Reporting Architectures Take Two

When I wrote this whitepaper, I thought I was already a bit behind the times. Certainly after all of the years of building data warehouses and data marts, people would not go back to reporting off of replicated databases would they? As it turns out, yes they do. To my amazement, seven years later my clients still struggle with the same issues.

So how can this happen? As far as I can tell, the answer seems to organic growth. What started out to be one thing, grew into something much bigger. In a way, that is a good thing! If your systems do not grow and expand, you are probably going out of business.

Does that mean that we should continue to build reporting solutions directly off of our transaction systems? When that no longer works, build a replication system and use that? When that no longer works, build a data mart? When that no longer works, build a data warehouse? The way I see it, if you are still in business and you are moving up the scale, you are on the right path.

One obvious danger that I have recently witnessed is over engineering a reporting system. Can you be all things to all people? I doubt it. What ends up happening is that you do most things OK, some poorly and the people that depend on your solutions get frustrated and end up looking elsewhere for their data.

While I continue to advise my clients to architect a scalable solution, you must start small. When talking to a client the other day, I advised "Think about as much of the problem space as you can for requirements and architecture. When it comes time to implement something, do as little as you can and still be successful." This naturally translates into quicker cycle time and more cycles. The benefit of this approach is that you continue to demonstrate reliability in achieving your goals and objectives. And who would not come back to someone who continuously achieves what they said they would?

One key that I have found to be rather successful in starting small is the Logical Data Model. The Logical Data Model can take in all of those requirements and hang them together for the big picture. That is your data blueprint. Print out big copies and put them everywhere so that people know what you are building and what it will look like when it is done. Then go build something and create a Physical Data Model scoped to only what you need at the time. This says that you have an eye on the future, but are focused on the present delivery.

So go out there and deliver something! Anything!!