WhereScape RED Methodology
The WhereScape Pragmatic Data Warehousing™ Methodology:
WhereScape RED encourages the use of a methodology. A standard approach speeds up delivery, and a consistent implementation assists with support and speeds up enhancements.
This paper describes in summary fashion the models, methods and processes that make up the essence of WhereScape’s Pragmatic Data Warehousing™ methodology as practiced by WhereScape designers and as implemented in the WhereScape RED data warehouse life cycle management product. This methodology is gently suggested in WhereScape RED, but can be subjugated by other approaches if preferred.
You can read an in-depth white paper outlining the WhereScape Pragmatic Data Warehousing Methodology in PDF format or see below for an overview of this topic. The full white paper is recommended reading for all practitioners looking for optimal warehouse solutions.
Overview
The WhereScape Pragmatic Data Warehousing™ Methodology, or PDW, has been in development by several dozen practitioners associated with WhereScape for more than a decade.
Born largely out of frustration with the boom-bust cycles through which data warehousing seems to pass – and the incessant name-changing and euphemism-making that accompanies this boom-bust cycle – PDW is focused on several (as we see it) self-evident objectives for a data warehousing or data marting project:
- Build the warehouse or mart in a fashion that involves business analysts in the process, builds their confidence in IT, and increases their commitment to the finished product: the warehouse or mart under development.
- Treat data sources as what they in fact are – constrains on the outer bounds of the functionality of any data warehouse or mart – rather than as the central design problem, which they assuredly are not.
- Treat the data warehouse or mart as a process, not a project – focus on iterative releases and rollouts that follow one another in quick succession, keeping the warehouse or mart evergreen, instead of treating the warehouse as a one-time, big-bang project that produces a finished monolith that endures in that form for ever more.
- Build warehouses and marts that present the dominant sets of required information for decision-making in the most appropriate form for the end user analysts and their chosen tool sets (in that order), saving edge-cases, arcane issues and special needs for subsequent iterations.
- Focus energy and technology on the three areas of the data warehouse life cycle that research and practice clearly show are the leading sources of failed warehouses and marts: poor schema design that distort, disguise or neglect critical information; overly lengthy deployment cycles that cannot keep pace with business change; and near-total lack of enhancement to warehouses and marts due to their overly brittle implementation.
In a nutshell: do it fast, do it properly, and then do it again. And again. And again.
At the outset, it’s important to note a few critical things about PDW, in order for the proper context to be established.
- We did not develop the PDW to sell it, per se. We developed it to improve our own skills and cycle times as practitioners.
- WhereScape RED, our data warehouse life cycle management software product, has its genesis in PDW. RED began as set of marginally-connected private tools several of us used to perform what we saw as basic activities associated with good data warehouse design, deployment and enhancement, and evolved into an integrated data warehouse lifecycle management product as our methods and practices began to cohere and form a coherent whole.
- Any methodology will produce better results, consistently, than no methodology or merely intuitive design and development behavior.
This last point we cannot emphasize enough. Methodology, per se, when embedded in software products, creates suspicion in the minds of designers and developers, due in part no doubt to the draconian ways in which so-called “upper CASE” tools popular in the 1980s and early 1990s demanded full and unthinking compliance with their ways of doing things. We are not interested in enforcing our practice; we are interested in seeing a common practice emerge across data warehousing projects within a given organization, because that common practice will produce predictable results.
Additionally, data warehouse design is not as mature as the transactional application design CASE tools tried to automate– not by a long stretch. There are still many valid instances in which ‘normal’ practice is either insufficient or not applicable to the problem at hand, and – as pragmatists – we believe that the situation at hand should always determine the methods used. In some cases, we have to tackle novel design or deployment problems for which no existing methods will suffice.
Finally, we believe that gentle suggestion is better than draconian enforcement. The techniques and methods that make up PDW can be extracted from PDW and transplanted into other methodologies – including the classic standard development life cycle (SDLC) waterfall-style project – with little or no rework. We practice PDW rigorously, and we believe our customers’ uniformly high levels of satisfaction and success are the best metrics of that consistent practice and the intrinsic value of PDW. But, when we came to embody PDW in WhereScape RED, we made the conscious decision to suggest PDW techniques, but not to enforce them. WhereScape RED has been, and can be, used quite successfully in data warehousing projects that employ a project methodology or a design methodology other than PDW.
As a final, introductory observation, we need to acknowledge that PDW it trades heavily in ideas first elaborated by Bill Inmon and Ralph Kimball. We freely and fully acknowledge our debts to those two pioneers.
For further information about the WhereScape Pragmatic Data Warehousing Methodology you can read our in-depth white paper available in PDF format.