Various architectural components of a data warehouse are shown in fig. 2.11 and explained below —
Load Manager
The load manager is the system component that performs all the operations necessary to support the extract and load process. This system may be constructed using a combination of off-the-shelf tools, bespoke coding, C programs, and shell scripts. The size and complexity of the load manager will vary between specific solutions from the data warehouse to the data warehouse but, as an indication, the larger the degree of overlap between source systems, the larger the load manager will be. However, it is worth noting that third-party tools will probably contribute a maximum of 20-25% of the total system functionality.
The architecture of the load manager shown in fig. 2.12 is such that it performs the following operations —
- Extract the data from the source systems.
- Fast-load the extracted data into a temporary data store.
- Perform simple transformations into a structure similar to the one in the data warehouse
Fig. 2.12 Load. Manager. Architecture
Warehouse Manager —
The warehouse manager is the system component that performs all the operations necessary to support the warehouse management process system is typically constructed using a combination of third-party systems: management software, bespoke coding, C programs, and shell scripts. The complexity of the Warehouse Manager is driven by the extent to which the operational management of the data Warehouse has•een automated.
Third-party tools will probably contribute a maximum of 40% of the total system, with the. the bulk of the contribution from systems management tools for automated backup/recovery/archiving. ,• The architecture of a warehouse manager shown in fig. 2.13 is such that it performs the,•ollowing. operations —
- Analyze the data to perform consistency and referential integrity checks.
- Transform and merge the source data in the temporary data store into the published data warehouse.
- Create indexes, business views, partition views, business sunburns against the base data.
- Generate denormalization if appropriate.
- Generate any new aggregations that may be required. •
- Update all existing aggregations.
Fig. 2:13 Architecture of a Warehouse Manager
Query Manager —
The query manager is the system component that performs all the operations necessary to support the query management process. This system is typically constructed using a combination of user access tools, specialist data warehousing monitoring tools, native database facilities, bespoke coding, C programs, and shell scripts. The complexity of query managers will vary between specific solutions and is driven by the extent to which the facilities are provided by user access tools or native database facilities.
Practically all the query managers I built in later development phases. Typically “the query manager’ is designed in the first build phase, once the database and user access toOtslechnologieShave been determined.
The architecture of a query manager is shown in fig. 2.14 is such that it performs the following operations —
(i) Direct queries to the appropriate tables
(ii) Schedule the execution of user queries.
In some cases, the query manager also stores query profiles to allow the warehouse manager to determine which indexes and aggregations are appropriate.
