HPE Shadowbase Online Loader (SOLV) and HPE Shadowbase ETL Overview

Before a database can be useful, it must first be loaded with data. HPE Shadowbase Online Loader (SOLV) and HPE Shadowbase ETL are utility products designed to perform this task, providing online and offline loading (or unloading) of a source database into a target database.

HPE Shadowbase (SOLV)

The HPE Shadowbase SOLV utility can load audited and non-audited HPE NonStop Enscribe source files and HPE NonStop SQL tables into any target environment and database combination supported by the Shadowbase line of data replication products (e.g., HPE NonStop Enscribe or HPE NonStop SQL targets, or other server targets such as Oracle, SQL Server, Db2, and MySQL). The Shadowbase SOLV utility can perform the load while regular replication is also taking place, consuming the source database changes and merging them with the data being loaded to keep the target fully synchronized with the source while the load proceeds.

Key Features

• Initial and refresh target data-loading utility
• Supports point-in-time bulk loading of source to target
• Allows for extract, transform, filter, and cleanse of the data being loaded
• Integrates with the Shadowbase replication engine to keep target synchronized during and after the load completes
• Can use the same (or different) conversion routines for replication as are used for loading
• Replicated events are consumed/merged with the loading activity, so there is no queue of changes to drain after the load completes
• Dynamically adjusts loading activity to efficiently utilize available resources
• Source and target can be opened for read-write activity while the load or refresh occurs
• SOLV does not write data to interim disk queues during the load, making the loading process much more efficient than other loading methods

Solution Brief: Shadowbase SOLV Product Suite
White Paper: HPE Shadowbase SOLV Product Suite — Online Database Loading, Copying, Synchronizing, and Unloading

HPE Shadowbase ETL Toolkit

Extract Database Changes or Initial Load Data into Flat Files

The HPE Shadowbase ETL Toolkit is used to extract database changes or initial load data into flat files for subsequent ETL loading into a target environment using:

• comma-separated value (CSV),
• fixed-position,
• variable length, and
• tab-delimited formats.

It can feed high-volume data quantities into a data warehouse, and row-based data into columnar databases. When coupled with HPE Shadowbase Online Loading (SOLV), the Toolkit can: extract select data from a source database; transform/cleanse it into the proper format; save it into flat files understood by the ETL utility; and load the files into a target database environment not directly supported by Shadowbase replication.

Using the Toolkit, source database change data can be periodically extracted from its audit trail, and processed into a flat file format for incremental loading into the target environment using a micro-batch update approach to keep the target synchronized with the source.

The Toolkit can also be used to extend the SOLV loading capabilities to allow for reading and injecting events from flat files into the Shadowbase replication engine for processing.

Key Features

• Trickle-feed replicated-events loading utility
• Trickle-feed replicated-events UN-loading utility
• Snapshot point-in-time bulk loading utility
• Snapshot point-in-time bulk UN-loading utility
• Allows for extract, transform, filter, and cleanse of the data to be loaded or unloaded
• Reads or generates comma-separated value (CSV) and fixed-position flat files for the data to be loaded or unloaded
• Source or target can be open for read-write activity while the load or unload occurs
• Leverages the Shadowbase Online Loader (SOLV) features for ETL processing

 Case Study: Vodacom’s One Year Recovery
 Case Study: A Major ISP Migrates from Sybase to HPE NonStop with No Downtime
Use Case: Transform Inflight Data Using Shadowbase Mapping