Part 8 — HPE Shadowbase Real-Time Business Intelligence Solutions

The previous web pages explained the evolution of RTBI, and the various technologies that have been used over the years to achieve it. Of these technologies, data replication is by far the most flexible, least disruptive, and easiest to implement. The HPE Shadowbase product suite provides the transactional replication needed for effective real-time business intelligence, real-time sales analysis, data insight, and business insight systems. These products also provide the infrastructure necessary to implement continuous availability architectures, including the HPE Shadowbase real-time data replication engine and the HPE Shadowbase Online Loading (SOLV) copy utility.

Case Study: Prescription Drug Fraud Prevention — An HPE Shadowbase Data Integration Case Study
Use Case: Ensure Quality Assurance Through Operational Analytics
White Paper: The Evolution of Real-Time Business Intelligence

HPE Shadowbase Real-Time Replication

The Shadowbase data replication engine provides high-speed data replication among a variety of databases and platforms. With a history in the decades and hundreds of large, high-volume, mission-critical, enterprise-class installations worldwide, the Shadowbase engine is a proven performer and is an ideal solution for interconnecting enterprise systems with an RTBI system.

HPE Shadowbase Software:

Provides bi-directional asynchronous data replication, supporting active/passive, Sizzling-Hot-Takeover (SZT, also known as Sizzling-Hot-Standby), and active/active system configurations. Shadowbase synchronous data replication is also available.
Is totally transparent to the applications. It requires no application modifications because it gets database changes from a transaction log or from database triggers or intercept libraries (real-time change data capture).
Runs on a wide variety of platforms against several databases; source platforms include HPE NonStop, Linux, Unix, and Windows. Target platforms include all source platforms plus OpenVMS and AS400. Shadowbase software replicates from NonStop SQL, Enscribe, Oracle, Sybase, and SQL Server databases to these same databases as well as to DB2 and MySQL databases, and can also replicate to any ODBC-compliant target database. (Please see our current list of supported databases and platforms.)
Has no disk-queuing points, thus, replication is very rapid. Replication latency is measured in the tens of milliseconds on many systems and performance is enhanced by its multithreading capability to handle massive loads.
Runs as check pointed process pairs and persistent processes on HPE NonStop systems so that it survives hardware and software failures. On other systems, its processes run as persistent processes under a monitor.
Guarantees the integrity and consistency of the target database even when running with multiple threads. It is configured to break source transactions into sub-tractions to enhance replay parallelism and performance.
Has extensive built-in data transformation rules that are augmented by user-supplied specialized rules.
Provides complete data collision detection and resolution facilities for active/active environments. Users extend its collision resolution rules with additional business rules tailored to their application’s business logic.
Is heterogeneous; any supported platform and any supported database may serve as a source system or as a target system in any mix.
Is easy to manage with its AUDMON process monitor, its AUDCOM command interface, and its Shadowbase Enterprise Manager (SEM), a Windows GUI that provides integrated command and monitoring support for its components running in a heterogeneous environment.

Extreme Availability with HPE Shadowbase Active/Active Support

Many HPE Shadowbase installations around the world are active/active, geographically distributed systems. These systems achieve availabilities in excess of six 9s (less than thirty seconds per year of downtime). Shadowbase replication provides two functions in an RTBI environment: it synchronizes the operational systems with the RTBI system and ensures multiple RTBI systems surge on, via an active/active or Sizzling-Hot-Takeover configuration.

Running an RTBI system as an active/active system simply requires that the Shadowbase engine replicates database changes between the two geographically separated RTBI systems. If one system fails, the other system is immediately ready to take over the RTBI services. With an active/active RTBI system, it is fair to say that the effective uptime of RTBI services is measured in centuries.

HPE Shadowbase SOLV – Online Copying

The critical online copying function needed to bring an RTBI system into service is provided by the unique copying utility, HPE Shadowbase Online Loading (SOLV), which:

Moves data from a source database to the target database without interrupting source database activity
Maintains target data that was already copied in a current state by applying changes to it as other portions of the source database are copied. There is no large queue of source database changes that needs to be applied following the copy
Maintains currency of those parts of the database that were already copied, which may then be used by applications at the target system while the copy is in progress

Another powerful feature of SOLV is inherent in its architecture. Since SOLV is not a utility separate from the HPE Shadowbase replication engine, it uses the Shadowbase replicator to move data from the source database to the target database. As a consequence, SOLV uses the same data transformation rules during the copy function that the Shadowbase engine uses during later data replication. This function avoids the problem of trying to conform the copy transformation rules with the replicator transformation rules so that both perform exactly the same. Rules are implemented only once because the same code is then used by both the load engine and the replication engine.

HPE Shadowbase Streams

Shadowbase Streams, another member of the Shadowbase product line, streams data generated by one application to other applications and facilities as well as target database environments. Shadowbase Streams for Data Integration and Application Integration provides the facilities for integrating existing applications at the data or event-driven level in order to create new and powerful functionality for the enterprise.

Solution Brief: HPE Shadowbase Streams for Data and Application Integration

HPE Shadowbase Streams:

Quickly and easily integrates changes made in any database into other data environments to keep that target information synchronized. The changes can be made in real-time or batched for periodic snap-shot or micro-batch updating
Enables interoperation between applications that were once isolated in an event-driven fashion in real-time. Critical data generated by one application is distributed and acted upon immediately by other applications
Supports many models for data distribution, including maintaining a remote database of critical data, sending critical data to client applications or servers directly via queues or client APIs, publishing data to applications that have subscribed to the data, and responding to poll queries from client applications
Is extensible, allowing the user to embed custom processing logic into the data processing path. It readily filters, transforms, and adapts data from one application or database environment into the format or protocol required by another application or database environment, all without requiring any changes to the existing application that is generating the data
Modernizes legacy applications by integrating diverse applications across the enterprise so that new and valuable services are generated to enhance competitiveness, to reduce costs or to increase revenue, to satisfy regulatory requirements, and to generally improve the user experience
Plays a significant role in the distribution of data from disparate sources throughout the enterprise in real-time, thereby facilitating the implementation of RTBI capabilities