Focus on Shadowbase Product Management

Keith B. Evans,
Shadowbase Product Management

Transform Your Data!

Today, the value of a company’s data cannot be over-estimated. Data has become the fuel to power new business opportunities. While exploiting data is nothing new, the amount of data being generated certainly is, and it is only now that we have the tools to gather and mine vast amounts of that data in real-time.

However, this exploitation of data presents some interesting challenges, such as how to move the data to where it can be actively exploited? Existing IT systems have typically been developed only within an existing and technologically familiar context (i.e., a new application is needed; therefore, build the new application). Companies also accumulate disparate IT systems via business mergers and acquisitions. The net of these processes is that applications (and the data collected and used by those applications), are often isolated, and the data managed by one application is not available to others. This type of impromptu growth seriously limits the possibility to exploit that data for new purposes. Fortunately, HPE Shadowbase Data Integration and HPE Shadowbase Application Integration software overcomes this problem by providing a data distribution backbone, enabling the real-time distribution of events and data between heterogeneous systems and applications, from wherever they are created, to wherever they are needed.

Yet, the problem doesn’t end there. It is highly unlikely that the format of the data/events created by one application would match the format requirements of another application (just as two people with no language in common cannot communicate). The wrong way to solve this problem is for each application to learn the other’s language, an overwhelming software development task, if anything more than a few applications are involved, and also one that would have to be repeated ad infinitum as new applications that need to interoperate are developed.

A much simpler and more repeatable and scalable solution is the practice of data transformation. With data transformation, the data/events created by one application are transformed, mapped, and changed into the formats required by other applications, while in progress as the data is distributed to wherever it is needed. In this way, the data arrives at the target application in the required format, such that no changes are needed to the application at all. I hear you ask, “But isn’t that data transformation process also an onerous and time-consuming task?” Fortunately not, because as well as providing the data distribution backbone, HPE Shadowbase solutions also include powerful capabilities to simply and easily transform change data into whatever format is required by the target database/application:

• SBMAP – a scripting “language” that can be used to inform Shadowbase software how to transform source data into target data formats. SBMAP is powerful, sophisticated, and extensible.
• SBDDLUTL – a utility that reads an HPE Enscribe DDL record definition and produces a “flattened” (normalized) DDL structure along with an SQL CREATE TABLE statement for the selected target SQL environment. This capability simplifies the replication of unstructured Enscribe data into structured SQL databases. SBDDLUTL includes features to allow manipulation of the source fields when creating the target columns, including dropping and/or renaming fields, transforming field data, and normalizing the non-normalized Enscribe data.
• User Exits – enable the inclusion and execution of customized user logic (program code) at various points in the Shadowbase replication stream and provide capabilities that are more complex than the scripting language. User Exits are extremely flexible, enabling almost any kind of data transformation, and can also perform specific field/column-level encryption and data tokenization.
• DBS Mapping – a scripting “language” for target-side Other Server platforms, including these capabilities: drop all events for a target table; drop columns and/or certain events for a target table; convert updates to inserts; concatenate (text) columns; and reformat and convert/replace characters.

Simply put, HPE Shadowbase software provides all the facilities required to distribute data and events from wherever they are created to wherever they are needed, in the required format, in real-time, without change to existing applications, while unlocking the value in your data for competitive advantage and building new solutions for your business. For more information, please see our white papers, HPE Shadowbase Streams for Data Integration and HPE Shadowbase Streams for Application Integration.

Please reference our Newsletter Disclaimer.