Every time we begin to study some technology or methodology of work, we always come to the question:
What is the best tool to work with?
A plausible but very delicate question. So, since I’ve been working with Oracle for a long time, and every technological context that I live with revolves around it, I will inevitably talk about Oracle tools.
Even when we think of a corporate solution the decision is not always so easy, especially when we move in the pocket.
But believe me, everything that will be shown about Oracle in the articles of this blog will always be presenting the best possible scenario to work with, as well as solutions to possible problems that arise in the course of time (Bugs for example).
And the subject of the time will be Oracle Data Integrator.
The Oracle Data Integrator Conception came into being in 2006. It came after Oracle acquired SUNOPSIS, a leading data integration company and a leading provider of heterogeneous E-LT (Extraction, Reading, and Transformation). As a result, Sunopsis’ data integration product became Oracle Data Integrator 10g after the acquisition.
In August 2010, Oracle made its first ODI updates with the release of Oracle Data Integrator 11g. This release allowed ODI to be integrated with other products, such as Oracle SQL Developer and JDeveloper, and introduced new features such as the JEE Agent and the ODI Console.
Although the 11g version was a breakthrough compared to ODI 10g, it was still not widely considered a typical ETL development tool. The concept of Interfaces, as a single unit of work for ETL versus the usual flow-based mapping approach found in most ETL tools, including OWB, led to a slower adoption rate. It took years, but ultimately, Oracle Data Integrator became stream-based with the release of ODI 12c, integrating some of the best features of Oracle Warehouse Builder into the current ODI product.
By the end of 2013, the initial version of ODI 12c was made available to the public. In addition to moving to flow-based mappings, ODI 12c also included:
- Integration with large data source;
- Life-cycle management capabilities;
- Many performance improvements, and;
- A migration utility for those still using OWB.
Oracle Data Integrator 12c is an integration platform that supports Use Cases such as:
- Business Intelligence (BI);
- Data Warehousing;
- Master data management;
- Data migration;
- Data integration and application integration.
It can be deployed using bulk, batch, real-time, and web services.
The Oracle Data Integrator platform is fully integrated with Oracle Database, Oracle GoldenGate, Oracle Fusion Middleware, Oracle Big Data Appliance and Exadata.
Its main functionality is based on an extraction, load and transformation architecture that leverages the capabilities of the source and destination databases used in data integration processes and for this topic I would like to give a good prominence. We’ve always heard of ETL functionality (I’ve even written an article about it), but with ODI 12c, things change a little bit about order of processes.
Instead of having the sequential process of:
EXTRACTION => TRANSFORMATION => LOADING
With ODI 12c we will have the flow:
EXTRACTION => LOAD => TRANSFORMATION
as the image below:
That is, with ODI 12c you will do all TRANSFORMATION work in the SOURCE or the DESTINATION. This way, you will have less processing load in the middle of the process, thus concentrating the heavier load between the high end processes (start and end).
This architecture enables the software to leverage the functionality, scalability, and performance capabilities of relational database management systems and large data systems.
An example of leveraging database functionality is native support for Hive, HBase, Sqoop, Pig, and Spark.
Key features include:
- Knowledge modules (KMs), which are the objects of integration – models, workflows and transformations – are used as building blocks for data integration processes.
A large set of KMs is provided by Oracle, but the Oracle Data Integrator user community also creates and shares KMs. It’s a tip worth checking out. - Connectivity. It allows you to connect to a wide variety of structured, unstructured, and semi-structured sources and goals.
- Metadata repository. Management that captures, presents and manages processes and metadata.
- Administration and Management, which is provided by the Oracle Application Development Framework and JavaScript Asynchronous and XML Framework.
- Increased volume support for Apache Spark, Apache Pig, and Oozie.
- Improved file, LDAP, JMS Queue XML, JMS Topic XML and XML connectivity.
- Increased release management capabilities for production, test and development environments.
In addition to everything that was listed above, I would like to highlight an important point for those who already work with OWB and intend to migrate to ODI 12C.
This latest release includes important enhancements such as increased interoperability with the Oracle Warehouse Builder (OWB) as well as a tool that enables migration from OWB to Oracle Data Integrator.
It can also be integrated with Oracle Enterprise Manager Cloud Control 12c to manage deployments of Oracle products.
Oracle Data Integrator users can deploy real-time data integration by leveraging Oracle GoldenGate, which enables faster, more efficient data loading and transformation in real-time.
Version 12c also provides enhanced parallelism, integration workflow development, release management, and large data support.
Oracle Data Integrator 12c, when used in conjunction with other products related to Oracle integration, is generally used by large companies, especially those with multiple uses and integration, and those who plan to continue expanding the resulting integration applications. Companies that use Oracle applications and devices typically use Oracle Data Integrator.
I end by stating that the tool is really fantastic, robust and very good to work with. Other than that, it can provide excellent job opportunities.
Everyone, I’m staying here.
Inquiries, please contact us.
Strong hug.
Eduardo Santana
bufallos@bufallos.com.br