Introducing ODPi Egeria – The Industry’s First Open Metadata Standard

Tools & Languagesposted by John Mertic November 5, 2018 John Mertic

Organizations looking to better locate, understand, manage and gain value from their data have a new industry standard to leverage. ODPi,...

Organizations looking to better locate, understand, manage and gain value from their data have a new industry standard to leverage. ODPi, a nonprofit Linux Foundation organization focused upon accelerating the open ecosystem of big data solutions, recently announced ODPi Egeria, a new project that supports the free flow of metadata between different technologies and vendor offerings.

[Related Article: Predicting code bug risk with git metadata]

Recent data privacy regulations such as GDPR have brought data governance and security concerns to the forefront for enterprises, driving the need for a standard to ensure that data providence and management is clear and consistent—supporting the free flow of metadata between different technologies and vendor offerings. Egeria enables this, as the only open source driven solution designed to set a standard for leveraging metadata in line of business applications, and enabling metadata repositories to federate across the enterprise.

“A consistent view on data across the entire landscape is essential for any organisation that wants to become data driven. Not just where the data is, but also the quality, the ownership, and the full lineage across the entire set of technologies used,” said Ferd Scheepers, chief information architect, ING, in the ODPi press release announcing Egeria. “The open metadata standard delivered by Egeria delivers this consistent view across all the technologies, while reducing the cost of metadata capture, and the management challenges of working with various data tool vendors.”

Egeria created a set of open APIs, types and interchange protocols to allow all metadata repositories to share and exchange metadata. From this common base, it adds governance, discovery and access frameworks to automate the collection, management and use of metadata across an enterprise. The result is an enterprise catalog of data resources that are transparently assessed, governed and used in order to deliver maximum value.

The ABC of open metadata and governance: Automation, Business Value, and Connectivity

Delivering this capability as open source was a critical part of the project, since multiple vendors must buy into this ecosystem. Vendors are not going to participate if one organization dominates the technology base. Thus the open metadata and governance technology is freely available with an open source governance model that allows the community of organizations and practitioners to develop and evolve the base for use in their offerings and deployments.

The first release of Egeria focuses on creating a single virtual view of metadata. It can federate queries across different metadata repositories and has the ability to synchronize metadata between different repositories. The synchronization protocol controls what is shared, with which repositories and ensures that updates to metadata can be made with integrity.

Egeria is built on open standards and delivered via Apache 2.0 open source license. The Egeria project creates a set of open APIs, types and interchange protocols to allow all metadata repositories to share and exchange metadata. From this common base, it adds governance, discovery and access frameworks for automating the collection, management and use of metadata across an enterprise. The result is an enterprise catalog of data resources that are transparently assessed, governed and used in order to deliver maximum value to the enterprise.

“Changing the availability and the quality of metadata will, in turn, improve the agility of the data scientist, as well as the transparency of the results they produce,” added IBM Distinguished Engineer and Master Inventor Mandy Chessell in the Egeria announcement. “Egeria simplifies metadata capture and management to create a consistent view of data across all tools an organization may use.”

Learn more about the Egeria project’s focuses on Automation, Business Value, and Connectivity by reading the full press release, or get involved directly at the Egeria GitHub page: https://odpi.github.io/egeria/

About the Author:

John Mertic is the Director of Program Management for The Linux Foundation. Under his leadership, he has helped ODPi, R Consortium, and Open Mainframe Project to accelerate open source innovation and transform industries. John has an open source career spanning two decades, both as a contributor to projects such as SugarCRM and PHP, and in open source leadership roles at SugarCRM, OW2, and OpenSocial. With an extensive open source background, he is a regular speaker at various Linux Foundation and other industry trade shows each year. John is also an avid writer and has authored two books “The Definitive Guide to SugarCRM: Better Business Applications” and “Building on SugarCRM” as well as published articles on IBM Developerworks, Apple Developer Connection, and PHP Architect.