January 20, 2020
Data Fabrics Add Augmented Intelligence to Modernize your Data Integration is the new Gartner research for data and analytics leaders. Gartner has identified Irion EDM among the Data Fabric ready platforms.
The evolution of Data Management models
Data Management is changing in depth. A disruptive paradigm shift is coming. The market demand for integrated platforms is growing. Those must be able to:
- Convert passive metadata into active metadata and manage it;
- Deliver many management services, that involve data in different formats, types, locations, volumes, speed;
The Irion EDM platform leverages on an original paradigm and proprietary technologies. They let the platform handle the main requirements of this new model. But most important is that the platform is entirely developed using Microsoft technologies .Net, SQL Server and to use Irion EDM you only need to know the SQL language.
Irion EDM and Data Fabric
A Data Fabric is a network of highly reusable services, microservices and other Data Management and integration components in the form of Data Management services. A single development team can build one or more modules and you create sophisticated Data Management solutions linking different modules.
Irion EDM breaks down a Data Management solution into specialized units of work. Each module is self-consistent and processes a set of input data to produce an output.
DELT® (Declarative Extract Load & Transform®), Irion i.p., enables these features. Irion EDM orchestrates modules at run time, optimizing executions using statistics and all available information. It can visualize the execution graph and the lineage.
Irion EDM allows creation of modules (the graph nodes) using SQL, MS-Powershell, R and Python. You can create modules using other languages or environments. It’s up to you the choice of optimal communication/collaboration methods to integrate them in the solution. To select, consider requirements about:
- technical characteristics
- volumes
- performance
- business continuity
Adopting the Data Fabric model allows creating Data Management solutions that, by design, access and process in an integrated environment, data:
- with different data organizations (relational DBs, flows, data lakes, …)
- in various locations (on cloud, on premises, hybrid)
and for people with different skills (like data engineers, data scientists, business analysts,…).
The platform wraps the complexity embedded in this kind of environments.
Irion EDM can connect to many data sources: it provides hundreds of connectors to reach the most disparate information structures and application systems. Modern or not, on premises or on-cloud. Powerful platform features even enable to develop specific and customized connectors. Irion EDM expose to modules as “virtual tables” all the data available in the various sources and the modules output datasets themselves. The IsolData® i.p. automatically manages (without coding) modules processed data. This avoids unnecessary data persistence. IsolData® is in fact the capability of Irion to isolate in a dedicated working space all it takes (input, output, temporary data) in order to perform a single processing unit of a solution developed in the Irion platform. In IsolData® temporary tables are automatically created and deleted when they are no longer needed. If some data created during the processing are necessary for future use (e.g. Data Quality controls that involve historical data), it is also possible to save these data in a persistent area.
- visualizes a graphical interface to navigate them
- manages the behavior of the solution engines
- compiles the solution documentation in natural language (on the fly, so it’s always up to date)
Irion EDM enables a collaborative environment. With a graphical UI, users can define additional metadata creating new entities, attributes, links. To do this, they must consider objectives, views and interests of their different business and IT communities.
These “user defined” metadata, along with the “operational” ones, drive the engines behaviour. As a result, you can freely define multiple abstract layers and connect them with “vertical” links: a bridge between different business languages. In this way, metadata play an active role. Irion EDM can enrich the model collecting internal or external information to feed metadata.
Dynamic events and properties can produce information such as:
- access frequency
- accessed platforms
- volumes
- usage patterns
- etc.
R and Python embedded engines in Irion EDM can carry out models for the advanced analysis (AI/ML) of these phenomena, i.e. to identify a set of typical use cases. As an application example of use cases, the solution can provide the data of interest to the users – even suggesting new ones – in an easier and more efficient way.
Conclusions
Irion EDM is ready to implement today the new generation of data-intensive solutions. It is the loom enabling the creation of Data Fabric systems.
Irion EDM is the future-proof Enterprise Data Management platform.
Want to know more about Data Fabric? Read the article by
Ehtisham Zaidi, Eric Thoo, Guido De Simoni e Mark Beyer
https://www.gartner.com/document/3978267
About the Author
Mauro Tuvo
Irion Principal Advisor
For over thirty years, Mauro has worked with Italian and European organizations supporting information asset management, offering management, solution design and market opportunity development, particularly, Enterprise Data Management with a focus on Data Quality, Data Governance and Compliance (GDPR, IFRS17 and Regulatory Reporting). Over the years, Mauro has gained vast experience defining and applying methodologies that make him a leading player in the market, as well as in academic and speaking contexts. Mauro has published articles and papers on Data Management and has taught masters and specialization courses at the University of Padova, Pavia and Verona. He is a conference and seminar speaker and has been a member of the ABI Lab Information Governance Observatory since 2011.