As part of integration, data fabrics, or organisation-wide analytics, developers have to pull in (or reach out and retrieve) data from disparate data sources.
The typical normalise and build a common/canonical data model of tables and fields that are populated with data from these different sources, and then utilised at an organisation level for reporting, machine learning and other practices.
It would be great to have a toolkit that allows user to quickly setup and define a canonical data model with these features:
- define a common data model through UI
- define data source contributors of this data model
- define how fields from the data sources are mapped into fields from the common model, include the ability to normalise/transform data on the way in
- introduce a set of interoperability business services/DTLs, and other utility specifically developed to quickly allow for onboarding additional data sources.
InterSystems advanced data modelling capabilities, schema free document databases, and experience in Information Exchanges all seem to be attributes that would help in developing this to.
It does not look like something has to be done on IRIS side. This mostly looks like the work for some external tool.
I have needed anything like that, but pretty sure, there are many tools like that (probably proprietary), suitable for that.