How the Combination of AI & Data Fabrics Is Going to Revolutionize Data Management

The one thing a properly functioning artificial intelligence (AI) model needs is data. Lots and lots of data. But rather than just blast data in its direction like a fire hose, the most effective approach is to optimize the data feed through a wide variety of vetting, curation, and analytical methods.

This requires not just substantial infrastructure but highly sophisticated management software. These elements, in turn, are blended into a fabric architecture capable of drawing data from multiple sources and dispersing it in multiple directions at once so as to provide the AI model with the broadest perspective possible.

This is a herculean job, of course. One that would typically demand a vast team of data scientists and IT resource managers. However, thanks to emerging technologies, many of these processes can now be automated, and the driving force behind this automation is none other than AI.

Mutual Support

This symbiotic relationship between AI and data fabrics is no accident. In fact, it has been carefully designed to complement the abilities of both technologies to form a more properly functioning whole. As Analytics Insight noted recently, data fabrics provide the kind of timely, accurate, and non-biased data to drive positive results from intelligent algorithms, while some of those algorithms enable fabrics to operate at the speed and scale to support AI training and operations.

Today’s data fabrics provide a wide range of advantages over more traditional network architectures when it comes to supporting AI models. For one, they enable real-time data access from diverse sources, regardless of location. They also break down the silo-based architectures that exist in most organizations and only serve to prevent full versions of the truth from emerging. And fabrics are meant to be not just scalable but highly agile, allowing them to pivot in the face of changing needs and environments.

In other words, a data fabric provides a holistic view of all information assets available to a given organization, both internally and externally, public and private.

Douglas Vargo, vice president of consulting services at CGI, says it does this by bringing three core functionalities to the AI training process and the broader data environment that drives modern business models:

• Data Ingestion: Prioritizes data in batch, real-time, and event-driven formats for analysis and decision-making;
• Data Storage: Organizes data into tiers of raw, transformed, and curated information to support low cost and high reliability;
• Indexing and Cataloging: Ensures visibility and searchability for faster discovery and retrieval.

Once these capabilities are established, they can be used to create AI models that streamline data sharing, collaboration, and innovation for advanced user applications and open-source AI platforms and libraries.

A Fabric Is Not a Mesh

But be careful not to confuse a data fabric with a data mesh, says Daniel Comino, director of digital marketing at data management platform developer Denodo.

In a data mesh, data is organized according to key business domains like sales, marketing, and customer service. This allows individual teams to take ownership of their data and use it to achieve the best outcomes for their areas of responsibility.

A fabric integrates data from multiple sources and then relies on automation to reduce complexity and ensure broad access and tight security.

There is no reason why meshes and fabrics cannot work together, of course, but in terms of AI functionality, it is the data fabric that produces the higher return.

How Data Fabric Governance Enables Reliable and Trustworthy AI through

One of the ways it does this is by creating AI models that are reliable and trustworthy, according to John J. Thomas, vice president, and distinguished engineer at IBM Expert Labs. With a fabric architecture, organizations can easily implement broad policies regarding access and use of data while also establishing guardrails to prevent AI models from going awry with the natural biases that exist in all data sets.

This kind of governance is crucial for the prevention of what many are starting to fear from AI – algorithms running amok and generating catastrophic consequences for businesses and perhaps the entire human race. By minimizing integration issues and enabling rules-based access, AI governance becomes both scalable and automated to produce consistent, repeatable processes with inner workings that are transparent, traceable, and ultimately accountable.

Under this framework, negative outcomes can be quickly identified and corrected before the damage reaches a critical stage.

It’s All About the?Symbiotic Relationship Between AI, Data Fabric, and Human Expertise

While AI certainly has the propensity to do much of the heavy lifting when it comes to building and maintaining the data fabric, this will also require a substantial amount of human labor. As with most other applications, AI is expected to do the grunt work while trained experts will assume the creative roles involving architectural development, oversight, and strategic planning.

At the same time, the fabric belies the notion that AI will just run loose in the enterprise, finding and correcting anything it deems insufficient. The fabric is both the playing field and the rules under which the game is played, and it needs a vibrant and experienced ground crew to ensure both the players and spectators are safe and the outcome is fair.

In this way, we can look at the emerging intelligent data ecosystem as a form of three-way symbiosis in which AI, the data fabric, and the human element all work together for their mutual benefit.