Informatica-Databricks Alliance Looks To Broaden Data Lakehouse Adoption

Deeper integration of Informatica’s data management tools with the Databricks platform will simplify data movement and transformation for data lakehouses and make data available to a wider audience of business users for analytical tasks.

Informatica and Databricks are expanding an existing partnership by integrating Informatica’s data management cloud software with the Databricks Lakehouse Platform.

The move will accelerate “data democratization,” the companies say, by making it easier to move massive amounts of data into the Databricks data lakehouse system and provide non-technical users with greater access to that data for business analytics.

“This allows us to basically democratize the ability to leverage data assets, prepare data assets, to work with data, for a much wider audience,” said Rik Tamm-Daniels, Informatica vice president of strategic ecosystems and technology, in an interview with CRN.

[Related: Databricks Boosts No-Code Data Analytics, AI Capabilities With 8080 Labs Acquisition]

Thursday the two companies announced that Informatica’s flagship Intelligent Data Management Cloud (IDMC) platform now provides cloud-native, low-code/no-code data integration with the Databricks Lakehouse Platform.

The new Informatica-Databricks integration enables no-code enterprise-grade data pipelines to run natively on Databricks, simplifying the process of moving massive volumes of data into Databricks. It also makes possible rapid data loading from databases into Databricks Delta (the company’s open-source data lake format) via Informatica’s Database Mass Ingestion tool.

The Informatica-Databricks integration also shifts data preparation and transformation tasks into Databricks SQL, the company’s software for running analytic workloads directly on huge stores of unorganized data within a data lakehouse. (Databricks says Databricks SQL helps a multi-cloud data lakehouse operate with up to 12x better price performance than traditional cloud data warehouses.)

These new capabilities, according to the two companies, expand data integration and data preparation – traditionally the domain of IT management and data engineers – to a wider range of business users who can benefit from analyzing lakehouse data.

“Data democratization is one of the biggest challenges that companies have to deal with in terms of getting value from their core data assets, their analytics,” Tamm-Daniels said. “It’s more data through our connectivity, it’s more users through the no-code experience, and it’s more scale through the database engine. This will help a. host of new users to get value and utility out of the lakehouse.”

Databricks is a leading proponent of the data lakehouse concept of making large stores of unorganized data – called data lakes – available for data analytics. It’s promoted as an alternative to traditional data warehouse systems where data is organized for business analytics.

The data lakehouse concept is evolving such that businesses are no longer asking “what is a data lakehouse?” to “how do I get started?” said Joel Minnick, Databricks marketing vice president, in an interview. And the Informatica-Databricks alliance will help with that, he said

“How do we ensure that the data platform folks are adopting going forward is a much simpler platform than the platform they’re coming from?” Minnick said. “Leveraging the simplicity of the tools that Informatica is putting together here to make this process very clean for users, by getting a lot of the heavy lifting out of the way, is a big enabler and helps customers ultimately be more successful with their data lakehouse.”

Informatica and Databricks have numerous joint customers who are looking for ways to expand their use of the two companies’ technologies and better leverage both systems together, Tamm-Daniels and Minnick said.

Both companies also work with regional and global systems integrator partners and the Informatica-Databricks integration will simplify data lakehouse implementations, Tamm-Daniels said. “They are the ones on the front lines of delivering this in a transformational analytical architecture with a data lakehouse.”

“Ultimately this is about how we help more customers embrace the lakehouse faster and easier so they can realize the benefits it provides by having that single source of truth, that one platform, to handle their traditional analytics/data warehousing workloads and their predictive analytics around data science and machine learning,” Minnick said. “The easier and simpler we can make that, it’s a win for everyone.”