Dremio Looks To Shake Up The Data Lakehouse Space With New Cloud Offerings

Dremio Cloud is now live with a free open-source edition which, combined with new data management and updated SQL query technologies, will simplify data lakehouse implementations and accelerate their adoption, according to the company.

Looking to expand the adoption of its data lakehouse technology, Dremio has flipped the “on” switch for its cloud lakehouse service and debuted new query engine and data management software that boosts the new platform’s capabilities.

The Dremio Cloud lakehouse, initially unveiled in July 2021, is now generally available in a free standard edition, according to the Santa Clara, Calif.-based company. A commercial enterprise edition provides advanced security controls, including custom roles and enterprise identity providers, and enterprise support options.

Dremio also debuted Dremio Sonar, a new release of the company’s SQL query engine that powers the cloud lakehouse system, and is providing a preview of Dremio Arctic, a new metadata and data management service that will work with Dremio Cloud.

[Related: Dremio Looks To Recruit, Enable SI And Consultant Partners WithDremio Looks To Recruit, Enable SI And Consultant Partners With Channel Program LaunchChannel Program Launch ]

“We think that to really experience the data lakehouse paradigm, you need to do it in production, at scale,” said Tomer Shiran, Dremio co-founder and chief product officer, speaking of the free edition of Dremio Cloud in an interview with CRN.

“Now, with all these new features in Sonar and with what we’re doing with Arctic – which is way beyond what you could ever do with a [data] warehouse – it’s giving people the chance to experience a data lakehouse [as] the best approach from a business standpoint,” Shiran said.

Dremio is one of a growing number of big data companies promoting data lakehouses as an alternative to traditional data warehouse systems. In January Dremio raised $160 million in Series E funding, putting its valuation at $2 billion.

The new Dremio Cloud allows data analysts, data engineers and data scientists use to tap into huge volumes of data for business analytics and other tasks. The cloud-native service is based on the Dremio SQL Data Lake Platform, which includes a query acceleration engine and semantic layer, and works directly with cloud storage systems.

“Dremio Cloud is the world’s first lakehouse platform that was built from the ground up for SQL workloads, including mission-critical BI,” said Dremio CEO Billy Bosworth, in a statement. “In the past, companies had to weigh the pros and cons of data lakes versus data warehouses. We’ve eliminated this tradeoff by providing a frictionless and infinitely scalable service that delivers the combined benefits of both.”

Data analysts, data engineers and data scientists can use Dremio Cloud to quickly access the increasingly huge volumes of data businesses and organizations are working with today. Dremio Cloud provides bi-directional query capabilities with business analytics tools such as Tableau and Microsoft Power BI.

The fully managed service is currently running on the Amazon Web Services cloud platform with plans to make it available on Microsoft Azure and the Google Cloud Platform later this year, Shiran said.

Dremio Sonar is an update of the lakehouse SQL query engine, the company’s core technology, that businesses and organizations use to perform business intelligence tasks – from ad-hoc queries to low-latency dashboards and reports – running against lakehouse-stored data.

Sonar is based on the open-source Apache Iceberg table format for analytic datasets (originally developed by Netflix and Apple) and the open-source Apache Arrow columnar memory data format. Sonar is more than twice as fast as the product’s previous generation, according to Shiran, and can insert, update and delete records directly on a lakehouse through its support for the SQL Data Manipulation Language (DML).

“It basically puts all data warehouse functionality on a data lake,” Shiran said. “You no longer need a data warehouse.”

Dremio Arctic is a data and metadata management service designed to provide data engineers and data scientists with what Dremio calls a “Git-like” experience, similar to the code version control and change management capabilities developers enjoy with the Git system. Arctic is based on the open-source Project Nessie transactional catalog technology.

“It automatically optimizes the data in a data lakehouse,” Shiran said, citing previously manual tasks such as data partitioning and managing file sizes that consume data engineers’ time.

Dremio Sonar is available now and is offered free with Dremio Cloud. Dremio Arctic is in preview

“Tableau is committed to making data and analytics technology more powerful, easier to use and integrated in the flow of business. The new single sign-on-enabled connectivity with Dremio Sonar and Arctic enables analysts to explore and visualize current, historical, and experimental data snapshots on cloud data lakes, without having to create and maintain countless copies of data, which will help surface insights faster,” said Brian Matsubara, vice president of global technology alliances at Tableau, a Dremio technology partner, in a statement.

Dremio Cloud, Sonar and Arctic also will create opportunities for Dremio’s systems integrator and consulting partners who design and implement data lakehouse systems for their clients and build applications and services that run on them, Shiran said.

“It’s a big benefit for channel partners because they can get up-and-running with a data lakehouse much more quickly and easily,” he said, citing the reduced time-to-benefit advantage of Dremio Cloud. “They don’t have to develop all this expertise at the infrastructure layer.”

Dremio Cloud and the related software should also provide a boost for open-source software adoption in the software-as-a-service realm where it is not as widely used as open-source downloaded software, Shiran said.

“This exposes users to the possibilities of open architectures” in cloud software, he said. “It’s a big win for customers, really, because when you think about a data lakehouse architecture, you want your data to be open.”