by Lightdash, an open source business intelligence (BI) platform that aims to challenge incumbents like Looker, is officially launching its commercial core product to the public today with the backing of $8.4 million in funding. The first round was Accel, with participation from Moonfire, Y Combinator (YC), Snyk founder Guy Podjarny and Gitlab CMO Ashley Kramer.
In its original form, Lightdash was known as Hubble when it graduated from YC’s S20 batch, with a focus on helping companies run tests on their data warehouse to identify data quality issues. These data quality metrics, it turned out, were most useful within BI tools, which co-founder and CEO Hamzah Chaudhary says no BI tool on the market supports. And so they pivoted the product to Lightdash and started working on the new project full-time in May 2021 to solve a big problem for data analysts.
“Today’s data analysts are becoming more and more like software developers, but they’re stuck with enterprise user interfaces that lock up business logic and slow them down.” Chaudhary he explained to TechCrunch. “Lightdash provides analysts with productivity tools that enable them to deploy BI at enterprise scales with much less effort.”
Lightdash is built specifically for dbt, a command line-based data transformation tool that allows analysts to transform raw data in their warehouse using SQL and their usual text editor. Dbt is the “t” in “export, load, transform (ELT)” and Lightdash in turn turns any dbt project into a “full stack BI platform”.
It is worth noting that Lightdash is both front-end and back-end. So for regular business users who may not know SQL (eg marketing or finance), Lightdash serves as a visual layer for dbt, while on the back-end data analysts and other more technical users can build custom flows of work and define enterprise-wide logic for metrics and KPIs, effectively “removing the complexity” of how they are calculated.
“Lightdash is focused on giving data analysts the tools to enable true self-service BI for the rest of their company.” Chaudhary said.
Lightdash dashboard Image credits: Lightdash
BI Basics
Business intelligence, for the uninitiated, is the process of mining, integrating and organizing disparate data sets for decision making. Big data insights are the name of the game, helping analysts draw meaningful conclusions, identify and visualize patterns, and predict future outcomes (eg sales forecasts).
The BI market is big business, pegged as a $23 billion industry in 2020 and projected to surpass $33 billion by 2025, which is likely why Google shelled out more than $2 billion to acquire Looker in 2020 and Salesforce it acquired Tableau for more than $15 billion before that.
Lightdash, for its part, serves as the primary gateway for companies looking to explore their data, implementing native integrations with many of the tools that make up the modern data stack, from dbt to Snowflake, Airbyte and Fivetran.
“Lightdash is built to be open and integrated into the modern data stack, not a closed system.” Chaudhary said.
Lightdash in action Image credits: Lightdash
The genesis of Lightdash can be traced back Chaudhary and co-founder CTO Oliver Laslett were together at UK insurance tech Cytora, where they were tasked with scaling the company’s data analytics output.
“We saw the huge gap in the quality of tools available to our data teams versus our software engineers, even though the tasks asked of our data analysts were just as technical.” said Chaudhary.’When we left Cytora, we knew we wanted to empower data analysts and data teams by providing them with tools that were more fit-for-purpose and up-to-date. We [then] worked as data consultants helping companies build their data stacks and eventually realized that the weakest link in the workflow was the BI layer because the BI tools didn’t integrate well with the rest of the data stack, didn’t support data-driven developer workflows, and made difficult for data teams to collaborate effectively.”
And that, perhaps, gets to the heart of what Lightdash is trying to do: it’s about helping data analysts and analytics engineers use their existing tools, like code editors, and enable teams to collaborate on scale. “A platform built to integrate with other tools,” such as Chaudhary says so.
The open source factor
While the main Lightdash project is open source, the company released a fully managed and hosted Lightdash Cloud service in beta in January, ahead of the release of the free self-hosted Community version in June. Today marks the public beta release of the core Cloud product, which has so far amassed a waiting list of around 600 companies.
“We always planned to have a commercial version of Lightdash, but we also wanted to make sure that the open source product was also usable – which is why almost the entire feature set of the product is available in the self-hosted open source version. Chaudhary said.
Looker and its ilk are the obvious comparisons here, but Lightdash’s open source credentials are one of its key differentiators, going some way towards SMBs and larger enterprises. Open source is a major selling point for security-conscious companies, as it means they have complete visibility into how their data is handled. It also means they can start small by deploying Lightdash to just one or two teams to test it out before expanding further into their stack if they like what they see.
“This is much more efficient compared to proprietary BI tools, where you often have to go through a long sales and procurement process before you start using the product itself.” Chaudhary said. “For many enterprises and startups, this is the preferred method to get started with new tools – it has the added benefit of being open source, already built to be deployed on-premise if needed, which is often a requirement for large enterprises ».
This is a model that has been successful for so many startups before it: an open source foundation for organizations that need full control and flexibility, with a commercial layer that removes much of the complexity and spadework for those who need it.
A quick look at the competitive landscape reveals a few other players in the commercial open source BI space, including Metabase which raised a $30 million round last year and Preset which raised about $36 million to commercialize the Apache Superset project. So it’s clear that there is real demand not just for BI, but open source BI backed by a fully supported commercial service.
Lightdash is a remote company, with its founders based in or around London and the rest of its 8-person team spread across Europe, although the company is incorporated in both the US and the UK with 8.5 million dollars in the bank, which includes a so far unannounced $2.4 million pre-seed round led by Moonfire, the company said it is now well-funded to accelerate its hiring especially in its product team, as well as expand a training program called Lightdash University, which is designed to “upgrade” BI teams.
