The Business Side of the Modern Data Stack
Has Analytics reached peak meme status?
Some in the Data Engineering and Analytics community would have us believe the end is near. The funding and venture money have run dry. The wrong leaders are guiding our community down a bad path. Valuations will never recover to peak levels of the early 2020s.
To an observer, this fear, uncertainty, and doubt may seem warranted. However, as data engineers, analysts, and scientists, we would be wise to take a step back and evaluate the opportunities that still lie before us. As Director of Growth at SDF, I am hopeful for what is to come, which is also supported through my research and analysis while studying at Columbia Business School.
Let’s poke some holes in the skepticism that currently pervades our industry, review the financial circumstances that led us here, and take a look at the future.
Clickbait With Doomsaying is Easy
LinkedIn has become the social network of choice for the modern business professional. It has also become the home of many an influencer who promise business tips, how to get a job, or how to break into a FAANG (MAANG?) Lately, I’ve observed the most famous data influencers respond to community and industry pressures through odd activities such as judging resumes, asking if AI is going to replace data analysts, and claiming the end of data and the modern data stack is near. (My take - if you are going to be pessimistic and advocate for shorting a position bring receipts.)
A recent article questioning the usefulness of “the modern data stack” highlights the shifting macroeconomic effects of heightened interest rates, a shift to cloud-first technologies, and a desire of both VCs and industry leaders to innovate with AI. This perspective of a changing future with AI is driven by the current macroeconomic environment of high-interest rates and shifting approaches to raising capital for startups.
What Circumstances Led to This Point?
Select data companies raised several rounds of funding during the era of “free money” from 2020 through 20221. The core go-to-market strategy for many data brands was to move fast and grow at any cost. (peep the conjoined triangles of success for inspiration) These costs frequently resulted in a burn rate that was 5 times (and sometimes even more) their annual recurring revenue. To quote a recent banker who visited Columbia’s campus, “A lot of wild things happened in the time of 0% interest, things that still don’t make sense, but made a lot of people excited.” Today, without low-cost VC money available, many teams have had to look to other strategies for profitability.
As opposed to viewing this as a regular economic occurrence in our industry, data influencers like those mentioned have started using fear to their advantage - yelling “fire” and going back on their predictions in an attempt to redirect attention toward their own products.
Inflation Control and the Cost of Money
Let’s dive deeper into the cause of today’s state of affairs. As many SaaS, Big Data, and Analytics companies were reaching peak valuations in 2022, Jerome Powell, chair of the U.S. Federal Reserve, was preparing to increase interest rates. No one knew how high the interest rates would climb, but many expected it to occur quickly. Even my old employer, Astronomer, raised a Series C in March 2022 that increased its valuation to a number well above where I think it sits today.2 (Crying deeply into my Carta App)
Money was so “cheap” with low-interest rates that Pitchbook analysts were calling out high-value opportunities in the commercialization of Open Source tech. These types of investment opportunities (what many data engineering products are) often take seven to ten years to materialize profitability as community-driven tech is hard to commercialize when the default option is “free.” This narrative of high-value opportunities in commercializing Open Source tech has changed over the past year as interest rates have climbed3 and the total amount of Venture Capital investments has fallen. In the SaaS Analytics business venture capital investments dropped 33% from over $10B in 2022 to $6.6B in 2023 (2024 is looking even more bleak).
Interest rates drastically declined during the era of COVID-19 and remained at levels as low as 0.05% from late 2020 through early 2022, fueling some open-source investments. Since 2022, the FED Funds Effective Rate, or as we colloquially call it the interest rate, has reached 5.33%.
Why does this number matter to us? Although we are not borrowing from the FED (though at one time you could get loans directly from the FED), this rate has economic effects on the liquidity of funding and the expected returns of different industries. When interest rates are high, investors typically gravitate to fixed-income investments like bonds. When rates are low, investors take on opportunities that meet their risk appetite.
It’s important to stop for a moment here and recognize that as expected with increasing interest rates, the total amount of VC investment has fallen. We can pose probing questions like a true MBA student and follow the MECE structure (Mutually Exclusive, Collectively Exhaustive) to establish some guiding questions.
Are there companies still being funded by VC firms?
Why are those companies receiving funds over others?
If the VC money has run dry is the room on fire and is the industry going to die?
The Strongest Will Prevail
Companies are still acquiring funding even though the total number of deals and investments has reached record lows. For example in the U.S., Seed and Angel Investments sit near a level lower than last year. Interestingly, the current level is equal to the average trendline if we had not experienced the massive levers being pulled at the FED during 2021/2022.
Who is Getting Money? Give Me the TL;DR Already
The market is right-sizing and will continue to do so with larger brands within the data community(for example, Snowflake just experienced a version of right-sizing). This also means that some brands will either stop operating completely or seek to be acquired by major corporations. These M&A movements will prove tricky with the increased application of antitrust laws (for example, Apple this past week was sued by the DOJ for monopolizing the smartphone market) and will likely lead to a few ill-fated IPOs or mergers of equals (for example, recent Customer Success tool Catalyst merged with Totango).
The most positive outcome of this time, which we as a community should focus on, is that the best brands often start during times of turmoil, which is right now.
As one of the toughest VC funding years continues, I am excited about the future and what is coming for the data landscape. I have no doubt that in five to ten years we will look back on this time as an era from which some of the most valuable data and analytic product brands emerged.
“The end of artificially cheap money should herald a fairer, more sustainable economy.” - Friends at the Atlantic
Complete speculation: the new hosted model is beautiful, though my tears are real
https://fred.stlouisfed.org/series/FEDFUNDS#