2023 was a turbulent year by most measures. We experienced the failure of Silicon Valley Bank. War broke out in the Middle East. The most prominent AI company in the world nearly collapsed in a matter of days. Throughout it all, venture capital investment activity slowed and many startups struggled as customer budgets for software line items were delayed, curtailed, or canceled. However, as Unusual’s co-founder and Managing Partner, John Vrionis, recently wrote:

“At the same time there is optimism and hope. Practically every leader is charting a course to take advantage of the transformative abilities of AI technology for their product. As we experience a generational shift in technical power, the blistering pace of innovation can be felt by teams everywhere. Weekly news is significant as leading research is published, new product launches occur, and regulation emerges.”

As we look ahead to 2024, we see bright days ahead for startup founders solving hard, important problems in infrastructure software. Here are our 10 predictions for next year.

‍

1. A handful of LLMs will become the de facto standard

Earlier this year, we wrote that work on open-source AI, especially LLMs, was the most significant driver in advancing the AI-native stack. These models close the gap on rivaling the performance of proprietary models and will make significant inroads in adoption, especially now that they are starting to be made available via many other platforms. While there was a proliferation of open LLMs this year, we predict that next year we will start to see mindshare coalesce around a handful of open-source LLMs, namely ones such as Llama 2 and Mistral 7B, alongside proprietary alternatives. Consider diffusion models, where we saw Stable Diffusion become the de facto open standard alongside proprietary offerings like Midjourney. A long tail of open LLMs will continue to exist, but only a few will capture substantial market share.

‍

2. AI infrastructure will become use case specific

Infrastructure is a means to an end and that end is almost always a user-facing application. While much of today’s AI infrastructure — from foundation models to LLMOps tools — has been designed to be general-purpose, the demands of specific application use cases will narrow the focus for infrastructure solutions. For example, applied AI in verticals such as healthcare, legal, or financial services will require new domain-specific models. More companies will train and/or fine-tune their own models on platforms such as Together. Embedding models from companies such as Voyage, or model providers including OpenAI, will support specialized datasets. Already, we’re seeing organizations looking to start with AI by leveraging it for a common set of business functions. General-purpose AI infrastructure will always be available and needed, but these “killer” app use cases will give rise to new opportunities for AI infrastructure founders.

‍

3. AI evaluation and security will come into focus

Our recent podcast with Naveen Rao, co-founder and CEO of MosaicML (acquired by Databricks for $1.3B), highlighted “unsolved” problems in AI that will present blockers for most companies to use AI in production. These include evaluation, which startups such as Quotient AI are addressing to ensure model accuracy, observability to maintain operational visibility into AI systems, and security to protect how sensitive data is handled by models.

‍

4. The agentic future will start to take shape

One of the most interesting developments in 2023 was the emergence of AI agent frameworks such as AutoGPT and BabyAGI, which in theory allow anyone to build an agent for any task. But what tasks? There have been fun PoCs such as using agents to automate DoorDash orders, but we won’t see agents en masse until there are clear workflows they can be used for and these frameworks mature in reliability and robustness. We expect that applied AI companies will be the first to selectively incorporate agent architectures to enable new workflows.

‍‍

‍5. The next $100M ARR enterprise gen AI product will emerge

In October, we learned that GitHub Copilot had surpassed $100M in ARR, one of the first gen AI products to reach this milestone (save OpenAI/ChatGPT). This is incredibly impressive considering Copilot was released only two and a half years ago and came out of technical preview in mid-2022. It also highlights the advantages in user base and distribution that incumbents will look to leverage to capitalize on massive opportunities. Where will the next $100M gen AI product focused on enterprises come from? We’ll likely know in the next 12 months.

‍

6. AI will “show up” in the public markets

Despite the huge potential and promised impact of AI, in the public markets, investors have mostly rewarded only a handful of the largest tech companies, e.g., NVIDIA and Microsoft, to date. The large cloud providers will continue to accrue significant spend now that offerings such as Amazon Bedrock are generally available. But in the next year, public investors will reward or ding companies who are (or aren’t) able to effectively incorporate AI to sustain competitive product advantages and/or to drive more efficient cost structures through greater infrastructure automation. As a result, AI has become a topic of strategic discussion in many boardrooms.

‍

7. AI regulation will become clearer and less clear at the same time

This year, angst over new risks introduced by AI catalyzed key actions by governments. The EU advanced comprehensive AI legislation while the White House issued an Executive Order. It’s up to NIST to determine the relevant US standards, guidelines, and best practices. With likely inconsistencies across geopolitical divides, smaller companies will be more challenged than larger ones to figure out relevant rules and reporting requirements. At the same time, this will present an opportunity for a new set of startups to enable responsible AI by providing tools for AI security and compliance. Companies such as Credo are helping enterprises with AI governance today. Who will be the next Vanta or Drata for AI compliance?

‍

8. Cybersecurity will continue to experience aggressive platform expansion 

New companies will emerge at the intersection of AI and security, both to secure AI and to use AI to streamline security workflows. But the main headline in 2024 for cyber will continue to be platform expansion. Palo Alto acquiring Talon Security and Dig Security, CrowdStrike buying Bionic, and Tenable scooping up Ermetic were just some of the notable security acquisitions this year. As long as CISOs continue to want fewer tools (and vendors) to manage, incumbents looking to grow NDR will continue to see opportunities to acquire best-in-class technology and bundle it with existing platforms. And strong challengers such as Wiz will continue strategically adding capabilities to their platforms. Where does this leave opportunities for startup founders? New problems are emerging in gen AI, non-human identity, and other areas.

‍

‍9. The modern data stack will consolidate

Over the last decade, data teams have cobbled together various tools to understand their key business metrics and answer all sorts of questions. There’s a data warehouse (Snowflake) and/or lakehouse (Databricks), something for ETL (Fivetran) and data transformation (dbt), orchestration (Airflow), BI and analytics (Looker), visualization (Tableau), observability (Monte Carlo), governance, cataloging, etc. It’s too complex. In 2024, we’ll see both vendors and customers move towards consolidation, including enabling greater automation across data pipelines by incorporating best practices from CI/CD pipelines and DevOps with open-source projects such as SQLMesh. We expect more acquisitions, integrations, and product expansion as vendors look to upsell data leaders looking to consolidate capabilities.

‍

10. Front-end dev tools will go full-stack

2023 saw continued innovation in developer tools, especially those for front-end engineers. Newer Javascript toolchains and runtimes such as Oven and Deno are maturing and “mini clouds” such as Vercel are growing their ecosystem with integrations for all the key components needed to build full-stack web applications, e.g., database (Neon), authentication (Clerk), and workflow engine (Inngest). The lines between front-end and back-end will continue to blur and developers will seek products that give them the best DevEx, least complexity, and fastest time-to-ship workflows, driving growth of Cloudflare, Fly.io, and platforms other than the large cloud providers.

‍

And there you have it — our predictions for infrastructure software in the coming year. If there’s one thing that’s certain, it’s going to be an interesting year. Cheers to 2024!

‍