Software developers have never been more productive. The recent proliferation of AI-powered developer tools is enabling engineers to build, test, and deploy applications at unprecedented speeds. Catalyzed by the initial release of GitHub Copilot in 2021, the impact of AI is now being felt across nearly all parts of the software development lifecycle.
These tools leverage code Large Language Models, or code LLMs, to assist developers in solving a breadth of programming-related tasks. In this post, we’ll discuss the current capabilities of code LLMs and highlight several examples of how they are being used across the emerging AI-powered developer toolchain.
What are code LLMs capable of?
Code LLMs are a unique class of LLMs that have been trained on a massive amount of source code across a variety of programming languages. As a result, they are remarkably versatile and adept at common programming-related tasks. Code generation was one of the first widely recognized use cases for code LLMs given its surprisingly accurate suggestion quality and integration into existing workflows without interrupting developer flow state. Within a developer’s IDE, code generation tools were capable of making suggestions about how a single line or series of lines of code should be finished.
Beyond code generation, code LLMs have demonstrated that they are increasingly capable of code reasoning tasks. These tasks may include prompting a code LLM to provide answers to questions about an underlying codebase or instructing it to directly execute operations over code. For more complex, multi-step code reasoning tasks, frameworks such as Parsel are utilized to enforce structure around how code LLMs break down tasks to achieve more accurate results. Recent examples with LLM-based agent frameworks such as BabyAGI and Auto-GPT have also shown the potential for code LLMs to perform certain larger, more complex programming-related tasks autonomously.
What types of developer tools are being built with code LLMs?
Since the launch of GitHub Copilot, many companies have been started to take advantage of code LLMs. Currently, the majority of tools are focused on pre-production workflows such as developing, testing, and maintaining code. Below, we highlight several categories that make up the AI-powered developer toolchain.
Code and test generation tools: Many startups have come out with promising alternatives to GitHub Copilot. Broadly speaking, these tools aim to compete with Copilot on the basis of suggestion quality, latency, security, and cost. Separate from the differences between products, these companies face the challenge of competing against the distribution advantage that GitHub has.
Code generation tools such as Tabnine, Codeium, Mutable, CodeComplete, and Tabby provide self-hosted offerings that can be fine-tuned on a customer’s codebase to achieve superior suggestion quality. Test generation tools such as Codium, Ventrilo, and Nova help automate the process of writing tests and enable QA teams to more accurately measure code integrity.
Terminals and IDEs: AI-native terminals and IDEs are reimagining the surface area where code is written and development work gets done.
New terminals such as Warp are rethinking an outdated developer experience and augmenting the command-line interface with powerful suggestion and reasoning capabilities using code LLMs. New IDEs such as Cursor are reimagining the interaction pattern between developers and code LLMs through a prompt-based interface.
Code maintenance tools: Developers spend a considerable amount of time on their day-to-day performing code maintenance, which takes precious time away from writing net-new application and business logic. As enterprise codebases grow larger and more complex, routine code maintenance tasks become non-trivial and time-intensive to resolve manually.
Code maintenance tools such as Grit and Second are leveraging code LLMs to automate tasks such as codebase migrations and code refactoring. They tap into a lesser-known capability of code LLMs to perform verifiable code transformations that are backed by Abstract Syntax Trees.
Semantic code search tools: Code LLMs allow developers to easily search and understand codebases using just natural language. Natural language serves as an abstraction over domain-specific programming languages and enables developers of varying skill levels to more easily navigate unfamiliar codebases.
Semantic code search tools such as Sourcegraph Cody, Bloop, and Buildt provide natural language interfaces for developers to search and understand code without needing to leave their IDE.
Coding agents: Newer LLM-based agents are increasingly focused on code.
Coding agents such as Magic, Factory, poolside, and Smol AI help developers solve complex, multi-step programming tasks without requiring developers to write a single line of code.
Our predictions for AI-powered developer tools
This new toolchain is unfolding in real time and we believe that it has the potential to completely transform how we build software. Although we are still in the early days of this transformation, we want to highlight two predictions below:
1. It will primarily be open
Similar to our view that AI-native infrastructure will be open, we expect the same to hold true for dev tools. Many developers already want or expect their tools to be open-source. Recent announcements of open-source code LLMs such as replit-code-v1-3b and StarCoder have laid the foundation for new open-source AI-powered developer tools, and we are excited about their potential.
2. AI will significantly reduce developer toil
Unusual Co-founder Jyoti Bansal often points to developer toil as one of the largest threats to software innovation and developer productivity. By leveraging AI-powered tools to tackle this problem, organizations can free up developers’ time to work on creative programming tasks.
We are very excited about the greenfield opportunities in AI-powered dev tools. If you are building in this space, we’d love to hear from you — please reach out to simon@unusual.vc or wei@unusual.vc.
Read more on AI
Why the future of AI-native infrastructure will be open
How good is your language model? Nobody knows yet
Create what you dream: The emerging Generative AI video and 3D landscape
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
