GitHub Copilot’s AI model improved with new features

Since we first launched GitHub Copilot, we’ve worked to improve the quality and responsiveness of our code suggestions by upgrading the underlying Codex model. We’ve also developed a new security vulnerability filter to make code submissions on GitHub Copilot more secure, helping developers identify unsafe coding patterns as they work.

This week we are launching new updates for Copilot for Individuals and Copilot for Business. New Easy Signup for Organizations— Makes GitHub Copilot more powerful and responsive for developers.

Let’s dive in.

Stronger AI models and better code suggestions

To improve the quality of code suggestions on GitHub Copilot, we’ve updated the underlying Codex model, resulting in significantly better quality code suggestions and faster time to deliver those suggestions to users.

Case in point: When we first launched GitHub Copilot for individuals in June 2022, on average over 27% of developer code files were generated by GitHub Copilot. GitHub Copilot currently lags behind developers’ code by an average of 46% across all programming languages. In Java, that number jumps to 61%.

This work means that developers using GitHub Copilot can now code faster than before thanks to more accurate and responsive code suggestions.

Here are the key technical improvements we made to achieve this:

• Upgraded AI Codex model: Upgraded GitHub Copilot to new OpenAI Codex models. This improves code synthesis results.

• Improved contextual understanding: Improved GitHub Copilot with a new paradigm called Fill-In-the-Middle (FIM). This allows developers to provide better crafting prompts for code suggestions. Instead of only considering code prefixes, we also leverage known code suffixes, leaving gaps in the middle for GitHub Copilot to fill. In this way, we now have more context about the intended code and how it interacts with the rest of the program. His FIM at GitHub Copilot has developed a variety of strategies to consistently generate higher quality code suggestions and serve them without adding delays.

• lightweight client-side model: Updated the GitHub Copilot extension for VS Code with a lightweight client-side model to improve the overall acceptance rate of code suggestions. To do this, GitHub Copilot uses basic information about the user’s context (such as whether or not the last suggestion was accepted) to avoid unnecessary suggestions when they might disrupt a developer’s workflow. Reduce frequency. This reduced his unnecessary suggestions by 4.5% and allowed GitHub Copilot to better serve each developer using it. And his second improved iteration of this client-side model, which shipped in January 2023, further improved overall code acceptance rates.

Rule out security vulnerabilities with new AI systems

We also launched an AI-based vulnerability filtering system that blocks unsafe coding patterns in real time, making GitHub Copilot suggestions even more secure. Our model covers the most common brittle coding patterns. hardcoded credentials, SQL injectionand path injection.

The new system leverages LLM to approximate the behavior of static analysis tools. And because GitHub Copilot runs advanced AI models on powerful computing resources, it’s incredibly fast and can even detect vulnerable patterns in incomplete fragments of code. This means that unsafe coding patterns are immediately blocked and replaced with alternative suggestions.

Here are some examples of vulnerable patterns generated by language models::

Note: GitHub Copilot can generate new strings for identification forms, such as keys and passwords, by mimicking patterns found in your data. The examples below do not necessarily represent existing or available credentials. Anyway, displaying these credentials in code is an insecure coding pattern that our solution addresses.

This application of AI fundamentally changes the way vulnerabilities are addressed, offering greater value than traditional, limited vulnerability detection tools.

Traditionally, security vulnerability detection tools were employed at build or release time along with repository-wide static code analysis tools. There are three important reasons for this.

• Because the code is fully formed, scanning tools have the full context of your codebase and can track dependencies that can determine if your code is vulnerable.
• Static analysis tools can safely assume that the code is syntactically correct and can leverage language compilers to improve accuracy.
• These phases do not have meaningful time constraints, so static analysis tools can spend more time detecting vulnerabilities and are not designed for rapid detection.

GitHub Copilot allows developers to reach magical flow states. This includes providing fast and accurate vulnerability detection directly from the editor. This filtering mechanism is the first important step in enabling developers to build more secure code with her GitHub Copilot. And continue to teach LLMs to distinguish between vulnerable and non-vulnerable code patterns. You can help improve GitHub Copilot by reporting vulnerable patterns that you identify in code suggestions. copylot-safety@github.com.

Start a build on GitHub Copilot

Upgraded AI model, better code suggestions, improved responsiveness and enhanced security – all these improvements are now available to developers using Copilot for Individual and Copilot for Business . We will continue to improve the experience for developers using GitHub Copilot. These updates are just the beginning.