Github Delivers Improved Ai Model New Capabilities And Business Edition For Copilot

GitHub Delivers Improved AI Model, New Capabilities, and Business Edition for Copilot March 03, 2023 By: Melinda-Carol Ballou, Katie Norton, Michele Rosen IDC's Quick Take On February 14, GitHub announced several updates to its Copilot coding assistant to improve the developer experience, as well as the general availability of GitHub Copilot for Business, which includes tools for managing Copilot across the enterprise. While companies digest the latest news about generative AI, developers are already using GitHub Copilot to increase their productivity by significantly accelerating the production of code. Product Announcement Highlights GitHub sent a Valentine's Day present to developers this year by releasing updates to its Copilot code suggestion tool as well as making GitHub Copilot for Business generally available. The February 14 updates include: Fill-in-the-Middle — Copilot now considers context both before and after the cursor to suggest code that is more contextually relevant. The Copilot extension for VS Code now contains a custom, lightweight model to reduce the frequency of unwanted suggestions An AI-powered security filter to help identify some insecure coding patterns A new version of the underlying Codex model GitHub also announced the general availability of Copilot for Business, which adds enterprise -level governance features: The ability to grant or revoke access to Copilot at the enterprise or organization level Activation at the team or enterprise level of the duplication detection filter Support for VPN proxies via self-signed certificates IDC's Point of View GitHub' s Copilot announcements come at a time when the demand for tools that boost developer productivity continues to increase. The large majority (71%) of respondents to IDC's 2022 PaaSView and the Developer Survey believe their organization's current IT environment will have to undergo major transformation over the next five years to support its business strategies and goals. Businesses are more dependent on software than ever and therefore more dependent on developers. IDC continues to predict that the current shortage of developers will continue for the foreseeable future, despite lingering economic headwinds. With almost a year' s worth of feedback and data on Copilot's use under its belt, GitHub has made improvements on two fronts — to the tool itself and to the tool's management features for enterprise use. Developers will appreciate the addition of a client-side model to help Copilot avoid making IDC #lcUS50455323 unwanted code suggestions, which can interrupt the developer's flow; IT managers will appreciate the fine-grained control over developers' access to Copilot and the ability to block code suggestions that are similar to code in public GitHub repositories for all developers in the enterprise. The mix of new features highlights GitHub' s intent to provide a tool that both increases developer productivity and developers will actually want to use. In the announcement, GitHub emphasized two data points: developers using Copilot are coding up to 55% faster, and up to 75% of developers who use GitHub Copilot reported feeling more fulfilled by their work. Given that 80% of developers surveyed by IDC indicate they have almost complete freedom to choose the development tools they use, it makes sense for GitHub to make sure that its improvements to developer productivity are also appreciated by developers. GitHub' s efforts appear to be working. The company reports that developers are now accepting more of Copilot's suggestions: in code written using Copilot, an average of 46% of the code is built using the tool across all programming languages, a number that jumps to 61% for Java code. This is an improvement over the 40% reported when Copilot was released to general availability in July 2022, which was an improvement over the 35% reported during the earlier preview. It's not clear yet what the ceiling will be for the amount of code generated by Copilot: 60%? 80%? As usage increases, it will be important to keep in mind the concomitant need to address potential software quality issues and the impact of architecture and design on application performance, flexibility, and resilience. This is especially relevant as more nonprofessional developers use coding assistants to create code. These developers also need to have access to value stream, project, and collaborative work management tools that provide visibility into the business priorities, processes, and value streams driving their organization's software initiatives. How Developers Are Using Copilot What is clear is that patterns are beginning to emerge in how developers use Copilot. Some of the common use cases are: Writing boilerplate code: Since Copilot is trained on code in GitHub's public repositories, it's especially good at suggesting the kind of repetitive code that needs to be included in every piece of software (the details of which vary by programming language and use case). Since boilerplate code is stereotyped, it's a perfect candidate for automatic code generation and the resulting increase in developer velocity. Test generation: Many developers find writing tests for their code tedious, but testing is increasingly important in the context of rapid, continuous deployment, particularly as business developers increasingly leverage Copilot to generate applications. GitHub Copilot's ability to autogenerate unit tests is a win-win for everyone involved: developers, DevOps engineers, and IT managers. However, while unit tests are a first step, more will have to be done to address the need for broader and deeper automated software quality (ASQ) capabilities as part of a Copilot implementation across testing phases and at the enterprise level. Helping developers get up to speed on new technologies and frameworks: Copilot can help developers cope with the proliferation of APIs, frameworks, and libraries that have made software development more complex than ever. Less experienced developers can use the tool to explore existing code; more experienced developers can use Copilot to explore how to incorporate new technologies into their code. Using Copilot in this way reduces the cognitive ©2023 IDC #lcUS50455323 2 load on developers caused by cont ext switching between applications to search sites like Stack Overflow for code snippets and solutions to coding problems. Copilot, ChatGPT, and Enterprise Use of Generative AI Since OpenAI's release of ChatGPT in November, interest in and awareness of generative AI has exploded. While the chatbot unquestionably represents a leap forward in the public's use of generative AI, there are many questions about how this technology will be used in the enterprise. That question is easier to answer where coding assistants are concerned. While companies experiment with the business applications of chatbots, tools like Copilot represent a practical way to begin leveraging generative AI to achieve digital transformation and innovation. Developer scarcity impacts many enterprises that are unable to meet the overwhelming demand for software solutions and creates challenges for developers trying to keep up with those demands. The addition of generative AI-based tools such as Copilot can help address these challenges by onboarding developers more quickly, assisting developers to bridge skills gaps, empowering a broader set of developers, and improving the efficiency of professional developers. GitHub' s February updates to Copilot include a new version of the underlying OpenAI Codex model, which generates Copilot' s real-time code suggestions. Both Codex and ChatGPT are based on OpenAI's GPT-3 large language model. Research by OpenAI indicates that improvements to the underlying model, such as Fill-in-the -Middle — the ability to consider context from both before and after the cursor — improves the model's performance for generating code more than for natural language. The researchers explained that this result is somewhat expected "given that code is a formal language and, as such, has more structure and less uncertainty." Coding assistants like GitHub Copilot may face fewer technical obstacles to success than natural language chatbots such as ChatGPT, but they continue to face legal challenges, including a class action lawsuit targeting the use of public repositories as training data, and enterprise concerns about the potential legal and security risks inherent in incorporating generated code into an enterprise code base. GitHub has already begun addressing the latter concerns with the addition of management and security tools in Copilot for Business. The addition of AI-based vulnerability filtering capabilities provides developers with real-time feedback that blocks and replaces insecure coding patterns with alternative suggestions, even in fragments of code. The model currently covers common vulnerable coding patterns such as hardcoded secrets, SQL injections, and path injections. The new security capabilities are table stakes for the success of Copilot for Business in enterprises that are facing increasing security threats. However, AI-enhanced static analysis also has the potential to address a gap in skills or a shortage of developers with development-related security expertise. IDC's DevSecOps Adoption, Techniques, and Tools Survey (IDC #US48599822, August 2022) found that insufficient knowledge and expertise regarding secure coding practices is the biggest barrier toward empowering developers to find and fix vulnerabilities. Catching and blocking insecure code as part of the code generation process and within the developer 's workflow could help mitigate the developer security capability shortfall, particularly when paired with the adoption of automated DevSecOps tools throughout the software development life cycle, such as GitHub Advanced Security. Beyond security vulnerabilities , many enterprise developers also need information about whether a given code snippet matches public code released under a restrictive license. A GitHub blog post in November described a new feature that would provide the ability to identify the license of the original ©2023 IDC #lcUS50455323 3 source for code suggestions similar to code found in GitHub public repositories, but this feature was not included in the latest release. Such a tool, coupled with the ability to apply enterprisewide or teamwide filters to which licenses are permissible for suggestions, could go a long way to addressing enterprises' licensing concerns. Conclusion In his blog post announcing Copilot for Business, CEO Thomas Dohmke said that GitHub will "integrate AI into every aspect of the developer experience" in the coming years. It's not hard to see how a combination of Copilot, ChatGPT, and "Hey, GitHub!," a tool being developed by GitHub's Next research group to enable developers to interact with Copilot via voice commands, could drastically alter the development process. As the proportion of code generated by tools like Copilot increases, it may become more accurate to describe developers as code curators, code evaluators, or even code architects, instead of as people who write code. That said, it will be vital to keep these humans in control of the development life cycle, given the legal and ethical risks inherent in the use of generative AI. Customers will rely on GitHub and other software vendors to provide additional tools to incorporate security, quality metrics, monitoring tools, and project management into Copilot to mitigate these risks. Subscriptions Covered: Agile Application Life-Cycle, Quality and Portfolio Strategies , DevSecOps and Application Security, Low- Code, No-Code and Intelligent Developer Technologies Please contact the IDC Hotline at 800.343.4952, ext.7988 (or +1.508.988.7988) or sales@idc.com for information on applying the price of this document toward the purchase of an IDC or Industry Insights service or for information on additional copies or Web rights. Visit us on the Web at www.idc.com. To view a list of IDC offices worldwide, visit www.idc.com/offices. Copyright

©2023 IDC #lcUS50455323 4