Earlier this year, GitHub launched Copilot Chat, a chatbot focused on ChatGPT-style programming for organizations enrolled in Copilot for Business. Copilot Chat most recently came to individual Copilot customers — those who pay $10 a month — in beta. And now, GitHub is making Chat generally available to all users.
Starting today, Copilot Chat is available in the sidebar in IDEs, Visual Studio Code, and Microsoft Visual Studio — included as part of the paid GitHub Copilot tiers and free for verified teachers, students, and maintainers of certain open source projects.
“As the home of the world’s developers, we brought to market what is now the most widely used AI development tool in history,” Shuyin Zhao, vice president of product management at GitHub, told TechCrunch in an email interview. “And completing the code was just the beginning.”
Little else about Copilot Chat has changed since the beta version.
The chatbot is still powered by GPT-4, OpenAI’s flagship AI generation model, tuned specifically for developer scenarios. Developers can ask Copilot Chat in natural language for real-time guidance, for example asking Copilot Chat to explain concepts, identify vulnerabilities, or write unit tests.
Like all artificial intelligence generation models, the model on which Copilot Chat is based, GPT-4, was trained on publicly available data — some of which is copyrighted or under limited license. Vendors, including GitHub, disagree fair use The doctrine protects them from copyright claims. But that didn’t stop coders archiving class action lawsuits against GitHub, Microsoft (GiHub’s parent company), and OpenAI for what they claim are open source licensing and IP infringements.
I asked Zhao if codebase owners will have the opportunity to opt out of the training now, should they wish to do so. He said there is no new mechanism for this with the wider release of Copilot Chat and instead suggested codebase owners make their repositories private to prevent them from being included in future training sets.
I have to imagine that the owners of the codebase won’t take too kindly to this suggestion – there are many reasons to keep copyrighted code public, not the least of which is crowdsourcing bugs. But GitHub apparently isn’t willing to back down on training data exceptions — or not yet, at least.
Genetic AI models, including GPT-4, also have a tendency to hallucinate or confidently make up facts — something that is particularly problematic in the field of coding. According to recent Stanford studydevelopers who use AI assistants to code tend to produce code that is less secure than those who don’t use AI assistants, in part because AI assistants introduce snippets of code that have bugs or have been deprecated.
Zhao said GPT-4 has “better performance” against hallucinations than the older model that once powered Copilot, and demonstrated exploit mitigation features such as filters for unsafe code patterns, which alert Copilot Chat users for vulnerabilities such as hard-coded credentials, SQL injections, and path injections. However, he emphasized the importance of close human scrutiny of any AI-suggested code.
“GitHub Copilot is powered by OpenAI’s models, which we’ve found to be the best models for the services we offer today,” said Zhao. “We’re in a really strong position to continue to empower developers with the AI tools they need to build better, more secure software at scale — and have fun while doing it.”
In October, Microsoft CEO Satya Nadella told analysts that Copilot had 1 million paying users and ~37,000 enterprise customers. However, it’s up to GitHub to make Copilot even more attractive so that it doesn’t lose ground to competitors and therefore lose cash.
According in a Wall Street Journal article, Copilot loses an average of $20 per month per user, with some customers costing GitHub as much as $80 per month. The high price of running the underlying AI models is reportedly to blame — a problem also faced by GenAI-encoded startup Kite, which forced it to shut down early last December.
As GitHub struggles to make Copilot profitable, Amazon continues to upgrade CodeWhisperer, perhaps Copilot’s best-resourced competitor.
In April, Amazon made CodeWhisperer free to developers without any usage restrictions. This month also saw the release of the CodeWhisperer Professional Tier, which added single sign-on with AWS Identity and Access Management integration, as well as higher limits on scanning for security vulnerabilities. A business plan for CodeWhisperer was released in September. And to in early November, Amazon “optimized” CodeWhisperer to supply “enhanced” suggestions for developing applications in MongoDB, the open source database manager.
In addition to CodeWhisperer, Copilot has competition from startups like Magic, Tabnine, Codegen and Laredo, as well as open source models like Meta’s Code Llama and Hugging Face’s StarCoder and ServiceNow.
