Nvidia, Hugging Face, and ServiceNow have released a family of LLMs called StarCoder2 aimed at code generation that delivers “new standards for performance, transparency, and cost-effectiveness. The family of models include a 3-billion-parameter model trained by ServiceNow, a 7-billion-parameter model trained by Hugging Face, and a 15-billion-parameter model trained by NVIDIA.
StarCoder2 has been trained on 619 programming and can perform specialized tasks like source code generation, workflow generation, text summarization, and more. Nvidia said developers can use it for code completion, advanced code summarization, code snippets retrieval and more to boost their productivity.
In terms of improvements over the original StarCoder LLMs, Nvidia said that the new 3-billion-parameter model matches the performance of the original StarCoder 15-billion-parameter model. This was achieved by using a new code dataset called Stack v2 which is seven times larger than Stack v1; new training techniques also mean that the model can better understand low-resource programming languages like COBOL, mathematics, and program source code discussions.
Commenting on the news, Leandro von Werra, a machine learning engineer at Hugging Face, said:
“The joint efforts led by Hugging Face, ServiceNow, and NVIDIA enable the release of powerful base models that empower the community to build a wide range of applications more efficiently with full data and training transparency. StarCoder2 is a testament to the potential of open source and open science as we work toward democratizing responsible AI.”
StarCoder2 is being made available under the BigCode Open RAIL-M license which allows royalty-free access and use. The source code is available on the BigCode project’s GitHub page while the models are available for download from Hugging Face.
If you want to learn more about the project, just head over to the BigCode project’s Hugging Face page where you can find StarCoder2 and the original StarCoder LLMs.
Source: Nvidia