Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Generative AI for code generation is becoming increasingly powerful and compact. Stability AI, best known for its stable diffusion text-to-image generative AI technology, has announced its first new AI model of 2024: the commercially licensed Stable Code 3B, available via membership.
Stable Code 3B is a 3-billion parameter model designed for code completion in software development. Despite its relatively small size, it can run locally on laptops without dedicated GPUs while offering competitive performance compared to larger models like Meta’s CodeLLaMA 7B.
The move towards smaller, more capable models began in late 2023 with Stability AI’s release of models like StableLM Zephyr 3B for text generation. Stability AI first introduced Stable Code in August 2023 and has since been improving the technology.
Enhancements to Stable Code 3B include not only suggesting new lines of code but also filling in larger missing sections, an advanced feature known as Fill in the Middle (FIM). The model’s training has been optimized with an expanded context size using Rotary Position Embeddings (RoPE), allowing for a context length of up to 100k tokens. This technique is also used by other large language models, including Meta’s Llama 2 Long.
Built on Stability AI’s Stable LM 3B natural language model, Stable Code 3B has been further trained with software engineering data to enhance its code completion capabilities while maintaining strengths in general language tasks. Its training data includes code repositories, programmer forums, and other technical sources, covering 18 different programming languages. Stability AI claims that Stable Code 3B performs exceptionally well on benchmark tests across multiple languages, including Python, Java, JavaScript, Go, Ruby, and C++. Early benchmarks suggest it matches or exceeds the completion quality of models twice its size.
The market for generative AI code generation tools is competitive, with larger models like Meta’s CodeLLaMA 7B being popular choices. On the smaller side, the StarCoder LLM, co-developed by IBM, HuggingFace, and ServiceNow, is another notable option. Stability AI claims that Stable Code 3B outperforms StarCoder across several programming languages, including Python, C++, JavaScript, Java, PHP, and Rust.
Stable Code 3B is available for commercial use through Stability AI’s new membership subscription service, launched in December. Members gain access to Stable Code 3B and other AI tools in Stability AI’s portfolio, including SDXL stable diffusion image generation tools, StableLM Zephyr 3B for text content generation, Stable Audio for audio generation, and Stable Video for video generation.