Meta has released Code Llama, a large language model (LLM) variant of Llama 2 specifically designed for programmers.

Meta has been taking a very different approach to AI and LLMs than its Big Tech rivals, making its models available for free and open-sourcing them. The company is continuing that track record with Code Llama, making it available free for both research and commercial use.

The company outlined the LLMs capabilities, as well as its availability in three different sizes:

Code Llama is a code-specialized version of Llama 2 that was created by further training Llama 2 on its code-specific datasets, sampling more data from that same dataset for longer. Essentially, Code Llama features enhanced coding capabilities, built on top of Llama 2. It can generate code, and natural language about code, from both code and natural language prompts (e.g., “Write me a function that outputs the fibonacci sequence.”) It can also be used for code completion and debugging. It supports many of the most popular languages being used today, including Python, C++, Java, PHP, Typescript (Javascript), C#, and Bash (see our research paper for a full list).

We are releasing three sizes of Code Llama with 7B, 13B, and 34B parameters respectively. Each of these models is trained with 500B tokens of code and code-related data. The 7B and 13B base and instruct models have also been trained with fill-in-the-middle (FIM) capability, allowing them to insert code into existing code, meaning they can support tasks like code completion right out of the box.

Programmers looking to get started with Code Llama can learn more here.