EleutherAI

A framework for evaluating language models

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

LLM application: Pythia-1|1.4|2.8|6.9|12B

Llemma 7B is a language model for mathematics. It was initialized with Code Llama 7B weights, and trained on the Proof-Pile-2 for 200B tokens. Llemma models are particularly strong at chain-of-thought mathematical reasoning and using computational tools for mathematics, such as Python and formal theorem provers.