LLaMA: Open and Efficient Foundation Language Models
Project description
LLaMA
This repository is intended as a minimal, hackable and readable example to load LLaMA models and run inference. In order to download the checkpoints and tokenizer, fill this google form
Setup
In a conda env with pytorch / cuda available, run
pip install pyllama
Download
Once your request is approved, you will receive links to download the tokenizer and model files.
Edit the download.sh script with the signed url provided in the email to download the model weights and tokenizer.
Inference
The provided example.py can be run on a single or multi-gpu node with torchrun and will output completions for two pre-defined prompts. Using TARGET_FOLDER as defined in download.sh:
torchrun --nproc_per_node MP example.py --ckpt_dir $TARGET_FOLDER/model_size --tokenizer_path $TARGET_FOLDER/tokenizer.model
Different models require different MP values:
| Model | MP |
|---|---|
| 7B | 1 |
| 13B | 2 |
| 30B | 4 |
| 65B | 8 |
Model Card
See MODEL_CARD.md
License
See the LICENSE file.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for pyllama-0.0.1.dev2-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 3164b9aa00ad1cf5378944243cdf10a8e2584d51f9199d5330863d6fb21b995f |
|
| MD5 | b1e6b2d3ee1cdbe68f349948cec3df3c |
|
| BLAKE2b-256 | a8ee708a47d46f882d1565acff2162cdaac3d64c4b8876971c1114ac985c130a |
