arey

Arey (अरे, sanskrit) — ind. Interjection of calling.

Arey is a simple large language model app.

🤖 Use any llama.cpp or ollama model.
💬 Chat with your favorite local models. CPU friendly 🍀
🙋 Ask the AI model with a single command.
📋 Supercharged in-context learning workflow ❤️ Edit your prompt in any editor and arey will generate a completion on save.
🔓 No telemetry, no internet, nothing to sell. Dedicated to the public domain.

🚧 Much more to come... See Roadmap below.

https://github.com/codito/arey/assets/28766/6b886e49-6124-4256-84d9-20449c783a34

Installation

# Install pipx if needed: `pip install pipx`
# Ensure ~/.local/bin is available in system PATH
pipx install arey

Please use WSL for Windows installation. Troubleshooting notes are here.

Usage

❯ arey --help
Usage: arey [OPTIONS] COMMAND [ARGS]...

  Arey - a simple large language model app.

Options:
  -v, --verbose BOOLEAN  Show verbose logs.
  --help                 Show this message and exit.

Commands:
  ask   Run an instruction and generate response.
  chat  Chat with an AI model.
  play  Watch FILE for model, prompt and generate response on edit.

On the first run, arey will create a configuration file in following location:

• ~/.config/arey/arey.yml for Linux or Mac systems.
• ~/.arey/arey.yml for Windows.

Please update the models section in the config yml to your local model path.

1. Ask Arey something!

arey ask "Who is Seneca? Tell me one of his teachings"

2. Chat with Arey

arey chat

3. Run Arey in play mode

Use to fine-tune a prompt in your editor while arey keeps completing your prompt on every save.

❯ arey play /tmp/arey_playzl9igj3d.md

Welcome to arey play! Edit the play file below in your favorite editor and I'll generate a
response for you. Use `Ctrl+C` to abort play session.

Watching `/tmp/arey_playzl9igj3d.md` for changes...

───────────────────────────────────── 2024-01-21 17:20:01 ──────────────────────────────────────
✓ Model loaded. 0.57s.

Life is short because it passes by quickly and can end at any moment. We should make the most of
our time here on earth and live a virtuous life according to stoicism.

◼ Canceled.

Watching `/tmp/arey_playzl9igj3d.md` for changes...

Development

# Install arey locally in editable mode.
> pip install -e .
> pip install -e .\[test\] # optional, if you wish to run tests

# Install with samples dependency if you wish to run them
> pip install -e .\[samples\]

With OPENBLAS, loading time for models is much smaller and inference is about 4-5% faster. Here's how to install llama-cpp-python with OPENBLAS support:

> pip uninstall llama-cpp-python
> CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --verbose

If you've a GPU, try the following installation instead.

> pip uninstall llama-cpp-python
> CMAKE_ARGS="-DLLAMA_CUBLAS=ON" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --verbose

Roadmap

• Chat and task modes for interactive or batch queries
• Define new tasks with only a prompt, no code. See docs/samples directory for examples.
• Markdown formatting for chat mode.
• Ollama support.
• Command support in chat. E.g., logs, change model, copy, clear, etc.
• Discover prompts from user directory
• Manage prompts and create new interactively
• Download models and manage them
• Release v0.1
• Add textfx
• Add offline knowledge bases and RAG. See https://library.kiwix.org/#lang=eng

License

Dedicated to the public domain with CC0. We'll be delighted if this tool helps you positively 💖