aboutsummaryrefslogtreecommitdiffstats

Privacy-First Command-Line AI for Linux

AI_ENV

Unlock the power of AI—right from your Linux terminal.

This project delivers a fully local AI environment, running open source language models directly on your machine.

No cloud. No GAFAM. Just full privacy, control, and the freedom to manipulate commands in your shell.

How it works

  • Ollama run language models on the local machine.
  • openedai-speech provides text-to-speech capability.
  • nginx add an authentication to the API.
  • AIChat is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.

Everything is free, open-source, and automated using Docker Compose and shell scripts.

It doesn't require an internet connection to work once the models have been downloaded.

Requirements

To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.

As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the qwen2.5:7b model.

The model qwen2.5-coder:32b is usable but slow with this configuration.

Note that more modest models like llama3.2:3b require much fewer resources and still allow you to do a lot of things.

On GNU/Linux, you must use the NVIDIA Container Toolkit.

Note that it is probably possible to run the project on other GPUs or modern MacBooks, but this is not the purpose of this project.

How to launch the server

Choose the models you wish to use in the docker-compose.yaml file.

environment:
      - MODELS=....

Add an API key to secure server access by adding a .env file like this:

LLM_API_KEY=1234567890

Create a user authentication for aichat Web UI:

htpasswd -c src/nginx/htpasswd user

Next, start the servers and their configuration with Docker Compose:

docker compose up --build -d

Then wait for the models to finish downloading using the following command to display the status:

docker-compose logs -f ollama_provision

How to use

The setup_desktop.sh script allows for copying a compiled static version of AIChat from a container to your host and configuring the tool.

AIChat essentials

A request to populate a demo database:

aichat "10 fictitious identities with username, firstname, lastname and email then display in json format. The data must be realistic, especially from known email domains."

Request a snippet of code:

aichat -m ollama:qwen2.5-coder:32b -c "if a docker image exist in bash"

To launch a chatbot while maintaining context:

aichat -s

Pipe a command and interpret the result:

ps aux | aichat 'which process use the most memory'

Using roles:

aichat -r short "tcp port of mysql"
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"

Go to the AIChat website for other possible use cases.

Text To Speech

For this features, use the speech.sh script like this:

./speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."

How to Use Remotely

The API authentication via Nginx allows you to open the API on the internet and use it remotely.

By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.

This way, you can securely use this environment remotely.

To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .

Example:

export TTS_API_HOST="https://your-remote-domain"
./tools/speech.sh ...

Web UI

A web application to interact with supported LLMs directly from your browser is available at http://127.0.0.1:8001/playground.

A web platform to compare different LLMs side-by-side is available at http://127.0.0.1:8001/arena.