# Privacy-First Command-Line AI for Linux

![AI_ENV](logo.webp)

Unlock the power of AI right from your Linux terminal.

This project delivers a fully local AI environment, running open source language models directly on your machine. 

No cloud. No GAFAM. Just full privacy, control, and the freedom to manipulate commands in your shell.

## How it works

* [Ollama](https://ollama.com/) run language models on the local machine.
* [speaches.ai](https://speaches.ai) provides text-to-speech capability.
* [nginx](https://nginx.org/en/) adds an authentication to the API.
* [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.

Everything is free, open-source, and automated using Docker Compose and shell scripts. 

It doesn't require an internet connection to work once the models have been downloaded.

## Requirements

To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.

As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [gemma3:12b-it-qat](https://ollama.com/library/gemma3:12b-it-qat) model.

On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit).

Note that it is probably possible to run the project on other GPUs or modern MacBooks, but this is not the purpose of this project.

## How to launch the server

Choose the models you wish to use in the `docker-compose.yaml` file.

```bash
environment:
      - MODELS=...
```

Add an API key to secure server access by adding a `.env` file like this:

```bash
LLM_API_KEY=1234567890
```

Create a user authentication for aichat Web UI:

```bash
htpasswd -c src/nginx/htpasswd user
```

Next, start the servers and their configuration with Docker Compose:

```bash
docker compose up --build -d
```

Then wait for the models to finish downloading using the following command to display the status:
```bash
docker-compose logs -f ollama_provision
```

## How to use

The `setup_desktop.sh` script allows you to copy a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configure the tool.

### AIChat essentials

A request to populate a demo database:
```bash
aichat "10 fictitious identities with username, firstname, lastname and email then display in json format. The data must be realistic, especially from known email domains."
```

Request a snippet of code:
```bash
aichat -m ollama:gemma3:12b-it-qat -c "if condition to check if a docker image exist in bash"
```

To launch a chatbot while maintaining context:
```bash
aichat -s
```

Using shell pipe:
```bash
cat README.md | aichat -m ollama:llama3.1:8b 'Check the spelling, grammar, and phrasing. Anwser the correction using diff format'
```

Using roles:
```bash
aichat -r short "tcp port of mysql"
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"
```

Go to the [AIChat](https://github.com/sigoden/aichat) website for other possible use cases.

### Text To Speech

For this features, use the speech.sh script like this:

```bash
./tools/speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
```

## How to Use Remotely

The API authentication via Nginx allows you to open the API on the internet and use it remoteli.

By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption. 

This way, you can securely use this environment remotely.

To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .

Example:
```bash
export TTS_API_HOST="https://your-remote-domain"
./tools/speech.sh ... 
```

## Web UI

A web application to interact with supported LLMs directly from your browser is available at [http://127.0.0.1:8001/playground](http://127.0.0.1:8001/playground).

A web platform to compare different LLMs side-by-side is available at [http://127.0.0.1:8001/arena](http://127.0.0.1:8001/arena).