Ollama run language models on the local machine.
speaches.ai provides text-to-speech capability.
nginx adds an authentication to the API.
AIChat is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.

Everything is free, open-source, and automated using Docker Compose and shell scripts.

It doesn't require an internet connection to work once the models have been downloaded.

Requirements

To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.

As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the gemma3:12b-it-qat model.

On GNU/Linux, you must use the NVIDIA Container Toolkit.

Note that it is probably possible to run the project on other GPUs or modern MacBooks, but this is not the purpose of this project.

How to launch the server

Choose the models you wish to use in the .env file. Put the embedding model first and the default model second.

MODELS=nomic-embed-text,gemma3n:e4b,gpt-oss:20b,llama3.1:8b,llama3.2:3b,gemma3:12b-it-qat

Add an API key to secure server access by adding a .env file like this (you can generate one with openssl rand -hex 16):

LLM_API_KEY=1234567890

Create a user authentication for aichat Web UI:

htpasswd -c src/nginx/htpasswd user

Next, start the servers and their configuration with Docker Compose:

docker compose up --build -d

Then wait for the models to finish downloading using the following command to display the status:

docker-compose logs -f ollama_provision

How to use

The setup_desktop.sh script allows you to copy a compiled static version of AIChat from a container to your host and configure the tool.

AIChat essentials

A request to populate a demo database:

aichat "10 fictitious identities with username, firstname, lastname and email then display in json format. The data must be realistic, especially from known email domains."

Request a snippet of code:

aichat -m ollama:gemma3:12b-it-qat -c "if condition to check if a docker image exist in bash"

To launch a chatbot while maintaining context:

aichat -s

Using shell pipe:

cat README.md | aichat -m ollama:llama3.1:8b 'Check the spelling, grammar, and phrasing. Anwser the correction using diff format'

Using roles:

aichat -r short "tcp port of mysql"
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"

Go to the AIChat website for other possible use cases.

Text To Speech

For this features, use the speech.sh script like this:

./tools/speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."

How to Use Remotely

The API authentication via Nginx allows you to open the API on the internet and use it remoteli.

By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.

This way, you can securely use this environment remotely.

To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .

Example:

export TTS_API_HOST="https://your-remote-domain"
./tools/speech.sh ...

Web UI

A web application to interact with supported LLMs directly from your browser is available at http://127.0.0.1:8001/playground.

A web platform to compare different LLMs side-by-side is available at http://127.0.0.1:8001/arena.