Update with last components and improve documentation.

This commit is contained in:
ben
2025-08-21 21:52:53 +02:00
parent 2595c44071
commit 576324fada

View File

@@ -2,7 +2,7 @@
![AI_ENV](logo.webp) ![AI_ENV](logo.webp)
Unlock the power of AIright from your Linux terminal. Unlock the power of AI right from your Linux terminal.
This project delivers a fully local AI environment, running open source language models directly on your machine. This project delivers a fully local AI environment, running open source language models directly on your machine.
@@ -11,8 +11,8 @@ No cloud. No GAFAM. Just full privacy, control, and the freedom to manipulate co
## How it works ## How it works
* [Ollama](https://ollama.com/) run language models on the local machine. * [Ollama](https://ollama.com/) run language models on the local machine.
* [openedai-speech](https://github.com/matatonic/openedai-speech) provides text-to-speech capability. * [speaches.ai](https://speaches.ai) provides text-to-speech capability.
* [nginx](https://nginx.org/en/) add an authentication to the API. * [nginx](https://nginx.org/en/) adds an authentication to the API.
* [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents. * [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.
Everything is free, open-source, and automated using Docker Compose and shell scripts. Everything is free, open-source, and automated using Docker Compose and shell scripts.
@@ -23,11 +23,7 @@ It doesn't require an internet connection to work once the models have been down
To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required. To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.
As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [qwen2.5:7b](https://ollama.com/library/qwen2.5) model. As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [gemma3:12b-it-qat](https://ollama.com/library/gemma3:12b-it-qat) model.
The model [qwen2.5-coder:32b](https://ollama.com/library/qwen2.5-coder:32b) is usable but slow with this configuration.
Note that more modest models like [llama3.2:3b](https://ollama.com/library/llama3.2) require much fewer resources and still allow you to do a lot of things.
On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit). On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit).
@@ -35,22 +31,22 @@ Note that it is probably possible to run the project on other GPUs or modern Mac
## How to launch the server ## How to launch the server
Choose the models you wish to use in the docker-compose.yaml file. Choose the models you wish to use in the `docker-compose.yaml` file.
``` ```bash
environment: environment:
- MODELS=.... - MODELS=...
``` ```
Add an API key to secure server access by adding a `.env` file like this: Add an API key to secure server access by adding a `.env` file like this:
``` ```bash
LLM_API_KEY=1234567890 LLM_API_KEY=1234567890
``` ```
Create a user authentication for aichat Web UI: Create a user authentication for aichat Web UI:
``` ```bash
htpasswd -c src/nginx/htpasswd user htpasswd -c src/nginx/htpasswd user
``` ```
@@ -61,13 +57,13 @@ docker compose up --build -d
``` ```
Then wait for the models to finish downloading using the following command to display the status: Then wait for the models to finish downloading using the following command to display the status:
``` ```bash
docker-compose logs -f ollama_provision docker-compose logs -f ollama_provision
``` ```
## How to use ## How to use
The `setup_desktop.sh` script allows for copying a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configuring the tool. The `setup_desktop.sh` script allows you to copy a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configure the tool.
### AIChat essentials ### AIChat essentials
@@ -77,8 +73,8 @@ aichat "10 fictitious identities with username, firstname, lastname and email th
``` ```
Request a snippet of code: Request a snippet of code:
``` ```bash
aichat -m ollama:qwen2.5-coder:32b -c "if a docker image exist in bash" aichat -m ollama:gemma3:12b-it-qat -c "if condition to check if a docker image exist in bash"
``` ```
To launch a chatbot while maintaining context: To launch a chatbot while maintaining context:
@@ -86,13 +82,13 @@ To launch a chatbot while maintaining context:
aichat -s aichat -s
``` ```
Pipe a command and interpret the result: Using shell pipe:
``` ```bash
ps aux | aichat 'which process use the most memory' cat README.md | aichat -m ollama:llama3.1:8b 'Check the spelling, grammar, and phrasing. Anwser the correction using diff format'
``` ```
Using roles: Using roles:
``` ```bash
aichat -r short "tcp port of mysql" aichat -r short "tcp port of mysql"
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')" ./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"
``` ```
@@ -103,13 +99,13 @@ Go to the [AIChat](https://github.com/sigoden/aichat) website for other possible
For this features, use the speech.sh script like this: For this features, use the speech.sh script like this:
``` ```bash
./speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)." ./tools/speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
``` ```
## How to Use Remotely ## How to Use Remotely
The API authentication via Nginx allows you to open the API on the internet and use it remotely. The API authentication via Nginx allows you to open the API on the internet and use it remoteli.
By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption. By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.
@@ -118,7 +114,7 @@ This way, you can securely use this environment remotely.
To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) . To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .
Example: Example:
``` ```bash
export TTS_API_HOST="https://your-remote-domain" export TTS_API_HOST="https://your-remote-domain"
./tools/speech.sh ... ./tools/speech.sh ...
``` ```