Update with last components and improve documentation.

This commit is contained in:
ben
2025-08-21 21:52:53 +02:00
parent 2595c44071
commit 576324fada

View File

@@ -2,7 +2,7 @@
![AI_ENV](logo.webp)
Unlock the power of AIright from your Linux terminal.
Unlock the power of AI right from your Linux terminal.
This project delivers a fully local AI environment, running open source language models directly on your machine.
@@ -11,8 +11,8 @@ No cloud. No GAFAM. Just full privacy, control, and the freedom to manipulate co
## How it works
* [Ollama](https://ollama.com/) run language models on the local machine.
* [openedai-speech](https://github.com/matatonic/openedai-speech) provides text-to-speech capability.
* [nginx](https://nginx.org/en/) add an authentication to the API.
* [speaches.ai](https://speaches.ai) provides text-to-speech capability.
* [nginx](https://nginx.org/en/) adds an authentication to the API.
* [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.
Everything is free, open-source, and automated using Docker Compose and shell scripts.
@@ -23,11 +23,7 @@ It doesn't require an internet connection to work once the models have been down
To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.
As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [qwen2.5:7b](https://ollama.com/library/qwen2.5) model.
The model [qwen2.5-coder:32b](https://ollama.com/library/qwen2.5-coder:32b) is usable but slow with this configuration.
Note that more modest models like [llama3.2:3b](https://ollama.com/library/llama3.2) require much fewer resources and still allow you to do a lot of things.
As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [gemma3:12b-it-qat](https://ollama.com/library/gemma3:12b-it-qat) model.
On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit).
@@ -35,22 +31,22 @@ Note that it is probably possible to run the project on other GPUs or modern Mac
## How to launch the server
Choose the models you wish to use in the docker-compose.yaml file.
Choose the models you wish to use in the `docker-compose.yaml` file.
```
```bash
environment:
- MODELS=....
- MODELS=...
```
Add an API key to secure server access by adding a `.env` file like this:
```
```bash
LLM_API_KEY=1234567890
```
Create a user authentication for aichat Web UI:
```
```bash
htpasswd -c src/nginx/htpasswd user
```
@@ -61,13 +57,13 @@ docker compose up --build -d
```
Then wait for the models to finish downloading using the following command to display the status:
```
```bash
docker-compose logs -f ollama_provision
```
## How to use
The `setup_desktop.sh` script allows for copying a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configuring the tool.
The `setup_desktop.sh` script allows you to copy a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configure the tool.
### AIChat essentials
@@ -77,8 +73,8 @@ aichat "10 fictitious identities with username, firstname, lastname and email th
```
Request a snippet of code:
```
aichat -m ollama:qwen2.5-coder:32b -c "if a docker image exist in bash"
```bash
aichat -m ollama:gemma3:12b-it-qat -c "if condition to check if a docker image exist in bash"
```
To launch a chatbot while maintaining context:
@@ -86,13 +82,13 @@ To launch a chatbot while maintaining context:
aichat -s
```
Pipe a command and interpret the result:
```
ps aux | aichat 'which process use the most memory'
Using shell pipe:
```bash
cat README.md | aichat -m ollama:llama3.1:8b 'Check the spelling, grammar, and phrasing. Anwser the correction using diff format'
```
Using roles:
```
```bash
aichat -r short "tcp port of mysql"
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"
```
@@ -103,13 +99,13 @@ Go to the [AIChat](https://github.com/sigoden/aichat) website for other possible
For this features, use the speech.sh script like this:
```
./speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
```bash
./tools/speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
```
## How to Use Remotely
The API authentication via Nginx allows you to open the API on the internet and use it remotely.
The API authentication via Nginx allows you to open the API on the internet and use it remoteli.
By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.
@@ -118,7 +114,7 @@ This way, you can securely use this environment remotely.
To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .
Example:
```
```bash
export TTS_API_HOST="https://your-remote-domain"
./tools/speech.sh ...
```