Update with last components and improve documentation.
This commit is contained in:
46
README.md
46
README.md
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
Unlock the power of AI—right from your Linux terminal.
|
Unlock the power of AI right from your Linux terminal.
|
||||||
|
|
||||||
This project delivers a fully local AI environment, running open source language models directly on your machine.
|
This project delivers a fully local AI environment, running open source language models directly on your machine.
|
||||||
|
|
||||||
@@ -11,8 +11,8 @@ No cloud. No GAFAM. Just full privacy, control, and the freedom to manipulate co
|
|||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
* [Ollama](https://ollama.com/) run language models on the local machine.
|
* [Ollama](https://ollama.com/) run language models on the local machine.
|
||||||
* [openedai-speech](https://github.com/matatonic/openedai-speech) provides text-to-speech capability.
|
* [speaches.ai](https://speaches.ai) provides text-to-speech capability.
|
||||||
* [nginx](https://nginx.org/en/) add an authentication to the API.
|
* [nginx](https://nginx.org/en/) adds an authentication to the API.
|
||||||
* [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.
|
* [AIChat](https://github.com/sigoden/aichat) is used as LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents.
|
||||||
|
|
||||||
Everything is free, open-source, and automated using Docker Compose and shell scripts.
|
Everything is free, open-source, and automated using Docker Compose and shell scripts.
|
||||||
@@ -23,11 +23,7 @@ It doesn't require an internet connection to work once the models have been down
|
|||||||
|
|
||||||
To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.
|
To run this project efficiently, a powerful computer with a recent NVIDIA GPU is required.
|
||||||
|
|
||||||
As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [qwen2.5:7b](https://ollama.com/library/qwen2.5) model.
|
As an example, I achieved good performance with an Intel(R) Core(TM) i7-14700HX, a GeForce RTX 4050 (6GB VRAM), and 32GB of RAM using the [gemma3:12b-it-qat](https://ollama.com/library/gemma3:12b-it-qat) model.
|
||||||
|
|
||||||
The model [qwen2.5-coder:32b](https://ollama.com/library/qwen2.5-coder:32b) is usable but slow with this configuration.
|
|
||||||
|
|
||||||
Note that more modest models like [llama3.2:3b](https://ollama.com/library/llama3.2) require much fewer resources and still allow you to do a lot of things.
|
|
||||||
|
|
||||||
On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit).
|
On GNU/Linux, you must use the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit).
|
||||||
|
|
||||||
@@ -35,22 +31,22 @@ Note that it is probably possible to run the project on other GPUs or modern Mac
|
|||||||
|
|
||||||
## How to launch the server
|
## How to launch the server
|
||||||
|
|
||||||
Choose the models you wish to use in the docker-compose.yaml file.
|
Choose the models you wish to use in the `docker-compose.yaml` file.
|
||||||
|
|
||||||
```
|
```bash
|
||||||
environment:
|
environment:
|
||||||
- MODELS=....
|
- MODELS=...
|
||||||
```
|
```
|
||||||
|
|
||||||
Add an API key to secure server access by adding a `.env` file like this:
|
Add an API key to secure server access by adding a `.env` file like this:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
LLM_API_KEY=1234567890
|
LLM_API_KEY=1234567890
|
||||||
```
|
```
|
||||||
|
|
||||||
Create a user authentication for aichat Web UI:
|
Create a user authentication for aichat Web UI:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
htpasswd -c src/nginx/htpasswd user
|
htpasswd -c src/nginx/htpasswd user
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -61,13 +57,13 @@ docker compose up --build -d
|
|||||||
```
|
```
|
||||||
|
|
||||||
Then wait for the models to finish downloading using the following command to display the status:
|
Then wait for the models to finish downloading using the following command to display the status:
|
||||||
```
|
```bash
|
||||||
docker-compose logs -f ollama_provision
|
docker-compose logs -f ollama_provision
|
||||||
```
|
```
|
||||||
|
|
||||||
## How to use
|
## How to use
|
||||||
|
|
||||||
The `setup_desktop.sh` script allows for copying a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configuring the tool.
|
The `setup_desktop.sh` script allows you to copy a compiled static version of [AIChat](https://github.com/sigoden/aichat) from a container to your host and configure the tool.
|
||||||
|
|
||||||
### AIChat essentials
|
### AIChat essentials
|
||||||
|
|
||||||
@@ -77,8 +73,8 @@ aichat "10 fictitious identities with username, firstname, lastname and email th
|
|||||||
```
|
```
|
||||||
|
|
||||||
Request a snippet of code:
|
Request a snippet of code:
|
||||||
```
|
```bash
|
||||||
aichat -m ollama:qwen2.5-coder:32b -c "if a docker image exist in bash"
|
aichat -m ollama:gemma3:12b-it-qat -c "if condition to check if a docker image exist in bash"
|
||||||
```
|
```
|
||||||
|
|
||||||
To launch a chatbot while maintaining context:
|
To launch a chatbot while maintaining context:
|
||||||
@@ -86,13 +82,13 @@ To launch a chatbot while maintaining context:
|
|||||||
aichat -s
|
aichat -s
|
||||||
```
|
```
|
||||||
|
|
||||||
Pipe a command and interpret the result:
|
Using shell pipe:
|
||||||
```
|
```bash
|
||||||
ps aux | aichat 'which process use the most memory'
|
cat README.md | aichat -m ollama:llama3.1:8b 'Check the spelling, grammar, and phrasing. Anwser the correction using diff format'
|
||||||
```
|
```
|
||||||
|
|
||||||
Using roles:
|
Using roles:
|
||||||
```
|
```bash
|
||||||
aichat -r short "tcp port of mysql"
|
aichat -r short "tcp port of mysql"
|
||||||
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"
|
./tools/speech.sh synthesize --play --lang en --voice bryce "$(aichat -r english-translator 'Bienvenue dans le monde de l AI et de la ligne de commande.')"
|
||||||
```
|
```
|
||||||
@@ -103,13 +99,13 @@ Go to the [AIChat](https://github.com/sigoden/aichat) website for other possible
|
|||||||
|
|
||||||
For this features, use the speech.sh script like this:
|
For this features, use the speech.sh script like this:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
./speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
|
./tools/speech.sh synthesize --play --lang fr --voice pierre "Bonjour, aujourd'hui nous somme le $(date +%A\ %d\ %B\ %Y)."
|
||||||
```
|
```
|
||||||
|
|
||||||
## How to Use Remotely
|
## How to Use Remotely
|
||||||
|
|
||||||
The API authentication via Nginx allows you to open the API on the internet and use it remotely.
|
The API authentication via Nginx allows you to open the API on the internet and use it remoteli.
|
||||||
|
|
||||||
By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.
|
By adding a reverse proxy like Caddy in front of it, you can also add TLS encryption.
|
||||||
|
|
||||||
@@ -118,7 +114,7 @@ This way, you can securely use this environment remotely.
|
|||||||
To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .
|
To use script tools in a remote context, use the environment variables TTS_API_HOST and modify AIChat config (~/.config/aichat/config.yaml) .
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
```
|
```bash
|
||||||
export TTS_API_HOST="https://your-remote-domain"
|
export TTS_API_HOST="https://your-remote-domain"
|
||||||
./tools/speech.sh ...
|
./tools/speech.sh ...
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in New Issue
Block a user