Skip to main content

Instalar

image.png

Instalar debian

Instalar paquetes

sudo apt install lshw curl
 lshw -C display
  *-display                 
       description: VGA compatible controller
       product: GM107GL [Quadro K2200]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:02:00.0
       logical name: /dev/fb0
       version: a2
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=nouveau latency=0 resolution=1920,1080
       resources: irq:38 memory:f2000000-f2ffffff memory:e0000000-efffffff memory:f0000000-f1ffffff ioport:2000(size=128) memory:c0000-dffff

Instalar Driver nvidia

Añade al final de cada línea del fichero '/etc/apt/sources.list' -> 'contrib non-free'

deb http://deb.debian.org/debian/ bookworm main non-free-firmware contrib non-free
deb http://security.debian.org/debian-security bookworm-security main non-free-firmware contrib non-free
deb http://deb.debian.org/debian/ bookworm-updates main non-free-firmware contrib non-free
deb http://deb.debian.org/debian/ bookworm-backports main non-free-firmware contrib non-free
Instala los paquetes
LINUX_HEADERS=$(uname -r)
sudo apt update
sudo apt -y install nvidia-driver firmware-misc-nonfree linux-headers-$LINUX_HEADERS dkms
sudo reboot
Comprueba la instalación que mostrará el estado de la tarjeta gráfica:
nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:05:00.0 Off |                  N/A |
|  0%   54C    P0    27W / 120W |      1MiB /  6144MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Instalar NVIDIA Container Toolkit

Debian 12 Bookworm : NVIDIA Container Toolkit : Install : Server World (server-world.info)

Instalar OLLAMA

ollama/ollama - Docker Image | Docker Hub

Ejecuta el contenedor usando CPU solo

 docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama

Ejecuta el contenedor usando gpu

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Ejecuta modelo

Now you can run a model:

docker exec -it ollama ollama run llama3

Back to top