How to install LocalAI with Docker step by step

Installing LocalAI with Docker is one of the fastest and cleanest ways to Set up a local environment to run AI models without relying on external servicesIn this post, we'll tell you how to do it: from the prerequisites for installation to a brief tour of the Local AI interface. We'll also see how to download and use templates depending on the type of hardware installed.

What is LocalAI and why install it with Docker?

Before installing LocalAI with Docker, it's helpful to understand exactly what this tool is. In short, LocalAI is an open-source project that presents itself as a A free alternative to the OpenAI APIBesides being free, it has the advantage that all processing is done on local hardware: absolute privacy.

As an alternative to the OpenAI API, it allows you to use the same routes, parameters, and request structure, but running the models on your own machine. Furthermore, with LocalAI You can load models in different formats, such as GGUF and GPT4All, and for different purposes.: language, vision, audio and images.

Something else that stands out about LocalAI is that It does not require a dedicated GPU.While it takes advantage of hardware acceleration where available, it runs smoothly on x86_64 and ARM64 architectures in CPU mode. It also features native support for agents and the Model Context Protocol (MCP) specification.

And why install LocalAI with Docker? Because deploying it with Docker containers eliminates dependency conflicts on the host system. Furthermore, this It is the most accessible and convenient option For the vast majority of users and developers, the following step-by-step installation works perfectly on any operating system: Windows, macOS, or Linux.

Requirements for installing LocalAI with Docker

Naturally, before we begin You will need to have Docker installed on your system.If you don't already have it, you can follow the official documentation for your operating system. You'll find step-by-step guides in the entries. How to install Docker on Linux y How to install Docker on Windows 10Once installed, verify that it is working correctly by running the command docker –version.

At the hardware level, the requirements for installing LocalAI with Docker depend on the model or models you want to run. Remember that the tool downloads and runs models locally. This means that, The larger and more powerful the model, the more hardware will be needed.: storage space, CPU cores and GB of RAM.

Step-by-step instructions for installing LocalAI with Docker

Let's see the Step-by-step guide to installing LocalAI with DockerFirst, we'll see how to perform a quick deployment using Docker CLI, ideal if you want to try out the tool. Then we'll explain how to deploy LocalAI if you have an NVIDIA GPU installed (the command changes slightly). And finally, we'll see how to perform an advanced deployment with Docker Compose if you need to apply more specific settings.

Step 1: Creating the Directory Structure

The first step is to create a directory structure, that is, a dedicated folder This is where the downloaded models and custom YAML configurations will be stored. Besides maintaining a cleaner organization, this step ensures data persistence even if the container is destroyed or updated. It's advisable to do this as a starting point regardless of the installation method. Open your terminal and run:

mkdir -p ~/localai/models cd ~/localai

Step 2: Selecting the appropriate Docker image

Less than a step, this is a clarification to keep in mind when installing LocalAI with Docker. The tool offers Images optimized for different hardware architecturesTherefore, choosing the right label (tag) is vital to achieving the right performance:

• CPU only: localai/localal:latest
• NVIDIA CUDA 12: localai/localai/:latest-gpu-nvidia-cuda-12
• AMD ROCm (hipBLAS): localai/localai:latest-gpu-hipblas
• Intel GPU (oneAPI): localai/localai/:latest/gpu/intel
• Vulkan Acceleration: localai/localai/:latest-gpu-vulkan

Step 3: Install LocalAI with Docker CLI

This is the main point and the most important step that will lead you to perform a quick and basic LocalAI installation with Docker. Depending on your hardware, use the corresponding command with the appropriate tag. For a Installing LocalAI with Docker on a PC without a GPU (CPU only)Use the following command:

docker run -d \ --name local-ai \ -p 8080:8080 \ -v $(pw)/models:/build/models:rw \ --restart unless-stopped \ localai/localai:latest

On the other hand, if your computer has an NVIDIA GPUThe command to install LocalAI with Docker changes slightly. Before running it, make sure you have the latest NVIDIA driver and the nvidia-container-toolkitThen, run the following command:

docker run -d \ --name local-ai \ -p 8080:8080 \ --gpus all \ -v $(pw)/models:/build/models:rw \ --restart unless-stopped \ localai/localai:latest-gpu-nvidia-cuda-12

Step 4: Advanced deployment with Docker Compose (recommended)

Finally, you can install LocalAI with Docker Compose if you're looking for a advanced, configurable, and permanent deployment of the tool. This method allows you to define environment variables, memory limits, and network configurations in a single declarative file. To do this, you need to create a file called docker-compose.yml within the ~/localai directory (see Step 1) by running the following command:

version: '3.8' services: local-ai: image: localai/localai:latest-gpu-nvidia-cuda-12 container_name: local-ai restart: unless-stopped ports: - "8080:8080" environment: - THREADS=8 - DEBUG=true - MODELS_PATH=/build/models - UPLOAD_LIMIT=15M volumes: - ./models:/build/models:rw deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu]

This command is adapted for a computer with an NVIDIA GPU. However, if you are running in a CPU-only environment, change the image to localai/localai/latest and remove the section deploy.resources.reservations.

Step 5: Start the service and manage the models from the interface

After installing LocalAI with Docker Compose, all that remains is Start the service in the background with the command docker-compose up -dIf you want, you can do it right away. verify real-time records to verify that the server has started correctly with docker compose logs -f local-ai.

Whether using Docker Compose or Docker CLI, at this point you can go to the address http://localhost:8080 for access the web administration panel (WebUI) of the tool. Once inside, click on the tab Models (Models) and look for a lightweight model optimized for testing, or install the one that the platform recommends based on the installed hardware.

Finally, click on Install LocalAI will download the file and automatically generate the YAML configuration in your directory. Now you can Chat with the model directly from the Local AI WebUI or connect them to other tools via server. Now all that's left is to explore and experiment with all the models you want.