How to import a manually downloaded GGUF model into LM Studio

Importing a manually downloaded GGUF model into LM Studio allows you to use any LLM you download from platforms like Hugging Face without relying on the built-in search engine. You can import it in two ways: by manually organizing the files in your local directory or by running commands in the terminal. Next, Let's see step by step how to integrate your custom models into LM Studio.

Why import a manually downloaded GGUF model into LM Studio

Importing a manually downloaded GGUF model into LM Studio will help you get the most out of your hardware in a completely private way. In what situations is it recommended to import models downloaded outside of the tool? It's advisable to do so. if the app's internal search falls short or doesn't fit your actual workflowThese are the main reasons for use:

• Models not listed in Hugging Face or private: If you are training your own model or if a developer shared a private .gguf file with you that is not published in public repositories.
• Interrupted downloads or external managers: The built-in download manager in apps often fails with slow or unstable connections. Using tools like wget allows you to pause/resume and download files from 10 GB up to 50 GB with greater reliability.
• Experimental versions or custom modifications: You can also consider importing a manually downloaded GGUF model into LM Studio if a model creator released variants with custom quantizations.
• When you don't have an internet connection: If you are in a location without internet access, you can transfer the GGUF file using a USB drive or local network and integrate it directly into LM Studio.

How to import a manually downloaded GGUF model into LM Studio.

Next, we explain Step-by-step instructions on how to import a manually downloaded GGUF model into LM StudioFirst, we'll see how to do this by manually organizing the files within LM Studio's internal models folder. Then, we'll show you how to import it by running a command in your terminal.

Organize the files manually

To import a manually downloaded GGUF model into LM Studio, you must Place it within the internal models folder, respecting the folder structure required by the application.To do this, follow these steps:

1. Open the local models folder: Open LM Studio, go to the My Models tab (the folder icon in the left sidebar), and click Open Models Directory. If you prefer to browse manually from your file explorer:
   • Windows: C:\Users\YOUR_USER\.cache\lm-studio\models
   • macOS/Linux: ~/.cache/lm-studio/models
2. Create the subfolder structure: LM Studio requires models to be within an author folder and a model subfolder. Create the folders following this path: models / [Author_Name] / [Model_Name]. Here's an example: models/QuantFactory/Llama-3-8B-Instruct-GGUF
3. Place the .gguf file: Move or copy your .gguf file directly into the subfolder of the model you just created.
4. Reload and select the model: Return to LM Studio. On the My Models tab or in the top model selection dropdown menu, click the Reload/Refresh icon (or restart the application if it's not detected immediately). The model will appear ready to be loaded into memory.

Sometimes, LM Studio is unable to load a modelIn a previous article we analyzed the Common mistakes and solutions Step by step, which you can apply. You can take a look to find out what's happening if you can't load a model and how to fix it.

Use lms import

Another way to import a manually downloaded GGUF model into LM Studio is running the following command in your terminal:

• lms import and follow the interactive instructions to complete the import process.
• This is the expected model directory structure of LM Studio:
  • ~/.lmstudio/models/
    publisher/
    model/
    model-file.gguf
• For example, if you have a model called ocelot-v1 published by infra-ai, the structure would look like this:
  • ~/.lmstudio/models/
    infra-ai/
    ocelot-v1/
    ocelot-v1-instruct-q4_0.gguf

Using the native CLI of LM Studio

LM Studio includes a command-line tool for manage and run models without opening the graphical interfaceTo import a manually downloaded GGUF model into LM Studio, you can also do the following:

1. Install the lms command: Open LM Studio, go to the Developer / CLI tab in the sidebar and click on Install 'lms' command line tool (or run lms Bootstrap in your terminal).
2. Load the model using LMS: Once the .gguf file is in the models folder, run: lms load –gpu max.
3. Start an interactive conversation: LMS chat.
4. Start a local server compatible with OpenAI: lms server start.

Advantages of importing a manually downloaded GGUF model into LM Studio: It is possible

Importing a manually downloaded GGUF model into LM Studio offers several Key advantages when we talk about performance, accessibility and customizationThese are the main benefits:

• Faster and more stable downloads: External download managers handle slow or unstable connections better than the built-in client. This allows you to pause or resume large, bulky downloads without losing progress.
• Use of own or private models: If you train your own local models, you will need to import them manually using one of these methods if they are not hosted in any public repository.
• Compatibility with offline environments: It allows you to transfer models to USB drives or local networks to work on high-security computers, or those without internet access.
• Disk space saving: You avoid duplicating files if you already use those same .gguf files in other environments like llama.cpp, Ollama, or AnythingLLM. You just need to move or link them to the LM Studio models folder.
• Greater version control: You ensure the use of an exact revision of the model without the risk of it changing or being unexpectedly updated in the source repository.

In conclusion, manually importing a GGUF model into LM Studio gives you complete control over your local environment. Whether you do this by organizing the files into subfolders or using the terminal, this procedure allows you to take advantage of custom quantizations, work offline, and manage your resources efficiently. By applying the ideas discussed in this article, you will be able to Personalize your development experience and boost your AI projects.