Getting a Meta Llama model onto your own hardware sounds simple until you hit the licence form, the 24-hour download link, and three different platforms all claiming to be the easiest route in. It is simple, once you know the order to do things in.
We help organisations move from AI experimentation to production systems at Shipshape Data, and open-weight models like Llama come up in that work constantly. Teams want to prototype without sending data to a third-party API, or they want a model they can fine-tune and own outright rather than rent by the token. Either way, the first step is the same: register with Meta, accept the licence, and get the weights onto a machine you control.
This guide walks through that process from a blank slate to a working local model. We cover what to prepare before you request access, how to pick between the available sizes and file formats, the three practical ways to pull the weights (Meta's own script, Hugging Face, and Ollama), and the errors that eat the most time when something goes wrong. Read the sections that match your setup and skip the rest; you do not need all three download routes.
What to sort out before you request access
Meta will not release any model weights until you have registered and accepted the licence terms, and your machine needs to be ready to receive files that run into tens of gigabytes. Do this preparation first. Turning up unprepared is the single biggest reason people end up requesting access twice.
A Meta account and the licence
Go to the Meta Llama site and click "Request Access". The form asks for your name, email address, and what you intend to use the model for. Meta then emails a signed URL that expires within 24 hours, so do not submit the request until your environment (Python, disk space, the works) is actually ready to go.
Use a work email address for the request if you have one. We have seen personal Gmail submissions sit in review noticeably longer than a company domain, though Meta has never published a stated reason for that.
Every major release, Llama 3, 3.1, and 3.2, has its own separate licence, and if you download through Hugging Face you accept it again there even if you already accepted it on Meta's site. Keep the confirmation email. The signed link is tied to the address you registered with and cannot be handed to a colleague.
Read the licence properly before you build a commercial product on top of it, not after. The Llama community licence is permissive for most businesses, but it carries a clause that requires a separate agreement from Meta if your product or service has more than 700 million monthly active users. Almost nobody reading this hits that threshold, but we have had a client ask us to check anyway before signing off a build, and it took five minutes to confirm they were nowhere near it. Worth the five minutes.
Hardware you will actually need
The size of model you choose sets the hardware bar, and there is a wide range on offer. The 1B and 3B Llama 3.2 models run on a modern laptop CPU with 8GB of RAM, though you will feel the slower inference speed within a few prompts. Anything above 13B parameters wants a dedicated GPU or a properly resourced cloud instance; trying to force a 70B model onto a laptop is a good way to spend an afternoon watching a fan spin.
Rough minimums by size, for reference:
- 1B to 3B parameters: 8GB RAM, 4GB GPU VRAM recommended
- 7B to 8B parameters: 16GB RAM, 8GB GPU VRAM recommended
- 13B to 14B parameters: 32GB RAM, 16GB GPU VRAM recommended
- 70B and above: 64GB+ RAM, 40GB+ VRAM, usually across multiple GPUs
If you run a quantised version in GGUF format through Ollama, you can roughly halve the VRAM figure against those numbers. That is what makes a 13B or even a 70B model realistic on a single decent consumer GPU rather than a rack of them.
Not all GGUF files are quantised the same amount, and the naming convention trips people up the first few times. Q8_0 is close to full precision and barely shrinks the file. Q5_K_M is a solid middle ground: noticeably smaller with output quality that holds up in most everyday tasks. Q4_K_M is the one most people end up on for local work, small enough to run on a mid-range GPU with output that is still reliable for chat and summarisation, though it starts to wobble on precise reasoning tasks. Anything below Q4 is really only for testing that a pipeline works end to end, not for anything you would put in front of a user.
Software you need installed first
Meta's official download script wants Python 3.10 or newer, plus Git and pip on your system path. If you are going the Hugging Face route as well, install its CLI too:
pip install huggingface_hub
huggingface-cli login
That login step needs a Hugging Face access token from Settings > Access Tokens, and it has to carry at least "read" permission on gated repositories. Skip that and the download fails with a 403 error the moment it starts, which is a frustrating way to discover a permissions setting five minutes into a download you thought was working.
Choosing the right model and file format
Meta has shipped several sizes across the Llama 3 family, and the format you pick decides which tools you can run it with afterwards. Get this wrong and you are re-downloading tens of gigabytes rather than tweaking a setting, so it is worth two minutes of thought before you start pulling files.
Model size and parameter count
Parameter count is the trade-off between capability and how much hardware you need to feed it. The small end, 1B and 3B models, is fast, cheap to run, and fine for classification, summarisation, and simple chat. Anything involving real reasoning, long documents, or instruction-following across a technical domain wants the 8B tier at minimum, and often 70B.
For most proof-of-concept builds we start with Llama 3.1 8B Instruct. It has already been fine-tuned for following instructions, so you get usable output straight away without doing any fine-tuning work yourself first.
Original weights or GGUF
You will meet two formats in practice:
- Original weights (PyTorch
.pthor.safetensors): the full-precision files, needed if you plan to fine-tune the model or serve it through Hugging Face Transformers. - GGUF: a quantised, single-file format built for local inference through tools like Ollama. Files are much smaller and run comfortably on CPU or a consumer GPU without the quality falling off a cliff.
If you want fast local iteration, grab GGUF through Ollama. If you are building towards a fine-tuned production model, use the safetensors weights from Hugging Face instead.
Downloading straight from Meta with the signed link
Once your access request is approved, the email you get contains a signed URL good for 24 hours, plus a working script rather than one large file to save. Meta hands you a shell script that fetches whichever model you tell it to.
Clone the download script
The script lives in Meta's public GitHub repository:
git clone https://github.com/meta-llama/llama-models.git
cd llama-models
chmod +x download.sh
Run it and paste the URL
From that directory, run the script. It asks for the signed URL from your email, then a model identifier from a numbered list.
./download.sh
Paste the URL cleanly, with no trailing space. A malformed URL fails the request with a 403 almost instantly, and it is easy to add a stray space copying from an email client. When it asks for the model identifier, use the exact string for the size and variant you settled on earlier, for example meta-llama/Llama-3.1-8B-Instruct.
The script pulls the weights, a tokeniser, and a checksum file into a local folder. Once it finishes, verify the files actually arrived intact before doing anything else:
md5sum -c checklist.chk
Every line should say OK. Anything marked FAILED is a corrupted file, and the fix is to re-download that one file rather than starting over.
Downloading through Hugging Face
Hugging Face is the platform most teams end up using for a Llama download, and it earns that position. The hub carries every official Llama 3 release, authentication runs cleanly through its CLI, and interrupted downloads resume instead of restarting from zero. If you would rather skip Meta's own script entirely, this is the faster route, as long as you have accepted the gated licence on the model's page first.
Pull the weights with the CLI
Open the model page, for instance meta-llama/Llama-3.1-8B-Instruct, and click "Agree and access repository". Once that is granted, pull the weights straight to your machine:
huggingface-cli download meta-llama/Llama-3.1-8B-Instruct \
--local-dir ./llama-3.1-8b-instruct \
--local-dir-use-symlinks False
The --local-dir-use-symlinks False flag writes real files rather than symlinks back into the Hugging Face cache, which matters if you want to mount the folder in a container or copy it to a cloud instance later.
A 403 error here almost always means one of two things: your token lacks read access on gated repos, or you have not actually accepted the licence on the model page itself, only on Meta's site.
Ollama, for the fastest possible start
If you want a quantised model running locally without touching raw weight files at all, Ollama is quicker than either of the above. Install it from the official site, then run one command:
ollama run llama3.1
That single line handles storage, picks a sensible quantisation, and starts serving the model behind an API at http://localhost:11434/api/generate. You can point any application at that endpoint without writing your own inference code. For quick prototyping, or for anyone who just wants to see what Llama can do before committing to a full pipeline, this is the path of least resistance.
Verifying the download actually works
A download finishing without an error message does not guarantee a working model. Skipping the verification step is, in our experience, the most common reason teams end up debugging what looks like an application bug hours later when the real problem was a corrupted file or a mismatched tokeniser from the start. Run a quick smoke test straight after the download, before you write a single line of application code against the model.
Load it and run one prompt
The Hugging Face Transformers library will load the weights and confirm the model responds:
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_path = "./llama-3.1-8b-instruct"
tokeniser = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16,
device_map="auto"
)
inputs = tokeniser("Explain what a transformer model is:", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokeniser.decode(outputs[0], skip_special_tokens=True))
torch_dtype=torch.float16 halves the memory footprint compared with the default float32, which buys you real headroom if you are close to your GPU's VRAM limit.
If the model loads but the output reads as garbled nonsense, check that the tokeniser you loaded actually came from the same directory as the weights. Mixing a cached tokeniser from a different release with a newer set of weights produces exactly this symptom, and it is easy to miss because the model does not throw an error, it just talks rubbish.
The errors you will actually run into
Four problems account for almost every support ticket we have seen on this. Work through them in order rather than starting the whole process again from scratch:
- "model not found" (OSError): caused by a wrong local path. Use an absolute path in
from_pretrainedrather than a relative one, especially if you are running from a different directory than the one you downloaded into. - CUDA out of memory: the model is too large for your VRAM. Add
load_in_4bit=Truewith the bitsandbytes library, or drop to a smaller model size. - 403 Forbidden on download: your token is missing gated access. Re-accept the licence on the Hugging Face model page and regenerate the token if the problem persists.
- md5sum FAILED: an incomplete or corrupted download. Delete that specific file and re-run the download script rather than deleting the whole folder.
Each of those has one deterministic fix. It is tempting to nuke the whole setup and start again the moment something goes wrong, but that usually just costs you another 24-hour access window for no benefit.
Skipping local hardware entirely and running Llama in the cloud
Not every project needs weights sitting on a laptop or an office GPU box. If you are building something that needs to be available to a team, or you do not want to manage GPU infrastructure yourself, the same models are available as managed endpoints across the major clouds, and the download process above becomes optional rather than mandatory.
AWS offers Llama models through Bedrock as a hosted API, billed per token with no infrastructure to provision. Azure AI Foundry does the same thing on Microsoft's side, and Google Cloud's Vertex AI Model Garden carries the Llama family too. All three skip the registration dance with Meta directly (the cloud provider has already done that), skip the hardware planning, and let you start calling the model with an API key inside a few minutes.
The trade-off is the one you would expect: you pay per token indefinitely rather than paying once in GPU time, and you lose the option to fine-tune the weights directly or run fully offline. For a genuine proof of concept, or for a workload with unpredictable usage that would sit idle on owned hardware most of the time, the managed route is usually cheaper and considerably less hassle. For a steady, high-volume workload, or anything with strict data residency requirements that rule out sending prompts to a third-party endpoint, owning the weights and the infrastructure underneath them starts to make more sense. We usually tell clients to prototype on a managed endpoint first and only move to self-hosted once the usage pattern is clear enough to size the hardware properly, rather than guessing upfront and over-provisioning.
What comes after the download
At this point you have registered with Meta, picked a model size and format that suit what you are building, downloaded the weights through whichever platform fits your workflow, and confirmed the model actually loads and responds. That is the whole of the download problem solved.
What usually comes next is harder than the download itself: wiring the model into something people actually use. A RAG pipeline over your own documents, an internal knowledge tool, or an AI feature bolted onto an existing product all involve architecture decisions, data preparation, and a deployment plan that go well beyond getting weights onto a disk. We have watched teams get a model running locally in an afternoon and then spend three months figuring out how to serve it reliably at scale, because nobody had thought through the second half of the problem.
That gap, between a working local model and something you can actually rely on in production, is where most of our work with clients starts. If you want a second opinion on your Llama setup, or help turning it into something production-ready, talk to us and we will give you a straight answer on what is actually missing.