Skip to content
NLEN
Illustration: Editing an Ollama Modelfile: system prompt and parameters

Editing an Ollama Modelfile: system prompt and parameters

By Ivo Donker - 5 August 2026

When you run language models locally through Ollama, you use by default the base settings supplied by the model's creators or the Ollama team. In many practical applications that is not enough. You may want a model to behave by default as a specific assistant, the output to be deterministic, or the context length to be increased. Passing extensive instructions or runtime parameters by hand on every API call or chat session is cumbersome and error-prone.

For this Ollama offers the concept of the Modelfile. A Modelfile is a simple text file in which you set a base model and combine it with a fixed configuration, a custom system prompt and specific parameters. Building a Modelfile creates a new, locally registered model that can be invoked with all its specific properties at once. In this guide we cover how a Modelfile is constructed, which instructions and parameters are available, and how to manage them effectively.

For downloading base models or installing the software itself, consult the guides on downloading and managing models and installing Ollama on macOS. This guide focuses solely on configuring Modelfiles.

What a Modelfile is and what it is not

A Modelfile is best compared to a Dockerfile. Where a Dockerfile describes how a software container is built, a Modelfile describes how a language model should be loaded and steered. The file bundles the following elements into a new model entity on your system:

It is essential to understand what a Modelfile not is: a Modelfile is not fine-tuning. In fine-tuning, a neural network's internal weights are adjusted through an intensive training process based on a dataset. A Modelfile changes no weight value at all. It changes only the context, conditions and decoding settings with which the existing weights are consulted. The result of a ollama create command based on a Modelfile is therefore a lightweight pointer to the existing weights plus a stored configuration set.

The main instructions in a Modelfile

A Modelfile is built from a series of specific instructions. Each of these instructions fulfills its own role in defining the model. Below we discuss the instructions that matter, with an example line for each.

FROM

The FROMinstruction defines the foundation of your new model. This can be an existing Ollama model already on your system or fetched automatically, or a direct path to a local GGUF file.

FROM llama3.2:3b

Or when using a local file:

FROM ./custom-model-q4_k_m.gguf

SYSTEM

With SYSTEM you set the fixed system prompt. This instruction gives the model its role, constraints, language preference or output format. For multi-line texts you can use triple quotes.

SYSTEM """Je bent een professionele tekstredacteur. Corrigeer taalfouten en verbeter de leesbaarheid zonder de inhoudelijke betekenis te wijzigen."""

PARAMETER

The PARAMETERinstruction sets specific values for Ollama's inference engine. With it you directly influence how the model selects tokens and how much memory it uses. You can place several PARAMETERlines one after another.

PARAMETER temperature 0.2

TEMPLATE

The TEMPLATEinstruction specifies how the system prompt, chat history and user input are merged into one long string fed to the model. This uses Go template syntax.

TEMPLATE """{{ .System }}
User: {{ .Prompt }}
Assistant:"""

ADAPTER

With the ADAPTERinstruction you can apply a separate LoRA (low-rank adaptation) adapter to the base model. This makes it possible to lay specific, fine-tuned layers over an existing base model.

ADAPTER ./my-custom-lora.bin

MESSAGE

The MESSAGEinstruction lets you supply predefined dialogues (few-shot prompting). This helps demonstrate the desired input and output format to the model before the user asks questions.

MESSAGE user Hoe maak ik een opsommingsteken in HTML?
MESSAGE assistant Gebruik de <li>-tag binnen een <ul>- of <ol>-element.

LICENSE

With LICENSE you can record the license terms of the derived model. This matters especially if you want to export or share the model within an organization.

LICENSE """MIT License"""

In depth: the main PARAMETER values

How a language model behaves depends heavily on the parameters steering logit processing and sampling. In a Modelfile you can tune these parameters precisely to your application area.

Parameter Default direction Effect of too high a value Effect of too low a value
temperature 0.0 - 1.0 Model becomes incoherent and hallucinates. Model becomes repetitive, stiff and rigid.
top_p 0.1 - 1.0 Raises diversity but reduces focus. Restricts the vocabulary too tightly to predictable words.
top_k 1 - 100 Allows rare tokens; risk of grammatical errors. Excludes relevant synonyms; answers become monotonous.
repeat_penalty 1.0 - 1.2 Model avoids necessary repetitions and synonyms. Model quickly falls into repeating loops of words or sentences.
repeat_last_n 0 - 256 Checks too far back; costs extra compute. Checks too short a span; sentence-level repetitions go unnoticed.
num_ctx 2048 - 131072 High memory pressure; risk of falling back to CPU. Premature loss of information from the conversation.
num_predict -1 (unlimited) Answers become needlessly long or keep running on. Output is cut off mid-sentence or mid-argument.
seed Random No effect (this is a specific integer). No effect (this is a specific integer).
stop Model-dependent No direct range (strings). Stops on the wrong characters. Model does not stop where wanted and generates extra roles.

A detailed analysis of the core parameters

temperature: This parameter steers the randomness of token selection. A high temperature (0.8 or higher, for instance) makes the model choose less likely tokens, which is useful for creative tasks. A low temperature (0.1, for instance) forces the model to pick the most likely tokens every time. Set temperature to 0.0 and the model switches to so-called greedy decoding.

top_p and top_k: These are sampling techniques for restricting token selection. top_k restricts the choice to the top K most likely next words. top_p (nucleus sampling) looks at cumulative probability: it chooses from the smallest set of tokens whose combined probability reaches the value P . Lowering top_p helps filter noise out of answers.

repeat_penalty and repeat_last_n: The repeat_penalty penalizes tokens that have already appeared recently in the text. A value of 1.0 means no penalty. A value of 1.15 discourages repetition. The parameter repeat_last_n specifies how many tokens the model looks back over to detect repetition (often 64 by default). If you notice a model getting stuck in an infinite loop of the same sentences, raise the repeat_penalty.

The context window (num_ctx) explained

One of the most important settings in the Modelfile is num_ctx, which sets the size of the context window in tokens. By default, many models in Ollama use a relatively small window of 2,048 or 4,096 tokens to conserve working memory. Modern architectures can often handle 32,000 to 128,000 tokens, but increasing this value has direct consequences for hardware load.

Processing the context requires the KV (key-value) cache to be held in memory. As num_ctx increases, memory usage grows quadratically or linearly depending on the model's attention design. When total memory requirements (the model plus the KV cache) exceed the graphics card's available video memory (VRAM), Ollama is forced to move some layers from the GPU to system memory (RAM) and the CPU.

The effect of this move is a dramatic drop in processing speed (generation speed in tokens per second). You notice this happening when the processing time for generating the first token (time to first token, or prompt evaluation time) climbs enormously, or when GPU fan activity drops while CPU load spikes to 100%. So determine carefully how much context you actually need for your application and test it on your local hardware. More background on how tokens are processed can be found on the page about tokenization explained.

Reproducibility: seed and temperature 0

In many software applications, automated pipelines or test environments, randomness is undesirable. You want the model to give exactly the same output on exactly the same input every time. This is crucial when comparing prompt changes, running automated evaluations or generating structured data such as JSON.

To make a model fully deterministic, combine two settings in the Modelfile:

PARAMETER temperature 0
PARAMETER seed 42

By specifying temperature to 0, the model picks the highest-probability token at every step. The seed (a randomly chosen integer, such as 42) fixes the pseudo-random number generator at a set point. This guarantees identical output as long as the hardware architecture and software version stay the same. This principle is central to running structured tests; read more about it on the page about reproducibility in benchmarks.

Setting stop sequences

The parameter stop specifies the character sequences on which the inference engine should abort generation immediately. As soon as the model generates the specified stop sequence, the process stops and the stop sequence itself is not shown to the user.

Setting stop sequences by hand matters greatly when building structured applications, agents or chat interfaces with clear role separation. An example of stop sequences in a Modelfile:

PARAMETER stop "<|im_end|>"
PARAMETER stop "Gebruiker:"
PARAMETER stop "---"

Be careful choosing stop sequences. If you set a stop sequence that can also occur in normal text output (a single period, a space or a common word), the model will cut the response off abruptly halfway. Use only unique tokens or clear separators that do not belong in the substantive answer.

Writing system prompts for smaller local models

Writing a system prompt for a local model of 3B, 7B or 8B parameters requires a different approach from instructing very large cloud models. Smaller models have a more limited ability to process and remember complex, narrative or contradictory instructions over the course of a conversation.

When a system prompt for a small model is too long or too narrative, instruction drift often occurs: the model forgets the constraints or gets confused about priorities. More in-depth advice on this is covered on the page about prompting for smaller models.

When drawing up the SYSTEMinstruction in a Modelfile, apply the following principles:

If you want the model to answer in high-quality Dutch specifically, see the guidelines on generating better Dutch with local LLMs.

Practical example 1: a Dutch-language summarization assistant

Below is a fully worked example of a Modelfile, designed to turn long texts into a concise Dutch-language summary according to fixed rules.

Create a file called Modelfile.samenvatting and add the following content:

# Basismodel specificeren
FROM llama3.2:3b

# Sampling parameters afstellen voor gestructureerde, stabiele uitvoer
PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 8192

# Vaste systeemprompt
SYSTEM """Je bent een gespecialiseerde assistent voor het samenvatten van Nederlandstalige documenten.

Hanteer altijd de volgende regels:
1. Geen inleiding of beleefdheden, begin direct met de samenvatting.
2. Schrijf in helder, professioneel Nederlands.
3. Structureer de samenvatting in maximaal 3 kernpunten met bullet points.
4. Houd de totale lengte onder de 150 woorden.
5. Gebruik uitsluitend informatie uit de opgegeven brontekst."""

Line-by-line explanation

Building and running the model

Open your terminal in the folder containing the file and run the command below to create the model under the name samenvatter:v1:

ollama create samenvatter:v1 -f ./Modelfile.samenvatting

Once the process is complete, you can start and test the model straight away:

ollama run samenvatter:v1

Practical example 2: a deterministic coding assistant

In this second example we adjust only the base model's parameters without imposing a system prompt. This is highly suitable when you want to keep the base model's original behavior and chat instructions but enforce strict, reproducible output for code generation, for instance.

Create a file called Modelfile.code-strict with the following content:

FROM qwen2.5-coder:7b

# Volledig deterministische instellingen
PARAMETER temperature 0.0
PARAMETER seed 1234
PARAMETER top_k 10
PARAMETER num_ctx 16384
PARAMETER stop "```"

Build this model through the terminal:

ollama create code-strict:v1 -f ./Modelfile.code-strict

This model will now produce exactly the same code on every call with the same prompt, without variation in syntax or explanation, which is ideal for integration into IDEs or automated tests. See integrating a local LLM into VS Code for applications of this.

Practical Ollama commands for managing models

Working with Modelfiles involves a series of CLI commands in Ollama. Here are the main commands for day-to-day management:

1. Creating a new model (ollama create)

This compiles the Modelfile into a new locally registered model.

ollama create mijn-model:v1 -f ./Modelfile

2. Inspecting a model's configuration (ollama show)

If you want to check which parameters, system prompt or license a built model contains, use ollama show with the flag you want:

# Toon de volledige Modelfile
ollama show --modelfile samenvatter:v1

# Toon alleen de systeemprompt
ollama show --system samenvatter:v1

# Toon de toegepaste template
ollama show --template samenvatter:v1

3. Reading an existing Modelfile as a starting point

A handy way to write your own Modelfile is to request an existing base model's Modelfile and save it to a file. That way you see immediately which template the model uses by default:

ollama show --modelfile llama3.2:3b > Modelfile.basis

You can then edit this file and adapt it to your own wishes.

4. Running and testing the model

Start an interactive chat session with your customized model:

ollama run samenvatter:v1

5. Cleaning up models (ollama rm)

When an experimental version of a model is no longer needed, remove it from your system easily:

ollama rm samenvatter:v1

Warning: do not adjust the TEMPLATE instruction lightly

It is tempting to rewrite the TEMPLATEinstruction in a Modelfile by hand. The warning is clear: only do this if you know exactly how the base model was trained. In almost all cases, adopt the base model's existing template.

Every language model (such as Llama 3, Mistral or ChatML) was trained during the alignment phase (RLHF/SFT) with very specific control tokens to separate the roles of `system`, `user` and `assistant`. When you change this structure by hand or replace it with a format of your own devising, the model no longer recognizes the role separations. This often leads to subtle quality degradation, such as:

If you build a Modelfile from a standard Ollama base model (through FROM modelnaam), Ollama adopts the correct template automatically. So leave the TEMPLATEinstruction alone unless you are importing a bare GGUF file that contains no template metadata yet.

Managing Modelfiles in practice

Since a Modelfile is a plain text file, it lends itself excellently to professional version control. Apply the following guidelines for managing your Modelfiles effectively within projects:

Further reading