How to Build Applications With the Ollama API and Python

By /

Running a model in a chat window is fun, but real value starts when your own program can talk to it. The Ollama API makes that easy, and Python is the quickest way to use it. Ollama runs a local server on your computer, and any script can send it a prompt and read the answer. No cloud account, no API key, and no per-request fee.

This Ollama API Python tutorial takes you from a fresh install to a working application. You will install Ollama, run a model, call the REST API, use the official Python library, stream responses, handle errors, and build a small command-line tool that turns meeting notes into structured data. The code uses current library features, but we have not run it for you, so test it on your machine and check the library documentation if a version differs.

Prerequisites

You need a few things before you start.

  • A computer with enough memory for a small model. An 8B model at 4-bit quantization needs roughly 5 to 6 GB, as explained in our guide to LLM VRAM requirements.
  • A recent version of Python 3. Version 3.10 or newer is a safe choice.
  • Basic comfort with a terminal and with reading Python code.
  • An internet connection for the initial download of Ollama and the model.

Install Ollama and Run Your First Model

Download the installer for your system from ollama.com/download. On Linux, the site provides an install script. On macOS and Windows, you run a normal installer. The official Ollama documentation has the current steps.

When the install finishes, open a terminal and check that it works:

ollama --version
ollama run llama3.1:8b

The first command prints the version. The second downloads the model if you do not have it, then opens a chat. Type a question, press Enter, and exit with /bye. The model name is just an example, so choose any model that fits your hardware from our guide to the best local LLMs for different tasks.

Ollama also starts a background service that listens on http://localhost:11434. Your code will talk to that address.

Call the Ollama REST API With curl

Before you write Python, it helps to see the raw request. Send a chat message with curl:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1:8b",
  "messages": [
    {"role": "user", "content": "Explain recursion in one sentence."}
  ],
  "stream": false
}'

On Windows PowerShell, quoting works differently, so save the JSON to a file and use curl.exe with the -d @file.json option, or skip ahead to the Python examples.

The server replies with JSON. The answer sits in the message field, under content. Setting stream to false returns the whole answer at once. Without it, Ollama streams the reply as a series of small JSON objects.

The Main Endpoints

Endpoint Purpose
/api/chat Send a list of messages and get the next reply
/api/generate Send a single prompt and get a completion
/api/embed Turn text into embedding vectors
/api/tags List the models you have downloaded
/api/ps List the models currently loaded in memory

Use the chat endpoint for most work, because it keeps the conversation structure clear. The embedding endpoint matters later, when you build a local RAG system.

Set Up a Python Project

Create a folder and a virtual environment, so the packages stay separate from the rest of your system.

mkdir ollama-demo
cd ollama-demo
python -m venv .venv
source .venv/bin/activate
pip install ollama pydantic

On Windows, activate the environment with .venv\Scripts\activate instead. The ollama package is the official Python client. We install pydantic because it helps us check structured output later.

Make Your First Ollama API Python Request

Create a file called hello.py and add this code:

import ollama

response = ollama.chat(
    model="llama3.1:8b",
    messages=[
        {"role": "user", "content": "Give me three tips for learning Python."}
    ],
)

print(response.message.content)

Run it with python hello.py. The call sends one message and waits for the full reply. The reply object has a message field, and the text is in message.content.

Chat Versus Generate

The library also has a generate function for single prompts. It takes a prompt string instead of a message list. Use chat when you need roles such as system and user, or when you want to keep a conversation going. Use generate for one-off completions where history does not matter.

Keep a Conversation

The model does not remember earlier turns by itself. You send the history each time. Add each reply to your message list, then send the list again:

import ollama

messages = [{"role": "system", "content": "You are a concise tutor."}]

while True:
    user_input = input("You: ")
    if user_input.strip().lower() in {"quit", "exit"}:
        break
    messages.append({"role": "user", "content": user_input})
    response = ollama.chat(model="llama3.1:8b", messages=messages)
    reply = response.message.content
    messages.append({"role": "assistant", "content": reply})
    print("Model:", reply)

Long chats grow the history, and the history uses context window space. If the history gets too long, trim the oldest messages.

Stream Responses for a Better Experience

A long answer can take many seconds. Streaming prints words as the model produces them, so the program feels responsive. Pass stream=True and loop over the chunks:

import ollama

stream = ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Explain what an API is."}],
    stream=True,
)

for chunk in stream:
    print(chunk.message.content, end="", flush=True)
print()

Each chunk holds a small piece of text. The flush=True argument forces Python to show it immediately.

Control the Model With Options

The options argument changes how the model runs. Two settings matter most.

response = ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "List five fruits."}],
    options={"temperature": 0.2, "num_ctx": 4096},
)
  • temperature controls randomness. Low values such as 0.1 to 0.3 give steadier answers, which suits extraction and code. Higher values give more varied text.
  • num_ctx sets the context window in tokens. A bigger window uses more memory, so raise it only when you need it.

Other options include num_predict, which limits the output length. Check the API documentation for the full list. Our guide on running local LLMs faster explains how these settings affect speed.

Choose the Official Library or Plain HTTP Requests

You can reach the Ollama API from Python in two ways. The official library is the easier path. It turns replies into Python objects, handles streaming for you, and raises clear errors. Most Ollama API Python projects should start there.

Plain HTTP requests with a package such as requests give you full control and one fewer dependency. They suit small scripts, locked-down environments, and cases where you want to see exactly what travels over the wire. Here is the same kind of chat call with requests:

import requests

reply = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "llama3.1:8b",
        "messages": [{"role": "user", "content": "Name three Python web frameworks."}],
        "stream": False,
    },
    timeout=120,
)
reply.raise_for_status()
print(reply.json()["message"]["content"])

The timeout matters. Without one, a stuck request can hang your program forever. The official library lets you set a timeout and a host when you create a client object. This helps when Ollama runs on another computer in your network.

from ollama import Client

client = Client(host="http://localhost:11434", timeout=120)
response = client.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.message.content)

Handle Errors Properly

Local programs fail in predictable ways. The server may not be running, the model may not be downloaded, or the request may be invalid. Handle these cases so your program gives a clear message instead of a long traceback.

import ollama

def ask(prompt: str, model: str = "llama3.1:8b") -> str:
    try:
        response = ollama.chat(
            model=model,
            messages=[{"role": "user", "content": prompt}],
        )
        return response.message.content
    except ConnectionError:
        raise SystemExit("Cannot reach Ollama. Is the app or 'ollama serve' running?")
    except ollama.ResponseError as error:
        if error.status_code == 404:
            raise SystemExit(f"Model not found. Run: ollama pull {model}")
        raise SystemExit(f"Ollama returned an error: {error.error}")

print(ask("What is a hash map?"))

A ConnectionError means the library could not reach the server. A ResponseError means the server answered with an error, and a 404 status usually means the model name is wrong or not downloaded. You can also add a request timeout and a retry loop for long-running jobs.

Get Structured JSON Output

Programs need predictable data, not free text. Ollama supports structured outputs: you pass a JSON schema in the format argument, and the model is constrained to follow it. Pydantic can generate the schema and check the result.

import ollama
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

response = ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Ana is 31 years old."}],
    format=Person.model_json_schema(),
    options={"temperature": 0},
)

person = Person.model_validate_json(response.message.content)
print(person)

The schema guarantees the shape of the output, but not the truth of the content. A model can still extract a wrong value, so validate anything important.

Build a Complete Example: A Meeting Notes Summarizer

Now combine these pieces into a small tool. It reads a text file of meeting notes and prints a summary, the decisions, and the action items. Save the code as summarize_notes.py.

import sys
from pathlib import Path

import ollama
from pydantic import BaseModel, ValidationError

MODEL = "llama3.1:8b"
MAX_TRIES = 3


class MeetingNotes(BaseModel):
    summary: str
    decisions: list[str]
    action_items: list[str]


def check_setup(model: str) -> None:
    """Stop early if the server or the model is missing."""
    try:
        installed = {m.model for m in ollama.list().models}
    except ConnectionError:
        sys.exit("Cannot reach Ollama. Start the app or run 'ollama serve'.")
    if model not in installed and f"{model}:latest" not in installed:
        sys.exit(f"Model not found. Run: ollama pull {model}")


def summarize(text: str) -> MeetingNotes:
    system = (
        "You turn meeting notes into structured summaries. "
        "Use only facts that appear in the text. "
        "If there are no decisions or action items, return empty lists."
    )
    for attempt in range(1, MAX_TRIES + 1):
        try:
            response = ollama.chat(
                model=MODEL,
                messages=[
                    {"role": "system", "content": system},
                    {"role": "user", "content": text},
                ],
                format=MeetingNotes.model_json_schema(),
                options={"temperature": 0, "num_ctx": 8192},
            )
            return MeetingNotes.model_validate_json(response.message.content)
        except ValidationError:
            print(f"Attempt {attempt} returned invalid data. Retrying...")
        except ollama.ResponseError as error:
            sys.exit(f"Ollama error: {error.error}")
    sys.exit("The model did not return valid data. Try a larger model.")


def main() -> None:
    if len(sys.argv) != 2:
        sys.exit("Usage: python summarize_notes.py notes.txt")
    path = Path(sys.argv[1])
    if not path.exists():
        sys.exit(f"File not found: {path}")

    check_setup(MODEL)
    notes = summarize(path.read_text(encoding="utf-8"))

    print("SUMMARY")
    print(notes.summary)
    print("\nDECISIONS")
    for item in notes.decisions:
        print(f"- {item}")
    print("\nACTION ITEMS")
    for item in notes.action_items:
        print(f"- {item}")


if __name__ == "__main__":
    main()

Create a file named notes.txt with a few lines of meeting text, then run:

python summarize_notes.py notes.txt

How the Example Works

The check_setup function asks Ollama for the list of installed models. It catches a connection failure and a missing model before any real work starts. The summarize function sends the notes with a system message that limits the model to facts from the text. It asks for output that matches the MeetingNotes schema, and it retries if the data is invalid. The main function reads the file, runs the steps, and prints a clean report.

Three design choices are worth copying in your own projects. The temperature is zero, which makes extraction more consistent. The schema forces a predictable shape. And the program fails with short, helpful messages instead of crashing.

Use the OpenAI-Compatible Endpoint

Ollama also exposes an OpenAI-compatible API under http://localhost:11434/v1/. If your code already uses an OpenAI-style client, you can often point it at this address, pass any placeholder as the API key, and keep the rest of your code. Compatibility covers the common features, but not everything, so test the calls you depend on.

Common Mistakes to Avoid

A few problems catch almost everyone at first.

  • Forgetting to download the model. A request for a model you have not pulled fails with a not-found error. Run ollama list to see what you have.
  • Ignoring the context window. The default window may be shorter than you expect, so a long document can be cut off. Set num_ctx when you work with long inputs.
  • Parsing free text. If your program needs data, use structured output instead of searching the reply for words.
  • Leaving the temperature high for extraction. Random variation helps creative writing and hurts consistent data extraction.
  • Sending an endless history. A chat that grows without limit slows down and eventually overflows the context window.
  • Skipping the timeout. A request without a time limit can block your whole program.

Limitations and Practical Tips

  • Small models make mistakes. Validate outputs, and do not use them for decisions without a check.
  • The first request after a pause is slow because Ollama must load the model. The keep_alive setting controls how long it stays in memory.
  • Library and API details change between versions. When something fails, check the ollama-python repository and the official docs.
  • The server listens only on your own computer by default. If you expose it to a network, add your own access controls, because the API has no built-in login.

Conclusion

The Ollama API turns a local model into a service that any program can call. You installed Ollama, made requests with curl and Python, streamed responses, handled errors, and built a tool that returns structured data. These pieces cover most small applications.

From here, you can extend the example in two directions. Add document search with our guide to building a local RAG system, or let the model take actions with the tutorial on a local AI agent.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top