Skip to main content
You are currently on a page documenting the use of Ollama models as text completion models. Many popular models available on Ollama are chat completion models.You may be looking for this page instead.
This will help you get started with Ollama text completion models (LLMs) using LangChain. For detailed documentation on Ollama features and configuration options, please refer to the API reference.

​
Overview

​
Integration details

Ollama allows you to run open-source large language models, such as Llama 3, locally. Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. It optimizes setup and configuration details, including GPU usage. This example goes over how to use LangChain to interact with an Ollama-run Llama 2 7b instance. For a complete list of supported models and model variants, see the Ollama model library.
ClassPackageLocalSerializablePY supportDownloadsVersion
Ollama@langchain/ollama✅❌✅NPM - DownloadsNPM - Version

​
Setup

To access Ollama embedding models you’ll need to follow these instructions to install Ollama, and install the @langchain/ollama integration package.

​
Credentials

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below: 
# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

​
Installation

The LangChain Ollama integration lives in the @langchain/ollama package: 
npm install @langchain/ollama @langchain/core

​
Instantiation

Now we can instantiate our model object and generate chat completions: 
import { Ollama } from "@langchain/ollama"

const llm = new Ollama({
  model: "llama3", // Default value
  temperature: 0,
  maxRetries: 2,
  // other params...
})

​
Invocation

const inputText = "Ollama is an AI company that "

const completion = await llm.invoke(inputText)
completion

I think you meant to say "Olivia" instead of "Ollama". Olivia is not a well-known AI company, but there are several other AI companies with similar names. Here are a few examples:

* Oliva AI: A startup that uses artificial intelligence to help businesses optimize their operations and improve customer experiences.
* Olivia Technologies: A company that develops AI-powered solutions for industries such as healthcare, finance, and education.
* Olivia.ai: A platform that uses AI to help businesses automate their workflows and improve productivity.

If you meant something else by "Ollama", please let me know and I'll do my best to help!

​
Multimodal models

Ollama supports open source multimodal models like LLaVA in versions 0.1.15 and up. You can bind base64 encoded image data to multimodal-capable models to use as context like this: 
import { Ollama } from "@langchain/ollama";
import * as fs from "node:fs/promises";

const imageData = await fs.readFile("../../../../../examples/hotdog.jpg");

const model = new Ollama({
  model: "llava",
}).bind({
  images: [imageData.toString("base64")],
});

const res = await model.invoke("What's in this image?");
console.log(res);

 The image shows a hot dog placed inside what appears to be a bun that has been specially prepared to resemble a hot dog bun. This is an example of a creative or novelty food item, where the bread used for the bun looks similar to a cooked hot dog itself, playing on the name "hot dog." The image also shows the typical garnishes like ketchup and mustard on the side.

​
API reference

For detailed documentation of all Ollama features and configurations head to the API reference
Edit the source of this page on GitHub.
Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.
⌘I