Skip to main content
Learn how to use LangChain with models on Predibase.

​
Setup

  • Create a Predibase account and API key.
  • Install the Predibase Python client with pip install predibase
  • Use your API key to authenticate

​
LLM

Predibase integrates with LangChain by implementing LLM module. You can see a short example below or a full notebook under LLM > Integrations > Predibase. 
import os
os.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}"

from langchain_community.llms import Predibase

model = Predibase(
    model="mistral-7b",
    predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"),
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    """
    Optionally use `model_kwargs` to set new default "generate()" settings.  For example:
    {
        "api_token": os.environ.get("HUGGING_FACE_HUB_TOKEN"),
        "max_new_tokens": 5,  # default is 256
    }
    """
    **model_kwargs,
)

"""
Optionally use `kwargs` to dynamically overwrite "generate()" settings.  For example:
{
    "temperature": 0.5,  # default is the value in model_kwargs or 0.1 (initialization default)
    "max_new_tokens": 1024,  # default is the value in model_kwargs or 256 (initialization default)
}
"""
response = model.invoke("Can you recommend me a nice dry wine?", **kwargs)
print(response)
Predibase also supports Predibase-hosted and HuggingFace-hosted adapters that are fine-tuned on the base model given by the model argument: 
import os
os.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}"

from langchain_community.llms import Predibase

# The fine-tuned adapter is hosted at Predibase (adapter_version must be specified).
model = Predibase(
    model="mistral-7b",
    predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"),
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="e2e_nlg",
    adapter_version=1,
    """
    Optionally use `model_kwargs` to set new default "generate()" settings.  For example:
    {
        "api_token": os.environ.get("HUGGING_FACE_HUB_TOKEN"),
        "max_new_tokens": 5,  # default is 256
    }
    """
    **model_kwargs,
)

"""
Optionally use `kwargs` to dynamically overwrite "generate()" settings.  For example:
{
    "temperature": 0.5,  # default is the value in model_kwargs or 0.1 (initialization default)
    "max_new_tokens": 1024,  # default is the value in model_kwargs or 256 (initialization default)
}
"""
response = model.invoke("Can you recommend me a nice dry wine?", **kwargs)
print(response)
Predibase also supports adapters that are fine-tuned on the base model given by the model argument: 
import os
os.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}"

from langchain_community.llms import Predibase

# The fine-tuned adapter is hosted at HuggingFace (adapter_version does not apply and will be ignored).
model = Predibase(
    model="mistral-7b",
    predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"),
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="predibase/e2e_nlg",
    """
    Optionally use `model_kwargs` to set new default "generate()" settings.  For example:
    {
        "api_token": os.environ.get("HUGGING_FACE_HUB_TOKEN"),
        "max_new_tokens": 5,  # default is 256
    }
    """
    **model_kwargs,
)

"""
Optionally use `kwargs` to dynamically overwrite "generate()" settings.  For example:
{
    "temperature": 0.5,  # default is the value in model_kwargs or 0.1 (initialization default)
    "max_new_tokens": 1024,  # default is the value in model_kwargs or 256 (initialization default)
}
"""
response = model.invoke("Can you recommend me a nice dry wine?", **kwargs)
print(response)
Edit the source of this page on GitHub.
Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.
⌘I