Skip to main content
Shale Protocol provides production-ready inference APIs for open LLMs. It’s a Plug & Play API as it’s hosted on a highly scalable GPU cloud infrastructure. Our free tier supports up to 1K daily requests per key as we want to eliminate the barrier for anyone to start building genAI apps with LLMs. With Shale Protocol, developers/researchers can create apps and explore the capabilities of open LLMs at no cost. This page covers how Shale-Serve API can be incorporated with LangChain. As of June 2023, the API supports Vicuna-13B by default. We are going to support more LLMs such as Falcon-40B in future releases.

​
How to

​
2. Use https://shale.live/v1 as OpenAI API drop-in replacement

For example 
from langchain_openai import OpenAI
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser

import os
os.environ['OPENAI_API_BASE'] = "https://shale.live/v1"
os.environ['OPENAI_API_KEY'] = "ENTER YOUR API KEY"

llm = OpenAI()

template = """Question: {question}

# Answer: Let's think step by step."""

prompt = PromptTemplate.from_template(template)


llm_chain = prompt | llm | StrOutputParser()

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.invoke(question)
Edit the source of this page on GitHub.
Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.
⌘I