Skip to main content
LangGraph provides built-in support for LLMs (language models) via the LangChain library. This makes it easy to integrate various LLMs into your agents and workflows.

​
Initialize a model

Use init_chat_model to initialize models:
  • OpenAI
  • Anthropic
  • Azure
  • Google Gemini
  • AWS Bedrock
pip install -U "langchain[openai]"

import os
from langchain.chat_models import init_chat_model

os.environ["OPENAI_API_KEY"] = "sk-..."

llm = init_chat_model("openai:gpt-4.1")
👉 Read the OpenAI integration docs

​
Instantiate a model directly

If a model provider is not available via init_chat_model, you can instantiate the provider’s model class directly. The model must implement the BaseChatModel interface and support tool calling: 
# Anthropic is already supported by `init_chat_model`,
# but you can also instantiate it directly.
from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
  model="claude-3-7-sonnet-latest",
  temperature=0,
  max_tokens=2048
)
Tool calling support If you are building an agent or workflow that requires the model to call external tools, ensure that the underlying language model supports tool calling. Compatible models can be found in the LangChain integrations directory.

​
Use in an agent

When using create_react_agent you can specify the model by its name string, which is a shorthand for initializing the model using init_chat_model. This allows you to use the model without needing to import or instantiate it directly.
  • model name
  • model instance
from langgraph.prebuilt import create_react_agent

create_react_agent(
   # highlight-next-line
   model="anthropic:claude-3-7-sonnet-latest",
   # other parameters
)

​
Dynamic model selection

Pass a callable function to create_react_agent to dynamically select the model at runtime. This is useful for scenarios where you want to choose a model based on user input, configuration settings, or other runtime conditions. The selector function must return a chat model. If you’re using tools, you must bind the tools to the model within the selector function. 
from dataclasses import dataclass
from typing import Literal
from langchain.chat_models import init_chat_model
from langchain_core.language_models import BaseChatModel
from langchain_core.tools import tool
from langgraph.prebuilt import create_react_agent
from langgraph.prebuilt.chat_agent_executor import AgentState
from langgraph.runtime import Runtime

@tool
def weather() -> str:
  """Returns the current weather conditions."""
  return "It's nice and sunny."


# Define the runtime context
@dataclass
class CustomContext:
  provider: Literal["anthropic", "openai"]

# Initialize models
openai_model = init_chat_model("openai:gpt-4o")
anthropic_model = init_chat_model("anthropic:claude-sonnet-4-20250514")


# Selector function for model choice
def select_model(state: AgentState, runtime: Runtime[CustomContext]) -> BaseChatModel:
  if runtime.context.provider == "anthropic":
      model = anthropic_model
  elif runtime.context.provider == "openai":
      model = openai_model
  else:
      raise ValueError(f"Unsupported provider: {runtime.context.provider}")

  # With dynamic model selection, you must bind tools explicitly
  return model.bind_tools([weather])


# Create agent with dynamic model selection
agent = create_react_agent(select_model, tools=[weather])

# Invoke with context to select model
output = agent.invoke(
  {
      "messages": [
          {
              "role": "user",
              "content": "Which model is handling this?",
          }
      ]
  },
  context=CustomContext(provider="openai"),
)

print(output["messages"][-1].text())
New in LangGraph v0.6

​
Advanced model configuration

​
Disable streaming

To disable streaming of the individual LLM tokens, set disable_streaming=True when initializing the model:
  • init_chat_model
  • ChatModel
from langchain.chat_models import init_chat_model

model = init_chat_model(
    "anthropic:claude-3-7-sonnet-latest",
    # highlight-next-line
    disable_streaming=True
)
Refer to the API reference for more information on disable_streaming

​
Add model fallbacks

You can add a fallback to a different model or a different LLM provider using model.with_fallbacks([...]):
  • init_chat_model
  • ChatModel
from langchain.chat_models import init_chat_model

model_with_fallbacks = (
    init_chat_model("anthropic:claude-3-5-haiku-latest")
    # highlight-next-line
    .with_fallbacks([
        init_chat_model("openai:gpt-4.1-mini"),
    ])
)
See this guide for more information on model fallbacks.

​
Use the built-in rate limiter

Langchain includes a built-in in-memory rate limiter. This rate limiter is thread safe and can be shared by multiple threads in the same process. 
from langchain_core.rate_limiters import InMemoryRateLimiter
from langchain_anthropic import ChatAnthropic

rate_limiter = InMemoryRateLimiter(
    requests_per_second=0.1,  # <-- Super slow! We can only make a request once every 10 seconds!!
    check_every_n_seconds=0.1,  # Wake up every 100 ms to check whether allowed to make a request,
    max_bucket_size=10,  # Controls the maximum burst size.
)

model = ChatAnthropic(
   model_name="claude-3-opus-20240229",
   rate_limiter=rate_limiter
)
See the LangChain docs for more information on how to handle rate limiting.

​
Bring your own model

If your desired LLM isn’t officially supported by LangChain, consider these options:
  1. Implement a custom LangChain chat model: Create a model conforming to the LangChain chat model interface. This enables full compatibility with LangGraph’s agents and workflows but requires understanding of the LangChain framework.
  2. Direct invocation with custom streaming: Use your model directly by adding custom streaming logic with StreamWriter. Refer to the custom streaming documentation for guidance. This approach suits custom workflows where prebuilt agent integration is not necessary.

​
Additional resources

⌘I