Workflows and agents

This guide reviews common patterns for agentic systems. In describing these systems, it can be useful to make a distinction between “workflows” and “agents”. One way to think about this difference is nicely explained in Anthropic’s Building Effective Agents blog post:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

Here is a simple way to visualize these differences: Agent Workflow

When building agents and workflows, LangGraph offers a number of benefits including persistence, streaming, and support for debugging as well as deployment.

Set up

You can use any chat model that supports structured outputs and tool calling. Below, we show the process of installing the packages, setting API keys, and testing structured outputs / tool calling for Anthropic. Install dependencies

npm install @langchain/core @langchain/anthropic @langchain/langgraph

Initialize an LLM

import { ChatAnthropic } from "@langchain/anthropic";

process.env.ANTHROPIC_API_KEY = "YOUR_API_KEY";

const llm = new ChatAnthropic({ model: "claude-3-5-sonnet-latest" });

Building Blocks: The Augmented LLM

LLM have augmentations that support building workflows and agents. These include structured outputs and tool calling, as shown in this image from the Anthropic blog on Building Effective Agents:

import { z } from "zod";
import { tool } from "@langchain/core/tools";

// Schema for structured output
const SearchQuery = z.object({
  search_query: z.string().describe("Query that is optimized web search."),
  justification: z
    .string()
    .describe("Why this query is relevant to the user's request."),
});

// Augment the LLM with schema for structured output
const structuredLlm = llm.withStructuredOutput(SearchQuery);

// Invoke the augmented LLM
const output = await structuredLlm.invoke(
  "How does Calcium CT score relate to high cholesterol?"
);

// Define a tool
const multiply = tool(
  async ({ a, b }: { a: number; b: number }) => {
    return a * b;
  },
  {
    name: "multiply",
    description: "Multiply two numbers",
    schema: z.object({
      a: z.number(),
      b: z.number(),
    }),
  }
);

// Augment the LLM with tools
const llmWithTools = llm.bindTools([multiply]);

// Invoke the LLM with input that triggers the tool call
const msg = await llmWithTools.invoke("What is 2 times 3?");

// Get the tool call
console.log(msg.tool_calls);

Prompt chaining

In prompt chaining, each LLM call processes the output of the previous one. As noted in the Anthropic blog on Building Effective Agents:

Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks (see “gate” in the diagram below) on any intermediate steps to ensure that the process is still on track.

When to use this workflow: This workflow is ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks. The main goal is to trade off latency for higher accuracy, by making each LLM call an easier task.

import { StateGraph, START, END } from "@langchain/langgraph";
import { z } from "zod";

// Graph state
const State = z.object({
  topic: z.string(),
  joke: z.string().optional(),
  improved_joke: z.string().optional(),
  final_joke: z.string().optional(),
});

// Nodes
const generateJoke = async (state: z.infer<typeof State>) => {
  // First LLM call to generate initial joke
  const msg = await llm.invoke(`Write a short joke about ${state.topic}`);
  return { joke: msg.content };
};

const checkPunchline = (state: z.infer<typeof State>) => {
  // Gate function to check if the joke has a punchline
  // Simple check - does the joke contain "?" or "!"
  if (state.joke && (state.joke.includes("?") || state.joke.includes("!"))) {
    return "Pass";
  }
  return "Fail";
};

const improveJoke = async (state: z.infer<typeof State>) => {
  // Second LLM call to improve the joke
  const msg = await llm.invoke(`Make this joke funnier by adding wordplay: ${state.joke}`);
  return { improved_joke: msg.content };
};

const polishJoke = async (state: z.infer<typeof State>) => {
  // Third LLM call for final polish
  const msg = await llm.invoke(`Add a surprising twist to this joke: ${state.improved_joke}`);
  return { final_joke: msg.content };
};

// Build workflow
const workflow = new StateGraph(State)
  .addNode("generate_joke", generateJoke)
  .addNode("improve_joke", improveJoke)
  .addNode("polish_joke", polishJoke)
  .addEdge(START, "generate_joke")
  .addConditionalEdges(
    "generate_joke",
    checkPunchline,
    { "Fail": "improve_joke", "Pass": END }
  )
  .addEdge("improve_joke", "polish_joke")
  .addEdge("polish_joke", END);

// Compile
const chain = workflow.compile();

// Show workflow
import * as fs from "node:fs/promises";
const drawableGraph = await chain.getGraphAsync();
const image = await drawableGraph.drawMermaidPng();
const imageBuffer = new Uint8Array(await image.arrayBuffer());
await fs.writeFile("workflow.png", imageBuffer);

// Invoke
const state = await chain.invoke({ topic: "cats" });
console.log("Initial joke:");
console.log(state.joke);
console.log("\n--- --- ---\n");
if (state.improved_joke) {
  console.log("Improved joke:");
  console.log(state.improved_joke);
  console.log("\n--- --- ---\n");

  console.log("Final joke:");
  console.log(state.final_joke);
} else {
  console.log("Joke failed quality gate - no punchline detected!");
}

Parallelization

With parallelization, LLMs work simultaneously on a task:

LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically. This workflow, parallelization, manifests in two key variations: Sectioning: Breaking a task into independent subtasks run in parallel. Voting: Running the same task multiple times to get diverse outputs.

When to use this workflow: Parallelization is effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results. For complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect.

// Graph state
const State = z.object({
  topic: z.string(),
  joke: z.string().optional(),
  story: z.string().optional(),
  poem: z.string().optional(),
  combined_output: z.string().optional(),
});

// Nodes
const callLlm1 = async (state: z.infer<typeof State>) => {
  // First LLM call to generate initial joke
  const msg = await llm.invoke(`Write a joke about ${state.topic}`);
  return { joke: msg.content };
};

const callLlm2 = async (state: z.infer<typeof State>) => {
  // Second LLM call to generate story
  const msg = await llm.invoke(`Write a story about ${state.topic}`);
  return { story: msg.content };
};

const callLlm3 = async (state: z.infer<typeof State>) => {
  // Third LLM call to generate poem
  const msg = await llm.invoke(`Write a poem about ${state.topic}`);
  return { poem: msg.content };
};

const aggregator = (state: z.infer<typeof State>) => {
  // Combine the joke and story into a single output
  let combined = `Here's a story, joke, and poem about ${state.topic}!\n\n`;
  combined += `STORY:\n${state.story}\n\n`;
  combined += `JOKE:\n${state.joke}\n\n`;
  combined += `POEM:\n${state.poem}`;
  return { combined_output: combined };
};

// Build workflow
const parallelBuilder = new StateGraph(State)
  .addNode("call_llm_1", callLlm1)
  .addNode("call_llm_2", callLlm2)
  .addNode("call_llm_3", callLlm3)
  .addNode("aggregator", aggregator)
  .addEdge(START, "call_llm_1")
  .addEdge(START, "call_llm_2")
  .addEdge(START, "call_llm_3")
  .addEdge("call_llm_1", "aggregator")
  .addEdge("call_llm_2", "aggregator")
  .addEdge("call_llm_3", "aggregator")
  .addEdge("aggregator", END);

const parallelWorkflow = parallelBuilder.compile();

// Invoke
const state = await parallelWorkflow.invoke({ topic: "cats" });
console.log(state.combined_output);

Routing

Routing classifies an input and directs it to a followup task. As noted in the Anthropic blog on Building Effective Agents:

Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs.

When to use this workflow: Routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model/algorithm.

import { SystemMessage, HumanMessage } from "@langchain/core/messages";

// Schema for structured output to use as routing logic
const Route = z.object({
  step: z.enum(["poem", "story", "joke"]).describe("The next step in the routing process"),
});

// Augment the LLM with schema for structured output
const router = llm.withStructuredOutput(Route);

// State
const State = z.object({
  input: z.string(),
  decision: z.string().optional(),
  output: z.string().optional(),
});

// Nodes
const llmCall1 = async (state: z.infer<typeof State>) => {
  // Write a story
  const result = await llm.invoke(state.input);
  return { output: result.content };
};

const llmCall2 = async (state: z.infer<typeof State>) => {
  // Write a joke
  const result = await llm.invoke(state.input);
  return { output: result.content };
};

const llmCall3 = async (state: z.infer<typeof State>) => {
  // Write a poem
  const result = await llm.invoke(state.input);
  return { output: result.content };
};

const llmCallRouter = async (state: z.infer<typeof State>) => {
  // Route the input to the appropriate node
  const decision = await router.invoke([
    new SystemMessage("Route the input to story, joke, or poem based on the user's request."),
    new HumanMessage(state.input),
  ]);

  return { decision: decision.step };
};

// Conditional edge function to route to the appropriate node
const routeDecision = (state: z.infer<typeof State>) => {
  // Return the node name you want to visit next
  if (state.decision === "story") {
    return "llm_call_1";
  } else if (state.decision === "joke") {
    return "llm_call_2";
  } else if (state.decision === "poem") {
    return "llm_call_3";
  }
};

// Build workflow
const routerBuilder = new StateGraph(State)
  .addNode("llm_call_1", llmCall1)
  .addNode("llm_call_2", llmCall2)
  .addNode("llm_call_3", llmCall3)
  .addNode("llm_call_router", llmCallRouter)
  .addEdge(START, "llm_call_router")
  .addConditionalEdges(
    "llm_call_router",
    routeDecision,
    {
      "llm_call_1": "llm_call_1",
      "llm_call_2": "llm_call_2",
      "llm_call_3": "llm_call_3",
    }
  )
  .addEdge("llm_call_1", END)
  .addEdge("llm_call_2", END)
  .addEdge("llm_call_3", END);

const routerWorkflow = routerBuilder.compile();

// Invoke
const state = await routerWorkflow.invoke({ input: "Write me a joke about cats" });
console.log(state.output);

Orchestrator-Worker

With orchestrator-worker, an orchestrator breaks down a task and delegates each sub-task to workers. As noted in the Anthropic blog on Building Effective Agents:

In the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.

When to use this workflow: This workflow is well-suited for complex tasks where you can’t predict the subtasks needed (in coding, for example, the number of files that need to be changed and the nature of the change in each file likely depend on the task). Whereas it’s topographically similar, the key difference from parallelization is its flexibility—subtasks aren’t pre-defined, but determined by the orchestrator based on the specific input.

import "@langchain/langgraph/zod";

// Schema for structured output to use in planning
const Section = z.object({
  name: z.string().describe("Name for this section of the report."),
  description: z.string().describe("Brief overview of the main topics and concepts to be covered in this section."),
});

const Sections = z.object({
  sections: z.array(Section).describe("Sections of the report."),
});

// Augment the LLM with schema for structured output
const planner = llm.withStructuredOutput(Sections);

Creating Workers in LangGraphBecause orchestrator-worker workflows are common, LangGraph has the Send API to support this. It lets you dynamically create worker nodes and send each one a specific input. Each worker has its own state, and all worker outputs are written to a shared state key that is accessible to the orchestrator graph. This gives the orchestrator access to all worker output and allows it to synthesize them into a final output. As you can see below, we iterate over a list of sections and Send each to a worker node. See further documentation here and here.

import { withLangGraph } from "@langchain/langgraph/zod";
import { Send } from "@langchain/langgraph";

// Graph state
const State = z.object({
  topic: z.string(), // Report topic
  sections: z.array(Section).optional(), // List of report sections
  // All workers write to this key
  completed_sections: withLangGraph(z.array(z.string()), {
    reducer: {
      fn: (x, y) => x.concat(y),
    },
    default: () => [],
  }),
  final_report: z.string().optional(), // Final report
});

// Worker state
const WorkerState = z.object({
  section: Section,
  completed_sections: withLangGraph(z.array(z.string()), {
    reducer: {
      fn: (x, y) => x.concat(y),
    },
    default: () => [],
  }),
});

// Nodes
const orchestrator = async (state: z.infer<typeof State>) => {
  // Orchestrator that generates a plan for the report
  const reportSections = await planner.invoke([
    new SystemMessage("Generate a plan for the report."),
    new HumanMessage(`Here is the report topic: ${state.topic}`),
  ]);

  return { sections: reportSections.sections };
};

const llmCall = async (state: z.infer<typeof WorkerState>) => {
  // Worker writes a section of the report
  const section = await llm.invoke([
    new SystemMessage(
      "Write a report section following the provided name and description. Include no preamble for each section. Use markdown formatting."
    ),
    new HumanMessage(
      `Here is the section name: ${state.section.name} and description: ${state.section.description}`
    ),
  ]);

  // Write the updated section to completed sections
  return { completed_sections: [section.content] };
};

const synthesizer = (state: z.infer<typeof State>) => {
  // Synthesize full report from sections
  const completedSections = state.completed_sections;
  const completedReportSections = completedSections.join("\n\n---\n\n");
  return { final_report: completedReportSections };
};

// Conditional edge function to create llm_call workers
const assignWorkers = (state: z.infer<typeof State>) => {
  // Assign a worker to each section in the plan
  return state.sections!.map((s) => new Send("llm_call", { section: s }));
};

// Build workflow
const orchestratorWorkerBuilder = new StateGraph(State)
  .addNode("orchestrator", orchestrator)
  .addNode("llm_call", llmCall)
  .addNode("synthesizer", synthesizer)
  .addEdge(START, "orchestrator")
  .addConditionalEdges("orchestrator", assignWorkers, ["llm_call"])
  .addEdge("llm_call", "synthesizer")
  .addEdge("synthesizer", END);

// Compile the workflow
const orchestratorWorker = orchestratorWorkerBuilder.compile();

// Invoke
const state = await orchestratorWorker.invoke({ topic: "Create a report on LLM scaling laws" });
console.log(state.final_report);

Evaluator-optimizer

In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop:

When to use this workflow: This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative writing process a human writer might go through when producing a polished document.

// Graph state
const State = z.object({
  joke: z.string().optional(),
  topic: z.string(),
  feedback: z.string().optional(),
  funny_or_not: z.string().optional(),
});

// Schema for structured output to use in evaluation
const Feedback = z.object({
  grade: z.enum(["funny", "not funny"]).describe("Decide if the joke is funny or not."),
  feedback: z.string().describe("If the joke is not funny, provide feedback on how to improve it."),
});

// Augment the LLM with schema for structured output
const evaluator = llm.withStructuredOutput(Feedback);

// Nodes
const llmCallGenerator = async (state: z.infer<typeof State>) => {
  // LLM generates a joke
  let msg;
  if (state.feedback) {
    msg = await llm.invoke(
      `Write a joke about ${state.topic} but take into account the feedback: ${state.feedback}`
    );
  } else {
    msg = await llm.invoke(`Write a joke about ${state.topic}`);
  }
  return { joke: msg.content };
};

const llmCallEvaluator = async (state: z.infer<typeof State>) => {
  // LLM evaluates the joke
  const grade = await evaluator.invoke(`Grade the joke ${state.joke}`);
  return { funny_or_not: grade.grade, feedback: grade.feedback };
};

// Conditional edge function to route back to joke generator or end
const routeJoke = (state: z.infer<typeof State>) => {
  // Route back to joke generator or end based upon feedback from the evaluator
  if (state.funny_or_not === "funny") {
    return "Accepted";
  } else if (state.funny_or_not === "not funny") {
    return "Rejected + Feedback";
  }
};

// Build workflow
const optimizerBuilder = new StateGraph(State)
  .addNode("llm_call_generator", llmCallGenerator)
  .addNode("llm_call_evaluator", llmCallEvaluator)
  .addEdge(START, "llm_call_generator")
  .addEdge("llm_call_generator", "llm_call_evaluator")
  .addConditionalEdges(
    "llm_call_evaluator",
    routeJoke,
    {
      "Accepted": END,
      "Rejected + Feedback": "llm_call_generator",
    }
  );

// Compile the workflow
const optimizerWorkflow = optimizerBuilder.compile();

// Invoke
const state = await optimizerWorkflow.invoke({ topic: "Cats" });
console.log(state.joke);

Agent

Agents are typically implemented as an LLM performing actions (via tool-calling) based on environmental feedback in a loop. As noted in the Anthropic blog on Building Effective Agents:

Agents can handle sophisticated tasks, but their implementation is often straightforward. They are typically just LLMs using tools based on environmental feedback in a loop. It is therefore crucial to design toolsets and their documentation clearly and thoughtfully.

When to use agents: Agents can be used for open-ended problems where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents’ autonomy makes them ideal for scaling tasks in trusted environments.

import { tool } from "@langchain/core/tools";

// Define tools
const multiply = tool(
  async ({ a, b }: { a: number; b: number }) => {
    return a * b;
  },
  {
    name: "multiply",
    description: "Multiply a and b.",
    schema: z.object({
      a: z.number().describe("first int"),
      b: z.number().describe("second int"),
    }),
  }
);

const add = tool(
  async ({ a, b }: { a: number; b: number }) => {
    return a + b;
  },
  {
    name: "add",
    description: "Adds a and b.",
    schema: z.object({
      a: z.number().describe("first int"),
      b: z.number().describe("second int"),
    }),
  }
);

const divide = tool(
  async ({ a, b }: { a: number; b: number }) => {
    return a / b;
  },
  {
    name: "divide",
    description: "Divide a and b.",
    schema: z.object({
      a: z.number().describe("first int"),
      b: z.number().describe("second int"),
    }),
  }
);

// Augment the LLM with tools
const tools = [add, multiply, divide];
const toolsByName = Object.fromEntries(tools.map((tool) => [tool.name, tool]));
const llmWithTools = llm.bindTools(tools);

import { MessagesZodState, ToolNode } from "@langchain/langgraph/prebuilt";
import { SystemMessage, HumanMessage, ToolMessage, isAIMessage } from "@langchain/core/messages";

// Nodes
const llmCall = async (state: z.infer<typeof MessagesZodState>) => {
  // LLM decides whether to call a tool or not
  const response = await llmWithTools.invoke([
    new SystemMessage(
      "You are a helpful assistant tasked with performing arithmetic on a set of inputs."
    ),
    ...state.messages,
  ]);
  return { messages: [response] };
};

const toolNode = new ToolNode(tools);

// Conditional edge function to route to the tool node or end
const shouldContinue = (state: z.infer<typeof MessagesZodState>) => {
  // Decide if we should continue the loop or stop
  const messages = state.messages;
  const lastMessage = messages[messages.length - 1];
  // If the LLM makes a tool call, then perform an action
  if (isAIMessage(lastMessage) && lastMessage.tool_calls?.length) {
    return "Action";
  }
  // Otherwise, we stop (reply to the user)
  return END;
};

// Build workflow
const agentBuilder = new StateGraph(MessagesZodState)
  .addNode("llm_call", llmCall)
  .addNode("environment", toolNode)
  .addEdge(START, "llm_call")
  .addConditionalEdges(
    "llm_call",
    shouldContinue,
    {
      "Action": "environment",
      [END]: END,
    }
  )
  .addEdge("environment", "llm_call");

// Compile the agent
const agent = agentBuilder.compile();

// Invoke
const messages = [new HumanMessage("Add 3 and 4.")];
const result = await agent.invoke({ messages });
for (const m of result.messages) {
  console.log(`${m.getType()}: ${m.content}`);
}

Pre-built

LangGraph also provides a pre-built method for creating an agent as defined above (using the createReactAgent function):

import { createReactAgent } from "@langchain/langgraph/prebuilt";

// Pass in:
// (1) the augmented LLM with tools
// (2) the tools list (which is used to create the tool node)
const preBuiltAgent = createReactAgent({ llm, tools });

// Invoke
const messages = [new HumanMessage("Add 3 and 4.")];
const result = await preBuiltAgent.invoke({ messages });
for (const m of result.messages) {
  console.log(`${m.getType()}: ${m.content}`);
}

What LangGraph provides

By constructing each of the above in LangGraph, we get a few things:

Persistence: Human-in-the-Loop

LangGraph persistence layer supports interruption and approval of actions (e.g., Human In The Loop). See Module 3 of LangChain Academy.

Persistence: Memory

LangGraph persistence layer supports conversational (short-term) memory and long-term memory. See Modules 2 and 5 of LangChain Academy:

Streaming

LangGraph provides several ways to stream workflow / agent outputs or intermediate state. See Module 3 of LangChain Academy.

Deployment

LangGraph provides an easy on-ramp for deployment, observability, and evaluation. See module 6 of LangChain Academy.

Get started

General concepts

Additional resources

Set up

Building Blocks: The Augmented LLM

Prompt chaining

Parallelization

Routing

Orchestrator-Worker

Evaluator-optimizer

Agent

Pre-built

What LangGraph provides

Persistence: Human-in-the-Loop

Persistence: Memory

Streaming

Deployment

Get started

General concepts

Additional resources

​Set up

​Building Blocks: The Augmented LLM

​Prompt chaining

​Parallelization

​Routing

​Orchestrator-Worker

​Evaluator-optimizer

​Agent

​Pre-built

​What LangGraph provides

​Persistence: Human-in-the-Loop

​Persistence: Memory

​Streaming

​Deployment

Set up

Building Blocks: The Augmented LLM

Prompt chaining

Parallelization

Routing

Orchestrator-Worker

Evaluator-optimizer

Agent

Pre-built

What LangGraph provides

Persistence: Human-in-the-Loop

Persistence: Memory

Streaming

Deployment