Welcome to Day 19 of #30DaysOfLangChain – LangChain 0.3 Edition! So far, we’ve built intelligent agents and collaborative workflows. But what happens when a conversation ends, or an agent’s task is paused? Without a mechanism to remember its past, the agent would lose all context and start fresh every time, leading to frustrating user experiences and inefficient processes.

Today, we dive into a critical aspect of building practical AI applications: managing agent state and persistence in LangGraph. We’ll learn how to store and restore the entire state of a long-running agentic conversation, giving our AI applications a memory.

The Challenge of Stateful Agents

In a multi-turn conversation or a complex workflow, an AI agent accumulates context: user messages, its own responses, internal data gathered, decisions made, and even the current position in a complex graph. If this state isn’t preserved, every interaction is a new one, breaking the flow and requiring users to repeat information. Persistence solves this by allowing the agent to pick up exactly where it left off.

LangGraph’s Checkpointing Mechanism

LangGraph provides a powerful feature called checkpointing to handle state persistence. When a graph is compiled with a checkpointer, LangGraph automatically saves the entire state of the graph at each transition point. When you invoke the graph with a specific thread_id, it first checks for an existing checkpoint for that ID and, if found, loads it, resuming the conversation. If no checkpoint exists, a new thread is implicitly started.

Key Components for Persistence:

  1. StateGraph and StateSchema (TypedDict): Your TypedDict defines the schema of your agent’s state. Crucially, fields annotated with Annotated[List[BaseMessage], add_messages] will automatically manage message history, saving all incoming and outgoing messages. Any other fields you define in your TypedDict will also be saved.
  2. SqliteSaver (or other Savers): LangGraph provides various Checkpointer implementations. SqliteSaver is convenient as it uses a file-based SQLite database to store checkpoints. Other options include Redis, PostgreSQL, etc.
  3. checkpointer argument in workflow.compile(): This is where you connect your SqliteSaver (or other Checkpointer instance) to your compiled graph, enabling automatic state management.
  4. config={"configurable": {"thread_id": "your_unique_id"}}: When invoking your compiled graph, you pass a thread_id in the config dictionary. This thread_id acts as the unique identifier for a specific conversation or session, allowing LangGraph to load/save the correct state.

Project: A Persistent Research Assistant

We’ll create a simple “Research Assistant” agent that answers questions in multiple turns. The core demonstration will be its ability to remember previous turns and any “facts” it has internally gathered, even if the application is restarted.

The workflow will be:

  • User asks a question.
  • Research Assistant answers and simulates gathering a “fact” related to the question.
  • The state is automatically saved.
  • In a subsequent invocation (simulating a resume or new turn in the same conversation), the agent remembers prior context and accumulated facts.

Before you run the code:

  • Ensure you have installed langchain-openai or langchain-ollama based on your LLM_PROVIDER.
  • You’ll also need langgraph[sqlite] (or langchain-sqlite directly) for the SqliteSaver.
  • Set your OPENAI_API_KEY environment variable if using OpenAI. If using Ollama, ensure Ollama is running and your chosen model is pulled (e.g., ollama pull llama2).
import os
from typing import TypedDict, Annotated, List
from langchain_core.messages import BaseMessage, HumanMessage, AIMessage
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_ollama import ChatOllama
from langgraph.graph import StateGraph, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.sqlite import SqliteSaver

# Load environment variables from a .env file
from dotenv import load_dotenv
load_dotenv()

# --- Configuration ---
LLM_PROVIDER = os.getenv("LLM_PROVIDER", "openai").lower() # 'openai' or 'ollama'
OLLAMA_MODEL_CHAT = os.getenv("OLLAMA_MODEL_CHAT", "llama2").lower() # e.g., 'llama2', 'mistral'

# --- LLM Initialization ---
def initialize_llm(provider: str, model_name: str = None, temp: float = 0.7):
    """Initializes and returns the ChatLargeLanguageModel based on provider."""
    if provider == "openai":
        if not os.getenv("OPENAI_API_KEY"):
            raise ValueError("OPENAI_API_KEY not set for OpenAI provider.")
        return ChatOpenAI(model=model_name or "gpt-3.5-turbo", temperature=temp)
    elif provider == "ollama":
        try:
            llm_instance = ChatOllama(model=model_name or OLLAMA_MODEL_CHAT, temperature=temp)
            llm_instance.invoke("Hello!", config={"stream": False}) # Test connection
            return llm_instance
        except Exception as e:
            print(f"Error connecting to Ollama LLM or model '{model_name or OLLAMA_MODEL_CHAT}' not found: {e}")
            print("Please ensure Ollama is running and the specified model is pulled (e.g., 'ollama pull llama2').")
            exit()
    else:
        raise ValueError(f"Invalid LLM provider: {provider}. Must be 'openai' or 'ollama'.")

# Initialize the chosen LLM
llm = initialize_llm(LLM_PROVIDER)
print(f"Using LLM: {LLM_PROVIDER} ({llm.model_name if hasattr(llm, 'model_name') else OLLAMA_MODEL_CHAT})\n")


# --- 1. Define Agent State ---
class AgentState(TypedDict):
    """
    Represents the state of our conversational agent.
    'messages' will automatically manage conversation history thanks to add_messages.
    'facts_gathered' is a custom field to demonstrate persistence of internal agent data.
    """
    messages: Annotated[List[BaseMessage], add_messages]
    facts_gathered: List[str] # To store internal facts gathered during conversation


# --- 2. Define Agent Node ---
def research_assistant_node(state: AgentState) -> AgentState:
    """
    A simple research assistant node that responds to user questions and
    simulates gathering specific facts.
    """
    print("\n--- Node: Research Assistant ---")
    messages = state['messages']
    # The last message in the `messages` list should be the current user's input
    last_user_message = messages[-1].content 

    # Define the LLM prompt for the research assistant
    # We include the full message history in the prompt to maintain conversation context
    prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a helpful and knowledgeable research assistant. Answer the user's question clearly and concisely. If the question implies gathering information, add a short, relevant 'fact' to your internal knowledge base about the topic. Keep your responses brief and directly answer the question."),
        ("human", "{question}") # {question} is implicitly the last message in `messages` here
    ])
    
    llm_with_history = prompt | llm # Chain the prompt with the LLM

    # Invoke the LLM with the current conversation history.
    # LangGraph will automatically pass `state['messages']` as `messages` to the prompt.
    response = llm_with_history.invoke({"question": last_user_message}) 
    ai_response_content = response.content

    # Simulate gathering a 'fact' based on certain keywords in the user's question
    new_fact = ""
    if "Python" in last_user_message or "programming" in last_user_message:
        new_fact = "Python is a versatile, high-level programming language known for its readability."
    elif "history" in last_user_message or "event" in last_user_message:
        new_fact = "The fall of the Berlin Wall in 1989 was a pivotal event in modern history."
    elif "space" in last_user_message or "planet" in last_user_message:
        new_fact = "Mars is often called the 'Red Planet' due to its iron oxide-rich surface."

    # Retrieve existing facts and append the new one if unique
    facts_gathered = state.get('facts_gathered', [])
    if new_fact and new_fact not in facts_gathered:
        facts_gathered.append(new_fact)
        print(f"  (Simulated) Gathered fact: '{new_fact}'")

    print(f"  AI Response: {ai_response_content}")
    # Return the updated state. LangGraph will automatically save this.
    return {
        "messages": [AIMessage(content=ai_response_content)], # Add AI's response to messages
        "facts_gathered": facts_gathered # Update the list of facts
    }

# --- 3. Build the LangGraph Workflow with Persistence ---
print("--- Building the Stateful Conversational Agent Workflow ---")

# Configure the memory (checkpointer).
# Using ':memory:' creates an in-memory database, which is good for quick tests.
# For persistent file-based storage across runs, use a file path:
# memory = SqliteSaver.from_conn_string("sqlite:///langgraph_checkpoints.sqlite")
memory = SqliteSaver.from_conn_string(":memory:") 

workflow = StateGraph(AgentState)

# Add the research assistant node to the workflow
workflow.add_node("research_assistant", research_assistant_node)

# Set the entry point of the graph. The conversation starts with the assistant.
workflow.set_entry_point("research_assistant")

# Define the edge: After the research assistant responds, the graph reaches an END state.
# For multi-turn conversations, the persistence mechanism handles resuming.
workflow.add_edge("research_assistant", END) 

# Compile the graph and enable checkpointing by passing the 'memory' object.
conversational_app = workflow.compile(checkpointer=memory)
print("Stateful Conversational Agent workflow compiled successfully.\n")


# --- 4. Demonstrate State Persistence ---
print("--- Demonstrating State Persistence Across Turns and Sessions ---")

# --- Scenario 1: Start a new conversation (Thread ID: 'user_session_abc') ---
# This is a unique identifier for this conversation/session.
thread_id_1 = "user_session_abc"
print(f"\n--- Scenario 1: Starting new conversation (Thread ID: '{thread_id_1}') ---")

print("User: What is the capital of France?")
app_state_1_turn_1 = conversational_app.invoke(
    {"messages": [HumanMessage(content="What is the capital of France?")]},
    config={"configurable": {"thread_id": thread_id_1}}
)
print(f"Current Facts Gathered: {app_state_1_turn_1['facts_gathered']}")
print(f"Messages count after Turn 1: {len(app_state_1_turn_1['messages'])}")

print("\nUser: Tell me about Python programming.")
app_state_1_turn_2 = conversational_app.invoke(
    {"messages": [HumanMessage(content="Tell me about Python programming.")]},
    config={"configurable": {"thread_id": thread_id_1}}
)
print(f"Current Facts Gathered: {app_state_1_turn_2['facts_gathered']}")
print(f"Messages count after Turn 2: {len(app_state_1_turn_2['messages'])}")
# At this point, the entire state (messages, facts_gathered) for thread_id_1 is checkpointed.

# --- Scenario 2: Resume the conversation (Thread ID: 'user_session_abc') ---
# Simulates a user coming back later or a new turn in the same session.
# LangGraph will automatically load the last saved state for this thread_id.
print(f"\n--- Scenario 2: Resuming conversation (Thread ID: '{thread_id_1}') ---")

print("User: What historical event happened in 1989?")
app_state_1_turn_3_resumed = conversational_app.invoke(
    {"messages": [HumanMessage(content="What historical event happened in 1989?")]},
    config={"configurable": {"thread_id": thread_id_1}}
)
print(f"Current Facts Gathered: {app_state_1_turn_3_resumed['facts_gathered']}")
print(f"Messages count after Resumed Turn 3: {len(app_state_1_turn_3_resumed['messages'])}")

# Verify previous history and facts are present:
print(f"  (Verification) First user message in history: '{app_state_1_turn_3_resumed['messages'][0].content}'")
if "Python is a versatile" in str(app_state_1_turn_3_resumed['facts_gathered']):
    print("  (Verification) 'Python' fact from previous turn is present in facts_gathered.")


# --- Scenario 3: Start a new, separate conversation (Thread ID: 'new_user_session_xyz') ---
# This demonstrates that different thread IDs maintain independent states.
thread_id_2 = "new_user_session_xyz"
print(f"\n--- Scenario 3: Starting new, separate conversation (Thread ID: '{thread_id_2}') ---")
print("User: Tell me about the planet Mars.")
app_state_2_turn_1 = conversational_app.invoke(
    {"messages": [HumanMessage(content="Tell me about the planet Mars.")]},
    config={"configurable": {"thread_id": thread_id_2}}
)
print(f"Current Facts Gathered: {app_state_2_turn_1['facts_gathered']}")
print(f"Messages count after Thread 2, Turn 1: {len(app_state_2_turn_1['messages'])}")


print("\n" + "="*60)
print("--- Persistence Demonstration Complete ---")
print("Notice how 'facts_gathered' and message history correctly persisted and were loaded")
print("across invocations within the same thread_id, but remained separate for different thread_ids.")
print("To see disk persistence (data saved to a file), change ':memory:' to a file path")
print("like 'sqlite:///langgraph_checkpoints.sqlite' in the code.")
print("="*60 + "\n")

Code Explanation:

  1. AgentState Definition:
    • This TypedDict is the blueprint for the information our agent remembers.
    • messages: Annotated[List[BaseMessage], add_messages] is key. LangGraph automatically handles appending incoming user messages and outgoing AI messages to this list. This is how the conversation history is built and, importantly, saved.
    • facts_gathered: List[str] is a custom field we added. It demonstrates that any custom data you include in your state schema will also be persisted. Our research_assistant_node will add simulated “facts” to this list.
  2. research_assistant_node:
    • This function acts as our conversational agent.
    • It takes the current state, extracts the last_user_message, and uses an LLM to generate a response.
    • Crucially, the LLM is prompted with the full messages history to maintain context.
    • It includes a simple if condition to simulate gathering a “fact” (e.g., about Python, history, or space) based on keywords in the user’s input. This new_fact is then added to the facts_gathered list in the state.
    • The function returns a new state dictionary containing the updated messages (with the AI’s response) and the potentially updated facts_gathered list. LangGraph automatically saves this new state if a checkpointer is configured.
  3. SqliteSaver for Persistence:
    • memory = SqliteSaver.from_conn_string(":memory:") configures our checkpointer. Using :memory: means the database exists only in RAM, perfect for quick testing without creating files.
    • For true disk persistence (where data remains even after your script stops), you would use memory = SqliteSaver.from_conn_string("sqlite:///langgraph_checkpoints.sqlite"). This would create a file named langgraph_checkpoints.sqlite in your project directory to store all conversation states.
  4. Graph Construction and Checkpointing (workflow.compile(checkpointer=memory)):
    • A StateGraph is defined with our AgentState.
    • Our research_assistant_node is added as a node.
    • workflow.set_entry_point("research_assistant") sets the starting point.
    • workflow.add_edge("research_assistant", END) defines a simple flow: the assistant responds, and the graph reaches an END state. For multi-turn conversational agents, the magic of resuming happens when you invoke the graph again with the same thread_id.
    • The most important line for persistence is conversational_app = workflow.compile(checkpointer=memory). This tells LangGraph to use our memory (the SqliteSaver) to save and load the state automatically.
  5. Demonstrating Persistence (invoke with thread_id):
    • Scenario 1 (New Conversation): We define a thread_id_1 (e.g., “user_session_abc”). The first invoke with this ID creates a new conversation entry in the checkpoint database.
    • Scenario 2 (Resume Conversation): Subsequent invoke calls using the same thread_id_1 are where persistence is demonstrated. LangGraph automatically loads the last saved state associated with “user_session_abc”. You’ll see that the agent remembers previous messages and the facts_gathered list correctly accumulates facts from prior turns, proving state retention.
    • Scenario 3 (New, Separate Conversation): We introduce thread_id_2 (e.g., “new_user_session_xyz”). This demonstrates that LangGraph maintains completely separate and independent states for different thread_ids, ensuring conversations don’t mix.

This project is fundamental for building any long-running AI application where you need to maintain context across sessions or user interactions, providing a seamless and intelligent experience.

Leave a comment

I’m Arpan

I’m a Software Engineer driven by curiosity and a deep interest in Generative AI Technologies. I believe we’re standing at the frontier of a new era—where machines not only learn but create, and I’m excited to explore what’s possible at this intersection of intelligence and imagination.

When I’m not writing code or experimenting with new AI models, you’ll probably find me travelling, soaking in new cultures, or reading a book that challenges how I think. I thrive on new ideas—especially ones that can be turned into meaningful, impactful projects. If it’s bold, innovative, and GenAI-related, I’m all in.

“The future belongs to those who believe in the beauty of their dreams.”Eleanor Roosevelt

“Imagination is more important than knowledge. For knowledge is limited, whereas imagination embraces the entire world.”Albert Einstein

This blog, MLVector, is my space to share technical insights, project breakdowns, and explorations in GenAI—from the models shaping tomorrow to the code powering today.

Let’s build the future, one vector at a time.

Let’s connect