ML Vector

Welcome back to #30DaysOfLangChain! On Day 1, we established the foundational concepts of Runnables and built our first simple LCEL pipeline. Today, we’re going to enhance our control over LLM interactions by diving into Prompt Templates and Output Parsers.

These two components are essential for steering the LLM’s generation process and then extracting clean, usable data from its often verbose output. With LCEL, integrating them into our pipelines is incredibly intuitive.

1. Prompt Templates: Guiding the LLM’s Response

Think of a prompt template as a fill-in-the-blanks form or a recipe book for your conversation with the LLM. Instead of sending raw, unstructured strings, prompt templates allow you to define a clear structure, inject variables, and even assign roles (system, human, AI) to guide the LLM’s behavior. For chat models, ChatPromptTemplate is the go-to.

• System Messages: Define the AI’s persona, capabilities, or constraints.
  • Example: If your system message is “You are a helpful assistant.”, the LLM will respond generically. If it’s changed to “You are a witty comedian specialized in short, clean jokes.”, the LLM’s entire tone and content will shift dramatically to align with a comedian.
• User Messages: Contain the actual query or input from the user, often with placeholders for dynamic content.
• Placeholders: Variables (e.g., {topic}) that are filled in at runtime.
  • Example: A template might be “Translate ‘{text_to_translate}’ into {target_language}.” Here, {text_to_translate} and {target_language} are placeholders you fill when you invoke the chain. This ensures the LLM always gets the correct context for translation, regardless of the input.

By using templates, you ensure consistency and clarity in your interactions, leading to more predictable and higher-quality responses.

2. Output Parsers: Structuring the LLM’s Output

LLMs are great at generating free-form text, but that text might not always be in the exact, structured format you need (e.g., a simple string, a JSON object, a list of items). Output parsers bridge this gap. They take the raw AIMessage output from the LLM and transform it into a more usable format for your application.

Think of an output parser as a data extractor or a specialized translator. Instead of just getting a block of text, you can tell it to pull out specific pieces of information and put them into a predefined structure.

• StrOutputParser: The simplest parser, as seen on Day 1. It extracts the raw string content from the LLM’s AIMessage. Useful when you just need the text.
  • Example: If an LLM responds “The capital of France is Paris.”, StrOutputParser simply gives you “The capital of France is Paris.” as a Python string.
• PydanticOutputParser: This is where things get powerful! It allows you to define a desired output schema using Pydantic models (Python classes that define data structures). The parser then includes instructions in the prompt to tell the LLM to format its output according to this schema. It then attempts to parse the LLM’s response into an instance of your Pydantic model, raising an error if the format doesn’t match. This is crucial for structured data extraction.
  • Why use it? Imagine you ask an LLM for contact information. Without a parser, you might get “John Doe, works at Acme Corp, Email: john.doe@acme.com, Phone: 555-1234.” This is hard to put into a database. With PydanticOutputParser, you define a ContactInfo model with name: str, company: str, email: str, phone: str. The parser instructs the LLM to output JSON like:

{
  "name": "John Doe",
  "company": "Acme Corp",
  "email": "john.doe@acme.com",
  "phone": "555-1234"
}

The parser then converts this JSON directly into a ContactInfo Python object, making it easy to access contact.name, contact.email, etc.

Using parsers, especially PydanticOutputParser, is a best practice for building reliable applications that depend on structured data from LLMs.

For more details, check out the official LangChain documentation:

• Prompt Templates
• Output Parsers

Let’s build an LCEL chain that takes a topic, generates a joke about it, and then parses the joke into a structured format (a Joke object) using PydanticOutputParser. We’ll also show a simple StrOutputParser example for comparison.

import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser, PydanticOutputParser
from langchain_core.pydantic_v1 import BaseModel, Field
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# Ensure API key is set
if not os.getenv("OPENAI_API_KEY"):
    raise ValueError("OPENAI_API_KEY environment variable not set.")

# Define the LLM
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.8)

# --- Example 1: Basic Joke with StrOutputParser ---

print("--- Example 1: Simple Joke (String Output) ---")

# Define the prompt for a simple joke
joke_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a witty comedian specialized in short, clean jokes."),
    ("user", "Tell me a joke about {topic}."),
])

# Build the simple joke chain
simple_joke_chain = joke_prompt | llm | StrOutputParser()

# Invoke the chain
topic_1 = "cats"
response_1 = simple_joke_chain.invoke({"topic": topic_1})
print(f"Topic: {topic_1}")
print(f"Joke: {response_1}\n")

# --- Example 2: Structured Joke with PydanticOutputParser ---

print("--- Example 2: Structured Joke (Pydantic Output) ---")

# Define a Pydantic model for our structured joke output
class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke.")
    punchline: str = Field(description="The punchline of the joke.")
    category: str = Field(description="The category of the joke (e.g., animal, food, tech).")

# Create a PydanticOutputParser from our Joke model
parser = PydanticOutputParser(pydantic_object=Joke)

# Define the prompt for a structured joke, including parser's format instructions
structured_joke_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a witty comedian. Generate a joke based on the user's topic "
               "and output it in the specified JSON format."),
    ("user", "Tell me a joke about {topic}.\n{format_instructions}"),
])

# Combine the prompt with format instructions from the parser
# This is a key step: parser.get_format_instructions() must be included in the prompt
final_structured_joke_prompt = structured_joke_prompt.partial(
    format_instructions=parser.get_format_instructions()
)

# Build the structured joke chain
structured_joke_chain = final_structured_joke_prompt | llm | parser

# Invoke the chain
topic_2 = "programming"
response_2 = structured_joke_chain.invoke({"topic": topic_2})

print(f"Topic: {topic_2}")
print(f"Joke Setup: {response_2.setup}")
print(f"Joke Punchline: {response_2.punchline}")
print(f"Joke Category: {response_2.category}")
print(f"Parsed Object Type: {type(response_2)}\n")

# Another structured example
topic_3 = "dogs"
response_3 = structured_joke_chain.invoke({"topic": topic_3})
print(f"Topic: {topic_3}")
print(f"Joke Setup: {response_3.setup}")
print(f"Joke Punchline: {response_3.punchline}")
print(f"Joke Category: {response_3.category}")

Code Explanation:

1. ChatPromptTemplate: We create two prompt templates. Notice how we use {topic} as a placeholder for the user’s input. For the structured joke, we also include {format_instructions}.
2. PydanticOutputParser:
   • We define a Joke class inheriting from BaseModel, specifying setup, punchline, and category fields with Field for descriptions (which help the LLM).
   • PydanticOutputParser(pydantic_object=Joke) creates an instance of our parser.
   • Crucially, parser.get_format_instructions() generates a detailed instruction string (often including JSON schema) that the LLM needs to format its output correctly.
   • We use structured_joke_prompt.partial(format_instructions=...) to inject these instructions directly into our prompt. This is vital for the LLM to know how to structure its response.
3. LCEL Chaining: The | operator effortlessly connects our prompt, LLM, and parser. The LLM’s output is fed directly to the parser, which then attempts to convert it into our desired Joke object.