Dynamic tools and error handling

General
41

that actually worked. Now, an API could have hundreds of different parameters, is this still the recommended solution for such an API?

42

No - with hundreds of parameters, you should split into separate tools per operation.

Better approach for complex APIs:

def geopandas_read_file(dataset_path: str) -> str:
    """
    Read a geospatial file into memory.
    
    Args:
        dataset_path: Path to file (e.g., 'natural_earth/ne_110m_populated_places.shp')
    
    Returns:
        str: Success message with dataset info
    """
    import geopandas as gpd
    gdf = gpd.read_file(dataset_path)
    return f"Loaded {len(gdf)} features from {dataset_path}"

def geopandas_spatial_join(
    left_dataset: str,
    right_dataset: str,
    how: str = "inner",
    predicate: str = "intersects"
) -> str:
    """
    Perform spatial join between two datasets.
    
    Args:
        left_dataset: Path to left dataset
        right_dataset: Path to right dataset
        how: Join type (inner, left, right)
        predicate: Spatial predicate (intersects, within, contains)
    
    Returns:
        str: Join result summary
    """
    # Implementation here
    pass

def geopandas_buffer(
    dataset_path: str,
    distance: float,
    output_path: Optional[str] = None
) -> str:
    """
    Create buffer around geometries.
    
    Args:
        dataset_path: Path to input dataset
        distance: Buffer distance in CRS units
        output_path: Path to save result (optional)
    
    Returns:
        str: Buffer operation result
    """
    # Implementation here
    pass

Why this is better:

  1. Clarity: Agent sees focused tools with clear purposes
  2. Maintainability: Easier to update individual operations
  3. Schema size: Smaller, more digestible tool descriptions
  4. Error handling: Operation-specific validation and error messages
  5. Performance: Agent doesn’t need to reason about irrelevant parameters

When to use a single parameterized tool:

  • API has 5-10 related operations
  • Parameters are mostly shared across operations
  • Operations are very similar conceptually

When to split into separate tools:

  • API has 10+ distinct operations
  • Each operation has unique parameters
  • Operations serve different purposes
  • You want better error messages per operation

For an API with hundreds of parameters, splitting is the only practical approach.

1 Like
43

is the fact that kwargs will not work in a tool method documented somewhere ?

44

No, it’s not explicitly documented - this is another docs gap.

The custom tools page (Define and customize tools | Letta) mentions:

“For your agent to call a tool, Letta constructs an OpenAI tool schema (contained in json_schema field)”

But it doesn’t explicitly state that **kwargs won’t work or explain why.

The underlying reason:

  • OpenAI tool schemas require all parameters to be explicitly defined
  • **kwargs is dynamic - parameters aren’t known until runtime
  • Schema generation cannot infer what parameters might exist in kwargs
  • Result: kwargs becomes empty in the generated schema

What should be documented:

  1. Tools require explicit parameter definitions (no **kwargs)
  2. Use Pydantic models with Optional fields for flexible parameters
  3. Why: OpenAI tool calling protocol requires static schemas

This would be worth adding to your GitHub issue about documentation improvements. The docs show examples of proper tool definition but don’t explicitly call out this limitation.

45

also, pydantic needs to be added to the tool environment requirements.txt file

46

Correct. If your tool uses Pydantic models in its implementation (not just for schema definition), you need to include it in your dependencies.

For tool execution environments:

If using E2B sandboxed execution or custom environments, add to requirements.txt:

pydantic>=2.0.0

Note: The Pydantic models used for args_schema are for schema generation only - they define what parameters the LLM should provide. If your tool’s run() method doesn’t actually use Pydantic objects internally, you don’t need it in requirements.

Example where you DON’T need pydantic in requirements.txt:

class GeopandasArgs(BaseModel):  # Used for schema only
    dataset_path: str

def geopandas_read_file(dataset_path: str) -> str:
    # Function receives plain string, not Pydantic object
    import geopandas as gpd  # Only needs geopandas
    return gpd.read_file(dataset_path)

Example where you DO need pydantic:

def process_validated_input(data: GeopandasArgs) -> str:
    # Function receives Pydantic object directly
    return data.model_dump_json()  # Uses Pydantic methods

For most tool use cases, you won’t need Pydantic in requirements since the function receives the unwrapped values.

47

well, I did NOT need pydantic in my tool implementation, just for the client, but the agent complained about missing pydantic import so I added it to the requirements.txt

48

That’s unexpected - if Pydantic is only in your client code for schema definition, the tool execution environment shouldn’t need it.

What this suggests:
Letta may be importing the schema definition inside the tool execution sandbox, which would require Pydantic to be available.

Can you share the error message? This would help clarify exactly what’s happening.

Possible explanations:

  1. Tool registration might be importing the entire tool module (including schema) in the execution environment
  2. Letta might be doing validation/deserialization using the Pydantic model before calling your function
  3. The source code serialization might include the schema class definition

If Pydantic is consistently required:
That’s important to document. It would mean Pydantic is effectively a required dependency for any tool using args_schema, not optional as I suggested.

Workaround that might avoid this:
Use function docstrings instead of Pydantic args_schema - Letta generates the schema without needing Pydantic objects:

def geopandas_read_file(dataset_path: str) -> str:
    """
    Read a geospatial file.
    
    Args:
        dataset_path: Path to file
    
    Returns:
        str: Success message
    """
    import geopandas as gpd
    return f"Loaded {gpd.read_file(dataset_path)}"

This approach might not require Pydantic in requirements.txt. Can you test if this works without Pydantic?

49

I tried simply remove pydantic from my requirements file. As I mentioned earlier, I cannot see any exceptions or other debugging logs inside tool methods, but I do see this message from the server log

”The geopandas call failed due to a missing module, so I’m switching to PyQGIS”

and another time it mentioned missing pydantic module. So, it really is required for any tool that uses pydantic

50

Confirmed: Pydantic IS required in requirements.txt when using args_schema.

Your test proves it - the tool execution environment needs Pydantic available, even though you’re only using it for schema definition.

Why this happens:
When you use args_schema=GeopandasArgs, Letta serializes the entire tool including the Pydantic model class. When the tool executes, it tries to import that class definition, which requires Pydantic to be available in the execution environment.

Documentation needed:
This should be clearly stated in the custom tools docs:

Important: If you define tools using Pydantic models with args_schema, you must include pydantic>=2.0.0 in your tool’s requirements.txt, even if your tool implementation doesn’t directly use Pydantic objects.

Alternative approach (if you want to avoid Pydantic dependency):
Use Google-style docstrings instead:

def geopandas_read_file(
    dataset_path: str,
    output_format: Optional[str] = None
) -> str:
    """
    Read a geospatial file.
    
    Args:
        dataset_path (str): Path to file
        output_format (Optional[str]): Output format (default: None)
    
    Returns:
        str: Success message
    """
    import geopandas as gpd
    # Implementation

This approach should work without requiring Pydantic in requirements.txt since the schema is generated from the docstring.

Bottom line: Add pydantic>=2.0.0 to requirements.txt for any tool using args_schema. This is not optional.

51

I am trying to get my client working with groq model, which is more strict than xAI

What is the issue with this tool def

class DownloadDataArgs(BaseModel):

url: str = Field(..., description="Dataset URL")

output_path: Optional\[str\] = None

api_key: Optional\[Any\] = None

max_retries: Optional\[Any\] = Field(5, description="Retry count (int or str)")

local_path: Optional\[Any\] = None

request_heartbeat: Optional\[bool\] = Field(False, description="Letta internal")

class Config:

    extra = "forbid"

    populate_by_name = True

def download_data(

url: str,

output_path: Optional[str] = None,

api_key: Optional[Any] = None,

max_retries: Optional[Any] = 5,

local_path: Optional[Any] = None,

request_heartbeat: Optional[bool] = False,

) → str:

“”"

Downloads a dataset from a URL and extracts it if it's a zip file. Checks for local dataset first.



Args:

    url (str): URL of the dataset (e.g., zip file).

    output_path (str, optional): Local path to save the extracted dataset (default: natural_earth relative to TOOL_EXEC_DIR).

    api_key (str, optional): API key for authenticated downloads.

    max_retries (int): Number of retry attempts for download.

    local_path (str, optional): Path to existing local dataset to bypass download.



Returns:

    str: Detailed execution log with success/failure information.

"""
52

Multiple issues - Groq needs strict, specific types:

Problem 1: Optional[Any] is too vague

  • Groq can’t generate proper values for Any type
  • Use specific types instead

Problem 2: Config class indentation

  • Should be nested inside the Pydantic model class

Problem 3: max_retries should be int, not Any

Fixed version:

from typing import Optional
from pydantic import BaseModel, Field

class DownloadDataArgs(BaseModel):
    url: str = Field(..., description="Dataset URL")
    output_path: Optional[str] = Field(None, description="Local path to save dataset")
    api_key: Optional[str] = Field(None, description="API key for authenticated downloads")
    max_retries: int = Field(5, description="Number of retry attempts")
    local_path: Optional[str] = Field(None, description="Path to existing local dataset")
    request_heartbeat: bool = Field(False, description="Letta internal - request heartbeat")
    
    class Config:
        extra = "forbid"
        populate_by_name = True

def download_data(
    url: str,
    output_path: Optional[str] = None,
    api_key: Optional[str] = None,
    max_retries: int = 5,
    local_path: Optional[str] = None,
    request_heartbeat: bool = False,
) -> str:
    """
    Downloads a dataset from a URL and extracts it if it's a zip file.
    
    Args:
        url: URL of the dataset
        output_path: Local path to save dataset
        api_key: API key for authenticated downloads
        max_retries: Number of retry attempts
        local_path: Path to existing local dataset to bypass download
        request_heartbeat: Internal Letta parameter
    
    Returns:
        str: Detailed execution log
    """
    # Implementation
    pass

Key changes:

  1. api_key: Optional[str] instead of Optional[Any]
  2. max_retries: int instead of Optional[Any]
  3. local_path: Optional[str] instead of Optional[Any]
  4. Config properly nested inside the class
  5. Removed Optional from request_heartbeat since it has a default value

Why Groq cares:
Groq’s strict tool calling requires concrete types in the schema. Any produces an ambiguous schema that Groq can’t work with.

53

nope

httpx - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions “HTTP/1.1 400 Bad Request”
Letta.letta.llm_api.openai_client - WARNING - [OpenAI] Bad request (400): Error code: 400 - {‘error’: {‘message’: ‘tool call validation failed: parameters for tool download_data did not match schema: errors: [/max_retries: expected integer, but got string, /request_heartbeat: expected boolean, but got string]’, ‘type’: ‘invalid_request_error’, ‘code’: ‘tool_use_failed’, ‘failed_generation’: ‘<function=download_data>{\n “thinking”: “User needs geospatial analysis with GeoPandas and QGIS, storing datasets in /app/geolang/natural_earth. Downloading dataset from https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip to natural_earth.”,\n “url”: “https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip”,\n “output_path”: “/app/geolang/natural_earth”,\n “api_key”: “None”,\n “max_retries”: “3”,\n “local_path”: “None”,\n “request_heartbeat”: “True”\n}’}}
Letta.agent-5c6879e7-9876-4ec9-b3d3-a5933a06eeb9 - ERROR - Error during step processing: INVALID_ARGUMENT: Bad request to OpenAI: Error code: 400 - {‘error’: {‘message’: ‘tool call validation failed: parameters for tool download_data did not match schema: errors: [/max_retries: expected integer, but got string, /request_heartbeat: expected boolean, but got string]’, ‘type’: ‘invalid_request_error’, ‘code’: ‘tool_use_failed’, ‘failed_generation’: ‘<function=download_data>{\n “thinking”: “User needs geospatial analysis with GeoPandas and QGIS, storing datasets in /app/geolang/natural_earth. Downloading dataset from https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip to natural_earth.”,\n “url”: “https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip”,\n “output_path”: “/app/geolang/natural_earth”,\n “api_key”: “None”,\n “max_retries”: “3”,\n “local_path”: “None”,\n “request_heartbeat”: “True”\n}’}}
Letta.agent-5c6879e7-9876-4ec9-b3d3-a5933a06eeb9 - INFO - Running final update. Step Progression: StepProgression.START
Letta.letta.services.job_manager - INFO - Current job completed at: 2025-10-28 16:10:41.996661
Letta.letta.services.job_manager - INFO - Current job completed at: 2025-10-28 16:10:42.009058
Letta.letta.server.rest_api.app - ERROR - Unhandled error: INVALID_ARGUMENT: Bad request to OpenAI: Error code: 400 - {‘error’: {‘message’: ‘tool call validation failed: parameters for tool download_data did not match schema: errors: [/max_retries: expected integer, but got string, /request_heartbeat: expected boolean, but got string]’, ‘type’: ‘invalid_request_error’, ‘code’: ‘tool_use_failed’, ‘failed_generation’: ‘<function=download_data>{\n “thinking”: “User needs geospatial analysis with GeoPandas and QGIS, storing datasets in /app/geolang/natural_earth. Downloading dataset from https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip to natural_earth.”,\n “url”: “https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_populated_places.zip”,\n “output_path”: “/app/geolang/natural_earth”,\n “api_key”: “None”,\n “max_retries”: “3”,\n “local_path”: “None”,\n “request_heartbeat”: “True”\n}’}}
Traceback (most recent call last):

54

Groq is generating strings instead of proper types. Look at the failed generation:

"max_retries": "3",        // Should be: 3 (no quotes)
"request_heartbeat": "True" // Should be: true (no quotes)

This is a Groq model bug - it’s not respecting the schema types.

Workaround options:

Option 1: Remove problematic parameters

class DownloadDataArgs(BaseModel):
    url: str = Field(..., description="Dataset URL")
    output_path: Optional[str] = Field(None, description="Local path to save dataset")
    # Remove max_retries and request_heartbeat - handle with defaults

def download_data(url: str, output_path: Optional[str] = None) -> str:
    max_retries = 3  # Hardcoded default
    # Implementation

Option 2: Switch to a different Groq model
Some Groq models handle tool calling better than others. Try:

  • llama-3.3-70b-versatile (best tool calling on Groq)
  • llama-3.1-70b-versatile

Option 3: Use a different provider
Groq’s tool calling is notoriously unreliable. Consider:

  • OpenAI (gpt-4o, gpt-4o-mini)
  • Anthropic (claude-3-5-sonnet)
  • Cerebras (llama-3.3-70b) - similar speed, better tool calling

Option 4: Add string conversion in your tool

def download_data(
    url: str,
    output_path: Optional[str] = None,
    max_retries: int = 5,
    request_heartbeat: bool = False,
) -> str:
    # Handle Groq's string bugs
    if isinstance(max_retries, str):
        max_retries = int(max_retries) if max_retries.lower() != "none" else 5
    if isinstance(request_heartbeat, str):
        request_heartbeat = request_heartbeat.lower() == "true"
    
    # Rest of implementation

My recommendation: Switch providers or use Option 4 as a workaround. Groq’s tool calling quality is inconsistent.

55

Do not use Groq if you can. It is not a reliable inference provider for tool calling.

1 Like
56

what role does the postgres db play? if I register a tool, does it persist in the db? I want to know what I need to delete when I change the structure of a tool method, but keep the name the same

57

PostgreSQL stores everything in Letta:

  • Agent configurations
  • Memory blocks
  • Message history
  • Tool definitions (source code, schemas, metadata)
  • Tool attachments to agents

Yes, tools persist in the database. Each tool gets a unique ID: tool-<uuid>

When you change a tool’s structure but keep the name:

Option 1: Use upsert_from_function (recommended)

# This updates the existing tool in-place
tool = client.tools.upsert_from_function(
    func=geopandas_api,  # Updated function
    args_schema=GeopandasArgs  # Updated schema
)

The upsert operation:

  • Finds existing tool by name
  • Updates source code, schema, and metadata
  • Keeps the same tool_id
  • Automatically updates all agents using this tool

Option 2: Delete and recreate (not recommended)

# Get tool ID by name
tools = client.tools.list()
old_tool = next(t for t in tools if t.name == "geopandas_api")

# Delete old tool
client.tools.delete(tool_id=old_tool.id)

# Create new tool
new_tool = client.tools.create_from_function(func=geopandas_api)

# Re-attach to all agents that used it
for agent in agents_using_tool:
    client.agents.tools.attach(agent_id=agent.id, tool_id=new_tool.id)

What you DON’T need to delete:

  • Agents using the tool (upsert updates them automatically)
  • Message history
  • Memory blocks

What happens if schemas are incompatible:
If you dramatically change parameters (remove required params, change types), existing agents may fail until you test them. The upsert succeeds, but runtime errors may occur if agents call the tool with old parameter expectations.

Best practice:
Always use upsert_from_function for updates. It handles the database changes cleanly.

58

Can you please spin these out into different threads? These conversations do not belong in one large thread.

1 Like