Agent Builder: server (FastAPI / Uvicorn Integration)

To keep the core framework lightweight, the dependencies (fastapi, uvicorn, pydantic) are imported lazily. The server is completely optional but easily attached to any agent via the AgentBuilder.server() method.

---

Configuration Classes

The server is configured using two strictly typed dataclasses (defined in configs.py). These classes govern the network bindings, routing, security, and tuning of the HTTP server.

2.1 CORSConfig

Defines the Cross-Origin Resource Sharing (CORS) policy for the API.

2.2 ServerConfig

The master configuration object passed to AgentBuilder.server().

Network & Binding

Endpoint Routing

Authentication & Limits

API Metadata & OpenAPI

TLS & Tuning

---

Data Models (Request & Response Schemas)

The server relies on strict Pydantic models for request validation and response serialization.

3.1 Chat Schemas

ChatRequest (Payload for POST /chat and POST /chat/stream)

ChatResponse (Returned by POST /chat) Contains the full turn result: response (str), response_markdown (str), message_id (str), trace_id (str), conversation_id (str), user_id (str), invoked_flows (list[str]), invoked_tools (list[str]), errors (list[str]), turn_number (int), is_blocked (bool), and latency_ms (int).

3.2 Conversation Management Schemas

ConversationCreateRequest

ConversationUpdateRequest

---

HTTP Endpoints API

Below are the routes exposed by the server. Authenticaton (Authorization: Bearer <api_key>) is required on all routes except /health, /info, /docs, and /.

4.1 Chat Endpoints

• POST {chat_endpoint}: Sends a message and blocks until the entire turn is complete. Returns a ChatResponse JSON object.
• POST {chat_endpoint}/stream: Sends a message and returns a StreamingResponse (Server-Sent Events). See Section 5.

4.2 Conversation Endpoints

• POST {conversations_endpoint}: Creates a conversation shell upfront. Returns ConversationCreateResponse.
• GET {conversations_endpoint}: Lists conversations for a user.
• Query Params: user_id (Required), limit (Default 100), offset (Default 0).
• GET {conversations_endpoint}/search: Case-insensitive substring search on titles and IDs.
• Query Params: user_id (Required), query (Required), limit, offset.
• GET {conversations_endpoint}/{conversation_id}/messages: Retrieves the full chronological transcript of messages.
• Query Params: limit (Default 200), offset (Default 0).
• PATCH {conversations_endpoint}/{conversation_id}: Updates metadata (e.g., title).
• DELETE {conversations_endpoint}/{conversation_id}: Cascade-deletes the conversation, messages, turn traces, and working memory.

4.3 Utility Endpoints

• GET {health_endpoint}: Returns {"status": "ok", "agent_name": "...", "uptime_s": ...}. Safe for load balancers.
• GET {info_endpoint}: Returns static metadata, versioning, capabilities (has_tools, has_memory), and a dictionary mapping of all registered HTTP routes.
• GET /: Auto-redirects to the Swagger UI (/docs) or /info if docs are disabled.

---

Server-Sent Events (SSE) Streaming

The POST {chat_endpoint}/stream route returns a text/event-stream response. It yields three distinct event types, allowing frontend UIs to render intermediate tool executions while waiting for the final response.

1. event: intermediate: Emitted whenever the agent calls a tool or generates an intermediate thought.

json data: {"type": "tool_call", "label": "Searching DB", "data": {...}}

1. event: done: Emitted exactly once at the end of the turn. Contains the full ChatResponse payload.

json data: {"response": "Here is the data...", "trace_id": "...", "is_blocked": false, ...}

1. event: error: Emitted on timeouts or fatal agent crashes. The stream terminates immediately after this event.

json data: {"error": "TimeoutError", "detail": "Agent did not respond within 120.0s."}

---

The AgentServer Class (Lifecycle Management)

The AgentServer class wraps the FastAPI app and the Uvicorn server, running it safely within an asyncio.Task so it does not block the main Python thread.

Properties

• app: The underlying FastAPI application instance.
• is_running: Boolean indicating if the server is accepting requests.
• effective_port: Returns the actual bound TCP port. This is exceptionally useful in automated testing when ServerConfig(port=0) is used to let the OS assign a random free port.

Methods

• start(): Builds the app, starts Uvicorn, and uses a polling loop to ensure the socket is successfully bound before returning control to the caller.
• stop(): Flags Uvicorn to exit and gracefully awaits the background task. The AgentBuilder automatically registers this method with the Teardown sequence to ensure the server shuts down before database connections are dropped.

---

Error Handling

• 504 Gateway Timeout: If the agent.chat() internal execution exceeds ServerConfig.request_timeout, the server catches the asyncio.TimeoutError and returns a clean 504 JSON response.
• 422 Unprocessable Entity: Malformed payloads (e.g., message exceeding max_message_length or missing user_id) are automatically rejected by Pydantic validation before touching the agent logic.
• 500 Internal Server Error: A global exception handler catches any unhandled exception (like a database disconnect), logs the full traceback, and returns a sanitized ErrorResponse to the client to prevent exposing stack traces to end-users.