LLM Types
The types module defines the standardized Data Transfer Objects (DTOs) used for all LLM interactions within the framework. By enforcing a common schema for requests, responses, and usage metrics, it ensures that the rest of the jazzmine framework (Enhancers, Summarizers, Agents) remains decoupled from the specific JSON formats of various API providers.
Behavior and Context
In the jazzmine architecture, these types act as the "Universal Language" of the LLM layer:
- Structured Communication: Every LLM response, regardless of whether it comes from OpenAI, Anthropic, or a local binary, is converted into an LLMResponse.
- Observability: The LLMUsage class provides a consistent way to track tokens and financial costs, which is eventually persisted in the TurnTrace.
Purpose
- Standardization: Mapping disparate provider outputs to a single internal format.
- Cost Tracking: Enabling per-turn and per-conversation financial auditing.
- Role Management: Defining the three-role system (system, user, assistant) used for prompt construction.
Class Breakdown
LLMUsage (Dataclass)
Tracks the resource consumption of a single completion request.
| Attribute | Type | Description |
|---|---|---|
| prompt_tokens | int | Number of tokens in the input prompt. |
| completion_tokens | int | Number of tokens generated by the model. |
| total_tokens | int | Sum of prompt and completion tokens. |
| cost | Optional[float] | The monetary cost of the request (if provided by the API). |
LLMResponse (Dataclass)
The unified container for model output.
| Attribute | Type | Description |
|---|---|---|
| text | str | The primary text content generated by the model. |
| usage | LLMUsage | Token and cost metadata. |
| model | str | The specific model identifier that processed the request. |
| latency_ms | Optional[int] | Time elapsed for the API call in milliseconds. |
| finish_reason | Optional[str] | Why the model stopped (e.g., "stop", "length"). |
| raw | Optional[Any] | The original, unmodified response dictionary from the provider. |
MessagePart (Dataclass)
Represents a single turn in a conversation history.
| Attribute | Type | Description |
|---|---|---|
| role | str | The participant role: "system", "user", or "assistant". |
| content | str | The actual text of the message. |
Usage Example
from jazzmine.core.llm.types import MessagePart, LLMUsage, LLMResponse
# 1. Defining a conversation fragment
messages = [
MessagePart(role="system", content="You are a helpful assistant."),
MessagePart(role="user", content="Hello!")
]
# 2. Reconstructing a response (usually handled by the Provider)
response = LLMResponse(
text="Hello! How can I help you today?",
usage=LLMUsage(prompt_tokens=20, completion_tokens=10, total_tokens=30),
model="gpt-4o",
latency_ms=450
)
print(f"[{response.model}] {response.text}")LLM Core: errors
The errors module defines a standardized exception hierarchy for all Large Language Model interactions within the jazzmine framework. Because different providers utilize different underlying transport libraries (e.g., httpx for OpenAI, boto3 for Bedrock, or subprocess for Local models), they raise different native exceptions. This module wraps those disparate failures into a consistent set of errors that the agent can programmatically handle.
LLM Core: utils
The utils module provides the helper logic required to handle token accounting and data normalization across different LLM providers. In a multi-provider environment, not every API returns token usage in the same format—or at all. This module ensures that the framework has a consistent baseline for tracking consumption and costs.