Jazzmine logo
Plug & PlayStart GuideGitHub
LLM Providers
Core reference

LLM Providers: AnthropicLLM

The AnthropicLLM provider integrates the Claude 3 family of models (such as Opus, Sonnet, and Haiku) into the jazzmine framework. Anthropic models are widely recognized for their high reasoning capabilities, strict adherence to complex system instructions, and industry-leading performance in long-context tasks. This provider uses the Anthropic Messages API, ensuring state-of-the-art interaction with Claude models.

1. Behavior and Context

In the jazzmine architecture, AnthropicLLM serves as a high-precision reasoning engine.

  • System Prompt Isolation: Unlike the standard OpenAI format where system instructions are part of the message list, Anthropic requires the "System" instruction to be passed as a separate top-level parameter. This provider automatically handles that extraction for you.
  • Strict Message Alternation: Anthropic requires messages to strictly alternate between user and assistant. The provider assumes the input history adheres to this conversational flow.
  • Mandatory Token Limits: Anthropic requires a max_tokens value for every request. The provider defaults this to 1024 if not specified, ensuring API compliance.
  • Header Management: Automatically injects the x-api-key and the required anthropic-version (set to 2023-06-01) headers for every request.

2. Purpose

  • High-Reasoning Tasks: Best suited for the MessageEnhancer or FlowSelector where identifying subtle nuances in user intent is critical.
  • Strict Instruction Adherence: Ideal for complex SequentialFlows where the model must follow a series of technical constraints without drifting.
  • Safety and Reliability: Claude models are engineered for reduced hallucinations and high reliability in professional environments.

3. High-Level API Examples

Example: Initializing the Anthropic Provider

python
from jazzmine.core.llm import AnthropicLLM

# Initialize with Claude 3.5 Sonnet
llm = AnthropicLLM(
    model="claude-3-5-sonnet-20240620",
    api_key="sk-ant-...",
    max_tokens=2048,
    temperature=0.0,
    timeout=45.0
)

# Usage remains consistent with the jazzmine BaseLLM interface
response = await llm.agenerate(messages)
print(f"Claude says: {response.text}")
print(f"Latency: {response.latency_ms}ms")

4. Detailed Functionality

__init__(api_key, model, base_url, **kwargs)

Functionality: Configures the Anthropic-specific authentication and initializes the httpx connection pools.

Parameters:

  • api_key (str): Your Anthropic API key (typically starts with sk-ant-).
  • model (str): The model ID (e.g., "claude-3-5-sonnet-20240620").
  • base_url (str): Defaults to https://api.anthropic.com.
  • **kwargs: Inherited parameters like temperature, max_tokens, and timeout.

_prepare_payload(messages, stream) [Internal]

Functionality: Orchestrates the conversion from MessagePart objects to the Anthropic JSON schema.

How it works:

  • System Extraction: Iterates through the message list and joins all system role messages into a single newline-delimited string passed to the system API parameter.
  • Turn Filtering: Collects only user and assistant messages for the messages API array.
  • Default Enforcement: Ensures max_tokens is present (defaulting to 1024).

_parse_response(data, start_time) [Internal]

Functionality: Parses the Anthropic block-based response format into an LLMResponse.

How it works:

  • Content Assembly: Concatenates all blocks of type "text" into a single output string.
  • Usage Normalization: Maps Anthropic's input_tokens and output_tokens to the standardized LLMUsage object.
  • Finish Reason: Maps Anthropic's stop_reason to the finish_reason field.

stream / astream

Functionality: Handles Anthropic's specific streaming event types.

How it works: It listens for server-sent events. It specifically looks for content_block_delta events where the delta type is text_delta. It yields these fragments to provide the user with real-time feedback.

5. Error Handling

  • LLMInvalidRequestError: Raised on HTTP 400 errors. This is common if max_tokens is missing or if the message roles do not alternate (e.g., two "user" messages in a row).
  • LLMRateLimitError: Raised on HTTP 429 errors when the usage tier limits are reached.
  • LLMInternalError: Raised for Anthropic-side server errors (HTTP 5xx).

6. Remarks

  • System Prompts: If you have multiple system prompts in your agent logic, this provider automatically merges them. It is best practice to group your system instructions at the beginning of your message list.
  • Alternating Roles: Anthropic is stricter than OpenAI regarding conversational structure. Always ensure your message list starts with a "user" message and alternates correctly.
  • Tokenizer Calibration: While this provider uses Anthropic's reported token counts, the internal estimate_tokens utility is calibrated for OpenAI-style tokenization and may slightly vary from Claude's actual counts.