Queue & Agent Sinks
Logger reference

Agent Collector Sinks

This module implements sink adapters for local/sidecar log collectors:

1. Introduction

  • VectorLogger
  • FluentBitLogger
  • PromtailLogger

2. Behavior and Context

These sinks are designed for environments where an on-host agent forwards logs to a centralized backend. Each sink:

  • supports sync (requests) and async (aiohttp) send methods
  • formats payloads to agent-specific schemas
  • uses short network timeout and fail-open behavior

3. Purpose

  • Local Forwarding Pattern: Send logs to nearby collector for buffering/transformation.
  • Platform Decoupling: Application code remains backend-agnostic.
  • Ops Flexibility: Agent pipeline can route to multiple destinations without app changes.

4. High-Level API & Examples

Example 1: Vector Sink

python
from jazzmine.logging.sinks.agent_sinks import VectorLogger

sink = VectorLogger(
    sink_config={"type": "vector", "url": "http://localhost:8686"},
    logger_name="api",
    level="INFO",
    use_json=True,
)

sink.initialize()
sink.info("vector health check", ok=True)

Example 2: Promtail Sink with Labels

python
from jazzmine.logging.sinks.agent_sinks import PromtailLogger

sink = PromtailLogger(
    sink_config={
        "type": "promtail",
        "url": "http://localhost:3100/loki/api/v1/push",
        "labels": {"job": "chat", "instance": "api-1"},
    },
    logger_name="chat",
    level="INFO",
    use_json=True,
)

sink.initialize()
sink.warning("upstream latency increased", p95_ms=240)

5. Detailed Class Functionality

VectorLogger

  • Default URL: http://localhost:8686
  • Payload: {timestamp, level, message, logger, ...kwargs}
  • Sync and async POST with JSON body

FluentBitLogger

  • Default URL: http://localhost:2020
  • Optional tag (defaults to logger name)
  • Payload is a list with one event entry
  • Sends to <url>/<tag>

PromtailLogger

  • Default URL: http://localhost:3100/loki/api/v1/push
  • Supports base labels from config and injects runtime level label
  • Payload uses Loki stream format with nanosecond timestamp
  • Appends kwargs as JSON string suffix to log line text

Shared Lifecycle

  • initialize sets endpoint metadata and marks sink initialized.
  • shutdown closes async session via background task.
  • log and alog send payloads with 5-second timeout and swallow exceptions.

6. Error Handling

  • Transport errors are swallowed in both sync and async methods.
  • Dependency issues are raised only if required imports are missing at module level/runtime.

7. Remarks

  • Collector endpoints should be local or low-latency to keep request overhead minimal.
  • Promtail and Loki payload formats are similar but not identical to generic HTTP log APIs.
  • For high-volume workloads, prefer async paths and adjust collector buffering accordingly.