LiteLLM

LiteLLM is a unified LLM gateway that provides a consistent API across 100+ model providers. XeroML integrates with LiteLLM via its built-in callback system.

Installation

pip install xeroml litellm

Setup

import litellm
from xeroml import get_client

xeroml = get_client()

# Add XeroML as a LiteLLM callback
litellm.success_callback = ["xeroml"]
litellm.failure_callback = ["xeroml"]

Or set environment variables:

XEROML_SECRET_KEY="sk-xm-..."
XEROML_PUBLIC_KEY="pk-xm-..."
XEROML_BASE_URL="https://cloud.xeroml.com"

Usage

After setup, all LiteLLM calls are automatically traced:

import litellm

# Calls any provider — all traced automatically
response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello, world!"}],
)

# Also works with Anthropic, Mistral, Cohere, etc.
response = litellm.completion(
    model="claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "Hello, world!"}],
)

LiteLLM Proxy

If you’re running LiteLLM as a proxy server, configure XeroML as a callback in the LiteLLM config:

litellm_config.yaml

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o

litellm_settings:
  success_callback: ["xeroml"]
  failure_callback: ["xeroml"]

environment_variables:
  XEROML_PUBLIC_KEY: "pk-xm-..."
  XEROML_SECRET_KEY: "sk-xm-..."
  XEROML_BASE_URL: "https://cloud.xeroml.com"

This traces all requests through the proxy regardless of which client is calling it.

User and Session Context

Pass user and session context as LiteLLM metadata:

response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    metadata={
        "xeroml_user_id": "user-123",
        "xeroml_session_id": "session-abc",
        "xeroml_tags": ["feature:chat"],
    },
)