Helicone

Proxy vs SDK#

LLM observability tools differ mainly in how they intercept calls. Langfuse and Phoenix prefer SDK-level instrumentation: import their library, decorate a function, and traces flow out. Helicone's default mode is the inverse — it sits in front of the LLM provider as a proxy. The application points at `https://oai.helicone.ai/v1` instead of `https://api.openai.com/v1`, adds a `Helicone-Auth` header, and the proxy logs the request and response on the way through.

The trade-off is integration cost vs. depth. The proxy captures everything that flows through it but knows nothing about internal application steps (retrieval, tool calls, planning). For application-internal spans, Helicone offers an async logging SDK and OTel-compatible ingest. Many teams use both: proxy for raw LLM calls, SDK for orchestration steps.

Getting Started#

from openai import OpenAI

# One-line change: rewrite the base URL, add the auth header
client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
        # Optional: tag this request for filtering in the dashboard
        "Helicone-User-Id": user.id,
        "Helicone-Property-Feature": "rag-search",
    },
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": question}],
)
# Trace now visible in Helicone with user, feature, cost, latency, tokens.

Custom Properties and Sessions#

Helicone's primary discriminators are HTTP headers. `Helicone-User-Id` groups requests by end user. `Helicone-Session-Id` groups multi-turn conversations. `Helicone-Property-*` headers attach arbitrary tags (feature flag, experiment arm, customer tier) that flow into filters and aggregations on the dashboard.

Because the discriminators are headers, the application stays decoupled from Helicone's SDK — switching observability vendors only requires changing what headers you emit, not rewriting code.

Gateway Features#

Sitting in the request path lets Helicone do more than log. The gateway can transparently:

• Cache identical prompts and return the cached completion (configurable TTL).
• Apply per-user or per-key rate limits before requests reach the provider.
• Route across providers (OpenAI ↔ Azure OpenAI ↔ Anthropic) with failover.
• Mask sensitive fields in logs (PII redaction) before they hit storage.
• Stream responses through the proxy without breaking SSE clients.

Pricing and Hosting#

Helicone Cloud has a free tier (10k requests/month) and usage-based paid tiers above that. The open-source self-hosted edition is Apache 2.0 and stores data in Clickhouse plus PostgreSQL plus MinIO. The self-hosted setup is the right call for regulated industries that cannot send prompts to a third party.

Helicone vs Langfuse vs Phoenix#

Pick Helicone when the priority is visibility on raw LLM calls without code restructuring. Pick Langfuse when prompt versioning and evaluation runs sit at the centre of the workflow. Pick Phoenix when retrieval debugging or wider ML observability matter. The three are not mutually exclusive — many production stacks proxy through Helicone and ingest application-level traces into Langfuse or Phoenix.