Batch and service-tier pricing
Track native Batch API jobs and regional or service-tier rate cards.
Batch API jobs (~50% off)
Native Batch APIs (OpenAI, Anthropic, Gemini Developer API) are async and file-based. Usage is only known after a job completes, so wrappers do not auto-capture them.
Call a track helper once the job reaches a terminal success state:
batch = client.batches.retrieve("batch_abc") # completed
sp.track_openai_batch(client, batch, metadata={"feature": "nightly-eval"})const batch = await openai.batches.retrieve("batch_abc");
await trackOpenAIBatch(openai, batch, {
metadata: { feature: "nightly-eval" },
});Each succeeded line emits one batch=true row with a deterministic idempotency key (batch_id:custom_id). Retries dedupe on the server. The server prices from explicit catalog rate_variants.batch legs (no runtime multiplier). Models without a batch variant fall back to the synchronous rate and log a warning.
AWS Bedrock and Vertex AI batch helpers are not available yet.
Regional and tier pricing
Some providers charge different rates by region, deployment type, or service tier. The SDK forwards these as first-class fields.
Precedence: per-call metadata > wrap options > auto-capture > model default.
| Field | Auto-captured? | Notes |
|---|---|---|
region | Bedrock | From client region or endpoint host |
deployment_type | No | Bedrock global/geo, Azure data zone |
service_tier | OpenAI | From response (flex, priority, ...); batch uses batch=true |
upstream_provider | OpenRouter | Routed upstream host from the response |
await openai.chat.completions.create({
model: "gpt-4o",
service_tier: "flex",
messages: [{ role: "user", content: "Hello!" }],
spMetadata: {
region: "us-west-2",
deployment_type: "regional",
service_tier: "priority", // wins over request + response
},
});OpenRouter: the SDK requests usage accounting and forwards the routed upstream as upstream_provider, plus OpenRouter-reported cost and BYOK upstream cost when present. No extra wrap option is required.