Batch and service-tier pricing
Track native Batch API jobs and regional or service-tier rate cards.
Batch API jobs (~50% off)
Native Batch APIs (OpenAI, Anthropic, Gemini Developer API) are async and file-based. Usage is only known after a job completes, so wrappers do not auto-capture them.
Call a track helper once the job reaches a terminal success state:
batch = client.batches.retrieve("batch_abc") # completed
sp.track_openai_batch(client, batch, metadata={"feature": "nightly-eval"})const batch = await openai.batches.retrieve("batch_abc");
await trackOpenAIBatch(openai, batch, {
metadata: { feature: "nightly-eval" },
});Each succeeded line emits one batch=true row with a deterministic idempotency key (batch_id:custom_id). Retries dedupe on the server. The server prices from explicit catalog rate_variants.batch legs (no runtime multiplier). Models without a batch variant fall back to the synchronous rate and log a warning.
AWS Bedrock and Vertex AI batch helpers are not available yet.
Regional and tier pricing
Some providers charge different rates by region, deployment type, or service tier. The SDK forwards these as first-class fields.
Precedence: per-call metadata > wrap options > auto-capture > model default.
| Field | Auto-captured? | Notes |
|---|---|---|
region | Bedrock only | From client region or endpoint host. Azure is not auto-captured; pass a location code such as eastus2 (not East US 2). |
deployment_type | No | Azure: "regional" or "data-zone". Bedrock: "global" / geo. |
service_tier | OpenAI | From response (flex, priority, ...); batch uses batch=true |
upstream_provider | OpenRouter | Which upstream served the call |
Azure: wrap AzureOpenAI as an OpenAI-compatible client. SuperPenguin detects Azure from the endpoint host, but typical Azure URLs use a resource name, not a region. Pass wrap-level fields so SDK estimates pick the matching rate card:
| Azure deployment | What to pass |
|---|---|
| Global Standard | omit both (default catalog price) |
| Data Zone | deployment_type="data-zone" |
| Regional | region and deployment_type="regional" |
Passing only region is not enough for Azure regional rates. region must be the Azure location code (eastus2, not East US 2 or us-east-2; case does not matter). deployment_type must be "regional" or "data-zone" (keep the hyphen). Connecting Azure for billed spend already uses invoice amounts; these fields only affect SDK estimates. Wrap examples: Python and TypeScript.
await openai.chat.completions.create({
model: "gpt-4o",
service_tier: "flex",
messages: [{ role: "user", content: "Hello!" }],
spMetadata: {
region: "us-west-2",
deployment_type: "regional",
service_tier: "priority", // wins over request + response
},
});You do not pass region or deployment_type for gateways. SuperPenguin captures:
- OpenRouter: billed cost, which upstream served the call, and whether you used your own key
- Vercel AI Gateway: billed cost on the call
- LiteLLM: billed cost when LiteLLM includes it on the response