Bedrock provisioned throughput bills when idle

A Bedrock model unit starts billing the hour you buy it and does not stop for a quiet week. At Cohere Command's published rate for a one-month commitment, $39.60 an hour, a single idle model unit runs to about $28,900 a month. The team that bought it has usually moved on to a different model, and the commitment renews itself.

Why a model unit bills whether you call it or not

Provisioned Throughput sells you capacity. On-demand pricing sells you usage. That distinction is the whole story.

A model unit is a throughput allowance for one model: a fixed number of input tokens and a fixed number of output tokens it will handle for you each minute. You pick a model, pick how many units you want, and pick a commitment length. AWS then charges you hourly for those units until you delete them. Use the allowance or leave it sitting, the hourly price is the same.

The commitment length sets your discount. AWS offers three, and the longer you commit the cheaper the hour gets. Here are the rates from the Amazon Bedrock pricing page, per model unit per hour.

Hourly rates hide their own size. Multiply by 730 hours in an average month and the numbers change character.

Those are single units. Capacity planning rarely stops at one.

AWS publishes rates for some models and points you at your account team for the rest, which includes most of the frontier tier. So the number that decides your case may not be on the pricing page at all. Your own Cost Explorer data carries the rate you actually pay, broken out by usage type, and it is the only public-to-you source for a model AWS quotes privately.

Provisioned capacity wins when your sustained throughput is high enough that the on-demand bill clears the flat hourly one. Bursty traffic loses, because the hours between bursts bill at the same rate as the hours during them.

Cloud spend that sits in cost of goods sold makes this arithmetic a margin question rather than an infrastructure one. If you report cloud waste as a gross margin KPI, an idle model unit is one of the cleanest lines you will ever put in front of finance.

What it takes to break even against on-demand

Cohere Command on demand lists at $0.0015 per 1,000 input tokens and $0.0020 per 1,000 output tokens on the Bedrock pricing page, so $1.50 and $2.00 per million. Divide a model unit's monthly cost by those rates and you get the volume where the flat fee stops being the more expensive choice.

Real traffic mixes the two, so your own break-even sits between those columns.

Billions a month is still an abstraction. A 730-hour month holds 43,800 minutes and the unit bills every one of them, so divide once more.

A one-month Cohere Command unit pays for itself only while you hold about 330,000 output tokens a minute, and it wants that rate at 3am on a Sunday as much as at Tuesday lunchtime. Put the same workload on a business-hours schedule and roughly three quarters of what you bought goes to hours nobody is awake for. Utilization decides this purchase, and the hourly rate you negotiated barely moves it. The same arithmetic governs where a Savings Plan breaks even, where the number lands at 73.5% utilization.

There is a second number, and AWS keeps it off the pricing page. A model unit carries a fixed tokens-per-minute allowance, and the Provisioned Throughput guide sends you elsewhere to learn it: "For more information about what an MU specifies, pricing per MU, and to request limit increases, contact your AWS account manager." Public rates tell you the volume that justifies the spend. Only your account team can tell you whether one unit can carry it.

Check them in that order, because the second can undo the first. If a single unit cannot serve your break-even volume you buy a second, the flat monthly cost doubles, and the break-even volume doubles with it. Adding units to reach the break-even moves the break-even. Sustained demand is the only thing that catches it.

What you buy when you purchase Provisioned Throughput

The purchase is a single API call, and one optional field in it decides whether you can ever walk away.

aws bedrock create-provisioned-model-throughput \
  --provisioned-model-name claude-haiku-prod \
  --model-id anthropic.claude-3-haiku-20240307-v1:0:200k \
  --model-units 1 \
  --commitment-duration OneMonth \
  --region us-east-1

Three flags are required: --provisioned-model-name, --model-id, and --model-units. --commitment-duration is optional and accepts OneMonth or SixMonths. Leave it out and you get a no-commitment Provisioned Throughput, which is the only one you can walk back. AWS lists that level in the Provisioned Throughput guide with a single sentence of privileges: "You can delete the Provisioned Throughput at any time."

Six months is the cheapest hour on the pricing page and the one you are stuck with longest. Pick the term against how sure you are of the workload, because the middle column is what you shop for and the right column is what you live with.

Two things sit between you and that command.

Your account starts with no model units for a committed purchase. The CreateProvisionedModelThroughput reference states it directly: "By default, your account has no model units for purchasing Provisioned Throughputs with commitment. You must first visit the AWS support center to request MUs." A committed Provisioned Throughput in your account therefore went through a support case at some point. Somebody asked for it on purpose, which is useful context when the audit below turns one up that nobody claims.

The model ID you purchase with is not the model ID you invoke with. Purchasing takes a contextual variant, with the context window on the end, so it reads anthropic.claude-3-haiku-20240307-v1:0:200k rather than anthropic.claude-3-haiku-20240307-v1:0. The supported models table carries the exact string for every model and region. It is also where you check whether the base model you want can be bought without a commitment at all: Titan Image Generator G1 V1 and V2 cannot.

The commitment renews itself

AWS renews a Provisioned Throughput at the end of each commitment term and keeps your existing configuration. No approval step stands in the way. A six-month commitment nobody is watching becomes a twelve-month one.

Three facts sit underneath that, and they stack badly.

A committed Provisioned Throughput by model units cannot be deleted before its term completes. AWS is explicit about it in the delete documentation. Deletion is the lever everyone reaches for, and for a commitment it is not available.

Cancelling auto renew is a separate action with its own rules. AWS documents it for Provisioned Throughput by Tokens, from the console Actions menu or by setting disableAutoRenew on the update API. Cancel it and the capacity stays in service to the end of the current term, billing the full fee, then deletes itself. You cannot re-enable the renewal afterwards, and you cannot modify that Provisioned Throughput again for the rest of the term.

The model-units update API is thinner than the cancellation story suggests. UpdateProvisionedModelThroughput documents two request fields, desiredModelId and desiredProvisionedModelName, and the CLI reference mirrors that. Before you plan on scripting the cancellation, check what your own build actually exposes.

aws bedrock update-provisioned-model-throughput help

There is a quieter problem in the inventory itself. Nothing in the list response tells you whether a Provisioned Throughput will renew. You get commitmentExpirationTime, which reads like an end date. It is a renewal date until somebody cancels the renewal, and the API gives you no field to tell the two apart. Track that state somewhere you control, because AWS will not hand it back to you.

Find your idle model units by hand

Start with the inventory. Provisioned Throughput is regional, so a forgotten commitment in a region you rarely open bills exactly like one you check every morning.

aws bedrock list-provisioned-model-throughputs \
  --region us-east-1 \
  --query "provisionedModelSummaries[].{name:provisionedModelName,units:modelUnits,commit:commitmentDuration,expires:commitmentExpirationTime,status:status,arn:provisionedModelArn}" \
  --output table

status comes back as one of Creating, InService, Updating, or Failed. Sweep the regions you have ever run inference in.

for region in us-east-1 us-west-2 eu-central-1 ap-northeast-1; do
  echo "== $region"
  aws bedrock list-provisioned-model-throughputs --region "$region" \
    --query "provisionedModelSummaries[].{name:provisionedModelName,units:modelUnits,commit:commitmentDuration,expires:commitmentExpirationTime}" \
    --output table
done

Now the traffic check, and the trap that makes most of these audits come out wrong.

Invocations against provisioned capacity are recorded in CloudWatch with the provisioned model ARN as the value of the ModelId dimension. The foundation model ID is a different series. Query anthropic.claude-... and you will see your on-demand traffic, conclude the model is busy, and learn nothing at all about whether the capacity you are paying for moved a single token. Use the ARN from the inventory above.

# GNU date. On macOS: date -u -v-30d +%Y-%m-%dT%H:%M:%SZ
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name Invocations \
  --dimensions Name=ModelId,Value=arn:aws:bedrock:us-east-1:111122223333:provisioned-model/abcd1234efgh \
  --start-time "$(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 86400 \
  --statistics Sum \
  --output table

Daily buckets are deliberate. CloudWatch keeps one-minute data for 15 days and five-minute data for 63 days, so a single bucket spanning your whole window can quietly return less than you expect once it crosses a retention boundary. Ask for a day at a time and add them up.

An empty result is the finding. No datapoints across 30 days means nothing invoked that capacity, and every one of those 720 hours billed anyway.

Retire or repoint the capacity

What you can do next depends entirely on the commitmentDuration you read in the inventory.

No commitment. Delete it now. Billing stops when the Provisioned Throughput is gone, not when the traffic stops.

aws bedrock delete-provisioned-model-throughput \
  --provisioned-model-id arn:aws:bedrock:us-east-1:111122223333:provisioned-model/abcd1234efgh \
  --region us-east-1

Deletion is immediate and it is permanent. A custom model behind the Provisioned Throughput survives the deletion, so you are giving up the reserved capacity rather than the artifact. Rolling back means purchasing again, at whatever rate applies on the day, so confirm the traffic is genuinely gone before you pull it.

Committed, on a custom model. You cannot delete it, and you can point it at something more useful.

aws bedrock update-provisioned-model-throughput \
  --provisioned-model-id arn:aws:bedrock:us-east-1:111122223333:provisioned-model/abcd1234efgh \
  --desired-model-id arn:aws:bedrock:us-east-1:111122223333:custom-model/mymodel.v2/wxyz5678ijkl

The restriction is tight, and AWS spells it out in the API reference. desiredModelId is rejected outright when the Provisioned Throughput is attached to a base model. When it is attached to a custom model, the target has to be either the base model that custom model came from or another custom model built from that same base. Roll back by running the same command with the original model ARN.

Committed, on a base model. Two options survive. Route real traffic to the endpoint so the capacity earns its rate, or make sure the term ends without renewing. Neither one recovers the money already committed, which is why the audit is worth running against your commitments well before the expiry date rather than after it.

Idle capacity that bills by the hour is a shape worth recognising across your stack. A Snowflake warehouse does the same thing between queries, and tuning AUTO_SUSPEND is the same audit with different nouns.

A quick decision tree

graph TD
  A[Provisioned Throughput] --> B{Invocations on the provisioned ARN?}
  B -- Yes --> C[Capacity is earning its rate: check units against peak throughput]
  B -- No --> D{commitmentDuration set?}
  D -- No --> E[Delete now: billing stops immediately]
  D -- Yes --> F{Attached to a custom model?}
  F -- Yes --> G[Repoint with desired-model-id to a model you actually serve]
  F -- No --> H[Route traffic to it, or stop the renewal before the term ends]

Common questions

Does AWS Bedrock Provisioned Throughput bill when there is no traffic? Yes. Provisioned Throughput bills hourly per model unit from purchase until deletion, and it makes no reference to how many tokens you sent. At Cohere Command's published $39.60 per model unit per hour on a one-month commitment, an untouched unit costs about $28,900 over a 730-hour month.

Can I cancel a Bedrock Provisioned Throughput commitment early? No. AWS documents that a Provisioned Throughput by model units carrying a commitment cannot be deleted until the term completes. Your options during the term are to send traffic to it, or (for a Provisioned Throughput attached to a custom model) repoint it at a different model with desiredModelId.

Does Bedrock Provisioned Throughput renew automatically? Yes. AWS renews it at the end of each commitment term with the same configuration, and no approval step interrupts that. Cancelling auto renew is documented for Provisioned Throughput by Tokens through the console Actions menu or the disableAutoRenew parameter, and once cancelled it cannot be turned back on.

How do I check whether my Bedrock Provisioned Throughput is being used? Query the CloudWatch Invocations metric in the AWS/Bedrock namespace with the provisioned model ARN as the ModelId dimension value. Traffic against provisioned capacity is recorded under that ARN rather than the foundation model ID, so querying the model ID returns your on-demand usage and tells you nothing about the commitment.

What is Bedrock Provisioned Throughput? Dedicated model capacity that you buy by the hour. You choose a model, a number of model units, and optionally a commitment term, and AWS holds that throughput open for you until the Provisioned Throughput is deleted. The hourly charge is identical on a quiet day and on your busiest one.

What is a Bedrock model unit? A model unit is a fixed throughput allowance for one model: a set number of input tokens and output tokens it will process each minute. AWS prices it per hour, and the rate depends on the model and the commitment length. AWS publishes rates for some models and asks you to contact your account team for others.

How do I purchase Bedrock Provisioned Throughput without a commitment? Omit --commitment-duration from create-provisioned-model-throughput. That buys the no-commitment level, which AWS documents as deletable at any time, and it bills at the highest of the three hourly rates. OneMonth and SixMonths buy the discount and take deletion away until the term ends. A committed purchase also needs model units your account does not have by default, so it starts with a support case rather than an API call.

Is Provisioned Throughput cheaper than on-demand Bedrock pricing? Only above a break-even set by your sustained throughput. One Cohere Command model unit on a one-month commitment costs $28,908 a month, against on-demand rates of $1.50 per million input tokens and $2.00 per million output tokens. That breaks even at about 19.27 billion input tokens or 14.45 billion output tokens a month, which works out to a sustained 440,000 input or 330,000 output tokens every minute of the term. Steady high-volume traffic clears it. Bursty or seasonal traffic usually does not, because the idle hours bill at the same rate as the busy ones.

How many tokens per minute does a Bedrock model unit handle? AWS does not publish it. The Provisioned Throughput guide says a model unit specifies how many input and output tokens it processes each minute, then sends you to your AWS account manager for the figure itself. So you can compute the break-even volume from the public hourly and on-demand rates, and confirming that one unit can actually serve that volume takes a conversation with AWS.

Where do I find the rate for a model AWS does not publish? Cost Explorer. Filter to the Bedrock service and group by usage type, and your negotiated model unit rate appears in your own billing data. The public pricing page carries rates for Cohere and Llama 2 model units and refers the rest to your AWS account team.

How OhChimp approaches this

OhChimp reads your Bedrock token usage from the AWS/Bedrock CloudWatch namespace, lists every Provisioned Throughput across your regions, and flags the ones with zero invocations in the window along with their model units, commitment term, and expiration date. Model unit rates are not public for every model, so those findings report the utilization facts rather than a made-up dollar figure, and the plan says plainly whether deletion is available or the term has to run out. You read the plan and click apply, and nothing changes in your account until you do. Savings are then checked against your real AWS bill.

See how the AWS connection works on the AWS integration page.

All OhChimp posts