Amazon Bedrock explained: model access, quotas and what RPM really limits

Amazon Bedrock is a managed API in front of a set of foundation models. You do not provision servers, you do not manage weights, and you do not choose an instance type. What you do manage, and what decides whether your application works under load, is throughput.

That throughput is expressed in two numbers per model per region, and understanding how they interact explains most of the confusion around Bedrock capacity.

Model access comes before quota

A fresh account with Bedrock enabled cannot call anything yet. Each model has to be enabled in the Model access screen of the region you intend to use, and several require you to accept the model provider’s end user licence before the request is granted.

This trips people up because the failure looks like a permissions problem. You have the IAM policy, the SDK is configured, the region is right, and the call still fails. Nine times out of ten the model simply has not been enabled in that region.

Access is also regional. Enabling a model in us-east-1 does nothing for eu-west-1, and model availability itself differs by region. Some models launch in a handful of regions and expand later, so the region you pick constrains the menu.

RPM and TPM, and which one you hit first

Bedrock applies two on demand quotas to each model:

• Requests per minute (RPM). How many InvokeModel or Converse calls you may make in a rolling minute.
• Tokens per minute (TPM). The total tokens processed in that minute, input and output combined.

Whichever limit you reach first is the one that throttles you, and which one that is depends entirely on your prompt shape. A chat interface sending short turns will run out of requests long before tokens. A document summariser pushing 40,000 token contexts will exhaust TPM while barely registering on RPM.

This is why a headline RPM figure alone is a poor description of capacity. If your average request is large, do the arithmetic: multiply expected tokens per request by target requests per minute and check the result against the TPM number.

Cross region inference

Inference profiles let a single call be routed to one of several regions in a geography. The practical effect is that your requests draw on the quota pools of more than one region, which raises the throughput ceiling without any change to your code beyond using the profile identifier instead of the plain model identifier.

There are trade offs. Requests may be served outside the region you nominated, which matters if you have data residency commitments, and latency varies with where a given request lands. For most applications the throughput gain is worth it; for regulated workloads, read the routing map first.

On demand against provisioned throughput

On demand billing charges per token with no commitment, subject to the shared quotas above. Provisioned throughput reserves dedicated capacity for a model, billed hourly, with a commitment term for the better rates.

Provisioned capacity makes sense when you have a steady, predictable, high volume workload and throttling is unacceptable. It is poor value for spiky or exploratory work, because you pay for the reservation whether you use it or not. Most teams start on demand, measure for a month, and only then decide whether a reservation pays for itself.

What varies between accounts

Two accounts with identical IAM setups can behave very differently on Bedrock, for reasons that have nothing to do with the code:

• Different models enabled, or enabled in different regions.
• Different applied RPM and TPM values, since quota increases are granted per account.
• Different region availability, if one account is operating where a model has not launched.
• Whether cross region inference profiles are in use.

When comparing accounts, the useful questions are which models, in which regions, at what RPM and TPM. A single number without that context does not tell you much.

Designing around the limits

A few habits keep Bedrock workloads stable:

• Measure tokens, not requests. Log input and output tokens per call from day one. You cannot reason about TPM without that data.
• Trim context aggressively. Token limits reward shorter prompts twice over: lower cost and higher effective request throughput.
• Separate interactive and batch traffic. Give user facing requests priority and let background jobs absorb the throttling.
• Fail over between models. If a smaller model is acceptable for some requests, routing overflow to it keeps the service responsive during spikes.
• Watch the CloudWatch metrics. Bedrock publishes invocation and throttle counts. Alarm on throttles before users report slowness.

Where Bedrock sits next to the rest of AWS

Bedrock capacity is entirely independent of EC2 capacity. An account with a high standard vCPU quota may have no model access at all, and an account with generous Bedrock throughput may not be able to launch a mid sized instance. If you need both, check both. The mechanics of the compute side are covered in AWS vCPU quotas explained, and the broader account picture in AWS account types explained.

If you are weighing platforms for inference work rather than AWS specifically, AWS vs Google Cloud covers how the two approach managed model APIs differently.