Placement groups and parallel streams: getting the bandwidth you paid for

Published August 18, 2026

The Gbps figure in an instance catalogue is a ceiling for the whole instance, not for one connection. On AWS Graviton4 we measured a single TCP flow reaching 4.96 Gbps on an instance whose aggregate was 39.72 Gbps — the same instance pair, in the same test run, over the same wire. That shape is advertised at 50 Gbps, so one connection collected about a tenth of the advertised figure, and an eighth of what the same instance managed across eight streams. Two configuration choices explain most of that gap: which placement group the instances are in, and how many streams your software opens. Neither is a tuning nicety. At default settings on AWS, parallelising is not an optimisation — it is how you obtain the bandwidth you are already being billed for.

What was measured, and where

iperf3 between a pair of instances, plus round-trip time, with 8 parallel streams and with 1. On AWS: two shapes, c8g.4xlarge and c8gn.4xlarge, in us-west-2, within one availability zone, across all three placement modes. On GCP: c4a-standard-16 in us-central1-a, within one zone, default placement only. On OCI: VM.Standard.A1.Flex at 16 OCPU in us-ashburn-1 AD-1, within one availability domain, default placement only.

The streams half of this article covers all three clouds. The placement half covers two: AWS across three modes, and GCP under a COLLOCATED policy. OCI's cluster placement group API was not available to us at all, so it contributes default-placement numbers only and is the one remaining hole. Everything here is one sample per condition, one shape per cloud, in three different regions, and every conclusion below is labelled with which footing it stands on.

The AWS matrix

Four things fall straight out of that table, and each is worth its own section.

A cluster placement group nearly doubles single-flow throughput

This is the finding that surprised us. A single TCP stream got 4.96 Gbps at default placement and 9.53 Gbps inside a cluster placement group — a 1.92× improvement — and it happened on both shapes, at the same pair of numbers to three significant figures. The 15 Gbps instance and the 50 Gbps instance have the same single-flow behaviour, and the same single-flow response to being placed close together.

Round-trip time moves in the same direction. On c8g, RTT fell from 0.170 ms to 0.072 ms, a factor of 2.4. On c8gn, from 0.134 ms to 0.079 ms, a factor of 1.7. A single TCP connection's throughput is bounded by its window divided by its round-trip time, so a shorter path plausibly explains a higher single-flow ceiling, and that is the obvious mechanism to reach for.

We are not going to claim it is the whole mechanism, because the numbers actively contradict the simplest version of it. Under a fixed-window model, single-flow throughput scales as 1/RTT, so the throughput ratio should equal the RTT ratio. On c8g, RTT improved 2.4× and throughput 1.92×. On c8gn, RTT improved only 1.70× but throughput improved 1.92× — a throughput gain larger than the RTT gain, which a fixed window cannot produce. And the two shapes landed on the same single-flow figures despite RTT ratios of 2.4 and 1.7: 4.96 Gbps default and 9.53 Gbps cluster, identical to three significant figures across two different instance types. Values that repeat that exactly across shapes look more like a per-flow cap being applied at two discrete levels than like a smooth window-over-RTT effect. We did not instrument deeply enough to tell the difference, and one sample per cell is not enough to try. What we can state is the observation, which is what you would configure against either way: at default or spread placement one flow gets 4.96 Gbps, inside a cluster group one flow gets 9.53 Gbps, on both shapes.

Note what does not change on c8g: aggregate 8-stream throughput is 14.89 Gbps in all three placement modes, identical to three significant figures. If your software already opens several connections, a cluster group buys you latency, not bandwidth, on that shape. If your software opens one, it buys you close to double.

c8gn only reaches its advertised 50 Gbps inside a cluster group

AWS advertises c8gn.4xlarge at 50 Gbps base and peak. We measured 49.74 Gbps — 99.5% of the advertised figure — inside a cluster placement group, and 39.72 Gbps, or 79%, at default placement. Spread placement gave 38.12 Gbps, about 76%.

So on the network-optimised shape, placement alone is worth about a fifth of the advertised 50 Gbps — the gap between 99.5% and 79% — even when you are running eight parallel streams. That is a meaningful gap to leave on the table on an instance carrying a 49% price premium over c8g.4xlarge ($0.9480 against $0.6381).

The contrast with c8g is instructive. c8g.4xlarge is advertised at 15 Gbps and returned 14.89 Gbps — 99% — at default placement, with no help at all. A 15 Gbps ceiling is low enough that ordinary placement reaches it comfortably, which makes the high-bandwidth shape the one that needs the placement work — the opposite of the intuition that premium hardware should simply be faster out of the box.

There is a catch in that reading, though, and chasing it down produced the most useful result in this section. Those 15 and 50 Gbps figures are not the same kind of number:

AWS publishes c8gn.4xlarge at 50 Gbps base and peak. It publishes c8g.4xlarge at a 15 Gbps peak over a 7.5 Gbps baseline — the shape of a burst allowance. Our standard runs are 20 seconds, well inside any plausible burst window, so on paper our 14.89 Gbps looked like a number c8g could not hold: 99% of its peak but 198% of its published baseline. The obvious inference was that a long transfer would decay toward 7.5, making c8g's 99% far less impressive than c8gn's 99.5%.

We tested that inference over five minutes, and it is wrong. A 300-second paired run at default placement held 14.89 Gbps in every 30-second block from start to finish. The slowest single second was 14.86 Gbps and the fastest 15.04; the last ten seconds ran at 0.999 of the first ten. Nothing in 300 seconds came within 6 Gbps of the published baseline. The burst rating behaved as the usable rate, flat for five minutes.

That is worth more to a reader sizing a transfer than the decay we expected to find. A 7.5-over-15 split invites you to budget for 7.5 Gbps; over a five-minute transfer the honest planning figure was double that. It is also a clean instance of the premise this whole series runs on — published figures are a poor guide to behaviour, in both directions, and the only way to know which is to measure.

Be precise about what this does and does not establish. What we measured: 14.89 Gbps sustained for 300 seconds on one c8g.4xlarge pair at default placement, with no decay. What we did not establish: that the 7.5 Gbps baseline never binds. It may apply over windows longer than five minutes, under network contention we did not encounter, or as a guaranteed floor rather than a post-credit cap — and this is one instance, one run. We have not run the equivalent 300-second test on c8gn. What we can say is that we went looking for a burst fallback on c8g and did not find one. Note the contrast with t4g on the CPU side, where the same suite did catch credits running out mid-run: burstable is not one behaviour, and network burst on this shape did not behave like CPU credits.

Both percentages are ours, computed against AWS's published figures: 49.74/50 = 99.5%, 39.72/50 = 79%, 38.12/50 = 76%, 14.89/15 = 99%. The measured numbers are from the table above; the advertised numbers are from the InstanceFinder AWS dataset. ENA Express was not enabled in any of the runs in this table, and jumbo frames were not explicitly configured, so these are stock-configuration results rather than a tuning ceiling. We have since measured ENA Express separately and it lifts the single-flow figure substantially — that is a subject of its own and gets its own article; treat every single-flow number here as a floor, not a limit.

Spread placement costs almost nothing

Spread placement puts your instances on distinct underlying hardware, which is the whole point of it: it is a fault-isolation tool, so a single host or rack failure cannot take out more than one member of a group. The usual objection is that it must cost performance.

On these measurements, it barely does. c8gn.4xlarge gave 38.12 Gbps spread against 39.72 default, about 4% lower, with RTT essentially unchanged at 0.129 ms against 0.134 ms — marginally better, which on one sample simply means the two are indistinguishable. On c8g.4xlarge, aggregate throughput was identical at 14.89 Gbps and RTT was 0.161 ms against 0.170 ms.

That makes spread a cheap default for anything where correlated failure is the risk you actually care about: replicas of a database, members of a quorum, instances behind one load balancer. You are trading roughly 4% of aggregate bandwidth on the network-optimised shape, and nothing measurable on the ordinary one, for hardware-level fault isolation. The one thing spread does not give you is the single-flow improvement — 4.96 Gbps, the same as default — because it is doing the opposite of what cluster placement does. You cannot have both fault isolation and minimal path length; that is a genuine either/or, and the table tells you what each side costs.

c8gn's 2.7× advantage requires multiple streams to exist at all

This is the section to read if you are about to pay the network-optimised premium. Compare the two shapes at default placement:

With eight streams, c8gn delivers 2.67× the aggregate throughput of c8g (39.72 against 14.89 Gbps) for 49% more money — a good trade if you need it, and better still if you also put it in a cluster group, where 49.74 against c8g's 14.89 is 3.3×. With one stream, though, c8gn delivers exactly the same 4.96 Gbps. Not slightly more. The same number.

And the per-flow ceiling is not an artefact of a short test. In the 300-second run described earlier, single-flow throughput came out at 4.965 Gbps — unchanged to three figures from the 4.96 measured over 20 seconds. So a flow that starts capped stays capped: the cap held for five minutes on the same shape whose aggregate held its full burst rating for the same five minutes. One sample, default placement, but it means the parallelise-first advice applies to sustained transfers and not just to short ones.

So the premium buys you nothing whatsoever unless your data path is parallel. A single-connection workload — one scp, one database replication stream, one client library that pools connections but only uses one at a time under low concurrency, one backup job writing over a single socket — runs at 4.96 Gbps on both shapes, and you would be paying 49% extra for an identical result. Inside a cluster group the single-stream figure improves to 9.53 Gbps, but again identically on both shapes, so the conclusion does not change: placement helps single flows, the network-optimised shape does not.

Which gives a clean decision order for anyone moving a lot of data on AWS. First check whether your transfer is parallel, because that is free and it is the difference between 4.96 and 14.89 Gbps on the instance you already have. Then use a cluster placement group, which costs nothing on the bill — though it is single-AZ and it gives up the hardware fault isolation that spread placement buys, so it is free in dollars rather than free in trade-offs. Only then consider paying for c8gn, and only if eight-way-plus parallelism is genuinely available in your data path. Buying the expensive NIC first is the most common way to spend 49% on nothing.

AWS is the only one of the three where a single flow cannot reach the aggregate

Everything above is an AWS story, and it would be easy to assume the per-flow ceiling is just how cloud networking works. It is not. With all three clouds measured at default placement and 16 vCPU, AWS is the outlier:

On GCP and OCI a single TCP connection reached essentially the entire link: 103% and 95% of the respective 8-stream figures. On AWS a single connection got a third of the measured aggregate on c8g and an eighth of it on c8gn. Two independent clouds, on different silicon in different regions, show no per-flow ceiling we could detect; one cloud plainly has one.

This is the strongest form of the argument this article exists to make, and it is worth stating flatly. On AWS, parallelising your transfers is not an optimisation — it is the only way to collect the bandwidth you are being billed for. On GCP and OCI, in this test, it was unnecessary. That has a practical consequence in both directions: code moved from GCP or OCI onto AWS can lose most of its transfer throughput without a line changing, which is exactly the kind of regression that gets blamed on the wrong component; and an AWS-shaped data path carried to another cloud may be carrying parallelism complexity it no longer needs.

The limits, stated rather than buried. One sample per cloud, one shape each, one zone or availability domain each, default placement only. The GCP figure exceeding 100% is noise plus per-stream overhead, not evidence that parallelising hurts — do not read it that way, and the same goes for OCI's 95%. "No per-flow ceiling we could detect" is what the data supports for GCP and OCI; "they have no per-flow cap" is not, and we are not saying it. These are all default-placement figures; the next section covers what happens when GCP instances are collocated, and OCI remains the one cloud whose placement we could not vary at all. And the GCP paired test initially produced nothing at all, which turned out to be an interactive package-configuration prompt hanging the startup script rather than anything about networking — worth knowing if you run this suite yourself.

Collocation buys latency everywhere. It buys throughput only where there is a ceiling to lift

Putting the two halves of this article together gives a rule that predicts, correctly, when placement will help you and when it will not. We measured a COLLOCATED group-placement policy on GCP as well, which makes the comparison two-way:

On both clouds, collocation improves latency. On only one of them does it improve throughput — and the reason is the per-flow ceiling from the previous section. Collocation cannot lift a ceiling that is not there. On GCP one flow already reaches the whole link, so there is nothing for a shorter path to unlock; on AWS a single flow is capped at a third of aggregate on c8g and an eighth on c8gn, so shortening the path roughly doubles it, and on c8gn it also unlocks the last fifth of aggregate.

The latency result on GCP is the largest single placement effect in the whole campaign. RTT fell from 0.116 ms to 0.054 ms, a 2.1× improvement, and 0.054 ms is the lowest round-trip figure we measured anywhere — below AWS's best cluster-placement result of 0.072 ms. Jitter improved too, from 0.089 ms to 0.068 ms, though it remains the highest of the three clouds.

So the decision rule is simple and it does not depend on which cloud you are on. If your traffic is latency-sensitive, collocate: it helped on both clouds we could test. If it is throughput-bound, collocation is worth real money on AWS and does nothing measurable on GCP — and on GCP you would be paying for it in availability rather than dollars, which is the next section.

Two things to hold the line on. "Throughput does not move" means no measurable effect, not a demonstrated null. The GCP aggregate figures are 20.87 Gbps collocated against 21.06 at default, which is one sample each and well inside noise; a real effect smaller than a couple of percent would be invisible to us. And the GCP result is plain COLLOCATED — we did not vary maxDistance, so a tighter constraint might do more. OCI could not be tested at all.

Measured against what each cloud publishes

Saturating a link is not the same as saturating the link you were sold. Checking the second thing means finding out what each cloud actually promises, and the three do not promise it in the same way — which matters more than the percentages do.

AWS and OCI both landed within a couple of percent of what they publish. OCI's 102% is a measurement marginally above a round published figure, which is what a comfortably-provisioned link looks like, not a miracle. GCP left about a third of its documented ceiling unused in our test, and we did not find a configuration that recovered it: eight streams did not, and neither did a COLLOCATED placement policy.

The three rows are not equally solid, and the differences run the same way as the results, so we are spelling them out. AWS publishes base and peak Gbps for each shape, which is the strongest basis here. OCI publishes a per-OCPU rate, so a Flex shape's ceiling is 1 Gbps multiplied by its OCPU count — note for anyone working from the same dataset that netGbps there is the per-OCPU base unit, not the whole instance. GCP publishes neither a per-shape nor a per-instance figure; its 32 Gbps is derived from a documented per-vCPU rate and series cap, which makes it the weakest of the three comparisons and the one to distrust first. All of these are documented maxima rather than guarantees, and each is one sample. One more thing about GCP's row: it is a default-placement figure, but unlike when we first wrote this, we now know collocation does not rescue it. A COLLOCATED policy left GCP's throughput unchanged, so the 68% is not an artefact of denying GCP the treatment that took c8gn to 99.5% — we gave it that treatment and the number did not move. "GCP left roughly a third of its documented ceiling on the table in our test, and collocating did not recover it" is supportable. "GCP underdelivers" is not, and it is not what we are saying.

Latency and jitter: an observation, not a verdict

The RTT and jitter columns are the least comparable numbers in this article, because the three clouds were measured in three different regions — us-west-2, us-central1 and us-ashburn-1. That compares three datacentres as much as three networks, so treat what follows as indicative only.

OCI's 0.356 ms round trip is roughly two to three times the AWS figures (0.170 and 0.134 ms at default placement) and about three times GCP's 0.116 ms. We have two OCI round-trip samples, 0.356 ms and an earlier 0.395 ms, so that figure is at least repeatable in magnitude; we quote the run that also produced throughput. Reported as an observation; it is not a verdict on Oracle's network, and a cross-region comparison cannot be one. Note also that AWS's own RTT varied more with placement than these clouds differ from each other: c8g went from 0.170 ms at default to 0.072 ms in a cluster group, which is a larger swing than the AWS-to-GCP gap.

Jitter runs the other way and is the one number that goes against GCP. Round-trip mean deviation was 0.089 ms on the GCP pair, against 0.046 ms on OCI and 0.006 to 0.016 ms across the six AWS pairs — so GCP is roughly 6 to 15 times noisier than AWS on this measure, depending on which AWS pair you compare against. For bulk transfer that is irrelevant. For a latency-sensitive request path it may not be, because tail latency is usually what matters and jitter is where tail latency comes from. One sample per cloud, and it deserves a dedicated test rather than a conclusion.

Can you even ask for placement? Checked on all three

Everything above assumes placement is a control you have. On AWS it is. On the other two it is more complicated, and we checked rather than assumed. All three verified on 2026-08-18.

✓ available, and its effect measured above  ·  × not available to us

On GCP, a COLLOCATED placement policy creates without trouble, but two practical wrinkles are worth knowing before you plan around it. The --max-distance flag is absent from gcloud 580.0.0, although the beta REST API does accept maxDistance and returned HTTP 200 — so the capability exists ahead of the CLI surface. And compact placement requires onHostMaintenance=TERMINATE, which means giving up live migration: your instances will be stopped for host maintenance rather than moved.

Now that we have measured the benefit, that trade-off is sharper rather than softer. What you get for forfeiting live migration is a 2.1× latency improvement and no measurable throughput gain. For a latency-sensitive service that is a real and possibly decisive win. For a bulk-transfer workload you would be giving up live migration for nothing you can measure — so on GCP, unlike on AWS, collocation is a decision to make on the shape of your traffic rather than a free optimisation to switch on.

On OCI, oci cpg cluster-placement-group exists as a command but returned 404 NotAuthorizedOrNotFound for us. That is the signature of a feature with limited availability that Oracle has to enable for a tenancy, rather than of a wrong invocation. Treat cluster placement on OCI as something you cannot assume you have, and confirm with Oracle before designing around it.

What we did not measure

• Every number here is a stock-configuration result. ENA Express was not enabled in these runs and jumbo frames were not explicitly configured. Both act on exactly the single-flow behaviour this article is about, and we have since measured ENA Express on c8gn: it raises the single-flow figure well above anything in the tables above, without touching the aggregate. So read the single-flow numbers here as a floor set by the defaults, not as a ceiling set by the hardware — the per-flow limit turns out to be a good deal more movable by configuration than this article on its own would suggest. That measurement is the subject of a separate article. Jumbo frames remain untested.
• One sample per cell. Every figure in the AWS matrix and the GCP table is a single measurement. The exact repeatability of 14.89 Gbps across three placement modes and of 4.96 and 9.53 Gbps across two shapes is reassuring about the instrument, but it is not the same as repeated runs on different days.
• OCI and GCP contribute one shape each, at default placement only. AWS was measured across two shapes and three placement modes; the other two clouds were not. Any statement here about how GCP or OCI behave rests on a single sample, and the single-flow finding in particular deserves a repeat run before anyone rearchitects around it.
• OCI placement could not be tested at all, so it is the one remaining hole: its cluster placement group API returns 404 for our tenancy and Oracle has to enable it. AWS and GCP placement are both measured, but on GCP only a plain COLLOCATED policy — we did not vary maxDistance, so a tighter constraint might do more than we saw.
• The three clouds' ceilings are not equally solid. AWS publishes base and peak Gbps per shape, which is the strongest basis here. Our c8g percentage is against its 15 Gbps peak rather than its 7.5 Gbps baseline — and a 300-second run showed it holding that peak rate with no decay, so treating the peak as the relevant denominator is what the measurement supports. OCI publishes a per-OCPU rate, so a Flex shape's figure is arithmetic on a published unit. Google does not expose network bandwidth per shape at all; the 32 Gbps is derived from a documented per-vCPU rate and series cap. A percentage measured against a derived number deserves less weight than one measured against a published one, and we would rather say that than present the rows as equivalent. For completeness: C4A's series maximum under Tier_1 networking is 100 Gbps, but that is a series ceiling rather than a figure for a 16-vCPU shape, Tier_1 was not enabled here, and it should not be used as an alternative denominator.
• All tests were within one availability zone. Nothing here describes cross-AZ or cross-region behaviour, where both latency and cost change substantially.
• Regions differ across the three clouds (us-west-2, us-central1, us-ashburn-1), which is fine for comparing silicon but not for comparing latency.

The short version

1. Count your streams before you buy a bigger NIC. On AWS, one flow got 4.96 Gbps on both a 15 Gbps shape and a 50 Gbps one. Parallelism is the free way to convert an instance's advertised bandwidth into throughput you can use, and at default settings it is the main one.
2. Use a cluster placement group for single-flow work. It took one stream from 4.96 to 9.53 Gbps on both shapes, and more than halved RTT on c8g (0.170 to 0.072 ms). It is also the only way c8gn reached its advertised 50 Gbps: 49.74 Gbps inside a cluster group against 39.72, or 79%, at default placement.
3. Collocate for latency on either cloud; expect a throughput gain only on AWS. A COLLOCATED policy on GCP took RTT from 0.116 to 0.054 ms — the lowest figure we measured anywhere, beating AWS's best cluster result of 0.072 — while leaving throughput unchanged. Collocation lifts throughput where there is a per-flow ceiling to lift, and GCP does not have one. On GCP it also costs live migration, so it is a decision rather than a free win.
4. Use spread when correlated failure is the risk. It cost about 4% of aggregate throughput on c8gn (38.12 against 39.72 Gbps) and nothing measurable on c8g. What it does not give you is the single-flow gain, so you are choosing between fault isolation and path length.
5. Only pay for the network-optimised shape if your data path is genuinely parallel. c8gn's 2.67× aggregate advantage at default placement is real with eight streams and completely absent with one, at a 49% premium. Put it in a cluster group and the advantage is 3.3× (49.74 against 14.89). And do not discount c8g on the assumption that its 15 Gbps is a burst rating it cannot hold: over five minutes, it held it.
6. AWS is the odd one out on single flows, not GCP. On both GCP and OCI a single stream reached essentially the whole link (103% and 95% of their 8-stream figures); on AWS it reached 33% on c8g and 12% on c8gn. Set against that, GCP's single flow was also the only one that left a large share of its ceiling unused — 68% of a derived 32 Gbps, against 99% and 99.5% for the two AWS shapes in their best configurations and 102% for OCI. One sample per cloud, one shape each, one zone each, so verify on your own workload before relying on any of it.

The rest of this benchmark run is written up in the provider differences article and the cost analysis. To compare the shapes themselves, see AWS, GCP and OCI, or Compare all.