See what the comparison found
One workload ran on EMR Serverless 6.15.0 and 7.5.0. Its report returned INSUFFICIENT_EVIDENCE; a seeded rounding difference returned DO_NOT_CUT_OVER.
Two comparisons, two verdicts
The seeded comparison adds one rounding difference to the same workload.
One workload — tpch-q1-reference-v1 — executed on both releases. 10 equality assertions ran and no differences were found. The report returned INSUFFICIENT_EVIDENCE because no estate inventory or static scan was attached.
| Field | Baseline | Candidate |
|---|---|---|
| Engine | EMR Serverless emr-6.15.0 | EMR Serverless emr-7.5.0 |
| Spark | 3.4.1-amzn-2 | 3.5.2-amzn-1 |
| Wall clock | 48.5s | 19.6s (−59.5%) |
| Metered cost | $0.0157 | $0.0202 |
A seeded rounding difference changed the output schema, row content hash, and three aggregates. The comparator returned DO_NOT_CUT_OVER.
- Tolerance for this comparison was
rel=0.0, abs=0.0— exact equality. Any rounding change fails the check. - The seeded run is
00g7nppt0p4fu80b, a real job run in the same account and region.
How a verdict is produced
Run a versioned workload on both releases, compare the results, and report what the evidence supports.
A verdict applies to one workload version. This comparison used tpch-q1-reference-v1.
Run the same workload on both releases. The reference comparison used EMR Serverless 6.15.0 and 7.5.0 in us-east-1. An assurance harness submitted those runs; SparkPilot's governed dispatch was not used.
Record output schema, row content hash, aggregates, wall clock, and reported resource use for both runs.
The reference comparison ran 10 equality assertions at the declared exact tolerance (rel=0.0, abs=0.0) and found no differences. Verdict: EQUIVALENT.
The reference pair returned INSUFFICIENT_EVIDENCE: its ten checks passed, but no estate inventory or static scan was attached. The seeded difference returned DO_NOT_CUT_OVER.
What gets compared
Three axes were exercised by the reference migration, each with the figure it produced.
Checks cover column types, a row content hash, and aggregates. The seeded fault changed all three.
Wall clock measured on each run, shown alongside the output checks.
Reported resource use priced at published rates. This estimate excludes other charges; missing usage reads not measured.
The negative control, in full
Five recorded differences from the seeded run at exact tolerance (rel=0.0, abs=0.0).
| Check | Baseline · emr-6.15.0 | Candidate · emr-7.5.0 | Result | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| q1_input_sample | |||||||||||||||
| Schema equivalence | 4 columns | 4 columns | PASS | ||||||||||||
| Row count | 100 | 100 | PASS | ||||||||||||
| Row hash | sha256 04553898f826… | sha256 04553898f826… | PASS | ||||||||||||
| Null counts | 0 nulls / 4 columns | 0 nulls / 4 columns | PASS | ||||||||||||
| Numeric aggregates | 9 aggregates | 9 aggregates | PASS | ||||||||||||
| q1_pricing_summary | |||||||||||||||
| Schema equivalence | 10 columns | 10 columns | DIFFERED | ||||||||||||
| |||||||||||||||
| Row count | 6 | 6 | PASS | ||||||||||||
| Row hash | sha256 984eb4735fc1… | sha256 5e7671c91ac6… | DIFFERED | ||||||||||||
| |||||||||||||||
| Null counts | 0 nulls / 10 columns | 0 nulls / 10 columns | PASS | ||||||||||||
| Numeric aggregates | 24 aggregates | 24 aggregates | DIFFERED | ||||||||||||
Compared exactly, with no tolerance. | |||||||||||||||
On a narrow screen, swipe horizontally to read the full table.
Run identifiers
The EMR application and job-run identifiers behind the figures on this page, in AWS account 400920695203, us-east-1.
- Reference migration
application 00g7nppcfcrf8i09 · job run 00g7nppqou31tg0b - Reference migration
application 00g7nppdqs2mr409 · job run 00g7npps1fnug00b - Seeded negative control
job run 00g7nppt0p4fu80b
Read the report itself
Read the source reports and sample assessment behind these figures.
- Reference migration report (Markdown) — the EMR 6.15.0 → 7.5.0 comparison, verbatim
- Seeded negative-control report (Markdown) — the same comparator returning DO_NOT_CUT_OVER
- Sample portability assessment (Markdown) — the assessment deliverable's shape, built from a fixture estate
- Raw run manifests (JSON) — 7.5.0 baseline and seeded control, the comparator's actual inputs
What this does not prove
These are the limits of the reference comparison.
The report records 0 jobs inventoried, 0 source files scanned, and an incomplete inventory. Estate-scale discovery and scanning remain unverified.
tpch-q1-reference-v1 across EMR Serverless 6.15.0 → 7.5.0. No second workload, release pair, or migration path has been taken end to end.
The reference verdict is INSUFFICIENT_EVIDENCE. Ten checks passed for one workload; the estate was not inventoried.
Both releases completed the reference workload successfully, so this comparison never had to reconcile two different failures. Comparing how a workload fails across runtimes has not been proven.
Where this sits
Compare workload outputs, runtime, and reported resource cost across Spark versions.
Today that comparison is proven on outputs, wall clock, and platform-reported cost, for one workload across one runtime-upgrade pair. The reference workload succeeded on both releases, so failure-mode divergence was not exercised — see the section above.
Each report names the workload, the releases, the checks, and the verdict.
Bring us a workload and a destination
Tell us your workload and target runtime. We will check whether we can run the comparison and show you the resulting report.