# Spark Portability Assessment — SparkPilot reference migration - NEGATIVE CONTROL, seeded difference injected (--seed-difference rounding). This run exists to prove the comparator can fail; it is not an assessment of the runtime upgrade

**emr_serverless:emr-6.15.0 → emr_serverless:emr-7.5.0**

## Recommendation

### Do not cut over — unresolved differences or blockers

`DO_NOT_CUT_OVER`

This report covers 1 workload(s) executed on both emr_serverless:emr-6.15.0 and emr_serverless:emr-7.5.0. No estate was inventoried, so this is a reference run of those workloads and not an assessment of an estate. It says nothing about workloads that were not run.

This recommendation follows from:

- 1 dual run(s) produced different output on emr_serverless:emr-7.5.0 than on emr_serverless:emr-6.15.0
- no estate was inventoried, so nothing is known about workloads other than the one(s) run here
- no source was statically scanned for incompatible constructs

## Summary

- **Decision:** `DO_NOT_CUT_OVER` — do not cut over — unresolved differences or blockers.
- **Jobs inventoried:** 0; **job definitions that cannot port as written:** 0.
- **Source files with blocking constructs:** none.
- **Distinct problems to fix:** 0.
- **Measured change:** emr_serverless:emr-7.5.0 vs emr_serverless:emr-6.15.0: wall clock +159.9%, metered cost +9.1%.
- **What this does not do:** SparkPilot converts nothing. Every problem above is manual work; the deliverable is the evidence, not the migration.

## Coverage

- Jobs inventoried: **0**
- Workloads executed on both platforms: **1**
- Source files scanned: **0**
- Job definitions parsed without error: **n/a — no job definitions were supplied**
- Estate enumeration: **not performed**

## Output equivalence

### `tpch-q1-reference-v1` — NOT_EQUIVALENT

emr_serverless:emr-6.15.0 vs emr_serverless:emr-7.5.0; 10 equality assertions executed.

| Dataset | Check | Severity | Detail |
| --- | --- | --- | --- |
| q1_pricing_summary | schema_type | DIFFERENCE | sum_charge: source type decimal(38,6) vs target type decimal(38,1) |
| q1_pricing_summary | row_hash | DIFFERENCE | content hash differs: source 984eb4735fc1b990… vs target 5e7671c91ac6d5cc… |
| q1_pricing_summary | aggregate_value | DIFFERENCE | sum_charge:max: source 1418207490.410181 vs target 1418207475.3 (tolerance rel=0.0, abs=0.0) |
| q1_pricing_summary | aggregate_value | DIFFERENCE | sum_charge:min: source 1391007086.081212 vs target 1391007105.8 (tolerance rel=0.0, abs=0.0) |
| q1_pricing_summary | aggregate_value | DIFFERENCE | sum_charge:sum: source 8427644657.016188 vs target 8427644818.5 (tolerance rel=0.0, abs=0.0) |


## Performance and cost

| Workload | Platform | Wall clock (s) | Spark | Cost (USD) |
| --- | --- | --- | --- | --- |
| tpch-q1-reference-v1 | emr_serverless:emr-6.15.0 | 48.5 | 3.4.1-amzn-2 | 0.0157 |
| tpch-q1-reference-v1 | emr_serverless:emr-7.5.0 | 126.0 | 3.5.2-amzn-1 | 0.0172 |

**emr_serverless:emr-7.5.0 vs emr_serverless:emr-6.15.0: wall clock +159.9%, metered cost +9.1%.**

_**Neither figure is a platform claim, and this report does not offer one.** The two runs were not given stated, identical capacity — no run manifest records a worker count or size — so wall clock cannot be attributed to the engine. Metered cost is not a clean substitute: EMR Serverless bills each worker for a **one-minute minimum**, and a run shorter than that is charged for capacity it did not consume, which inflates the cost of the faster side. To obtain a defensible figure, re-run both sides at stated, identical capacity on a workload long enough to clear the billing minimum._

_Cost is the platform's own reported resource consumption for the run priced at that platform's published public rates. It is not a bill, and it is not a modelled forecast -- the consumption figures come from the run itself. Where a platform exposes no consumption feed the cell reads `not measured`, which is never rendered as zero._

_**Single observations. Run-to-run variance has not been characterised — these figures bound nothing and are not a benchmark.** Repeat both runs several times before treating any delta as real._

## Portability risks

_No known-incompatible constructs found by the inventory or the scanner._
## Evidence

Every claim above traces to one of these artifacts:

- **EMR Serverless pricing**: https://aws.amazon.com/emr/pricing/
- **EMR job run (emr615-baseline)**: application 00g7nppcfcrf8i09 / job run 00g7nppqou31tg0b in AWS account 400920695203, us-east-1
- **EMR job run (emr750-seeded)**: application 00g7nppdqs2mr409 / job run 00g7nppt0p4fu80b in AWS account 400920695203, us-east-1
- **billed utilisation + rates (emr615-baseline)**: 0.208 vCPU-hr x $0.052624/vCPU-hr + 0.831 GB-hr x $0.0057785/GB-hr = $0.0157 (EMR Serverless us-east-1 x86_64; from aws emr-serverless get-job-run; recomputable in scripts/generate_reference_migration_report.py)
- **billed utilisation + rates (emr750-seeded)**: 0.227 vCPU-hr x $0.052624/vCPU-hr + 0.907 GB-hr x $0.0057785/GB-hr = $0.0172 (EMR Serverless us-east-1 x86_64; from aws emr-serverless get-job-run; recomputable in scripts/generate_reference_migration_report.py)
- **comparator guard mutation run**: scripts/verify_equivalence_guards.py
- **manifest probe**: src/sparkpilot/migration/workloads/probe.py
- **raw run manifests**: docs/evidence/runs/
- **reference workload**: src/sparkpilot/migration/workloads/reference_tpch_q1.py

---

_Generated 2026-08-14T06:00:01+00:00 from commit `b10aac5`. Any platform rates cited above were read on that date and change without notice._