Preflight reference

Preflight checks before dispatch

SparkPilot checks the run before dispatch. A failed check blocks submission through SparkPilot; it does not stop charges from existing AWS resources. Use the check codes below to understand the verdict and its engine-specific scope.

The three statuses

A failed check blocks dispatch. Warnings allow it.

Pass

The check passed. Other checks may still block dispatch.

Warning

This check allows dispatch but needs attention, such as an outdated EMR release or a budget warning.

Fail

A failed check blocks SparkPilot dispatch. No job is submitted for that attempt. Existing AWS resources may still incur charges. Read the check detail for the next step.

What the preflight checks

Check codes appear in the verdict. Some checks apply only to the engines or onboarding modes noted below.

Identity and cross-account access

Checks the cross-account role and execution identity used for dispatch. SparkPilot does not require long-lived AWS access keys.

environment.customer_role_arn
iam.assume_role_chain
iam.runtime_identity
iam.no_static_credentials
issue3.sts_caller_identity
issue3.iam_simulate_principal_policy
issue3.eks_describe_cluster
issue3.irsa_trust_subject

The STS caller-identity, per-action IAM simulation, EKS describe, and execution-role trust (IRSA) checks run on EMR on EKS with BYOC-Lite onboarding only. On other engines, preflight enforces policy and quota gates but does not simulate per-action IAM permissions before submit — see the limitations page for the exact scope.

Environment readiness

Checks that the target environment is provisioned and configured for dispatch.

environment.status
environment.virtual_cluster
emr.security_configuration

Engine and configuration compatibility

Checks the EMR release, execution role, logging destination, and Spark configuration.

config.emr_release_label
config.emr_release_currency
config.graviton_release_support
config.execution_role
config.log_group_prefix
run.spark_conf_policy
run.vpa_compat
run.s3_express_config

Cost and capacity

When a team budget is configured, preflight can warn or block before dispatch. The YuniKorn check compares requested vCPU with the configured queue maximum; it does not measure live queue headroom. A blocked run's current in-product explanation can misstate the budget cause.

team_budget
yunikorn_queue_capacity

Governance policies

Configured global, workspace, and environment policies appear as individual checks. Hard rules block dispatch; soft rules warn.

policy.<rule_type>

Onboarding readiness (BYOC-Lite EMR on EKS)

For EMR on EKS environments onboarded with BYOC-Lite, SparkPilot validates the cross-account and in-cluster wiring your onboarding created — such as the OIDC association, the execution-role trust, the EKS access-entry mode, cluster and namespace configuration, IAM pass-role, and Spot capacity.

byoc_lite.oidc_association
byoc_lite.execution_role_trust
byoc_lite.access_entry_mode
byoc_lite.eks_cluster_arn
byoc_lite.eks_namespace
byoc_lite.namespace_collision
byoc_lite.iam_pass_role
byoc_lite.spot_capacity

This lists representative onboarding checks, not the full set, and they run only on BYOC-Lite EMR on EKS environments.

Fine-grained data access (Lake Formation)

For environments using Lake Formation, preflight checks release support, the service-linked role, and whether the execution role has data grants. It does not prove access to every dataset the job will use.

fgac.emr_release
fgac.service_linked_role
fgac.lf_permissions

Current scope

Per-action IAM permission simulation covers EMR on EKS with BYOC-Lite onboarding. Budget checks can block dispatch; the resulting failure screen may attribute a budget block to environment setup. See current limits for other coverage gaps.

Current limitsSee the pilot guide