Legal

Privacy Policy

What we collect, what we deliberately do not, and where the boundary sits between our control plane and your AWS account. Every statement here describes a real data flow in the product — not a template.

Last updated: 9 August 2026
Effective date: [EFFECTIVE DATE]

Draft pending legal review — not yet in force

Every value highlighted in brackets below — for example [LEGAL ENTITY NAME] — is a placeholder that must be completed before this policy is relied on. The descriptions of what we collect and how it flows are accurate as of the date above; the bracketed items are the corporate, jurisdictional, and retention facts still to be filled in.

The short version

Your Spark jobs run in your AWS account. The datasets they read and write stay there — they do not flow to us.
We hold control-plane records: who you are, which environments and runs exist, what state they reached, what they are estimated to have cost, and an audit trail of control-plane actions.
Sign-in runs through Auth0. We receive an opaque user id and a verified email address. We never see your password.
Paid tiers are invoiced today. Card payments, when enabled, run through Stripe's hosted pages — card numbers never reach our servers, and we store none of them.
No advertising trackers, no third-party analytics SDKs, and no sale of personal information.

The short version is a summary, not the agreement. The sections below are.

1What this policy covers

This policy covers sparkpilot.cloud, the SparkPilot application at app.sparkpilot.cloud, and the SparkPilot API. It describes what [LEGAL ENTITY NAME] collects when you visit the site, evaluate the product, or use it as a customer.

It does not cover your own AWS account, your Spark applications, or the data those applications process. You control those, and your agreement with AWS governs them.

2Your Spark data stays in your AWS account

SparkPilot is a control plane, not a data plane. Your jobs execute on infrastructure inside your AWS account, under an execution role you create, and they read and write your S3 buckets directly. Those bytes never transit SparkPilot.

Three specifics, because these are the questions security reviewers actually ask:

The cross-account role you grant us can read and write exactly one bucket — the job-artifacts bucket you nominate when you deploy the stack. It carries no permission over your other buckets or your data lake. You attach your own data-access policy to your execution role separately, and we never assume that role.
SparkPilot deliberately does not parse the input and output paths inside your job arguments. Before dispatch it reads a single byte of your entrypoint script, using a ranged request, purely to confirm the object exists and is readable.
When a job references a secret, SparkPilot passes the AWS Secrets Manager ARN — never the value. The job fetches the value itself, inside your account, with your execution role. The control plane never reads, stores, or transmits the secret value.

How the access boundary is enforced

3What we collect

Grouped by where it comes from and why it exists. Nothing here is collected for advertising or profiling.

Account and identity: From Auth0 when you sign in: an opaque user identifier and, where your identity provider has verified it, your email address. We store that identifier, the email, your role, your team and organisation, and when you last signed in. We do not receive or store your password, and we do not collect your name or profile picture.
Organisation records: Your organisation name, plan tier, entitlement status, and the invitations you send — the invitee's email address and the role you assigned them.
Environment configuration: The AWS identifiers you give us so we can reach your account: role ARN, ExternalId, execution-role ARN, cluster ARN and namespace, virtual-cluster or application identifier, job-artifacts bucket, and event-log location.
Job and run records: Job definitions, entrypoint URI and digest, arguments, Spark configuration overrides, and for each run: the AWS job-run id, state, attempt, timings, who submitted it and from which surface (UI, API, Airflow, Dagster, or CLI), the failure reason AWS returned, and pointers to the log group, log stream prefix, and Spark UI in your account.
Failure diagnostics: When a run fails we read its logs to classify the failure and store the single matching line, truncated to 500 characters, so the diagnosis can be shown again later. We do not copy log streams into our database.
Cost and usage records: Estimated vCPU-seconds, memory-GB-seconds, and an estimated amount per run. If you deploy the optional reconciliation stack, we also store summed amounts per run and per environment queried from your own Cost and Usage Report. We pull aggregates, never line items.
Audit events: Control-plane actions — identity binding, run submission, dispatch, policy and budget decisions, plan changes — recorded with the acting identity, the source IP address of the request, a timestamp, and details of the action.
Correspondence: Whatever you send us by email or through the contact form on this site, and our replies.
Technical records: Our servers and load balancer produce ordinary request and application logs. These can include IP addresses and request paths, and — when an AWS call fails — your role ARNs, log group names, and region.

4What we read from your AWS account, and what we do not

We reach your account only by assuming the cross-account role you created, gated by an ExternalId we issue you, and only with short-lived credentials from AWS STS. We hold no long-lived keys into your account.

What we read: EMR virtual-cluster and job-run state; the cluster, node-group, add-on, and access-entry configuration preflight needs; IAM role, trust-policy, and OIDC-provider details for the roles the template names; log events under the SparkPilot log-group prefix; and objects in the job-artifacts bucket you nominate.

What we do not read: your other S3 buckets, your data lake, your Cost Explorer or Organizations data, your Secrets Manager values, or anything the template does not grant. The template is the entire grant, it is published before you deploy it, and deleting the stack revokes all of it.

5Notebooks, job code, and artifacts

There are two places where your own code sits in, or passes through, our systems. We state them plainly, because a policy that skipped them would be wrong.

Notebooks you create in SparkPilot are stored in our database in full, as Python source, because the notebook file is what gets submitted as the Spark script. Treat a SparkPilot notebook as content you have handed us to hold.
When you deploy an artifact through the SparkPilot API, the file passes through the control plane on its way to your own job-artifacts bucket. We do not keep a copy of the bytes; we record its URI and digest.

6When you use the contact form

The form on this site collects your name and work email, and optionally your company, a use case, and a message. We store those together with the IP address the request arrived from and a timestamp, and we email the submission to our team through Resend, our email provider. That notification also includes the page you submitted from, and it is set to reply directly to the address you gave.

A hidden anti-spam field and a signed form token are checked and then discarded — neither is ever stored. Submissions are rate limited by originating IP address, and a repeat submission from the same email and IP address within a short window is rejected rather than stored twice.

If you would rather not use the form, email [email protected] instead.

Contact page

7Payments

Paid tiers are currently billed by invoice. When Stripe card checkout is enabled, you enter card details on Stripe's own hosted pages and manage the subscription in Stripe's customer portal. Card numbers, expiry dates, and security codes never touch SparkPilot's servers, and we store none of them — not even the last four digits.

What we store is Stripe's identifiers for your organisation: the customer id, the subscription id, the subscription status, and the plan tier they map to. Stripe sends us signed webhook events when a payment succeeds, fails, needs action, or is disputed, and we record the resulting plan and entitlement status.

Stripe's own handling of your payment information is governed by Stripe's privacy policy, not this one.

8Cookies, local storage, and analytics

This marketing site sets no cookies. It is a static site with no analytics SDK, no advertising tag, and no third-party font or script hosts — the fonts are served from our own origin.

The signed-in application sets two cookies of its own: a session cookie (HttpOnly, Secure, SameSite=Strict, expiring with your token and no later than eight hours) and a companion CSRF token cookie. Both are strictly necessary to keep you signed in safely. Auth0 sets its own cookies on its own domain while you are signing in.

Both surfaces remember your light or dark preference in your browser's local storage under the key sparkpilot.theme. That value never leaves your browser.

Our domains sit behind Cloudflare, which terminates TLS and can inject its own cookieless Web Analytics beacon at the edge. It reports aggregate page-view and performance data to us. It sets no cookie and does not build a cross-site profile of you.

9How we use the information

We use what we collect for these purposes and no others.

To operate the product: authenticate you, run preflight, dispatch and track runs, show diagnostics and cost, and enforce the roles, policies, and budgets you configure.
To bill you and keep your entitlement accurate.
To keep the service secure and accountable — the audit trail, rate limiting, and investigating abuse.
To support you and answer what you send us.
To improve the product using aggregate operational data such as error rates, failure categories, and feature usage.

11Other disclosures

We also disclose information where the law requires it, where it is needed to establish or defend legal claims, or to protect the rights and safety of people using the service. If SparkPilot is ever acquired or merged, information may transfer to the acquirer, remaining subject to this policy or a successor policy no less protective.

We will update this page before adding a provider that processes personal information on our behalf. A data processing agreement is available: [DATA PROCESSING AGREEMENT — AVAILABILITY AND CONTACT].

12Where your information is stored

Control-plane data is stored in AWS in the US East (N. Virginia) region, in a database encrypted at rest, with credentials held in AWS Secrets Manager. Our own application logs are retained in AWS CloudWatch for 30 days.

If you are outside the United States, using SparkPilot means your control-plane information is processed in the United States. The transfer mechanism we rely on, and any regional representative, are: [INTERNATIONAL TRANSFER MECHANISM], [EU / UK REPRESENTATIVE].

13How long we keep it

We keep information for as long as your account is active and for as long as we need it to run and account for the service. Our specific periods are: account and organisation records [RETENTION — ACCOUNT DATA]; run, job, and cost records [RETENTION — RUN AND COST RECORDS]; audit events [RETENTION — AUDIT EVENTS]; contact-form submissions [RETENTION — CONTACT SUBMISSIONS]. Application logs in AWS CloudWatch are deleted after 30 days by configuration.

You can delete runs, environments, projects, and notebooks yourself in the product. Ask us to close your account and we will delete or de-identify the rest within [ACCOUNT DELETION WINDOW], except where we are required to keep records for tax, accounting, or legal reasons.

Logs and data in your own AWS account are yours. Deleting your SparkPilot account does not delete them.

14Your choices and requests

Email [email protected] and we will act on it. You can ask for a copy of the personal information we hold about you, ask us to correct it, ask us to delete it, or object to a particular use. We will verify who you are before acting, and we will not treat you differently for asking.

If a data-protection law gives you rights over your information, we will honour requests made under it. Our formal position under specific regimes — the legal bases we rely on, our designations, and any appointed representative — is: [DATA-PROTECTION REGIME DETAIL — GDPR / UK GDPR / CCPA].

15How we protect it

Our security posture is set out in full on the security page, and we would rather you read that than a paragraph of adjectives here.

In short: access to your AWS account is by assumed role with a per-tenant ExternalId and no long-lived keys; traffic to SparkPilot is served over TLS; the control-plane database, dispatch queues, secrets, and logs are encrypted at rest with a dedicated AWS KMS customer-managed key; access is enforced by role and scoped to teams and environments; and control-plane actions are recorded as audit events tied to the acting identity.

SparkPilot holds no formal security certification today, and nothing here implies one. No system is perfectly secure — if you find a problem, email [email protected].

Security page

16Automated decisions and AI

SparkPilot makes no automated decision about you that produces a legal or similarly significant effect. Run diagnostics and remediation suggestions are rule-based.

The product contains an optional assistant that would send scoped run and cost metadata to a third-party model provider. It is switched off, and no customer data goes to any model provider today. If we turn it on, we will name the provider on this page before it processes anything of yours.

We do not use your content, code, or notebooks to train machine-learning models.

17Children

SparkPilot is a business product and is not directed at children. We do not knowingly collect information from anyone under 16, and we will delete it if we learn that we have.

18Changes to this policy

We will update this page when our practices change, and we will change the date at the top when we do. If a change is material we will tell your organisation's administrators by email or in the product before it takes effect.

19Contact us

Write to [LEGAL ENTITY NAME], [REGISTERED ADDRESS], or email [email protected] with anything about this policy — including access, correction, and deletion requests.