How to evaluate
an aviation safety platform.

1. Does it carry one operational picture, or are modules siloed? Ask the vendor to take a single occurrence and trace it through to a quality finding, a corrective action, and a training record. If the demo requires copy-paste between modules, you have a four-tool problem disguised as one tool.
2. Does the AI give the same answer twice? Re-run the AI classification on the same occurrence three times in front of you. Different output = no audit defence three years later.
3. Where does my data live, and who can see it? Three sub-questions: isolation enforced at the database (not the app)? data ever used to train an AI model? export to JSON or CSV in 24h on request, no friction?
4. How long is onboarding actually?If the plan starts at "kickoff workshop" and ends at "go live" with three boxes in between, push back. Real onboarding has weekly milestones with named deliverables.
5. What happens to a finding once it's "closed"? A checkbox is not a closure. Effectiveness verification is the gate the regulator audits you against — your platform should enforce it structurally.

The deepest architectural divide in aviation safety platforms is whether they treat records as connected nodes in one operational picture, or as isolated rows in per-module databases. The first lets a Safety Manager open an occurrence and see the related audit findings, the training competency that lapsed, and the corrective action in flight — without an integration project. The second produces a four-tool stack where every cross-module question becomes a manual reconciliation.

The verification test: ask the vendor to show you a finding from their QMS module that links back to the SMS occurrence that created it, and forward to the training record that closed the loop. If the demo requires a copy-paste between modules, you have a four-tool problem disguised as one tool.

The audit-replay scenario: in three years, the regulator opens an old occurrence and asks why your team classified it the way they did. If the AI gives a different answer on that re-run than it did at the time, you cannot defend the original decision — and your audit trail is broken.

The verification test:ask the vendor to re-run the AI classification on the same occurrence three times in front of you. If the output differs even slightly, the AI is not designed for an aviation safety audit — you would have a compliance problem you didn't know you were buying.

The right posture: AI calibrated against your enumerated taxonomy, outputs that are consistent and reviewable, every run replayable, every output citing the source records it referenced, and a human always validating before the result becomes part of the operational picture.

The aviation safety platform you choose holds your most sensitive operational data — the occurrences your crew reports, the audit findings, the training non-compliances, the regulator submissions. Three questions, three yes/no answers:

• Are my records isolated from every other operator on the platform — and is that isolation enforced by the platform itself, not just the marketing slide?
• Will my organisation's data ever be used to train an AI model — for you, for any provider?
• Can I export everything as JSON or CSV in 24 hours, on request, with no friction?

Weasel words on any of those three are red flags. "We have controls in place" is not the same as "enforced at the database." "Generally we don't" is not the same as "never."

ICAO-aligned should mean a platform that ships with the actual taxonomies — ICAO occurrence categories enumerated, the standard 5×5 risk matrix configurable to your organisation, the Annex 17 threat catalogue, the Annex 19 SMS pillar structure. It should NOT mean a marketing badge with no traceability into the data model.

The verification test:ask which Annexes they cite, in what current version, and how they handle amendments. A platform that handles ICAO Annex 17 amendments as a structural change-impact analysis (each amendment ripples through dependent requirements, policies, training records, and evidence) is a different product from one that put "Annex 17 ready" on a slide.

Ask for a written onboarding plan with weekly milestones and named deliverables. If the plan starts at "kickoff workshop" and ends at "go live" with three boxes in between, push back. A real plan looks like this:

• Week 01: tenant provisioned · data model mapped to your operation
• Week 02: ICAO + your taxonomies live · risk matrix calibrated
• Week 04: legacy occurrences imported with AI classification applied + locked on the way in
• Week 06: workflows + SLA clocks + owner/verifier rules dialled in for your org structure
• Week 08: team running the platform daily · weekly cadence moves to monthly

If a vendor needs 12 months, ask what the other 10 months cover. Usually it's their data-model design happening on your time and budget.

A corrective action is not closed when implementation finishes. It is closed when a named verifier confirms — with evidence — that the action actually reduced the risk it was designed to address. EASA Part-ORO and FAA Part 5 both make effectiveness verification mandatory. Many platforms treat it as optional metadata.

The verification test:in the demo tenant, try to close a corrective action without the effectiveness verification step. The system should refuse. If it lets you through "just for the demo", the platform doesn't enforce the regulation it claims to support.

Three pricing patterns to scrutinise:

• Per-occurrence pricing. Incentivises under-reporting. Just Culture and an open reporting system are the lifeblood of an SMS — you do not want a price tag attached to each report a crew member files.
• Per-record-type modules sold separately.If the "SMS license" doesn't include hazards, or the "QMS license" doesn't include findings, you're paying twice for the same operational graph.
• 6-month implementation fees.A long pre-paid implementation period covers the vendor's time-to-value risk, not yours. Healthier models are short pilots with monthly subscriptions and no lock-in.

Healthier patterns: per-tenant flat, per-tail-aircraft, or per-employee with unlimited reports.

Six red flags to watch in vendor demos:

1. "We can do that, we'll show you next call." (Translation: it doesn't exist yet.)
2. Demo data so polished it's clearly hand-built. (Reality is messier; their UI may not handle it.)
3. Vague answers about data residency. (You need a specific region and a specific contractual clause.)
4. No published roadmap. (Where is the product going? You're committing for 3+ years.)
5. Cannot show you a real customer's tenant, even sanitised. (Either no customers, or customers won't consent.)
6. Cannot quote a price without 5 follow-up calls. (Pricing opacity scales with future negotiation pain.)

Exit options:ask for the data export contract clause BEFORE you sign — what format, what timeline, what cost. If the answer is "we'll work it out", walk away. The clause needs to be specific: JSON or CSV, complete tenant export, within 24-48h on request, at no marginal cost during contract or for 12 months after.

Run each shortlisted vendor through these four:

1. The same real occurrence. File it, classify it, link it to a related finding, create a corrective action, verify effectiveness, close. End-to-end. Note how many tools you touched.
2. The same regulator.Load your local civil aviation authority's mandatory reporting requirements. See whose system encodes them as structured rules (not just documents).
3. The same Friday-afternoon edge case.A pilot files an occurrence at 16:55 on Friday; the safety manager is on a four-day weekend. What does the system do? (SLA clock starts ticking, escalation tier 2 triggers Saturday, accountable manager auto-CC'd Sunday morning — that's a real workflow.)
4. The same export. Request your tenant data back in CSV + JSON within 24h. See who delivers cleanly and who stalls.

If a vendor passes all four, you have a shortlist of one.