Methodology

This page documents exactly how AutoPilotWatch processes, classifies, and presents autonomous vehicle safety data. We believe transparency about our methods is essential for responsible data journalism. If you have questions about our approach, please contact us.

What is SGO Data?

In June 2021, NHTSA issued Standing General Order 2021-01 (SGO), requiring manufacturers and operators of vehicles equipped with Automated Driving Systems (ADS) or Level 2 Advanced Driver Assistance Systems (ADAS) to report certain crashes. This was the first mandatory, standardized crash reporting requirement for AV-equipped vehicles in the United States.

ADS-equipped vehicles must report any crash within 24 hours. ADAS-equipped vehicles must report crashes involving a fatality, injury requiring hospitalization, airbag deployment, pedestrian/cyclist involvement, or a vehicle tow-away.

This reporting asymmetry is the single most important thing to understand about the data: ADS reports everything, ADAS only reports serious crashes. Any analysis that compares raw ADS and ADAS counts without acknowledging this difference is fundamentally misleading.

Data Processing Pipeline

We process the raw SGO data as follows:

1. Ingestion:Raw reports are downloaded from NHTSA's public SGO dataset as structured CSV/JSON files.
2. Parsing: Each report is parsed into structured fields including date, location, vehicle information, system type, and severity.
3. Deduplication: Reports are deduplicated by Report ID, keeping only the highest version number. When a manufacturer submits an updated report for an existing incident, the newer version supersedes the original.
4. Classification:Incidents are classified as ADS or ADAS based on the reporting manufacturer's designation in the SGO filing.
5. Severity assignment: Each incident is classified into one of three severity levels (see below).
6. Aggregation: Statistics are aggregated by manufacturer, vehicle model, state, and time period.
7. Validation: Automated checks verify data consistency — incident counts match across views, fatality totals are consistent, and no duplicate Report IDs remain.

Crash Severity Classification

We classify each incident into one of three severity levels based on fields in the SGO report:

ADS vs ADAS Classification

The distinction between ADS and ADAS is critical:

• ADS (SAE Level 3-5):The vehicle's automated system handles all driving tasks. Examples: Waymo robotaxis, Cruise driverless vehicles, Zoox shuttles.
• ADAS (SAE Level 2): The system assists but requires continuous human supervision. Examples: Tesla Autopilot/FSD, GM Super Cruise, Ford BlueCruise.

We use the manufacturer's own classification from their SGO filing. Note that Tesla's "Full Self-Driving" is classified as ADAS (Level 2), with the recent exception of a small number of ADS-designated reports.

Deduplication Details

NHTSA's SGO system assigns each incident a unique Report ID. Manufacturers may submit multiple versions as more information becomes available. Our pipeline keeps only the highest version, ensuring each physical crash is counted once with the most complete information.

Manufacturer Rankings

Our manufacturer rankings are based on total SGO incident count. We rank by raw count by default because it's the most transparent metric. However, we strongly caution against interpreting raw rankings as safety rankings:

• A manufacturer with more vehicles will naturally report more incidents
• ADS operators report all crashes, inflating their counts relative to ADAS
• Without miles-driven normalization, raw counts are misleading

We provide fatality counts, injury counts, ADS/ADAS splits, and per-vehicle breakdowns for more nuanced analysis.

Consumer Complaint Processing

Consumer complaints are pulled from NHTSA's complaints API and filtered for AV/ADAS-related keywords including: autopilot, self-driving, autonomous, phantom braking, lane departure, FSD, full self driving, driver assist, and related terms. This keyword-based filtering may miss some relevant complaints or include tangentially related ones.

Data Freshness

We update our database on a monthly cadence:

• SGO data: Pulled monthly from NHTSA's published dataset.
• Complaints: Pulled via NHTSA's API, filtered for AV/ADAS keywords.
• Recalls: Pulled via NHTSA's Recalls API.
• Investigations: Pulled from NHTSA's ODI database.
• Safety ratings: Pulled from NHTSA's NCAP API.

How We Score Safety

Our manufacturer safety scorecards combine multiple dimensions:

• Incident classification: ADS and ADAS incidents are tracked separately and never combined for ranking.
• Severity weighting: Fatal incidents carry the highest weight, followed by injury crashes, then property-damage-only.
• Recall & complaint signals: Active recalls, open investigations, and high complaint volumes are supplementary indicators.

We deliberately avoid producing a single "safety rank" number because, without miles-driven normalization, any such ranking would be misleading.

Known Limitations

• Reporting thresholds differ: ADS has a lower reporting threshold than ADAS, making raw count comparisons misleading.
• No exposure normalization: Without miles-driven data per manufacturer, we cannot calculate true crash rates.
• Fleet size varies enormously: Tesla has millions of ADAS-equipped vehicles; some ADS operators have hundreds.
• Self-reported data: Manufacturers self-report incidents. Quality and completeness may vary.
• Reporting lag: Incidents may take weeks or months to appear. Initial reports may lack details.
• CBI redaction: Some fields are redacted as Confidential Business Information.
• Not all AV crashes included: Only crashes meeting SGO thresholds are reported.
• Causation not determined: SGO reports do not establish whether the ADS/ADAS caused the crash.
• Keyword filtering for complaints: Our complaint filtering is keyword-based and may miss relevant reports or include tangential ones.
• Historical coverage: SGO data only goes back to June 2021. Pre-SGO incidents are not in this database.

Interpretation Guidance

When using this data, remember: more incidents does not necessarily mean less safe. A manufacturer with more vehicles on the road will naturally have more reported incidents. Always consider fleet size, reporting thresholds, and miles driven when drawing conclusions. See our glossary for definitions of key terms.

Peer Review & Corrections

We welcome peer review of our methodology. If you identify errors in our data processing, classification, or interpretation, please contact us at info@thedataproject.ai. We take accuracy seriously and will issue corrections promptly. All significant corrections are documented on our changelog.