Let's get out there and wreck some planes
Posted: Sat May 04, 2019 12:40 am
Just kidding. An attempt at humor. And my intent is not to induce more Cirrus-bashing. It's more like journalist-bashing.
Here's what sparked it. last month Aviation Consumer published an article regarding the Cirrus accident record. The conclusion was it has improved a lot over the years. But something in the fatal accident percentage comparing them to the DA40 (33% ???) didn't look right, so I got to looking into it myself. Using what I think was his (Paul Bertorelli) stated methodology, i.e., using NTSB data for US accidents, I found some things I expected but some I did not. Summarily, here's what I found with respect to the percentage of accidents in the US that have been fatal:
1. His overall statement about Cirrus improvement is true, qualitatively.
2. The SR22 fatal accident percentage he came up with of 35% for all time looks accurate to me (76/120).
3. He showed a 41% fatal accident percentage all time for SR20. I conclude about 31% (22/72).
4. He stated 33% fatal accident percentage for the DA40. I find 21-23% (depends on whether to include DA40NG's - 7/34 or 7/31).
Contrary to his conclusion I believe the SR20 and DA40 are doing substantially better in this particular metric than he states, and both far better than the 22.
I went a bit further and did rolling 3 and 5 year averages of this statistic for the 3 types. All 3 show a downward trend (albeit somewhat "noisy"), with the numbers as of the end of 2018 being:
1. SR22 3-year: 26%, 5-year: also 26%.
2. SR20: 3-year: 18%, 5-year: 19%.
3. DA40: 3-year: 13%/9%, 5-year: 9.1%/7.1%
The DA40 numbers are particularly quirky and noisy as there has been only 1 DA40 fatal accident in the US after 2012, in 2016. My conclusion is the SR20 (and its pilots) with respect to this particular metric are both doing substantially better than the SR22. The DA40/pilot population accident record in the US is almost totally off the radar to the point it could be considered statistically useless. 4 of the 7 US DA40 fatal accidents occurred in the period 2009-2012, with the remaining 3 scattered across the remaining 13 years I included (2002-2018). The SR20 has had 4 years of zero fatal accidents scattered across that period, while the SR22 has had no such years in that period.
Why the title of the thread? There is such a disparity between what I came up with vs. Mr. B's is that I conclude that he somehow under-counted non-fatal accidents for both the SR20 and DA40 and/or over-counted fatal accidents. I have identified some quirks in the NTSB data and its search criteria page that could induce this sort of problem. Why this also didn't seem to be true for the SR22 conclusion I'm unsure, but it might have to do with the much higher numbers for the 22 either cancelling stuff out or making it moot. <SARCASM ALERT> So if we could bend up some more DA40's without hurting anyone we'd get more love from A/C next time around
I did come across something else interesting in the data. I counted actual fatalities recorded for each fatal accident. FWIW SR20 fatal accidents have a statistically significant higher likelihood than either the SR22 or DA40 to involve more than 1 fatality in a fatal accident.
Here's what sparked it. last month Aviation Consumer published an article regarding the Cirrus accident record. The conclusion was it has improved a lot over the years. But something in the fatal accident percentage comparing them to the DA40 (33% ???) didn't look right, so I got to looking into it myself. Using what I think was his (Paul Bertorelli) stated methodology, i.e., using NTSB data for US accidents, I found some things I expected but some I did not. Summarily, here's what I found with respect to the percentage of accidents in the US that have been fatal:
1. His overall statement about Cirrus improvement is true, qualitatively.
2. The SR22 fatal accident percentage he came up with of 35% for all time looks accurate to me (76/120).
3. He showed a 41% fatal accident percentage all time for SR20. I conclude about 31% (22/72).
4. He stated 33% fatal accident percentage for the DA40. I find 21-23% (depends on whether to include DA40NG's - 7/34 or 7/31).
Contrary to his conclusion I believe the SR20 and DA40 are doing substantially better in this particular metric than he states, and both far better than the 22.
I went a bit further and did rolling 3 and 5 year averages of this statistic for the 3 types. All 3 show a downward trend (albeit somewhat "noisy"), with the numbers as of the end of 2018 being:
1. SR22 3-year: 26%, 5-year: also 26%.
2. SR20: 3-year: 18%, 5-year: 19%.
3. DA40: 3-year: 13%/9%, 5-year: 9.1%/7.1%
The DA40 numbers are particularly quirky and noisy as there has been only 1 DA40 fatal accident in the US after 2012, in 2016. My conclusion is the SR20 (and its pilots) with respect to this particular metric are both doing substantially better than the SR22. The DA40/pilot population accident record in the US is almost totally off the radar to the point it could be considered statistically useless. 4 of the 7 US DA40 fatal accidents occurred in the period 2009-2012, with the remaining 3 scattered across the remaining 13 years I included (2002-2018). The SR20 has had 4 years of zero fatal accidents scattered across that period, while the SR22 has had no such years in that period.
Why the title of the thread? There is such a disparity between what I came up with vs. Mr. B's is that I conclude that he somehow under-counted non-fatal accidents for both the SR20 and DA40 and/or over-counted fatal accidents. I have identified some quirks in the NTSB data and its search criteria page that could induce this sort of problem. Why this also didn't seem to be true for the SR22 conclusion I'm unsure, but it might have to do with the much higher numbers for the 22 either cancelling stuff out or making it moot. <SARCASM ALERT> So if we could bend up some more DA40's without hurting anyone we'd get more love from A/C next time around
I did come across something else interesting in the data. I counted actual fatalities recorded for each fatal accident. FWIW SR20 fatal accidents have a statistically significant higher likelihood than either the SR22 or DA40 to involve more than 1 fatality in a fatal accident.