Page 1 of 1

Let's get out there and wreck some planes

Posted: Sat May 04, 2019 12:40 am
by Rich
Just kidding. An attempt at humor. And my intent is not to induce more Cirrus-bashing. It's more like journalist-bashing. 8-)

Here's what sparked it. last month Aviation Consumer published an article regarding the Cirrus accident record. The conclusion was it has improved a lot over the years. But something in the fatal accident percentage comparing them to the DA40 (33% ???) didn't look right, so I got to looking into it myself. Using what I think was his (Paul Bertorelli) stated methodology, i.e., using NTSB data for US accidents, I found some things I expected but some I did not. Summarily, here's what I found with respect to the percentage of accidents in the US that have been fatal:

1. His overall statement about Cirrus improvement is true, qualitatively.
2. The SR22 fatal accident percentage he came up with of 35% for all time looks accurate to me (76/120).
3. He showed a 41% fatal accident percentage all time for SR20. I conclude about 31% (22/72).
4. He stated 33% fatal accident percentage for the DA40. I find 21-23% (depends on whether to include DA40NG's - 7/34 or 7/31).

Contrary to his conclusion I believe the SR20 and DA40 are doing substantially better in this particular metric than he states, and both far better than the 22.

I went a bit further and did rolling 3 and 5 year averages of this statistic for the 3 types. All 3 show a downward trend (albeit somewhat "noisy"), with the numbers as of the end of 2018 being:

1. SR22 3-year: 26%, 5-year: also 26%.
2. SR20: 3-year: 18%, 5-year: 19%.
3. DA40: 3-year: 13%/9%, 5-year: 9.1%/7.1%

The DA40 numbers are particularly quirky and noisy as there has been only 1 DA40 fatal accident in the US after 2012, in 2016. My conclusion is the SR20 (and its pilots) with respect to this particular metric are both doing substantially better than the SR22. The DA40/pilot population accident record in the US is almost totally off the radar to the point it could be considered statistically useless. 4 of the 7 US DA40 fatal accidents occurred in the period 2009-2012, with the remaining 3 scattered across the remaining 13 years I included (2002-2018). The SR20 has had 4 years of zero fatal accidents scattered across that period, while the SR22 has had no such years in that period.

Why the title of the thread? There is such a disparity between what I came up with vs. Mr. B's is that I conclude that he somehow under-counted non-fatal accidents for both the SR20 and DA40 and/or over-counted fatal accidents. I have identified some quirks in the NTSB data and its search criteria page that could induce this sort of problem. Why this also didn't seem to be true for the SR22 conclusion I'm unsure, but it might have to do with the much higher numbers for the 22 either cancelling stuff out or making it moot. <SARCASM ALERT> So if we could bend up some more DA40's without hurting anyone we'd get more love from A/C next time around :D

I did come across something else interesting in the data. I counted actual fatalities recorded for each fatal accident. FWIW SR20 fatal accidents have a statistically significant higher likelihood than either the SR22 or DA40 to involve more than 1 fatality in a fatal accident.

Re: Let's get out there and wreck some planes

Posted: Sat May 04, 2019 1:46 am
by Steve
Rich:

You should forward your data to Bertorelli, perhaps as a letter to the editor of AC. He will either print it with a response, or ignore it. Either way, it will provide information about the accuracy of his analysis.

Steve

Re: Let's get out there and wreck some planes

Posted: Sat May 04, 2019 2:12 am
by Rich
Steve wrote: Sat May 04, 2019 1:46 am Rich:

You should forward your data to Bertorelli, perhaps as a letter to the editor of AC. He will either print it with a response, or ignore it. Either way, it will provide information about the accuracy of his analysis.

Steve
I've gone back and forth whether to bother. It's not that his article is biased and overall seems like good information. This one metric just seemed way off and I now am sure that it is. FWIW, in the past, he has often understated the already low numbers in the DA40 accident record.

I haven't tried to assess the equivalent numbers for several other types he had in his comparison. They may or may not be accurate. But doing this for Cessna 172s and such widely common types with vast numbers of years of data is way more than I'm ambitious enough to tackle.

Re: Let's get out there and wreck some planes

Posted: Sat May 04, 2019 7:14 pm
by Keith M
Statistics about the relative safety of SR22s are all very well, but you do get a better choice of colors compared with our DA40s!
SR22LR.jpg
:D

Re: Let's get out there and wreck some planes

Posted: Mon May 06, 2019 8:01 pm
by Don
Here was the safety record from Aviation Consumer, back in 2012 when all North American DA40's were gas powered.
Total accidents on the left. Fatal accidents on the right.
I once herd someone refer to the DA40 as, "The Volvo of the Sky".
GA Safety Record by Aircraft type..jpg

Re: Let's get out there and wreck some planes

Posted: Tue May 07, 2019 8:23 pm
by Rich
Don, which issue was this in?

Re: Let's get out there and wreck some planes

Posted: Wed May 08, 2019 4:13 pm
by Don
Rich wrote: Tue May 07, 2019 8:23 pm Don, which issue was this in?
January, 2012 Rich.

Re: Let's get out there and wreck some planes

Posted: Wed May 08, 2019 11:18 pm
by Rich
Thanks, Don.

I'm trying to deduce methodology in the past and present. Digging into the long-ago article clarified some things, but is still befuddling. A January Issue was probably composed sometime in late 2011, so data would have been incomplete for 2011 in that issue. There's an implication that NTSB data was the basis for accident counts. But here are a few weird things:

1. If you include all available NTSB data for the DA40 through 2011 and do not filter to US accidents alone, you do get about 30% a fatal accident percentage at that point in time.
2. But conversely, the SRxx data, while clearly much higher than the DA40, do not produce percentages as high as the 2012 graph shows.
3. Where they get DA42 fatalities I have little clue. There is not a single fatal DA42 accident to this day in the NTSB data, though there have been quite a few appearing elsewhere (such as ASN WIKI) dating back as early as 2006.
4. The article alludes to being informed by Diamond about a fire in a DA20 crash where the plane hit power lines, but the author states none could be found in the NTSB database. But it's there, big as life - 9/21/2005, in Texas.

Re: Let's get out there and wreck some planes

Posted: Thu May 09, 2019 2:00 pm
by CFIDave
Rich wrote: Wed May 08, 2019 11:18 pm 3. Where they get DA42 fatalities I have little clue. There is not a single fatal DA42 accident to this day in the NTSB data, though there have been quite a few appearing elsewhere (such as ASN WIKI) dating back as early as 2006.
Yes, no fatalities among Diamond twins in North America, although there have been some unusual fatalities of DA42s in Europe including aircraft being shot down and a suicide. In North America, the only accident (non-fatal) that I can recall is a pair of CFIs who stalled a DA42 too high over the runway at KLNS and collapsed the gear. My understanding is that plane is flying again.