HN Reader

I'm surprised the "Actual vehicle tested may be different" section didn't mention the alleged fraud Ford engaged in with the 2015 F150 where they welded in extra crash safety bars only on specific configurations out of the factory... that just happened to be the exact configurations that were being sent to the IIHS for crash testing.

I can offer two takes on the IIHS:

(1) A remarkable example of a private agency that has pushed safety standards beyond what the government would do own its own, and

(2) An organization that has persuaded Americans to buy larger vehicles than they would have otherwise with all the associated costs (e.g. the “affordable car” crisis) and risks (to pedestrians.)

The IIHS is an organization of insurers so they are particularly concerned about quantifiable monetary costs. And when it comes to that much more of the benefit of larger vehicles is in avoided minor injuries such as broken bones which are more common than death and life changing injuries. The public focuses on the latter and the psychology is such that some people will spend another $50k on some German vehicle and spend the rest of their days at the dealer getting it fixed or subject their children to the trauma of riding in a minivan. (To generation X the minivan is like the toxic PFAS GenX)

IIHS claims that compatibility has improved between large and small vehicles but that large vehicles are still a menace to other road users

https://www.iihs.org/topics/vehicle-size-and-weight

Tesla aced another independent out-of-sample crash test [1].

The danluu article also mentions an iSeeCars report at the bottom that says that Teslas have a high amount of fatalities per mile driven [2]. However, while they claim to be using official US estimates of fatalities data, they normalized it with estimated mileage using unknown proprietary iSeeCars data:

> iSeeCars analyzed fatality data from the U.S. Fatality Analysis Reporting System (FARS). Only cars from model years 2018-2022 in crashes that resulted in occupant fatalities between 2017 and 2022 (the latest year data was available) were included in the analysis. To adjust for exposure, the number of cars involved in a fatal crash were normalized by the total number of vehicle miles driven, which was estimated from iSeeCars’ data of over 8 million vehicles on the road in 2022 from model years 2018-2022. Heavy-duty trucks and vans, models not in production as of the 2024 model year, and low-volume models were removed from further analysis.

Some numbers are also surprising, such as the Model Y having almost twice the fatality rate (10.6 per billion miles) as the Tesla average (5.6) --- suggesting that there's a wide gulf between it and the Model 3 in terms of fatality rates, which seems difficult to explain. Ultimately, iSeeCars is a small VC-funded startup with very few people and so it is unclear if their methodology is actually good or not.

For a better blog post about whether Teslas are safe, here's a post from Brandon Paddock [3] from a year ago, which seems more or less objective and unsurprising. Here is the conclusion that he drew upon analyzing the FARS data:

> Tesla’s fatal accident rate is nearly identical to that of the Audi A4 series, and far lower than a standard Ford mid-size sedan. In this case, the Ford’s accident rate is more than 4 times higher than the Tesla Model 3.

[1] https://www.teslarati.com/tesla-model-3-aces-safety-tests-ch...

[2] https://www.iseecars.com/most-dangerous-cars-study

[3] https://brandonpaddock.substack.com/p/are-teslas-the-most-or...

An excellent overview, one thing I'd like to take issue with is this:

    We can also observe that,
    in the IIHS analysis, many
    factors that one might want
    to control for aren't [...]
    One way to see that the numbers
    are heavily influenced by
    confounding factors is by
    looking at AWD or 4WD vs. 2WD
    versions of cars. They often
    have wildly different fatalty
    rates even though the safety
    differences are not very large 
    (and the difference is often
    in favor of the 2WD vehicle).
    Some plausible causes of that
    are random noise, differences
    in who buys different versions
    of the same vehicle, and
    differences in how the vehicle
    are used.

I have no idea why that is either, but the author is surprising confident in declaring AWD a confounding variable, especially after the earlier "Actual vehicle tested may be different" point they made.

A typical AWD will have a transfer case, front driveshaft and other front drivetrain components v.s. a sister car in a 2WD configuration. The overall weight and distribution will be different.

One might e.g. suspect that in a crash the front driveshaft might fold upwards and assist in structurally compromising the passenger cabin.

Spending a little time in Japan, it is hard not to see the general difference in size of vehicle compared to what I'm used to seeing in the US. I'm incredibly curious to know if other nations have similarly small cars compared to the states?

I'm also very curious on other impacts the vehicle sizes have on things. Easy to think it contributes to so many cyclist with basically no helmets. Curious if data backs that.

>Checking to see if crash tests are being gamed with hyper-specific optimizations isn't really feasible for someone who isn't a billionaire.

hmm, operation "Trick Elon into Wasting His Money" has published its latest trick!

From the "Bonus: reputation" section:

> As we've seen over the past few years, loudly proclaiming something, regardless of whether or not it's true, even when there's incontrovertible evidence that it's untrue, seems to not only work, that kind of bombastic rhetoric appears to attract superfans who will aggressively defend the brand.

What does by Dann Luu mean by "loudly proclaiming something, regardless of truth, [seems to work]"? Let's try to make the claim more precise and testable.

Since I'm interested in the broader claim, let me "carve out" (put aside) Dan's specific point about attracting superfans, which I find persuasive. With that said, here are some attempts to reframe the broad claim:

Reframing #1: "Some people are persuaded of false things when a company loudly proclaims them." Yes, this seems obvious. But it isn't particularly useful in guiding action. The claim doesn't say anything quantitative.

Reframing #2: "On net, the accuracy of a population's belief goes down when a company loudly proclaims a false belief." Maybe, maybe not. I would expect it largely depends on what happens next; i.e. whether there is some kind of response. Here are two kinds of reactions that favor not lying. First, getting caught in a lie can damage brand reputation. Second, lying calls more attention to an issue which would have otherwise drifted out of public awareness as the news cycle churns. These depend on how much credibility people assign to the organization and their claims.

Reframing #3: "On net, it is in an organization's self-interest to loudly proclaim a falsehood about its quality." Maybe. It depends on many factors. This is the most interesting public policy question! If such a tactic works (see reframing #2 above), it is clearly a negative externality (a downstream effect than an actor does not feel directly).

How does wise public policy address externalities? One classic way is to internalize the cost -- by making the actor feel the pain themself. In #3 above, one way to internalize the externality would be to impose some penalty or cost for lying. How to do this in a targeted, effective, non-delayed, and legal way is non-obvious to me.

By non-delayed I mean temporally-immediate consequences provide better signal to correct a decision. This isn't just human nature; it is fundamental to the credit-assignment problem in reinforcement learning.

The problem of delayed feedback goes deeper than just credit-assignment; it has to do with fairness too. Often, the decision-maker isn't a monolith: the leadership composition changes over time. It may be impractical to punish one cog in the system without imposing a cost on the rest of the system, many of whom had nothing to do with the decision.

The 2024 update in this post (which claims the worst brands are Tesla, Kia, Buick, Dodge, and then Hyundai, in that order) uses data from an iSeeCars blog post[1], which claims to have gotten their data from the U.S. Fatality Analysis Reporting System[2]. There's one important detail they mention in the blog post:

> To adjust for exposure, the number of cars involved in a fatal crash were normalized by the total number of vehicle miles driven, which was estimated from iSeeCars’ data of over 8 million vehicles on the road in 2022 from model years 2018-2022.

Note that the mileage data for each model comes from iSeeCars, not FARS. We don't know how accurate it is. There could be selection effects because iSeeCars only gets odometer data for cars that are for sale or have been sold, not vehicles that people keep. If for example, someone reserved a Model Y and sold it above the original price as soon as they got it (a common practice for new Tesla models), and the second owner kept the car indefinitely, iSeeCars would only see the odometer reading at the early sale. Tesla mileage would also skew low in sales data for anyone who bought an electric car, then regretted it for one reason or another, and exchanged it for a gas car. And since the rate they are measuring is fatalities per vehicle mile traveled, not per passenger mile traveled, any model that tends to contain more people will skew higher in the stats.

If you query FARS data to get all Teslas involved in occupant fatalities from 2018-2022, there are 141.[3] If you split out the data by model, there were a total of 22 Model Ys involved in occupant fatalities from 2018-2022 (actually 2020-2022, as the Model Y was released in 2020). If there were 10.6 Model Ys involved per billion miles, then (according to iSeeCars) the total number of miles traveled by Model Ys was 2.07 billion. By the end of 2022, the US had approximately 484,100 Model Ys on the road.[4] If you divide the total vehicle miles traveled by the number of vehicles, then the average Model Y had less than 4,300 miles on its odometer. That seems suspiciously low, as Tesla's 2021 report put the average Model Y at 13,000-14,000 miles per year.[5]

This artifact in mis-measuring mileage would also explain why other recently released models seem to have high fatality rates. The Honda CRV Hybrid was released in 2020. The Buick Encore GX was released in 2020. US production of the Hyundai Venue didn't ramp up until 2020. The Toyota Corolla Hybrid was released in 2019. Most people who buy a new car either sell it quickly (because they dislike it and want something else), or they keep it until it gets old. So anyone collecting milage data from car sales will have data that is skewed low for new models because the higher mileage sales haven't happened yet.

It may be the case that I'm using different reports than iSeeCars did. It's hard to know whether they're looking at vehicles involved in fatal crashes, drivers involved in fatal crashes, drivers killed in fatal crashes, occupants involved in fatal crashes, or occupants killed in fatal crashes. FARS lets you query all of these, and the iSeeCars post doesn't make it clear which data they used.

Another odd thing about the iSeeCars data is that they claim an average of 2.8 vehicles involved in a fatal accident for every billion miles traveled. But in 2022, the US had 13.3 deaths per billion miles traveled by car. Unless the average accident involves 5 fatalities, it seems to me that the iSeeCars data has some significant issues.

1. https://www.iseecars.com/most-dangerous-cars-study#v=2024

2. https://www.nhtsa.gov/research-data/fatality-analysis-report...

3. https://cdan.dot.gov/query I don't know how long they retain generated reports for, but mine was available at https://cdan.dot.gov/files/files/b14dcfb4-1f45-4eeb-abda-744... when I made this comment.

4. https://en.wikipedia.org/wiki/Tesla_Model_Y#Sales states 172,700 sold in 2021 and 231,400 in 2022. Other estimates put around 80,000 sold in 2020, for a total of 484,100.

5. See page 79 of https://www.tesla.com/ns_videos/2021-tesla-impact-report.pdf

I haven't seen anyone talk about how the fleet of cars on US roads is now older than ever. Approximately 12 years now. That's 12 years of safety engineering improvements that aren't there. I am not advocating for government handouts or another "cash for clunkers" program to get them off the roads. But I think it's something that people should consider when shopping for a used vehicle.

I wish car testing was less about the safety of the people inside of the car and more about the safety of the people outside of it, like pedestrians, cyclists and other vehicles.

After all, people buying a car already have a strong incentive to purchase something that is safe for themselves, but how many people put any thought into reducing the risk that the force upon others?

For context, pedestrian and cycling deaths have increased for the past decade in North America [0] and it is known that tall blunt hoods increase fatalities [1]. Yet, nothing is being done about it in NA as far as I know.

[0] https://www.npr.org/2023/06/26/1184034017/us-pedestrian-deat...

[1] https://youtube.com/watch?v=YpuX-5E7xoU