Driverless cars worse at detecting children and darker-skinned pedestrians say scientists::Researchers call for tighter regulations following major age and race-based discrepancies in AI autonomous systems.
deleted by creator
There’s still a huge racial disparity in tech work forces. For one example, at Google according to their diversity report (page 66), their tech workforce is 4% Black versus 43% White and 50% Asian. Over the past 9 years (since 2014), that’s an increase from 1.5% to 4% for Black tech workers at Google.
There’s also plenty of news and research illuminating bias in trained models, from commercial facial recognition sets trained with >80% White faces to Timnit Gebru being fired from Google’s AI Ethics group for insisting on admitting bias and many more.
I also think it overlooks serious aspects of racial bias to say it’s hard. Certainly, photographic representation of a Black face is going to provide less contrast within the face than for lighter skin. But that’s also ingrained bias. The thing is people (including software engineers) solve tough problems constantly, have to choose which details to focus on, rely on our experiences, and our experience is centered around outselves. Of course racist outcomes and stereotypes are natural, but we can identify the likely harmful outcomes and work to counter them.
I hate all this bias bullshit because it makes the problem bigger than it actually is and passes the wrong idea to the general public.
A pedestrian detection system shouldn’t have as its goal to detect skin tones and different pedestrian sizes equally. There’s no benefit in that. It should do the best it can to reduce the false negative rates of pedestrian detection regardless, and hopefully do better than human drivers in the majority of scenarios. The error rates will be different due to the very nature of the task, and that’s ok.
This is what actually happens during research for the most part, but the media loves to stir some polarization and the public gives their clicks. Pushing for a “reduced bias model” is actually detrimental to the overall performance, because it incentivizes development of models that perform worse in scenarios they could have an edge just to serve an artificial demand for reduced bias.
I think you’re misunderstanding what the article is saying.
You’re correct that it isn’t the job of a system to detect someone’s skin color, and judge those people by it.
But the fact that AVs detect dark skinned people and short people at a lower effectiveness is a reflection of the lack of diversity in the tech staff designing and testing these systems as a whole.
They staff are designing the AVs to safely navigate in a world of people like them, but when the staff are overwhelmingly male, light skinned, young and single, and urban, and in the United States, a lot of considerations don’t even cross their minds.
Will the AVs recognize female pedestrians?
Do the sensors sense light spectrum wide enough to detect dark skinned people?
Will the AVs recognize someone with a walker or in a wheelchair, or some other mobility device?
Toddlers are small and unpredictable.
Bicyclists can fall over at any moment.
Are all these AVs being tested in cities being exposed to all the animals they might encounter in rural areas like sheep, llamas, otters, alligators and other animals who might be in the road?
How well will AVs tested in urban areas fare on twisty mountain roads that suddenly change from multi lane asphalt to narrow twisty dirt roads?
Will they recognize tractors and other farm or industrial vehicles on the road?
Will they recognize something you only encounter in a foreign country like an elephant or an orangutan or a rickshaw? Or what’s it going to do if it comes across that tomato festival in Spain?
Engineering isn’t magical: It’s the result of centuries of experimentation and recorded knowledge of what works and doesn’t work.
Releasing AVs on the entire world without testing them on every little thing they might encounter is just asking for trouble.
What’s required for safe driving without human intelligence is more mind boggling the more you think about it.
But the fact that AVs detect dark skinned people and short people at a lower effectiveness is a reflection of the lack of diversity in the tech staff designing and testing these systems as a whole.
No, it isn’t. Its a product of the fact that dark people are darker and children are smaller. Human drivers have a harder time seeing these individuals too. They literally send less data to the camera sensor. This is why people wear reflective vests for safety at night, and ninjas dress in black.
This is true but tesla and others could compensate for this by spending more time and money training on those form factors, something humans can’t really do. It’s an opportunity for them to prove the superhuman capabilities of their systems.
That doesn’t make it better.
It doesn’t matter why they are bad at detecting X, it should be improved regardless.
Also maybe Lidarr would be a better idea.
They literally send less data to the camera sensor.
So maybe let’s not limit ourselves to using hardware which cannot easily differentiate when there is other hardware, or combinations of hardware, which can do a better job at it?
Humans can’t really get better eyes, but we can use more appropriate hardware in machines to accomplish the task.
That is true. I almost hit a dark guy, wearing black, who was crossing a street at night with no streetlight as I turned into it. Almost gave me a heart attack. It is bad enough almost getting hit, as a white guy, when I cross a street with a streetlight.
These are important questions, but addressing them for each model built independently and optimizing for a low “racial bias” is the wrong approach.
In academia we have reference datasets that serve as standard benchmarks for data driven prediction models like pedestrian detection. The numbers obtained on these datasets are usually the referentials used when comparing different models. By building comprehensive datasets we get models that work well across a multitude of scenarios.
Those are all good questions, but need to be addressed when building such datasets. And whether model M performs X% better to detect people of that skin color is not relevant, as long as the error rate of any skin color is not out of an acceptable rate.
The media has become ridiculously racist, they go out of their way to make every incident appear to be racial now
Isn’t that true for humans as well? I know I find it harder to see children due to the small size and dark skinned people at night due to, you know, low contrast (especially if they are wearing dark clothes).
Human vision be racist and ageist
Ps: but yes, please do improve the algorithms
Part of the children problem is distinguishing between ‘small’ and ‘far away’. Humans seem reasonably good at it, but from what I’ve seen AIs aren’t there yet.
Yeah This probably accounts for 90% of the issue.
Easy solution is to enforce a buddy system. For every black person walking alone at night must accompanied by a white person. /s
DRIVERLESS CARS: We killed them. We killed them all. They’re dead, every single one of them. And not just the pedestmen, but the pedestwomen and the pedestchildren, too. We slaughtered them like animals. We hate them!
Probably could have stopped that headline at the third word.
Maybe if we just, I dunno, funded more mass transit and made it more accessible? Hell, trains are way better at being automated than any single car.
Yes, but also improve kid and dark skin people detection tools, they don’t work just for driving cars. Efficient, fast and accurate people detection and tracking tools can be used in other myriad of stuff.
Imagine a system that tracks the amount of people in different sections of the store, a system that checks the amount of people going in and out of stores to control how many are inside… There’s a lot of tools that already do this, but and they works somewhat reliably, but they can be improved, and the models being developed for cars will then be reused. I+D is a good thing.
An AI that can follow black people around a store? You might be into something.
The trains in California are trash. I’d love to see good ones, but this isn’t even a thought in the heads of those who run things.
Dreaming is nice… But reality sucks, and we need to deal with it. Self driving cars are a wonderful answer, but Tesla, is fucking it up for everyone.
Strongly disagree. Trains are nice everywhere in the world. There’s no reason they can’t be nice in the US. Cars are trash. Strip malls are trash. Giant parking lots are trash. The sky high cost of cars is trash. The environmental impact of cars is trash. The danger of cars is trash. Car centric urban planning is trash.
Self-driving cars are safer… than the most dangerous thing ever. But because cars are inherently so dangerous, they are still more dangerous than just about any other mode of transportation.
Dreaming is nice, but that’s all self-driving cars are right now. I don’t see why we don’t have better dreams.
Trains in California suck because of government dysfunction across all levels. At the municipal level, you can’t build shit because every city is actually an agglomeration of hundreds of tiny municipalities that all squabble with each other. At the regional level, you get NIMBYism that doesn’t want silly things like trains knocking down property values… And these people have a voice, because democracy I guess (despite there being a far larger group of people that would love to have trains). At the state level, you have complete funding mismanagement and project management malfeasance that makes projects both incredibly expensive and developed with no forethought whatsoever (Caltrain has how many at-grade crossings, again?).
This isn’t a train problem, it’s a problem with your piss-poor government. At least crime is down, right?
A single flir camera would help massively. They don’t care about colour or height. Only temperature.
I could make a warm water balloon in the shape of a human and it would stop the car then. Maybe a combination of all various types of technologies? You’d still have to train the model on all various kinds of humans though.
cars should be tested for safety in collisions with children and it should affect their safety rating and taxes. Driverless equipment shouldn’t be allowed on the road until these sorts of issues are resolved.
I’d assume that’s either due to bias in the training set, or poor design choices. The former is already a big problem in facial recognition, and can’t really be fixed unless we update datasets. With the latter, this could be using things like visible light for classification, where the contrast between target and background won’t necessarily be the same for all skin tones and times os day. Cars aren’t limited by DNA to only grow a specific type of eye, and you can still create training data from things like infrared or LIDAR. In either case though, it goes to show how important it is to test for bias in datasets and deal with it before actually deploying anything…
In this case it’s likely partly a signal to noise problem that can’t be mitigated easily. Both children and dark skinned people produce less signal to a camera because they reflect less light. children because they’re smaller, and dark skinned people because their skin tones are darker. This will cause issues in the stereo vision algorithms that are finding objects and getting distance to them. Lidar would solve the issue, but companies don’t want to use it because lidars with a fast enough update rate and high enough resolution for safe highway driving are prohibitively expensive for a passenger vehicle (60k+ for just the sensor)
Darker toned people are harder to detect because they reflect less light. The tiny cheap sensors on cameras do not have enough aperture for lower light detections. It’s not training that’s the problem it’s hardware.
This has been the case with pretty much every single piece of computer-vision software to ever exist…
Darker individuals blend into dark backgrounds better than lighter skinned individuals. Dark backgrounds are more common that light ones, ie; the absence of sufficient light is more common than 24/7 well-lit environments.
Obviously computer vision will struggle more with darker individuals.
Visible light is racist.
If the computer vision model can’t detect edges around a human-shaped object, that’s usually a dataset issue or a sensor (data collection) issue… And it sure as hell isn’t a sensor issue because humans do the task just fine.
Which cars are equipped with human eyes for sensors?
Damn ageist AI!
Racist lighting!
The study only used images and the image recognition system, so this will only be accurate for self driving systems that operate purely on image recognition. The only one that does that currently is Tesla AFAIK.
Weird question, but why does a car need to know if it’s a person or not? Like regardless of if it’s a person or a car or a pole, maybe don’t drive into it?
Is it about predicting whether it’s going to move into your path? Well can’t you just just LIDAR to detect an object moving and predict the path, why does it matter if it’s a person?
Is it about trolley probleming situations so it picks a pole instead of a person if it can’t avoid a crash?
Conant and Ashby’s good regulator theorem in cybernetics says, “Every good regulator of a system must be a model of that system.”
The AI needs an accurate model of a human to predict how humans move. Predicting the path of a human is different than predicting the path of other objects. Humans can stand totally motionless, pivot, run across the street at a red light, suddenly stop, fall over from a heart attack, be curled up or splayed out drunk, slip backwards on some ice, etc. And it would be computationally costly, inaccurate, and pointless to model non-humans in these ways.
I also think trolley problem considerations come into play, but more like normativity in general. The consequences of driving quickly amongst humans is higher than amongst human height trees. I don’t mind if a car drives at a normal speed on a tree lined street, but it should slow down on a street lined with playing children who could jump out at anytime.
Anyone who quotes Ashby et al gets an upvote from me! I’m always so excited to see cybernetic thinking in the wild.
Im not expert, but perhaps thermal camera + lidar sensor could help.
It’s amazing Elon hasn’t figured this out. Then again, Steve Jobs said no iPhone would ever have an OLED screen.
We should just assume CEO’s are stupid at this point. Seriously. It’s a very common trend we all keep seeing. If they prove otherwise, then that’s great! But let’s start them at “dumbass” and move forward from there.
