My husband is a large man with a serious expression. People sometimes cross the street when he approaches.
One day, he was walking down the street when he saw what he perceived to be a dangerous character.
They both chose to cross the street at the same time to avoid the other.
Eyes may be the mirrors of the soul, but the shapes of our features do not indicate our character.
23
@Bridget - similar problem. I'm big, and I've had people ask me "what's wrong" or "what are you mad about", out of the blue. The explanation I finally was able to give people is that the expression is simply what my face does when I'm not using it.
The alternative, if they don't accept the explanation and keep asking (why does that happen), is that I'm angry because someone doesn't believe what I'm telling them.
17
This sounds potentially really dangerous, especially in the hands of an authoritarian. Given that the US government at the highest level now rejects real science I would be very concerned where this can lead.
31
So many misguided comments here it’s difficult to know where to begin.
We live in the safest time in human history. That we must live in constant fear of so-called criminals is a false narrative, largely perpetuated by Trump and the Republicans to move us to exactly the type of police state being advocated by many of those commenting here.
Misidentification is a real concern in a surveillance state, especially for minorities who already must bear the burden of police harassment.
And by the way, exactly what is illicit or even criminal behavior? We are living in a time when an increasingly unaccountable chief executive believes he should be able to make the rules as he sees fit. What is legal today could be criminal tomorrow. Ask those living in China. As we move away from a country governed by laws to one governed by men, the surveillance state will be able to intimidate just by indicating something that is now legal could be illegal in the future, and those videos of YOU will remain on government servers forever.
Wake up people. We are NOT living at risk of being impacted by criminals but we ARE at increasing risk of losing our civil liberties. Read the history of the Third Reich if you need to see the path we are on and the outcome.
95
crime not a problem? I say, depends on your zip code among other things. Some streets have a shooting every week and in others people do not know what gunfire sounds like.
4
Note that the "2016 paper" that was uploaded to arXiv is not peer-reviewed and has not been published in any journal.
My father, Henry Eisenman, was a survivor of Auschwitz. Like millions of other Jews, he lost most of his family due to the insanity of "racial purity."
He told me this story. A while after he was liberated, a German doctor saw him for a check-up. When the customary medical exam was over, he looked at my dad with curiosity and asked whether he could take some "measurements." My dad agreed, and out came the calipers.
Every aspect of my father's head shape, eyes and nose, and the ratios of all these to each other, were measured. When the doctor was finished, he proclaimed to my father, with no small sense of irony and almost laughing: "You'll be happy to know you have a perfectly shaped Aryan head. Congratulations."
There's only one race, and we're all that race: Homo sapiens. And the idea that the newest A.I. technology should be implemented AGAIN to further a pseudo science about faces being predictive of criminality is a scary thought.
Mark Eisenman, Toronto
4
Obvious rejoinder to arguments for phrenology and physiognomy: Ted Bundy and any good con man!
1
It's hard to say which is worse junk "science" or junk "research", this article or the 19th century physiognomists the article relies on to tell us about modern day face recognition researchers. If the "Privacy Project" wasn't overloaded with lawyers and journalists, at the expense of people who have actually worked with and written pattern recognition algorithms, it wouldn't probably merit such harsh criticism. But going all the way out on the furthest limb in a gotta-go-viral reverie through history with tenuous connections like citing Chinese face recognition attempts at "criminal faces" and "debunking" them using talking about the U.S. justice system.
I worked in face recognition and other types of pattern recognition for the better part of a decade. Not once did anyone avail themselves of Nazi eugenicists or 19th century physiognomists or the theories they peddled.
What your article shows is that idiot businesspeople will buy any pseudoscientific garbage to try to completely eliminate risk when hiring the people they want to be totally loyal and subservient but also to show no loyalty to in return. What it shows is that Stanford researchers will publish just about anything to try to put their name in public, much like MIT's media lab of old.
And it shows that journalists, lawyers, and psychologists are groups of people that, unlike real scientific researchers, will gladly go into someone else's discipline and opine without understanding it, to feign expertise.
We have always been obsessed with how people look. How do white suprematists look. Is it better not just to engage than find some artificial formula for sorting human character. What kind of character does a person who creates these programs have?
"AI" is perfectible technology entirely subject to human biases and misuse, it's not magic.
Evoking AI here as so many do as a "call to authority" argument only reveals the same level of reasoning that gives power to voodoo dolls. Anyone who does it is either a manipulator or a dupe. Which are you?
And let us not forget the forgettable Szondi Test, which purported to be able to determine if you were a psychopath, or a psychotic, based on your liking of various faces. Of course it was complete trash.
But, nothing dies, it is still available on-line.
1
The author conveniently mentions Galton and Lombrosso, whose work has long been criticized, but omits Sheldon and Kretchmer, in more recent times, whose studies are still valid and interesting. Sheldon's theory of personality is still listed in books detailing theories of personality, and although it is very difficult to apply, it is still a valid view of personality based on somatic differences. There is nothing sensational about it, just hard analytical work. The Editors should have been suspicious of an article that omits a lot of the data amassed by Sheldon and Kretchmer. Admitedly, Kretchmer is less known in America, but his theories are nonetheless valid. Personalities are not easy things to determine, and we need all the data we can get.
The author is not conflating facial recognition with anything, but establishing the history of misuse physically identifiable features with human behaviour, and risks involved which play to scientifically proven age old human bias.
You don't need AI to know mischaracterising an argument then claiming misspeaking is a tell for someone with an agenda. In this case, it's most likely seeking funding for a research or development project, if it's not straight up tr trying to sell one of these bogus systems...
1
This article misses the current abuse of facial recognition in China for surveillance for a "social credit score" and to suppress minorities. These systems aren't even close to 95% accuracy in identification so to suppose they can evaluate character, criminality, sexual orientation or emotions is ridiculous.
1
In urban cities in China, if a man or woman jaywalk, cameras identify the miscreant and automatically deducts the fine from their bank accounts. No hearing, no appeal. It happens in minutes.
2
The more often you are correctly identified by facial recognition, the less likely that it will make a mistake. I went through customs of two countries and both used facial recognition to identify me. In both cases, after the machine determined that I am who I say I am, then I went to the human that stamped my passport.
As for using facial recognition to determine character or personality, complete nonsense. Humans can find patterns in known random data, no doubt so can computers. Finding a temporary pattern is not a reliable indicator of anything.
5
The Italian anthropological criminologist Cesare Lombroso (1800s) was a pseudo scientific racist. (One predictive indicator of criminal intent was the slope of the forehead). My experience as a trial investigator in criminal cases leads me to suspect that the vast unreliability of eyewitnesses to interracial crime is also driven by an implicit belief in anthropological criminology. Some uncertain eyewitnesses pick a photo from a lineup based on skull characteristics, posture, etc. They won’t admit that implicit racial bias. Dr. Liz Loftus (UC Irvine) has written widely about eyewitness identification issues.
6
P.S. Does Hollywood’s typecasting of villains ‘fitting a certain appearance’ influence us as a society of potential witnesses to incriminate innocent people or to convict, as jurors, innocent defendants. Woe to the criminal defendant who could be cast in roles of bad guys in movies and television. Woe to the innocent defendant that looks like Hannibal Lecter, Agent Smith, Norman Bates, Nurse Ratched, Keyser Soze, Noah Cross (Chinatown). Better to look like Opie grown up (Ron Howard).
4
Face it. We are falling fast into Dystopia where junk technology supplants junk science at our altar of progress.
3
Committing to memory features "exceedingly strong in amativeness" -- who knew? -- and amused that in 2019 we're still seeing articles on phrenology. Looking forward to next week's article on efficacy of snake oil.
2
The book Mismeasure of Man by the late Stephen J Gould, published in 1981 , gives a detailed and accessible account of the science and the social and political contexts of eugenics.
Worth a read in these times...
3
On the other hand, human-designed faces can be almost a fool-proof way of detecting villainy. Refer to Disney and others.
1
The author of this article is using a trick- conflating two completely different activities (using facial recognition to identify individuals, and using similar technologies to make value judgements about groups of people). the former is a legit technique, the latter is not. Conflating them intentionally like this is trying to convince people the former is wrong. I note from the comments that most people have picked up on this rhetorical trick,
3
The author is not conflating facial recognition with anything, but establishing the history of misuse physically identifiable features with human behaviour, and risks involved which play to scientifically proven age old human bias.
You don't need AI to know mischaracterising/misspeaking is a tell for someone with an agenda. Seeking funding for a project? Or trying to sell one of these bogus systems?
3
Eyewitnesses to crimes usually try hard to correctly identify an offender in police photo and in-person line-ups. Oftentimes, they misidentify a suspect and an innocent person is convicted. Sometimes the lineup is tainted by detective malfeasance, more subtly, implicit biases in the eyewitness influence the process. Those implicit biases are frequently grounded in the eyewitness’ tacit acceptance of anthropological criminology where facial characteristics are indicators of guilt. Maybe the skull structure validates the misidentification as a confirmation bias. There are subconscious heuristics leading the eyewitness to incriminate an innocent man or woman. As a criminal trial investigator over ten years, I observed this often in vetting eyewitnesses before trial. Dr. Elizabeth Loftus at UC Irvine is a respected expert, has written often about her research.
She is not among the rogue pseudo-science criminologists (like Cesare Lombroso) that misuse junk science to condemn classes of people because of their forehead shape.
3
If we put a cop by the entrance to a stadium and he recognizes a man from a wanted poster--how is this different from having an electronic device recognize him? Even if there is an error, we wouldn't accept a legally consequential recognition based only on a cop's recollection of a poster or an algorithm. Is it an invasion of privacy to be recognized in a public place?
An invasion of privacy problem may begin after recognition, if a company (most likely) or the government sets about recording legal movement or behavior.
3
He ‘thinks’ he recognizes...
This article has a lot of conclusions without a lot of backup. I was waiting for the studies that used AI to debunk these claims that personality and physiology are related. But seems we don’t have them yet because none are cited.
6
@Mmm Why do you need AI to debunk the claims? Research is addressed. Further, I think you are asking that the researchers prove a negative when it should be the other way around.
7
Two different ideas presented here: the ridiculous notion that behavior can be “predicted” by facial recognition and the legitimate use of facial recognition for “identification” of known criminals. As with any identification or surveillance system, there are legitimate privacy concerns for law abiding citizens. Having to present identification before voting, for example, is a reasonable invasion of privacy if one wishes to prevent those not eligible from voting. The same is true of firearms registration and driving privileges. The constant fear is that the government will abridge our rights in the name of the “common good”. AI technology increases their ability to do exactly that. Sadly, that’s already a reality.
7
It is silly to compare older 19th century technology and methods to today’s machine learning which can learn patterns, some we can’t even recognize.
If a pattern is there, it is there.
If a pattern is not there, it is not.
5
Patterns may be likened to correlations. Correlations do necessarily connote causation or the nature of things.
7
Sure, but correlation is not causation.
With AI/ML we outsource pattern recognition, not critical thinking.
4
@Siri That's a cop out, like saying there is a correlation between smoking and lung cancer, but then adding "but correlation is not causation!"
2
In the 1980's I worked at Harrah's Reno. What was interesting is that there was one person who people seemed to mistake for me on a regular basis. Talk about a doppelganger. He worked in a different department, but we looked almost alike to the point of people thinking we were brothers. We weren't, he was from Kansas, and I was from California. After all, how many ways can you make the human face?
6
Seems to me "appearance" is being conflated with genetically conferred characteristics. While it may not be possible to infer someone's character from that person's physical characteristics determined at birth, it is certainly possible to infer much about a person's character from their appearance, since so much of appearance is determined by the person's (conscious and unconscious) choices of how they present themselves, through inter alia their grooming, clothing, hygiene, mannerisms, and in today's world certainly tattoos. We all judge each other based on such characteristics every day, especially as it pertains to those about whom we have little other information. Of course such inferences are far from perfect, but they aren't random either.
8
My Mom used to say "You get the face Nature gave you until you're 50, after 50 you get the face you deserve." Or maybe life's slings and arrows sculpt a reflection of lived experience and one's inner life.
5
The author is deliberately conflating two distinct subjects:
1. Predicting someone’s character from his/her appearance, which has long been discredited and which no one is suggesting be used (this is called a straw man); and
2. Using video surveillance, facial recognition software and image databases to identify persons who have committed crimes.
A recent survey showed that 97.6% of criminals are against video surveillance and facial recognition software. That makes me all for it.
10
No, they are not conflating data with predictive ability. They do, however, suggest that the statistical tools are badly misapplied and are used to justify authoritarian projects and prejudice.
4
one simply cannot judge a book by its cover...
3
Yet many books are bought and sold by their covers nonetheless. Hence, despite the truth of the maxim, often it is the contrary which is practiced.
4
OH! At first I thought you were talking about China or Russia, the places, and dictatorships, that tRump so admires.
5
Q. " . . . When will we finally learn we cannot predict people’s character from their appearance? . . .."
A. At about the same time as MLK's dream --of a day when we judge others by their character and not by color of their skin --
becomes reality.
5
@SpecialKinNJ - Thanks for reminding everyone of MLKs dream.
The hitch is that we cannot on first meeting of someone, let's say someone whom you think might become a friend, learn much about what is on and in their minds, information that might tell you something about their "character"
I mention this because Dorothy Roberts, in her brilliant book, "Fatal Invention, How Science....Reinvents Race in the Twenty-first Century" wrote on page 1 that the first thing people (I suggested to her, add American people) enter a room with a variety of people in it is to try to place a "race" name on each. Not me.
Only-NeverInSweden.blogspot.com
1
Sorry but this article convolutes recognizing individuals with trying to predict something about a person based on how they look. Those are two TOTALLY different things. By all means let's have an informed discussion of AI, facial recognition, and surveillance but can we at least stay on topic!?
16
@Paul R. S.- I agree and tried to deal with that problem in my comment or comments. One thing at a time is a good guide for columnists.
Only-NeverInSweden.blogspot.com
2
No "convolution" (sic) involved (I think you meant conflates?) . You just didn't understand the context nor the conclusions. I suggest you try a reread.
1
What a dumb premise for an article. Yes facial recognition can be used for racist studies and junk studies, but no, that doesn't make AI "racist."
In New York there was a recent spate of hate crimes in which rainbow flags were set alight outside of a gay bar. Does the author now believe that fire is "homophobic/"
To the best of my knowledge as working professional in the tech field (not a handwringing wonk generating clickbait), most uses of facial recognition have nothing whatsoever to do with trying to predict behavior - much less predict behavior based on race.
If the NYT's editorial department is truly interested in shaping the ethics conversation around this and other emerging technologies. They should (a). get input from people who are actually working on these technologies and (b). refrain from ridiculous claims that are easily dismissed and instead focus on the actual nuances at play.
7
@JP
You miss the premise behind Chinoy's argument. Facial recognition has a history--a history that involves identifying people based on phenotypical characteristics, creating categories based on these characteristics, assigning values to these categories --and that history cannot be ignored.
Just because people want technologies in 2019 to be seen as "neutral" because "we don't do that anymore" doesn't make it so. Also: we do.
12
But the underlying algorithm written into the AI can be racists when it reflects the programmer’s subtly racist assumptions about certain types of faces or expressions.
1
The Nazis believed that the Earth is round.
That doesn’t cause me to believe that it is flat.
Face reading in China is a tradition that goes back thousands of years.
Like other aspects of traditional Chinese science, it lacks the theoretical underpinnings developed by the Europeans, but it describes reality just the same.
I refer you to the very informative book, “Secrets of the Face” by Lailan Young.
3
If miss-perception and conjured inferences is reality, then sure, the Chinese get their world right in this regard.
2
"Chinese wisdom"?
An oxymoron if I've ever seen one.
1
Superstition and conjecture are too often the rule of thought.
Interesting that the first article in the Times in modern times dealing with systems for classifying people based on their appearance is given us by Sahil Chinoy, a Times Opinion section graphics editor.
The article deals with many important subjects such as eugenics and the efforts on the part of racists to first assign people to "races" based on appearence and then arrange those "races" in a racial order, with people who most closely resembled the researcher racist placed in the "race" at the top.
Since this subject is never treated seriously in the Times I must offer modest thanks to graphics editor Chinoy for the article but I wonder why the Times does not give us articles on this subject, the classification of Americans on the basis of appearance, the attention it deserves by enlisting experts including but not restricted to genome researchers.
It appears that Times Editors find the US Census Bureau system perfectly acceptable and useful, perhaps forever.
I do not and I often name leading scholars who share my view. Those scholars are otherwise never named in the Times and their work seems of no interest at all to Race/Related.
I wonder why.
I also wonder why there are only 64 comments in print at this article that has been OnLine since the 10th.
Only-NeverInSweden.blogspot.com
Citizen US SE
3
If many genes are involved in brain development and many genes are involved in facial development, then it is statistically very unlikely that no gene involved with brain development has any correlation with any gene involved in facial development. It's also physically impossible for all of the brain and face genes to be uncorrelated because they are physically attached to each other on our chromosomes. As final proof, human facial features correlate with human behaviors, while chimpanzee facial features correlate with chimpanzee behaviors. The differences in features and behaviors along individual humans are obviously more subtle, but the underlying genetic processes are identical.
1
Putting labels such as “eugenics” on ideas the author disagrees with is a cheap shot. Eugenics was an internationally recognized science until it was discredited by the Nazi ideological misuse of it. But the fact that scientific discoveries may have morally troubling implications does not mean they are incorrect. Nature cares nothing for our morals. “All men are created equal” is an aspirational slogan, not a fact. Humans are manifestly not equal in their intellectual endowments. It would be morally comforting to conclude that there is no connection between facial features and character but it is not true. There is a body of research that shows that the ability to recognize and react to faces is inborn in human infants. Standards of beauty are universal, though culturally inflected. People across different cultures prefer beautiful faces to ugly ones; kind faces to threatening or angry faces and so on. This article flies in the face of all the research done in evolutionary biology and cognitive science in the last 15 years. Science denialism in the name of political correctness is no better than science denialism in the name of industry profits.
8
@Mor
Eugenics is alive and well -- we screen pregnancies for serious health issues and terminate nonviable or severely disabled fetuses.
2
@Mor
Eugenics didn't disappear. It was rebranded as "euphenics" by the Nobel Laureate Joshua Lederberg at Stanford in the 1950s. Lots of scientists believed that they could disaggregate the potential of genetic knowledge from "master race" claims of the Nazis, much like the scientists who believed that they could disaggregate the potential of atomic knowledge ("Atoms for Peace") from the bombing of Hiroshima and Nagasaki.
3
Nicely contextualized.
3
I had to look very hard to find very much in this article that would merit such a sensationalist headline involving something "racist." The whole piece is a confusing agglomeration of barely-related topics where race had nothing to do with it.
8
@John - John I actually submitted a comment in which I begin with a new version of the headline that I think reflects better what the article deals with. The headline is not written by the author.
There are also some strange things going on here as concerns comment review. The article is long and as you suggest, an agglomeration. After 24 h or longer there are only 64 comments and the only one of my submissions is a reply to someone who seems to think being Italian is to belong to an Italian "race"
As for "race" having nothing to do with "it" not sure what "it" is in your sentence.
Times writers use "race" and racism as synonyms so I often can be sure what the writer is really trying to tell me.
I mention that as a former upstate New York resident, about 40 years in Rochester. I write from a perspective influenced by 22 years living in Sweden where Swedish medical and genome researchers and also a leading science journalist, Karin Bojs, think American researchers use of "race" is non-scientific, not at all logical, and often misleading.
Only-NeverInSweden.blogspot.com
Citizen US SE
1
How does the author know that we cannot predict people's character from their appearance? Evidence? Only that it hasn't worked in the past.
Why is this racist? That's the all-purpose insult used to denigrate any person or opinion a NYT commenter disagrees with.
12
This is just a tool to help law enforment find criminals or suspects faster. During the Boston bombing the police and FBI reviewed the videos of the area where the explotiona occured. Then the two brothers' pictures surfaced as main suspects and later they were caught. Facial recognition will do the process faster.
9
Oh, I don't know. To me, Jeffrey Epstein's mouth looks a little self-indulgent.
And Donald Trump's eyes seem cold and empty.
While no one's precisely Dorian Gray's portrait, some people do look what they are.
5
You are not using the concept objectively ( blindly ) because you already have biases towards the faces you are evaluating.
1
Oh well, it will get better. Just because something may have had some racial undertones when discovered doesn't mean it can't change.
5
...then again there’s Jeffery Dahmer, Ted Bundy, etc.
Not much predictive value in their physiognomy.
An interesting topic.
Obviously, we haven’t heard the last of it.
3
Imagine you are part of a group of people exercising your constitutional right to freedom of speech by protesting some policy of the government, such as the banning of Islamic immigrants. You notice you and others have been photographed by some plainclothes person who drifts through the crowd. What you don’t know is that these photographs will end up in a computer database of citizens flagged as potential subversives or even terrorists. With good facial recognition software, you are now marked for life as dangerous. What happens when a potential employer does a security check, and the report comes back about you being a risk, so they pick somebody else? You will have way of knowing why you can’t get a job.
4
It's good to have this article published in the midst of the Epstein-Acosta controversy this week. Be honest: when you first heard about a wealthy child rapist getting a sweetheart plea deal from a Trump-allied former prosecutor, and then you saw a photo of Alex Acosta -- what went through your mind when you saw his face?
I'm also reminded of how so many people last autumn demanded public, live-streamed video testimony by Christine Blasey-Ford, on the grounds that the American people and their Senate could evaluate her credibility simply by watching her talk.
It's a deeply-rooted human impulse, thinking that we can instantly judge character by appearance and mien. We all love to flatter ourselves by believing that we have a special superpower spidey-sense, an intuition that magically reveals liars and cheats at a glance.
But this impulse is unfounded, arrogant, and unjust. We should judge people on their character, and this can only be established gradually over time, by accumulated evidence of their behavior.
6
The title seems to be misleading. While this piece does a good job of presenting the history of facial analysis to predict character traits, I was disappointed in the failure to address the real modern day concerns behind our current usage of facial recognition technology. In addition to privacy concerns, there are issues with accuracy in identifying non-caucasians, and thus serious implications for law enforcement.
32
The flaws in handling faces of people in under-represented groups is partly that fewer faces of such groups are in data available to researchers. For example, some datasets used in early research had public figures, or college students. This is cheaper than collecting actually representative facial information. With better funding, data, and engineering these problems will probably go away. The flaws of this tech is thus a temporary problem. We as a society should be more worried about other long run dangers from truly reliable tech, like big brother style abuses.
2
Talk about an absolute non-sequitur. Facial recognition software isn't used for 19th Century personality critiques. It's used as a forensic tool, in the same way as fingerprints (dactylography, if you will) and DNA. It does not INCLUDE. It EXCLUDES. The MORE refined a description of a criminal suspect , the LESS likely other are to be arrested or inconvenienced. It's not a tool to enhance racial profiling. It's a tool to reduce it, i.e., the ideal description of a single individual eliminates everyone else of similar race and gender. Facial recognition software is good now, and getting better - like most software, exponentially
And all this is obvious. Were it not for the subtle Ludditism that has crept into Times technical reporting, this technology would be as welcome an advance as DNA once was. In stead, we get a recital of dismal 19th Century prejudices that are absolutely irrelevant to the modern use and purposes of the technique. Astonishing and sad.
16
The fact that physiognomy has failed in the past doesn't mean it will necessarily fail in the future. Certainly, some things can be learned from a face: emotions like anger and fear.
Mr. Chinoy is simply expressing his prejudices, and using the dirty word (racist) he recently learned on the playground. Children often do that to shock the grown-ups.
14
@Jonathan Katz ... and the resurgence of phlogiston is due any time now.
Seriously, that which has been proven to be scientifically unsound remains scientifically unsound.
12
This is simply not true. For the a long time people thought man couldn’t fly and then planes were invented, that metal wouldn’t float and now we have steel ships. x-rays allow us to se inside the body which prior to that was scientifically unfeasible.
When science and technology advance to the point were we can further understand the human genome and develop early testing to identify behaviors and traits, watch how many people abort these “undesirables”.
Do not underestimate the advances of technology and write off something that was once not feasible.
5
REI HR uses Hirevue for the interview process.
Hirevue is a firm that uses this "science" to assess candidates for viability.
1
This opinion piece is outrageously misleading. Like trying to discredit astronomy with tales of past frauds of astrology and fortune tellers.
Yes, the old analysis attempted to analyze a person's character and predisposition to anti-social behavior based on facial features and skull shapes. That has nothing to do with today's facial recognition techniques which attempt to create a digital "fingerprint" from a picture, and then use those files to try to find matches, as is done with fingerprints.
This new technology has its own shortcomings and errors. But instead of addressing them, the author prefers to try to confuse.
Applying technology to old photographs, showing what those people might look like today, decades later, has had some remarkable results. The NJ murderer John List was found decades later when his aged appearance was broadcast on a national television show.
And perhaps this technology will help in identifying murder and accident victims who presently go unidentified.
11
You missed the part about the Israeli company and the HR company among others who are doing just that. It is not about Luddites vs. Technology. It is about unscrupulous and ignorant persons and organizations and governments which apply dubious science and ethics to the tools that are unseasoned.
2
Same problem as people looking for your physical posture:
When my superior thinks to detect my feelings by analyzing my physical attitude like crossing my arms.
I answer that he can observe my appearance but not read my thoughts.
Laurent Kling
1
This is being recorded.
2
Of course the old ideas about skull shape are nonsense.
However, facial appearance can also be a strong signal of a person's personal choices. It is grooming, it can display care for one's health, it can display athletic activity or a gym rat. It can show us a lot about a person.
It can also mislead us, if shaped by someone for conscious deception. However, that works only because most people don't do that, but instead use their choices to show the world their face, who they are.
Where does facial recognition fit in all that? The wrong place. The people it is looking for are exactly the ones who would make the effort to conceal. I've even read of special glasses designed to dazzle facial recognition software, along the lines of (illegal) license plate covers.
However, our common experience of reading faces leads to automatic assumptions. To assume is error, ass-u-me in the old military teaching. However, we might easily slip from one thing to the other quite different thing without realizing our mistake.
3
@mon Ray....I am German and know from history that innocent people should fear this heightened surveillance state.
4
We have a legacy facial recognition program. We call it eyewitness testimony. The chances are 50 per cent the computer performing the scan and recognition will have an IQ less the median level of the human population. I will take my chances with the non-human program.
4
Mr. Chinoy is mixing up apples, oranges, and pears, and occasionally, watermelons,
The assessment of character via physiognomy has been thoroughly discredited, period.
The use of AI in identification is not absolute, but it’s validity and usefulness is established; the serious issues of privacy and misuse it raises are another matter.
Paul Ekman’s identification of universal facial expressions across racial and cultural groups is well established, though initially there was vehement incredulity from some colleagues. There are cultural differences in how and when certain expressions are suppressed, but not in the emotions associated with them. Again, this has nothing to do with assessing character.
Attempts at behavioral risk assessment, e.g. at airports, is yet another entirely separate matter. Some of Ekman’s early and provisional formulations did not hold up, and were superseded, but are apparently still used by some who want to discredit him.
It is not helpful to conflate these matters. The issues associated with them are too important!
11
@AustinTexan: I agree with a lot of what you say; reading emotions on faces (certainly do-able way above chance) is very different from predicting personality from faces. As an aside, one of my students was (apparently) able to predict Big 5 personality category at above chance rates from faces - but just barely above chance, so I don't really believe his results, and haven't tried to replicate them. I do think it is still an open question. Your genes determine a great deal about your personality, and they also determine what you look like. But the same argument could be used to suggest that your personality could be predicted from your fingerprints - most likely false.
With respect to Paul Ekman's theories, though, his data bears close inspection. As I recall, the Fore tribesmen thought all of the images of caucasians were angry! And while the six "basic" emotions probably exist, it is silly to think that there aren't a panoply of much more subtle mixtures of emotions and cognitive states (e.g., drowsiness) that can be read from the face.
@Gary Cottrell Thanks for your comments.
Yes, there are certainly subtle mixtures of emotions, as well as cognitive states. Ekman's basic point, I believe, was that there was a set of basic facial expressions that corressponded to the same emotions in different cultures. Prior to Eckman, the general belief (assumption?) was that facial expressions were essentially totally culturally determined, rather than culturally modulated.
None of the articles I've read on facial recognition acknowledge the profound diferences cosmetic surgery and dermatology can make in a person's expression and even bone structure. And these are being used with greater frequency and effect nowadays than ever before, thanks to advances in the materials and technques.
There was a film noir --- I forget the name --- where a criminal (possibly Bogart) underwent "plastic surgery" to change his face and evade detection. That was ahead of its time but the idea is now fully implementable.
Today. with the ability to add malar cheekbones, reduce or advance chins by surgery or implants, narrow or widen the jaw, change the nose width and angle, nostril size and shape, etc. dramatically, and erase expression/emotion lines from foreheads and nasalabial folds, change ear position, and now even change eye color using lasers (not available publicly yet), a person can alter their face drastically in its underlying composition and surface expression.
Even children and very young people are having these types of procedures, both for medical reasons and for cosmesis, despite value judgments people place on those decisions. It's a social reality and an increasing one.
So I foresee that the application and reliability for this technology, if those indeed exist, will be reduced as more and more people change their faces.
2
In my opinion, this article is fundamentally flawed. Drawing analogies between some facial recognition technologies and physiognomy does not make their content or purpose identical. Attaching loaded words like “racist” is unwarranted not rooted in logical argument.
There are many good reasons, both ethical and scientific, for questioning, if not opposing, the rush toward employing automated facial recognition techniques. Stick to discussion of those reasons.
8
Sounds similar to handwriting analysis. You can identify people by their handwriting, but you can't really analyze their personalities that way. Trumps gigantic signature might be an exception
2
@DL -- Handwriting is a choice, and it can reveal things about a person. Not everything, not all that is assigned to it, but more than a little. This is especially true of someone who is using it deliberately as a gesture, such as Trump does.
3
As far as I know, facial recognition (FR) software is being used by the federal government simply to confirm an individual's identity (disclaimer: I work for the State Dept in Consular Affairs, and every visa applicant's photo is run through facial recognition software).
The software is not particularly good, it often will call up siblings as matches, for example. We use it simply to make sure no one has changed his name and is attempting to obtain a visa under another identity. Occasionally someone who has been entered into the database because of terrorism concerns will come up as a match.
In any case, no one uses the FR system to predict anything about an individual's tendencies or personality.
19
@David Ricardo-- There are people you don't know who are using facial recognition for other purposes than confirming identity including screening for genetic diseases.
2
It seems like a slippery slope from physiognomy to eugenics. We are heading straight into a global Orwellian authoritarian dystopia, where individual privacy and freedoms are increasingly restricted by pervasive electronic surveillance systems. Anywhere in the world, these "facial recognition" systems could be easily used to discriminate, exclude, maybe even imprison or sterilize people who do not conform to the ideals of the regime in power. This is scary stuff.
8
ML/deep ML fits a complex function to a set of parametres, and is able to find patterns that are difficult to see, but correlation isn't causality. Scientists are under a lot of pressure to publish in high profile journals (I'm a scientist) so they tend to find "sexy" ways to sell their work but the media isn't far behind on this also. We seem to be deep in a PR driven world, where appearance matters more than what is underneath. It's not just with facial recognition.
1
Excellent article. It is one thing to use AI to recognise faces (eg. Going through immigration at the airport), it is another to use it to make inferences about a person’s character and emotional state. It’s even a longer bow to use them like a crystal ball to predict future criminality or behaviour. This is the real issue with loss of privacy - not that others know things about us, but how that information might be put together and used without your knowledge to determine your life. Worse still Pseudoscience gives this a level of objectivity and certainty it does not have. Our current understanding of discrimination is nothing compared to what is coming.
8
We forget that humans are good at interpreting faces. Really good. We’ve designed computers to perform a similar process, with similar outcomes. The more we see a person, the easier they are to recognize. If they’re smiling, they’re probably happy; frowning, upset; neutral, bored. Humans can “read” faces of their own race more easily. Computers, having practiced on mostly white faces, struggle to interpret POC.
Facial recognition technology can be a substitute for human input, e.g. for taking classroom attendance. Currently, it does not substantially improve on human input. If you don’t think *you* could perform a face-related task, it’s a good bet that computers can’t either.
1
Structure of face: no relationship to criminality. Expression on face: whole different story (sometimes).
2
@LetsBeCivil
I'll bet some of the most beguiling smiles are on white collar criminals and although a true smile, involving many muscles is difficult to fake, that's not the case for a psychopath.
4
The racist history behind law enforcement began with slave catching under the Constitution and the Fugitive Slave Act before devolving into the carefully carved colored exception to the 13th Amendment's abolition of slavery and involuntary servitude aka prison.
Alcohol prohibition and the war on drugs were both callously, clearly ,cleverly cruelly cynically and hypocritically focused on limiting non- white European Anglo-Saxon Protestant peoples socioeconomic, political and educational status and rights.
7
@Blackmamba
There is no exception to the 13th Amendment.
Most alcohol bootleggers were Appalachian. Their ethnicity was most often Scotch-Irish.
1
@Blackmamba
What on Earth are you on about? Do you think WASPs were never charged or convicted of crimes?
2
One more example of what might be called "AI Fate". Before, the events that befell you could, in the absence of any other evidence, be ascribed to Fate, but now your entire life could be circumscribed by algorithms operating invisibly, denying you credit, a job, a mate, or a home. There is no court of appeal and no way of ever knowing how or why these things are happening to you. Eventually, even the people who wrote these programs will have either moved on or died but they will just keep churning away, creating endless misery in their wake.
The Chinese seem to be the current front-runners in creating this dystopia but one thing you can be sure of, all these procedures can and will be overridden or gamed for those who are deemed too important to submit to such base humiliation. While everyone else is watching their backs, making sure they don't bring down Big Brother's wrath, those in charge will be cavorting like thousands of Jeffrey Epsteins with complete impunity.
17
I appreciate the argument. But I always liked a story about Abraham Lincoln. He turned a man down for a job and was asked why. "I didn't like his face," was the answer. When reproached that that seemed like a superficial reason, Lincoln replied, "Every man over forty is responsible for his face." This suggests, though, that our faces (shaped by our habitual expressions) evolve over time to show our true character, not that the face we were born with says anything about us.
1
@Kyle Gann - or our faces evolve over time to match our dogs.
Give a listen to Talking Heads "Seen and Not Seen", especially the last line.
3
Speaking of Lincoln, he was reportedly accused once of being "two-faced." His reply was along the lines of, "if I were two-faced, would I be wearing this one?" I know how he felt.
3
@Kyle Gann
And Lincoln never even saw Exhibit A, Mitch McConnell.
1
Actually, I think you can tell something about a person's character from his or her face. Look at Anthony Scaramucci, for example. Is that a face you would trust? ( I'm Italian, so no singling him out on ethnic grounds.)
7
@Cynthia Starks - and people are naturally (and I mean by our entire millions of years of evolution) suspicious of people who don't look like us and our family. So how useful is that now that the average small band of hunter-gatherers is encountering hundreds or thousands or more faces every day?
4
@Cynthia Starks - Fortunately, Cynthia Starks, you are still free to express your personal opinion, since you live in the United States, but if you live by this belief, then you may be engaging in serious misjudgments.
And then you end by telling me that you are Italian so not singling him out on ethnic grounds.
Italian in my vocabulary refers either to nationality (citizen of a country) or nativity (born in that country) or even both.
As a dual citizen of US and SE I encounter this problem often. Since I learned Swedish late in life any "real Swede" can hear instantly that my first language must be American English. Fine. But occasionally this leads a "Swede" to say to me "Du är inte svensk" because they define being Swedish in almost essentialist terms.
Only-NeverInSweden.blogspot.com
The Mooch was ny neighbor in my hometwon. His son and my daughter went to HS together. I see him around town at the deli, the bookstore. He is an Italian elf who bounces around like the Energizer Bunny. I do not endorse his politics. I respect his business accomplishments. He was not a swamp creature, at least. He was booted out faster than anyone else by Trump which just goes to show, he was a never a true Trumper but was testing the waters of politics and was too naive to play.
Great piece. Thank you. I want my privacy and will be resisting to the very end as I don’t agree to any of this and disagree with the commenters who think it is only criminals who are against this. Its Life liberty and pursuit of happiness. Not life liberty and surveillance.
26
Surveillance using public records and revealing the mysteries of personality and behavior are rather different things of course. One in fear of their civil liberties from this technology should skip the DMV, abandon social media, and stop committing crimes in public.
11
@Frunobulax, would also suggest not going through Customs at SFO Airport wherein they immediately take a facial photo (super duper sized) every time one flies international from the NCal Bay Area and who knows where all this information goes. Analyzing my facial features would surely have me institutionalized by now, lol.
I was just reading a historical fiction book, (based on actual people and events) which had generally been extremely insightful in the characters’ psychology, and pointed out that whether a king was handsome bore no relationship to whether he could rule and rule well;
Yet suddenly, the author spent several paragraphs pointing out how one king was obviously ‘weak-chinned’, among other facial attributes meant to convey his character. There was also a comment on how an unattractive face reflected evil.
The sudden shift in tone, and the seriousness in which it was made, almost made me question myself ? I have also encountered in certain other philosophies and religions the belief in connecting external beauty with the inner life. And yet we know, intellectually and empirically, that this is not true.
3
Maybe it's a newer book whose author has too been overwhelmed with the ubiquity of face-creeping and the heretostayism of commentators who only mildly criticize it, and decided midway: if they can't beat 'em, join 'em.
1
Sahil Chinoy is a graphics editor so should know--and acknowledge--that there are many well documented applications of facial analysis in diagnostic medicine.
Researchers have published open source software that can correctly identify about 1/3 of all known genetic disorders from any unobstructed facial photo taken at birth or any age thereafter. (Downs Syndrome for example)
While very helpful to doctors, I fear this software could be used to discrimante against people in employment, housing, immigration, etc. While various laws protect disclosure of personal diagnosis info from confidential records, you can't hide your face in the age of photo ID and ubiquitous video surveillance.
22
Much the same arguments could be made against fingerprints, DNA analysis or wanted posters in the post office for that matter. The protection of civil liberties is paramount, but that doesn't mean ignoring technology breakthroughs.
25
@james haynes - for it to be a breakthrough, the results need to be accurate. Just as it's no sure thing that modern scanning can detect personal character traits by looking at a face, it's still difficult to scan claims from software firms and diagnose the level of profitable hype vs. actual validity.
Don't ignore technological breakthroughs - but never fail to scrutinize them to make sure it's not just old flim-flam dressed up in new tech.
7
According to a recent survey, 97.6% of criminals were against surveillance cameras and the use of AI to identify perpetrators of crimes.
Similarly, in the late 1800s and early 1900s surveys of criminals were undertaken and 96.8% of them were against the use of fingerprints to identify perpetrators of crimes.
I am pretty sure a huge majority of law-abiding Americans support any techniques, including surveillance cameras, that will make their lives safer, reduce crimes and apprehend criminals.
The only people who have to fear surveillance cameras and AI identification are those involved in illicit behavior.
36
@Mon Ray -- "that will make their lives safer, reduce crimes and apprehend criminals"
That's the catch.
Does it make non-criminals safer, or put them at risk of mis-identification and ugly police contracts?
Does it reduce crimes, or divert police resources?
Does it actually apprehend criminals who would not otherwise be caught?
Those who fear constant surveillance include those mis-identified, those with a false sense of security, and anyone like George Orwell's characters encountering universal surveillance by a "well meaning" government run by self serving, small minded people with a bit of power gone to their heads.
54
I'm a law-biding American and I don't support "any technique." I don't want insurance companies, cops, hospitals or companies using my data to make it hard for me to live. I'm not doing anything wrong. And I'm surprised that so many NYT commenters agree with your harsh and draconian attitude about the growing surveillance state we are becoming.
15
@Mon Ray - where is a verifiable link to that survey you mention? Where is data to back up your "I'm pretty sure" statement?
Are you "pretty sure" you're fine with the same technology automatically ticketing you for doing 30 in a 25mph zone, or all sorts of other profitable minor infractions towns like to fund themselves with? Cameras and storage and software cost money, so it needs revenue to offset. Automatic fining by facial recognition could be a revenue source waiting for new infractions.
You jaywalk any? You guilty of any minor moving violations where the cops use their "he's a white guy" to give you a pass? Talk on your cell phone while driving an an area that bans that? That's a wasted revenue opportunity if you look at it that way - and just think of the speed trap concept on steroids.
I assume you aren't Hispanic, so don't have to worry that current Administration efforts against "illegals" won't sweep you up - or are you advocating that every citizen be forced to carry a national identity card?
Basically - are you comfortable to have you and all of your family members be photographed any time outside of your home, and for those photos to be stored indefinitely, and be searchable, and be subjected to any latest kind of "this person might be guilty of" algorithms, and for you to have to deal with potential fines and penalties and possible arrest for things based on accurate or possibly inaccurate identification of you or a loved one?
10