Discrimination against women persists in other ways. Take the obituary column of the NYT - on a good week, you will find obits for perhaps 5 women compared to 20 men. This discrepancy has been pointed out to the NYT by my wife a couple of years ago. Did they add more successful women? No, they didn't. And usually the women they report on in the obit are those in the arts, whether they participated as actors or patrons. You rarely see any woman who had a career in science or health care, or social work, or teaching. We know there are lots of them out there who have done outstanding work for our country. For shame, NYT!
44
"There is a widespread belief that software and algorithms that rely on data are objective." says Ms. Miller.
Well, Ms. Miller, two things:
1. If by objective, you mean completely devoid of human input, there is no such thing as an objective algorithm, since algorithms are designed by humans. If by objective, you mean once the procedures are designed by humans, their execution does not involve humans, algorithms that do not use human input as part of the process are indeed objective. BUT ...
2. Objective (using the second definition above) and unbiased/non-discriminatory are not at all associated despite popular misconceptions. It is easy to device a completely objective (again, using the second definition) and extremely discriminatory algorithm.
Well, Ms. Miller, two things:
1. If by objective, you mean completely devoid of human input, there is no such thing as an objective algorithm, since algorithms are designed by humans. If by objective, you mean once the procedures are designed by humans, their execution does not involve humans, algorithms that do not use human input as part of the process are indeed objective. BUT ...
2. Objective (using the second definition above) and unbiased/non-discriminatory are not at all associated despite popular misconceptions. It is easy to device a completely objective (again, using the second definition) and extremely discriminatory algorithm.
10
Algorithms are written by people. People are biased, not objective. Daniel Kahneman et al. have proven this.
11
There is actually a legal analysis that is quite apt. When a facially neutral statute (analogous to an algorithm) has a disproportionate impact on a protected class (race, religion, gender, sexual orientation), then the burden should shift to show that the classification (algorithm) both serves an important interest, and that it is the least discriminatory way of addressing that interest.
Why shouldn't we evaluate these kinds of algorithms the same way? We may not be dealing with governmental actions, but shouldn't similar rules apply in covered commercial and otherwise covered interactions?
Why shouldn't we evaluate these kinds of algorithms the same way? We may not be dealing with governmental actions, but shouldn't similar rules apply in covered commercial and otherwise covered interactions?
15
It's not that algorithms are carrying forward human bias, it's just that they are not designed (yet) to care about how someone feels when confronted with reality. The only thing the algorithm wants to do is put forward the search results MOST people are seeking. If most people wind up looking at male CEOs, those results will be pushed to the top as most relevant. That says more about the reality of the workplace environment than inherent bias in any algorithm. If searches for African Americans yield ads or results related to incarceration, it says more about either: (a) over incarceration or (b) unusually high crime rates. Either way, the algorithm is presenting what it has found to be true. When the real world statistical measurable behaviors change, so will the results. Just as a final note, if someone is truly interested in finding female CEOs, they can simply type that. Maybe if enough people were interested in that search, the more generic search term ("CEO") would yield more balanced results. Apparently, that's not the case.
18
It's time we realized that 'discrimination' is a much misused term - women are not nearly as successful as CEOs as men have been, and some outcomes are far more likely than others - eg. watch an true crime show and note the frequency African-Americans turn out to be the perpetrators.
12
Comments like this one are why I left Phoenix.
17
"women are not nearly as successful as CEOs as men have been"
Might not this be because of the historic and current "discrimination" against women? Duh.
Might not this be because of the historic and current "discrimination" against women? Duh.
16
I disagree with your statement that women are less successful as CEOs. What is your proof of that assertion? As for true crime shows, you cannot say they depict a cross-section of actual crime -- someone has picked the crimes and criminals to feature, and you don't know what the criteria for selection might be.
Or maybe you are a troll and I am being gullible in even responding here...
Or maybe you are a troll and I am being gullible in even responding here...
12
Here at the NYTimes at DotEarth you can see the pernicious result of the NYT algorithm. A "fair and balanced" choice of people who have learned to game the system does not distinguish between the truth and the constructed dishonesty of the fossil-funded mirror universe that emphasizes biased "science" to deceive. Credible expertise is 80-99% in agreement about how accumulating heat-trapping greenhouse gas emissions increase heat in the system (global warming) resulting in a disturbed planetary circulation (climate change).
"About that consensus on global warming: 9136 agree, one disagrees"
http://blogs.scientificamerican.com/the-curious-wavefunction/about-that-...
These verified (me pot vs. kettle, sorry) commenters use their platform to bully and mischaracterize both science and their fellow commenters, and have consistently driven away good discussion between interested and interesting voices, enabling each other.
---
Similarly, Google searches on global warming/climate change subjects promote "the world's most popular science blog" WattsUpWithThat and many others who have learned to game the system so their less credible efforts appear on the first page of any search.
For example, try the first page of a search on Richard Feynman/climate. Those of us who knew him before he died agree that he would make short work of this dishonesty. But the search follows traffic created by a purposeful campaign.
"About that consensus on global warming: 9136 agree, one disagrees"
http://blogs.scientificamerican.com/the-curious-wavefunction/about-that-...
These verified (me pot vs. kettle, sorry) commenters use their platform to bully and mischaracterize both science and their fellow commenters, and have consistently driven away good discussion between interested and interesting voices, enabling each other.
---
Similarly, Google searches on global warming/climate change subjects promote "the world's most popular science blog" WattsUpWithThat and many others who have learned to game the system so their less credible efforts appear on the first page of any search.
For example, try the first page of a search on Richard Feynman/climate. Those of us who knew him before he died agree that he would make short work of this dishonesty. But the search follows traffic created by a purposeful campaign.
11
In "real" science (including medicine, social science, etc.), we generally use competitive, peer-reviewed professional journals for solid information. This means, generally, not blogs, not commerically sponsored information, not random comments, etc. It preserves the intergrity of scince to have agreed-upon data sources, not algorithms based more on popularity. Popular does not really mean "true" inspite of advertisers' preferences (for profit over "science).
20
Thanks Lori. In the manufactured debate over climate science, it goes a long way beyond that. You may see at DotEarth that all of the credible expertise and science you mention is treated as a big "scam" or conspiracy, and clean energy development is continuously misrepresented. Say anything at all, and in a few minutes it will have been turned on its head and used to imply the opposite.
8
"The reason for the difference is unclear. It could have been that the advertiser requested that the ads be targeted toward men, or that the algorithm determined that men were more likely to click on the ads."
No, it's not unclear. You answered your own question.
No, it's not unclear. You answered your own question.
7
That is why I occasionally run a script to confuse the heck out of Google, to make it think I am an uber terrorist. I run random searches using the Homelandie's 'terror words' list. Not often enough to put a big load on Google, just enough to mess up what it thinks about me.
(The script is at homelandistan.com/goofle.html . Feel free to copy to your own machine. It opens one popup window for the searches, so you can look at the code and run it off your own server if you want to be sure it is not malware ridden.)
(The script is at homelandistan.com/goofle.html . Feel free to copy to your own machine. It opens one popup window for the searches, so you can look at the code and run it off your own server if you want to be sure it is not malware ridden.)
3
@MCE: "That is why I occasionally run a script to confuse the heck out of Google, ..."
Why do you want "to confuse the heck out of Google"? Wouldn't it be simpler to never log in to a Google service and to always clear your browser cookies and history?
Why do you want "to confuse the heck out of Google"? Wouldn't it be simpler to never log in to a Google service and to always clear your browser cookies and history?
2
Algorithms do not discriminate --- algorithms used to make decisions by a person or company's can be discriminatory. For example, an algorithm that says when "A + B is negative then discriminate" is harmless until unleashed upon the public. In other words symbols express ideas --- no more. Speed doesn't kill --- speeding drivers kill.
1
Wrong. In today's internet every decision can made by a machine. If a driverless car causes an accident is a human at fault?
3
Bias is endemic to human nature - so live with it.
As for failed targeting, blame the programmer - or perhaps the data that reached the algorithm that he (or she) wrote.
Human frailties continue to abound, don't they?
As for failed targeting, blame the programmer - or perhaps the data that reached the algorithm that he (or she) wrote.
Human frailties continue to abound, don't they?
4
"Computers are useless. They can only give you answers."
4
Depending on the question and the construction of the question.
Do you "know" that said answer really does that?
Do you "know" that said answer really does that?
2
The sad part of this (along with all the comments that say "algorithms aren't the problem") is that the fix is pretty straightforward: don't use gender in your algorithm.
11
Now we are going to write algorithims that have to run through an application that applies a politically correct veneer? Talk about a corruption of integrity and quantitative methodology. Ridiculous.
12
"Research from the University of Washington found that a Google Images search for “C.E.O.” produced 11 percent women, even though 27 percent of United States chief executives are women."
Nothing, absolutely nothing, is in this article points to the algorithm as the reason for this. It might well be, but no reason for that conclusion is given.
Why does this media outlet continue to inflict on us articles involving statistics by people with little or no evident statistical training?
Nothing, absolutely nothing, is in this article points to the algorithm as the reason for this. It might well be, but no reason for that conclusion is given.
Why does this media outlet continue to inflict on us articles involving statistics by people with little or no evident statistical training?
7
The original paper does indeed mention algorithms and use statistics.
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
http://dub.washington.edu/djangosite/media/papers/unequalrepresentation.pdf
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
http://dub.washington.edu/djangosite/media/papers/unequalrepresentation.pdf
3
Simple rationality of an algorithm does not shield it from legal challenges. Where an algorithm produces a disparate impact upon a protected class of individuals, or when it produces a result that is inseparable from impermissible profiling, it may in fact be both illegal and unconstitutional.
4
Part of the issue here may be that following Bayesian principles of rationality can be inconsistent with socially accepted ideas of fairness to individuals. Knowing what categories a person falls into can give us information about him or her. But often we think that it is odious to use that information, because we are not evaluating the person on the basis of his or her individual qualifications, but rather using a stereotype. We think it is ethically wrong, to that extent, to be rational (and, indeed, that might be true).
16
Algorithms are amoral? So are guns. The "morality" of an object or procedure lies in its human use. It's disingenuous to create something, and then to throw up your hands and say "hey, not my fault what it does". Maybe that worked for the Christian creator god, but not so much for Oppenheimer.
This lack of human common sense and nuance is what bugs me about the adulation of programming, and the redefining of intelligence as "that which can be controlled and predicted by rules". I don't know why we aren't more appalled at being reduced to quantification.
My yahoo email account keeps giving me ads for age-appropriate heterosexual dating services. They have my age and sexuality right, but I'm happily married, and even if I welcome eye candy, the guys they show are viscerally disgusting to me. They look leering and crude. Not one of them strikes me as intellectual, professional, or artistic. My most casual acquaintance, were I single, wouldn't try to fix me up with one of them. I've even contacted yahoo twice to say that I don't want this ad. It's still by far the most frequent ad that pops up, far more often than the kinds of things I routinely shop for online.
We're more than the sum of our data. I hadn't thought of how misdirected ads reinforce prejudices, and practices such as automated hiring are even more corrosive of social trust. We need more in-person contact: our human senses have evolved to read interpersonal nuance.
This lack of human common sense and nuance is what bugs me about the adulation of programming, and the redefining of intelligence as "that which can be controlled and predicted by rules". I don't know why we aren't more appalled at being reduced to quantification.
My yahoo email account keeps giving me ads for age-appropriate heterosexual dating services. They have my age and sexuality right, but I'm happily married, and even if I welcome eye candy, the guys they show are viscerally disgusting to me. They look leering and crude. Not one of them strikes me as intellectual, professional, or artistic. My most casual acquaintance, were I single, wouldn't try to fix me up with one of them. I've even contacted yahoo twice to say that I don't want this ad. It's still by far the most frequent ad that pops up, far more often than the kinds of things I routinely shop for online.
We're more than the sum of our data. I hadn't thought of how misdirected ads reinforce prejudices, and practices such as automated hiring are even more corrosive of social trust. We need more in-person contact: our human senses have evolved to read interpersonal nuance.
11
"I've even contacted yahoo twice to say that I don't want this ad."
Install Adblock and be done with it.
Install Adblock and be done with it.
3
I run these ads for a living, and I've got to say - the reason they show the ads to you, even though you're happily married, is because ads targeted to people matching your profile (demographic, search history and more) have historically generated more revenue than ads that weren't targeted to your demographic.
But if you're not clicking on those ads, and still seeing them, there are at least two possible reasons why.
1 - The advertiser doesn't care. They're trying to build a brand, and the way you do that is to get eyeballs on your product.
2 - The agency running the ads is not doing their job.
2a - There's a thing called "frequency caps" where you can - duh - set a cap on the frequency with which someone sees an ad. A common frequency cap rule would be "stop showing this ad to this person if they see it five time without clicking on it."
2b - yahoo impressions - an impression is when you see the ad - are generally cheaper than a google impression. It's entirely possible that the agency has a set budget, and has promised to deliver, say, 2,000,000 impressions for that budget. The less they pay for those 2,000,000 impressions, the more they make.
In any event, if you're truly not clicking on the ads, and they keep showing them, someone's not doing their job right. But I'll bet plenty of other people are clicking them, or they'd stop running them.
But if you're not clicking on those ads, and still seeing them, there are at least two possible reasons why.
1 - The advertiser doesn't care. They're trying to build a brand, and the way you do that is to get eyeballs on your product.
2 - The agency running the ads is not doing their job.
2a - There's a thing called "frequency caps" where you can - duh - set a cap on the frequency with which someone sees an ad. A common frequency cap rule would be "stop showing this ad to this person if they see it five time without clicking on it."
2b - yahoo impressions - an impression is when you see the ad - are generally cheaper than a google impression. It's entirely possible that the agency has a set budget, and has promised to deliver, say, 2,000,000 impressions for that budget. The less they pay for those 2,000,000 impressions, the more they make.
In any event, if you're truly not clicking on the ads, and they keep showing them, someone's not doing their job right. But I'll bet plenty of other people are clicking them, or they'd stop running them.
12
Another funny, everybody-is-on-summer-vacation (or, at least, the editors) piece. The author writes, "As a result, say researchers in computer science, ethics and law, algorithms can reinforce human prejudices." If researchers in science, ethics, and law are getting paid to do research to tentatively make this conclusion, they are overpaid.
By definition an algorithm discriminates. It says yes to this and no to that. Where the this and the that are delineated is irrelevant. It could be the people writing the algorithm; it could be the people giving the criteria the algorithm writer uses; it could be those who did the research on which those criteria were developed.
By definition an algorithm discriminates. It says yes to this and no to that. Where the this and the that are delineated is irrelevant. It could be the people writing the algorithm; it could be the people giving the criteria the algorithm writer uses; it could be those who did the research on which those criteria were developed.
13
I could also just be measuring the behaviour of the population, a behaviour many would see as undesirable or at least one that should not be reinforced.
1
Of course this is complete bosh. Algorithms are written by coders who use business rules defined by a client. We know exactly how they produce what they produce. The bias is in the people setting the business rules. It's not magic. A computer isn't a mystery sentient being that does stuff we can't figure out. It does what it's programmed to do and Google alludes to that in their response.
The ads are targeted toward men and they work very well.
The problem isn't in the machines. It's in the people making the decisions.
The ads are targeted toward men and they work very well.
The problem isn't in the machines. It's in the people making the decisions.
22
Most programmers find that their business clients don't have a clue about software or about what they really want.
5
Algorithms are based on data that it can analyze and use past information and apply it to the present or future. The apparent bias is caused by the historic/structural discrimination in the system and not the people who wrote it. The smart data analysts and business managers should know the limits of the data and algorithms and only use it as a guide to making decisions, and be open to changing algorithms or guidelines that are flawed. Business managers should be looking at potential bias and change their behavior when confronted with it.
FYI, my bona fides include being a data analyst and manager using large data sets, so I have some idea of how the process works.
FYI, my bona fides include being a data analyst and manager using large data sets, so I have some idea of how the process works.
5
Well, it takes an algorithm to become a verified commentor for the NY Times, what does that say?
11
This article (like much of the Times' content touching on race/gender as of late) verges on the Orwellian. "Algorithms can reinforce human prejudices?" Well, *objective reality* and *math* can reinforce human prejudices. I bet searches for "mugger" or "criminal" are more likely to return images of men than women. Is this sexist, or does this merely reflect the fact that muggers and criminals are more likely to be men? Any statistical analysis done on any crime dataset will yield the same result -- perhaps the statistical concepts of mean, mode, and standard deviation should be re-configured with "a lot of care and thinking about the ways we compose these technical systems," in the words of the Berkeley researcher quoted in they article? Perhaps the traditional legally-defined crimes of robbery and assault should also be re-configured to correct this gender imbalance as well? Perhaps all science and math should be re-configured to support our political beliefs, a la Marx, Pol Pot, and Plato?
10
Verging on Orwellian is bad, sure. But in a nation where equal treatment is a high value, showing concern about how prejudices get passed down is a far cry from Cambodian genocide. Keep references to Pol Pot out of this.
6
But the problem is not that the algorithm returns a statistically representative number of photos of women CEOs. The problem is that it underestimates the true number of women CEOs. There's no need to undermine math, only to get the math right.
10
It is possible for something to reflect "objective" reality and still be sexist. Because obviously, objective reality is sexist itself. Why mirror sexism back onto itself?
Why are men more likely to commit violent crimes than women? It's called gender socialization. We teach men that it is "natural" for men to be more violent than women; we teach them that aggression and masculinity often overlap. We conflate chauvinism with being "macho", and thus more attractive to women. Is that a form of sexism? Absolutely.
Why are men more likely to commit violent crimes than women? It's called gender socialization. We teach men that it is "natural" for men to be more violent than women; we teach them that aggression and masculinity often overlap. We conflate chauvinism with being "macho", and thus more attractive to women. Is that a form of sexism? Absolutely.
2
People don't realize how much search engines target us by demographics and history, rather than the search terms. Years ago, if you entered four different words, search engines placed results with all four words above others in "organic" (read: unpaid) results.
Now, search engines believe they know better what you seek, so it may be that out of your unpaid results, the top few drop one or more terms because another term is more popular. What you asked for is harder to find.
Results are skewed by those around you as well as your own preferences, not just advertising targeting. It's difficult to try to get "unbiased" results. There is a private mode that supposedly ignores who you are, but you pay in not being able to preserve your search history as easily. You have to be "you, as the search engine interprets you," or the equivalent of a online non-person, who can't maintain a history because you aren't "real."
Results skewed by neighbors or demographics are results skewed to keep me like others, keep me in my place, and keep me making money for businesses. The idea that we might want to see perspectives beyond ours, and see them easily, flies in the face of algorithms designed for marketing and adapted to everything else.
It's part of why we have deeper and deeper political divides. You can't tell someone, "Google it," because what they see and you see will differ. Reality according to search.
Now, search engines believe they know better what you seek, so it may be that out of your unpaid results, the top few drop one or more terms because another term is more popular. What you asked for is harder to find.
Results are skewed by those around you as well as your own preferences, not just advertising targeting. It's difficult to try to get "unbiased" results. There is a private mode that supposedly ignores who you are, but you pay in not being able to preserve your search history as easily. You have to be "you, as the search engine interprets you," or the equivalent of a online non-person, who can't maintain a history because you aren't "real."
Results skewed by neighbors or demographics are results skewed to keep me like others, keep me in my place, and keep me making money for businesses. The idea that we might want to see perspectives beyond ours, and see them easily, flies in the face of algorithms designed for marketing and adapted to everything else.
It's part of why we have deeper and deeper political divides. You can't tell someone, "Google it," because what they see and you see will differ. Reality according to search.
9
Try an unfiltered search engine -- I have been told that duckduckgo is such a thing.
The author fails to discriminate between factual differences, patterns that reflect commercial third party input and those that are biased. If I search "high crime cities" and those cities have a disproportionate number of African American residents is this racists? The basic problem with imposing social engineering on facts is that you not only distort markets, you cover up real issues that require remediation. People shouldn't be shocked that if you search for Bailbonds men that it targets males. Guess what, males commit more crimes.
5
Yes, but WHY do men commit more crimes? Could it be because we as a society condone mail violence? We expect men to be violent the same way we expect women to be sexy.
By reinforcing the notion that it is just "objective fact" that men commit more crimes than women is to misunderstand bias altogether. Media bias--the widespread portrayal of men as naturally or fundamentally more violent--reaps what it sows. The key to understanding patriarchy is that it negatively affects men as well as women.
By reinforcing the notion that it is just "objective fact" that men commit more crimes than women is to misunderstand bias altogether. Media bias--the widespread portrayal of men as naturally or fundamentally more violent--reaps what it sows. The key to understanding patriarchy is that it negatively affects men as well as women.
2
'If I search "high crime cities" and those cities have a disproportionate number of African American residents is this racists?' Maybe. What are the facts? Your choice of question appears to be unconsciously racist.
1
I agree that societal biases can affect behaviors but it is not the job of search engines to socially engineer outcomes. If you do a search for guns, should we place articles on how bad gun violence is on the top of natural search? Should a search on murders result in articles on the affect on crime of broken homes, school graduation rates, etc? Men do commit more crimes and the why can be debated. BTW - to suggest that it is a derivative of a patriarchal society is like saying the reduction in murder rate is because society is getting heavier. Correlation is not causality.
Age discrimination can be built in with or without intent to override skills, degrees, experience, etc. You have to game the job system by listing terms but even that will not eliminate hidden quals such as too many degrees, too many jobs, too many years in the workforce, years, moves, etc.
On ads, I have an unusual search profile resulting in many popups for telephone lineman's gloves, work boots, glow in the dark vests, Lidar equipment, etc when I actually buy very girly shoes and Yoga stuff. Numbers/clicks lie. That said, they should be cross marketing those gloves. My garden has thorns galore. Tempting.
On ads, I have an unusual search profile resulting in many popups for telephone lineman's gloves, work boots, glow in the dark vests, Lidar equipment, etc when I actually buy very girly shoes and Yoga stuff. Numbers/clicks lie. That said, they should be cross marketing those gloves. My garden has thorns galore. Tempting.
3
Search engines like Google use Algorithms to eliminate websites they feel are using words for more hits from their searches. But many websites like mine cease to exist on search. Most of this has to do that I don't do ads, actually it is a cookie free, free website you can visit, sort of against what the gods of the internet accept these days.
I imagine if this get used shills from Google will be complaining that I shouldn't complain because I get free email, I don't get free email I've always paid for it. Likewise I shouldn't complain because I get free search, if I want ads I guess I'd search on Google but I don't.
Long ago the internet was sold as the Information Highway, thanks to Google and other companies it is just a bunch of ads, information doesn't matter anymore and algorithms are just a way to decide what is the best ad to show you next.
I imagine if this get used shills from Google will be complaining that I shouldn't complain because I get free email, I don't get free email I've always paid for it. Likewise I shouldn't complain because I get free search, if I want ads I guess I'd search on Google but I don't.
Long ago the internet was sold as the Information Highway, thanks to Google and other companies it is just a bunch of ads, information doesn't matter anymore and algorithms are just a way to decide what is the best ad to show you next.
2
Kenexa, an IBM company that provides job applicant assessment services, believes that a lengthy commute raises the risk of attrition in call-center and fast-food jobs. It asks applicants for call-center and fast-food jobs to describe their commute by picking options ranging from "less than 10 minutes" to "more than 45 minutes. The longer the commute, the lower their recommendation score for these jobs, said IBM's Jeff Weekley in a 2012 Wall Street Journal article.
Applicants also can be asked how long they have been at their current address and how many times they have moved. People who move more frequently "have a higher likelihood of leaving," Mr. Weekley said. These "insights" are based on algorithmic decisionmaking and machine learning.
Are there any groups of people who might live farther from the work site and may move more frequently than others? Yes, lower-income persons, disproportionately women, Black, Hispanic and the mentally ill (all, protected classes).
The Kenexa findings are generalized correlations The insights say nothing about any particular applicant. The application of these insights means that many low-income persons are electronically redlined. Employers do not even interview, let alone hire, qualified applicants because they live in certain areas or because they have moved. The reasons for moving do not matter, even if it was to find a better school for their children, to escape domestic violence or due to job loss from a plant shutdown.
Applicants also can be asked how long they have been at their current address and how many times they have moved. People who move more frequently "have a higher likelihood of leaving," Mr. Weekley said. These "insights" are based on algorithmic decisionmaking and machine learning.
Are there any groups of people who might live farther from the work site and may move more frequently than others? Yes, lower-income persons, disproportionately women, Black, Hispanic and the mentally ill (all, protected classes).
The Kenexa findings are generalized correlations The insights say nothing about any particular applicant. The application of these insights means that many low-income persons are electronically redlined. Employers do not even interview, let alone hire, qualified applicants because they live in certain areas or because they have moved. The reasons for moving do not matter, even if it was to find a better school for their children, to escape domestic violence or due to job loss from a plant shutdown.
10
“...Even if they are not designed with the intent of discriminating against those groups, if they reproduce social preferences even in a completely rational way, they also reproduce those forms of discrimination...”
If an algorithm returns the statistic than men search for online porn 100x more often than women, is the algorithm discriminating against women?
Is that not sexism?
Isn't everything?
If an algorithm returns the statistic than men search for online porn 100x more often than women, is the algorithm discriminating against women?
Is that not sexism?
Isn't everything?
1
An advertiser asking that their ad for a CEO be target exclusively toward men would clearly be discriminatory, but if the algorithm determines a preference based on success rate (user clicks) - and if women don't click on ads for CEO positions, than I don't think a computer search engine has the responsibility to fix these societal issues.
If we wanted gender-blind search engines, my web experience would have to include feminine hygiene product ads, and my wife would be bombarded with after-shave ads.
We need to address the social root cause in our schools and media outlets. Using the web for social engineering seems like treating the symptoms, not the disease.
If we wanted gender-blind search engines, my web experience would have to include feminine hygiene product ads, and my wife would be bombarded with after-shave ads.
We need to address the social root cause in our schools and media outlets. Using the web for social engineering seems like treating the symptoms, not the disease.
3
I agree with your ideas here, but it's also important to recognize first, the power of the internet to subtly influence people's opinions and beliefs, and second, the "echo chamber" effect. When, for example, you and your wife are subjected to gendered ads, this has profound effects on the shaping of gender. When site algorithms determine that we belong in Category X, they spam us with stereotypes of what Category X must want. For example, around Halloween, I am subjected to ads for a bevy of sexy costumes. Based off of my purchasing history, the site has identified me as a woman, and is now urging me to buy...sexy nurse outfits. This is not just a "symptom" of negative gender stereotyping, but a circuitous cause in and of itself that feeds into the vicious cycle of female hypersexualization.
1
Oh, great! That's what we were missing, sexist algorithms. What's next racist computers?
1
Ms. Miller's column is useful, if for no other reason than that it exposes the naivete of the author, and perhaps millions of technology users, who magically think that algorithm-generated messages are anything other than tools that one set of humans have created in order to attract, manipulate, and prey upon other humans.
5
Computer code is written by people, and those people tend to be white man-boys. Spend some time in an area that real estate developers call a 'tech center' and you will see who these people are; their cultural constructs are the basis for a good amount of online life, and ultimately, online results.
Swipe right for yes, left for no.
Swipe right for yes, left for no.
3
Facebook has nothing to do with faces, books or friends.
9
If you're going to write about whether algorithms "discriminate," you have to define the term, or describe what constitutes non-discrimination. Otherwise, you leave the question begging.
Further, as stated, there is human influence, both at the programming level and as a result of users' previous search histories. So to the degree non-discrimination is defined as separate from humans contact, you've already lost.
But do you actually even want non-discrimination if that's how it's defined?
If not, then you're left with distinguishing between which human behavior is acceptable and which isn't.
Further, if you're going to allow targeting, at least in certain circumstances, it becomes even messier.
Lastly, terms like "rationality" and "objectivity" do not necessarily deserve their gold standard status. See my article on the latter:
“Is it Objective to ‘be Objective’ about Sustainable Business Metrics?: Part XIV,” Sustainable Brands, http://www.sustainablebrands.com/news_and_views/new_metrics/matt_polsky/....
A difficult challenge, but maybe we need to go there, even apart from the subject of this article.
Further, as stated, there is human influence, both at the programming level and as a result of users' previous search histories. So to the degree non-discrimination is defined as separate from humans contact, you've already lost.
But do you actually even want non-discrimination if that's how it's defined?
If not, then you're left with distinguishing between which human behavior is acceptable and which isn't.
Further, if you're going to allow targeting, at least in certain circumstances, it becomes even messier.
Lastly, terms like "rationality" and "objectivity" do not necessarily deserve their gold standard status. See my article on the latter:
“Is it Objective to ‘be Objective’ about Sustainable Business Metrics?: Part XIV,” Sustainable Brands, http://www.sustainablebrands.com/news_and_views/new_metrics/matt_polsky/....
A difficult challenge, but maybe we need to go there, even apart from the subject of this article.
So am I to believe that now my PC is a homophobic, xenophobic, misogamist, and bigoted racist? Is there some kind of rehab or reeducation camp I can sent it to so it will realize the error of its ways?
1
So much for “do no evil”.
5
We all have biases, and there is nothing government can or should do about ordinary biases. What we need to be concerned about is bigotry. The point at which government has a legitimate concern is only when a bias becomes bigotry and not before.
2
Yes, anyone I describe as a bigot (this includes most liberals) must be silenced!
What is bigotry if not a system that steers me away from actual, specific jobs because of my race/gender/socioeconomic class? These systems have real effects on real people, and just because they aren't in-person acts doesn't lessen their effects on my life prospects.
1
Confidence in algorithms is inversely proportional to the denial of our own cognitive dissonance. Computers will be always handicapped by logic, like human beings by desire.
3
Why blame it on the algorithm when we know that advertisers can explicitly pick their target audience? Would we be surprised if women happened to see more ads for lipstick? The default assumption should be that it was the advertiser's choice until there is proof that the advertiser didn't include gender in their target profile.
1
In a deeper sense, discrimination is the purpose of these algorithms. Not discrimination based on group identity, but based on my behavior. Of course "discrimination" in that sense is normally unproblematic: you walk into my shoe store, ask to buy shoes, and based on your behavior I sell you shoes. But the discrimination exerted by the internet is fundamentally different: the marketer is responding not to my interaction with him, but to clandestine observations of my behavior with others, and he is reinforcing my behavior by changing the offers & the environment that I encounter.
The reason we object to discrimination in the first place is that we believe that, as far as possible, we should all live in the same world, with comparable opportunities & challenges. We believe in freedom, which entails individuality & unpredictability. But we have structured the internet to pigeonhole people based not on interests they express, but on statistics or stereotypes derived from their past behavior--which may reflect any number of arbitrary constraints. It not only predicts behavior, but it makes behavior more predictable. Of course the algorithms don't prevent me from doing something unpredictable, but they do shape my experience in order to make me more predictable & profitable, rather than more free.
The reason we object to discrimination in the first place is that we believe that, as far as possible, we should all live in the same world, with comparable opportunities & challenges. We believe in freedom, which entails individuality & unpredictability. But we have structured the internet to pigeonhole people based not on interests they express, but on statistics or stereotypes derived from their past behavior--which may reflect any number of arbitrary constraints. It not only predicts behavior, but it makes behavior more predictable. Of course the algorithms don't prevent me from doing something unpredictable, but they do shape my experience in order to make me more predictable & profitable, rather than more free.
9
The only solution is a new law (or, in our present situation, Executive Order) requiring that rich people and homeless people receive equal information regarding the purchase of Maseratis and Ferraris. Anything else is obviously discrimination by those racist Republicans.
19
Excellent sarcasm (which often doesn't play well) - and I mean that sincerely!
An algorithm is a set of instructions to produce a given result. It's only as good as the analysis of the person that designed and implemented it. That person's analysis may not be adequate, for example sending ads for high paying executive positions to well paid persons in the field will perpetuate the status quo if there differential pay for the same job between men and women, say. The advertiser's targeting instructions may be a problem.
The suggestion that simulations should be used to test algorithms is the best one because even if an algorithm is "designed from scratch to be aware of values and not discriminate" unanticipated consequences can be expected. Data may be objective but it must be interpreted/understood to be useful, which re-introduces the human factor.
The suggestion that simulations should be used to test algorithms is the best one because even if an algorithm is "designed from scratch to be aware of values and not discriminate" unanticipated consequences can be expected. Data may be objective but it must be interpreted/understood to be useful, which re-introduces the human factor.
3
At least when it's less targeted, humans can think more for themselves.
I'll wager they can do this any time.
This is nothing new, except perhaps to the New York Times. (Not to the Wall Street Journal, which has carried a number of fine articles on this topic.) After all, it's people who write the algotithms, and when it comes to such things as job searches and applications for schooling, loans or housing they're typically DIRECTED to make sure that certain types of people - the ones their clients prefer - get the 'edge'. (Read: 'We're in the business of weeding out those who don't conform to our ideals, whatever THOSE might be.' And you wonder why the morale of job-seekers, if it went ten miles higher, would still be below the surface of the Dead Sea?)
But you needn't take this from me. Just check out the Medical Quack website and see for yourself. Its originator and blogger, a lady named Barbara Duck (You CAN'T make that up), wrote code for years and invented the term 'algo-duping' to describe just what sort of ridiculousness and downright wrong ideas come from full reliance on these things. Then again, SHE once was suspended from her Google account, apparently because her name was deemed impermissible; as she told the story on her site, "Perhaps they thought I was a real duck."
The algorithms may be based on the theorem of Bayes, but the results have been looking more and more like something concocted by Rod Serling.
But you needn't take this from me. Just check out the Medical Quack website and see for yourself. Its originator and blogger, a lady named Barbara Duck (You CAN'T make that up), wrote code for years and invented the term 'algo-duping' to describe just what sort of ridiculousness and downright wrong ideas come from full reliance on these things. Then again, SHE once was suspended from her Google account, apparently because her name was deemed impermissible; as she told the story on her site, "Perhaps they thought I was a real duck."
The algorithms may be based on the theorem of Bayes, but the results have been looking more and more like something concocted by Rod Serling.
18
The problem with search algorithms is that they are based on Bayesian statistics which means they amplify conventional expectations. Bayesian statistics are about manipulating dependent probabilities: if you know A, what now are the odds of B as opposed to the case where that knowledge was lacking.
Data mining, basically tallying up past behavior, is used to build what are called "the priors", that is, the expectations and their weightings, so if most men have higher paying jobs than women, this fact is baked into the algorithm. The result will be that if the algorithm knows A, so and so is a woman, then B, having a well paying job is less likely. The result is that a woman will not be shown higher paying jobs. This is a property of the algorithm. It works the same way a spelling program might turn "giat" into "goat" rather than "geat", because "goat" is a more commonly used word. (Beowulf might have been a Geat, but how many people read Beowulf.)
In the case of spelling correction, there is a bias towards the conventional, which is why I turned off that stupid word completion stuff on my iPhone. It always guessed wrong. In the case of people looking for a job, the use of such an algorithm is simply discriminatory. It is no different from the old fashioned division of job listings into male and female categories, a practice long discontinued.
Data mining, basically tallying up past behavior, is used to build what are called "the priors", that is, the expectations and their weightings, so if most men have higher paying jobs than women, this fact is baked into the algorithm. The result will be that if the algorithm knows A, so and so is a woman, then B, having a well paying job is less likely. The result is that a woman will not be shown higher paying jobs. This is a property of the algorithm. It works the same way a spelling program might turn "giat" into "goat" rather than "geat", because "goat" is a more commonly used word. (Beowulf might have been a Geat, but how many people read Beowulf.)
In the case of spelling correction, there is a bias towards the conventional, which is why I turned off that stupid word completion stuff on my iPhone. It always guessed wrong. In the case of people looking for a job, the use of such an algorithm is simply discriminatory. It is no different from the old fashioned division of job listings into male and female categories, a practice long discontinued.
27
Job ads should be targeted to people with the strongest indicators of interest and qualification for that job. If those people, targeted based on those non-gendered indicators, happen to be majority male (because society currently has more qualified male applicants in that field than females ones) then so be it. That's society's problem and it's not obviously an algorithm's responsibility to fix it.
BUT, if the exact same outcome results from the opposite chain of events (the algorithm actually explicitly targets males because males have been assumed to be more likely to have the qualifications and interest in that job) that is gender discrimination and not acceptable.
The Carnegie Melon study is interesting because the "participants" were automated bots, not actual men and women. This allowed their online identities to be made the same in EVERY way except for gender, which was specified under their profile's google settings. The bots were programmed to automatically search a list of the top 100 employment websites. Based only on this identical search history and the bot's gender, Google showed a specific ad for a high paying job 1852 times to the male group but just 318 times to the female group.
I haven't read all of the study in detail, so there may well be other points to quibble with. But at least in terms of demonstrating that google can target job related ads based solely on gender -- a legally protected attribute -- this study seems to have definitively succeeded.
BUT, if the exact same outcome results from the opposite chain of events (the algorithm actually explicitly targets males because males have been assumed to be more likely to have the qualifications and interest in that job) that is gender discrimination and not acceptable.
The Carnegie Melon study is interesting because the "participants" were automated bots, not actual men and women. This allowed their online identities to be made the same in EVERY way except for gender, which was specified under their profile's google settings. The bots were programmed to automatically search a list of the top 100 employment websites. Based only on this identical search history and the bot's gender, Google showed a specific ad for a high paying job 1852 times to the male group but just 318 times to the female group.
I haven't read all of the study in detail, so there may well be other points to quibble with. But at least in terms of demonstrating that google can target job related ads based solely on gender -- a legally protected attribute -- this study seems to have definitively succeeded.
17
Algorithms now provide "clinical decision support" to doctors. These are increasingly built into the software a doctor's employer provides to the doctor. The software is secret and it's purchased by the institution, not the doctor. When the software is designed, does it maximize revenue for the hospital or what's good for that patient? How are we to know? In the paper days, medicine was all open source and peer-reviewed. Now, decision support software is secret and it's not regulated because the doctor is "in the loop". However, the doctor is now subject to the vendor's or the hospital's bias.
14
Yes. This is terrible. Amd another example of profits before people. Monetize before truth. Science only it is a handmaiden (or butler) to economics.
You used to be able to (generally) trust your physician because he/she took a professional other to "do no harm" but that is superceeded by an employment contact which says "make more money." Or even "perception is reality", there is no such thing as "science", except as a commodity.
You used to be able to (generally) trust your physician because he/she took a professional other to "do no harm" but that is superceeded by an employment contact which says "make more money." Or even "perception is reality", there is no such thing as "science", except as a commodity.
Clinical decision support is designed to assist physicians at the point of care. Doctors do have a say in what alerts they want to use and are not forced to act against their judgement just because of an algorithm. An example of a clinical decision support would be alerting the doctor if they had e-prescribed a drug that was contraindicated with another drug already listed in a patient's chart. Some systems allow the user to create their own alerts based on their patient's needs. It's certainly not a perfect system, but I think your fear that it is designed to maximize hospital revenue may be off-base....
Algorithms don't discriminate. They are simply recipes for creating code that solves a problem. They also don't "decide" or "make choices". The code written to the algorithms specification performs calculations on data and returns a result.
The crux of the issue is that the author wants code to solve a problem that is even more complex than the problems Google constructed code to solve. So:
"take the input data (users profile, web history, etc...); weighted advertiser preferences; perform calculations; return highest scored ad"
becomes:
"take input data, perform calculations, then filter the result through some type of anti discrimination code that will perform ad substitution in accordance with the new societal imbalance algorithm."
I'm sure Google's working on that type of filtering technology, but it would be a bewilderingly complicated endeavor - especially considering that they have to factor advertiser preferences into the mix. What happens when these preferences are in conflict with desireable societal aims?
We are witnessing yet another manifestation of the collision between advancing technology and the deeper societal dynamics it impacts. As it stands now, Googles search functionality is not yet advanced enough to adequately mitigate the externalities created by... Googles search functionality.
The crux of the issue is that the author wants code to solve a problem that is even more complex than the problems Google constructed code to solve. So:
"take the input data (users profile, web history, etc...); weighted advertiser preferences; perform calculations; return highest scored ad"
becomes:
"take input data, perform calculations, then filter the result through some type of anti discrimination code that will perform ad substitution in accordance with the new societal imbalance algorithm."
I'm sure Google's working on that type of filtering technology, but it would be a bewilderingly complicated endeavor - especially considering that they have to factor advertiser preferences into the mix. What happens when these preferences are in conflict with desireable societal aims?
We are witnessing yet another manifestation of the collision between advancing technology and the deeper societal dynamics it impacts. As it stands now, Googles search functionality is not yet advanced enough to adequately mitigate the externalities created by... Googles search functionality.
3
"What happens when these preferences are in conflict with desireable societal aims?" We get another example of what's wrong with capitalism, especially market-driven consumer capitalism. Google's public motto may be "do no evil," but as a private corporation, its real motto is "make more money."
pretty sure NBA teams and colleges have algorithms to id potential college/AAU players. Also sure this results in zero women being chose and Afr. Am. being selected at a much greater percentage than their representation in the population. Isn't that just perpetuating old stereotypes too?
4
This is so funny, now using statistics is discriminatory. If one points out that Blacks admitted to the Ivy League usually have SAT test scores 450 points lower then Asians you are called a racist. If one points out that 6% of the population (Black males) account for 50% of total US homicides you are called a racist. So why is it any different that computer algorithms, based on statistical probabilities, will not be called racist as well. It seems in our Politically Correct world any kernel of truth is to be called racist if the narrative does not fit the liberal propaganda.
72
I don't know... maybe the context in which a person introduces specific stats makes a difference in terms of how people perceive the introducer. Also the way he frames his stats to make his point within that context. For instance, a person could say 6% of the population (Black males) account for 50% of total U.S. homicides - and we get an impression of Black males. Then again, using the same data - a person could say that in a given year, out of a population of roughly 20 million (Black males), roughly 6000 murders were committed (by said population, almost entirely against said population). Gives a different impression of the vast majority of Black males. So, outside of an appropriate context - sure I could see why someone would question the motivation of someone blithely offering up the stat you chose.
7
Political correctness is about to ruin our country. I have no problem if you say to me Asians score 450 points higher than white kids on their SAT's. IF those are the facts they are the facts. The main issue is how do we fix that? If we are afraid of hurting feelings with statistics we will never fix anything.
1
Why would anyone call you a racist for pointing out things like that? Are you sure that the next sentence out of your mouth is not the one people think is racists?
What would happen if millions of people decided to coordinate a bunch of useless information intended to the full companies to believe that they're searching for their product when in fact they are doing the opposite?
2
Discriminate? Look up that word in a dictionary that was published before 1970 -- that's what they're supposed to do.
Now then, what if the algorithm just calls it as it sees it?
Now then, what if the algorithm just calls it as it sees it?
4
Algorithms don't call it as they see it. They attempt to call it as they think you see it. It's not objective.
It's a psychological truism that people fulfill expectations. If the world they access through search is expecting them to be a certain way, and feeding information accordingly, isn't that making it harder for all of us to be ourselves?
It's a psychological truism that people fulfill expectations. If the world they access through search is expecting them to be a certain way, and feeding information accordingly, isn't that making it harder for all of us to be ourselves?
1
Computer science is a huge industry: there are millions of people who understand it. Can the NY times please get some of those people to write articles about computer science topics? An algorithm is a clear and detailed description, written is a way designed for humans to read and understand, of how to solve a given problem. An algorithm is, almost by definition, not a 'black box'. If you want to know what an algorithm is intended to do, you just ask "what problem is this algorithm meant to solve?".
Since google is paid by the advertiser each time a user clicks on the ad they have been shown, it is obvious that google's algorithm is designed to solve the problem "show users the ads that they are most likely to click on". The standard way to do this is to gather as much past information you can about every individual interaction: the search terms used, cookie history, and ads that were clicked and not clicked. To allocate an ad to someone doing a search, the algorithm gets the current search terms, cookie history etc. and then asks "in my database of past information, when the current search term, cookie history etc. was present, what ads were most likely to be clicked on?". Those ads are the ones shown.
If you want to change the behavior of this algorithm, you have to either change the problem it is designed to solve (don't show people ads they are most likely to click on; instead show ads you think they should be clicking on), or get women to click on CEO ads.
Since google is paid by the advertiser each time a user clicks on the ad they have been shown, it is obvious that google's algorithm is designed to solve the problem "show users the ads that they are most likely to click on". The standard way to do this is to gather as much past information you can about every individual interaction: the search terms used, cookie history, and ads that were clicked and not clicked. To allocate an ad to someone doing a search, the algorithm gets the current search terms, cookie history etc. and then asks "in my database of past information, when the current search term, cookie history etc. was present, what ads were most likely to be clicked on?". Those ads are the ones shown.
If you want to change the behavior of this algorithm, you have to either change the problem it is designed to solve (don't show people ads they are most likely to click on; instead show ads you think they should be clicking on), or get women to click on CEO ads.
61
Anupam Datta, one of the Carnegie Mellon study authors, avers, "Given the big gender pay gap we’ve had between males and females, this type of targeting helps to perpetuate it."
This sounds more like social-agenda driven sloganeering than it does serious academic research. There appears to be nothing in the study to back up such an assertion.
Shame on Carnegie Mellon.
This sounds more like social-agenda driven sloganeering than it does serious academic research. There appears to be nothing in the study to back up such an assertion.
Shame on Carnegie Mellon.
42
Great Post "Whatever"
What I'm wondering is how long it is going to be, before our culture will concede that there no longer is
A GENDER PAY GAP.
It is very well established, that after you account for the different employment choices that the two genders make, this gap is thought to be maybe 4.8-7.1% (US Dept-Labor@wiki)
Since Females are now graduating college more frequently & faster than males, by a good clip
. . . soon it will be us, lads, who are getting taken by the lassies.
INTERESTINGLY ENOUGH, though:
"the pay gap is greater for older women than younger women"
What I'm wondering is how long it is going to be, before our culture will concede that there no longer is
A GENDER PAY GAP.
It is very well established, that after you account for the different employment choices that the two genders make, this gap is thought to be maybe 4.8-7.1% (US Dept-Labor@wiki)
Since Females are now graduating college more frequently & faster than males, by a good clip
. . . soon it will be us, lads, who are getting taken by the lassies.
INTERESTINGLY ENOUGH, though:
"the pay gap is greater for older women than younger women"
Funny, you touch on the opposite two sides of the same coin depending whether you're talking from the female perspective or the male.
You make the point that if you look at the same job, the men and women in it make pretty similar pay.
But then when you say "soon it will be us, lads, who are getting taken by the lassies" you imply that women will be getting higher paying jobs then men, rather than caring that presumably the men who do manage to get the same jobs, in this hypothetical world where women are getting higher ranked jobs then men, will still be getting the same pay -- there will just be fewer of them.
Either stance is fine on it's own, and I know the second comment was meant jokingly, but I still found it an interesting switch in logic based on the gender in question.
You make the point that if you look at the same job, the men and women in it make pretty similar pay.
But then when you say "soon it will be us, lads, who are getting taken by the lassies" you imply that women will be getting higher paying jobs then men, rather than caring that presumably the men who do manage to get the same jobs, in this hypothetical world where women are getting higher ranked jobs then men, will still be getting the same pay -- there will just be fewer of them.
Either stance is fine on it's own, and I know the second comment was meant jokingly, but I still found it an interesting switch in logic based on the gender in question.
Of course there is still a pay gap. One of the more glaring examples occurs in nursing, a traditionally female occupation. http://well.blogs.nytimes.com/2015/03/24/stubborn-pay-gap-is-found-in-nu....
1
Were men and women in question visiting the same site?
What Google is good at is targeting the ads based on the browser history as it sees it. Some ads are linked to Wired, some - to LinkedIn, and so on. I probably see a higher amount than normal of ads for the 100k+ jobs because I work in one, and these ads are targeted to the industry.
What Google is good at is targeting the ads based on the browser history as it sees it. Some ads are linked to Wired, some - to LinkedIn, and so on. I probably see a higher amount than normal of ads for the 100k+ jobs because I work in one, and these ads are targeted to the industry.
5
This is an excellent question and gets to the heart of the moral problem I think. The answer is that "men" and "women" had the exact same viewing history in the Carnegis Melon study.
This is because the experiment was conducted using bots that made artificial google accounts, set their gender to male or female in settings, and then searched a list of the top 100 employment websites. Google then showed the high paying job ad to 1852 times to the "male" group of bots and only 318 times to the female group.
I haven't read the whole study so I can't speak to other quality controls (e.g. did they define "high paying job ads" as the indicator of discrimination before hand, or interpret as such after it showed up with higher odds for male bots) but at least on controlling for ALL differences between males and females other than their sex, this study certainly was quite rigorous.
This is because the experiment was conducted using bots that made artificial google accounts, set their gender to male or female in settings, and then searched a list of the top 100 employment websites. Google then showed the high paying job ad to 1852 times to the "male" group of bots and only 318 times to the female group.
I haven't read the whole study so I can't speak to other quality controls (e.g. did they define "high paying job ads" as the indicator of discrimination before hand, or interpret as such after it showed up with higher odds for male bots) but at least on controlling for ALL differences between males and females other than their sex, this study certainly was quite rigorous.
2
So more men than women have the type of pathetic search histories that lead Google's algorithms to believe that they'd be likely to click on an ad making bogus promises about high paying jobs? And this is supposed to indicate that men are getting all the breaks? Sure, men have advantages, but seeing spam advertisements isn't one of them. Granted, this type of thing might become a more serious problem in the future though.
6
@Nina: "... this type of thing might become a more serious problem in the future though."
It already is. News articles may be displayed based on a user's history, location, and what other users have viewed.
How algorithms decide the news you see
By Jihii Jolly
May 20, 2014
http://www.cjr.org/news_literacy/algorithms_filter_bubble.php
It already is. News articles may be displayed based on a user's history, location, and what other users have viewed.
How algorithms decide the news you see
By Jihii Jolly
May 20, 2014
http://www.cjr.org/news_literacy/algorithms_filter_bubble.php
The "Right to Forget" law imposed by Spain on Google needs to be adopted in this country. I know of someone who was president of the university in Texas and who was falsely accused of wrong doing by an alumnus and large donor of the University. He was forced to resign from the University because the regents did not want to loose the support of this alumnus. The action made the headlines in the local papers, never mentioning the false accusations. He was exonerated. Even though the event happened a year and a half ago, whenever his name is searched on Google the first items that show up are the inaccurate newspaper stories. He's an academic with a PhD in education and an authority on Hispanic students issues relating to entry into colleges and universities. He's written numerous blogs on those issues on Blogspot, owned by Google, and they hardly show up on a search. The bottom line is that he is looking for a new position and has lost several opportunities, because perspective employers have searched him on the Internet and the inaccurate stories on him constantly appear at the top of the search. I hope a major law firm will take up a class-action lawsuit on behalf of thousand of Americans who are in a similar situation and advocate for a "Right to Forget" law in the United States.
2
I couldn't disagree more. Such a law would allow people to cover up their history of crimes, frauds, and lies.
What, he's too stupid to mention this to employers when he applies for a position? Maybe they didn't hire him because they figured he should be internet savvy enough to realize he needed to disclose this history when applying?
Well, of course. Algorithms are first designed by humans. Worst of all, often humans who have no training in psychology, sociology, etc. Same GIGO as always. If the orginiators don't reallty know enough about human behavior, the algprithm certainly won't. These just implement to processes designed by humans who are more likely to be mathematicians or technies than social scientists with training in both quantitaive - and qualitative - research. It is one of the reasons I don't like algorithms. And, what if you are an "outlier"? Then, too bad for you. Stats are usually a measure of central tendency; wide ranges and outiers are just filed down or ignored. That may be more relevant to a social scientist than a stat techie. Same GIGO as always.
3
You have no idea what are you talking about. This world perfect. Algorithms reflect whatever the reality is. Google doesn't hardcode "transgender this...". Algorithms learn from book, queries, blogs whatever. They don't differentiate. Algorithm learning comes from evidence. Fact is not subjective, it is what what it is. I believe all branches of science learn from evidence or experiments including all the "logies" you mentioned.
3
RK: What is "reality"?
1
prejudiced humans
1
"Algorithms, which are a series of instructions written by programmers, are often described as a black box; it is hard to know why websites produce certain results."
That isn't really what's meant by the term "black box".
Computer science is sometimes described as the study of abstraction. Computer programs are highly complex entities, too complex to understand all at once. Accordingly, the practice of computer programming revolves around abstracting complex processes into single, simple entities.
For example, computing the average of a list of numbers involves adding together the list of numbers, and then dividing that sum by the quantity of numbers which were in a list. This simple algorithm involves additional overhead when implemented on a computer, like creating storage for the values, etc. Altogether, you're looking at 5 or 6 lines of code.
Furthermore, programming languages give you tools to name that averaging process, so that when you need the process again, you simply use it's name rather than typing out the same 5 lines of code. That name is an abstraction: you can think about it as the simple result it produces rather than the complex steps it uses. Because we no longer care how the process gets its answer, just that the answer is correct, we say that it's a "black box", i.e. it's underlying machinery is concealed, facilitating its use in a complex program.
The black box abstracts away complexity; it does not mean we don't understand how output is computed.
That isn't really what's meant by the term "black box".
Computer science is sometimes described as the study of abstraction. Computer programs are highly complex entities, too complex to understand all at once. Accordingly, the practice of computer programming revolves around abstracting complex processes into single, simple entities.
For example, computing the average of a list of numbers involves adding together the list of numbers, and then dividing that sum by the quantity of numbers which were in a list. This simple algorithm involves additional overhead when implemented on a computer, like creating storage for the values, etc. Altogether, you're looking at 5 or 6 lines of code.
Furthermore, programming languages give you tools to name that averaging process, so that when you need the process again, you simply use it's name rather than typing out the same 5 lines of code. That name is an abstraction: you can think about it as the simple result it produces rather than the complex steps it uses. Because we no longer care how the process gets its answer, just that the answer is correct, we say that it's a "black box", i.e. it's underlying machinery is concealed, facilitating its use in a complex program.
The black box abstracts away complexity; it does not mean we don't understand how output is computed.
6
I think 'black box' is being used to refer to the statistical properties of machine learning algorithms, not CS issues like memory storage. For example, it's not always clear what predictors cause a complicated predictive algorithm like a random forest to decide that a user is likely to click on a certain ad.
5
When an algorithm produces a result that conforms to stereotypes it is because these types of search algorithms are designed to mimic human behavior. They are a reflection of human behavior rather than a perpetuator of it. Faulting algorithms for encouraging stereotypes feels like a shift of blame. Computer scientists (I am one) see themselves as a form of mathematician. We create formulas to solve a problem, and it feels like a perversion to modify those formulas to reduce bias that is not the fault of the formula. When Google presents the image results for CEO it displays the ones that you are most likely to click on. I do firmly believe that we struggle with massive gender and race based inequalities in this country, I just don't think that the way to deal with this is to make one and one equal three because two has been over represented.
51
If the reflection is the only option offered, it perpetuates itself and encourages it in others.
1
Good people use conscious choices to overcome involuntary biases. It is not unreasonable to ask our technology could to do the same.
But it is as difficult and nuanced a question as achieving diversity in schools and workplaces or integration in housing. It raises as many ethical questions as where to draw the line on hate speech.
This article facilely dismisses the issue by condemning those who release imperfect technology as "not thinking about the issue."
It is a difficult question, and there will be no answers we all agree on.
But it is as difficult and nuanced a question as achieving diversity in schools and workplaces or integration in housing. It raises as many ethical questions as where to draw the line on hate speech.
This article facilely dismisses the issue by condemning those who release imperfect technology as "not thinking about the issue."
It is a difficult question, and there will be no answers we all agree on.
1
Search algorithms are designed to pick up on biases in underlying data streams. That an algorithm shows discriminatory results is merely evidence that the society that is producing the algorithm's underlying data stream is discriminatory. Getting rid of discriminatory results would require programming to explicitly do so.
3
The internets are racismist and sexismist! We all knew it!
2
This article is completely clueless. The algorithms - i.e., the machines - are built to "learn" how humans operate in real life and do it for them, thus saving time. When you observe the machine reproducing biased stereotypes, it is simply telling you what you (human beings) actually are. If you have to rewrite each algorithm to artificially change its results in individual cases, then the point of having an algorithm - to save time - is lost.
In addition, you will not then be "eliminating" something (discrimination) from the social process, you will be adding something - artificial, machine-enforced preferences for specific ethnicities and gender - that didn't exist before. Once the programmers get to do this without anyone watching, subject to no legal review, you will have finally thrown civil liberties out the window.
In addition, you will not then be "eliminating" something (discrimination) from the social process, you will be adding something - artificial, machine-enforced preferences for specific ethnicities and gender - that didn't exist before. Once the programmers get to do this without anyone watching, subject to no legal review, you will have finally thrown civil liberties out the window.
66
I don't thing Tony is describing the case accurately.
To "save time" (a questionable assumption - I think the goal is primarily to provide desired data), algorithms are already perpetuating assumptions based on demographics. Tools designed for marketing are making decisions on our behalf, based on superficial similarities to other searchers.
The solution is not more targeting, but less.
To "save time" (a questionable assumption - I think the goal is primarily to provide desired data), algorithms are already perpetuating assumptions based on demographics. Tools designed for marketing are making decisions on our behalf, based on superficial similarities to other searchers.
The solution is not more targeting, but less.
1
In addition, you will not then be "eliminating" something (discrimination) from the social process, you will be adding something - artificial, machine-enforced *discrimination* for specific ethnicities and gender.
In other words, you would be programming political correctness into the algorithm.
In other words, you would be programming political correctness into the algorithm.
1
Mr. Longo is right. instead of artificially patching up algorithms for their political incorrectness, lets fix the underlying problems: that African Americans live in cyclical poverty, that woman work in less well paying jobs.. etc etc.
1
@Upshot: 'A recent Google search for “Are transgender,” for instance, suggested, “Are transgenders going to hell.”'
Google gives *four* suggestions, so citing only one example is misleading. And that suggestion is a valid theological question.
When I start with "are reporters", one of the suggestions is "are reporters biased", but none are "are reporters going to hell".
Presumably, Google users ask different questions about transgenders and reporters, and the suggestions reflect that. So what is the problem?
Google gives *four* suggestions, so citing only one example is misleading. And that suggestion is a valid theological question.
When I start with "are reporters", one of the suggestions is "are reporters biased", but none are "are reporters going to hell".
Presumably, Google users ask different questions about transgenders and reporters, and the suggestions reflect that. So what is the problem?
24
The problem doesn't lie with the algorithms. The problem lies with people who use bogus statistics in their endless quest to stir up gender and race hatred.
Take Miller's example of "research" from the University of Washington. Miller reports that it found that "a Google Images search for “C.E.O.” produced 11 percent women, even though 27 percent of United States chief executives are women."
27 percent of CEOs are women? Catalyst reports that women currently hold 4.6% of CEO positions at S&P 500 companies.
When people visualize "C.E.O."s, they usually have in mind the heads of quite large companies, not their local hardware or jewelry store. So the reported finding of discrimination was in fact generated by the researchers' manipulation of what we commonly understand by C.E.O. (or possibly by their sheer incompetence).
Take Miller's example of "research" from the University of Washington. Miller reports that it found that "a Google Images search for “C.E.O.” produced 11 percent women, even though 27 percent of United States chief executives are women."
27 percent of CEOs are women? Catalyst reports that women currently hold 4.6% of CEO positions at S&P 500 companies.
When people visualize "C.E.O."s, they usually have in mind the heads of quite large companies, not their local hardware or jewelry store. So the reported finding of discrimination was in fact generated by the researchers' manipulation of what we commonly understand by C.E.O. (or possibly by their sheer incompetence).
80
And regardless of the percentages of CEOs of each sex, there's no reason to assume that online photos of each sex of CEO are divided in the same proportion.
2
The researcher states pretty clearly that they are using Bureau of Labor Statistics data. They probably don't care what you "visualize" your ideal C.E.O. to be.
1
I first thought this must be the apotheosis of the discrimination mania gripping our media and political culture. But why should that be? When every difference and anomaly is conflated with 'bias', the possibilities are infinite.
12
Mathematical formulas (executed by means of computer program) discriminate. Giving wordsmiths and lawyers a chance to muck things up by claiming evil intent by the mathematicians, computer programmers or both. If turning another computer program loose to find extreme cases or statistical nits has its day in court the class action trial lawyers will own the world.
Filters that adapt to data fed through them by change over time and data stream what gets through and what does not. Discriminating the useful from the harmful is what filtering algorithms have always been about.
Filters that adapt to data fed through them by change over time and data stream what gets through and what does not. Discriminating the useful from the harmful is what filtering algorithms have always been about.
2
Nice piece Claire. I agree with almost all if it although I would take issue with the notion that algorithms are necessarily discriminatory. At our company (Entelo) we've created an algorithm to help companies with recruiting members of under-represented groups. For example, if a company has been having a hard time finding women to join their engineering team or difficulty recruiting African-Americans to join their company, they can use our algorithmic approach to better identify job candidates who would have the right skills and experience and be more likely to be a member of an under-represented group.
I'd be happy to discuss it with you more anytime!
I'd be happy to discuss it with you more anytime!
1
You produce software that discriminates among job applicants based on gender and race?
23
So basically, your company has created a discriminatory algorithm by programming in bias to specific groups of people.
4
I do all kinds of searches on all kinds of topics because I'm a curious person. But most of my searches have absolutely nothing to do with my personal life. And I do searches for non-computer literate friends who want me to do research for them.
And even though I don't see ads (because I use software that blocks out as many ads as possible), when I do get ads, they almost never -- no, make that they ARE never -- for any product that I want to see.
And I find it HI-LAR-I-OUS how often my bleeding-heart-liberal eyes are offered ads from right-wing Republican politicians (who will only get my vote if I'm voting from a turning B-B-Q spit).
Oh, and those "recommended" movies from Netflix are 90% wrong, too.
Algorithms might work on impressionable tweeers and arrested-development adults with only two brain-cells to pass among them for their use, but for thinking adults? They are hopelessly inaccurate. Why any thinking business person would do business with companies that employ them, well, pretty much explains the American corporate mind-set (or lack, thereof).
And even though I don't see ads (because I use software that blocks out as many ads as possible), when I do get ads, they almost never -- no, make that they ARE never -- for any product that I want to see.
And I find it HI-LAR-I-OUS how often my bleeding-heart-liberal eyes are offered ads from right-wing Republican politicians (who will only get my vote if I'm voting from a turning B-B-Q spit).
Oh, and those "recommended" movies from Netflix are 90% wrong, too.
Algorithms might work on impressionable tweeers and arrested-development adults with only two brain-cells to pass among them for their use, but for thinking adults? They are hopelessly inaccurate. Why any thinking business person would do business with companies that employ them, well, pretty much explains the American corporate mind-set (or lack, thereof).
10
Bless you! Now can we get rid of the NYT device that "suggests" stories for me? You don't know me from Adam.
8
Until recently all algorithms were coded by hand, now with machine learning having made reasonable progress, algorithms are coded by algorithms or rather programs learn from the data they are presented with. We were used to getting exact results from software programs (correct or otherwise) but now since we have programs that learn and modify its algorithms, we see how error prone this learning process can be. Results provided by such algorithms can never be exact (because by its very nature, learning is error prone). Recently we saw two examples of how error prone it is - Google’s algorithm tagged a black couple as gorillas and a search for top criminals returned images of the PM of India. It’s not the fault of these learning algorithms, it’s natural for them to deliver imprecise results.
We need software systems to be clearly delineated, right now we have both coexisting in the same sphere which can be confusing for us when we are presented with results that is inexact when we expect it to be precise.
We need software systems to be clearly delineated, right now we have both coexisting in the same sphere which can be confusing for us when we are presented with results that is inexact when we expect it to be precise.
3
What gives you the idea that this is an error at all?
There is no evidence of such. I would expect ads targeting people with an arrest record to target blacks because blacks are vastly more likely to have an arrest record - 1/4th of black males end up in jail or prison at some point, compared to closer to 1/20th of whites.
That's logical.
Likewise, high-interest loans are only of interest to poor people - they're specifically designed for high-risk individuals. That's why they have higher interest rates.
These people don't have a clue what they're whining about.
CEOs show men? Most of the most famous CEOs are men. Search for Applejack and you'll find a pony as well as an alcoholic beverage.
Ads are allowed to target various groups. The real question is whether or not the ad is specifically targeting men (in which case, why are women showing up?) or if it is targeting based on other things. What kind of job was it? If it was an engineering job, you would expect it to show up about 5x more often for men than women due to, well, there being a lot more male engineers.
There is no evidence of such. I would expect ads targeting people with an arrest record to target blacks because blacks are vastly more likely to have an arrest record - 1/4th of black males end up in jail or prison at some point, compared to closer to 1/20th of whites.
That's logical.
Likewise, high-interest loans are only of interest to poor people - they're specifically designed for high-risk individuals. That's why they have higher interest rates.
These people don't have a clue what they're whining about.
CEOs show men? Most of the most famous CEOs are men. Search for Applejack and you'll find a pony as well as an alcoholic beverage.
Ads are allowed to target various groups. The real question is whether or not the ad is specifically targeting men (in which case, why are women showing up?) or if it is targeting based on other things. What kind of job was it? If it was an engineering job, you would expect it to show up about 5x more often for men than women due to, well, there being a lot more male engineers.
21
While I think this article overstates the responsibility (not to mention capability) of algorithms to correct rather than reflect societal issues -- I think the comment that it's appropriate for engineering jobs to target men because there are way more male engineers than females ones captures exactly why simply reflecting society's current state is problematic.
There was a time when men vastly outnumbered women in colleges. Clearly in retrospect it has turned out that women are just as interested in and capable of getting a college degree. Therefore, if colleges had rationalized that it was okay to only recruit men because only men attend college (I have no idea whether any did), they would have been using an unequal system to justify perpetuating inequality. I imagine the same will turn out to be true of engineering -- most engineers are males, because our system expects engineers to be males, rather than vise verse.
Returning to algorithms -- it's not immediately obvious to me what responsibility we can foist on the designers of ad targeting algorithms. As a first stab, I would venture that ads should not be targetable based explicitly on characteristics that are legally protected against discrimination. However, if targeting "people whose search habits indicate an interest in engineering" targets mostly men because those two characteristics are currently correlated, then that's not the ad-maker's responsibility.
There was a time when men vastly outnumbered women in colleges. Clearly in retrospect it has turned out that women are just as interested in and capable of getting a college degree. Therefore, if colleges had rationalized that it was okay to only recruit men because only men attend college (I have no idea whether any did), they would have been using an unequal system to justify perpetuating inequality. I imagine the same will turn out to be true of engineering -- most engineers are males, because our system expects engineers to be males, rather than vise verse.
Returning to algorithms -- it's not immediately obvious to me what responsibility we can foist on the designers of ad targeting algorithms. As a first stab, I would venture that ads should not be targetable based explicitly on characteristics that are legally protected against discrimination. However, if targeting "people whose search habits indicate an interest in engineering" targets mostly men because those two characteristics are currently correlated, then that's not the ad-maker's responsibility.
3
How does Google know the sex of the person at the other end of the line? I have a gmail account, but google has never asked me for my sex, and I don't use my real photo for my avatar. Maybe it can assume based on my name, but not everyone uses a real name.
2
I think you are just describing how algorithms work. They infer based on available data, such as your name. The fact that "not everyone uses a real name" just means the algorithm will get some things wrong, which is approximately the issue this article is talking about.
1
You can see what gender Google has inferred about you at https://www.google.com/settings/ads. This page also provides you the option of editing these inferences. You could also completely opt-out of behavioral advertisements.
27
It uses an algorithm to guess - more precisely, to estimate the probability - based on searches launched from your computer in the past, ads you have clicked, items you've purchased with Google Pay, and probably the content and distribution of postings you have made to Google Groups. Each search or click is associated with an array of probabilities: male vs. female, age range, income range, formal education, location, marital and parental status, hobbies and interests, religious affiliation, political leanings. A search for support stockings in women's sizes nudges your profile toward female and older. A search for a crucifix nudges it towards Catholic. Time spent at HuffPost nudges it towards liberal; time at TheBlaze towards conservative. Over time, the aggregate of your online activity builds into a profile which is definitely not 100% accurate but which is good enough to target online ads to you.
1
This is exactly why I use AdBlock in my browsers (Chrome and Firefox). I don't have to see ANY ads.
18
This could manifest itself in other ways. Loan applications, Google searches, or basically anything that involves some kind of automated decisioning.
3