It would be interesting if something like this can work
Complete drivel. Exactly what do they mean by "discrimination"? It sounds like the authors mean "discrimination" is whatever result they don't approve of.
Garbage In is Garbage Out is garbage thinking and a useless point. Any data in and garbage out as a result of a poor algorithm is what is of concern.
Republicans frequently use good data through a biased algorithm to come up with garbage gerrymandered districts out. This is an area where a provably unbiased algorithm could be constructed to generate unbiased map divisions - indeed, it is quite basic, learned in freshman computer science courses.
Statistical algorithms designed to analyze data generated by other algorithms for bias is already available. Just use it!
The problem is the bias for money and power in people. That is one algorithm that can't be removed from humans.
For the record, artificial Intelligence is the alchemy of computer science. Since the earliest days of digital processing, pundits have been promising that humanity is on the verge of unlocking the secrets of the human mind and translating them into neat little binary baskets.
Coders, on the other hand, know this to be utter nonsense. Nothing we are doing or can do with existing languages and architectures even remotely approximates the intelligence of the average invertebrate.
While current processors are exponentially faster than previous generations, nothing has really changed in the way we work with data. We still process information in isolation, without context or nuance, and today’s machines are no more capable of intuiting or innovating than the original ENIAC. Just because computers can now process billions of bytes of discrete data in a fraction of a second, does not make them intelligent.
If you need to worry about something, you can skip AI and concentrate on what the government and corporations are already doing with your readily available and criminally distributed personal data.
3
A standard reference for algorithm bias is needed to assist discriminatory outcomes. The metric system establishes the meter, second, and kilogram, international standards that civilized nations adopted.
Is it not possible to construct the binary equivalent to measure algorithms, and render a "radar operating characteristic" value for the public, or a public representative, to assess? Legislation will only enable leverage through regulatory capture.
GIGO is an axiom.
See also Sturgeon's Revelation (Law)...
1
The wonderful ability of programs to sift through databases of data and to make associations and to report things which human researchers would take a long time to determine is great. But the assumption that this will lead to a machine intelligence which will develop understanding independently of any programmer is magical thinking, it's just speculation in the imagination. Algorithms return defined results in finite steps, that is all. It is for humans to give the results meaning. Data that reflects biased previous outcomes is only going to reproduce those outcomes if the people using these wonderful tools blindly use the previous results to determine their current selections without thought.
The situation described with data used being used to perpetuate biases because the data reflects biases is misleading. The difficulty described has nothing to do with garbage in/garbage out. Garbage is false or irrelevant data being processed by algorithms giving outcomes which conform with all the valid logical conditions and procedures in the algorithm but because the data is wrong produce results that are wrong. The situation being described reflects accurate results from the data input but the results it reflects the biases in the data input. If the data entered is not biased, neither would be the output.
If the selection system is biased rather than the preparation for premed students produces a cohort that is unfairly weighted against minorities and women, then correct the selection system. Don't create an automated system that perpetuates the biased selection system. If the problem is biased preparation, then focus upon preparing all students fairly.
In this case the data in and out is perfectly fine. What it illustrates is not. That previous admissions to medical school shows biased admission rates is not false. How to remedy that cannot be achieved by changing the actual data. Either change the selection process so that it is not biased, or change the ability of the applicant pool to produce unbiased results.
3
Racism is still reflected in population studies across our whole society. That is not a problem so far as the studies are concerned. The problem is how this society has not remedied those disproportional outcomes.
If the data used for comparisons are kept current, the results should reflect the most likely outcomes, even outcomes reflecting racial inequities. As attempts are made to remedy the affects of racism, the success or failure of those efforts will be reflected in the new data.
Better to work with unpleasant truths that reflect reality than to censor them and produce distorted reports that obscure the unflattering reality.
Algorithms are just mechanisms that complete mathematical tasks in finite steps, they are nothing more.
2
Only a week back I attended an AI/ML conference where the presenters were from Google, MS, Apple, Tesla...,. One presentation from Google actually show cased racial biases built into the AI systems currently in use by law enforcement authorities, so yes regulation in such cases makes sense. Note though AI as a field is still evolving a lot of research is still happening, there are many potential applications of AI, we are just starting, we don't even know - to rush to regulate something we don't fully understand may not help. After all the credibility of those who apply AI and predict results is also at stake and so are the $$ & the businesses, regulations can wait so innovations can happen - when the AI world matures, regulations can follow.
2
I was having breakfast in Colby, Kansas this morning when Facebook suggested that I try a nearby restaurant on Malcolm X Boulevard. Colby's on the western Kansas flatland, where there's a lot of corn and wind farms. But there's certainly no Malcolm X Boulevard.
Gotta love those algorithms.
1
"Gotta love those algorithms."
Your anecdote has nothing to do with "algorithms". It has to do with whether you have location tracking enabled and with how you were connected to the internet.
Gotta love people who don't understand technology.
1
The whole problem of an expert system whose image of the world has been skewed by biased data was foreseen in the 1940s by science fiction writer Isaac Asimov, who included a story on the idea in his "I, Robot". The story was funny; the modern real-world situation isn't, particularly when people have been brainwashed to think that "computers don't make mistakes".
And the problem is not in the "algorithms", but the data.
7
@Charlesbalpha: "The problem": where is it? It's in the data, sure. But also the algorithms, which are just programs in a computer. And in the methods by which these programs were developed. If you want good algorithms you have to validate them. That means developing them using highly disciplined methods that have been studied; testing them with carefully supervised data; doing trials (like for drugs); having a system for reporting problems and adverse outcomes; the owners being legally accountable; and periodic audits and inspections. In other words, its a bit like what the FDA does with medicines. We do need regulations, but they have to be designed by real experts, not seat-of-the-pants by politicians.
2
@Charlesbalpha
I recommend most of this, but 99Percent is right about the algorithms.
Not gonna work. As the phrase says that the authors cite, this bill and this analysis are both garbage in garbage out.
The things that bake in when algorithms are developed come from how ideas are developed, which is always from small side project to pilot to small product to production to scaling up.
At the beginning of that process is a tiny project by some people who think well and work well together, using data that somebody just grabbed or shot videoed to have something to write code with. And that data is the data that is most likely to bake in.
So rather than pretending that this is some huge cultural event in which we have big company cultures subject to regulation and watchdogging possible from huge federal agencies every step of the way, we'd be far better off if the legislators involved, and the newspapers who want to publish analysis and opinions like this one, would go to little software shops to do their data gathering, not to federal agencies and to law schools and elite universities. Those people have no idea how a bottom up piece of software is born.
2
The idea that we should all be judged and identified by data that has been stolen from us, first by coordinated denial and GWOT infrastructure design and now by lawfare, is an ambitious act of class-warfare.
It is not well received. This ain't what I went to war for.
2
"Significantly, the bill does not clearly prohibit algorithmic bias or unfairness."
This is probably because the bill likely does not tackle the thorny problem of *defining* 'algorithmic bias or unfairness.' Other commenters have pointed out some of the many problems inherent in trying to outlaw 'algorithmic bias or unfairness,' but there is a better approach.
An employer can be held liable for discrimination based on an established history of discriminatory behavior against, say, African-American job applicants, so apply the same empirical criteria when the resume' screening is performed by a computer algorithm. Instead of allowing the company to deflect responsibility for the outcomes onto a dumb machine or a mediocre software engineer, however, hold the company's chief executive personally responsible. Treat the CEO as if he had personally gone through the resumes and thrown out all the graduates of Historically Black Colleges & Universities, all the applicants with names he thought "sounded black," or all the applicants with home pages that identified them as African-American.
It will probably only take one instance in which a CEO is thrown in jail for allowing his company to use software whose decisions would be considered discriminatory had they been made by a human for this problem to disappear, quickly.
2
I find it upsetting that there are many Indian and Pakistani and African doctors in our communities but very few American born blacks. It does appear that there is conscious exclusion of US born black minorities based on names and origins. Among lawyers it is even worse in that there are so few migrants at all and even fewer blacks. These are the two highest earning professions.
1
The bill is pretty much worthless because it relies on impact statements. Imagine if a manager at Facebook had written an impact statement at the time of their IPO in 2012. It certainly wouldn't have predicted Cambridge Analytica or any of the political intrigues we are now seeing.
The only way regulation is going to work is to put a rigorous data firewall between every corporation that eliminates all data transmission about consumers, except that needed fulfill a transaction, and to destroy it soon thereafter. No more buying and selling of private data.
1
There's something missing here.
The issue should never be if an algorithm is fair, racially balanced. It should be if the algorithm is correct. If the algorithm is selecting less minorities as potential medical students because fewer meet the minimum criteria for the program - that's correct, and the issue is with schools. If it's doing so because they didn't go to the 'right' (expensive) college - then it's a bias that is not correct.
There are many real statistical differences between racial groups. Algorithms should not enhance them, nor assume them, but when it comes to something like a criminal record, that is a reasonable datum to use when deciding who a bank might hire.
Machine learning is tricky, and it is all about the sample and the data the program is handed to use to learn from. To judge it's results only by race, without considering if the result was fair and correct won't help anything.
There's a common news segment, where two identical resumes are submitted to several businesses - one with a traditionally minority name, one with a traditionally white name. And the traditionally white one consistently gets far more calls. That's a very valid type of test. Checking if more black or white (or hispanic, asian, gay, jewish, and all the other minority groups that don't fit merely into black and white) people are hired does not tell the tale.
4
@SusanStoHelit It's a bit more complicated than that. Real-life example: bail. On average, people who live in 'bad' neighborhoods skip bail more often than average, because more of them have risk factors (drug addiction, etc.) So would it be fair for an algorithm (such as COMPAS) to deny you bail based on your zip code?
Even though there is a real statistical association, I would argue not. I would say that the prosecutor would need to show that you, personally, are e.g. a drug addict, not merely that you have some trait that statistically correlates with higher risk. But others might disagree. Nice to see a fellow Pratchett fan BTW.
The underlying assumption of this piece is that any differences between different groups are clearly due to racism producing flawed data and not either behavior or genetics - despite a wealth of real world evidence to the contrary.
Racism isn't what makes white people's skin burn when exposed to the sun. Racism isn't what causes Asian to be far more likely to be lactose intolerant. Similarly, we know that different cultural groups behave differently. Obesity rates in the south may be impacted by poverty, etc, but there are also differences in diet and exercise.
In other words, different population segments are different so of course the data aren't going to be identical. And in fact, if the prevailing data sets about say who is creditworthy or who might in fact make an outstanding employee, big data is best positioned to uncover that information via a free market. That is, in fact, the assertion behind this piece - that fine employees, credit-worthy borrowers, and future valedictorians are at risk of being left behind because they may not match the prevailing assumptions about what an employee, borrower, or valedictorian looks like.
But since things like race or gender aren't likely to be good predictors, other factors can and should replace them because let's be honest, the driving color in American commerce isn't white, black, or brown. It's green.
6
"For example, training an algorithm to select potential medical students on a data set that reflects longtime biases against women and people of color may make these groups less likely to be admitted."
Except that beginning in 2017, more women than men were admitted to U.S. medical schools. But don't let the facts intrude on your argument.
6
"But don't let the facts intrude on your argument."
The authors were giving a hypothetical example, although their use of the phrase "longtime biases" exposes their ignorance of the subject.
If the training data does not correspond to current conditions, a decision system will return invalid results.
And your example illustrates the problem -- medical student demographics have been changing over time, so training data needs to be updated to reflect those changes.
2
I'm afraid if we try to program "fairness" into algorithms, we'll find that the program will crash like the 787 MAX8, trying to obey one attitude sensor which is out of control. If we implement the "I give up" algorithm coming soon to the 787 MAX whereby the software detects a conflict between the two sensors and drops control in the pilot's lap, we will be attempting to guess which of two opposing "fairness" measures is best. And if the 787 MAX software gets a third sensor which breaks a tie between two differing sensors then we'll just get a third opinion of what's "fair". Better again to drop the decision in the pilot's lap. In software programs, you'll get a pop-up which says "Unable to decide what's fair. What do you think?" In Government, we'll get what we have now: Decision Doldrums.
1
Algorithms are a new form of institutional racism. We anticipated this problem and built a solution called Goaloop, an independent platform that connects the world through goals, now a client of Yale Law School’s Entrepreneurship & Innovation Clinic. We connect the world not by ‘who’ you know, but by ‘what’ your goals are. A goal to learn guitar matches a goal to teach guitar, and sell guitars, and make guitars. Connecting us by the common denominator of our goals expands social circles, transcends zip codes, ideologies, genders, classes and other traditional boundaries to help us all reach our individual and collective potential. We are the world’s first level marketplace, connecting goals from Wall Street to Main Street to the sidewalk vendor and your mobile office, enabling true competition. Goaloop uses the goal as a humanistic unit to transcend boundaries often used to divide us. Our algorithms will help people see through biases that blind us every day. Every goal is a footprint of common ground. Algorithms will help make our common ground clearer. Goaloop is patent pending. If you are interested in Goaloop, please contact us - many team members are connected to Columbia University.
The Algo Accountability Act has critical intentions, but its enforcement is unrealistic for a tome of reasons. First, many cannot recognize biases, racism or sexism off-line. Second, big networks are ad platforms first, their models scale social divisions. Goaloop is a structural change.
9
@Lori Terrizzi "Goaloop uses the goal as a humanistic unit to transcend boundaries often used to divide us. Our algorithms will help people see through biases that blind us every day. Every goal is a footprint of common ground." Well said!
2
@Kelly Thanks! That first phrase within your excerpt above is actually a comment made about Goaloop by a (world renowned) writer.
2
@Lori Terrizzi
This is a great concept. An open platform for everyone to network in an attempt to achieve their goals and support others with similar goals.
1
True enough, but you left out garbage algorithms. Some of them, as in any human enterprise, are written by people with garbage views toward their fellow human beings, and are the mathematical expression of those views. At the very best, enough care may not be taken to avoid the authors' biases in algorithms, even when those using them kinda sorta meant to.
2
"... you left out garbage algorithms."
There is no such thing.
"Some of them, as in any human enterprise, are written by people with garbage views toward their fellow human beings, and are the mathematical expression of those views."
Obviously you don't understand the problem. In machine learning systems, the training data influences the results that the system returns when queried. Training data is not "written by people with [whatever] views".
3
An algorithm is just a process, not necessarily related to computer programming. You need to clarify, up front, what the heck you're talking about...algorithms for what?
3
"You need to clarify, up front, what the heck you're talking about...algorithms for what?"
If you read enough Times articles, OpEds and comments, you will see that the meaning of "algorithm" has been so corrupted that it has no relation to what computer scientists mean by the term.
Now, "algorithm", as used by know-nothings, means "computer program" or "app" or "software".
BTW, the bill does not use the word "algorithm" anywhere except in the title. The bill starts by defining the term "automated decision system". Follow the link in the OpEd.
Interesting article but it misses the point. We are force-fed the notion that algorithms are the 21st century version of how we view religion during medieval times -- as a force capable of resolving all issues. In fact, algorithms are more subject to human error than human beings are -- humans input the formulas algorithms use to make their calculations. At least humans have the ability (at least at times) to question the validity of our assumptions. Machine/algorithms lack that ability. The real question is whether algorithms really have a place in decision making, especially where much of it is based on human assumptions, not facts.
5
Humans are capable of self scrutiny. No matter how much learning you program a computer to execute it will not change itself. It is math not humanistic thinking here.
It is up to us how much we are willing to surrender our thinking and entrust our decision making to machines ultimately boxed.
No machine will, in parallel to what it has been programmed to do, search for its limitations at what it is doing.
Humans, when they are up to it, can.
Above does not address the cases when from the very beginning the algorithms are created and data is selected with the intent to manipulate.
2
In our innovative capitalist system providers will quickly jump in to serve the “victims” of alleged algorithm discrimination. Much preferable to unwieldy laws enforced by the slow moving bureaucracy. Let’s not provide more easy money to the lawyer class.
2
@dksmo
I hardly think that "innovative capitalist system providers will quickly jump in to serve the 'victims' of alleged algorithm discrimination". Can you give a concrete example of how you imagine this working?
Here's an example of how currently one category of algorithm doesn't work in that way at all: the algorithms now in use to program voice navigation systems clip out high frequency voices at some level, rendering them inaccessible to many with weak, high or otherwise "abnormal" voices. I know this because I have such a voice, due to a disorder, and I have decades of fruitlessly trying to interact with such systems. If there is no "press 0 for the operator" or if the "operator" isn't human, we are effectively locked out from the business world, making hotel/flight/car reservations, etc. And this happens often to such 'victims' - any outlier from the norm. Nobody has rushed to fix this for us, believe me.
An engineer friend (who was well-versed in such matters) explained that this is a pure cost-cutting design feature of the algorithms, engineered to minimize the transmission bandwidth required for voice recognition. I've often considered that we should bring a class action suit against such companies. Apple Watch is about the only voice rec system I've found that recognizes my voice. This isn't just frustrating, it's debilitating in today's world. Thanks a lot, A.I.!
Manipulating data to inform decisions related to credit, housing loans, employment, life choices, and almost every activity of the social human, has become the de rigeur way to make decisions. Additionally, data is not being manipulated by algorithms per se. The output of any algorithmic process is deduced from the three-legged stool of input data, the mathematical algorithm, and the implementation of the algorithm. At each of these stages, many assumptions have to be made, each of which has an indelible impact on the outcome. Furthermore, these assumptions can vary from practitioner to practitioner.
There are many commentators, who have advocated banning algorithms completely. Leaving aside the feasibility of doing this, algorithms implemented with a social conscience can be immensely beneficial. As such, the tendency of companies to design runaway models should be curbed. There is no reason to throw the baby out with the bathwater.
3
Please don't forget the job discrimination no one seems to mention - ageism. There is a law making age discrimination in employment "illegal for workers over 40", but it has almost no teeth. The AI bots described herein could make that problem much worse since current stats used to find preferred workers are based on this de facto discrimination.
In our culture and even our tax laws we presume that older people want to stop working around age 66. Some, even many, do. Many of us though, highly qualified with strong work ethics and work habits, are pushed out in lay-offs, early retirement offers, etc. and suddenly find that while our high skills are valued, they are not valued in us.
"We are looking for long-term employees." when average employment tenures are 5 to 7 years? "I see it has been quite a while since you took college courses." because I have been working in advanced tech. "Are you familiar with MS Office and Sharepoint and social media apps?" Presumably I am too old to have learned these skills in school. These are real questions I have been asked in interviews. To all those 'kids' who interviewed me: I has to self-train most of the software I ever learned because, like my peers I was ahead of the curve you climbed.
Employers must be required to submit stats to regulators on who applied, who was interviewed, and who was hired by as to race, gender, age, etc.
8
You failed to mention the obvious advantage of algorithmic decision making over traditional approaches: algorithms can be objectively evaluated for bias, unlike traditional, human-based decision making, in which personal biases can be easily concealed.
30
@David Eike: If it's "hard for even their designers to know exactly how outcomes come about," then if concealment is relevant to the detection of bias in the human case how is it possible to evaluate algorithms any more objectively?
6
@Mark
I do not agree with that assertion. I programmed databases for over 30 years. I never built an algorithm I did not understand.
5
@David Eike: Suppose the algorithm, looking for patterns, finds a predictive relationship between nonpayment of loans and what brand of sneakers the person wears. Is that, objectively, bias? Why or why not?
6
The article suggests that "fair and nondiscriminatory" is a well defined or well understood concept. Is it?
If one variable is correlated with another, how is it determined which, if either, is entering into the algorithm's decision-making process? If the algorithm admits only a few minority applicants into Stuyvesant, is it discriminating on the basis of race or ethnicity? Or is it discriminating on the basis of test scores, which happen not to be distributed equally by race or ethnicity?
42
@Mark,
Wrong stating of the right idea. Actually, those admitted in Stuyvesant are 75% minority students.
6
@Mark,
Curiously wrong cliche. Actually, 75% of those admitted in Stuyvesant are minority students.
4
@areader: I stand corrected! Thank you. But I hope you see the point I was trying to make.
3
Bureaucracy has no place in science. Training neural networks is an incredibly complex procedure, and it does not need oversight from ignorant politicos who can barely master email. With regard to medical algorithms, the problem is largely self-correcting, since by definition such an algorithm has to work for its target population, and men and women obviously have different medical needs. As to discrimination, perhaps we should first define our terms. If an algorithm is used in the criminal justice system to predict the likelihood of recidivism and it assigns higher probability to an Africa-American based on the statistical data, is it discrimination? Facts are facts. Politicians may argue why African-Americans have higher crime rates, or offer measures to change the situation, it they cannot outlaw data.
63
@Mor
"Facts are facts." Not!
However, there is wide variation in how facts are compiled, how they are interpreted and how they are used. Cherry-picking "facts" is also a favorite way to attempt to defend a position that could be entirely fallacious.
A prime example refuting the assumption at all facts are created equal is the prohibition of data collection on gun violence by the Center for Disease Control for the past 20+ years, based on NRA lobbying efforts during that time.
Further, the way in which other important "facts" are collected vary widely, e.g., how incidents of police shooting of civilians are recorded and reported.
Ultimately, it is not the data sets that are "outlawed," but the flaw in the manner in which and reasons why such data are collected.
10
@wysiwyg oh really? Facts are not facts? What a perfect summary of Trump’s post-truth era. Your post is a textbook example of misdirection and logical fallacy. The prohibition on collecting the gun-related data is a prime example of the LACK of relevant facts. You can’t train an algorithm on missing data. And if you have any actual proof that criminal statistics are skewed, why don’t you show it? Defending a position with even flawed facts is better than defending it with pure demagoguery.
5
Data is data; but selecting certain subsets of the data or various weighting of the data can ‘game’ the system. Many times it’s a ‘you tell me what you want and I can manipulate the data to give it to you’.
9
We're right there to hold tech companies accountable, but we never hold minorities accountable for their own lack of success. Is this a liberal bias in data interpretation that needs to be corrected? Whatever good things are happening in the economy they never seem to benefit minorities. We automatically look in the mirror to see what we're doing wrong; we never look at them to see what they might be doing wrong, if anyone suggests doing that they're immediately shouted down as racists so any progress that could be made following up that lead is promptly ruled out. Why? If correct data happens to lead to correct answers is it inadmissible if the answer to which it leads isn't the one we determined up front to be the one we want--well, you know, the politically correct answer? Is that scientific?
5
We don't need a law to regulate algorithms. All we need is for the courts (and/or Congress) to make it clear that when a company is accused of discriminating based on under-representation of protected groups with respect to the relevant matter, the company will not be able to defend itself by claiming that "the algorithm made me do it" unless and until the company can prove that the algorithm does not discriminate. Either that, or all algorithms should be required to attend diversity training programs.
1
We need to update Mark Twain -- There are three kinds of lies: lies, damned lies and algorithms.
4
How can politicians offer to improve something they're not good at? Algorithms are a human constructs based on selected criteria. Let's first have the difficult conversation of why we have these anomalies between races and what can be fixed thru self reliance. Let's get past our embarrassing history and focus on progress not perfection.
3
I've been a data scientist for 13 years or my whole career.
Here's the deal with any model. It's only as good as the data that it get's and if biased data is fed into a model then biased results are going be returned. This is especially true with historical data. Also, if there is no human intuition in how the data is constructed and what the models are selecting for variables (independent) then there is a lot of room for biased or wonky results.
If there isn't proper governance at the business to test and re-test models on development/in-time/out-of-time data then there will be problems. There's lots of technical issues that need to be test for prior to rolling out a deterministic algorithm in a production environment.
Let's take XYZ corp, if high achieving employees are historically all one population. Data selected for our model is based on historical outcomes. In today's world there is more diversity with other groups entering the overall population. We build our model on historical data and see that it's working well in predicting high achievers from low achievers. Then we apply our model to current data and it starts selecting applicants based on specific characteristics. Suddenly, we see skewed results with the model selecting more of one particular population then the new populations. Why? The model wasn't the issue here. The issue was the data having bias in it particular to one group even if we tried to remove that bias by not using gender or race.
5
"Then we apply our model to current data ..."
IOW, machine learning models are always obsolete in a changing environment.
The obvious solution is to continuously update the model. But doing that work partly negates the work saved by using automation. So there is a Catch-22: You have to do work to save work.
The industry's STRENGTH is its workforce of young, smart, IT savvy math wizards and computer engineers. The industry's WEAKNESS is its workforce of young, smart, IT savvy math wizards and computer engineers. Without more diversity initiatives that go beyond gender and ethnicity, and include age and income diversity, the algorithms are likely to continue missing the mark. So let's hear it for algorithms that address the needs of old(er) folks on limited or fixed incomes!
6
Because of systematic discrimination engineers and math whizzes are largely Asian or white. So it is not a surprise the algos are biased.
1
One of the problems with the Algorithmic Accountability Act is that it would be the 21st Century equivalent of the infamous "The Indiana Pi Bill" in which a legislative body is establishing mathematics by fiat, specifically by indirectly defining the value of pi away from its true value.
Well, algorithms are really no different. The results they give are based on mathematics and if mathematics reveals uncomfortable truths or actions, so be it. Congress's bill is tantamount to requiring these algorithms to give the wrong answer. That's unacceptable.
For this reason, I believe that we need simply remove the algorithms from many areas of life.
12
"We need make algorithms fairer"
If facts are unfair we must use something else instead of the facts.
14
@areader
Well said!
2
Algorithms must be thought of a dynamic not static. As learners over time, not unchanging filters. Likened to the human learning process, algorithms are expected to change over time. And that with real life field experiences, those algorithms make errors and become more accurate in ways not considered. For example the journalist’s reference to an algorithm that misidentified (some) blacks as gorillas. This error can be thought of as the surprising, but not unheard of with young child doing the same thing. And with experience, the child as well as the algorithms learn finer determinations that take in additional items for more accuracy. And in that gorilla example, that is exactly what happened. Learning.
What must also be guarded against are laws and political influences that prohibit discoveries and determinations we may not like. Prohibiting a big data algorithm from looking for correlations by race, color, creed, or gender would miss useful information of race linked sickle cell anemia, melanomas, Tay Sachs disease, and breast cancer. Who knows what today’s extraordinarily big data, from all manner of sources, could find? Certainly some of those findings will not be politically correct, but lead to better understandings and remedies. We must tolerate some level of “mistakes” as the algorithms learn. We must accept some level of discrimination in observations as that is an important element of the algorithms ability to benefit to us.
3
I would really like more information about exactly what data is being used. Opinions not backed by fact are useless.
1
"I would really like more information about exactly what data is being used."
As the authors suggest, some of that is considered "proprietary". However, there are numerous books on machine learning. Check your library.
Good grief. Another attempt to apply the concept of “Disparate Impact” when the inputs are beyond the expertise or abilities of regulators. Well, disparate impact is just a euphemism for the use of quotas—impose fines or mandate changes until proportionality is reached.
But disproportionately does not always, or even usually, imply bias. The NBA is the most famous example. The NBA is African-American in a way that defies randomness. That is a fact. Is it racist? I doubt it.
The search for “hidden bias” is the perfect tool for some because it can’t be disproven. It can’t even be seen—it is hidden—so one can only rely of disparate impact to root it out. And having found disproportionality, one can then enforce one’s desired political outcome.
15
Looks like a new opportunity for class action law firms.
6
Here's how you make them fair: Get rid of them.
3
big government yea~ ~ ~
it worked in Venezuela and Maoist China. Right?
6
A few observations:
The analysis in AI models is fairly inscrutable. The computer is finding patterns using different slices of data; there's really no way to peer into the black box and figure out precisely what it's doing.
It would be nice if the people proposing and drafting this legislation included computer scientists, not just lawyers and those with non-technical backgrounds.
The sort of standards they're proposing "fairness, bias", etc., viewed cynically, seem like a thin pretext for second guessing any decisions made by a tech company that someone is unhappy with. If that's all they're trying to accomplish, perhaps the Senate and House should create a social justice panel that would decide all matters of discrimination, bias, and racism, including those arising from "algorithms". The panel's decision would be subject to potential veto by Twitter - it would be overturned if aggrieved Twitter users expressed sufficient outrage regarding the panel's decision.
7
From the headline and initial comments, I was ready to lambast this legislation. But after reading the details I understand the purpose. Machine learning algorithms are NOT like algebra or even a test grading system. They are powered by massive amount of data and have literally millions of subcomponents, and even their creators cannot fully describe why all the systems decide what they do once the model is created.
Furthermore, even after the machine learning model is created, the accuracy has been shown to "drift" over time due to changes in environment or new data. Again, forget what you knew about math formulas or basic computer science. This is not the same.
Therefore, I think it is perfectly rational to mandate that once these complex models are created, that they undergo some form of certification that their output is not flawed or biased. Think of it as getting an inspection/alignment for your car. The factory certainly intended to make the wheels straight, but that doesn't mean they don't need monitoring & adjusting.
1
@Ridley Bojangles: Yes, but it follows from what you said in your first two paragraphs that humans aren't smart enough to perform the certification. We need AI for that.
We live in a world of correlations.
Right or wrong, many discernible facts about any one individual tend to cluster together with similar individuals across society. Knowing things like where they live, what they like, how they spend their money - kind of doesn't matter which specific variables you pick - can tell you a lot about many other aspects of their life (Bayes). Some people are more or less predictable in this way but broadly most people fit this pattern in a material way.
We should endeavor to design statistical algorithms to treat individuals fairly and with respect, but if we include any useful variables we will quickly find that computers/black-boxes will merely detect the patterns as they exist in the real world. It is a feature of statistics.
How we deal with this - and what level of categorization we are ultimately comfortable accepting - is likely to become one of our biggest challenges in the rapidly-approaching AI-dominated future.
6
Yes I wish this would happen. I've quit going to concerts bc of the pricing....maybe one day there will be a conscious capitalism.
Some may argue Vilarino’s cartoon has racist overtones. When does representation turn into negative stereotype?
Most people do not understand the breadth of American racism, from racist legislation and disenfranchisement to aversive racism. Southern racists expressed their bigotry directly to your face. Northern racists expressed their bigotry behind your back. Even good white liberals express racist sentiments when jobs and wealth are threatened. Many professions excluded members of racialized subcultures and consigned them to live in specific neighborhoods.
Algorithms are mathematical formula that model natural phenomena and social processes and are only as good as the quality of data from which they are derived.
The proposed legislation is problematic because it suggests that without proportional outcomes, the algorithm must be flawed. Why not consider the possibility that algorithms predict outcomes based on behavior?
Most medical schools rely on admissions committees to make the final decision based on a comnination of GPA, MCAT scores, and recommendation letters. Some schools do not require MCATs. Algorithms may be used at a preliminary stage to crunch grades and scores. Each requirement is potentially problematic. Most med schools do not distinguish institutional rigor in evaluating the caliber of the GPA. Recommendations can reflect serious bias.
Algorithms cannot replace judges, however flawed the stream of decisions emanating from our Courts.
2
The MCAT is racist if schools are not requiring it. What it tests is racially and culturally biased just like the GRE or LSAT. We have very few black lawyers and US born black doctors in the US especially in comparison to Asian ones. That needs to change.
Demanding constant oversight for algorithmic sorting defeats the entire purpose of having an algorithm. If you don't trust algorithms to be fair, then don't use them, this half-measure is ridiculous and absurd. The entire point of an algorithm is to automate the process of sifting through data to find matching criteria.
When you're sorting people the criteria you're matching for are usually abstract and difficult to evaluate on a logical basis. It's why we have evaluative tests, personality quizzes, questionnaires with ranked choices, it tries to convert a person's complex, abstract personality traits into sortable numeric data. Empirical observations can go into the algorithm to, that's how the advertising algorithms work, they record the number of times a person clicks on something or how many seconds they spend looking at a particular video, that's all added up and converted into a probability to repeat that same behavior or likelihood to engage in some other activity because of it. Similarly, answers to surveys and questionnaires that reveal personal info can be used to calculate probabilities for certain life outcomes by running those answers against national statistics for any given demographic.
It always boils down to numbers. Numbers aren't perfect, they only estimate based on past data, but they've proven far more more accurate than anything else. If you don't like what the math shows you, then don't use it.
11
"Algorithmic Impact Assessments." Yet another regulatory trap American companies will fall victim to. First, there will be the assessments themselves. Then there will be the disputes about whether the assessment were properly performed. And, finally, there will be the lawsuits, exponentially expanding the notion of algorithmic discrimination to its most preposterous extremes.
And while American companies are chasing their own tails through this regulatory bog, companies in Asia will be free to invest in how they can make their products more efficiently.
15
In 1976, Joseph Weitzenbaum of MIT wrote a prescient book entitled "Computer Power and Human Reason: From Judgment to Calculation." It was a user-friendly tome written by the creator of the ELIZA program (an early attempt at AI), and in it he explained how the computer could easily be programmed in ways that would reflect the unconscious biases of its programmers. Given the dominance of Caucasian males in technology, it is not difficult to understand why there may be discrimination in the ways in which such algorithms emerge.
A quote from the book that always remained with me (and placed on the wall of my office) was: "The range of answers one gets depends upon the domain of questions one asks."
Thus, it is the domain of questions that programmers use to create these algorithm that needs to be expanded. Not sure if legislation can define nor require a "reasonably possible" algorithm since it would still be determined by the programmer and his/her programming administrator. "Transparency" to expose these biases would likely be impossible "because of concerns about proprietary information or gaming the algorithm," as Kaminski and Selbst.
The most rational longer-term response to this troubling conundrum would be to ensure that these companies recruit and develop staff that reflects the diversity of our society. Legislation could help, but employees whose life experiences diverge from a white male privileged position would be equally, if not more, helpful.
1
@wysiwyg: Maybe in a sense ELIZA was an "early attempt at AI," but I thought it was meant to *mimic* intelligence when in fact it was not doing anything very deep at all. Maybe it showed that we tend to perceive patterns when there are none.
2
@wysiwyg
The blind spot of programmers (& Weitzenbaum, as you represent him) is the illusion that biases and self interest are extricable from either algorithms or thought. Algorithms simply hide their human purposes, biases and all, behind a wall, hidden from the human beings actually trying to communicate and interact in a digital format. More diverse staff at big tech might diversify the biases, but the deeper problem is that algorithms by-pass the interactive human communication we've always relied on to work out differences, defend ourselves, and change others. They allocate us according to categories and analyses that may or may not reflect our intentions, while hiding their purposes (& their ideology) behind a mask of "rationality."
6
@Mark
That is exactly the point. Thanks!
If a computer forecasts a certain person is likely to be foreclosed on his mortgage, why are we pretending he will make all of the payments again?
5
@Ryan
I'm a bit confused on your comment but here's a crack at a response:
There are a myriad of factors as to why someone would foreclose on a mortgage the biggest three are death, job loss / business foreclosure or divorce. These major life events account for the vast majority of all foreclosures.
Most people are penalized 7 years before they have the credit quality to get a new mortgage. Those 7 years have to be in good standing.
If you pay all your bills get a new job, find a better partner (if that's your thing) and are not dead then after those 7 years there is a enough new history for you to qualify and the Fico model will score approve you.
Because the hypothetical person you conjure to your mind in posing this question is a stereotype. The actual person may NOT default, and the bias you would bring to the program, as you did to your question ( and I to my reading of it) makes the Program biased. The article explains this quite clearly.
@Ryan
If the algorithm predicts that you will commit a crime, why not put you in jail now, and solve the problem? This is the problem with organizing society by algorithms & data. Predictions made in the interests of those who write the algorithms are a means of control. Racial "biases" are the least of the problem.
1
"The bill, called the Algorithmic Accountability Act ..."
Despite the title, the bill itself never uses any word related to "algorithm".
The bill starts by defining an "automated decision system". And that exposes the bias of the bill itself. It fails to hold *organizations* responsible for decision-making, regardless of how that decision-making is accomplished.
Another problem is with the term "computational process" in the definition. That could include manually adding numbers on a sheet of paper. So the definition fails to distinguish *automated* processes from *manual* processes when the processes produce the same results.
4
Suppose reality is skewed one way, for whatever reasons.
The majority of people in the real world who did X are from group Y.
Is the goal to develop algorithms that reflect that? Or are algorithms that reflect that considered biased?
Or is it to find things that somehow predict outcomes different from current reality so it will look unbiased?
Suppose the algorithm says even though the majority of people who did X are from group Y, it's really due to some other factor, Z. And Z is spread scross the population in a nondiscriminatory way.
So the algorithm starts to predict that factor Z, not being a member of group Y, predicts who will do X.
But reality doesn't bare that out. The people from group Y who also have factor Z appear superlikely to do X. And people from other groups who have factor Z are slightly more likely to do X.
And a bunch of people from group Y without factor Z also do X, but the algorithm does not predict that. It is simply reality.
Is that algorithm considered successful?
9
"algorithmic decision-making" is an oxymoron. Ban it altogether if possible.
2
'"algorithmic decision-making" is an oxymoron.'
That phrase is a mistake by the authors. Later they use the phrase "human decision-making", which should be distinguished from "_automated_ decision-making".
Better yet, they should distinguish between "automated decision-making" and "_manual_ decision-making". And the bill should do the same.
The problem here is that lawyers don't understand that the word "algorithm" can refer to a *manual* process:
"algorithm: A process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer." (Oxford)
"algorithm: 'broadly': a step-by-step procedure for solving a problem or accomplishing some end" (Merriam-Webster)
1
@MKR, AI is the future of the service industries.
@MKR
Automated decision making is redefining how we think, act and live. If the tools are properly used to enhance our own knowledge and decrease complexity of large data sets then humanity will benefit enormously. If we had the complex climate models and computational power a 100 years ago we wouldn't be in the mess that we are in.
As usual, our politicians & pundits are concerned not with making our society better or more fair, but with making the injustice & exploitation colorblind. Democracy is based on a shared world, not on worlds tailored for every individual's pocketbook & political preferences. If communications media served their purpose, they would unite people in community, not divide them into exploitable economic & political markets. Technology is waging a fundamental assault on the foundations of democracy, but because its profits are so high, our only response is to dress up the exploitation with political correctness.
2
From the article:
they must, as Mr. Booker put it, “regularly evaluate their tools for accuracy, fairness, bias and discrimination.”
The real problem is that often, accurate and fair assessment leads to bias. We've moved affirmative action back one step. Which may be a good thing. But don't pretend it's fair or unbiased.
6
The Algorithmic Accountability Act? Please tell me this is a sick joke. With all of the problems in our country, this is what legislators are focusing on? This is the same government that can't even get a regular budget passed without shutting the entire system down. The same government system that struggles to get the slightest things done due to such deep political gridlock. Yet you are going to find bandwidth & political capital to regulate Algorithms?
Right! At least we now know Sen. Booker wasn't serious about running for President. This would be a goldmine for public interest lawyers who would bury the tech industry in litigation and frivolous lawsuits. Missing from this article and this point can't be emphasized enough this proposed bill is nothing more than a progressive fantasy, a political stunt. It has zero chance of passing now or in the near future. The far left despises the fact that the internet is leaving them behind, obliterating their influence. Maybe they can't stand the fact the "people" who's will on Earth they purport to represent will need them less & less. When the Pope saw the first press in 1440 he was horrified. He's said to have shouted “This will destroy That. The Book will destroy the Edifice.” It was ecclesiastic terror before a new force: printing. It signified that one great power was to supplant another great power. And now the Internet is destroying a different Church. The power of far left... forever??? One can only hope.
29
@Bill Brown
"When the Pope saw the first press in 1440 he was horrified. He's said to have shouted “This will destroy That. The Book will destroy the Edifice.” "
That wasn't a Pope. That was the fictional priest in Victor Hugo's "Notre-Dame de Paris".
1
@Bill Brown
The only thing the internet has destroyed so far is our innocent belief that we would mostly use it for good, well maybe except for a few porny types, but we'd know how to rope them off from our children's eyes under the counter...
Plenty of evidence now that we can't be trusted to do that! The apple done been bitten.
This proposed law is an absolutely terrible idea. It effectively mandates Maoist style "correct thought" restrictions on all big data analytics. The problem is that in the real world people do differ in average in various ways across ethnic categories and acfross gender categories. Forcing these clear statistical differences to disappear from data mining results requires an expensive and dishonest over-ride of any analytic tool applied to big data. This is an information-destructive, unhelpful approach to dealing with ethnic and gender inequality, based on government-enforced ignorance.
60
Bureaucracy and more laws just create more problems and more costs. The free market will solve these problems. If an algorithm isn’t putting forward half (ie females) of the most eligible candidates or isn’t advertising a house to a significant portion of potential buyers, then it will be surpassed by a better algorithm. This isn’t systemic or willful racism, this is the result of bad or not properly trained algorithms.
16
Lawyers rarely succeed at controlling something they don't actually understand.
Look at the too big to fail banks or Wall St in general. Every attempt to chastise, regulate and rein in has met with 'unintended' or worse consequences. Still plenty of money just now more concentrated with a few big players.
Legislators will once again be chasing their tails trying to control something they don't understand and have no ability to create themselves. Instead, encourage everyone to be a creator and legislate the resources.
26