“...question assumptions about memory and orient themselves toward current events in a new way.” Memory! Whose? Whose selection of current events? What happened to facts, determining objectivity and relevance of sources? Hypothesis and synthesis? Students in a class you describe could not pass a rigorously constructed and standardized test of those skills because they probably spent a semester in the Groves of Academe studying doctoral dissertations and absorbing the authors’ prejudices. It is a waste of time to test for critical and original thinking and application of those skills when the class was a semester long indoctrination. The piece is in fact an argument for not holding college level instructors accountable for the cost of tuition.
4
Employers blaming universities for not training the students adequately and universities blaming schools for sending unprepared students for university education would not help solve the problem education sector is facing. The whole idea of forced education is flawed as it undermines the very law of division of labor based on preferences and productivity. Employers blaming universities for sending inadequately trained graduates is another ill-founded belief. They are avoiding the fact that at the given wages they cannot absorb the supply of labor, which is too much due to excessive zeal to educate everybody by providing generous financial support. The scholarships make sense only when they are disbursed based on meritocracy, i.e., to the students who show promising prospects. On the wage front, the wages in certain sectors are definitely far above what they should be due to lack of competition, protectionism, and resource misallocation. Until the appropriate correction happens there, the labor-employment mismatch would continue with parties involved blaming each other forever.
12
The root cause is lack of a solid foundation built in K-12 (and of course the root cause for that is societal, economic, cultural...). Perhaps the late John Barlow (lyricist for the Grateful Dead) said it best: "you ain't gonna learn what you don't want to know". Granted this is somewhat unfair - today's college freshman desperately want to learn and excel but if you are still struggling with percents or can't put very basic historical events in proper sequence then you simply aren't ready for college level scholarship. Though old fashion, I benefited greatly from the NY State SAT tests used in the 1960's which gated each grade advancement. These statewide curricula where tough but when you got to college you 'knew stuff'.
Question though: what is 'neoliberal austerity'?
19
Amen. Students in upper division science classes who cannot deal with percentages and who cannot take the square root of a number between zero and one (square root of 0.04 = what? The answer is 0.2, not 0.02) are not going to score well on an exam. Like Guy, I went through elementary school and high school when there were frequent "achievement" tests. My school system did not use the results to advance or hold back students, because the teacher-assigned grades did that well enough. But the frequent national assessments did provide reminders that the stuff we were being taught in school was stuff that we were expected to know for adult life. Contrary to the cutesy saying, just showing up would not be enough.
14
I would like to beat the attendance drum one more time here in the comments. Why are we not tracking class attendance by students in higher ed.? Attendance is top priority on the job - don't show up - get fired. Does this sound too much like big brother? K12 and the workplace sure track attendance - why are undergrads exempted? Think it's not really a problem? Then just keep your head in the sand! Are we too afraid to know just how much of that expensive education is going into the trash bin?
So how many total classes and how much money is being burned in this manner? Here is a nice USA Today article on the topic. These results are self-reported by students who may or may not underestimate the number of classes missed. I suggest a more robust attendance tracking system that doesn't rely on self reporting.
http://college.usatoday.com/2016/02/26/the-cost-of-skipping-class-by-the...
Here are the main points. Students miss 20% of classes. Over four years this comes to 240 classes. This amounts to $7200 of wasted tuition at cheaper state universities ($30/class) and $25,000 at private universities ($100/class).
8
This is an excellent article, in my opinion. Thank you, Molly Worthen.
I would add, Worthen notes that push to evaluate learning "preys especially on less prestigious schools" and that highly selective schools do not bother with it. Probably because highly selective, prestigious schools are selling their prestigious name and their prestigious old school tie, not their ability to teach knowledge or critical thinking.
Sadly for dedicated teachers everywhere, critical thinking is not really what today's companies want in employees. Even universities do not value critical thinking. The goal is set and the job is to achieve the goal. A employee who points out that the goal as specified does not demonstrate what it purports to demonstrate will be shown the door, at a company or at a university.
Devising a scheme to bring in more money is the actual skill that companies and universities desire. If the scheme involves misrepresenting or outright lying, that's okay. Shyster thinking, not critical thinking, is in demand. And whistleblowers are not welcome.
26
at least no one has suggested you carry a machine gun to class (yet)..
two truisms-- the purpose of college is learning to think, not to accrue facts; the cost of college has skyrocketed, it's the educational equivalent of the epipen, it needs to come down to a reasonable amount
7
Molly Worthen has hit the nail on the head with this op-ed piece. I teach in higher education and most often teach the neediest students trying to get through despite financial, home and housing problems. Despite the reality of our students, our classes are becoming neoliberal factories for jobs that may or may not exist. We have no data how students do once they leave our school. We have data on jobs that exist in the state, and so try to encourage students with $$, but most of us know how rocky things are for the poor. We have no actual data on whether our students actually get the jobs we train them for. One final word: we have examined the big data and pay mightily for that, and still there is no consistent funding for tutors, trained classroom assistants, embedded tutors, supplemental instruction, or paying students to do jobs on campus (so that they feel part of a community and perhaps feel encouraged to finish in a timely fashion --- what about $ for internships that might bring them skills seen as "soft"? None of that either. Neoliberal models of education are strangling us.
13
Assessment has *zero* scientific validity. When I first entered my full time position, I was shocked that a room full of PhDs could sit by meekly and nod their heads with assent at the idea that from a class taken in year one to a class taken in year three, we could measure how much "problem solving" ability had been gained by students through looking at their performance in *completely different assignments.* It was enough to boggle the mind. Then I figured out--everyone really was just nodding because they knew they had to pretend to drink the koolaid in order to get tenure, and assessment was a pet project of administrators--a costly pet project at that, not only in terms of funding but in the hours dedicated by full timers to meaningless performance of compliance.
33
Bravo--excellent analysis of this current dysfunctional element in modern academia. The same universities who are creating the big assessment bureaucracies are also increasing the disparity between their overpaid top administrators and faculty. I love your point about universities' purpose as providing space where capitalist pressure should be resisted in order to foster critical intellectual investigation. The chair-warmers in the Orwellian departments of student assessment should be terrified of the truths revealed by this article!
18
"All this assessing requires a lot of labor, time and cash." Painful, yes. And detracts from my actual teaching time.
District administrators obsess over percentages, data, the posting of numbered Common Core standards and learning targets. I'm given additional books to read for Professional Development which mandate teacher scripts, tell us to use the pronoun "I," not to teach facts because students have phones. We watch power point screens with circles marked ASSESSMENT, CURRICULUM, INSTRUCTION, move from not overlapping to—you guessed it! Overlapping!
These companies are after the 500 billion spent on public education annually. The cookie-cutter standardization of instruction, the scripts, the assessment of teachers by posting of standards developed by David Coleman, the posting of targets—all these de-emphasize content knowledge. Seems knowing your subject becomes the least important thing a teacher needs.
I've been in education for thirty years —at the secondary level—three public school districts, private schools, university level.
Put your money in K-3. They need the highest of all teacher/student ratios. Fifteen students for each teacher. Ten. If you make sure they know how to read, then I don't have to meet fourteen-year-olds without that skill who are dating.
22
Professor Worthen, your article hits the proverbial nail on the head. SLO assessments fail to realize the whole personhood of today’s adult learner. There are so many outside factors that influence the learning process, especially at the community college level, where most students work full- or part-time to make ends meet on their journeys toward self-improvement. Plus, on the faculty end, I hear myself say the same thing over and over, “If all I had to do was teach . . .”
11
Thanks, Prof. Worthen, for pointing out to those who haven't already seen it, or who are afraid to say it, that the Assessment Emperor wears no clothes!
22
Assessment done to get it done is a waste of everyone's time. Thoughtful deep assessment - as the author points out - is just another way to see we need to be mindful of our pedagogy, think deeply, and look to see if we are accomplishing our goals. It takes time to do well - which is why we resist it. That's not a problem with assessment per se. It's a problem with assessment imposed from the outside and not understood or embraced. It's a problem with overworked faculty and overburdened students and administrators trying to squeeze one more ounce out of our hides. We assess all the time. That's what teachers do. It's the documentation of the assessment that I so resent having to produce. Just like my students do when I ask them to waste their time with busywork assignments.
We have recently decided as a department to embrace assessment. We have figured out what we want it to do for US and OUR STUDENTS. It's already shown results. And in the process, we've made the administrators happy. But we can do that because we are professionals in this trade (we're a department of psychology and psychologists measure things) and we have some members of our department who have put a lot of time into learning to do this well. Most department don't have the luxury.
22
Thank you, thank you, Ms. Worthen. You could so easily insert "high school" wherever the words "college" and "university" appear in your article. After teaching high school English for 28 years, I retired when a career I had once loved became unendurable. Each day felt like a chapter from FAHRENHEIT 451, ANIMAL FARM, or 1984 (and sometimes even HUNGER GAMES!). Your article expresses so articulately the issues my colleagues and I faced in our school system--which was and is, of course, pressured by the state--when the business of data collection grew more important than the actual education and well-being of the young human beings in our care.
18
I teach at a college where an extraordinary number of talented faculty's energies are diverted into assessment activities. As Dr. Worthen said, neoliberal austerity policies have used assessment as a way to deflect political blame onto colleges for intransigent social issues, including poverty and a technological divide. (My students all work--many of them are on the late night shift.)
Thus not only is there administrative bloat at our larger university, there is less energy going into into the disciplinary expertise and teaching itself and more into measuring the less that is happening. And serious discussions about how to measure and what to measure just don't occur since there's always some deadline pressure. As educational institutions are forced to satisfy diverse "stakeholders" whether the state or private grant-givers, the more they have to prove that their new education pill is the cure-all. This sometimes encourages real thought and innovation but little long-term follow-through since the next grant requires something else. To convince the tax-payer, foundation or parent that the money is being well-spent, as Worthen says, is costing everyone a fortune. We have enshrined the business model in education (with what exactly as the bottom-line?) instead of the educational model in business. Now we have a public that believes what they read on Facebook and student debt is in the stratosphere.
26
Molly, I'm not worried about the prospects of the survival of the idea of a university, whatever the scale or whether public or private, as long as there are professors in them like you.
Especially in the humanities, but even in science and technology fields, where there are pockets of thoughtful and focussed dialogue like those in your classes, the experience of those students will be its own reward for them long after they graduate.
I truly hope your essay becomes well known. Perhaps it could become a favored leaflet in the offices of high school counselors. Please read this, it could say to those seeking some advise about college, and then we'll talk!
14
The most reliable test of knowledge acquisition I believe is testing. The limits of testing are primarily the ability of the tester to ask the right questions of the test taker. Perhaps a computer could design the questions based on the input by the teacher relative to the course material presented. The obvious next limitation is the quality of the teacher in imparting the course material to the student. And, of course, how motivated is the student to learn. The motivated student will learn because of self interest regardless of the other limitations.
3
Well here's something that might come as a surprise to you: The purpose of a college education is not merely "knowledge acquisition". Equally if not more important is the development of new knowledge and a demonstrated ability to think in a clear and fresh way about the subject. Not every course has simple objective standard like a high school algebra class. A well written exam question might elicit as many excellent and distinct responses as there are exam takers.
American contempt for expertise and intellect begins with the notion that important ideas are always reducible to a simplistic common denominator. It ends in a society that lags in development compared to its peers, eventually to the point of irrelevance.
20
Neoliberalism is just Trumpism Lite. The ends are the same so the means are trivial. This essay explains exactly what that looks like. Instead of an Orange hate-spewing liar, you get modern day phrenology; meaningless date masquerading as science. The assessment industry is emblematic of everything that the Democrats need to cut loose if they want to start winning elections and providing much needed economic relief to the American people.
6
The education industry should give up its chokehold on knowledge... stop denigrating WikiP (I did once hear a very impt professor acknowledge WikiP but those who go to second rate schools tend to turn their noses up ro down. as you will -- Anyone can edit WikiP -- that's true and some people do a great job. ) and NO ONE EVER MENTIONS. COURSERA, UDACITY, AND EDX-- the three online platforms for mostly free education. Administrators should mostly be fired... so far as the adjunct professors and graduate students who do at least 50% of the teaching and the grading?? humm equal pay equal work.. No more union pay byt number of years doing a job.. wages???
Americans love to pay for the unnecessary and not have the essential-- food, transportaion, education, whatever. (unless it's made in China in which case we often get a really good deal).
IMO there is no reason why people should have to pay to go to law school to become lawyers.. (Abe Lincoln didn't.) If you can pass the bar, you should be able to practice.Ditto many branches of medicine with some hands on. Can't wait til much of medicine and driving is done by robots..
BTW people who want to learn unless they are badly misled which some professors weem to specialize in.;;; do learn. So much easier in some ways now than ever before
1
Years ago, comedian Professor Irwin Corey, a self-made intellectual of great depth, proposed in one of his bits, the "Five Dollar University". For five dollars, he would teach you, in five minutes, everything you'd remember five years after having graduated college. Five minutes was probably too long, but that's just advertising.
4
"Teaching it is not a cheap or efficient process. It does not come from trying to educate the most students at the lowest possible cost or from emphasizing short, quantifiable, standardized assignments at the expense of meandering, creative and difficult investigation."
"Elite" colleges do not make any effort to strive for affordability any more than "elite" private schools or "elite" public school districts. The parents who spend their own funds to enroll their children in elite private schools or who pay a premium on their housing to reside in affluent school district do not view their spending as "throwing money at a problem". Rather, they see their tuition costs and higher home prices as an investment. In the meantime, those who resent paying broad based taxes for "other children" see low test scores as evidence that their precious tax dollars are being spent "inefficiently". The desire for cheap and efficient education only exists when voters are seeking a rationale for lower taxes and when voters see education as an "expense" as opposed to an "investment".
12
What is a "rubric"? What is it called out in the real world?
5
Welcome to my world!
—a public high school teacher
5
I think the Times should solicit suggestions for best proposed definition of "analyticity."
8
1. Hire excellent educators
2. Give them the support needed to do their job
3. Get out of their way
4. Don't make them do stupid stuff
I know, so radical. And yet so true. Guaranteed to work.
19
And how do you know who the excellent educators are unless you can compare how much their students learn with how much the students of other teachers learn?
6
In 2007, I wrote an essay against Outcomes Assessment for The Chronicle of Higher Education. The title, "A Pedagogical Straightjacket," tells all. Afterwards, I received several emails at my college email address, some of them in support of my stance, but a surprising number of them attacking me personally. Two came from college presidents, a couple from deans and administrators, and some from parents. They wrote such things as, "You can run but you can't hide," and, "The day when professors like you can get away without teaching anything substantive are over."
There's a whole industry out there that's invested in Outcomes Assessment--which is to say, it's not going to go away anytime soon. I tell people to watch for the day when universities begin offering Ph.D.'s in the subject.
A Pedagogical Straightjacket:
https://www.chronicle.com/article/A-Pedagogical-Straitjacket/20446)
18
There are doctoral degrees essentially in outcomes assessment.
IF you spend so much time WEIGHING your lambs (in anticipation of making LOTS of money on their sale), that you have little time to FEED the lambs ......
15
Amen to that!
Amen, sister.
Blah, blah, blah. Very happy to be retired and out of the discussion. Give up!
3
So we should pay her and shut up?
4
Having a child who had to pass an 8 part United States Coast Guard exam in order to graduate from the United States Merchant Marine Academy, there was no doubt about his body of knowledge. Without passing he couldn't graduate or get a job as a Marine Engineer. But also have two other children with strictly liberal arts degrees (Religion and Classical Studies) I take issue with the idea that these degrees are anti "capitalism". The liberal arts degrees conveyed strong critical thinking and writing skills, which are attractive to many employers. Not every student can be everything to every employer, but it's true that in today's working environment, employers want their new employees to hit the ground running with minimum employer training. A liberal arts graduate is at a disadvantage to prove their worth to a company with very little work experience.
6
This lively and thoughtful essay documents how I felt back in the 90s, when I too was an Asst. Prof and found assessment sharp, awkward and boxy. I'm now a jaded administrator who has seen faculty with no purpose or reason to their classes. Asking faculty to identify at the beginning of the term what they hope to teach and then measuring if they taught it is important.
Assessment isn't about what the students learned. It's about how well you fulfilled what you promised that you'd teach. If you say you're teaching students how to write an argument and you grade with As, Bs, Cs and Ds, you've told the students how well you think they absorbed your content. It's all about you. If someone else uses a matrix to evaluate those same papers, and it turns out that your papers lack a central point -- more so than your colleagues -- well then, you have your work cut out for you. Next time you teach, make sure one of your learning outcomes focuses on how to write a cogent thesis -- and then make sure you teach that. Has it become an industry? Sure. Can we go back to the good old days where your grades demonstrate that your students learned. Sorry, I'm too old and have seen too many cases when that just isn't the case.
10
As a high school teacher (31 yrs), I am saddened that our universities are now plagued with accountability systems that reduce the value of education to workplace skills. I am sorry those who devoted careers to transmitting the learning and thinking that best represents and build humaneness and humanity are now being sized up so narrowly. The education of a liberated mind is not the same as the education of a worker. The version of this accountability system has born no fruit in secondary education. We produce students the majority of which we know are not prepared. We spend four years complaining about students who act as if they have never seen our content before, are lazy, and don't study. Then we rig it so they graduate with strong enough grades to be admitted to college. Worse, students naively think that the grades they earn in HS do carry an imprimatur of readiness. They have no reason to gear up their poor learning skills, and spend their first semester of college finding out, at great cost, they don't belong. The most disadvantaged-first gen college, minorities- question that they belong and soon drop out. Others lower their sights, change majors and extend their stay to five and six years, at great cost. Teachers do not know how to teach for deep understanding, independent thought, long-term retention, and far transfer, for most students. We give crammable tests and shallow assessments to beat our accountability system, and stick it to the next level.
4
As a college professor, I’ve been troubled by the demand for assessment because I think it misconstrues the nature of what we do in higher education. We are not faucets filling vessels with premeasured amounts of knowledge. The “knowledge” I impart today is obsolete tomorrow. We educators are farmers. We plant seeds in all kinds of environments to see which ones will grow. We tend to our planting’s, content in our knowledge that if we have planted and tended, some time in the future, our efforts will yield sweet fruit.
13
I am glad to see that finally we are starting to question higher education. In my view, after the #meToo, it is the next bomb to explode.
Universities are now run by administrators who have no understanding of learning/teaching or curriculum and no background in academics. Only way they can pursue their personal agendas (their agendas certainly do not include student or program outcomes; even if they did, these administrators would not be able to articulate what they are) is to force academics to produce the learning outcomes and their measures so that these admins can use them in most cases to claim credit for successes and to blame the faculty for failures. In a way, we are converting our processes into something to legitimize the existence of a bunch of unqualified people with no credentials to hold the positions they do.
I would like to argue, however, that if we can teach our students real critical thinking and creativity that are build on a set of basic skills, they would perform well at any job related to their fields. For instance, Employers want college graduates to be able to write a sensible paragraph - sadly many college graduates cannot do so. Note that someone who is taught creativity and critical thinking typically writes well. So, in my view, there is nothing wrong with making sure that students have the skills employers demand and these skills tend to be the basic skills that are prerequisite;yet, as educators, we would like to do not only that but more.
3
Testing is not only a source of big bucks for businessmen but it is also a road block that limits the number of individual able to enter a profession, whether its nursing, law, accounting and a variety of others. Such limits which of course impedes competition and protects the salaries of those already in the field. Once asked to design a curriculum or a short course to help get high school grads up to speed in science so they can pass a Pre-test, required for admission into two year nursing schools at two year community colleges. One example of the futility of these tests - a multiple choice question ask; what color does litmus paper turn when immersed in vinegar? Of course vinegar is acid, but tell me how knowing that color is any measure of ones understanding pH or acid-base chemistry. These tests a just an artificial barrier to immigrants and kids with poor language skills.
3
The business Admin class had a meeting the College President. Here is his conflict, the in class students hate a professor. The alumni say he is the one who got it right. Feedback from graduates down the road.
1
I thought “measuring learning outcomes” is what grades are for.
101
Great piece about a serious and malignant tendency in education. But the author's troubled surprise is a hoot to those of us trying to teach high school, where this nightmare (standards-based, proficiency-based, NCLB etc) has hamstrung education for 20 years anyway. More often than not, it's the law. Resistance is hard but ongoing.
Welcome to the war, professor.
5
While overall Dr. Worthen addresses the majority of well founded objections to the "learning outcomes" agenda she missed two important considerations
One: That the drive for this data collection scheme is largely encouraged by a an administrative class of bureaucrats who have never spent any time at the front of the class and are disengaged from higher order thinking on a routine basis. As binary thinkers they live a checkoff box world.
and more importantly
Two: It enables the acceleration of hiring part time (adjunct) faculty who can deliver a cookie cutter curriculum intended to produce reproducible quantifiable, and in most cases meaningless, results.
I taught college (full time) for over thirty seven years. I was present when this nonsense first reared its ugly head. Almost universally the only faculty who embraced the concept were the weakest intellects and poorest classroom performers. They couldn't teach but they could follow instructions to produce testable results.
The net effect will be that we will again see a landscape of elite universities and colleges that don't feel the pressure to respond to inanities like "learning outcome measurablities", and a dumbed down vocational system for the masses. At least we'll have less chance of producing a discontented rabble as they won't know what they've missed.
9
We first weaken public schools with this nonsense, and then move on to the best universities in the world. What is the endgame?
9
I agree 100 per cent with all the points except please stop blaming “capitalism.” There is such a thing as bureaucratic self interest and the “man of system” mindset as described by Adam Smith. A real capitalist would say let universities compete for students and if they succeed why should a government mandated accreditation and assessment czar come in and tell them how to teach?
1
It’s not clear that our political leaders, in particular the Republicans so fond of the insipid, sycophantic “Fox and Friends” and their ilk, want education to turn out critical thinkers. Better that we stick with an education system that turns out shallow “thinkers”, who can be convinced that a bombastic, second-rate reality TV star is “presidential”?
9
If colleges attempt to measure learning outcomes, the author should visit a high school. High school buy into more debunked and mindless data and learning outcomes in a month than colleges deal with in a year.
Differentiated instruction is completely rejected as a instructional methodology by every expert at the university level, yet DI is still clung to in suburban Chicago by clueless admins.
Districts demand qualitative and quantitative assessments every couple of weeks. Yet no one actually understand big data or what it means. Schools have opted to kill accelerated classes to shove more unprepared students into AP classes. The result is more profits for the College Board, more stress for more students, and districts who supposedly feel better because more students are taking the AP tests and that metric is used by US News as if it were the final authority. Then admins pat themselves on their collective backs for a job well done and share bonuses back and forth for the work that classroom teachers conduct who themselves are ineligible for bonuses.
3
This problem is not confined to colleges. I know a number of former secondary and high school teachers (my mother for one) who spent a large part of each working day not teaching but filling in data sheets to satisfy some bunch of faceless state and federal bureaucrats. And they changed with each administration as did the non-teaching work load.
Data garbage In, Garbage students out.
4
I vote for a forced curve... everywhere.
“I do not feel threatened emotionally when presented with multiple perspectives” and scores them on metrics like the “intrapersonal affect scale.”
Jesus wept.
3
"Good morning class. Welcome to Physics 303: Quantum Systems. The learning outcomes for this class are described in the syllabus. It will be clear to you what you will know and what you will be able to do at the end of the semester. I have designed a series of assessments, that are a measure of how successful I am at teaching you these skills. If I am as good a teacher as I should be, all of you will do well on these assessments. While I expect you to do the homework and read the textbook, I understand that if you do not do well, the responsibility lies with me, and I will certainly have to revisit my pedagogy as well as consider whether I have been inspiring enough. If, as I expect, I am successful and you have mastered the material in Quantum Systems, as demonstrated by your performance on the outcomes assessments, please understand that you are not guaranteed an A in this class. The administration is also concerned with grade inflation. But since this is a course in quantum systems, where uncertainty is a given, I have no problem randomly assigning grades with a distribution between A and F with an average of B-/C+. If you were among the unfortunate students to receive an F, please don't worry. This is no reflection on either of us, as the assessments will show. I will certainly allow you to continue on in Physics 304: Advanced Quantum Systems. Have a good day and see you tomorrow!"
7
students pay too much for degrees that mean too little
2
"four methods for evaluating learning, which include testing software and rubrics to standardize examinations, e-portfolio platforms to display student projects, surveys and other tools". Really?
I went to one of those big midwestern land-grant colleges. They seemed to have a good idea of how to judge clear and independent thought, though I got put into Freshman English 1 because I did poorly on the grammar placement test. The TA (a mere poorly-paid PhD candidate) was able to judge that I belonged in honor's English and pushed for my transfer.
As to the modern types, how about first learning what the words mean, and applying them correctly. Chief among them is the abject failure to grasp the meaning of "rubric". Try this (from the OED) on for size: https://en.oxforddictionaries.com/definition/rubric
Good grief.
3
Very good essay, Dr. Worthen. "Learning Outcomes" is the penetration of a simplistic notion that understanding and knowledge can be quantified. If what students learn at colleges and universities can be quantified, it can serve the interests of academic merchants and politicians who need to show that what they do is objectively valid. The push to measure "outcomes" comes at the price of real, theoretical, inclusive, subjective, individual learning. This issue has been discussed in Europe since Aristotle was introduced to Bologna and Paris, Oxford, Cambridge and Salamanca. You can measure skills, that is to say, how to do certain concrete, observable things, like making shoes or building an edifice, But, you cannot measure a world view based on the multiple sources of science. This is abstract knowledge and thinking that during the Renaissance came to constitute the mark of real EDUCATION. This is humanistic education that, unfortunately and increasingly, is missing among the political elite of the United States. This is the reason why, the nation led, for the most part, by uneducated men steeped in religious dogmas, has gone into a tailspin of decadence. Men who do not take science seriously, who think that they can measure knowledge to make it marketable, severely restrict what young people need to know in a complex world that requires critical thinking based on a variety of fields to be useful for guiding society toward the future.
9
Hello (rhetorical). This is what has been happening at the secondary and primary educational level across the land - well meaning reformers supporting charters and school choice and less well meaning reformers trying to end public education. All because we can supposedly quantify school learning with a growing testing industry that gives us 'data'. This is what Bloomberg/Klein 'reforms' in NYC education were all about. Measuring students/teachers/schools by data and insisting no matter what that at least 10% or more must fail as per Welch/GE business methods. Ignoring every other factor and focusing only on the one in school metric that matters most - the teacher. I knew it was a matter of time before these debates hit the universities. Will be interesting to see how it plays out. My hunch: because universities employ mostly men, and because professors are accorded more respect than preschool teachers, it will be different.
1
I taught at a university that added "learning outcomes' to its course syllabi. There was a big, expensive effort to shift the whole university to emphasize a set of objectives that defined (according to someone) a well-educated graduate. So, every class has a set of outcomes attached, much like the fringe at the bottom of a scarf. And about as relevant. Despite an entire VP department pushing every department to add the words to the syllabi, there has been -- after 5 + years - not a single effort to actually MEASURE students on those supposed outcomes. Teachers still require term papers, tests or other measures of knowledge accumulated/regurgitated or sometimes cribbed from 3rd parties. But nothing has been done to test whether the individual class requirements actually create the kind of learning that would lead to the outcomes. It has been a laughable fraud, mere window-dressing and another example of educational theory being embraced with no firm handle on how to implement or evaluate the results.
2
In my senior year of my PolySci major, I took a graduate level course in International Relations thought by the senior professor in the department, a pleasant and open man who taught quickly but well if you paid attention.
He had a demonic mind when it came to testing. No multiple choice exams for his course. He haves us sixty questions each of some import, and asked us to answer any thirty with short essays in three hours.
If you knew the work, it was the easiest exam in the world. If you didn't, you rarely completed ten of the questions. Class size was cut to workable size with students who wanted to learn after the first four weeks.
We covered a lot of ground, at deal was added because we all grasped the opportunities to be in a class whhere we could add to our bases of knowledge far beyond what the syllabus required. Everyone had fun learning, even our professor.
2
I am approaching retirement after over three decades as a college instructor, and the "assessment" movement that has taken over the accreditation process is one big reason that my final day cannot come too soon. Who assesses the assessors, I wonder? Shouldn't they have to demonstrate that the enormous financial and time burdens they have heaped upon the faculty and administrators of countless educational institutions measurably improves "outcomes?" Let's add another layer: Meta-assessment! We can invent another lucrative industry to employ all of the PhDs who can't find jobs in higher education. Win-win, right? because more assessment is always" value added."
15
Dr. Al ... The "peer reviewers" that do the accreditation assessments are usually faculty drawn from other institutions considered to be peers of the institution being reviewed. I've been a peer-reviewer and the teams on which I served did not view the process as seeking to impose a burden but rather facilitating an opportunity to allow the institution being reviewed to show how and how well it was attaining its stated mission. It really came down to "show us what you intend to do as regards educating your students" and then "show us how well you are doing that." Within very broad boundaries, it was up to the institution to decide what education it intended to provide. All of the regional accreditors require that faculty have control of the curriculum. There also is a financial stability aspect to accreditation but that is still rather loose and it catches only the most egregious situations.
At my institution the assessment process is faculty driven (with reports going to those in administration). Depending on what is being assessed, we use a variety of assessment techniques. Assessing fact based knowledge is assessed differently than assessing procedural knowledge.
6
"Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning."
This should be the approach of education at all levels, beginning with preschool, if we want to raise a nation of joyful learners with lifelong commitments to learning and intellectual risk, exploration and creativity. These are the qualities employers are saying they need in their employees to successfully compete in the modern world of international innovation. And I believe these are also the qualities that our nation needs in its citizens to successfully continue to live in a democracy. We need to be able to risk exploring each other's ideas with an open mind, without always having to prove that our own ideas are the "right" ones, if we are going to manage to extricate ourselves from this polarized morass we're currently struck in. But unfortunately when we "measure" young children's "success" by how early they learn to perform a few very specific concrete skills (reading, writing, math) we teach them early on that there are "right" and "wrong" answers and that there is no room for or value in further questioning or exploring. Let's restore curiosity and discovery as the source of meaning in all levels of education.
8
17+ years teaching at state research university and private colleges as adjunct and full-time: There is only one way to measure success and learning outcomes, keep in touch with your alumni and track what they end up doing. This is more important then all the studies, forms, and evaluations out there.
The top schools have more successful alumni. The less successful schools have few or no successful alumni. This is true in any field. If a student gets a degree in English or art or science and—with any of those degrees—succeeds in business or literature or education or gets a Nobel, that is a win for the college and the faculty.
Individually accessing faculty or students for learning outcomes at any level (Freshman to senior faculty) is pure hogwash designed to make deans and state legislatures feel like they have some control over the college.
Education and learning is not sales or a business where results can be tangibly seen immediately.
Steve Jobs dropped in on a Reed College calligraphy class and he brought letterforms/typography and a design focus to Apple—and with no final college degree. His professors would be shammed by most student outcomes on that one. Yet, not too many years later, calligraphy helped found and then make the most valuable company in the world. But art, design, and humanities classes like that, are the first to be cut and the lowest paid faculty on any campus.
14
That method only assesses the admissions office. How about assessing value added, not the institution's selectivity? It's been done, and the highly selective institutions do not do well at all...
2
Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning.
That sums it up very well. That is what college used to be. It is not anymore. Yes, these students are going to make many mistakes and their grades may be poor. The awe ha moment can take a few years to kick in. Learning does not happen in 1 moment, it can takes many years. Many of us remember those moments while in a brain storming session during work when a learning moment in college flashes before us. We realize that time in college which becomes relevant in a moment at work. We make a connection. We build our background knowledge. We expand our thinking. These moments are spontaneous. Hopefully, they kick in sooner and not later.
5
This is a key point:
"If we describe college courses as mainly delivery mechanisms for skills to please a future employer, if we imply that history, literature and linguistics are more or less interchangeable “content” that convey the same mental tools, we oversimplify the intellectual complexity that makes a university education worthwhile in the first place."
I said much the same thing last summer in the Chronicle of Higher Education (https://www.chronicle.com/article/Assessing-the-Intangible-in/240744). We in higher education will have a hard time justifying our continued existence (especially at our price point) if we tell the public that all we teach are easily measured content and skills. Richer and more meaningful learning happens; we have to tell that story.
9
Outcomes on course syllabi have been mandatory for decades to gain accreditation in engineering schools. They are useful in providing goals, both for students and profs.
They do not, of course, include how to achieve the outcomes. That aspect is totally neglected and left up to the prof. At the worst, outcomes become all the course is about, and measuring attainment of outcomes becomes artificial. Unfortunately, that is the easy route actually preferred by many institutions, who prefer their faculty to spend their time on raising money and glad-handing donors.
3
I agree with many of Dr. Worthen's observations. However, as a junior faculty member many years ago, I found the assessment of my departmental academic programs vital. Senior faculty, the entrenched powers in the department, were reluctant to consider the ways in which the program curriculum was not working for our students. Assessment provided a shared collection of information that reduced the power inequities between the senior and the new faculty and opened a discussion about curricular change. I still find the assessment of courses themselves redundant at best, but conversations--based on evidence-- about how courses work together to form a coherent educational opportunity for students are also part of the assessment requirements. Those are useful.
10
JP ... "but conversations--based on evidence-- about how courses work together to form a coherent educational opportunity for students are also part of the assessment requirements. Those are useful."
Agreed ... assessment by itself is not terribly helpful. It is what is done with the information learned from the assessment that matters. The conversations you have had with colleagues that led to improving the education of your students were valuable. At my institution those conversations are both formalized and informal, but our goal is improving the curricula and teaching so that what we intend students to learn is more likely to be learned.
2
It is disturbing the extent to which some of the commenters here who are faculty show contempt for their students. While Worthen gestures in the direction of the very real problems of K-12 education in the United States and of a market driven culture, her primary purpose seems to be defending the right of faculty to teach according to the loftiest of intellectual ideals. I don't disagree but the larger context in which we find ourselves can't just be waved away as someone else's problem to fix. The author, and presumably many of the faculty who have commented here, gained their positions through participating in fiercely competitive, meritocratic struggles and they are speaking from very privileged positions. It's not a new observation, but it bears repeating: faculty too often wish to teach and reward only students who resemble them or seem to have the potential to do so. Being asked to address the very real and difficult problem of how to teach to all of the students in the classroom is part of what assessment and outcomes are about. It is the case that many creative and devoted faculty work hard to do right by students who do not come in well prepared and intrinsically motivated. It is also the case that there are faculty who blame students for not fitting that description and thereby absolve themselves of responsibility for them. Faculty are not solely responsible but they're part of the social fabric that produced these problems and are therefore implicated in them.
8
"It is disturbing the extent to which some of the commenters here who are faculty show contempt for their students..."
Those who don't teach would find the extent of contempt that today's students have for their professor pretty disturbing...
8
if you think that the position of adjunct faculty--who teach most of the undergraduate courses in most institutes of higher learning--is "very privileged," you need to get out more.
In the sense that we come from middle-class backgrounds, and are generally white, yes,we are privileged. But the conditions under which we try to meet our students' needs are anything but privileged,and as for our salaries and benefits? They are shockingly inadequate. Assessment initiatives add to our workload without increasing our pay,and that is a problem.
4
I teach in the computer science department at a small midwestern liberal arts school. I’m used to material that’s about as hard as it can get. But, I’ve also taught courses in critical thinking - very soft, with lots of writing.
With my technical background, I used to measuring and quantification, and feel if you can’t measure it you really don’t understand it. But I in teaching a soft class I had no difficulty ranking and measuring the work of my students. And, did any of us ever have trouble telling our good teachers from the bad ones?
It’s hard for me to believe the push-back against assessment is just a way of trying to escape responsibility for demonstrating that one is doing a great job -- or not.
4
" I used to measuring and quantification, and feel if you can’t measure it you really don’t understand it. But I in teaching a soft class I had no difficulty ranking and measuring the work of my students. "
The typos in your comment indicate the inadequacy of trusting spellcheck without proofreading. I have no difficulty ranking and measuring this comment, and I don't need a computerized instrument to do so. Sorry if this causes you embarrassment. I'm sure you're a good teacher who wants all your students to learn. But trusting technology has its limits.
1
"Assessments" are most useful for evaluating the effectiveness of TRAINING, not so much for EDUCATION. Training is what is done in the military... soldiers have to know what buttons to push, triggers to pull, lights to watch, on the various devices they have to work with. They don't understand the mathematics of cryptography or ballistics. They are not expected to ask questions or to offer improvements based upon their understanding of how their sophisticated rifle, radar, fighter plane, tank, cryptographic device, etc. works. They have to be REtrained pretty much every time a new generation of their device appears. Some employers are looking for workers who work like soldiers, who do what they are told to do, come in trained to do the job and operate the machinery (training which the employers used to do).
Other employers seek workers who can contribute to the improvement of their products and services because they are critical thinkers and have the understanding ("education") to allow them to see possibilities other than the existing products, services, and processes.
State legislatures have traditionally focussed on creating a pool of the first kind of worker. High Schools used to be sufficient to produce that pool of workers but as technology has transformed most businesses, more 'training' is required to produce the pool of workers ready to go from school to job over a weekend.
The most successful, innovative, 21st century companies require educated workers.
8
As a former public school teacher and University professor, all these efforts to quantify learning outcomes is an illusion---and very sophisticated illusion, but an illusion. All classrooms are filled with countless social, political, economic, biological, pedagogical variables that influence the learning process---we will never know what set of variables result in "student learning." Even if we hit on some scheme of variables, we would then have to figure out how teachers would conform to model of teaching that aligned with this ideal learning scheme---attend any faculty meeting and see how realistic that goal would be. The very unscientific truth about teaching and learning is getting an email or a call from a former student who either asks a question about something I had said in class or tells me that what I said worked for him in his profession. The research is clear on what students remember from the classes they take---very little. What students do remember are not facts, but personalities, or a fundamental idea that sticks--now how you quantify those two outcomes will always remain a mystery.
4
If students are so interested in market-ready skills then I humbly suggest that universities start measuring and enforcing attendance. Showing up is THE market ready skill. This is comparatively easy to measure and overall attendance rate as a percent could go on a transcript.
So why aren't we going for this low hanging fruit? Attendance is the responsibility of the student so you can't blame the institution or the professors. Maybe there is not enough money to be made from tracking attendance.
14
Here are the facts: according to the OECD, the United States ranks lower on several measures of educational outcomes than many other industrialized nations based on findings from the Program for International Student Assessment (PISA). We are 17th in Reading; 37th in Math; and 28th in Science among the 44 countries who participate in the assessment.
We used to be at or near the top, long before testing became in vogue. It is unfair to blame this decline on higher education, or the grades for that matter. As a lifelong educator, I have observed that good educational outcomes stem from parents who value education and insist on instilling that value onto their children, who are then nurtured by good teachers left alone to creatively practice their craft without governmental interference.
Parents need to redirect children from their device screens into books and hold their kids accountable for developing their innate mental capacities, which they all possess. Good outcomes will be a natural consequence.
7
When I began teaching high school English 49 years ago, college instructors decried students' ability to write a coherent sentence and blamed it on high school teachers. High school teachers blamed their students' inability to punctuate on middle school teachers, who blamed their students' inability to read at grade level on elementary school teachers. And elementary school teachers blamed their students' poor skills on ... wait for it ... parents.
"It's the parents" is a very old mantra. I thought then, and I believe it now, that public education's mission is to help every student learn, regardless of who their parents are. Just as high schools offered remedial reading classes and colleges offered remedial writing classes, you take the students where they are and help them advance. This decades-old trend of "accountability," demanding that "taxpayers" get their money's worth, is a bunch of bunk. Students who want their money's worth work hard to learn. I've had excellent teachers and I've had poor teachers. But at the end of the day, if a college or university requires a reasonable syllabus and the instructor/professor presents intriguing, challenging textbooks and lectures or other learning opportunities, the rest is up to the students.
The medical, legal and teaching professions have certification exams that measure what new professionals must know. That type of culminating "assessment" is much more valuable than meaningless standardized testing "assessments."
6
I agree with the author that the methods of evaluating students' learning has become another of the bureaucratic behemoths that sap time and money from the education process. That dynamic needs to be regularly reviewed for both its efficacy and for its influence upon the education itself. It is a well known scientific principle that the mere observation of a process alters it, and uncertainties abound with any observed results.
But the basic thought of assessing how well education is doing its job should never be in question, particularly given the trillions of public dollars that have flowed into the industry over the years.
When many of our colleges turn out graduates who remain at the same level of functional illiteracy as when they entered as freshman, one really has to wonder what, if anything, is being taught.
5
Agree in part but we come from the other side of the campus, namely science. It is much easier to determine if the student knows the material and can extrapolate the logical thinking to other endeavors. Clear concise writing and the ability to present the information is a requirement. It is quantifiable. We also do assessments before even admitting them to a major and sadly many don't make it. Much of that blame goes to the k-12 system. However, that is not a true measure of assessment. The real measure of assessment is 2, 5 and 10 years after they graduate. All too many colleges and majors shout huzzah after graduation and never follow their students in life. Within our department, there is a voluntary conscious effort to follow them after graduation and on the whole, that is where we assess our students. It works and so do the majority of students.
2
As a biology Professor for 42yrs, I disagree that paper or computer testing is a predictor of future success, generator of new ideas, productive leader in their field. Numerous studies conclude that GPA, GRE Mcat etc scores predict the success of a student in course work, whether at the undergrad or graduate level. These scores do not, however, predict whether an individual succeeds in his or her respective work after "formal" education has been completed. I sat on many committees that selected candidates of admission to grad school or hired Assistant Professors. We used every numerical descriptor and intuition available. Still in my experience fewer than 50% thrive or become leaders in their field 5yrs out of training.
1
Correct, but I never mentioned testing.
Like anything, assessment can be done well or badly. When done well, it brings clarity and consistency to what the university, program, and course are trying to achieve, making it more likely those goals will be met. The data can be more or less convincing and meaningful. Portfolios provide samples of work that students have selected and commented on; the reflection is meaningful and educational. Other good data include how well biology grads do on the MCAT, the medical school entrance exam, how many get jobs in their field, etc. Assessment is time-consuming but it provides some basis for trying to improve our programs.
4
I was in a secondary school situation where teachers were required to do assessments but also required to develop the assessment tools themselves since there was no budget for these tools. For several terms, I agonized over spending time on something I didn't have the skills for, only to discover later that my fellow teachers were just making up data and sending it in. When employees have mandatory tasks aren't meaningful or effective, they will simply find their own ways provide what is expected.
3
As a faculty member and later in my current position as the Director for Teaching, Learning, and Assessment at California State University, Monterey Bay, since 1997 I have been deeply involved in meaningful assessment that improves teaching and learning. It has transformed my own teaching and courses and I have seen it transform others, for the better. In a recent presentation at the 2018 Annual Meeting of the Association of American Colleges and Universities in Washington D.C., we shared our research on how assessment done right can engage faculty members in productive conversations that help them and the institution better facilitate student learning. When done thoughtfully, assessment helps us improve teaching and learning. Just because assessment can be poorly done and a waste of time and resources does not mean it has to. It is easy to find examples of meaningful and effective learner-centered assessment happening at different kinds of institutions across the nation. Those need to be highlighted so we can collectively improve our ability to use assessment to help us and every one of our students learn and flourish.
9
No one particularly likes finding out that they are not as good at what they do.
But, given the enormous amounts of public...and private...money that goes into education, it seems rather short sighted NOT to make some sort of meaningful assessment as what is being received for what is being spent.
And while I agree that the assessment processes should not become bureaucratic nightmares for all...and far exceed in expense their worth...I find it somewhat self-serving for the author to condemn either the intent, or the process, or both.
Teachers, at whatever level, are not the unassailable monoliths that they see themselves as.
Our teaching ranks have far too many incompetents hiding behind the muscle of a union or the life-time protections of tenure to continue to be immune from critique and the occasional brutal housecleaning.
3
An awful lot of hot air in this attempt to delegitimization evaluation. Isn't the evaluation system the strongest tool to measure performance? There are of course, many reasons for poor performance, only some of which, might be found in the classroom. But without measure, we cannot understand the degree or rate of progress.
6
So answer me this: Many of these students are going to leave the university with between $40,000 and $100,000 in debt. Was it worth It? Prove it
8
Every business and occupation runs on these numbers; like others, I am so glad to be retired.
3
"It preys especially on less prestigious schools and contributes to the system’s deepening divide into a narrow tier of elite institutions primarily serving the rich and a vast landscape of glorified trade schools for everyone else."
That about sums it up.
15
This line actually struck me as reinforcing a stereotype about higher ed that writers such as Frank Bruni have been arguing against for years. The "narrow tier of elite institutions" part rings true, perhaps now more than ever; but writing off the rest of the 4000 colleges in America as "a vast landscape of glorified trade schools" seems both misguided and, frankly, obnoxious.
Is the author including her own "big state university"--one of the best in the country, I believe--in this flippant dismissal of a lot of fine colleges delivering excellent educations to millions of students?
3
When we manufacture cars we can set up successful quality control evaluation systems to measure quality, why? Because all cars on that line are identical in every way. Quality measurement in this case has real meaning and use because quality measurement avoids waste and importantly the high cost of the quality measurement purchases continuing customers. Thus, the product and customer become enhanced.
When we educate humans we cannot set up successful quality control evaluation systems to measure quality, why? Because all humans in our classes are unique in every meaning of the word, quality measurement in this case has no real meaning or use, the expenditure becomes waste and the high cost of that quality measurement yields diminished education. Thus, the student and the educational system become inept.
If you want quality in cars, use a sophisticated quality evaluation system and develop it continually.
If you want quality in education, hire the very best faculty you can and develop them continually.
If there be any truth in these thoughts, then education has lost its trust in human value. This is then evidence that had we spent the same dollars in education on human capital instead of human measurement, we would have achieved a richer quality in both the student and the professor.
25
You had me up to this statement: "If you want quality in education, hire the very best faculty you can and develop them continually."
I agree that good faculty are the key to quality education, but how do you determine "the best faculty"? You need some kind of measurement, some kind of evaluation. I don't know that the tenure system was the best way to evaluate faculty, but it has been all but demolished in many institutions.
Our education system is suffering from the factory-like approach to evaluation and therefore to the education process itself. I read a stunning statistic about a year ago, that 70 percent of the undergraduate classes in colleges and universities were being taught by adjunct faculty. I used to be in that teaching corps. We were part-time, poorly paid and had little voice in how things were run in our department. This watering down of college and university faculty is still a problem, and it will not be solved by requiring "assessments" that do not take into account who the teacher is or what the teacher does, as well as what the students are learning or not learning.
1
Testing and assessments have poisoned the educational system from the ground up. It isn't geared to communicate or think independently. Trump sends his son to St.Andrews the tuition is $40,000 a year with a 12 to one teacher ratio. Private schools are exempt from the endless testing public schools are subject to and their funding is decreased the lower the test scores. None of this makes sense except for the elite, their disregard for those not in their socio economic realm is evident in all aspects of society.
9
Do you honestly believe that the elite institutions primarily serve the rich? Are you totally unaware that most of the Ivy's now pride themselves that at least 1/3 of their admitted freshmen class are first generation students or members of underrepresented minorities?
As for the other 2/3, yes, some are from very wealthy families, some are foreign students who somewhat manage to pay full price, and the rest fall somewhere between poverty and wealth and get substantial financial aid.
Surely you know this, or does it not suit your agenda?
5
But who will assess the assessors?
4
State legislators? Accreditation agencies? All of this stuff goes on
in between annual meetings at expensive hotels and well paid marvels of the moment.
A simple question: are teaching and learning more like continuous improvement in business or more like a faith journey?
1
If you torture your data long enough,
it will tell you exactly what you want to hear.
Funny to finally have higher ed making this complaint. Nothing, and I mean nothing had one more harm to generation of students than the test-and-punish regime ushered in by NCLB and doubled down on with CCSS/RTTT and the DWP.
10
Oddly, the author notes that "highly selective colleges" haven't jumped on the assessment-outcomes bandwagon. I've worked at one such university -- and it has embraced "learning assessments" with a vengeance. In fact, the author's own citation notes that the "vast majority" of US universities now carry out such assessments. Are they helpful? Hard to know or guess from this article, which is too busy claiming that assessments prey on "less prestigious" schools than examining pros and cons. The glaring irony here is the author's throw-away reference to "the big state university" at which she teaches -- as if the University of North Carolina at Chapel Hill, one of the most prestigious public universities in the country, is an open-enrollment community college. Please. On its web page, UNC-Chapel Hill notes its students are "distinguished not only by their academic credentials, but also by their extraordinary creativity, perseverance, compassion, and humility." In other words, lots of well-prepared, upper-middle class kids attend. Even so - don't they also deserve to have an education that matters -- even if it has to be measured?
45
As a college faculty member for nearly thirty years, I was so glad to finally read a sensible article about this issue. The press for "assessment" is not, as its propenents claim, a demand for better quality or "outcomes" (how I hate that word), it's a demand the eductaion reduce itself to its utilitarian skeleton. What can one expect from "assessments" that are driven by buzzwords coined by people with no idea about what higher education does? It's really just an assault on education, or at least on any education that does anything beyond mere professional preparation.
162
And so, by your remarks, we are supposed to continue to pay extraordinary amounts of money to "professors" without gauging just how much they are contributing actually to educating students in their class rooms?
If we allow you to define both what is taught and the measurements of success, haven't we lost all objectivity as to what "education" actually means?
Sadly, Dr. Potter, your remarks are exactly the type of hubris that independent assessment is designed to flush out of its oh-so-secure hiding place behind union contracts and professional "tenure".
In my extensive educational life, I had but one or two "professors" who cared anything about their students except for their hand-picked masters and doctoral candidates. And certainly those candidates were far more interested in their own educations than those of the classes that they were teach for their "professor" who was off doing "research".
2
The fact is, there is no better predictor of outcomes in life than the ability to read by the 3rd grade.
In other words, the battle for education in college, is largely won or lost many years earlier.
It's easier to blame teachers, or blame schools, then to fix the more comprehensive and complex social issues that all contribute to educational outcomes.
The same issues raised here for Higher Education, apply even more so in preK, primary and secondary education.
17
Amen.
One thing the author (wisely) left out, to not seem whiney, I suppose, is the responsibility students themselves must bear. How do you figure in binge drinking, frisbee-playing and any number of other distractions that have an impact on (not) learning -- not just among disadvantaged students but possibly even more among the most advantaged students. You can lead a horse to water, even do your best to make that water look like the nectar of the gods, but you can't make him drink it.
11
My research into writing assessment and development has indicated across many settings, k-12, undergraduate and graduate school, that “duration of work” is a key quality of the process of composing highly rated work. Duration of work means the time between getting the idea or assignment and completing the final draft. It does not imply work every day during the interval, only that the work remains open to change. Institutions and teachers who both allow for the time to be at least a month, AND teach students how to reconsider and revise or extend in the meantime, create writers who produce more highly rated pieces. To do so, writing teachers need to be able to read, respond, and mentor as writers, not, as Ms. Worthen so capably points out, test and respond like automatons in the Pin Factory.
7
The purpose of assessment of student learning is not to generate data for accountability, but rather to improve student learning, and if you want to improve student learning, you do that in the classroom. Having learned the techniques of assessment in 2004 working on accreditation mandated assessment for my university, I found that the most rewarding way of using them is directly in my own teaching and grading. Assessment at the school or university level is too general and too far removed from the teaching and learning. I am a philosophy professor and we teach complex issues of thinking, reasoning, knowledge and ethics. I started using rubrics that I developed for my own writing assignments shortly after I learned about rubrics. I use them not to generate data about my students, but to provide feedback to my students on their papers. I then allow my students to rewrite their papers (they are quite motivated when they see the grade generated by the rubric) and for each rewrite I score the rubric and provide written or oral comments back to them. Invariably, the students' writing on the paper improves!
6
America's education system?
The state of education in America is best gauged by imagining, assessing, how it would respond if a new Copernicus, Darwin or Nietzsche would appear in society, in short, how it would respond in face of a profound jolt of truth, an intellectual earthquake suffered by system. My belief is that the educational system has not at all studied the profound effects of truth on societies, how even a handful of thinkers historically could change things entirely for a society, not to mention has it prepared itself along lines of designing itself as an earthquake proof building, only this time the earthquakes to be suffered are truths to society. In short the educational system is quite conservative regardless of political/economic leanings. It does not produce minds likely to cause much disturbance to itself not to mention will it admit such minds from without, and it is not at all made for the more profound jolts of truth.
The education system in America seems more crowd control, behavior modification, workplace preparation than anything else. A design set up by power political/economic in society to feed itself. Neither power nor education system has much tolerance for truth, not to mention powerful jolts of such which history demonstrates regularly arrive in the form of profound individuals. But my belief is education system, society itself, must be designed as a truth receiving and sustaining system, withstanding even most profound jolt of truth.
5
Very provocative points - there are certainly arguments to be made that powerful elements in the US today don’t want inquiry, rather, they want adherence to dogma. Perhaps always present, but omnipresent, in my view today. The financial crisis, by breaking to college to job machine, made a bad trend much worse.
At the same time, I always think back to the Coleman Report, and think le plus ca change, le plus c’est la meme chose! Focusing on the what versus the why is a bridge to no-where!
"Teaching it is not a cheap or efficient process. It does not come from trying to educate the most students at the lowest possible cost or from emphasizing short, quantifiable, standardized assignments at the expense of meandering, creative and difficult investigation."
And therein lies the fallacy in the entire "reform" movement in public education, which is designed to use standardized tests to identify "best practices" that can be scaled up to help "deficient schools" improve their performance as measured by standardized tests. The "failing" public schools serving those who do poorly on standardized tests, like the "less prestigious colleges", gear their curriculum to increasing their test scores while the public schools serving affluent and well educated children-- who do well on these tests without coaching--- offer a wider array of courses and opportunities.
10
I have come to see some usefulness in certain kinds of assessment, such as discussion within a department about the sequence and purpose of courses within the major. However, I agree with the author about the unreliability of assessment data. For instance, if we are to evaluate a senior-level writing course, we must assume that students have completed all of the other writing-intensive courses required at our institution. However, many students transfer in college credit from high school and many others transfer to us in their sophomore year. How are we to know whether poor performance on those senior-level projects is due to failure in our courses or gaps in the courses students took elsewhere? Beyond that, how do we ever know that any particular artifact reflects what that student really knows? The student might have had four other major assignments at the same time, have been hungover, have been exhausted from a part-time job (as the author suggests), have been struggling with grief after a breakup or losing a loved one. I have been astounded that social science experts at our institution and elsewhere do not regard assessment data with the same skepticism that they would bring to peer reviewed work. The process is often a sham. Another negative byproduct is that assessment methods rarely measure or celebrate distinction. They instead measure "good enough" benchmarks that reduce teachers and students to mediocrity.
8
During my decades in higher ed, the biggest change occurred after the Great Recession. With state funding drying up severely, the bean-counters took over and demanded the type of neat & tidy quantifications that work well in some fields but is lousy for education, which is inherently big, messy, and not at all a one-size-fits-all approach (you've got 10, 30, 100, 250 people in a class, all of whom learn differently). Hence, "assessments" and "Student Learning Outcomes" became in vogue. While perhaps well-intentioned, they have been and always will be an abysmal failure unless we’re willing to spend the time, effort, and personnel to do so on an individualized basis for each student, which will likely never occur for obvious reasons.
With the bean-counter mentality now firmly in place, most colleges are forced to take in many HIGHLY unqualified students. Administrators and politicians have readily jumped on the bandwagon, choosing to blame the vulnerable professors. My colleagues and I have taught so many students who truly don't WANT to learn, yet all higher-ups simply state that the blame rests entirely on us. Thus we are forced to lower our standards to produce graduates (thanks, bean-counters!), and ironically, those that do so are deemed better teachers by our administrators, while those of us who resist and try to maintain high standards are deemed "ineffective."
Higher ed has changed drastically from decades ago, with virtually none of it for the better.
11
As a UK humanities grad student with two years of teaching experience, all I can say is hear, hear.
But I just don't see how it could possibly change. Universities are run by administrators. Administrators don't shrink administrative behemoths.
6
I'm the assessment coordinator for a school located in a state university. Faculty work extremely hard delivering knowledge to their students. They are sincerely interested in their students' learning and continually assess it across the semester with exams, papers, projects, Socratic dialogue, and observation. They know what they are doing and are the best judge of their students' learning.
The formal assessment process, however, is a different story driven by the external accrediting agency of the school. We engage in time-consuming, wasteful, and non-productive efforts directed at pleasing the agency - or it threatens to revoke our accreditation. The agency charges high fees while claiming non-profit status, delivers little value, and despite the internally driven, significant, visible improvements in the education of our students, its sole concern is whether we comply with their arbitrary standards. The agency shows up every five years to 'assess' the school and we, in turn, engage in in a two-day Kabuki dance intended to please our overlords. They leave and we then return to the real business at hand: educating our students. Professor Worthen's whistle-blowing is spot on.
26
I'm the assessment coordinator for my program. Posters who think this is about faculty accountability or improving teaching techniques are mistaken. This is about the curriculum, not individual faculty members. None of this effort goes toward improving teaching techniques. (Faculty who cannot teach are identified the old fashioned way: through student feedback.)
Quantitative assessment of student learning outcomes is all about collecting information for various accreditation agencies. The faculty are asked to assess specific student learning outcomes in their courses, using course-specific materials. Their assessment bubbles up through the assessment process to track learning outcomes. Any problems revealed by this process are addressed at the curricular level.
I've been doing this for 10 years. Despite the vast amount of faculty time spent collecting and collating it, the quantitative data that we are forced to collect with course-based evaluations rarely flags a problem in our program, much less lead to meaningful changes. The continuous-improvement efforts that have made a real difference in our program have come about the old-fashioned way: through the qualitative, thoughtful reflection of faculty members and feedback from our advisory committee.
This should not be a surprise. The score on one question on a final exam is completely inferior to the qualitative judgement of a faculty member who has spent a semester working with the students.
20
Assessment is easy: pre-test; post-test; correlate the two; regress on the residuals, per question or theme; analyze the data. What is hard is finding the appropriate course material in the first place. That takes tactical and strategic planning, and close relationships with the students' potential employers --if any; some students attend university in order to learn, but not necessarily to learn something that employers value. It also takes money and hard work to do that well. That's where the Fed should come in, but chooses not to. I suppose that there are an insufficient number of neoliberals serving in the fascist regimes of the nadiring nabobs of negativism to do so. Perhaps later 2018 that will change?
1
The author leaves out the way administrators selectively use such meaningless metrics to exercise control over teachers and punish ones who are unorthodox or bold, or whom they simply have a personal grudge against. There is no practice or policy so empty or counterproductive that clever scoundrels can't put it to good use.
10
As an excellent test taker I applaud any method of appraisal that does not involve having to deliver actual results.
5
High stakes testing has devastated American education. It is a predatorial industry which profits off of the notion that standardized tests produce stronger labor forces. However, other than countries which struggle with basic literacy, there is no correlation between stronger economies and high scores in tests like the P.I.S.A. exams.
High stakes testing in the classroom has made education boring and meaningless. Brains develop via human experience which requires critical thinking skills. Temporarily memorizing baseless facts just to forget them in pursuit of the next exam is useless. The outcome is not just a student not prepared for college but citizens ill prepared to protect their republic.
9
Why not SAT in, GRE out (both free). Compare and contrast. Done.
I have taught at universities in the past, all the bizarre tests you are describing sound like a waste of time.
But as one poster noted, you can only work with the students you have--I tried to set high expectations and remediate as much as possible.
High schools need to do better.
3
Get money out of the performance system in education, and the outcomes will improve.
6
Critical thinking should not be the be all and end all of education. Being able to sit with a 'problem', insight, intuition, emotional regulation and a host of other non-rational ways of 'thinking' are all worthwhile capabilities that can be learned.
3
As publishers have lost their stranglehold on textbook sales, they have turned to developing platforms that they can control...like automated assessment instruments and online courses. They push the learning outcomes movement to make a profit, not to educate the student.
8
Rupert Murdoch is one of the most well known investors in this industry. Fox News was practice for his plans for education. First he makes the old unable to think for themselves, then starts in on the young.
1
The author buried her lead: "The push to quantify undergraduate learning is about a century old, but the movement really took off in the 1980s." This is not an education issue. It is a political issue. In the mid-70's, the country was hijacked by the corporatist right. Ronald Reagan and a succession of extreme Republican right-wingers and Democrat neo-liberals, legitimized privatization and commodification of the public domain, running government like (and by) business. Profit motives have driven public policy on education, health care, infrastructure, environment, justice, safety nets, and governance itself, using easily gamed "data-driven" and "evidence-based" metrics. E.g., “ed reform” is destroying public schools with charter schools using standardized test scores to de-professionalize and de-unionize teachers. Charters get stellar scores by throwing out students who can’t be drilled into getting good test scores; then they boast a 100% graduation rate. Their over-regimented students later struggle with independent work. The rejects, shunted to public schools, suffer disrupted education and feelings of failure. This excellent article should be part of a comprehensive documentation of the destruction of our basic social institutions by conservatives, whose goal is to eliminate all arms of government but those that promote wealth accumulation by the American oligarchy.
10
Student: I need some help picking courses. What will I learn in your class?
Prof 1: A, B and C, and we’ll know because you will be able to do X, Y and Z.
Prof 2: You will learn something intangible, ineluctable, ineffable, and unfortunately, also unmeasurable.
Student: Hey thanks! You made that easy!
4
Direct and truthful article about a topic I have been dealing with in my college departments for the last 15 years. Another way to make colleges as ineffective as many high schools and then point to teachers and higher education as ineffective! At last, a light!
5
One of the main problems at some higher end research institutions is that teaching is an afterthought to research. In the "publish or perish" setting at the University of California many professors are there to do research and are forced to teach. In many instances research aptitude and teaching aptitude are non overlapping magisteria. By way of example, my organic chemistry professor's English was so incomprehensible that most of the class could only understand every fifth word that came out of his mouth. By the end of the quarter the only students left attending lectures were those who spoke Hindi. All others had to make due with Classical Notes.
2
God save us from the assessment bureaucrats. How am I ever going to find a college for my daughter that doesn't waste its money (my money) and its faculty's time on "quantifiable measurement"? Just teach, already, and try in some small degree to instill a love of life-long learning.
14
college is a little late to be instilling a life long love of learning. don’t you think?
Bravo! The emperor has no clothes. Learning objectives trivialize education.
7
I was a professor of intellectual history for over thirty years. My intellectual formation was grounded in my relation (sorry, nothing sexual) to my teacher. It was a master disciple relation, personal knowledge. My problem as I began my teaching career was to teach what I had learned in the rationalized, bureaucratic structure of a modern university, the very structure that emphasized teaching outcomes. I did that by transforming my knowledge in such a way that it satisfied the administrative requirements.
A second problem was presenting the material in such a way that students in large undergraduate classes (around 200/class) could feel comfortable with what I was doing. To do this, I set up specific course requirements. Moreover, each assignment was given a numerical grade. At the beginning of each class I handed out a numerical grading scale: how many points would give you an A, how many points a B, and so on. One last thing: the scale by which I graded the students was not the scale I used. The scale I used was set 5 points below the scale I told the students: If I told them the cut off for an A was 90 points, my actual cut off was 85 points, and so on.
So it is possible to square the circle of teaching something intuitive and visionary with the quantitative rationality that permeates our modern world. Buddhists have a term for this ability to take what a culture gives you and transform it into a vehicle for teaching something important. It is called skill in means.
4
"The reasonable man conforms to the world as he finds it. The unreasonable man insists that the world conform to himself; therefore all progress is made by unreasonable men (and women!)." George Bernard Shaw
So are we discussing education or are we discussing the workings of the corporate mind, and how it has marketed itself successfully to the citizenry at large?
The corporate world and its useful handmaiden, the political world, has no understanding of nor interest in education. Operating on the basis of its Iron Law, if you can sell it, it's true, this mind looks for markets and market opportunities. And it has found a large and useful one in the part of human endeavors once called education.
Broadly well-informed, deeply-thoughtful and reflective, highly-skilled, unorthodox minds are useless at best to this parasitic alliance, and profoundly dangerous at worst. We don't "buy" anything.
I teach way off in deepest Asia at an International School, IB students in the 11th and 12th grade, and one would think I might be insulated from this swirl of Newspeak baloney. But no.
I/we also have suffered the afflictions of data worship and technology bias, the belief that data ALWAYS speaks for itself, and that doing anything at all while mediating it through technology exalts and purifies it somehow.
The mostly invisible parts of the modern world that make life worth living were not created at the end of a cost/benefit chain, nor did they make any company rich. And yet . . .
6
The whole system is archaic and not working at all. This argument that college is some great intellectual adventure is wrong and has been at 95% of colleges. One or two classes in four years are even worth taking. Teachers not teaching much naturally resent being shown that this allusion of great intellectual conversation they are awakening in students is wishful thinking. College is no great intellectual adventure anymore if it ever was....it is just a piece of paper you have to have to show persistence for 95% of students. The emperor has no clothes.
1
I hate to say it, but one should not be too surprised that colleges - and their professors - are now bearing the brunt of attacks by those corporations that foist ideas of "quantifiable data, showing how much students have learned"; "learning assessment"; analyticity; outcome-based education; etc., in efforts to discredit current practices in the hopes of selling high-priced assessment tools, thus enriching their bottom lines.
Corporations, some foundations, and venture capitalists were all too successful in denouncing P-12 schools and teachers therein. Colleges are just the next set of pockets to raid.
4
There is only so many hours in the day, and only so many hours in the week. This development assumes either a) professors can trim the hours they spend on reflecting and thinking about their classes without much being lost or b) the technology and rubrics being deployed to collect these assessments are so easy they will have little time impact.
Both assumptions are false. And I cannot believe we are actually arguing this. Just ask public school educators how much their ability to be creative teachers has been compromised by the rise of the all-important testing regimens of latter years.
The administrative mania in the US ultimately does not trust teachers to teach. So they want to measure us 10 ways till Sunday. But look: we either want teachers to teach, or to be data collectors for an administrative regime. Can't have both. Only so many hours in a day.
The fact this puzzles our education leaders disturbs me no end. And also there is an insidious underbelly to it: if something can't be measured, it must not be worth much.
And thus are we surrounded by one of Oscar Wilde's nightmare people, who who know the price of everything, but the value of nothing.
It is bad for the classroom. It is bad for the mind. It will turn our schools into nothing but job prep factories. And thousands of years of culture flushed away as if so much useless dross.
This is, I fear, a slow moving "Cultural Revolution" like what China suffered under Mao, capitalist version.
14
Assigning grades is not particularly difficult - unless of course you want to do it fairly and accurately. These "learning outcomes" analytics companies are offering a service that takes care of the easy, low hanging fruit. They provide statistics that show trends and satisfy academic bureaucrats but they almost never get at the truth - what did the students learn and how hard did they work learn it?
Over my years of teaching I've settled into a kind of triage process that I apply to each of my students: Did they try to do the work? Did they finish the work? And, did they do the work well? That means for me, the subtle grading questions are typically reserved for the final group whereas a more sociological introspection is applied to those that struggle with the first two tests.
5
The tech has its place, certainly.... but it is marketed in ways that pretend it can do what it clearly can't.
As education transitions, as it must, charlatans from Google and Apple, especially, have been parading out a bunch of pricey 'edu-toys', designed primarily to capture their young audience for later exploitation.
Access to relatively inexpensive and powerful PCs, however, gives far more bang for the buck.
Often they provide the all-important 'data', which the taxpayers are encouraged to demand, and woe be him/her to defies the new orthodoxy....
Much of it, like Alexa, Cortana, and Siri are very slick but ultimately have more to do with consumption than cognition.... much more...
2
Thank you, Molly. This is the most honest, perceptive, articulate, accurate, and constructive thing that I have read about the state of higher education in 30 years as a professor at a large state university. I have witnessed, and have had to deal with, the same things that you report here. I teach in the physical sciences (chemistry), which is much easier to justify to the bean counters than an area of the humanities, and yet I experience the same frustration. The interaction between a teacher and a student, in any subject, is an art form. I think that it is safe to say that no one who is in the field of teaching is in it for the money - we do it because we like the work. It's a shame that well-intentioned people who value intellectual development and are trying to improve it are slowly killing that student-teacher interaction by laying on these accounting procedures that don't really measure what we are trying to achieve. We will cope, but this extra burden is slowing us down and taking time away from the important things. Hopefully the day will come when people will trust teachers to teach.
7
I should like the test constructors to please make a true-false quiz on the relationship of Goethe and Mozart. Surely it can all be mathemetized.
2
T.S. Eliot wrote in an essay in the early part of the 20th century that education can't do a decent job if the larger society doesn't know what it's about. Sums up a lot: 1) with our society in the condition it's in, it's natural that education, pre-K through 16, would harbor myriad problems, and 2) with no real national mission or vision of any kind for education, schools are whipped to and fro under various administrations and with sundry fads.
Assessment as it is done now is one of the poorest excuses for accounting for student learning, or for measuring teacher or school progress. The business model laid on schools years ago, further fueled by various companies preying on learning institutions, has given education a data-driven hunger that is myopic, unreliable, and invalid. An individual student's learning is the last thing that most numerical assessments tune into -- multiply that times a classroom times a school times a district and you get a boatload of numbers but no edible tuna.
Unfortunately, we have zero political leaders genuinely interested in any kind of critical, creative, or practical solutions to our educational mess. A new paradigm is needed -- we've tinkered with the antiquated model, and found that putting new doorknobs on the log cabin just doesn't do it. Meanwhile, the potential of millions of children and young people is squandered, wasted, and squashed, all while data crunchers profit and teachers grow numb. Now -- guns for them! Madness squared.
6
Unfortunately, most colleges has become the new high schools. As such we do need objective measures of learning accomplishment to assure that these expensive colleges are preparing the next generation of workers for the challenges ahead. To say that these measures favor the rich is absurd, They favor the learned.
1
K-12 schooling is explicitly not "higher" education, and because of that it does not prepare young people for college. It prepares them for exactly one perspective, and to ignore or scorn all others. I flunked out of college twice because I had never been told why I was there. And then I met some German students who told me: you are there to learn to recognize and evaluate perspectives. Aha! After that I was on my way to a Ph.D. in philosophy and a happy fifty-year career. A recent book by Philip Tetlock, Superforecasting, relates in detail the power of aggregating perspectives.
Stuck in the 20th century? The PUBLIC education I got is what parents pay 10s of thousands of dollars for today. It's a commercial tool and the internet is not magic and it runs on POWER AND ELECTRICITY. So when that stops, the analyses will stop. I don't call that an advancement (and I have a Masters degree from a "prestigious" school) Quit whining and TEACH our children. This country is way behind all the other industrialized nations of the 1st world. TEACH OUT CHILDREN. This commercial foolishness does not do it.
The PoliSci 101 course I took pass/fail as a senior pursuing an Engineering degree has been the most impactful course I took as an undergraduate. This course shaped the way I have viewed the world in the 25 years since graduation. There is no quantitative measurement of 'learning outcomes' that could possibly capture the benefit of this course to me and it would be a shame if such measurements had prevented me from taking this class. I believe Ms. Worthen captures this need very well in her penultimate paragraph when she speaks of 'the gift of time in an institution where curiosity and discovery are the source of meaning.'
12
Can we put this in k-12 so these unprepared students don't have to go to college unless they have the means to and it's ok for not everyone to have a degree? Isn't higher education needed in this country since the standard of public general education is just so bad? What's the point of fixing it at the top when we could address them earlier when student inpact could be great? And we have way too many colleges. With technology, we can all take online classes or just read books on subjects we want rather than have students go through so much requirement for a degree. Those degrees themselves aren't really preparing students for the real world anyways. Just like the students who have to work to pay for college, I've seen exhausted, incompetent or just inexperienced teaching assistant instructors in a big research university especially in STEM classes. They are supposed to be in PhD programs or have PhDs but they struggle teaching the basics. Why are they doing this for? Oh yeah, money. STEM PhDs usually get grants and fellowships but they have to work below min wage to help with their departments. There are some amazing adjunts professors and some tenured professors are horrible. I had an old man, a sweet guy who has been in the department for years, but who really should have retired 10 years ago, as a part time professor just because the department liked him. And this department had to be accredited to be a legit engineering department.
First, It's not all that challenging to get into a decent college nor does it take much work to graduate with that sheep skin.
Second, dad and mom can pay cash, you are in like Flynn regardless of high school performance.
Third, if you must finance your education graduating with a debt the size of a mortgage, you had better choose a career in health, IT management or, finance.
The ridicules expense of pertaining the skills for a profession that speaks to a person passion is gone. High school and secondary education are both simply mills that spit out graduates paying close attention to the race, the % of graduates.
Hear, hear!
But tell that also to the admissions office, which puts pressure on high schools and high school students to accumulate points of various kinds, and so on down to the kindergartens. It's worst for high school students, though, because they experience a terrible competition for places at universities.
As an educator I know instruction and assessment leaves students unprepared for college, in large majorities. Teachers are not trained to do anything but find strategies that carry labels like student-centered, or apply faddish approaches like PBL, personalized learning, flipped classrooms, etc. These claim to teach 21st century skills, a synonym for critical thinking and advanced problem solving. The problem? Teachers are not prepared to do this meaningfully. We are the products of the system we are to reform. Teacher training programs produce teachers with little understanding of cognitive science. We learn strategy after strategy that suggest rigor and higher order thinking, but we confuse the task with thinking and we never assess for deep understanding , durability of learning, and transferring learning to contexts beyond the classroom. Teachers present to students who do not know how to study, due to the fact that after all the high-minded strategies that students pass through, we assess with crammable tests, and infer from that that learning has taken place, and we have met our obligation to teach. Weeks late or months later, students can't pass the test any longer. But they too, think they have learned, because they have been compliant and earned the A's or B's that communicate a job well done. Disadvantaged kids are the first to fall away when these poorly developed skills fail them in college. Many more follow.
9
Measuring learning outcomes has its place and if introduced for the right subjects and without overestimating the validity of tests that measure learning outcomes it can be very useful....as an adjunct to inform teachers how to tweak their curriculum.
However, the current obsessive data driven approach to education masks the one true means to improving teaching and therefore student learning, and that is continuous, meaningful teacher training. But that is expensive, so burdening teachers with number crunching, which produces bucketfuls of data of dubious value, serves at best to check off ideological boxes of accountability and objectivity.
20 years of High School teaching and countless training sessions on data driven learning assessments have done little to improve teaching and learning in my classroom. But the few times I have received subject specific training, which were usually teacher-expert led, I was usually given materials and strategies that help me and my students improve learning.
We all know what really works, we just don't want to pay for it!
9
As a public school teacher, I'm asked to measure students' learning outcomes on a regular basis, and for good reason. If I can't show concrete evidence that students are learning, then I shouldn't get to keep my job.
It blows my mind that these college professors are getting paid primarily for their research, and that they're being allowed to drone on and on without ever checking to consider whether students are comprehending the lesson. There are many exceptions to this, of course, but as a whole the system doesn't place any accountability for college professors to ensure students are learning. Far too many college professors fail to engage in student-centered instruction, and focus on their own egos more so than scaffolding the skills students need to learn to survive in the real world (surviving can be economic, but also existential).
In secondary public teaching, we don't get the excuse of: "Oh, these students aren't learning because of social factors." Despite the macro-factors beyond our control, we're still held responsible for students' mastery, and college professors should be too.
3
Zachary,
The overwhelming majority of college professors are not paid to do research. It is a small minority of professors that have tiny teaching loads, especially nowadays. In fact, most of the people teaching at universities are not paid to do any research, e.g. part-time adjuncts that universities hire to teach classes like writing and math. Those that are paid to do research primarily do so at large research universities. At my university for example, 45% of our total income comes from research grants, written by professors. Research, in this sense, keeps the lights on, and moreover, keeps our universities among the best in the world.
Nobody is looking to make excuses. Nobody is saying we should not assess university teachers. A contentious relationship between k-12 and universities serves no one.
13
I, too, am a high school teacher and I know well that schools do not prepare students for post-secondary education, in the main. We do use the excuse that macro-factors are beyond our control, even though there are few sympathetic ears. Then we graduate students, the vast majority of whom cannot succeed in college because we teach, use strategies, and apply assessments that are crammable, and require no long term retention of content or skills, and we definitely cannot indicate that we do anything that would prepare our students to transfer their learning to college. Almost any measure, PSAT, SAT, ACT, etc. shows that students tank complex questions that require inference and complex reasoning. We are the ones who pass on students who study poorly and who think learning is flashcards and recall, even after being passed through so-called student centered activities and strategies. That is why only the best prepared from strong socio-economic backgrounds have a high probability of staying in college and eventually graduating.
14
This is an inaccurate portrayal of college. Most faculty are paid mainly to teach and are evaluated mainly on their teaching. They are required to give exams so that student performance can be graded. Teaching is evaluated multiple times by direct observation and review of teaching materials and student evaluations. These weight heavily in the tenuring process.
Assessment of student learning outcomes is none of these things. It is an add-on activity that says your standard examination and grading scheme isn't enough so you need to do a second layer of grading in a particular numerical format that literally was invented to evaluate the quality of durable manufactured goods.
66
When I hear about measurement schemes (in education and elsewhere), I think of the phrase "you get what you measure". So, what is the "what" we are measuring for? What do want to "get"?
Education is such a big part of the overall parenting process (getting children from newborns to new adults who know enough to have a good chance at success in life (and not just in ways that can be monetized), we really should start out with a definition about what "good" looks like in terms of a well informed, ready for life, new adult.
Not that this would be an easy definition to arrive at, but without it all the measurement schemes are pretty much just chasing our collective tails.
When thinking about a definition, it would be worth considering that there really is more to a successful life than work (even though the other things may not appear to be monetizable, they are still important). e.g. IMHO STEM is not enough.
Preparing children for "work" is not enough, but might seem to be if the main or only KPI focuses on what can be monetized. Aren't we hoping for more than just "survival" for our children? Can we include other things that make life meaningful and worthwhile? And include them intentionally in an education framework? And go beyond the middle of the bell curve with that and allow for a wide range of interests, whether they are monetizable or not?
If we can't define it then measurements are a distraction, because they will not be informative or meaningful.
6
I've been teaching mostly composition at a medium sized mid-western university for more than 25 years. When I started, in our department we had 6 grad students TA's who taught one comp class each; each regular full-time faculty member also taught one comp class each term. We now have 35 TA's teaching comp classes; most of the rest are taught by contingent faculty who teach 2/3 time and make about $14,000 a year. Our department is huge, by far the largest on campus, about 80 people teaching at any one time. Last week I was told that starting this spring those of us teaching composition and any other courses that meet a liberal studies requirement will now have to gather and hold every document our students write or present and--in addition to whatever other assessment we provide students (individual conferences, written feedback, etc.)--we will have to evaluate every item we collect from every one of our students and judge whether it "Meets Expectations," "Exceeds Expectations," or "Fails to Meet Expectations." Every single item. Every essay, every journal entry, every essay quiz, every written reflection, etc. My guess is that it won't take long before we see less and less work being assigned--and the students will most definitely suffer for it, all in the name of Outcomes Assessment.
16
The people in the assessment monolith actually look at this stuff, talk about it in small meetings, ponder the data as if they were reading "two Corinthians," and then divining some numerical index for determination of annual evaluations which they want to link to salary increases but cannot. Why? If they could it would all be automated and the people in the monolith would no longer be needed. And then? Composition Teachers would be free to be with their students and enrich their lives.
2
Thank you for this article. A story: recently my former alma mater (I have a PhD and work as an assistant professor) has made major cuts to my former department, cutting incoming graduate students by half and letting programs go extinct by not replacing retired professors. There are pros and cons of these changes, but in relation to assessment, this department had been told of the importance of the assessment measures, and had spent a number of years demonstrating that not only did they meet their objectives, but did so with excellence. Of course they were trying to satisfy what was asked of them, but in the end, it didn't matter at all, since the cuts were justified by decreasing undergraduate enrollment (note: this is a department in the humanities).
All this is to say, assessments are used to justify administrative decisions, many of which have already been made. The larger problems of "neoliberal austerity" and a detrimental disconnect between the wildly varied standards of k-12 and higher-ed (e.g. unprepared students) are instead set aside to blame professors. Yes, we will collect data and report it. These numbers can help us measure some kinds of success. But the narrative that what goes on in the classrooms is the primary problem with education has got to stop.
16
Fairly objective reasoning of college/university education, a unique time and place where we learn to think for ourselves, develop social skills, learn how to enjoy studying in a cooperative environment (instead of a competitive one), as opposed to trade schools geared to teach us skills job-ready but not necessarily a well-rounded individual, a 'renaissance man/woman' with the primed imagination only a liberal, and secular, institution may provide. Open mindedness, or at least tolerance, for opposing ideas, demanding formal and informal discussions and, hopefully, be able to come closer to the truth, a beauty to behold, is what we may call a cultured person. Priceless. Relevant. Even transcendental, beyond our own limitations and frailties, bigger than ourselves. How in the world could you measure all this, without amputating our mind's unique wonder?
6
"The trouble with you, Kay, is you tell it like it is to people who don't want to hear it like it is."
"You made me look bad in front of my people."
Don't try to tell me that American businesses want employees with critical thinking skills. I know they DON'T!
American businesses want employees who won't upset the apple cart!
As I learned while getting my MBA: "American businesses are operated primarily for the benefit of executive management" and that's the apple cart which must not be upset.
As for me, I have concluded that the primary reason discussions such as this one go round and round over and over is a failure to understand (or to acknowledge) that the primary purpose of the American educational system through the level of a college is to keep people off the labor market for as long as possible.
This began with the GI bill and has only grown more and more focused. Students go to school and to universities to 'get a piece of paper' not to actually learn anything. What a student needs to do to get the best grade and what that student needs to do to learn the material are generally quite different and frequently incompatible. Large numbers of students have figured this out and are gaming the system just as the system is gaming them.
Consider how many job descriptions list a four-year degree in ANY SUBJECT! If employers don't care about your major, why should you? If employers don't care about what you have learned, why should you?
10
Spare me the bloated rhetoric.
I work in institutional effectiveness for a NYC-area college. As an assessment professional, my belief, and that of many of my peers, is that it ultimately comes down to improving student learning. Don’t we all want to ensure that our students are learning what we purport to be teaching them? And by the way—assessment is happening campus-wide. That means all the administrative and support departments too. We all have to define what success means and work towards articulating it in a meaningful way to our stakeholders. If we aren’t doing well, then corrective action is needed. So please stop playing the “over-burdened, under-paid faculty member forced to jump through hoops” role.
3
The very existence of "institutional effectiveness" as an employment category--there could not be a better example of the condition our colleges are in. Managerialism has won the day.
16
Rebekah,
This editorial is not saying let's get rid of assessment, but let's find new ways of doing it, ways that take into account the fact that what students learn is not entirely the result of their teachers, and that producing the ideal worker is not entirely what an education is about. It's a nice idea to think that decisions are made due to a rational analysis of the numbers, but that's not how it always works. The "bloated rhetoric," I'm afraid, can also be a powerful force.
8
Seriously? I’ve been a professor for nearly twenty years I’ve yet to see assessment make any improvements in student learning. Assessment wastes my time and the university’s money. Period.
5
Much like the almost religious fervor that has seeped into the politisphere for free market solutions to every problem, there is a similar nearly religious belief that everything can be quantified and that once quantified made meaningful.
I like to think of data and data analysis the same way I think of the term "natural." Natural food sounds good, wholesome. We should all eat Natural. Unless we plan to ingest yew berries or nightshade, both of which are fully natural and often organic. Natural can be both wholesome and toxic. So can data.
Not everything can be meaningfully quantified. Some things require qualification instead. But qualification requires expertise, judgement and expense, and the result is both subjective and not easily compared.
Data is great until you remember early computing. Everyone then knew of Garbage In Garbage Out.
Figuring out how to teach critical thinking is not something that will be data driven. If your class is busily absorbing pictures on Facebook that are patently false, are shown that they are false, and still believe them to be true, no amount of data is going to help assess the "learning outcome."
7
This is what was said about measuring health care quality/outcomes 10 years ago. In that case prefect was the enemy of improvement. Perfect measures don't exist, sometimes you need to start.
2
I’m with you, but before the assessment regime, there was essentially no mechanism to generate the type of collegial and useful conversations about teaching you say are necessary. Every discussion I ever had about teaching was anecdotal, and there were department members with who I never discussed teaching, even if I knew them quite well.
If nothing else the assessment regime has forced faculty to have discussions about teaching and learning that are something more than complaints about classroom banalities and bad writing.
3
There's a basic confusion between the results of papers and tests, which are based on large amounts of knowledge, reflection, and skills of innumerable types, all of which are at issue throughout a course, and so-called "student learning outcomes" or SLOs, which are usually two or three fairly random things labeled skills that may be tangential to the course, that are taken to be independent of any "content" or actual knowledge, that may or may not ever be read and used to improve courses, and that in many cases actually involve long-term development rather than any particular course--they tend not to be readily teachable skills like typing. So, the idea that a system of education becomes rigorous only when SLOs are implemented is ludicrous. As professors, we are all in on student learning; that is our entire mission in the classroom. SLOs have little or nothing to do with that, and it gets much worse as we all are instructed by administrators to design courses backwards with only these same minute SLOs and no actual course content, because they think knowledge doesn't matter in the Internet Age.
10
Josh ...
I do not know where you teach but at my university the outcomes assessed are drawn from the formal program and course outcomes established in the curriculum development and revision process. We faculty decide which course/program outcomes will be assessed and they always are the ones we deem most important. They definitely are not "two or three fairly random things labeled skills that may be tangential to the course." Perhaps the problem at your institution is not assessment per se but how assessment is being done.
I am a bit confused by "we all are instructed by administrators to design courses backwards with only these same minute SLOs and no actual course content" as faculty control of the curriculum is an accreditation requirement of all the regional accreditors, including SACS which includes the state of Florida. Is your institution SACS accredited?
1
The author hit the nail on the head in her statement: "the system’s deepening divide into a narrow tier of elite institutions primarily serving the rich and a vast landscape of glorified trade schools for everyone else."
Since IQ tests were outlawed for hiring purposes (Griggs vs. Duke Power Co. 1971) companies have used a college degree as a surrogate for having adequate intelligence to do a job. But now that college degrees are proliferating and many graduates can't do basic math or write an intelligible paragraph, only the elite colleges provide a meaningful degree.
But this is more because of their selectivity than what they teach. Let us not forget that both Bill Gates and Steve Jobs were college dropouts. We need to significantly reduce the number of colleges and students going to college if we want a degree to mean anything. But the vested interests in the education industry will prevent that from happening.
8
The lack of self-awareness of a professor at UNC-Chapel Hill complaining about academic oversight is pretty rich, but, I digress. I cannot speak for any higher education institution other than those where I have worked, which is something the author does frequently, but in my experience assessment has been completely different. Assessment is faculty-driven by a quest to know what their students are learning and retaining in order to prepare them for their post-graduation experience. Additionally, while she repeatedly states that assessment burdens small regional institutions, those are the very institutions that do assessment best. At any of the places I have worked assessment has never been viewed as a financial burden. The reason we spend as much time as we do on it is because the faculty are committed to providing quality educational experiences for students. Finally, I taught political science for ten years. As I've watched my former students flourish I have rarely considered whether they remember Polk's presidency. Rather, I hope they still remember the communication, research, and critical thinking skills I tried to pass along in my classes. Those are the skills our students need because with the exception of higher education there are few fields where someone can simply be an expert on one thing their entire career. Furthermore, the desire to ensure our students are prepared is not dictated by tuition. It is a moral imperative, not financial justification.
3
just think about the assessment as a complicated intellectual problem, solution to which will help to evaluate the teachers, Universities and improve the outcome for students. Isn't it what the college education is about (at least according to the author)?
Bravo! Worthen's concern reflects another kind of outcome--the gullibility of FOX News viewers. I graduated from a small liberal arts college that required all kinds of courses in the first two years before going on to the courses largely in my major in the last two years. During those first two years, did I complain about all those seemingly irrelevant courses? Yep. Am I now grateful for them? Forever.
7
Setting learning objectives and measuring outcomes can be great in Science and Engineering -- graduates need to learn specific skills in order to build a bridge, for example. Many scientists and engineers need to pass licensure tests that focus on specific skills that are required to do a job. Personally, I found including learning outcomes on a syllabus to be extremely helpful for classes in science and engineering. The students could see exactly what they were supposed to get out of the class and how it would help them in their future careers. It can work great.
I can understand the problem for the humanities, though. People in different disciplines often think differently and have different goals. What works for scientists and engineers often won't work for others. Sometimes you just need to hire good faculty then get out of their way and let them teach.
That said, how many classes in state schools are now taught by graduate students rather than actual professors? The proliferation of classes taught by graduate students, post-docs, etc. is a big problem the learnings assessments aren't going to solve.
7
American universities are becoming high schools because, as far as I can tell (and I teach at a university part-time), administrators want to maximize profits by letting in as many people as they can, more or less regardless of whether they are ready for university-level work. As a teacher, when faced with a critical mass of unprepared students, you do what you can, but ultimately you have to lower the level of assignments, readings and your own expectations, all the while inflating grades. I have told myself many times: This is the level of student that my employers have given me, with the implication that these students should set the level at our school. I then follow suit. After all, how can I impose unrealistic standards (gleaned from own undergraduate days 30 years ago) and then fail half or more of my class?
182
Accurate observation. We have assumed that college is "good" for everyone. We actually have few opportunities for people who don't go to college--so we change our colleges to accommodate those people as best we can, hoping that it will socialize them and change them in a helpful way. This is expensive and not always very effective. College is generally a safe holding tank for many, while they mature, but it requires lots of top down action and support to recruit and retain these students and to keep them from doing poorly.
It's possible that we need new institutions that provide education and training at all stages of life and work. It is almost certain that we need new kinds of apprenticeship and training programs for young people who do not flourish in college. We need to recognize the diversity of needs and abilities and goals in young people, and give up the idea of a single mold.
13
There's a push now to pressure open-admissions public universities to become more restrictive in their admissions: university funding formulas are are now based on student success (graduation rates) as well as the enrollment itself. (Universities are also having their remedial courses taken away from them.) So the university realizes (or is told directly, in case they don't take the hint, which happened at my university) that interventions to increase retention and graduation rates are expensive and ineffective, but increasing admissions standards is effective. What's really going on is that the weaker students are being shunted--by design--to community colleges. This saves the state money, because community colleges are less expensive to operate (relying heavily on low-credential, part-time instructors). The result is very poor success rates for the students involved. But this is not necessary. There are public universities--Rutgers University at Newark, and Georgia Southern University--that specialize in admitting "at-risk" students, including minority students, and that have high graduation rates for these students (50-60% or more)--equal to their non-at-risk, non-minority students.
4
I started my career in higher education 25 years ago thinking that assessment was a straightforward task of unquestionable importance. I still think it is a very important task, but I no longer trust any assessment tool--ones I write myself or ones developed by specialists--to tell me everything I would like to know about how my students have improved their minds while in my courses. I keep trying to improve my teaching and my assessment, but I have learned that there are limits to what I can know. Most importantly, I have learned that focusing only on the easily measurable learning outcomes potentially leaves out some of the most important learning.
I read one editorial in an educational journal that said, in effect, that we shouldn't try teaching students to be creative scientists until we figure out a good way of measuring how effective we are in teaching students to be creative. At the same time I was reading how hard it was for researchers in the field of creativity studies to define and measure creativity. I was also reading other editorials saying that what schools really needed to do was to teach students to be more creative. Does a conscientious educator despair of trying to teach students to be more creative just because creativity is a hard thing to measure? I would hope not.
2
This excellent essay hits home for me as a person with many years in public education, beginning in the early 1970s. As the accountability movement, with its demands for quantifiable assessments, hit public education in the late 70s, there were debates within public education about the rise of the movement to focus all our efforts on what could be measured. We had a saying that "You treasure what you measure." We might now call this a meme, and for many of us it was a call for more complexity in our understanding of how to assess learning.
In Maryland, and in many other states, the Business Roudtable became a dominant player in education reform in the 1980s. Our top administrators began to promote the idea that schools must teach the skills that employers need. I do not oppose this idea but only its dominance over other values of education. High-stakes tests became the centerpiece of this focus on workplace skills. Nationally, that has led to much opposition but little understanding of the issues involve. It is a topic we need to pursue.
4
People love to complain about tests. Assessments are essential in many facets of industry, careers and life. University life is no exception.
When I was in the military some loved to complain about the annual physical fitness test. ...That it didn't really measure if one was physically fit or not. Those who passed the fitness test were approved for professional development courses and subsequent promotions. Those who failed the fitness test were left behind.
Which military leaders would you prefer?
4
At least on the college level, tests are not evidence of what folks men by "assessment." I've been teaching philosophy for 30 years, and like all my colleagues, I've assigned tests, essays, and other assignments that I have graded as measures of how well students have mastered material. Assessment is supposed to operate at some "higher" or more "objective" level where one tries to measure whether professors are accomplishing what they set out to accomplish in the classroom, and the professors evaluation of progress or success is no evidence at all. The idea is that one needs some kind of hard data, some objective measurement with numbers to verify that successful teaching is happening. One enormous problem with this is that with any subtle, complex elements of education, you are left trying to assign mythical numbers to things in order to "prove" results, a little like trying to assign a numerical score to love.
5
There are many kinds of tests, and many tests are essential. Almost all teachers engage in testing of some sort. But what the assessment industry has designed are nothing like tests. Professors give tests that measure knowledge of subject matter and the ability to make use of that knowledge in different ways. The managerial assessment people want to force another level of testing on professors and their tests--and this managerial class does this without knowledge of subject matter. It's quite literally ignorant. And the ignorant are in bureaucratic control.
2
Much of the problems in education stem from lack of reading skills. Believe it or not, most high school students do not read. If you want to improve performance in higher education adopt programs from scholars like Penny Kittle and Stephen Krashen, proponents for high volume reading, high interest reading while at school. Too many lectures, too much sage on the stage in our current culture. Instead, get students reading. Watch them read. Give them an accessible library to choose from. And, most importantly, Give them TIME at school to read. Less "assessment" more joy reading.
12
I taught English as a foreign language to college students for thirty-odd years in Seville, Spain and have seen the very same scenario and transformation you describe play out here. For most of that time my colleagues and I enjoyed a good deal of autonomy as to course design and teaching and evaluation methods. That came to a fairly abrupt halt when the university system shifted from having us try to teach the students to master a general ability to speak, read and understand a foreign language, a gradual process which is very difficult for many people in a classroom setting, to having us try to instill a series of specific "skills and competences" which would allow students to check off some items to put on their CVs. A whole Europe-wide assessment industry grew up to first advise us and then require us on how to teach to their standards. Hours and hours then had to be spent to design and pre-test exam items and formats so that a predetermined percentage of students could answer appropriately. Test scores have possibly improved but it seems to be more the result what is commonly known as "dumbing down". The focus on individualised teaching and testing on what was actually taught and hopefully learned in the class changed dramatically. Personally I was able to retire before the full weight of this new paradigm set in, but my former colleagues inform me that it has only become more mechanical and nightmarish.
I see many responses here from teachers; and students?
6
You are right that this is a planetary virus. Managerial creep and managerial fake-measurement can be found anywhere that people who have knowledge are organizationally subordinate to people without real knowledge. This is one important factor in the decline of universities. Supporting a managerial class in addition to teachers and researchers is also an important factor in the high cost of college.
3
Most of the curricula of colleges and universities have been taught for over a century and academics know how to measure students' success with regards to what that they have studied. But since the 1960's technology has changed so much of how we work and live that the relevance of the existing curricula with regards to what people need to succeed well enough to justify what they spend time learning has become critical to the future of higher education. A lot of academics may think that they can use their knowledge and intellectual skills to determine the changes that must be made without actually collecting a lot of data about things which are needed to adapt curricula but without hard and representative data from how education affects people's abilities to succeed, it's just a lot of hypotheses in need of confirmation. That's where the statistical work becomes important.
I graduated with a bachelor of arts degree and I could think and express myself very well but my math and science knowledge was thin and eventually I found that I just had to go back to school to learn those subjects better. The work place requires technical skills not generally needed a half century ago.
1
Here are the only learning outcomes I value:
Students complete their formal education with a desire to learn more, an interest in the wider world, generosity of spirit, even toward those different from themselves, and a critical eye cast on the conventions of the "culture" that the mass media and the business world try to pull us into.
21
You said it. Bravo.
2
The points here make good arguments for apprentice programs, something much more common in Europe than here in the USA: https://www.theatlantic.com/business/archive/2014/10/why-germany-is-so-m... ("Why Germany Is So Much Better at Training Its Workers").
We have too many colleges and universities for our own good. What this has done, to a large extent, is create a lot of debt-ridden young adults and created an over-abundance of administrators trying to measure things they do not understand.
Ms. Worthen's points are well taken, but a good bit of painful dislocation would need to occur before we could better balance our "system." There are so many vested interests that calcify our ability to adapt on a national scale.
4
Schools should teach students not only information, more importantly: how think for themselves. Skepticism and thinking for oneself will make for good citizens.
2
I am a high school teacher who is very familiar with the assessment culture that is now trickling up to the university level. Another reason I think this has been happening during the last couple of decades is due to cheap and ubiquitous computing technologies that make vast amounts of data gathering possible. Let alone the cultural obsession with quantifying everything, we now have the "material conditions" to make all this possible.
10
In one humanities course at the University of Minnesota, before any of this assessment stuff, I was assigned 73 books to read for a 3 credit course, 1/6 of my workload that term. The other classes: Physics and Mathematics. Of course none of the students could read 73 books, that was not the point. The point was:
YOUR ARE HERE. AT A UNIVERSITY. GROWN UP. You will get impossible assignments. Get over it. Deal with it.
And I got thru that somehow. Assessed critical thinking? Who knows what particular thing I learned. But I was there, at a big time university with brilliant professors and star struck students. What I got was just that - BEING THERE. It was worth every penny.
7
A University is not about some particular thing - which might be be literary criticism applied to Hemingway or solving a differential equation. Its not that. Employers hire a grad for the transformative experience a student gets - just being around Nobel prize winners. A student gets grades or test scores, but they might not matter. He or she will not solve math or do criticism on the job (probably). Instead, a grad must tackle problems not imagined in 10 years, such as:
o Classifying synthetic proteins
o Dealing with adversarial examples in AI
o Furnishing a moon base
This stuff they cant teach in schools. Its never been done.
1
Your are here? Maybe others are at a university. You appear not to be.
It's just as bad in K-12. Everything we teach is suppose to be data driven; even if it's not applicable to your content area.
For example, I teach art. My kids have a pre & post test they have to take that's partly written. As well as demonstrate understanding of certain skills like shading, drawing 3/D shapes, 1 pt perspective & so on. I use it to show growth.
Meanwhile, I am suppose to collect data throughout the year & submit it. All the other content areas have fancy programs on iPads to track what their kids do.
I have asked what data do I need to collect, how & why. No one can tell me. I've had to create my own vehicle by picking a skill that we will continuously work on throughout the year & focus on improving students who are weak on that. Usually something like the fine motor skill of cutting intricately w/ scissors.
I can tell you after the first week of school who has problems w/ scissors. I make a point of creating projects to practice the skill every month w/ all the kids. Most will improve but some will not. The point is I didn't need data points to tell me where some students need support. I don't need to document that this kid hasn't ever used scissors, needs practice & is working on it. I just create the practice time as well as other techniques to get them more comfortable & confident in their skillset.
Who am I documenting this for? No one really. I'm just proving that I'm doing my job by filling out log sheets every month.
Data isn't everything.
15
It occurs to me as a teacher that there is a not-particularly-well-hidden agenda here - it is much easier to simplistically assess whether learning objectives have been met in the natural and applied sciences than it is in the social sciences and humanities, which have traditionally been targets of conservative administrators and politicians who have contributed to the commodification of university education. To me, the worst thing about the proposed means of assessment is that it completely overlooks the role of "testing" as a means of gaining ongoing feedback as a means of restructuring future lessons to re-teach points learners find problematic. To be successful, teachers have to "read the room" in the same way other performers do - something that one-size fits all assessment ignores, and software cannot measure. As Einstein is said to have said "Not everything that matters can be measured, and not everything that can be measured matters".
17
In trade schools things can be metered;that's what makes them trade schools. A university should be dealing with those things that cannot.
15
Puh-leeeeeeze. What do you think thoughtful professors do every semester of their lives. Good professors test based on the content that they have "professed"--not on the curve. Thus an A means you mastered 90% or better of the material covered by a particular exam. The rub is this: how good are the exams? Excellent teachers, many of whom teach gigantic classes which require machine graded tests, are not only excellent at teaching but also at constructing unambiguous questions that reflect by their frequency in the exam the emphasis and importance of a particular area of the curriculum as taught.
The pressure to raise grades is real and when you combine teacher rating/tenure trials it gets ruthless. But the pressure to remain true to your mission and the subject matter is a greater moral imperative. And Real Profs do it...
3
I teach college courses and have worked in my institution from the instructor to the academic dean level. But before that, I worked for many years in high technology marketing. Essentially, I believe that the current wave of assessment is like asking a marketing team whether their product is really good before it it hits the market. "Well, yes, of course," they will dutifully assert. Unfortunately, markets often have something different to say. Our approach in education is like testing a product for compliance with tech specifications and believing that is the same as market success.
The only way to validate whether colleges do a good job, in my mind, is to see what happens with students after they leave the hallowed halls, or are out in the market, so to speak. Testing before release will fail to to validate market readiness, though the student may meet spec. (sorry to refer to humans like software products, but the analogy holds). This means tracking, which is easy for some institutions, but not for others.
I'm not convinced we really want the answer to market readiness though. When I periodically suggest that the departments teaching our prerequisite courses work with us to evaluate how students are faring in the skill sets they teach, they invariably make the sign of the cross and mumble something about "how would you guys understand what we teach."
4
The idea of "market ready" is not quite what it seems. There are many institutions that teach specific skills paired with job placement, an effective way to use ones money for a definite payoff. There is another thing called an education. The fact is that the market changes, and quickly. There is an intellectual and critical tradition in educational institutions that create the sorts of individuals that can help all fields to adapt and change as they always do. The sciences and the humanities and the social sciences and the arts have changed dramatically to better serve their purpose of understanding the times in which we live and build upon and revise the knowledge of the past. It takes thoughtful and critical people to keep things advancing and to deal with the challenges we will face as a society on the most local and on global levels. I have spent many hours dealing with assessment issues, I have learned a few useful things but mostly I have felt that this time directly took away from time I could have spent with students or revising my curriculum and making my classes ever better.
4
Excellent article! I hope it is widely read. The big problem I have seen with outcomes assessment at universities and standardized testing in K-12 is the same. Some things we can teach are easy to assess and some are very difficult to assess. Generally the more important skills and ways of thinking and working are more difficult to assess that the mastery of factual information and procedures for doing things. As pressure mounts for assessment data, schools have shifted their curricula toward what is easy to assess and away from what is most important to learn. This makes no sense at all. Educators should decide first what students need to learn, and then assess it as time allows. Instead, they are pressed to teach what is easily assessed, and add what is important as time allows.
21
I beg to differ. I think assessment and accountability are necessary in higher education. I began my career as a commercial lender and later switched to teaching, beginning as an adjunct and progressing to be full-time, tenure track faculty. Frankly, I have been shocked at the relative lack of accountability in the teaching profession. In the banking industry, I was subject to annual performance reviews based on more or less objective performance metrics. As an instructor for the past 11 years, I was given no training and have had one (1) classroom observation. The students fill out evaluation forms at the end of every semester, but per faculty governance rules, these are not a required factor in promotion and tenure decisions.
Without the demands of assessment, I do not think that some faculty members would reflect on outcomes. Without the emphasis on measurable objectives, I think that some faculty would continue to employ the same pedagogy, methods, and material that they have used for 20 or more years. I disagree that assessment is trying to standardize teaching - we have the freedom to define objectives and outcomes within wide parameters that can be tailored to our subject matter. The key is to put your money where your mouth is. If you say your institution is producing well-rounded, market-ready, communication competent graduates, then prove it within a statistical bellcurve of data.
12
In your background as commercial lender you must have experienced the phenomenon of “redlining”. This is when lenders refuse to invest in certain communities because the data says that those communities are an investment risk. Interestingly those same redlined areas correspond perfectly to areas inhabited by people of color. In education, assessment data usually is cited in a similar fashion. Coincidence? Or racism?
2
Allison, your final words gave you away — “market-ready.” Oops, and I almost missed “statistical bell curve of data.” What do you teach? I doubt it is literature or any of the humanities. The author is from the University of North Carolina. My daughters both graduated from that estimable institution. One majored in English and the other, in French. They also took the traditional two-year broad-based core courses that made them well-rounded. They both took in depth history courses that couldn’t be evaluated by a multiple-choice test. They are both gainfully employed, though not in their major. Their broad, humanities-based education provided them with a great foundation for any kind of career. They are very adaptable and have changed careers more than once and been successful at anything they pursued.
1
There is a difference between knowledge and wisdom. Wisdom in academics cannot be measured in the same way as knowledge or factual information. Too often these days the 2nd & 3rd level university's use online classes to both save money and classroom space. This too demeans the experience of higher education--which in part to having critical interaction between students, as well as students and professors. A well-prepared class can make learning inspiring. How does one measure inspiration?
Unlike sports where one can easily measure ability--most students are unaware how bright and well prepared their classmates are. There is an interesting quote: IF ONE DOES NOT ASK THE RIGHT QUESTION--THE ANSWER IS MEANINGLESS! I sense from this article that too many testing algorithms likely begin with poor questions. The data/answers produce are often meaningless.
11
I taught introductory Astronomy survey course in the local community college district for several years; at first the course content - syllabus, text, demonstrations, exams, and so on - was up to me.
After some time away I came back as an adjunct at the district's all-online "campus," where I was presented with the whole gamut of pre-prepared materials, including a list of 19 "course outcomes" (one for each chapter of the text), each of which could serve as the basis for a semester-long course. For example, "Describe current cosmological models and their implications on the past and the future." Really?
This was absurd. My students weren't going to become professional astronomers. The survey course objectives needed to address what is really important: that students learn to become scientifically literate citizens.
10
In the preface of his monumental "Lectures on Physics", Richard Feynman modestly quotes Edward Gibbon's statement: "The power of instruction is seldom of much efficacy except in those happy dispositions where it is almost superfluous". Even if learning assessments were done right, it is by no means obvious that they would be telling much about the teachers.
11
I can't defend the "learning outcomes assessment industry" but I believe can defend "learning outcomes assessment" as a practice. It's possible it's not being carried out well at certain schools but, when approached for what it is, it can help provide some basic information to a simple question: what are students learning after a given learning experience (a learning experience can be the completion of a course, program or a degree). It typically gets more complicated the larger the scope, i.e., trying to assess the total knowledge and skills attained by a student after completion of a BA in History can be more challenging than assessing what they learned after a single History course); but the alternative is to not even attempt to understand what students are learning after a given learning experience. I know it takes extra time and it's difficult but I still support assessment efforts that try to measure what students are learning. The cottage industry that has sprung up around such efforts does not have my support. I believe that colleges have all of the homegrown brain-power they need to produce useful learning outcomes assessments.
7
Molly does a good job at highlighting the symptoms that arise from a numbers driven system, but could it be possible that we're testing the wrong things and getting the wrong numbers?
The teacher who changed my life the most was an Art History professor who's final exams focused on how I might explain an Art historical perspective to someone who hadn't taken the class. It wasn't just a call and response but rather made use the information that I gathered in the class and speak both critically and winsomely on the subject. End of the class I got a grade, but it was a total change-up on how I dealt with classroom knowledge.
11
I'm 73 years old and have had a fairly successful work life. If I had had to pass all the standardized tests that students now must take, I'd still be in eighth grade.
19
Constant standardized testing mostly hammers home what we already know, those students who test well have good test-taking skills, whether they be in elementary school or in higher education.
A percentage of the most galvanized learners who work hard, make gains, stay curious, are good citizens of our academic community and exhibit tremendous potential for success in life do NOT test well due to the deliberately confusing way test questions are constructed or organized on the page.
The emphasis on standardized test scores as the main way to measure academic gain or need is simplistic and narrow-minded.
10
It became obvious to me within a couple years of my undergraduate work 50 years ago that grades were a combination of incentive for the student, and sorting mechanism for wherever they were going next: High school grades were designed to help colleges select students; college grades were designed to help employers choose employees. I've earned 3 masters degrees since then -- the most recent in 2016. I prefer to take courses pass-fail, because I learn more, and because getting an A is not the reward it was when I was 19.
I have taught high school English, English Comp in community college and journalism in university night school. Writing is a skill -- a thinking skill. You learn to write better by reading well written literature or journalism and by writing and rewriting. I didn't mind correcting papers, or editing stories, but I hated having to distinguish between an A paper and a B paper. I cannot imagine having to "assess" learning objectives using some standardized test. I considered my work successful if my high school or community college students were able to write good papers for their subsequent classes and if my journalism students were able to a) publish their work and/or b) get a writing job.
10
The great irony of "learning outcomes" is that students are learning less, but there's much more paperwork to pretend that they're learning more, or for that matter something.
30
I view the articulation and assessment of learning outcomes as a moral imperative for higher education. Students, their families, donors and taxpayers make major investments in higher education, expecting that students will learn important things and at an appropriate level of rigor. They deserve evidence that students do indeed learn those things. It's incumbent upon us in higher education to define those important things responsibly, considering not only what we want to teach but what our students most need to learn and what society and employers most need from our graduates.
But if Dr. Worthen describes her experiences accurately, I do sympathize with her to a certain degree. Being told what to do, without opportunity for input, certainly brings out my passive aggressive side! But I think we in higher ed are on the cusp of a transformation, from assessing learning outcomes just to get it done and satisfy an accreditor, to doing truly meaningful, worthwhile assessment that helps us give students a truly great education. That's what learning outcomes should be all about.
But much of what Dr. Worthen's comments are based on anecdotes, not systematic evidence on the state of learning outcomes assessment in American higher education. Simply contacting the National Institute for Learning Outcomes Assessment would have given her a better informed understanding, and I encourage her to do so.
14
Looks like she had - simply reading the piece...
"Pat Hutchings, a senior scholar at the National Institute for Learning Outcomes Assessment (and former English professor), told me that 'good assessment begins with real, genuine questions that educators have about their students, and right now for many educators those are questions about equity. We’re doing pretty well with 18- to 22-year-olds from upper-middle-class families, but what about — well, fill in the blank.'”
It sounds to me like she talked to Pat but didn't explore other NILOA resources. Their surveys, for example, have found that locally-developed assessment tools such as rubrics are far more prevalent--and useful--than the surveys and published tests she mentions. NILOA's website has good examples of assessment work that's been worthwhile in understanding and improving student learning.
I don't understand why some think colleges should teach mostly "market-ready" skills. That what vocational technical schools do. Colleges are for more than "market-ready" skills. The broad courses they teach help people fit into the world, make them able to associate with and be able to speak with people who are interested in many things, whether it's theater, literature, visual arts, engineering, architecture, medicine, anything. A liberal arts education helps us learn to fit into the world, no matter what part of the world we're in.
25
I spent 15 years as a adjunct at a community college. Generally speaking the majority of the class had multiple demands of work and family. Others where struggling academically, some in recovery and trying to find a new way in life, others coping with mental health concerns.
In a class of 25, by mid-term there were a core of 15 students in regular attendance, the rest intermittent or dropped out.
Whatever their circumstance in almost every case rest assured someone said you need to get some college education if you want a better job and improve yourself. And, I would remind them to continue their education if they wanted prosperity.
Our college uses placement tests for English and math. Those that scored low were encouraged to take a noncredit refresher course. Unfortunately, it was determined it wasn't in the student's interest to be directed to a noncredit course, so the process was ended.
Getting students to stay engaged in the class, read the assigned reading at least once seemed to be real challenges. The classroom energy dissipates over the semester especially as the holidays approach or summer break. The weight and grind of life and education takes it toll on the students.
I don't know if I improved their critical thinking, or skill sets. I think that only comes with accumulated experiences and time. And isn't that the essence of critical thinking. I have to hope their experiences with my teaching were relevant and helpful in their life.
12
With Mr. Piscopiello as a caring exception, the motto too often is a variation off an old saying from the Soviet Union: They pretend to learn, and we pretend they're learning. Look, we can show it to five decimal places ...
I really enjoyed this essay, but it's vital to turn our focus to the fact that an industry built upon the symbiotic relationship between accrediting agencies and the educational technology industry has become the behemoth that drives this ship. College administrators are trained to dance only to the tune this multi-billion dollar industry is playing, and frankly face real consequences for failing to keep dancing. While Dr. Worthen is correct to label this phenomenon as part-and-parcel of the Neoliberal turn in Western culture, nothing will blunt this distorted assessment trend without challenging accrediting agencies themselves who--like academic administrators themselves--justify their existence by the number of human bodies dedicated to "institutional research," campus Assessment offices created, and reams of paper produced.
11
I've been a college professor for twenty years. I've attended the regional accreditation training conferences where we are taught the importance of assessment and how to do it. Everyone agrees that assessment is part of a continuous improvement cycle. However, when I ask why we are not taught how to plan and implement the changes called for in the assessment, I'm told it is important, but not covered. As I see it, assessment is only half of the equation. Acting on the information to make change is the other half. Assessment that leads to actions and changes is what I can accept. Assessment for assessment's sake is missing the point.
16
I chaired one of my college's assessment committees charged with putting an assessment program in place and evaluating its performance. I learned the literature and got the training.
The evidence overwhelmingly indicates that the assessment regime exists solely for the purpose of satisfying bureaucratic needs, and has nothing to do with student learning.
The impetus behind it comes from the federal Department of Education via the regional accrediting agencies, which threaten to withhold reaccreditation unless the institution gets with the program. Culturally speaking, assessment reflects a misguided belief that learning is as quantifiable as the number of cycles a microwave oven will run before it dies.
I once asked my team to fan out among the sessions at a major national assessment conference and count any time that the assessments being described had improved student learning. The result was zero.
I soon learned that national assessment awards are given for creating assessments, not for improving student learning.
The critical scholarship is also clear. Assessment exists purely to simplify the job of bureaucrats in policing those beneath them. Accrediting agencies get simple markers for which institutions should be reaccredited. They get to say they have measured us.
We faculty are angry over the time and dollars wasted. But history tells us that it is going to take time before this regime collapses of its own inherent contradictions.
214
It's difficult to say this because I do believe in the ability of good government to provide leadership, but the Department of Education has played a not insignificant role in the managerial takeover of the universities, the demise of faculty governance, and the rising cost of higher education. The universities themselves have paid a high cost to stay on the good side of the Department of Education, an institution that allows people who don't know much to rule over people who know a lot.
4
The goal of higher education is to learn to think critically, to expand one's understanding of self and others, and to be introduced to the great thinking of history and the humanities. This is a process of becoming that has nothing to do with being assessed, because the end goal is person-hood - which is subjective. Assessment is appropriate in cases where a person is studying to learn specific job skills, and it's measured in the form of grades and whether or not the person ultimately qualifies for the job they're studying for. Anything beyond that stems from colleges and universities seeking to justify over-inflated cost of tuition.
21
I was a department chair at a large state university when the state legislature demanded us to prove that graduates were learning what they were supposed to know. None of the obvious numbers at hand would satisfy their need for "outcomes assessment." First, the university sampled graduating seniors to take a Saturday exam delivered by one of those testing services. Of course the seniors had no motivation to do well, and many made pleasing designs on the bubble sheets. So that was a fail.
Second, the university punted the problem to the departments. As Josef Martin wrote in his wonderful "To Rise Above Principle" (now back in print), "If a thing is not worth doing, then it is not worth doing well." I knew we needed to be anonymous in the middle of the pack in response, so we asked a PhD candidate in industrial/organizational psychology to devise a five-minute questionnaire for our graduating seniors; they had to come in for a final signature, so we could survey all of them. We also tried to track down a sample of graduates after a few years to ask their employers to rate them. Minimal effort, following Martin's dictum.
But the other department chairs let us down. Some managed to do less or even nothing at all, and our efforts were lauded by the administration as exemplary.
Sadly, this led to all sorts of things to divert our attention from what mattered, thus proving that "no good deed goes unpunished."
6
As a Ph.D. graduate who's taken countless standardized tests, and as a father whose children have done likewise, I am convinced that the standardized testing industry -- which sounds very much like the learning assessment industry, and in some cases is indeed sold by the same companies -- is a racket.
Witness, for example, the fact that the same companies who own and sell standardized tests also own and sell test-prep material -- material that skews test results in favor of the affluent who can afford test-prep material.
As for critical thinking: everyone claims to value it as a skill, but how many students are required to take an actual critical thinking course (I mean a good old Logic course)?
No, that would make too much sense. Instead, we expect students to pick up critical thinking filtered through some other discipline like science or math.
Critical thinking, aka Logic, is a branch of Philosophy and for thousands of years (that is, since the time of the ancient Greeks) it has been an integral part of a liberal education. And Philosophy, in turn, is rooted squarely in the Humanities: let's help to revive the perceived value of the Humanities, and of Philosophy, by pushing administrators to require students to directly study critical thinking as a stand-alone course in its own right.
68
Measuring learning is not an easy enterprise.
It's worth remembering the story of the economist (or was it sociologist?) who lost a valuable piece of jewelry in a large dark field with a single streetlight. After the social scientist had spent nearly an hour looking under the light, his fellow historian asked, "why do you keep looking in one small part of the field where you lost it?". He answered her that "that's where the light is".
What we can 'measure' about learning is in most cases either "under the light" and stunningly obvious, or else it is dubious at best: toting up numbers that claim to represent fuzzy and poorly defined qualities is a recipe for false precision and bogus results, and that's what a good portion of the "Learning Outcomes Assessment" I've encountered looks like.
16
When the companies that create assessments are held accountable to nobody except their investors who only expect big profits, how does any good come from it? The accountability movement, accountable to no one!
26
I think about my own college education in 1970's, and the lessons and experiences that made lasting impressions on me, the nature of those impressions, the direction and magnitude of their effects and how such experiences might be measured. I'm at a loss. How could one assess these experiences that opened my mind, filled me with awe and wonder, or that demonstrated the pricelessness of well-timed irreverent humor. How could you assess my learning to appreciate of the beauty of things known or my genuine sense of humility in the face of vastness of things unknown. College was more of an experience of coming of age and learning to view the world. Higher education put me in the company of adult men and women who were inspired by ideas and passionate about truth, clarity and honesty and I learned from them that those were admirable passions. My college education started me on the path of learning to learn and it encouraged me to press on. It was where began to achieve my adult sense of selfhood. I doubt much of that can be handily assessed.
20
The purpose of education is to teach people how to think originally, creatively and independently. Knowing how to get an answer begins with being curious which is more critical than the answer. Being able to say "I don't know" and learning from finding a "wrong" answer were my two most meaningful lasting educational "learning outcomes". Curiosity is everything on the road to gaining more knowledge.
18
As a professor at a small university with a significant population of first generation students, I find this analysis a little cynical. Assessment is stressed by our administrators because of its importance to reaccreditation, but we don't spend inordinate sums of money on it. And while I'll admit that not all of colleagues feel as I do, I personally enjoy the assessment process because it gives me the opportunity to reflect on my teaching. It doesn't have to take up a lot of my time because it is built on in to the process of planning and executing my courses. My courses aren't perfect. My assessment reflect that, and it helps me do better the next time. That's the point: continuous improvement.
15
Yes! It is ultimately about continuously improving, whether it be student learning or operational effectiveness.
I work in a massive, urban CC system. We have about 4-6 admin people at HQ who supervise this for a living at an expense of probably, $60-100K each. Dept chairs and vocational program coordinators produce semesterly reports that required anywhere from 40-100 hours of data collection and report drafting. The faculty shared governance component of assessment procedures is about 30 or so FT faculty who meet monthly to discuss assessment practices. That's 60hrs of work (30 people x2hrs) per month from the instructional leadership ranks who are all being paid probably $20-30/hr. At the high end these meetings cost about $1500/month.
I'd wager my institution spends more than $2mn on this, and if we don't there will be a process with no faculty input, or there will be major depts and programs who are absent from the data, and our accreditors will likely scold us, or worse, for not having data. And data needs to demonstrate "continuous improvement."
Millions of dollars wasted on this at my system alone.
4
A good teacher should be clear about what they are teaching and what they expect to learn. If they have not thought a lot about how they think the information can be used and communicated, they do a disservice to their students. Take history - the simple idea of if you don’t learn the lessons of history you are bound to repeat them. One is with new generations, more and more of the knowledge and wisdom of past generations is lost. So if you teach about WWII, Korea and Afghanistan, you need to present history differently to different age groups.
If a teacher doesn’t account for this and the fact that historical understanding changes over time, students carry around false information thaterers harms them. Stating desired outcomes and allowing them to be challenged is part of the learn process. Thinking of yourself as a guide to knowledge rather passing facts shows the teacher understands their role.
2
About 14 years of my career were taken up with delivering or developing classroom and on-the-job training for approximately 1 million hourly and salaried employees. The other 11 years of my career were spent delivering training as one part of my job as I operated restaurants, or 'consulted' with those who did. I came from an academic background, English and American Lit. Here are three things I learned:
1.) A college degree in humanities enabled me to think critically about the 'whys' of my jobs and the 'hows' of getting things done (the 'whats'). Those degrees, and the college professors as well as my high school teachers, enabled me to communicate why, what, and how.
2.) 'Learning assessments' are useful only in areas of specific 'whats', such as cooking a burger, developing coding, calculating the carrying capacity of a bridge, etc. Important stuff, with the common characteristic of being measurable.
3.) 'Learning assessments' are close to useless in measuring the ability to think critically and creatively, and communicate clearly the results of that thinking. 'Learning assessment' does not tell us about the vitally important aspect of learning that Worthen touches upon: preparation that enables the individual to not just learn, but to apply skills - that may be entirely cerebral.
As Worthen mentions, that preparation starts before college; indeed, it starts in earliest childhood with reading and discussion in the home.
35
Universities are now facing the same pressures that K-12 have been dealing with for quite a while. The emphasis on assessment is clearly changing the what we mean by education, because despite what companies say to sell the product, they cannot really assess understanding and complex thought, and they definitely cannot assess the relationship between teacher and student. We end up teaching to the test, in the process losing opportunities for real learning to take place. Sometime, hopefully soon, we will realize that this was a mistake--or maybe by then all citizens will have forgotten how to analyze a complex problem and will never know the difference.
20
I teach at one of those comprehensive state universities, we have a ton of students who are first in their families in college, and most of our students work. Assessment is a major problem, it's making it harder and harder to really teach. It's worse in my field (math) than in Prof. Worthen's, but it's bad for everyone. Prof. Worthen is right though that it's much worse at a school like mine than at a high end school (and she teaches at UNC which is high end). My so goes to the University of Rochester, very high end, and I don't think he's had one professor mention a thing about assessment yet. Assessment does nothing for anyone, except for those who sell the assessment products, and the bureaucrats wh need data points to justify their jobs.
23
This article would be a lot more useful if it made constructive suggestions for how we *should* assess schools and teachers. There are lots of examples of bad assessments here, lots of arguments in favor of the value of other things that aren’t valued by existing assessments, but absolutely nothing at all about what we *should* do instead. If the argument is simply to not assess at all, to make no effort to distinguish between better and worse, that’s not going to fly as long as the taxpayers are footing the bill.
2
And why exactly isn't the professors evaluation of their students work a valid assessment? And if it isn't, that's for professors to fix amongst themselves. Grade inflation is a bad thing, but it's a recent thing, and there's no reason it can't be made to disappear.
9
I've been teaching at the university level for about 20 years. I assess learning all the time. I give assignments, of all kinds; I make my students write a lot; I make them explain things to their fellow students; I make them do larger projects that include digital media and research on primary documents. I know pretty much at every turn in the semester how the learning thing is going, at least in the immediate timeframe. that's my job. And then, of course, there is the very strange thing about learning and teaching: it can operate on a kind of delayed action. How many times have any of us, as students, wondered why on earth we are studying this or that, or absorb some skill or information rather idly, only to have it fit into an entirely different context, in terms of our own maturity, experience or new practices and environments years later, and suddenly realize how much indeed we learned years ago..So to answer your question, Mr. Desjardins, I think those of us in tilling the fields of higher ed have lots of ways we assess learning. They just aren't particularly comprehensible to testing agencies and administrators or they can't be boiled into stats and norms.
11
There is a lot of complaining but the professor does not offer an alternative solution. The problem is that universities do not want to be transparent and accountable.
The other day I came across a course at a major university with tuition well over $30,000 per year. It had one professor and 20 teaching assistants for about 500 students. The school is charging over $1 million for the course.
2
Wow, that's a pretty luxurious university: 20 TAs for 500 students, or only 25 students pre TA? Where I teach, budgetary pressure means that each TA supports at least 69 students (3 sections of 23), so for over 500, I partner with only 7 TAs.
Sarcasm aside: I believe that trying to account for courses by dividing student tuition and multiplying instructional costs is very bad accounting. Universities really do provide more than classrooms and teachers, you know. And no university -- well, let me correct myself, no serious non-profit university -- would ever allow a student to graduate with a list of courses with 500 students. I teach courses of 500, but I also teach undergraduate courses limited to 22, which are required for majors. There's a mix, with different resources applied to each. That "$1 million for the course", if one accepts that calculation, is paying for labs, seminars, field trips, one-on-one tutoring, and a bunch of other academic activities, as well as counselors, advisors, specialists in the byzantine Federal and state bureaucratic requirements universities must follow, and much much more.
9
The only thing worse than no data, is false (or misleading) data. You don’t need to have a solution to know something is wrong.
1
It is important for people to look closely at real problems at least once in their lives. Through this, they learn that real problems are complex, that easy solutions may have unintended consequences, and that going along with the crowd is not necessarily a path to truth. If citizens can develop some depth of understanding, we can begin to address critical issues for the survival of civilization, such as climate change and nuclear war. Well educated people are typically hard to manipulate, in that they require that arguments have contact with reality, and should have learned not to fall in love with their own ideas.
5
At the northeast US university where I teach, the assessment issue becomes problematic when combined with our Incentive Based Budgeting Model. The IBB causes different colleges and programs in the university to compete against each other for students and enrollment. They compete by promoting themselves with jargon and "data" from L.O. Assessment initiatives. While the desired learning outcomes are good to consider from teaching and learning perspectives, they do not mean the same thing in different contexts, areas of study, or practices. "Critical thinking" in the humanities can mean something quite different that "critical thinking" in the health sciences - even within health sciences, from class to class, such "skills" might mean something entirely different. The presentation of comparitive data therefore has a sanitizing effect, and the terms of learning outcomes successes are vague at best and meaningless slogans at worst. Join our program because you will be a better critical thinker? When programs then use this data to prove student success and compete for resources, the game is stacked in favor of those programs that lead most clearly to employment along a linear path. Programs that can prove a marketable product will win every time. There's no room here for John Dewey's "ability to maintain doubt and carry on a protracted inquiry." Imagine if we could admit, cultivate, and sustain something as human as doubt in our lifetime endeavors. What might that look like?
11
I'm a nurse in a special education preschool. Therapists (OT, PT, ST, Social Workers) and teachers are required to capture extensive amounts of data, sometimes for children who literally were born with half a brain. Report card submission is required 5 times a year, for 18 month--5 year olds with a huge range of abilities. Between data-collection and writing extensive notes on any therapy or teaching "sessions" with children to satisfy Medicaid requirements (even if the child doesn't have Medicaid: HIPAA forbids us knowing, so we write
them for all kids) there is barely time to actually teach, and have a functioning classroom. I've heard for years that curriculum required in NYS public schools is despised by teachers, with good reason. It's all about "teaching to the test".
If Trump has his way, we'll soon be required to carry firearms as well.
I'm relieved to be close to ending my career rather than just starting out.
11
worthen is ignorant about learning outcomes and assessment.
-no outcomes so no guide, contract, transparency, or accountability. we're paying for it. you want to create you own assessment world that nobody but you understands or recognizes, do it on your dime, not mine.
not articulating learning outcomes in terms of the evidence required for achievement is bad practice
1
Insults are easy, Mr Westerhosen, but your comment does not suggest that you recognize any of the many difficulties in measuring something as multi-dimensional and contentious as a 'learning outcome'. Has it not occurred to you that what you consider to be "evidence", and to be "achievement", may be vehemently rejected by someone else with just as much license and authority to speak about students and their futures as you might have? Before we can "articulate learning outcomes in terms of the evidence required for achievement," wouldn't we need some consensus on what "evidence" and "achievement" would even look like? I assure you, no such consensus exists among those lucky and privileged enough to be teaching college students today (and I feel very lucky and privileged to be doing so).
8
I don't think you understand what you're paying for here.
My university just created a Vice President of Assessment. That administrator is paid well over $100K/yr, and has a fleet of staff to help him with Assessment. They spend their time collecting and collating assessment data from the various schools (Humanities, Engineering, Fine Arts, etc). Each of these schools have at least one Dean whose job it is to collect assessment data from each of the academic programs. Each of those programs has an Assessment Coordinator responsible for creating, distributing, and collecting Assessments of Student Learning from the faculty. The departmental Assessment Coordinators prepare a condensed departmental report and forwards it to the Deans, who condense the condensed reports and send it to VP of Assessment, who produces a glossy brochure announcing that all of our students have achieved a vague set of student learning outcomes, such as "demonstrated quantitative reasoning ability."
That's what you're paying for. That's the kind of system Dr. Worthen is protesting here. Hardly the model of transparency and accountability that you would like to see.
4
consensus occurs when the student understands the outcomes and takes the course. the evidence the teacher requires is produced by the test and must be aligned with the outcomes. if teachers do not articulate the outcomes, there can be no consensus between any stakeholders. there will always be people who disagree with the content of standardized (ie consensus based) tests, but there is NO consensus in Worthens approach because it is entirely personal.
Ms. Worthen's essay is a sad parody of liberal arts taboos. Taboo #1: never test, lest you discover that "disadvantaged" kids actually learned less than the evil patriarchy of kids who study. Taboo #2: never admit that students are customers who fork over $200,000 or more to learn facts that will increase their future earnings. Taboo #3: never admit that private ownership (i.e., capitalism) has been highly effective at delivering wonderful things to almost everyone, whereas Marxism and Socialism have impoverished millions.
Ms. Worthen claims to be interested in critical thinking and questioning assumptions, but her essay reveals a religious faith in academic orthodoxies and debunked economic theories that are at odds with reality.
4
Corporate interests have turned our country into a nationwide plantation system. We're not literally enslaved, but we are living with the 21st century version of it.
18
What does “It’s a bit like the old Soviet Union. You speak two languages,” mean??? The Soviets produced Sputnik AND the first astronaut when the US was claiming that the Russians were "stealing" their knowledge from US scientists. Capitalism does NOT want a highly educated, inquisitive general population. Such people might just be able to see the profit motive for what it really is: A way to ensure that a small minority of wealthy people take advantage of the large majority.
2
I was an alternative student in the late '80s, which is to say: an older adult returning to college. My marketable skills were already high but hard if not impossible to validate -- and I needed to change careers. Since I was majoring in engineering, much of my learning had an objective basis. Yet, I found the non-engineering classes the most interesting. There, I truly excelled. Why? Frankly, I had already learned to think analytically on a much broader level than my younger classmates.
I graduated in three years magna cum laude. I'm not sure this would have been possible if I had been fed years of Pablum masquerading as measurable learning. I was from an earlier generation that had been educated differently, and, I would argue, better. I went on to work in my primary field of study, but always noticed the shallowness of my younger colleagues. They were bright and hard working, but lacked depth beyond their work-related skill set. They solved problems well, but had a near total blindness to broader implications of the work they performed.
Consider me an unintended experiment, and consider the implications. College needs to educate not train, but that seems to be the norm today.
28
as an Arts Degree holder I have read that Arts Degrees make the best managers - people who can think across the broad spectrum of social factors that contribute to success or failure in business.
At uni the joke observation was that engineering students could not hold a conversation.
Now retired decades later I'm a local activist happily contributing to a better community life for all around me, while those who worked as engineers can't seem to quite figure out why those bad people did what they did.
2
Oh, come on, we all know education is a waste of time. You can do just about anything without one. Look who's President! And all those college graduates couldn't do a thing about it.
4
Years ago when the push for SLOs began, it was pretty clear that the march toward standardized tests for colleges had begun. And who was behind it? Textbook publishers. They had already begun creating online coursework to accompany their over-priced textbooks, saw the exponential potential profit, and started emailing requests for teachers to send them their 'best practices' so they could quickly learn how to teach without ever actually teaching. Then began the "why don't you want to do what's best for your students and require our ($$$) online portion of the textbook?" guilt tripping. Now they want to use SLO statistics to plan all-online universities, and this has already happened in CA thanks to Gov Brown whom they have seduced and who has declared that an online college is where all the budget $$ will be going now. What a disaster. Teaching is an art, not a training for corporate America. This would never have happened if administrators had not decided years ago that higher education was a business not a moral imperative, a profitable industry not a human right. Cookie cutter citizens ahead.
16
Several valid points in this piece go right out the window with this (almost) last line: "That's how we produce the critical thinkers American employers want to hire." In fact, "critical thinkers" are defined in different ways by different people, and crucially, by different institutions—which is precisely why this professor ends up "using the language of the capitalist marketplace to speak to [her] students as customers" at the university. This result then tells us the definition of "critical thinkers" that is used by these American employers she has in mind here: People who keep their thinking within the confines of the so-called capitalist marketplace (as this "capitalist marketplace" is defined by these same employers, of course).
What the American "capitalist marketplace" wants (and truly needs) is obedient thinkers, which means people who've been duly certified by the university as "critical thinkers." These "critical thinkers" are then safe to hire, as they have thereby demonstrated a trained inability to think critically about—and thereby act critically on—the tyrannical structure that is employing them.
Indeed, critical thinkers have always been the scourge of employers in this so-called capitalist marketplace, and these thinkers are certainly not sought after by corporations anywhere. And my guess is that if this university professor instead told her students that, she wouldn't be the "critical thinker" here.
12
it might be that members of accrediting associations have a stake in this. I see a link there. In addition, many faculty at smaller schools as well as those schools that have a high percentage of part-timers, have found themselves forced to take on the meaningless job of "learning assessment," and staffing committees because there is no one else. If a school goes from 1,200 students to 2,800 students and the full time faculty drops from 75 to 65, while administrative and coaching position increase, the faculty have to di the work, unless, like I, they refused to pay the stupid game.
it's easy to write angry professor epistles. It's much harder to try to help the university manage through the demands the grew out of the Higher Education Acts last revisions. Rather than rant at what you perceive to be wasted energies, why not try to help the university tell the story of how well students are learning under your guidance? And then he oped drifted off into another topic at the end, still a rant but on a different topic. I urge you to try to find positive offerings instead of raging at a machine you're helping to fix.
2
If you wish to make America great again, it will require (re)prioritizing pre-school through higher education.
4
Thank you so much, Molly Worthen! Please, please, lead a movement against assessment in academia! Real teaching of the humanities is not something that can be quantified; what can be quantified is not real teaching. To quote the headmaster in The History Boys on a superb English teacher in his school, “It isn’t that he doesn’t produce results. He does. But they are unpredictable and unquantifiable and in the current educational climate that is no use. He well may be doing his job but there is no method that I know of that enables me to assess the job that he is doing.”
18
The word "job" is mentioned three times in your piece, in regard to...
1. Software companies with the audacity to try to automate you out of part of your job
2. Over-emphasis on job-ready skills - at the expense of "the real purpose" of a university education
3. Full-time jobs that your students may currently need to hold down - as part of paying for their university education
The heck with your students and their current or future jobs...
If you don't care, I care even less - I just want your job...
Actually, my bot does...
It already took my job, but needs a second one so it can afford for itself a university education...
You know, the kind of insightful stuff that separates bots - and STEM students - from real human beings...
2
One advantage to having Betsy DeVos as Secretary of Education is her general incompetence. Getting nothing done may be the better alternative.
During the George Barack Bush Obama administration Education Secretaries Margaret Spellings and Arne Duncan preached this doctrine of learning outcomes with dictatorial fury. As President Bush-Obama was a Republican-Democrat there was no permissible alternative. I live in California where a rogue assessment bureaucrat very nearly ended the community college system of the city of San Francisco. I'm glad to see the New York Times publishing a critical editorial like this one from Molly Worthen rather than the typical newspaper editorial of the past ten years assailing the evil teachers unions for fighting any and all progress in education.
6
You work at UNC. If UNC and other schools liked it refused to participate in this and refuse to participate in the ridiculous accrediting institutions it would not matter. That accrediting bodies are for schools like ITT tech.
1
Nope. It is through accreditation, like a system of checks and balances, that colleges are able to offer and receive federal financial aid for their students.
It's hard to produce well educated college graduates when the grade and high schools are turning out garbage. So many of the teachers lack the basic skills, themselves. Even when they do, they are often shackled by the administration and school board with their teach to the test mantras.
4
Unfortunately, this is a very one-sided and shallow discussion of the matter, one that can only be appreciated by those who already agree with the author.
You know some faculty members who don't like assessment? You don't say?
2
To measure the amount of juice in an orange, squeeze it into calibrated glass. Who cares If it’s sweet and worth drinking.
6
.....and if you think it's difficult to do and of this in a normal classroom setting, try doing it ONLINE, given the explosion in that mode of learning....!!
Uhhhh...what? American employers want critical thinkers? Who told you that? American employers want people who don't have to be told twice how to do a simple thing. Once, we can take. Once, we get. Once, we remember being told ourselves. Twice, it's a red flag.
American employers want people who show up and work. At the lowest level, name one American employer who looked at a cashier and said to herself, "Now, there's a real critical thinker." At the highest level, name one American employer who looked at an executive vice president and didn't say, "Now, there's a guy who wants my job."
Through some combination of second-guessing and substituting your opinion for the American employers', you're misunderstanding what they want. You're so heavily invested in what you think American employers ought to want you've got yourselves convinced you're right.
Take your "third irony," for example. Just where do you get off, how do you reconcile "resist(ing) capitalism" with being all for giving American employers the critical thinkers you believe they want? I know. It's for their own good. The American employer and her legions of workers who you require have lives of intellectual endeavors of which you approve. You know better.
Well, maybe you do. Maybe if we listened to you we'd all be better off. Maybe you'd be able to answer satisfactorily the question my mom used to ask me: if you're so smart, why ain't you rich?
1
This is abject nonsense. Educators resort to a kind of mysticism or voodoo to claim that their outcomes are immeasurable. That would be fine if taxpayers didn't subsidize the higher ed Leviathan to the tune of several hundred billions of dollars annually.
As a physician, imagine if I told patients, families, insurance companies and regulators that I just couldn't measure my outcomes, they're undefinable and you just can't capture that fairy dust of "health". I'd be laughed out of the profession.
3
And yet, I have heard or read more than once that medicine is as much art as science. And how many times have I asked my family practice doctor (who I respect very much) a question such as "how did I get pneumonia?" only to have him answer, "If I knew the answer to that question, I'd win the Nobel Prize for Medicine."
Of course, the question here is: what should be measured and how should it be measured? So many of the "assessments" being foisted on teachers at all levels are the equivalent of taking a person's blood pressure to determine whether they have pneumonia or a sinus infection. The measurement is easy and it provides some information, but it does not assess or measure what's needed.
2
Actually, as your patient I hope that your outcome IS "health" and not just a bunch of test results. I've seen plenty of problems with physicians who test and measure and follow protocols but lack understanding of what's really going on with the whole patient and how help them back to health. Metrics are useful, but they are not the be-all, end-all.
1
Hmmm.. so the "20th century model" of education gave us broad spectrum antibiotics, people on the moon, jet aircraft, personal computing.. the "21st century model" gets us Candy Crush Saga on our cell phones and something called "analyticity". (I note that the NYT spell check rejects this as a word. So it should. So should we all.) I have been a college professor for 31 years at what is called in the rankings a "Highly selective Liberal Arts College". The impact of the assessment culture foisted on the public schools by various initiatives in the last 18 years has been nothing short of devastating. The students are still super bright, but they are pig-ignorant. One of my best described her High School experience as "Etch-a-Sketch education": Draw an elaborate picture for a term, take a test, erase the picture, draw a new picture, repeat. Teachers are forced to teach to tests to keep their jobs, there is no emphasis on scholarship or cumulative learning. Two weeks ago not one of my students could tell me where the Mississippi River emptied into the sea -never mind where it started. This is how a civilization dies, with an emphasis on false metrics and imagined accomplishments.
34
“The power of instruction is seldom of much efficacy, except in those happy dispositions where it is almost superfluous.”
― Edward Gibbon
3
Sing it sister. We are 2 task forces and a committee away from "No College Left Behind".
10
Too bad the author is really out of step with the way her professional organization, the American Historical Association, has been heading. You can complain all you want, but in reality, there is a whole lot of bad teaching going on out there in American universities. Ask your local students. Good learning assessment should be faculty driven and should offer opportunities to faculty to discuss what is really achieved in the classroom, not provide a forum to rant on about what silliness your regional accreditor demands of your institution. Not all of us are in SACS.
3
The fabulist assessment industry claim course based examination scores shouldn't be used assessment metrics. So what is the instructor to do: use GIGO (garbage-in-garbage-out) just like the history professor who asked his students on unrelated topic on Sumari culture to satisfy the assessment bean counters. SAD!
2
The problem isn't learning assessmet; it's that you have not yet figured out how to do it and thereby improve teaching. Talk to the Department of History in the UW Colleges.
2
Amen. Amen and amen.
6
Beautiful.
3
Here is the problem with deciding what is the successful outcome of learning: you don't know. There a few areas in which you need to display competence (a doctor/dentist for example), but for most areas you don't know what is going to be needed in the future, what is going to work, what will be successful and that is because you can't know what is going to be needed. What we need is people who are educated, flexible, and know how to learn. How else can we deal with a rapidly changing world?
4
If you want to improve universities, set the bar higher for entry. If you don’t have the grades or ACT scores, go to community college first.
1
Thanks so much for this article. I can see some usefulness to "assessment" as a practice -- it can help teachers become more aware of their own techniques. However, when my department stumbled through the process of coming up with a rubric years ago, I realized that none of our home-made tools would be deemed adequate by the Inquisitors. The whole thing was designed for failure . . . so some assessment company could come along and deliver us the instruments of torture. And that's pretty much what happened. I knew it would because the public schools already had to use those instruments. Some day, we will look back at those instruments when we're trying to figure out why Johnny couldn't read.
4
This is just a really great article, one that we really need in this battle against the banality of assessment. Of course, professors care about making sure students learn, deeply and profoundly. But let me give you one glaring example of the kind of asinine "outcome" the assessment industry measures: at my university one measurable outcome for students in an intro to psych class is that "100% of students will agree that understanding psychology was an essential part of the course." Duh.
And even though our undergraduate student body is not even 2000 students, we still have four (maybe five, one might be "acting") "associate vice provosts" running around with laptops and spreadsheets, and constantly being flown to yet another assessment conference that charges thousands of dollars in registration fees.
That's where the tuition money is going, folks.
22
Spot on, Fester. Our earliest versions of SLOs reflected ambitious goals that reflected what we actually taught and how we assessed student learning. We soon learned the folly as the next step foisted upon us was testing those SLOs specifically, quantifying the results, and being measured as an institution accordingly. Unless we were willing to spend many uncompensated hours on this stupidity, the SLOs needed to be considerably dumbed down to assessment size, and quickly. Mission accomplished.
If we created a hierarchy of needs for most university students a la Maslow, the lowest level would be to gain the credential which is required (often for little reason) for most jobs that allow the job-holder to enter the middle class. Next would be marketable skills that will make them attractive to an employer. Third would be the mastery of a discipline, and fourth might be to develop a state of consciousness that allows for greater insight into humanity or some branch of science.
.
Some of any educator's students are going to reach each of those levels. Very few are ever going to have classrooms where every student reaches the third or fourth level, even at Harvard or Stanford, because every classroom has students who are there despite their lack of interest in the subject, but they need it for their degree. If you can't handle the reality of that situation, you shouldn't be teaching in college. I teach college, and it is inspiring when students reaching the third and fourth level come to talk to you. But most students just take notes, pass the tests, and move on. You can't let that get you down. Education is many different things to many different people. All that an educator can do is offer the opportunity to learn -- you can't force a student to learn, and you're going to give him or her a C if they master the basics. Professors who fantasize that they are gurus offering a higher level of consciousness to every student need to get out of their ivory towers.
5
These faculty who reject learning outcomes assessment would protest loudly if a scholar in their discipline made bold claims with no other evidence than "Trust me! I know what I'm doing."
Yet these same faculty want the public to accept the claim that students are learning in their classrooms with no evidence whatsoever. They want to say "Trust me! My students are learning." with no evidence to support it. It's an accountability dodge.
1
Faculty in the humanities expect scholars in their fields to write persuasive books and articles, with clear arguments and evidence that is supporting their points. They expect their students to write persuasive papers, with clear arguments and evidence that is supporting their points. They don't expect either their peers or their students to advance interpretations of poetry, philosophical concepts, or historical events with quantifiable data.
9
Who said anything about quantitative data? Your reply is steeped in the mistaken notion that only quantitative data will satisfy demands for learning assessment. But qualitative measures are perfectly valid forms of assessment when done with some forethought.
What's funny about your reply is that you sketched the outline of a basic qualitative assessment: persuasive paper; clear arguments; relevant and convincing supportive evidence. Congratulations! You've got yourself a rubric!
Once what was learned in American colleges College was primarily the result of intense reading, attendance and attentiveness at related lectures, student/teacher discussion, and exploring the subject at hand through individual research and the writing of reflective papers. In other words, students, under the guidance of scholars, were responsible for what they did, or didn't accomplish.
Due to the open admissions policy of so many schools today, and the resultant "nobody flunks out as long as they pay their bills on time" attitude, it's no wonder that many students don't learn much. Besides being unfair to professors by holding them accountable for the learning of students who put forth a mediocre effort at best, attempting to "quantify" the unquantifiable is an exercise in futility. The concept belongs in the dumpster along with the "standards-based, data-driven" edu-babble, which has reduced what once was a complex, intellectual endeavor to the banality of a checklist of "skills" on a rubric, most of which are so vaguely constructed as to be meaningless.
5
Seems to me that a necessary condition to solve, or better to ameliorate, the problems described here is to [somehow?] improve [a lot!] the effectiveness of high schools [especially those with few or no students from elite families]. Prof Worthen says it: the difficulties non-elite colleges face are the results of high school graduates being unprepared for college-level work. The pressure for useless but costly "assessment" at the college level would, I think, disappear of the high school problem could be significantly ameliorated.
A professor of mine, 60 yrs ago in an engineering course, said "there is no such thing as teaching; there is only learning." This was a bit of an oversimplification, but: Critical thinking can only be learned by doing it (e.g. discuss the headlines in a serious newspaper or the news on last night's Fox TV); problem solving can be learned only by doing it (put the students in small groups and give them a few hard problems with one week to solve them); writing skills can only be learned by making each student write and then critiquing their work.
I see no reason whatsoever why every high school in the country can't do this. Of course, outcomes will vary. But the first thing to do is to try it.
1
Ahh, the tech goal of quantifying everything. I feel sorry for young people.
7
After 40+ years of teaching in a well-known private university, I agree with Dr. Worthen: attempted measurement of "Learning Outcomes" is an earnings-driven ploy of pseudo-educators and educatasters, seconded by the politically correct chorus of leftist radical Democrats and their likes. Only the accomplishments in the life of a person educated in a university can attest to the value of education.
4
Based on my return to college a few years back after decades away, I'm not sure what could have been assessed. It was one big political indoctrination factory. Students were reading as college assignments, novels I had been assigned in junior high, but were analyzing them only for how they revealed western culture to be oppressive, racist, and misogynistic. The entire experience was like Kafka mixed with Orwell (thanks to the speech code), but without the charisma of Rod Serling. Derrida was a visiting prof. - and I have to say post modernism (which seemed somehow to be the driving force behind the entire enterprise) is the most irrational ridiculous 'philosophy' one cold ever hope to dredge up from the bottom of a barrel. There was, though, one stoic aging professor of Shakespeare - amidst this landscape that might have been painted by Bosch or Dali on acid - one small island of sanity and that old man delivered the best, most enjoyable and enlightening class I have ever taken in my life. One class amidst the chaos. We happy few!
3
Relax, YaddaYaddaYadda. Your professors clearly did teach you, and you clearly did learn.
Is it so hard to teach students to think critically and to actually teach them something? Colleges and Universities are always insisting they are teaching kids to "think" with no real evidence that they actually are, and no way of proving that they are teaching them anything else.. Employers have no way of comparing what graduates of different school actually know. A degree in, for instance, English literature, can mean very different things from different institutions. As a result, employers favor kids from prestigious institutions over kids from lesser schools because there has been selection of students through higher admission standards. They also look for degrees in subjects like business and engineering where they can have a rough idea of what students have actually learned.
I would suggest that we adopt a national series of comprehensive exams for college graduation as they do in many countries in Europe. This would insure that all graduates with a French degree, for instance, have at least minimal knowledge and skills that an employer can rely upon. We already do this in accounting (also law and medicine, which are graduate degrees). This is why a CPA from a city college can compete with an Ivy League grad, while this is untrue in almost any other discipline. An employer can know that a graduate from a rural state school has acquired the same knowledge as Harvard grad. It levels the playing field and holds schools to account.
1
What if the student's goal is not to become "thoughtful" and "talented" (many students are both before they set foot in college), but to gain "market-ready skills" that will ensure her or his ability to pay off the debts accumulated to pay the money-poohing-poohing professors?
The critical thinking element in appreciating Milton is a wonderful thing, but unless the same intellectual effort applies to a task someone will pay to have done, it's a wonderful thing for the idle rich.
Do most students go into debt to revel in "the intellectual complexity that makes a university education worthwhile"? Most university graduates I know don't seem to be much into intellectual complexity (as demonstrated by the fact that many of them voted for Trump).
51
You can clearly do both--that is, seek to develop thoughtful students with market ready skills. I think it's pretty clear that a dynamic, ever-changing market calls for "market-ready" skills that are equally dynamic and ever-changing. This is a call for teaching the ever-so-market-ready skills of critical thinking, thoughtfulness, creativity, collaboration, curiosity, and so on.
5
This email partakes of the same mistake that so many critics of college education, and unfortunately, parents and students, continually make. Quite simply, the argument that a college education should enable you to get a job overlooks the more important factor: once you get a job, you have to KEEP the job, and MOVE FORWARD in the job. In any job on the level that a college graduate would hope to get, immeasurable qualities such as critical thinking and effective communication, sensitivity to diversity and audience, are necessary in order to work with employers, colleagues, clients or customers, and other stakeholders. I am an English professor, and when I teach a sophomore poetry class, I tell the students that at some point, you will be WRITING FOR YOUR LIFE. The deeper skills we teach our students may not come fully into play until years after they graduate. And the ones who learned best will, eventually, be the leaders in business who in turn lament the poor communication and critical thinking skills of college graduates.
10
What market ready skills? Nothing that a student does in a corporation will be taught at a university, except for the few simple skills (coding C, writing a paragraph, making a business plan) that will get the student in the corporate door. And they are simple. Granted, the students see those basic skills, and getting in the door is a big deal. But corporations dont realy need that - they have plenty of C or Python coders. They need people who can reinvent the entire computer experience. How do they get them? They hire people who are 5 years out of school who can point to real know-how they got, mainly outside of the university as it stands today.
3
Assessment, as I have worked on it as a faculty member and administrator, is supposed to help a program determine how well they are doing on their own aspirations for their students, beyond getting grades and collecting credits. In the arts, almost all programs have senior recitals or portfolio reviews. Multiple faculty view them to record and share their judgments on where the department is doing well or not so well, based on what their students can demonstrate. They use that information to make their own decisions about what to do more, less, or differently.
3
I chased the "targets" and "standards" in my classroom for about twenty years... the problem was those the targets kept moving and once we mastered the old standards new standards replaced them. I gave up and started figuring out how to build on the two really important skills my students needed: a love of reading and the ability to write something interesting.
Those two skills don't show up on standardized test, but are the most important skills to master as a human.
16
Don't forget to include us community colleges in this discussion. I'm a member of an "outcomes task force" that was described as "accreditation driven but faculty led." It likely apocryphal at best, but Einstein purportedly said, "Not everything that counts can be counted." Welcome to neoliberalism in higher education.
10
Is the author's thesis that we should not a attempt to measure the economic value of education because it's hard, expensive, or otherwise contrary to the notion of liberal education? what a laughable position to take! We can and should do both as a society! If the practical skills are a promise - with no measurement - how are the worthy programs separated from the unworthy programs? No, the debate must be focused on:
1) What professions do society currently need more of? We allow anyone to major in any program even when there may be no market demand for that major - we don't just allow it, we give them loans and grants and tell them it's a good idea!
2) how do we educate people into needed professions? this will require the Assessment of Outcome. This must be a conversation between industry and education, else degrees are reduced to empty signals.
unfortunately, as the author so clearly demonstrates, academics feel a sense of insulation from the practical world, where practical and measurable skills matter.
2
And THAT is the difference between the academic world and the "practical world" and it's a very good thing. The University existed before free-market ideology and it will exist afterwards. It exists to allow culture to critique and modify culture which can only be achieved with a perspective that draws experience from outside the "practical world".
9
Exactly right! People forget that Universities are custodians of civilization. The entire civilization can be reconstructed if one university were to be left standing after a nuclear holocaust.
Is there some way to assess without unduly warping the ways people teach and learn to fit the form and content of the assessment instrument?
How do the Finns assess educational outcomes?
I always thought education was about spending quality time figuring stuff out under the direction of people who were good at helping me do that. One of the things I had to figure out was how best to communicate what I’d figured out to those people. Time on task? Bah humbug.
I’m glad I didn’t ignore the 9th grade English teacher who insisted we learn to take good notes and organize them. That’s probably crucial for many of us who learn more by hearing than by reading. I’m glad my kids learned more than I did about how to structure their decision-making.
But taking meandering exploration out of the college experience will probably work out about as well as taking recess, P.E., art, and music out of K-12.
7
I was a community college history professor for 22 years and can not speak definitively of my students' "learning outcomes." I started every class with THINK questions, which provided about 50 per cent of the student's grade. THese THINK questionsprovided the basis for class discussions. I included four review sessions each semester. My few exams focused on what a student thought and why.
Some students didn't seem to benefit much from my attempted immersion into fact-based thinks. A number of students, at the time, spoke highly of the critical skills they had acquired. On occasion I have heard appreciation from students who had been in my class 10-15 years ago. I believe that my students,with my portfolio method, absorbed and retained more than from a traditional lecture. I shudder at the expense and efficacy of endeavoring to prove this statisticslly. Indeed, I stopped subjecting my students to the computerized evalluationsthat my college required. Instead, I solicited anonymous student responses to a two-page questionnaire and shared the results with my students on the final day of class. Their personal broadly ranging comments i found credible. I shared them with senior college administrators, who also found them more credible than their computerized evaluations that few students (or professors) took seriously.
5
My university jumped on the "Assessment" bandwagon a long time ago, and frankly, it's a waste of time. Ultimately, it is a response to conservatives demanding "accountability" for taxpayer money and how it is spent. The problem is that we are so deeply caught up in the assessment game, when we could be better spending our time more creatively in our teaching. And you know what? All of this time spent on assessment and endless report writing still does not satisfy those critical of higher education. It's just another excuse for slamming "those liberal university elites." I challenge everyone one of these critics to spend some time on a college campus today and see how hard faculty and staff work and how inspiring students can be when allowed to stretch their wings with real learning--then they will see some outcomes!
15
"Here is the third irony: The value of universities to a capitalist society depends on their ability to resist capitalism, to carve out space for intellectual endeavors that don’t have obvious metrics or market value." Yes. But no matter how much you adjust the content or even address deep rooted structural challenges to learning, NONE IF IT MATTERS if education costs well into the six figures, the quality of university degrees are hugely disparate based on what you pay, and student debt is one of the biggest problems facing the country. No matter how much professors want to pretend the curriculum is undermining capitalism, they are participants in one of the leading drivers of inequity in America.
3
I think that colleges should concentrate on learning of the market-ready skills. That's why the students go to colleges and that's why they pay so much money. On the other hand, I don't think that orientation on the market-ready skills prevents the students to be exposed to complicated intellectual problems. I don't think that it is Sun zero game: either marketable skills or intellectual problems. I think they should enhance each other. And I don't understand that hostility against tests. How would we know what work what doesn't? How we will know what courses are great and which ones are the waste of time and money? You don't like the today's testing system? propose another one, explain how it is better and let discuss. Otherwise, the phrase that testing doesn't lead to better learning, makes no sense because we don't know how this 'better' learning is defined.
31
I disagree and until recently so would have most incoming students. It's only been over the past few decades that a majority of incoming undergrads began to say the main purpose of going to college was to service an employer and earn a high income. Before then many went to college for intellectual stimulation and to help build a better society. Similarly, back then the nation sent its youth to college in large part to lay the foundation for a society, not just gratify the labor needs of employers and the one percent. But than again the last thing Trump and his running dogs want are an educated populace. For them there's little profit in that.
7
This essay is right on point, a painful point. I first encountered "learning outcomes" in the 1980s at a "poorly funded" college that mostly taught 1st generation college students. All the faculty shook their heads. What was this? One more task for overburdened and poorly paid faculty? Didn't grades already measure "outcomes?" We engaged in a lot of passive resistance, no one wanted to do this nonsensical task.
Overall this so-called movement is, beside being a money maker for private companies, an attack on higher education, an attempt to impose corporate "discipline" and reduce respect for teachers and what we do, After all, if the outcomes can be quantified and measured, faculty could be replaced entirely by online learning, although you can bet that this will never happen at the Harvards of this world.
At the end of my career I taught at the University of Michigan, an elite public. Thirty years after we at Purchase College were quietly resisting, none of the Michigan faculty had yet heard of "outcomes assessment," but unfortunately they probably will.
8
Well said. May I add a quote from Oscar Wilde, "A cynic is a person who knows the price of everything and the value of nothing."
I would say that is where the corporatization/commodification of our lives has led.
"Not everything that can be counted counts; not everything that counts can be counted." - William Bruce Cameron
9
While attending graduate school in California, I was required to take the Myers-Briggs Personality Inventory, as well as the Hall-Tonna Values Survey (which claimed to "measure objectively" such esoteric values as "Ecority" and "Minessence"). It annoyed me to waste time and energy on such nonsense. Psychology and sociology are called "soft sciences" for a reason. They are long on hypotheses and short on cold, hard, scientific evidence. When I confronted my professors about this New Age twaddle, they told me, "The scientific method is just another way of knowing." They dismissed hard science because they knew they had NOTHING that could be measured and quantified objectively. The professors responded to my questions by dismissing me from my graduate program--even though I had PASSED all my classes. They didn't like me challenging their sacred "theories."
This nonsense over "learning outcomes" sounds pretty similar--shady businesses claiming that they can "measure" the un-measurable and "quantify" what cannot be easily nailed down--knowledge, wisdom, experience, and sound judgment. These companies have all the trappings of science (lab coats, tests, surveys, computers, and slick software that claims to "analyze" what passes for data). Then they give you a colorful, easy-to-read PowerPoint that has all the scientific value of a horoscope. Colleges and universities are buying into this con-game because they are desperate to have "numbers" that justify their tuitions.
4
The morass the college professor describes here is worse in spades at the high school level in the US. Teachers are leaving the profession en masse as they witness firsthand what it means to reduce education to bean counting.
Administrators love it as it keeps the teachers out of their hair and in the bread line of rubrics production and endless quantifying and recording of every moment of the purported learning process. And the mountains of specious data mined from continuous assessment and regular Big Standardized Tests empowers state and federal bureaucracies to wield the golden carrots and executioner's ax of "Accountability."
After I at first snickered at Trump's babble of compensating teachers "a little" extra for duty as hired guns, I thought being trained to carry and employ a firearm might be the only thing to keep me sane and in the classroom.
4
Not all that is taught is quantifiable, but it is just as true that what is often quantifiable is frequently not an accurate measurement of anything at all. Assigning values to abstract constructs and then making assumptions about teaching and learning based on those assumptions can distort the picture as well. Effective teaching and engaged learning, while the ultimate goal, do not always correlate to one another. There are so many other factors.
Should we then stop assessment? No, it has value. Refinement and selection are key and they must make sense in the context of the student and teacher. This relationship is not static nor monolithic.
Do we overuse and abuse assessment? Absolutely. This is true not only in academia where I work, but also in just about every aspect of daily interaction. It is the result of bean counters taking over the world. Therein lies the problem.
2
I'd like to comment on:
"Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning."
Time for serious exploration comes in graduate school. There is no such time in a u/g program. U/g's are taking four or five courses a term. Students are simply in the position of having to be satisfied with coming up with a couple of thoughts just beyond the superficial and moving on to the next subject.
Most faculty were outstanding students and they have a very poor mental model of students who aren't in the top 2%. Their ideal student is a mini version of themselves: someone who was really fascinated by learning and getting A's, and someone who could be a strong graduate student should they decide to go to grad school. I call this the mini-me syndrome.
This is not to say that attempts to quantify assessment of student learning are not misguided. But very few faculty have had a job outside of academia and hence very few know what is actually of value to the 95% of students who do not go on to grad school. For higher ed to get out in front of this, it needs to *actually* understand its customers needs. I say this as a faculty member of a highly-ranked public U.
2
I know it is fashionable in academic circles to complain about assessment. I did not particularly enjoy writing the last assessment report. I'm sure doctors, who once "knew" bloodletting worked, did not like how evidence based practice interfered with the doctor-patient relationship. Learning outcomes assessment is an attempt to bring evidence bases practice to the university, and of course, elite institutions do not worry about it, because their students already know how to learn and express their learning and no one really questions the value of their degrees.
The problem is that some degrees are worthless. Not just because they will not improve the student's earning potential but because the students were never challenged academically. Mainly this a problem with for profit colleges, but some programs at non profits have questionable value. Hence the metric. The students paid x dollars and improved this much on our scale.
Those of us, who know we add value to our students educations are going to have to prove it. Clearly the writer's University assessment system is not working for her. Maybe an objective should be that each student will complete a "serious exploration of a complicated intellectual problem" but again serious, complicated and intellectual will need to be defined and assessed.
The “critical thinking” goal of modern liberal arts curricula is completely wrong-headed. We don’t need people that swoon over Shakespeare and know how do to close analysis of his texts. We need people who can write and act with the vigor and relevance to life that Shakespeare’s work represents, and can hold that knowledge of Shakespeare along with knowledge of other major contributors to human cultural history. It’s the output that counts, not the inner mental state, and output can be assessed. It might not be measurable by computer programs, but the low quality of mechanized assessment tools is not a valid argument against assessment itself.
But assessment is a product of those "low quality mechanized assessment tools." Hence the assessments are low quality as well and of little value. The burden for showing that your conclusion is correct is on you who make it, not on those who see no measurable value in concocted learning outcome assessments.
How do one "write and act with the vigor and relevance to life that Shakespeare’s work represents" and not teach Shakespeare to an otherwise culturally illiterate student body? You learn from the best that a culture has to offer not solely by teaching to economically fruitful ends.
I think every student in college should be forced to take an entire semester of logic so as to lighten the burden of humanity's spurious arguments. It may not make them money but it will help them identify a conman when they hear one.
1
I've worked at a large university for some time. Assessment tools and the findings of institutional research have led to policy changes that have had real, beneficial effects. Some of these policy changes have led to more hiring: advisement, career counseling, tutoring, not just "administrative bloat". I can think of no examples of anyone changing their course to satisfy assessment expectations, nor can I think of any reason why a regional accreditor would encourage this behavior.
This is a very overbearing argument founded on anecdotal data that seems not to be terribly representative.
1
Amen to all that. I taught here for 25 years in the last century and watched the emphasis veer from engaging the intellect to bestowing a credential as the state government busily set about starving the State University of New York. Some of my students actually became interested in the subject matter; more often they were more interested in gaming the existing system and getting their tickets punched. I can only imagine what it must be like now.
3
A few brief thoughts on a very complex topic from a long-time university professor:
1) As a greater percentage of the population attends college, we are reaching further and further down in the ability distribution (and, yes, there is such a thing as "scholastic ability" and "general intelligence"). It is not possible to teach teach various college subjects at the same level of complexity, the further you go down in the ability distribution. Thus, many colleges and universities today have become, in fact, glorified high schools (and sometimes, junior high schools).
2) At the university level, assessment often becomes ridiculous. As an example: We evaluate students in university science writing classes on how well they "master" formal writing systems (such as APA style), when many students come in not writing or reading at good high school levels.
3) At universities, "evaluation" is almost always done at the level of individual classrooms and teachers. I challenge many universities to do broad assessments of random samples of graduating seniors on basic skills such as reading, writing, basic mathematic proficiency, etc. I have no doubt that such assessment would show that many graduating seniors cannot read, write, or do basic mathematics at anything approaching a university level. Universities won't do such assessments because they would show how profoundly they are failing at their mission.
3
I teach writing and literature at a community college that, until recently, enjoyed the reputation of being one of the best in the nation. To the extent that reputation was deserved, it was because faculty, administrators, and staff worked together to support innovative, caring, and certainly rigorous learning environments. Professor Worthen has written a devastatingly accurate assessment of the slow yet implacable process whereby the institutional flexibility required for creativity--flexibility that can only flourish in a climate of trust--has been replaced by fear of legislators whose ever-diminishing financial support is attended by ever more strident demands for "accountability." In a climate where trust and creativity steadily erode, such accountability is no longer guided by qualitative concerns and conceptions of measurement. Overworked faculty thus scramble to produce quantitative "data" that, as Worthen suggests, rarely has use other than seeming to keep howling legislators at bay. The one thing Worthen doesn't discuss is that this work can only be done with the cooperation of faculty. If faculty unions would make resistance to the erosion of their institutions a real priority, the whole process could be brought to a halt. But this would take considerable courage. The survivors of the Stoneman Douglas shootings are demonstrating a level of moral courage that may well prove revolutionary. We need such courage now more than ever.
19
I'm a bit surprised the Professor Worthen doesn't mention that the current assessment craze came out of university engineering colleges, where our accreditation agency, ABET, has been requiring quantitative metrics for many years. While this method does have some applicability in engineering, physics, etc., it's hard to see how one could possibly come up with a viable assessment tool for the arts or humanities. Bureaucracy run amok, again.
8
A reading of the history of TQM (Total Quality Management) will show that education was not alone in the 1980's. Every industry began to feel the onslaught of TQM. For me, one relevant observation is "what you measure, is what you will get." That is, be careful what you ask for, lest you get it.
4
Amen. At my university, Simon Fraser University, in Vancouver, BC, we successfully beat back a push for these vile initiatives. But they keep coming back. It's nauseating how quickly they have become an unquestioned good.
If it ever comes to it, my learning outcomes will simply be: students will have read what they have read, written what they have written, listened to what others have had to say and shared what they have to say, and hopefully learned a lot in the process, much of which I will never ever be able to predict, forecast, or measure and much of which will never be useful in the boring sense that everyone wants learning to be.
9
I think this is spot on and one could go on to write a book about it. I especially appreciated the connection between politicians (and voters) who want better outcomes, but are too lazy to use the very critical thinking skills they advocate to achieve them. Instead they use the shallow rubrics of standardized testing and data mining (never mind whether the data would pass muster using the scientific method) to "pretend" to be doing something. They simply can't be bothered by the heavy lifting of critical thinking - and enough of the public buys it.
1
unfortunately we are living at a time when even a good high school education does not provide adequate preparation for getting a decent paying job. at the same time high schools seem to be doing a worse job of educating students. colleges and universities are forced to do the job high schools once did and at the same time provide too much remedial education. colleges and universities need to stop providing remedial education and get back to accepting only the 25 percent or so of the population that can actually benefit from advanced education. high schools need to regain their former position of providing a quality education at that level that prepares students for good paying jobs.
1
I think that if college professors today were being honest, they’d admit that they have a lot of students who are simply not college material. And lest that be taken the wrong way, it must be emphasized that completing four years of higher education is not just a matter of brainpower.
3
This probably won't go over well, but the author gave examples of faculty clearly doing the bare minimum and still getting mad that assessment 'doesn't work'. Just 'throwing a question on the final to appease the assessment office' is obviously not going to get you meaningful data. You have the opportunity to get out what you put in, so don't get mad at your assessment office because they want you to do better. If you expect students to put thought and effort into their work, but you don't apply the same to your pedagogy and teaching philosophy then what are you even doing. Assessment doesn't need to be complicated or so controversial. It's meant to start a conversation about what's working and what isn't in your class and giving you the time and space to do so. Basically, faculty should just chill.
3
On a different tangent...
This seems to be a great example of why the digital era has been marked by low productivity growth. A lot of time and effort based on un-supported assumptions about efficiency. The whole Learning Outcomes industry, all those people beavering away at their computers and all of that hardware and software probably haven't improved anything. A waste of money that could be better spent on teachers and tutors.
6
The assessment industry is just one function of a much larger movement that has overtaken universities. Call it corporate managerialism. A fundamental value of traditional universities was faculty governance, a bottom-up governance, so to speak, in which highly educated and committed teacher-researchers were the decision-makers when it came to important academic questions. The people who really knew stuff made the educational decisions.
Universities are now essentially governed by an enormous managerial class and executive administrators who operate with a corporate managerial style. They quantify and rationalize and manage and gain power and drain off wealth from the educational enterprise. They in turn are responding to larger management powers in the federal government and often in the states, too. Accreditation agencies are just one silly form of this creeping managerialism where people who don't know much have power over people who do.
The managerial empires that have been built are only growing stronger. And universities themselves are undergoing a legitimation crisis. The future looks tough.
9
Perhaps students should simply all get participation medals. That way, no one’s feelings would be hurt by bad grades. Until, that is, they emerge into the real world and discover that standards and results do indeed matter.
Bravo! and thank you for these common-sense but very important insights into the monetizing of yet another sector of life: education. (The same thing has happened in health care, of course. Probably lots more 'industries.') Why do we let ourselves be sold these projects and goals? Because too few people bother to tell us that the emperor really doesn't have any clothes on. They don't have the intellectual capital or the moral courage to speak as well as you have. Keep up your crusade to EDUCATE!
1
I am a recently retired California community college teacher and have been on the front line of the pushback against this largely bogus attempt to quantify student learning outcomes. As a former faculty union president for the Contra Costa College Disrict and member of the Faculty Assoc. Of Califormia Community Colleges (FACCC) I am very pleased to see this discussion finally occuring at a National Level.
Many of my colleagues have said for years that if you can show me a "data driven asssessment" of learning that helps students to learn more please share it with us. We have had a learning assessment tool for years that is also quite broken due to bias and corruption called grades.
This notion that creating a rubric of terms and then "measuring" the succcess of those terms in the classroom is absurd and time consuming.
Helping students acquire knowledge and hopefully wisdom requries passionate teachers who care about their students and who have the time to develop appropriate, accessible curriculum not more data.
As this debate continues I encourage everyone to reflect for a moment on 5 or 10 facts, ideas, things, they actually learned in college if the were lucky enough to attend. We don't take memorized names and dates with us for long. . . . .
2
Absolutely true! I teach at the kind of university the author mentions.
1
As a former college professor, I would point out that one root of this problem is the notion that student are paying for an education.
You cannot buy an education, only the opportunity to get an education. If you fail to take advantage of that opportunity it is your own fault, not the institutions'.
425
Well said. There's the saying: "One does not go to college to receive an education, but to GET an education."
2
I strongly disagree. Learning is almost always a partnership, both sides need to be committed to the outcome. Suggesting that all learning failures are the fault of the student misses that there are a large number of professors who are much more talented at research than at teaching. Had my university focused on teaching skills for those in front of my classes, I would have been able to learn a lot more over the time I spent in university.
3
My ballet teacher used to say "I can't dance for you." It is an expression I use in my English composition classes.
5
Newspaper editorials are not really where debates about learning and assessment happen at a high level. The caricatured depiction and familiar critiques are well past worn thin. No one who thinks about learning theory would make any of the assumptions Worthen rightly dismisses.
1
I can’t think of a single aspect of life that has not been “commodified” since Reagan took office. When educational administers, often teachers who failed in the classroom, sold out the idea of a university back in the 80’s, they valorized the concept of education as a product. Republican ad Democratic state legislators and their corporate sponsors effectively transformed the university from what it had been for the last few hundred years into state sponsored job training which saved, and continues to save, corporations millions. And Americans went for it hook, line and sinker. So now everybody wants a piece of the pie. So underfunded colleges and universities admit anyone because they need the tuition. Ready for college work or not. They all might as well be selling hot dogs.
2
"Some schools use lengthy surveys...which claims to test for qualities like “truthseeking” and “analyticity.”" Even abstracted and re-presented in a slightly risible context, the notion of seeking the truth and the ability to analyze seem like respectable goals.
Now, how to best do this? Take this existing measurement tools and refine them until they do what they are ostensibly intended to do.
"Employers report repeatedly that many new graduates they hire are not prepared to work, lacking the critical thinking, writing and problem-solving skills needed in today’s workplaces,”
The purpose of a university is primarily research. It is this upon which faculty members are chosen and promoted. If individuals were compensated and promoted on their ability to convey information then they might be better at it.
" We’re doing pretty well with 18- to 22-year-olds from upper-middle-class families, but what about — well, fill in the blank.”
My graduate education promoted the concept that equity was a trade-off against efficiency. At the time, I had strong suspicions that, this trade-off was a bogus dichotomy, but it fit the promoted"ceteris paribus" framework that made conclusions based subsequent analysis seem objective.
Lip service must be paid to the concept of equity but only the resources that will be channelled toward it will be those required to keep the "natives" from becoming restless.
I attended a public university where the standards for teaching were nonexistent. I very often taught myself entire semester courses due to my instructors inability to teach. I got so little out of these classes due to nonexistent pedagogy and poor structure. I received an A in my intro to statistics class but learned very little about probability and statistics. I realized I was wasting my time by attending classes due to how inept the instructor was.
I understand the concern over learning assessments leading to bureaucratic frenzy. But universities should take a hard look at how effective their teaching is and how to improve it. Most professors don't receive formal pedagogical training and I experienced the consequences.
151
Thank you. It was surprisingly hard to find anyone who agreed that there needs to be some measure of effectiveness that is not decided on by the course instructor them self.
1
You dont take a class to learn the fundamentals - which you get out of the text book. You take it to learn the intangibles. What matter is not how well you instructors handle the fundamentals; how well do they know intangibles, and how prepared are you to seek them out.
3
Outcomes do not pedagogical competence make.
3
In my career working with low achieving schools, I always began by having the teachers write on post-its the three top reasons they believed students did poorly in school. We used those post-its to chart their responses into categories: political, financial, the district, the parents, the teachers, the students themselves. Lots of post-its in every category but one: the teachers. Then I would ask them which of those categories they had control over. It’s a big “aha” when they realize that the only one was “the teachers.” Armed with the data on the power strong teachers have to change student outcomes, we could begin our work. Articles which cast blame everywhere without looking inward serve no purpose.
1
Try being a high school teacher. Three different courses, five classes, daily. Lesson plans due five days in advance. Student failure is teacher failure. Your fired. We need reform at all levels of education, not more tests.
1
Superb. Thank you. This disease is also infecting the HE system here in U.K. It is pointless, it becomes an exercise in box-ticking rather than one of encouraging inquisitive minds to explore and grow.
5
And let's not forget the evil twin of learning assessment - standardized metrics for faculty productivity! All faculty are being shoved into one assessment box, with much of the income heading to a firm called Academic Analytics, which produces statistical analyses of individual and departmental productivity. Everything fits into the algorithm - number of publications, ranked quality of the journal, number of citations,etc.
Everyone in academia is learning how to game the system, and to skew their activities and research into the paths that are incentivized under this system. So one good paper becomes two mediocre ones if possible (an extra publication), areas of endeavor that are not as well-rewarded are shorted (let's be honest: teaching and community service), all in an arms race to prove that you and your department are more excellent and deserving of funding than everyone else. At our university, all new faculty hiring decisions are made centrally, with the overwhelming consideration being how this hire will increase our research profile, rather than thinking about the curricular needs of a department.
Many of us, and our children, shook our heads at the stupidity of the assessment regime in elementary schools, and that has now moved up the educational ladder. It's No Ph.D. Left Behind!
2
As a public university professor I, too, am frustrated by this trend, however well-meaning it may be.
The assessment tool I'd most like to see is whether the person being evaluated (student or otherwise) has the intellectual tools and curiosity to distinguish objective evidence from opinion; separating data from analysis/interpretation. The proliferation and blind acceptance of opinion-driven lies and Fake News is a major symptom of the poor analytical skills that poison our culture and lead to terrible policy decisions.
Lets insist on embracing objective, rational analysis.
2
The courses to be gotten rid of are the ones where students get enough factual background and analytic skills to see what the Republican Party is up to, with respect to universities among other things.
2
The elephant in the classroom is obvious to those with experience in higher education; and, as this author points out, "assessment" won't budge the elephant.
Throughout my career as a math professor - first at a state regional school, and then at an "elite" school (Wash. U. in St. Louis) - I saw shocking evidence of the real problem in higher education, and how that problem condemns most K-12 students to a poor education.
The corrupting of higher education has been going on for decades. As far back as 1980, deep thinkers like Clark Kerr and David Riesman saw the problem. Here for example, is David Riesman from his book, "On Higher Education: The Academic Enterprise in an Era of Rising Student Consumerism".
“…advantage can…be taken of [students] by unscrupulous instructors and institutions...Like any other interest group, the student estate often does not grasp its own interests, and those who speak in its name are not always its friends…”
Since 1980 it has gotten much worse, as the Spelling Commission, quoted in this article, reported.
Not much has changed, though. That is because it is hard for the public to really comprehend how our universities work. In my small way, I am trying to change that by telling, and documenting, stories from higher education that let people see for themselves how shocking things have become - and, importantly, how they effect ALL of education in America. The blog is inside-higher-ed.
2
I would be much more sympathetic to the writer of this article if they took ownership of their own shortcomings. I certainly am a fan of the best of higher education, especially since I am a professor myself. I also share the progressive politics that the writer, no doubt, espouses.
But that is the essence of the problem! I don’t even have to wonder what their political leanings are. We academics make a big deal about “diversity“; nevertheless, we do a really bad job of entertaining diversity of ideas. We pay lip service to the notion of intellectual freedom, yet we insist on “safe spaces” in our universities. Claim to offer analytical perspective, you’re so much of what we do offer is political correctness.
I agree with the author that a real contribution that a university can offer in a capitalist society is to resist the worst of capitalist impulses. It is even useful to think about what could replace capitalism. However, in order to make such contributions, researchers need to accept, if not necessarily agree with, what capitalism actually is and what capitalism aims to do. It is my observation that most academics outside of finance departments (where I teach and grit my teeth against capitalist cheerleading)don’t understand this. otherwise, they would do something more about tuition economics besides pressuring finance department to take in Chinese students who can’t speak English, but who can pay “full freight”.
Surely SOME accountability is needed here.
Why don't we get the statistics how many got jobs right out of college showing they "may" have picked up a few snippets of the vast knowledge they were fed in the class from K-PG
I am sure all parents and students would love to see that before choosing their half million investment location in cash and kind and hard work these days.
Been there, fought that. Welcome to the upside down world of k-12 education, Ms. Worthen. A world where students were only cogs to be measured and pretested and tested and post-tested. It all started with the simplistic ideas behind Bush 2's No Child Left Behind.
Real teachers had no place in education beginning with NCLB. Suddenly, there was only room for politicians. I shifted from being a passionate-to-my-bones teacher to thinking about early retirement.
Obama and his Education Secretary, Arne Duncan, dreamed up an "improvement" to NCLB, which, if anything, was worse! 'Race to the Top' continued the insanity that George Bush began and finally drove me out of teaching. Sometimes, it seemed, I had more clueless politicians in my classroom than students.
Now it is the turn of our higher education colleagues to fight these destructive policies. We have your back. We have been fighting these damaging assessment battles a long time.
So much for the hallowed social contract whereby we academics pretend to teach, our erstwhile students pretend to learn, and well get our summers off!
We need teacher evaluations. Some teachers are good and some teachers are bad. If you want to quantify a good teacher versus a bad one though, you don't need an investigative review. Go on ratemyprofessors.com. You'll find the list of compliments and/or complaints fairly consistent. Statistically speaking, it doesn't take long to paint a picture.
I remember one professor was almost universally described as "nice person, great teacher, lectures with his eyes closed." They were 100 percent right. I really enjoyed that class.
If you're confusing academia with professional student outcomes though, you're already in trouble. College is about learning how to learn. The various academic fields are really just switching lenses on the same camera. You might pick up a few technical skills along the way but students aren't supposed to focus on the camera. They're supposed to decide where to point it. If you haven't figured that out by senior year, no teacher can help you.
1
So you say that colleges are succeeding with upper middle- class students. How about middle-class students, whom you fail to mention?
And does every student in college actually belong there? How much "dumbing down" can you do? I say more trade schools. They can earn more $ this route too.
1
Things to consider.
Are you a person who roams around a lot, sniffing everything over large areas or are you are detailed person burrowing in greater details over a more restricted area?
Are you the bearer of a "head" well-made or well-fed?
So everything may have more to do with the correct ratio between area covered versus depth. If you could measure that ratio across individuals you probably would come up with a Bell curve.
How does critical thinking fit into that framework? Is how you pack groceries in paper bags at your local food-store a better "metric" than a multiple-choice test where the there are only 5 possible answers?
The next time you wrestle with such issues and you happened to have just boarded a plane see if you can talk to the captain for a second and ask if her his/her training was mostly via learned outcomes and multiple-choice tests.
People in the education research field are fond of saying that the plural of anecdotes is not data. The more important truth, however, is that the plural of bad data is not good data. Indeed it is often worse than no data at all. This pseudo-rigorous quantification of quality is one of the most destructive trends in American life today. An analogy can be made to studies on health, where it is now recognized that the enormous majority of them reach meaningless conclusions:
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0184604
From what I have seen, most data on education, of which the "learning assessments" Molly Worthen is talking about are a good example, are downright atrocious. You fill out surveys and forms for three years at two high schools or three colleges, and all of a sudden you have become an "expert" on the subject, named to committees at your university, appointed director of a "center for teaching excellence," and getting grants from government funding agencies. After which, of course, you are empowered to collect even more bad data.
I have no idea if it is even possible to collect good data on a subject so complicated as what constitutes good teaching. The most glaring problem that we seem unable to fix is that we have a horrible system of social promotion of kids from one grade to the next today, and it is working its way upward from the high schools into college and university.
Oy Vey!
3
A major error made by the assessment industry is the assumption that the important things that happen during education can be quantified using the questionnaires and tests that they can create. Education is ultimately a human interaction and the attempt to reduce human interactions to something simple enough to quantify using the tools of mathematics and physics is doomed to failure. Maybe someday a super-intelligence will develop a quantitative understanding of how humans learn, but we are just barely able to learn effectively and we are simply not smart enough to develop a quantitative description of effective education. Effective assessment provides a few clues that wise educators can use to make adjustments in their methods. Claims that what is quantified is even a small part of what actually occurs during good education are ignorant and arrogant.
1
The rise of the Internet of Things is going to put a real monkey wrench into higher education. Instead of having faculty waste their time inventing "objectives" and later documenting how well students are "meeting the objectives", university administrators will be able to tap into the cell phones and security cameras of the campus of the future, and see whether students are really studying about exploring for oil, building rockets, etc., or are wasting their time talking about gun control or climate change or sex or other topics irrelevant to their stated educational objectives. The faculty themselves will continue to decline in importance in higher education.
Think I'm joking? Wait twenty years and get back to me.
Administrators want those "numbers" because they don't want to have to be in the business of the actual transmission of knowledge that is deeply learned and critically evaluated. In other words, they are laziest bunch ever, trying to burden teachers for no other reason than to make themselves look as if the administrators have done some kind of work when all they are doing is living off the sweat of the teachers' brows.
2
K-12 education has been dealing with that garbage for almost ten years. Computerized testing takes up way too much time and tells you nothing about 5 year olds. What is even worse is that a computer determines benchmarks for reading and math proficiency (not very reliable), and then children are placed in computerized programs for remediation. Essay's on line. Don't even get me started. The people hired to evaluate them don't even have to be educators. Who needs teachers. So, welcome to the craziness post-secondary educators!
1
I applaud Ms. Worthen for a fine article. As a graduate of a first-rate liberal arts college, and as an Instructional Designer for various high-tech firms, I have long been aware that the concept of "behavioral objectives" is often misapplied. It is an excellent tool for training on specific skills, but training is not the same as education. Education is to training as analog is to digital, or as love is to sex. The human dimension-- what is not measurable-- is what we lose when we rely on data-driven analyses to measure education. As Ms. Worthen beautifully concludes, "there's just no app for that."
3
Just in case some have not heard, the job of college or university professor is arguably the best job in the nation. Know why? For the same reason employers rarely if ever hire someone smarter than they; administrators just cannot compete with them logically or intellectually, especially face to face, and thus risk a fate which corporate stooges deem worse than death itself. Public embarrassment. Tenure is way down the list, and don't let anyone try to convince you otherwise. It almost persuades me to a universal truth, that the sophists now rule the day. The problems of performance in education often boil down to the perverse incentives, created by accountability metrics which measure things right instead of measuring the right things. It is exactly the methods of the electoral college, and the main reason we have the administration we have in DC.
This article is spot-on.
I teach at one of those mediocre mid-western universities that has for the last decade admitted scores of unprepared students, as well as several gaggles of unprepared third-world students each semester whose only qualification is the ability to pay tuition. My university is one of those whose administrators are obsessed with assessment, have loaded the administrative ranks with assessors and generally wasted faculty time with no end of tedious and banal busy work, so obviously without merit or intellectual content that we can only wonder if they believe in it themselves.
What can be done about all this rot and waste in higher ed? Well, I'm going to do my part by retiring and responding to articles like this one as the opportunity arises. Mostly because it now amuses me and I am now out of reach of the administrators. Looks like there should be no end of such opportunities in the foreseeable future.
340
What role do faculty play in a situation like the one you describe? If universities are increasingly admitting unprepared students, due to the alleged lower quality of high school education and the smaller size of the pool of college applicants, then faculty have two options: either they create stronger developmental education and college bridge programs in order to prepare students so they can access the curriculum, or they close down programs which do not have a sufficient number of qualified applicants to make the program financially viable, of course, with the risk of losing their jobs. After all, faculty own the curriculum. But to sit down and wait for retirement, doing nothing but complaining about and obstructing any effort to examine whether the university is fulfilling its mission, whether students are meeting stated program goals, whether faculty are engaging in reflective pedagogy informed by disciplinary standards and conducive to student learning, while cashing a respectable paycheck and seeing scores of students mortgage their lives with six-figure college loans, is not an acceptable option. It is sorely unethical. It violates every principle of social justice when you realize that students are, for the most part, members of the working class and/or racial and ethnic minorities. There is no honor in your position. Indeed, someone who refers to students in such deprecating terms does not belong in an institution of higher learning.
3
I can almost guarantee that the answer is “none”. As an elementary school teacher (in the second largest school district in the country) I can attest to the fact that we have ZERO control over curriculum. There are grade level standards that must be taught and everything from the books to assessments are chosen by the state. Everything is top down mandates and administration is there to micromanage. I know many readers will think this is a public school problem, but remember that we have a union that protects us when we stand up for our students and their education, so it has allowed us some leeway. Private school teachers and definitely most charter school teachers don’t have that benefit, so conditions are even worse for their students in regard to “owning the curriculum” and having the power to do what they think will work best for students. Of course no one thinks of that when they’re bashing teacher unions.
5
As a fellow teacher, I understand your frustration with bloated administrations and the administrivia that superiors load onto the backs of faculty, but I take exception to your dismissal of what you call "unprepared third-world students... whose only qualification is the ability to pay tuition." Many of these students come from both developed and developing nations (Japan, Korea, China, India) that have intensely competitive education systems. While some are indeed unsuited for the demands of an academically rigorous institution, most are simply dealing with an unfamiliar culture and a lack of English proficiency. Your burnout does nothing to help these students, whose problems have solutions. Enjoy your retirement, but do not disparage the intelligence, potential, or real abilities of international students.
3
I was teaching at Iowa State University in 1982 when student course evaluations were first introduced, followed by grade inflation, followed by the assessment described here. It's like the industrialization of education.
9
In the humanities the idea is to learn to think more deeply, which usually means learning more about the factual or contextual background of the material, and also learning how to delve deeper than you have into that material. Not all students start at the same place -- which is a good thing in a humanities course -- and that means they won't all end up at the same place. The course I teach where it's easiest to state learning outcomes is on the concept of infinity. But even here I expect my math majors to end up at a place pretty different from where my art history majors or literature majors end up. What do I want them to know at the end? Cantor's diagonalization proof and also how to prove that the square root of 2 is irrational. But I also want them to think about what these thrilling concepts mean for the kinds of things they think about in their own fields. I don't expect my math majors to apply the idea of multiple infinities to the sermon about eternity in Joyce's Portrait of the Artist as a Young Man, though I hope they derive great pleasure from Borges' "LIbrary of Babel." I don't expect my literature majors to be able to prove the associative law in arithmetic (which my math majors have a hard time doing). I do expect them to see how vistas open up, and how many vistas there are. Should I put that down as my learning goal? I can say for sure that the students who take this class seriously learn something deep.
1
Perhaps a historical review of how employers view young adults just starting employment is due? Did they ever think the new adults had the right skills or mix of skills? I suspect not. Much like older generations have always felt the younger ones were flighty and unreliable.
I think it's an age thing, not a learning thing.
Plus, really the thing to measure is how well your students do once graduated, not while studying.
I've heard quite a bit of rumbling about first-in-family going to colleges and having extra difficulties. Perhaps some extra support for those students would be helpful? Not sure what would be most useful, asking them would be a good place to start.
1
So university will become like K-12? Education will be replaced by endless
meaningless standardized tests. Teaching and learning will become the ability to produce what the testers are looking for ... not curiosity, love of learning, or ability to think of something not on the test. Oh my.
6
There are socioeconomic issues that impact learning. They also impact employment. Employers who compensate university graduates at higher levels than non-university educated employees should expect a highly performing employee. That is the world of work. I assume, it is how you keep your position; with high performance levels.
I would argue that the best way for universities to avoid "shifting blame for student failure wholly onto universities" is to develop leaning outcomes and then holding students accountable for achieving them. If the students cannot achieve those outcomes, it if fully on them, period. In other words, some students should not be allowed to graduate if they cannot meet the standards of their major.
The only time I ever been upset with universities is when I have hired a university graduate who is not skilled in the area of their degree. The university has left me with the dirty work of terminating the employee; a job the university should have done prior that individual ever arriving at my place of work.
3
As an educator, I support explicit written course outcomes, because it helps protect students from sloppy teaching. At the same, when I reflect on my own college courses (biology major), the most valuable learning would not be captured by an outcome written on a syllabus, because it was affective, not concrete. It had to do with forming values, wonder, making connections between disciplines and figuring out for myself what this all had to do with me. In early childhood education, there used to be a saying, "The curriculum is what actually happens." In other words, what students actually experience, and how they actually process may be a far cry from what the instructor assumes is happening. Yes, the best teachers do know their outcomes and how to teach to them BUT they also how to be responsive to what students are actually experiencing and what they actually need. They do not teach the Outcomes. They teach the students.
3
It's sad testament to our society that anecdotal complaints from a tiny segment of our nation, business owners, led to so much wasted time, money and effort. Why is that when these folks speak, our government can only ask, "How high?". It is also distressing that the leaders of our nation's universities did little more themselves. These people are teaching our young adults to think critically?
8
How can the Critical Skill of "Learning to Learn'--
needed for a lifetime of joy as well as rapid change?
How can that be measured? Can it be?
When did we decide that the only value in Education was the Employers' needs?
5
Another challenge to successful assessment is the culture of student-as-consumer and the entitlement it creates for both students and their parents. Meaningful assessment requires that an educator be both willing and able to tell students who aren't meeting the objectives that they are failing, that they don't deserve an A or even a B. In academia today, the onus is always placed on the faculty to ensure that every student learns -- which is unrealistic when your students, being as they are varied humans, include some who are unmotivated, lazy, resistant, etc.
If you cannot call a student out for shoddy work for fear of a lawsuit, falling tuition revenue, or even bad online reviews (such as Rate My Professors), then any assessment you do is automatically meaningless. Students will be passed with flying colors whether they learn anything or not. Supposedly the overall institutional assessment is supposed to detect and eliminate that, but in reality, whether it's in K-12 or higher ed, there is too great an incentive to game the system for assessments to have those kind of teeth. What would the institution do with a student who is not meeting the benchmarks? In my experience, few are willing to tell a student who lacks motivation or ability that they just can't cut it and show them the door (and in the public school system, that's not an option). If the assessment has no consequence for the students, it will produce no real change. Learning takes effort from two.
19
Measuring learning outcomes creates a mountain of tedious busywork, leading educators to spend more or more of their time looking over their shoulders toward administrators and bureaucratic functionaries, who are usually paid more, and less and less time attending to their students and their discipline.
In my experience, administrators have no end of appetite for quantitative data that can be read at an executive glance from a graph, until that data turns up something inconvenient. At one time, I taught and took a lead role in curriculum development for an "adult school" in accreditation trouble. The administration launched a grand plan to prove the school's value to students and employers. Unfortunately, the outcome assessment data showed that the more classes students took from the school, the poorer they performed on outcome assessments. Oops.
Now, there were many reasons for that result that were not an indictment of the education offered, starting with the likelihood that the more classes students had to take to complete their degrees, the more likely they were unprepared and poorly performing students to begin with. But delving into factors beneath the pretty graphs was of no interest to the administration. So that report was buried in a hastily dug grave, and that was the end of outcome assessment for a good long while.
4
My oldest, a Merit Scholar, won a 50% scholarship to a very prestigious large private liberal arts university. It still cost me about $200K. Afterwards he told me that it was a waste of money, that it would have been better to have simply given him the money while he educated himself. At least he might have been able to make a down payment on a house.
He said that it simply wasn't challenging, that most of the students weren't that serious, and that the faculty was not very demanding. In the decade since he has learned far more in self-study of history, literature, music, and philosophy than he did at university, for free.
In the last 50 years, several forces have coincided to create the perfect storm which has hollowed-out American education from top to bottom (excluding medicine, nursing and STEM).
Faculty used to out-number administrators; now administrators far out-number faculty with resulting exploding costs.
The parental culture which used to prevail at home and school has been overwhelmed by a peer culture which is inimical to education and which has rapidly absorbed the culture of the lumpen-proletariat. Most parents and teachers seem to have either joined with, or acquiesced to this lumpen culture.
"Progressive" reforms of education have resulted in an enormously dumbed-down curriculum of low expectations which addresses all sorts of fluff while ignoring what used to be the core of an education.
Wander through a college bookstore and see for yourself.
15
Excuse me if I point out another factor: the rampant consumer culture that has overwhelmed notions of what higher education is about. If your son received nothing of value from a "prestigious private liberal arts university," it's more than likely because he and perhaps you expected higher education to be something done to him and for him. To the contrary, the value of the education one receives at true institutions of higher learn is the value of the education one takes it upon oneself to obtain. This is the reason quite fine education can be had from "lesser" schools by students intent upon making the most of what courses and professors there are. Ultimately students educate themselves, wherever. The benefit of better schools and faculty is to make that process easier. Your money buys opportunity, not results. Nowhere will education be done for students as passive consumers just jumping in their sleep through the curricular hoops.
3
If your son was a merit scholar, you should have spent and extra $60K and sent him to an Ivy League school. As a product of one, I can attest that the education was magnificent.
"On many campuses, faculty must include a list of skills-based 'learning outcomes” on every syllabus and assess them throughout the semester.'
I fail to see how this is considered onerous. In fact, I can't imagine what faculty were doing before things like this were mandated. Just present some material and hope for the best? Wouldn't letting students know precisely what they'll be expected to learn over the course of a semester just be an obviously good practice? Is there some benefit in hiding this information?
As someone who spent 8 years earning 3 degrees, I think it's great that universities are finally starting to put some expectations on professors, many of whom can't be bothered to show up and teach, let alone gauge student accomplishment.
5
I have 3 degrees - two of them at the graduate level earned at public universities. In all my time as a student I never experienced a faculty member who didn't bother to show up and teach. Today as an educator I experience that most teachers, many of them working as adjuncts, are if anything over prepared and underpaid. They are eager to impart their passion for knowledge and critical thinking and are run ragged by a bureaucracy whose reason for existence is to collect data from faculty rather than actually spend time improving the conditions of students and faculty that allow for learning. These administrators, as well, did not enter the profession to collect data. Once the data is collected, little of substance is done to make necessary changes or changes are mandated upon the faculty without proper support. The problem with education today is--and I am referring in particular to public education at all levels, which is what the vast majority of students depend on--, though it is required and valued, it is not valued enough to be adequately resourced. Nor is the profession of teaching valued enough to offer educators an environment where they can put their focus on students and learning, including their own ongoing education in a rapidly changing world. The data gathering so far has not lead to accountability on the part of those who can help make education better by budgeting adequate moneys to support educators and students, namely, our state legislators.
4
This is an excellent essay that hits upon a number of important points -- and problems with the current bureaucrat-driven assessment culture. When pundits complain about the rising costs of higher education and the flocks of assistant Deans running around, assessment on steroids is one reason why.
I believe it had been said: not everything worthwhile is quantifiable, and not all quantification is worthwhile.
7
And don't kid yourself into thinking that education is the only field where mindless bureaucratic thinking prevails. Hospitals and our "health care delivery system" are rife with it. There are Six Sigma folks and a myriad of others, supposedly "quantifying" nursing care, yet good nursing care does not readily lend itself to being quantified. Good nursing care is something other than filling out quarterly reports, marking boxes on electronic records, or doing a task as quickly and "efficiently" as possible. Sometimes inefficiency is called for. sometimes spending time with a frightened, sick, or anxious patient is what really needs to be done. but efficiency is the highest virtue to bureaucrats, not compassion, expertise, or simply caring for the patient.
1
I knew a college chemistry professor who said the purpose of the exams was to inform him what the students were learning and where they were falling short; or more to the point, what he was doing well, and where he needed to improve.
16
You hit the nail on the head. We are in a world of pain when assessment is more about Faculty demonstrating that they’ve met an administrative mandate about assessment than what the student’s have learned.
2
Since the tax payer revolt of the 1970's and the rise in the popularity of laissez faire capitalistic slogans and assertions of the natural science of free markets all the lessons learned during the gilded age, the great depression, and the boom economy that ended in the 1970's have been trashed and education has been steadily defunded as a luxury irrelevant to the robust growth of national economies. The outcome is that funding for education has become scarce and colleges and universities simply cannot educate all who would benefit and offer any courses of study that anyone might want to pursue. That means identifying what to fund and what to await for more funding. Measurement of utility and outcomes of what is being done are essential to do this in a rational manner.
1
The 20th-century model only works for students who’ve already been prepared for that style of learning before they show up on campus.
Students who come from backgrounds where getting a college education was not always a given often struggle to learn the rituals that make up a college course. A detailed syllabus with explicit course goals helps them get oriented, and regular assessments that are aligned with these course goals helps them gauge whether they are on track throughout the semester.
That’s not an endorsement of the corporate assessment industry. But rejecting their model of learning outcome assessment should not lead us to return to the 20th-century model. It just means we need better ways of assessing learning outcomes for the students that show up in our classrooms today.
7
What are the students seeking in the learning environment described in this article? Does it matter? How does the learning assessment industry assess these issues?
Students should enter higher education with the intent to learn something. It is their own responsibility to obtain that learning. If they actually attend these institutions with the understanding that the mere obtaining of a piece of paper called a diploma will be the ticket to their own riches and happiness, I believe they are misguided. The students are paying for something. It is their responsibility to get what they paid for. If the people working within the organization somehow prevent the student from obtaining that, then I would be concerned. I doubt that is happening.
3
Maybe if we hadn't savaged academic standards in the pursuit of enrollment dollars years ago, there wouldn't be so much skepticism regarding what students are really learning. Their grades would tell the world what they had learned, or not.
9
I am an instructional designer working in higher education.
Two thoughts: Most educators - even at the highest level - do not know how to write proper Learning Outcomes. If you were to review the LO for the course I teach (on the side), you would see the kind that pertain very specifically to "...serious exploration of complicated intellectual problems". Writing LO is a woefully neglected craft which, it seems, only instructional designers have practiced.
Second: You've touched upon some legitimate points about the mania of assessment in HE. However, Learning Outcomes are not just for students. They provide a foundation for designing what the course ought to be about - it has to be about *something*.
Unfortunately, there are lots of excellent instructors who are not skilled in the *design* of instruction so that the course feels coherent to learners, the readings and media are relevant, and the basis of demonstrating learning has any connection to the point of taking the course.
The reality of education is that *good* learning experiences require some basis from which instruction is to be designed, much like the charter for building a building informs the oversight of an architect.
Learning Outcomes might be annoying to critics, but without them, courses are chaotic (I've seen plenty!), and degree programs would be incomprehensible.
4
Why, yes, it's amazing the race ever got out of our caves without learning outcomes to guide us
Thanks for the commercial.
1
Here is my reply to the repliers: Who are you going to believe about this issue? Someone who bitches and moans about LO from an institutional assessment perspective, or someone who literally works all day with faculty to design online courses for adult learners AND also teaches online to those same learners?
I see from the replies here that the only rebuttals that can be offered are snark. Let me know when you have something meaningful to add to the conversation. In the meantime, leave the real work of providing meaningful higher education experiences to the experts. (How's that for a commercial?)
While I support assessment for improvement in the classroom, it often seem that a lot of assessment is just to say or prove that some assessment was done. I rarely see a follow up plan to try and improve based on what the assessment showed.
6
Finally, a force that might temper the intrusiveness of accrediting bodies.
No one, not this author, disputes the merits to assessing learning outcomes. They have, however, become an end in and of themselves, dis-empowered faculty, empowered very expensive and wasteful administrative staff at colleges, and made the lives of faculty painful.
We all can go in and teach to produce good "learning outcomes" scores. We know this is not good teaching, nor good practice, nor an ethical and moral thing to do. I have asked professors at B-Schools at top ranked institutions; they couldn't care less about accreditation. The lives of faculty at mid-tier institutions, or local and regional institutions, have now been rendered absurd by the accreditation regimes and police. I end up spending more time administering assessment, assembling reports, sitting in meetings, and a whole bunch of things when my principal job is to learn new ways of firing up the imagination and learning of my students, and produce new theoretical and practical knowledge with research.
If, as a faculty in an unknown B-School, I resist this assessment regime; I am the problem. When Molly Worthen questions it, I hope it triggers some discussion and meaningful action.
Thank you Molly Worthen.
Kalidan
87
Actually, Kalidan, a number of us are skeptical about learning outcomes. Personally, I can't control the outcomes of my students. I can offer them the opportunity to learn, and I can try to create an environment in which learning will occur. But I absolutely cannot make learning happen, nor can I predict what will be learned when learning does happen.
1
Thank you, Dr. Worthen, for emphasizing that education extends beyond job training, to understanding the world around us whatever our field of study. As a teacher, I've been trying to forward the idea that this is just as critical in high school as it is in college, trying to lead students to actually use their minds for more than the achievement of a high grade, or often even a passing one. Preparing them for higher education, for the exchange of ideas with people from other places and backgrounds, for the lure and love of curiosity can be a tough task, but always worth the effort.
20
Good article that covers the most important points. Before I retired I was a library administrator at a state university. Outcomes assessment was just picking up steam at the end of my tenure. Along the way, I was lucky enough to teach one course per semester for seven years: an invaluable experience for an administrator. At the beginning of each term I would ask the students how many had jobs. Generally, the answer was: everybody. The last time I taught, I asked who worked at least 35 hours a week. About a third said yes.
Back in the late '90s, academic vice presidents were passing out a book that explained the secrets of successful teaching. The big secrets were 1) establish relationships with students and 2) assure that the student is spending 20 or more hours a week on classwork. Well, I can guarantee you that none of my students had the time for 20+ hours a week of classwork.
I used to say that I was proud to excel at outcomes assessment, because I was a literature major, and my strength was in a particular bureaucratic genre: 'assessment fiction.' Frequently, my very serious colleague deans did not get the humor in that.
24
Wow. This gets it right, though may be a little short on cynicism. I'm a retired psychology prof who had a pretty good background in measurement (Ph.D. from UNC) so drew many assessment assignments. I worked on projects at the university, college and department level. The groups I worked with won awards at each level from visiting assessors of assessment. I found a couple of plaques cleaning out my office. I've pointed out to my former colleagues that my real talent was finding activities that made it look like we were doing something and applying the results to improve instructions (that's key for the visitors) without inconveniencing anyone too much. I was always seriously concerned about what my, or our, students were learning, but that was independent of assessment. The basic problem with standardized assessment is that you basically cannot find tests the scores of which do not reflect mostly general cognitive ability, i.e. whatever the SATs measure. Elite universities don't worry much about assessment because their kids will blow the top off any standardized measure regardless of what they might be learning. Ive had ideas over the years about how we might assess the skills of our graduates, but nobody is going to make the investment real assessment would require. Senior projects in the real world, not too related to any particular discipline, would be a start.
14
In 1960, 20% of college-age youth were enrolled in college. Today, it's close to 70%. Did high school students become that much smarter or have the standards been lowered to admit more?
There are certainly some college students who, in the past, would have been overlooked or otherwise dissuaded from attending college. But that many? And then we complain that many college graduates do not have the appropriate skills. Big shock!
We need more quality trade schools to provide the skills necessary for today's job market without saddling students with barely manageable debts. College is not for everyone.
143
NCES Digest of Stats shows 45% of HS grads in 1960 in college.
https://nces.ed.gov/programs/digest/d99/d99t187.asp
There are plenty of trade schools and apprenticeship opportunities for those who actually desire them, but we live in a FREE society and many folks desire the prestige of a college degree, no matter how ill informed.
I am a public school educator, and this is all very familiar. I spend far more time with documentation, assessment, and follow-up or training regarding these activities than I do with my students. This time does not come out of my contracted school day, either. Some of the requirements have to do with funding (e.g. billing Medicaid for school-based therapy services), and some have to do with NY requirements for schools where enough students in our high-poverty district did not do well enough on the state-mandated assessments.
If you have any choice in the matter, college and university faculty and administration, buck this trend before it takes over your vocation as educators.
19
If colleges didn't have to make up for the failings of public schools to teach children ... well ... pretty much anything, there'd be no need for this level of assessment in universities.
As it is, a BA is now just a stand-in for what a high school diploma used to be: evidence that someone can read, write, and do basic math.
Too late!
Thank God someone is finally calling out the assessment industrial complex. Anyone with any appreciation of learning knows that the most important things to know are also the hardest to measure. What assessment mainly aims to do is standardize teaching, so that passionate, committed, creative faculty are hammered down to a readily measurable baseline of mediocrity, one that can be replicated easily in on-line learning "systems."
154
Ms. Worthen explains, articulately, the inherent conflicts with the use of proprietary outcome measurement tools in undergraduate education. I believe, though, that the distinction made between higher education as a place to learn 'critical thinking skills,' versus a place of 'delivery mechanisms for skills to please a future employer,' is too simplistic. Unfortunately, the cost of higher education has increased nearly 20 fold in the past 30 years, and post baccalaureate job salaries have not increased proportionately to this. Consequently, graduating from college with good critical thinking skills no longer cuts mustard for the investment made. If the astronomical costs for higher education are to be justified at any level, at least the graduates should be given adequate skills to pursue sufficiently paid careers. I'm sorry, I absolutely support 'learning for the sake of learning,' but not at the current price tag.
6
The cost has not risen appreciably as percent of GDP, however, the states have diminished per student funding over "the past 30 years" shifting a greater portion of costs on to students.
https://nces.ed.gov/fastfacts/display.asp?id=65
In any case, little of this has much to do with the "industry" of learning assessment which has seen outsized growth not least, and gone unmentioned in this otherwise good essay, by the accrediting agencies, and their shepherds in DC.
Outcomes assessment has very little to do with what you call "learning for the sake of learning" which, apparently, is a distaste for curriculum and not for perennial students. Believe me, outcomes assessment has as little value for certifying the quality of welding as it does the writing of a poem.
1
So what "skills" are these, specifically? I hear this all of the time, but never any specifics.
Thank you, thank you, thank you. You have said what all of us who work in higher education know, but seldom dare to say. Colleges and universities aren't failing our young people; our society is failing them, and this failure begins long before they get to college.
Watching a friend with his 9-year-old daughter the other day gave me a great example of how and when critical thinking is really taught. It's not expecting an 18-year-old who has been spoon fed information to spit back on a test her whole life to suddenly be able to analyze a history text and apply it to current politics. It's taught by parents interacting with children, asking them questions and guiding them through understanding their world, trying different approaches, making connections, and using those approaches in other situations. This is not learned by watching UTube. It's time consuming, taxing, and impossible for a teacher to make up for when it doesn't happen at home.
The second societal problem is that kids who were not academic superstars used to have other options for making a living. Now, with blue collar options closing and paychecks shrinking, we're expecting colleges to magically turn kids who would not have gone to college 20 years ago into "knowledge workers." We should somehow make up for deprivations in academic and social preparation, and we should do it on a shoestring budget, preferably online, keeping expensive face time to a minimum. No surprise we fail so often.
17
Permit me to summarize: Too much time and effort spent on testing.
Not enough time and effort spent on teaching and learning.
I agree. So it seems like we need to re-evaluate testing.
There are, I think, two very different kinds of testing.
1. One kind of testing is needed to measure outcomes against a stated objective. State licensing exams for Law, Medicine and Engineering fall into this category, And our professional schools should be oriented towards licensing, albeit, we may need to update the licensing exams regularly to reflect changes in the profession. And pedagogy depends on this type of testing, to identify the preparedness of the student to move on to the next level of education. It is also used for diagnostics, in identifying the weaknesses in the student that require shoring up.
2. Another kind is needed to evaluated the effectiveness of educational methods. This second kind is spurious at best, and at worse, extremely wasteful. However, there is a legitimate need to identify fraud in education. This kind of testing requires a before and after approach that measures the distance the student has travelled under the methods to be tested. Schools that attempt to educate the less-prepared students might show a significant result in the distance the student has travelled, even though the student might still not be prepared for the next level. This still leaves the question as to whether the effort to help the less-prepared student is cost-effective.
4
The number of administrative positions U.S. colleges and universities has more than doubled in the last 25 years, while number of full-time tenured faculty – the people who actually create the curriculum – has been constant or declining, replaced by adjunct professors, lecturers and other instructors.
The rise of assessment industry, with its coterie of consultants, accrediting agencies, and textbook publishers, doesn't provide any value to students and faculty, whose time and energy it wastes, but it does satisfy the need of the bean-counters in the administration to justify their enormous salaries by counting beans.
223
The Cleveland Federal Reserve Bank has done surveys about employment at US colleges and universities, and this is their latest report:
https://clevelandfed.org/newsroom-and-events/publications/economic-comme...
Interesting points covering 1987-2013—all measured as percentages of total employees--
* The widespread belief that administrative and management staff have increased is NOT true—not changed at all.
* Clerical/secretarial staff numbers have declined.
* Other professionals (IT, lawyers, librarians, etc.) have increased.
* Faculty members have increased.
* Full-time faculty have decreased, as part-time faculty have increased.
One can conclude that, as public universities experienced a decrease in state support funding, more part-time adjunct faculty have been hired to save money. Likewise, clerical staff numbers have decreased as more people do more of their own clerical tasks on their PCs, and secretaries are being shared by more people. The increase in other professionals is more difficult to explain, perhaps because governments have increased reporting requirements, universities compete for students (= tuition $) by providing more amenities, etc.
2
Thank you for pointing out that a college education is key to not only obtaining employment but producing a society capable of meeting the challenges each day brings. I have been at institutions that scramble to obtain these quantified learning outcomes to understand student learning or low graduation rates. They waste funding resources that can be otherwise productively funding programs to aide the student to obtain their goals and the university goal which should be to provide an education.
2
I cannot say if these learning assessments are correct, or testing what should be tested. I do know that If you can’t measure something, you can’t improve It. Outside of education this seems to be a well accepted fact. Saying the assessments are testing the wrong things or not accounting for demographics is not a reason to stop assessments but a reason to improve the assessments.
8
"Outside of education this seems to be a well accepted fact."
Well, perhaps a mass delusion. It is why we have business leaders who know the cost of everything but the value of nothing.
1
The most salient point the author makes here is this one:
"Without thoughtful reconsideration, learning assessment will continue to devour a lot of money for meager results."
Without question, the focus here is on the wrong thing. The final product of a college level course should involve original thinking and ingenuity, for which a standardized assessment can't be given. Part of the role of the professor is to assess student work, and by and large college professors are very capable of this.
That there should be focus on improving college instruction in general, however, and on using valid data to do so, not wrong-headed. The reason for the perception of an academically degraded generation of students now is that college is no longer just an option for the academically elite - it's a necessity for all who desire a middle-class life. In response to this necessity, we are finding more ways to get more students to college. For most of these students (most of all students, really), it's well understood that lecture-style instruction is ineffective. What's less well understood at the college level is how ineffective seminars are for everyone but the "top" students. More development on on-the-spot assessment and intervention from professors with ALL of their students is definitely needed if improving instruction is a goal, as it ought to be.
4
I am a CUNY faculty member at a community college and agree with much Molly Worthen writes, but, based on assessment processes we've developed over the past decade, would argue that they can be meaningful, and can be the basis of productive conversations about what we do in the classroom, and how our students learn.
To be meaningful, assessments of learning outcomes must be understood within the context of the college, and the students we serve. BUT we are not off the hook for assessing simply because our students might be "disadvantaged." In fact, done correctly, assessment can anticipate and defend against the forces of neo-liberalism and the systematic defunding of US higher ed. Frankly, I'd rather be forced to come to terms with what we do poorly, what we can do better, and what we do well in a process that is owned by, and driven by, my colleagues, than to wait for cookie cutter assessments that might trickle down from on high.
5
Thank you, Molly Worthen, for so pointedly calling our attention to this modern-day educational emperor without clothes. After a lifetime of teaching English literature at the college level, I have become convinced that measurable student learning outcomes are a figment of the utilitarian imagination. Curiosity, love, and wonder always take circuitous paths toward understanding—and true understanding, like Shakespeare's star, "whose worth's unknown, although his height be taken," is always immeasurable.
82
There is no "right answer" on an English essay, or there shouldn't be. If there is, it can be found in Google. It's not a puzzle with a right answer, it's an opportunity to show your stuff, wake up the TA reading the paper, make her smile. If you do that, however, it's likely more your inherent smarts than anything you learned in the class about "critical thinking". That said, a university provides an opportunity to try out stuff like that, if you've got it in you. The job for the faculty is to bring out what's inside the students, and the curriculum is only the context, not the purpose. Of course along the way, one picks up a lot of specific knowledge and skills, as a side-effect.
This is happening at universities everywhere... Thanks to Dr. Worthen for speaking out. I just forwarded her piece to the Executive Council of my faculty union - because this is a place where the voice of the faculty can be a valuable counterweight. This is coming from administrators, many of whom are enamored with online learning and with "job-ready" skills, rather than with educating students. There are certainly excellent deans and provosts out there who do care about education, but the whole move towards slicing, dicing and numerically tagging is coming from the ranks of the administration...
10
Its been almost 50 years since I was in university and things do seem to have changed. Apparently, there is now an emphasis on ensuring that the students are learning something and the lecturers are being held responsible for that. That is very different from the time that I was in school. We had the advanced calculus professor who read from the text book. We had our calculus TA who forgot to present a topic that was tested on the examination. he justified his error by saying that we, the first-year students learning calculus, should have known that it was in the course and learned it. Students were on their own and if they learned something or not, it was of little concern to the faculty and the university. Of course, there was my electromagnetism professor (a full professor) who lectured from a text book. It wasn’t the text that he assigned for the course but why not.
So, things have changed and universities are now held responsible for students learning something. Why is this not a good thing?
9
If you read the article, the point Worthen is making is that the responsibility is being shifted onto those who have the least ability (time or power-wise) to do anything about it. Universities aren't being held responsible at the highest level, but rather their lowest-paid, most-overloaded instructors are being made to take everything on themselves with little recourse. Tenured professors who teach one course a semester will never be removed for phoning it in, but the contract lecturer will be booted if enough students say they didn't learn enough, despite the four-course teaching load. Further, the assessments they use to measure whether students learn don't measure what they purport to measure--least of all student evaluations. All recent studies have shown this.
The biggest irony in all of this (and it's mentioned in the article) is that those universities that have the highest rankings are least likely to do assessments at all. But these universities are often where students report getting the least contact with their professors, who get little reward for being good teachers and throw all their energy at research and publication.
2
I tell my students that the day they receive their degree, they will know the time spent had value if they understand how much they don't know. Maybe the professors who did not spoon feed you were teaching you a larger truth: that curiosity and learning are life-long processes that we are never finished with, and that the tools for learning laid out in the classroom are the same ones you must use for the rest of life. If you wait for knowledge to land on you, you've missed it.
Assessment, as it is currently done now, would not have solved any of those problems. Assessment is not about evaluating individual faculty and TAs. It does not require testing every student in every section of every course, and problematic faculty are not identified. Assessment is about collecting statistics about population-level performance, to demonstrate that the program (not individual faculty) are producing students who have achieved broad, nebulous outcomes, such as "ability to reason mathematically."
It seems sad that so much money and effort has been spent on assessment when there doesn't even seem to be agreement on what it is, exactly, the students should be learning.
This article raises good points about assessment, but i'm disappointed that the author continues to bear down on some false dichotomies. Universities should be trade schools AND teach critical thinking/lifelong learning skills. It is true that students need to be treated like students and respect the student/professor relationship AND ALSO it is true that students are customers and clients paying for a service. All these things can be simultaneously true and I think we'd have better universities if we stopped going to war about which vision or role of the university is correct.
4
As a math professor at a college in California, I also see the trend that the writer identifies that's overwhelming higher education in America. Everyday I am bombarded with persistent appeals by textbook publishers to adopt their web-based curricula because "our unique digital solution" will simplify student assessments and quantify all learning outcomes. The goal is clear: Move everything online, let students take charge of their own learning (whatever that means), and you, the teacher, slowly disappear. Buzzwords like 'critical thinking', 'data-driven outcomes' and 'real-life problems' are liberally sprinkled in their glossy brochures and in the hefty, expensive tomes called textbooks. In this clash of jargon and analytics, single mothers, full-time employees and low-income students desperate to improve their lot with education are left holding a huge overdue bill but with diminishing returns every passing year. Teaching how to think and engage in protracted inquiry will not come from one-time assessment test per subject per semester/quarter but from teachers and students working together across months and even years to explore the deeper questions and solving problems worth solving, even if accompanied by a high risk of failure.
14
Thank you for a compelling argument. Let me add another thought: humans are remarkably complex and varied, yet we continue to search for standardized and simplistic descriptions of student performance to evaluate them, their teachers, and the institutions teaching them. But we do not know very much about what works. Throughout my 63 year experience as a student, a parent of students, manager in a fortune 500 company, and as college teacher for the last 11 years of my work life, I have seen one untested or proven fad after another take over schools and curriculum (and training programs in big business). I have also seen that good teachers have historically given lip service to the fads and continued to do what works for them, adding and deleting activities in the classroom as the teacher adapts to his or her growing skills and changing circumstances. But as Ms. Worthen points out, the assessment craze is not only of little value, but it saps the meager resources of time and money available for classroom instruction. Finally, a thought to consider: how many of the educators whose classes you attended do you remember? When you teach a class, or a person at work, or a child at home, I bet you rely on what has worked for you in the past as a learner-- because that is a life line to reality: that worked, even if only on you!
51
While I agree with much of what Professor Worthen says about learning assessment in this piece, I think she gets a couple of pretty fundamental things wrong—encapsulated by this paragraph near the end of her piece.
"Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning."
Treating "market-ready skills" as distinct from the "serious exploration of complicated intellectual problems" is, or should be, a false distinction if students are approaching their learning the right way. It is learning how to grapple with complicated intellectual problems, and all that goes into that effort, that prepares students to achieve at the highest level in their lives and careers.
As for the statement that the university is our students’ final and best opportunity to explore complicated intellectual problems: any professor worth their tenure should be striving to ensure that that is not the case. If you are not teaching with the goal of instilling in your students a lifelong interest in examining complex ideas, you are not doing your job.
29
As noted in a recent essay here in the NYT, it has been a long time goal of so-called conservatives to whittle education down to job training. Once you permit that to creep into your educational programming, no matter how false a distinction you might think there is between job-readiness and intellectual commitment, you will almost automatically lose the latter. You cannot be thinking about ideas and ideals freely if you are simultaneously wondering if this will get me some kind of job.
1
I'm a retired elementary school principal. Several years ago the school's science coordinator complained vociferously that one of the teacher's did not follow the 3rd grade curriculum. Instead he had his students study the moon, its phases, its origins, etc. She wanted me to reprimand him for deviating from the science program. I asked her what she and the curriculum guide expected of 3rd grade students at the end of the grade, i.e., what do we want them to know and be able to do after immersion in the science curriculum. I also asked how she would evaluate whether the students achieved those expectations, e.g., some performance assessment preferably both written and hands-on. If the students in this class studied the moon all year but were, nevertheless, able to demonstrate an appropriate level of achievement of our curricular goals, then I would congratulate this teacher; otherwise, I would address the teacher's non-compliance. Indeed, this teacher was so professionally competitive that if he was made aware that his students fell short of the mark (what the typical student ought to know and be able to do at this juncture in the curriculum) that our learning community (represented by grade, school, district, state and nation) had developed for 3rd grade science, he would have taken corrective action without my having to express any criticism.
4
This is one of the finest critiques I have seen on the issue of assessment, the bottom line being that what constitutes the education of a human being is to a great extent unquantifiable. Unfortunately, educational institutions have been forced to operate out of a business model. I was a college teacher and department head in a Midwest liberal arts school when quantifying learning became the big issue in the '90s. My department produced a lot of data for the North Central Association and other entities, but what we measured could only be an imperfect evaluation of what students took away from their learning experiences.
44
While the writer describes what the situation as it is today, I don't buy every explanations that she offers.
For one, data-driven assessment is not the issue. Having data to support decision-making is a good approach. The challenges are to determine what to measure and how to measure it.
Further complicating the picture, is this fact: In the rush to promote "having a college degree" as THE meal ticket to a better life, our society/government have collectively conflated the two aspects of higher ed, namely, academia track and research, and career track. What is the career prospect of "intellectual history"? Realistically it's close to zero. Yet, the faculty is left with the task of justifying of "falling enrollment" by trying to conflate its importance for real-world job market ("critical thinking!", "analytical mind!"). Some subjects are really for pure academia pursuit. To conflate their importance for job markets in order to justify the high price tags that students pay for that degree, is a very wrong-headed approach.
It's far too easier an explanation to say that 18-22yo in upper-middle-class students have little issues, yet those students in poorly funded colleges from power class are struggling. Let's face it: They all pay a high price tag (relative to their affordability) to attend college. If college can't serve them to a point where they can get a job, or they might be better off in community college, then what's the raison d'etat of this writer and her faculty?
9
The problem is the "data" itself. The point here is that the assessment regime lives and dies on numbers, and so more and more assessment tools must be created to generate numbers. "How well, on a scale of 1-5, did you learn X?" is typical of the assessment outcome "measurements" used. The bottom line is that after all of the extra effort put into the assessment regime, it really does not offer any more useful information to an engaged teacher than that teacher would have gotten through grading students' work.
4
" If college can't serve them to a point where they can get a job,"....
That is not directly measurable. I have a builder friend with a degree in physics, a plumber with a degree in history (who I wish would write a book on the Vietnam war), and I have three degrees, only one of which, for the most part, was required for my career. Or was it? Who knows? I've other friends with no formal post high school education who earn substantial livings, that would, if to do it again, find it difficult to gain employment in today's market. What people do or, do not, do with their education and how much they are willing to pay for it is as much a choice and personal freedom as any others we have granted to us.
No teacher objects to assessments, but people need to know the whole story.
The assessment is the camel nose under the tent. In no time, the test is used to determine pay and funding. As soon as that happens, teaching is replaced w test prep.
The difference between teaching and test prep is clearly seen in NYC where, until recently, not 1 student from Success Academy could gain admission to the city’s top magnets.
I teach in public, my own children attended public through 8th grade; privates are infinitely superior because they are free from the pay/funding tyranny that turns true teaching into test prep.
16
To be soundly reliable measurements of comparative differences must be in terms of numbers which can be mathematically analyzed and confirmed. There are all kinds of errors and biases possible when people try to quantify qualitatively varying abilities. That does not mean that there are not ways of comparing people’s success in learning al subjects taught in schools, teachers do it as part of their job. However, measuring the knowledge retained long after matriculation is completed and the relevance of that to the knowledge needed for careers or to excel generally is not measured, but to evaluate the usefulness of a particular curriculum really is essential.
The reliability of the measurements is critical. If the reliability is not close to foolproof, they will create enormous problems. People under great pressure to show certain results are not going to care about the reliability of measurements but just what they seem to indicate, and they will act upon what it is because they must act.
1
Any statistician attempting to run validation tests on most assessment data would weep/laugh. I know from experience. Accrediting bodies neither want nor care about validation testing - they just want data saying SOMETHING.
2
This has already happened to health care. Hospitals, first under the influence of consultants, then on their own, started trying to quantify everything. They focused on those things that could be measured easily at the expense of others,
like quality of care, that are much harder to quantify. It hasn't been pretty. Increased throughput looks like a winner when you ignore the externalities. I'm afraid there is no escape from the MBA monster.
40
Strongly agree. Physicians have become subordinate to non-medically trained managers who develop protocols and procedures based on bureaucratic and economic rather than medical concerns. I agree too, that there seems to be no escape. There are rare independent primary care physicians who take patients on a yearly fee basis, but they have limited access to hospitals.
5
Brava, Molly Worthen! Now, if only you could persuade all of the college administrators and accrediting agencies that are so insistent in compellingfaculty members to devote time to Learning Outcomes Assessment! The basic premise of Learning Outcomes Assessment is sound: faculty members need to determine what they want students to learn, and then ensure that curricula, courses, and assignments are designed with that end (the "outcomes") in mind. But compiling ever more data in order to measure the difficult-to-quantify outcomes of, say, a philosophy essay is misguided. What would Socrates say about the effort to assess and quantify whether students are living an "examined life"?
14
What a wonderful analysis. I hope this is read by university faculty and administrators all over the country.
As a long time university faculty member, my colleagues and I have experienced so many of the same misgivings about the big push for "outcomes assessment" and doubts about the utility of this very costly process.
At the end of a recent presentation about the next stage of my own university's assessment process (done at least in part in response to the mandates of our regional accrediting body), I asked the presenting administrator what process, if any, they had designed to determine whether the whole assessment program was in fact doing any good? The response was pretty much a blank. Seems that despite all the rhetoric about how we all needed to empirically measure outcomes, the assessors hadn't thought to ask whether all that assessing actually does any good?
94
Excellent point. I have found, however, that the assessment class, especially when entrenched in government, has sufficient funds to design and publish studies that meet their aims. Such studies can be refuted, but that takes time and resources, and most teachers and researchers care more about their students and their own efforts to contribute to human knowledge.
6
They didn't dare to ask, because they're afraid of the answer.
1
Owing to my squeaky wheel tendencies, I was put on a committee about "assessment" before our last accreditation started. A representative of our accreditor came to one of our meetings. After some conversation, one committee member asked, "If what we decide about learnings outcomes assessment doesn't meet your organization's expectations, will we lose our accreditation," and he (to his credit) said: "absolutely not". So, the threat of accreditors is largely a paper tiger in most situations, used by administrators or by consultants to achieve other ends, I suspect.
1
Thanks to NYT, for publishing this and other writing by Molly Worthen. This is a particularly thorough short set of thoughts that tend to get flooded over by the opposite in public discussion. Very nearly all of it was roughly in my own mind after a decade of teaching –– BY 1975! At an increasingly strong college of the sort that she commends, I and like-minded teachers both supported each other and still had to battle the 'outcomes programs' –– sometimes winning a bit but clearly losing the longterm trend.
One factor in the difficulty of the battle is that we were resisting simplistic thinking, most fundamentally the notion that some perfect system of education can be constructed. Teachers and students are humans in particular situations; they need to be allowed to try do better continually and in mutual respect and cooperation.
19
I concur with everything that my distant UNC colleague writes here, but as a professor in a Humanities discipline and director of a core interdisciplinary sequence, I recognize some small but significant gains from the emphasis on student learning. Before assessment, I thought about my teaching goals rather than the student learning goals, that is, I focused on what I wanted to cover in class each semester and each day, assuming that the students were learning what I was teaching. Assessment disabused me of that assumption, and more importantly, put some responsibility for the gap between my teaching and their learning on me. I think that made me a better teacher. Second, students benefit when they are aware of and engaged in the course learning outcomes, if they are clear and well-designed. Critical thinking is indeed the Holy Grail of outcomes, but it is not particularly helpful to list as a goal in itself.
That said, those gains are rarely worth the cost and effort that we expend on them. I usually find the process of assessment exhausting, despite the occasional nuggets of gold that we discover.
The treasure trove comes later. The real measure of a student's critical thinking is years after graduation. One of my professors used to say that teaching is an obscure art: you couldn't know what effect you have on your students unless you could see their bookcases in 30 years. I'm proof. He's gone now, but my bookcases, my thinking and my teaching bear his fingerprints.
262
This is well said. Although I lament the waste of money in the assessment industry, I think it is salutary that professors (like me) and administrators have given more attention to figuring out what students are absorbing and how best to meet them where they are.
3
Teaching is for high school where you care about "student learning". Presenting the material, at the highest level of scholarship, is for the university. You should not care so9 much about "student Learning". That is no longer your responsibility as your students are now adults. In the real world, your students will encounter "teachers" who dont care if the student learns - and very often. You should care, of course, but not so much that it gets in the way of scholarship.
3
There is an obvious parallel to standardized testing in schools. I have long supported testing, because it is simply crazy to suggest that we have no way of determining, other than through self-serving impressionistic guesswork, whether students are learning anything.
Time has worked its will on my views. I still support the idea of testing, but have no faith in our education bureaucracy to carry it out. From "teaching to the test" to outright fabrication of test performances, the bureaucracy and teachers' unions have largely outlasted the political consensus for student testing.
I have taught graduate students for 20 years. In my field, we receive little formal instruction about how to teach when we begin, and there is little discussion of what, precisely, we should be imparting to our students. On the one hand, I welcome a conversation that asks us to be mindful and transparent (harder than it looks) about what it is we do in the classroom, and why it should be expected to produce a given result for most students. On the other, the creeping gobbledygook Worthen describes so well seems likely to consign this inquiry to the same fate standardized testing has met. Specifying, with reproducible consistency and accuracy precisely how our efforts should transform student understanding is difficult work. It demands introspection, dialogue, and a willingness to learn about how minds other than our own actually work. I am not sure we are up to it.
6
The author makes some valid points. Taking it a step further, we really need to question whether a bachelor's degree should be the only portal to the American middle-class. We place so much emphasis on obtaining this credential that it has become the be-all and and-all to good jobs, decent living and a happy life. Many 18-22 year olds are not ready for higher education, including many who graduate from the most prestigious college prep academies. I also laugh whenever I hear employers lament about the lack of "critical thinking" skills of recent hires. How many 20-year veterans of the workforce can demonstrate these skills themselves?
27
Agree that the bachelor's degree is far too narrow a portal to the middle class and to meaningful work. But broadening the entry means transforming or winnowing our institutions of higher ed. They are already suffering some legitimation doubt, and young men, in particular, seem to be less convinced than ever. The universities are so bloated with managerial excess and non-educational services and goals that it is hard to believe that they have the flexibility to adapt.
2
My students can interpret the HR diagram (completely trivial “skill”), but they can’t tell you 2 reasons why you’d never need an umbrella at night on Mars. Or why it’s cold on the top of a mountain.
Most of what can be reduced to a multiple-choice year question is useless.
And shouldn’t we aspire to see what 20-year graduates cannot do and then fix it?
2
It is obvious that a large minority of the citizenry cannot. I sure would be nice to improve that though.
As a high school teacher in NYC public schools I'd like to welcome you to the club. We have been arguing for years about the wasted dollars that moved from children and classrooms to private firms that do absolutely nothing to actually improve outcomes for students. It is much more profitable to skim this money from schools, and now institutions of higher learning, than fixing underlying causes of the poverty and lack of focus that harm students and are, studies show, 90% of the reason student achievement falls short.
287
I think it's important to note that many of these changes are backed by private organizations such as the Lumina Foundation, and not by the faculty or unions. Simplifying assessment and outcomes makes it easier to put classes online, which may be the ultimate goal for those who see our students as a lucrative market. I want my children to be taught by real-life, highly-qualified teachers and professors why will teach them how to think and how to live. Our public universities and schools are national treasures.
130
The author nicely captures the essence of what a colleague of mine calls the metrization of education. If we can't measure it (student learning, college rankings, etc), it is not real. To what end? How does it end? In less learning I fear.
74
I would add that it's the MBA-ization of education. If it can't be counted, it doesn't count. We see it in business, where metrics are the goal and not the tool they should be.
4
Thank you for the well developed and important thoughts. These are the targets we hope students reach. It is important to remember that assessments and competency based education works well in some areas and are challenging to institute and use in others. Implementing them in a meaningful manners, means that we need too understand what they are good for, how to leverage the tools they provide, and how to develop content in a meaningful way. And these are the issues so eloquently defined in this piece.
7
In what areas of higher education do assessments and competency-based education work well? I can't think of any. I'm sure I can't think of every area, so please help.
2
Thomas,
I believe competency-based learning began in the Army, where soldiers were taught the skills necessary for the accomplishment of rote tasks, such as the disassembly, cleaning, and reassembly of items such as M-1 rifles and field telephones, or the procedures for replacing tank treads. Then some ed professor had the brilliant idea of applying the practice to abstractions such as logic and critical thinking, where real students and teachers became the victims.
1
malibu frank: Thanks.
I have observed for years that we have become a culture where nothing is valued unless a number can be attached to it.
Thank you for this piece.
314
I'm not sure what Professor Worthen is afraid of. If my job could be replaced by a computer program, it would be. Similarly, if a college professor isn't adding educational value beyond what a few YouTube videos or a good textbook would, he or she shouldn't be employed. The only way to determine this is with standardized tests, imperfect though they may be. If you're not teaching anything testable, you're probably not teaching anything useful.
Also, I take issue with the criticism of "elite institutions plus trade schools" as a dystopian outcome of our educational system. In fact, the most competitive economies - Singapore, Germany - do just that. While the top 10-20% of students attend university, the rest are guided to a trade, often while still teenagers.
Compare this to the example in the article, where students at Arkansas State University learn about "cross cultural connections" and "empire". What career exactly are we preparing them for?
27
I teach college writing and literature. The tests students take to be placed in classes are notoriously unreliable so I don't have high hopes for achievement tests. Before everything is based on testing, we need to be clear about what proper standards of achievement are. As to your comment about Youtube videos or textbooks, dispensing the materials as provided and testing students on it is exactly what many professors (adjuncts) are being paid for. That this won't get the job done seems to be lost on administrators. We have never had a European system. Until there is a full-scale reform of education, students need to be exposed to all kinds of subjects so that they learn to think; to consider points of view other than their own; and so that they come out of school with some understanding of the world we live in because a lot of them are not getting any of that in high school.
16
"I take issue with the criticism of "elite institutions plus trade schools" as a dystopian outcome of our educational system. In fact, the most competitive economies - Singapore, Germany - do just that. While the top 10-20% of students attend university, the rest are guided to a trade, often while still teenagers."
The US system of government has moved steadily away from the levels of support that the most competitive economies put into their educational systems. You can hardly propose that we strive to emulate them when we won't pony up for the costs,
"What career exactly are we preparing them for?"
Wasn't it part of the author's point - indeed, the theme she ended on - that this was not the correct question?
2
Even though I am a supporter of standardized testing for the professions, like Law, Medicine and Engineering, I do not think education is limited to professional training. Indeed, the word 'education' has traditionally been aimed at the development of character, which involved much more than intellectual skills. And admission to professional training often rested on the character of the individual more than his/her intellectual abilities.
2
Try a classroom full of 30 year olds, over 60% of whom are women and mainly single moms, who either completed secondary school in a woefully underfunded and rigid system and who work full time, tend to a family AND go to college. Now make 60% of them Hispanic, Black, and Native American. These students are heroes for being where they are and doing what they do, and their growing skepticism of "the way things are" and an ability to express themselves in clear and correct sentences, and to use numerical data rather than preconceived notions to support their ideas is nothing short of phenomenal. What is "out of date" is the Mr. Chips vision of a leafy British (or Connecticut) university quad full of upper middle class males and children of the nobility that informs "rubrics" set by testing companies who are selling their wares to State legislatures.
200
I'll take a class of single mothers any day. They are some of the most motivated and well-organized students I teach.
4
Yes, those are challenges, no doubt. But perhaps you should ask yourself this question: If all these struggling students cannot cope academically, is it the college's fault for admitting them in the first place, and then make them pay, and to see them struggle along, to eventually fade away? Would these students be better learning a trade, or attending community college, rather than attending a college to get a piece of paper called "degree" that can't help them land a job? You tell me.
2
As a college teacher, I agree with all this except characterizing the assessment movement as Neo liberal. As far as I can tell, colleges and universities are under attack from the right who want to defund everything but the military and are looking for data to use to do it. And didn’t Margaret Spellings work for Dubya? That was a neo con administration.
152
Neoliberalism doesn't mean progressive; it means Reagan/Thatcher/the Chicago School of economics wherein everything is based on whether there's a measurable and short-term return on monetary investment, is always the automatic answer for a troubled public utility, and the state should stay out of the business of regulating the economy except to the extent that it helps huge for-profit corporations.
63
Doug, "neoliberal" is a code word whose meaning is very similar to "neoconservative". I have yet to see a definite difference between them, except that the neos began in different schools of thought, converging later into a right-wing elite orthodoxy. (I'm willing to learn differences if there are any, but they are both members of the right-wing elite.)
The Chicago School is far superior to the Frankfurt School. Unfortunately, it is the Frankfurt School which has prevailed and now dominates education in America, UK, and W. Europe. It has destroyed what used to be an amazing edifice of learning; we see the results all around us.
Thank you for articulating so well what university teachers have had to deal with in the course of our careers: the mostly fraudulent, time consuming and costly attempts at "assessing" teaching. Like healthcare, education should be considered a public good, and paid for by the collectivity, by us "the people", as in most other advanced societies.
351
I agree that education should be considered a "public good." However, using evidence of outcomes to improve the effectiveness of both health care and education strengthens the public good by showing where improvements can be made.
One can debate how to assess learning outcomes (there are many ways to do so, no one of which is sufficient) but unless learning outcomes are measured in some manner we cannot know whether we are being successful, both as faculty and as a society, in providing the education we seek to convey.
Education, both K-12 and higher education, needs improvement in the US. Our best schools are world class but most are just average among developed countries (and a few would be an embarrassment in a developing country). Just as our best health care facilities are world class, overall our health care system in the US does not compare well to other developed countries. We need to stop thinking our best schools are typical and look more at the median and bottom half for improvement.
Although I believe more money is needed for education at all levels, I also believe that just throwing money at education has not worked and cannot work. Assessment of learning outcomes is one tool, not the only tool, to determine where additional money is likely to yield needed improvement. Student success is a complex topic; there is no single factor that determines it. Many improvements are needed across many domains, each of which is necessary but not sufficient.
2
Dave, the shibboleth of "measurement" has captured your mind. No, "measurement" is not everything and cannot capture everything. As a couple of experts I know through a friend have said, measurement is hard or it's worthless. Doing hard measurement means devoting resources to it instead of to teaching, since the community is not willing to pay for both.
7
"Doing hard measurement means devoting resources to it instead of to teaching, since the community is not willing to pay for both."
Thomas ... Perhaps that is true in your community; in mine we do both and the teaching improves from assessing the student learning outcomes. Such assessments are not perfect, but they are better than not finding out whether students are learning what is expected in the curriculum. They may not capture "everything" but they do capture a lot of useful information. If we do not find out what students are and are not learning, how can we get better at ensuring what is intended to be learned, be it propositional or procedural knowledge, actually is learned?
When I was an executive in business I believed in continual improvement of our processes. We practiced Deming's PDAC cycle in doing that. In academia, where I now am, I believe that assessment of outcomes is part of continually improving the education we offer to students, be that through curriculum design or how we teach.
The author’s plaint may be a day late and a dollar short.
We once effectively acculturated children in our public schools, teaching them the basic skills and knowledge required to economically compete and clothing them in an understanding of who they were as Americans and citizens. Undergraduate work was intended to develop higher-order thinking abilities for those with the capacity to exploit them in life, and prepare some for highly specialized training at the graduate level. No longer. While that public school process appears to remain effective for the children of upper-middle-class and wealthy families, with all the economic and emotional supports they can offer children, it appears to have largely failed almost everyone else except the most gifted.
In such a new reality, where we have failed badly in a retail way during a child’s first eighteen years of life, undergraduate environments have become the venues for trying to make up for what we failed to do earlier as a society. Given that this may be a young adult’s last chance to prepare for being a useful and contented social cog, can it be surprising that we would measure the outcomes of those four years with increasing intensity and desperation?
We all know what is said to run downhill, and the author and her colleagues sit disconsolately at the bottom of that hill. Her plaint questions why the original purposes of her vocation can’t remain what they once were.
They can’t because we’ve mucked up everything else.
70
The politicians are unwilling to provide the support necessary for upgrading the education of low-income children. Examples: Nurses and learning specialists who go to the homes of small children to teach the mothers/fathers how to help their children develop intellectually, e.g., by reading to them daily. Assigning experienced, capable teachers to low-performing schools with discipline problems. Assigning adequate social support staff to deal with the extra-scholastic issues children in poor areas often have. Making sure the buildings in poor sections are in good shape. Making sure that one or two jobs provide a living income for a family with children so the parents have the time and energy to work with their children, such as in reading to them, and emotionally supporting them. Those are just a few that come to mind. All have been tried successfully in other countries like France and Canada, or here on a small scale but then never properly expanded.
7
Thomas:
It's not the politicians but the people. To fix our primary-through-secondary problems so that the schools serve a far larger percentage of our population adequately would require that such education be largely severed from local funding and control. People don't want that, as it likely would dilute quality in wealthier communities in order to buttress it in poorer ones; and because a more centrally-controlled education might not insist on teaching children that the world is 6,000-10,000 years old.
However, if I wanted to wax useful on our current public education woes, I have a lot of ideas on the subject; but I'd probably choose a forum in which to do it that allowed me more than 1500 characters (including spaces) to develop those thoughts. My comment here wasn't aimed at solutions, but to chide the author on kvetching about the consequences of ineffective public education to over-measured assistant professors of history at the U. of North Carolina at Chapel Hill.
4
The culture no longer supports child-rearing. In fact, the cultural messages children are bombarded with work against it.
1
I understand the concerns of the writer. I have taught in high schools and universities and then worked in program evaluation. It is difficult as a teacher to find out what students have learned and to say with some degree of objectivity, that one student deserves an 'A' while another deserves a 'C'. I would ask how the writer assesses students in her classes. My guess would be that she gives essays, but I found that students who write well do better on essays across the board than weaker writers, no matter what they have learned. Rubrics are a great way to impose some objectivity. At the other end of the question, people who invest in education and manage it have legitimate questions about what students have learned. Asking whether certain programs are more effective than others is a legitimate question. I certainly do not know the answer to finding the best measures for learning, but to suggest that there are no measures, or that seeking good measures of learning is a waste of time is simply unfathomable. I think that what the writer is primarily concerned with is the way the measures get used. If administrators blame poor learning outcomes on the teachers and ignore the effects of relative poverty and social capital, that is a different and important question.
57
In your teaching career, perhaps you have run across the notion of a "straw man" argument. For example. "My guess would be that she gives essays, but I found that students who write well do better on essays across the board than weaker writers, no matter what they have learned. Rubrics are a great way to impose some objectivity."
As it happens, I also teach at a university. I believe Professor Worthen is saying that she is teaching intellectual history, not remedial writing. Critical thinking and effective communication are just skills, the by-products of good instruction along with meaningful effort by students over the span of a college career, not the reasonable subject matter or focus of college level courses.
14
I believe the writer is concerned with both the poor quality of the usual measures, in that they don't measure what they claim to or what matters, and the misuse of measures to make administrative decisions.
5
I, too, teach at the graduate level of a major university. Critical thinking and effective communication are important aspects of what students must learn. You are correct that these are skills; not all knowledge is of facts (propositional knowledge), some is of how to do something (procedural knowledge). Both types are necessary for success in life, career, and education.
One of the courses I teach is the pre-dissertation course in research methods. In this course both propositional and procedural knowledge are expected to be developed.
All faculty somehow assess student learning (excepting those who record an A grade for all students ;-( ). One of the issues in assessment of learning outcomes is the validity of the assessment. There are many ways to assess learning outcomes, no one of which is wholly sufficient by itself. Assessing propositional knowledge requires a different approach than assessing procedural knowledge. What I do know is that "I know it when I see it" lacks reliability as an assessment method, both as to inter-rater reliability and, often, as regards intra-rater reliability.
9