My company participates in a nearly identical if not the same program detailed in the study. It drives me crazy because unless you are wearing a heart monitoring device all of the incentives are based on a step program. So what do you get? People meandering around shaking their arms. The problem is there is no real education or incentive on actually getting fit, so people figure out the best way to earn points and do that. It rewards moving slow and long regularly. A hard workout will "earn" you about 20% of the points you need for a day, but you can really rack it up if you mow the lawn or walk around the zoo or play with your kids for an hour. Not that those are bad things, but if you know how to work out well and efficiently you know those activities aren't going to create any significant change in any but the most sedentary of people. Worst of all, you can see how it is all designed from a marketing perspective to advertise to these companies all of the benefits their employees are seeing from increased activity etc. It's all show and no substance. The wellness program makes a lot of money, the fitness gadget companies make a lot of money, the employees get a little pocket change, the companies fork out a bunch of money, no one really benefits other than having fun graphs to show how much better everyone is doing.
3
One thing you can be sure of. Someone or something is making big money running the "wellness program".
3
A program that gets people to go the the gym 7.4 times a years over 3.8 days a year is not a success. A program like that is a waste of time. A trip to the gym every 2 months will do absolutely nothing to improve your health.
I can tell you from experience, corporate employees just find these programs annoying and intrusive - I've never seen anyone change their behaviors for the good based on these programs.
6
It sounds like people didn't like the exercise program at work. Maybe it was too hard to change clothes, was at an inconvenient time, or they didn't want to exercise with colleagues. Possibly the same people were exercising elsewhere.
3
Good point about disrupting a workday to change clothes to exercise. Our existing office work a day wasn’t designed for that. Time to re-think substituting office drudgery with a health break.
1
What if the intervention really mattered? My observational study in my practice looked at ten kids with chronic ear infections. Ear infections begin from bacteria in the back of the nose and when the kids washed their noses 4 times a day with a solution of xylitol their ear complaints were reduced by more than 90%. Xylitol helps our immune defenses in the nose work better. We call it defense medicine to contrast with the offensive kind normally practiced when we use antihistamines to turn off the back up defense that is the bothersome runny nose. Why does the nose run? Why not to get rid of the pollutants it senses! Why not turn the bothersome symptoms off? Why not hobble the defense of your favorite football team? They would lose; and so do we! Look at the increases in nasal problems–sinus and ear infections, allergies and asthma–since 1965 when antihistamines were made available OTC. Look at commonsensemedicine(dot)org for more on this.
3
As part of their "Health" program, my company sends out "News Letters" - one week the advice was to "always carry Kleenex on your person" :-)
2
Another variable that is usually not accounted for is the "social desirability" of the research question, which skews participation and results.
1
If I may throw a wrench into the machinery, does the very fact that incentives were offered to participate create a selection bias affecting the results? At least on the observational side. I suspect that health insurers will try to use this data to argue that we just need to offer more, umm, compelling incentives to move more people into the participant column.
This is of concern because some employers offer carrots to encourage participation, and some use rather punitive sticks. There are probably debates about what is a carrot and what is a stick, but I would say a stick is a cost that the employee cannot avoid except by participating, such as a higher health insurance paycheck deduction.
I am not sure this study answers whether carrots and/or sticks improve employee health and/or reduce costs, but it may be misinterpreted as doing so. To answer that question, we would need to compare outcomes at employer A, offering a wellness program with no incentives, vs. employer B, a program with carrots, vs. employer C, who uses sticks to "encourage" participation, and vs. employer D with no wellness program at all.
7
Article would have been more informative had it been written well. I read through it and couldn't figure out what the meaningful conclusions were.
For example, "went to the campus gym 3.8 days per year, and those who participated in it went 7.4 times per year. " and "Those in the control group went 5.9 times per year, and those in the intervention group went 5.8 times per year." Can anyone help me understand how 5.9 is significantly different from 5.8?
My conclusion?
See https://www.frameworksinstitute.org/
1
@Erik Kengaard 5.8 is not significantly different from 5.9, which is why the wellness program showed no significant improvement when viewed as a controlled trial.
2
This is precisely why we have all been deceived into thinking that regular EXERCISE CAUSES good health and healthy outcomes. All of those exercise studies are observational in the exact same way the observational analysis of this “wellness” program is. But in reality, regular exercise simply REFLECTS the healthier behaviors and eventual healthy outcomes of those who are the more able and willing to exercise.
As here, “The most likely explanation is that participants differ from nonparticipants in very important ways,” said Julian Reif, one of the study’s principal investigators. “Therefore, when [exercise or activity level are examined], the differences seen between those who take advantage of it and those who don’t are due to differences in the people rather than differences from the program.”
Show me some large-scale, long-term, all-taker, randomized controlled trials of exercise with hard-outcomes.
Surprise, there aren’t any. If there were, as here, they would likely show no difference between couch potatoes and avid holier/healthier-than-thou exercise types. Indeed, forcing couch potatoes to exercise, as per current recommendations, would likely harm or kill them sooner.
2
@SRP
Bingo: "Show me some large-scale, long-term, all-taker, randomized controlled trials of exercise with hard-outcomes."
1
@SRP We know there are acute and long-term physiological benefits of exercise, which will reflect in decreasing the risk of a variety of diseases. This will improve quality of life and in most accounts, "health" so I am a little confused on your conclusion as written.
This article identifies that study design can impact the results. Workplace wellness as a program is what is inefficient at improving health, not participation in exercise.
5
@SRP "Show me some large-scale, long-term, all-taker, randomized controlled trials of exercise with hard-outcomes" where people are either forced or forbidden to exercise for their entire lives? There's no way that could possibly be feasible or ethical. Not to mention unnecessary given mountains of evidence on the health benefits of exercise.
You might as well run trials to see if raising children in locked, isolated cages from birth to age 18 has any negative effects. There's no way to know without large-scale, long-term, all-taker, randomized controlled trials!
Thank you though, for reminding me of this gem: https://www.smbc-comics.com/comic/bias .
There is a reason the behavioral sciences - psychology, epidemiology, and nutrition - are called the soft sciences. Any scientist that thinks one RCT can show causation proves that they belong in a "soft science." They do not truly understand that causation cannot be inferred simly by statistical power. Statistics by its nature is to show correlation. Plus/minus is the only way to infer true causation.
2
Any scientist that thinks one RCT can show causation proves that they belong in a "soft science."
@daTulip I don't know how many epidemiologists you know, but this is not at all the case. Epidemiologists know that most things are multi causal and that reaching a conclusion based on one study of any kind is a ridiculous notion. Epidemiology is not a behavioral science, BUT if you think you can solve issues of human disease without thinking about behavior you're foolish.
3
@Kernyl
I completely agree, but behavioral scientists love to state causation based on significant differences rather than true +/- data that is fully controlled. One can infer or suspect a causal relationship, especially over time, but certainty cannot be obtained this way.
The scientists commenting and the author seem to be missing this point here.
1
@daTulip
Epidemiology is not science any more than political science is a science Epidemiology is a branch of the study of "history".
I am a participant in this study and was very interested to read the initial results. A few clarifications are needed: 1) This wellness program was different in that it wasn't a benefit offered to all campus employees. We learned about it through email and many people simply overlooked it. But the fact that nearly 5,000 employees participated is quite amazing. (I think we have 10,000 to 12,000 faculty/staff.) 2) Most of the people who participated in the study are administrative or civil service staff, meaning they work full-time, 12 months of the year. As one commenter suggested, faculty have more flexible work schedules. They do, but they weren't the predominant group of participants. 3) This study occurred during a time when the state of Illinois had cut funding to higher education, and we hadn't had a pay increase of any kind for 2+ years. I was motivated to participate simply because of the financial incentives, including free health screening--I was in the highest incentive group, fortunately, but I also am someone who already is health-conscious. 4) The Illinois marathon is a community event and at least 80% of the nearly 20,000 participants are completing shorter distances: 5Ks and half-marathons. Many participants do one of the distances as their first-ever race, many end up walking. Yes, there's a competitive group aiming for Boston, but many local folks see it as a motivator to become more fit. So it was very creative for the researchers to use this data point.
11
Very interesting study and reported really well (unusual for NYT Health).
Important to see that after statistical correction - even with AI help sussing out artifactual correlations - an "observational" study would still show big effects based on volunteer bias.
What does that say about a million studies of the sort "those who ate the most fish lived 5% longer than those who ate the least"?
10
I can't help but read this in Aaron's voice after being a big Healthcare Triage fan for years. Good stuff. Thanks Aaron!
This study caused a lot of chatter six months ago when it was published but as people read beyond the headlines and blogger frenzy, most concluded that it was a beautifully designed randomized trial that unfortunately studied a wellness program design that few experts would expect to move the needle on risk reduction. It's an early indication that one particular design didn't engage the population much... This is not an indictment of all corporate wellness programs...
See Cleveland Clinic's response for meaningful perspective: https://www.cleveland.com/letters/index.ssf/2018/02/not_all_population_h...
6
Reading many of the comments, I'm disheartened by the poor understanding of randomized control trials by what is likely a well educated readership.
4
These wellness programs wouldn't be necessary if employers weren't so fixated on getting as much work as possible out of every employee to the detriment of their lives outside of work. If there were enough employees so that people didn't have to be available 24/7 on jobs that don't require that we could exercise after work, over the weekends, take real vacations, eat better, etc. Hire more people, pay them better, provide more vacation time, and stop forcing us to be available all the time and we might be healthier without "wellness" programs.
21
Here's a novel idea for employers about wellness programs- save yourselves and your employees the money and time spent on all those fancy degree'd biostatiticians and MBA's who are paid to essentially tell them what they want to hear from those *randomized* trials, and just plain offer your employees a reasonable discount to join any gym they prefer.
The level of greedy, arrogant chutzpah by these corporations of demanding these punitive financial arrangements where access to their employees personal health information is used as a weapon just shows, sadly, how much control and power we've allowed to be handed over to them, far too easily. Corporations building all these fancy high end on-site gyms and then demanding employee exclusivity without offering alternative choices, just smacks of Orwellian dystopianism. God help us all.
7
I get the feeling most people don't know how much their employers pay for their medical coverage. Many companies institute wellness programs because they get a break on the insurance premiums when a significant number of employees sign up; the employees who don't sign up pay the full cost of their share of medical premiums. The cost of health insurance in the USA is out of control. Insurance companies and the entire "health" industry is designed to make profit. Health "care" has nothing to do with it; it's all about the money.
7
The most important factor in any study of human beings is N, the number of people studied. Since the cost of the study varies significantly with N, most studies are small (under a thousand) and are thus often not reproducible.
In the particular reported study, the fact that gym visits per YEAR were under ten indicate almost universal non-participation by both the control and the intervention groups. Clearly, simply offering a wellness program does not make people actually participate in other than a token manner.
4
The difference between the studies is that the clinical trial shows no overall benefit for the company, while the observational studies show benefits for the individuals who choose to participate. Will companies use this data to end or curtailprograms?
2
Method can't replace theory, but can hide motivations.
Randomized test and control trials were invented in agricultural research in the late 19th century; double blind was invented thereafter. We don't need to revisit a methodology tried and proved for over 120 years.
The standard approach in the behavioral sciences is to use observation (ethology, demography, etc.) at the front end in order to put boundaries around an issue and to formulate the hypotheses for rigorous testing.
This is separate from outcome research, which is only a form of observation but typically has better measures and can sweep much larger samples.
The idea that controlled studies are unethical means -- compared to what alternative? Too expensive? Maybe, but do you want to know, or to guess?
Controlling by regression or path analysis is asking quite a lot of behavioral data, especially real world data. It can't cure the basic flaws of bias and validity and is almost never cross validated with split samples. Ask any "nutrition scientist" or drug researcher how well it works between the clinical trial and the actual population efficacy.
But sure: generations change, lessons are forgot, standards grow to seem arbitrary, costs must be controlled -- really, dad, do we have to do it the same way as grandpa did? Well, now you know.
3
I understand that this is about how statistics are interpreted however my question is were there different programs offered? It may be that the program itself wasn’t as motivational as it could be.
In MA, the State, through the Executive Office of Elder Affairs and The Executive Office of Health and Human Services, is pushing these short-term wellness programs for seniors. They refer to the workshops as "evidence based." But the evidence is based on self-perception questionnaires that attendees complete soon after completing the workshops. What amazes me is that based on this very "soft" evidence, public dollars as well as private foundation funds (Tufts Foundation) are supporting the workshops as a means to reduce health care costs. Federal funds through the Association of Community Living are also being used to expand these programs. The workshops appear to provide a relatively easy way for the State and Feds to show they are doing something, even if that something has no real evidence behind it.
1
Of course there's a difference in people because unlike diseases, physical activity and type of activity is a conscious choice. Some people are more likely to want to do it, enjoy it, or are motivated by other individual factors.
Some people don't associate these things with a workplace. Some people do just fine without having their employer help them along. Some don't want to do these things in the way that they are offered.
My own example may not be typical, but I will absolutely not share any health information that could fall into the hands of my employer. I simply don't trust them when they say it's all confidential. That information is between my doctor and me. (Do they control for plain orneriness?) I am, as many people are, fully aware of the benefits of wellness and the options. These programs are a good service for those who don't, or need this kind of motivation, but that is a particular individual quality.
I am skeptical about using a university as a population. University faculty have an atypically flexible schedule which can influence an exercise regimen. They also get a lot of time off. It's probably also a very different demographic from full-time staff. So it's a bifurcated population that you wouldn't find in a corporation, say. I know it says they controlled for that, but it still seems like not the best of test populations.
And BTW, if going to the gym 7 days *a year* is considered success, I'd say they need to rethink the whole program.
5
The Public Schools claim falsely, that lifetime earnings and presumably happiness increase in direct proportion to the amount of schooling that one has, Wrong!
These “studies” are based on pure selection biases. Kids who finish high school or college or grad schools are more disciplined, have better family structures, start off with more family money, are more motivated to succeed and above all, have higher IQs than do folks who don’t excel in the academic milieu.
I could as easily claim that more schooling makes kids dumber and less successful in life than they would have been had they walked away from the PS and colleges; prove me wrong.
3
@erwin haas"I could as easily claim that more schooling makes kids dumber and less successful in life than they would have been had they walked away from the PS and colleges; prove me wrong."
This is just attitude, not a well-considered conclusion. You can't substantiate that claim - the data doesn't bear it out. Education correlates directly with income and achievement. Whether that correlates with happiness is another matter, but it's not the fault of getting a good education. As for proving it, I don't have time to do your homework.
3
@erwin haas
I hope you don't supervise studies.
As stated, the public schools are exactly right. Earnings do increase as one's level of education increases (for the group, but not necessarily for any individual). Without the additional education, in a credential-mad society, most people will make less money.
Yes, the people who finish high school and college and even get advanced degrees are different from the people who don't. But that doesn't change the fact that people with more education make more money (on average) than those who make less. Correlation and causality are quite different.
Happiness is a completely different issue and probably has much less to do with education level than with other things. On the other hand, being desperately poor is probably not the easiest route to happiness and lifetime satisfaction in the US.
"I could as easily claim that more schooling makes kids dumber and less successful in life than they would have been had they walked away from the PS and colleges; prove me wrong."
This is just nonsense. We can't prove a negative.
1
It seems many of the concerns about selection bias could be alleviated by looking at the “wellness” of participants in the year before they participated and compare that with the year during which they participated. Did participation actually change anything?
Second, using cost as an inverse proxy for “wellness” is inherently flawed. Sick people who have an aversion to doctors are scored as “well” and healthy hypochondriacs fall in the unwell bucket.
“If your experiment needs a statistician, you need a better experiment.”
- Ernest Rutherford
13
@Tone. I completely disagree. If you don't have someone with a solid understanding of statistics, you'll likely end up with a very poor study. Any good statistician knows that simple methods combined with good experimental design is the better approach. But lots of non-statisticians don't fully grasp the limitations of their findings.
2
@Tone
Ironically, Rutherford's most famous and most significant experiment relied on statistics for its conclusions.
The university where I work outsourced the wellness program to a multinational corporation with a questionable reputation but with financial incentives for the university. In order to participate, we had to accept their privacy and data sharing policies which are simply unacceptable if one cares about privacy at all. To opt out, however, means a financial hit each month. The same university has also accepted a gigantic soft drink company sponsorship, which promotes the consumption of sugary beverages. All vending machines on campus must include this company's products. The hypocrisy is breathtaking. I decided to take the financial hit rather than participate.
22
Folks seem to (conveniently?) forget that the final step in the scientific method is replication.
7
Let us not forget that randomized trials are fraught with their own form of selection bias... in this case ‘deselection bias.’ The subjects must agree to be randomized, willing to take either the intervention or control. Far too often subjects asked about participating in a study agree, but only if they can get the experimental intervention. Thus, those who agree are themselves fundamentally different from the general population, even if they are similar to each other. Furthermore, some subjects might agree to randomization hoping for the chance of receiving the experimental treatment, and might not put forth any effort if assigned to the control arm (a risk in unblinded studies). Whose to say which set of results represents the ‘truth’ when both designs have limitations?
12
@Michael Thompson....Why would you choose to run an unblinded study?
@Michael Thompson
The head of a specialty clinic at a university in Seattle told me that I would probably never get into a drug trial. This doctor participates in many trials, but he told me that the corporations running the trials want people who have nothing wrong with them but the target illness and that illness should not be too severe. Therefore, the trial is explicitly not for the people who need the drug the most.
The first condition is more understandable than the second...unless the goal is simply to get the drug approved regardless of whether it will help the severely ill.
When drugs are advertised on TV, there is often the statement "for moderate to severe" (arthritis, IBD, etc.) But. if that doctor is correct, the chances are the drug was never tested on the severely afflicted. I guess we discover that by paying thousands of dollars and testing it ourselves.
2
@Michael Thompson. I'd rather have a valid causal inference based on a subset of the population than bad science based on observational analysis of the full population. Yes, the soft spot of RCT's is the question of whether the findings can be applied broadly (external validity). But the ability isolate if the treatment led to effect is far, far more important.
I've followed the "disease management" aka wellness companies for many years and find that most of the data used to sell these plans to employers is adjusted in the same way drug makers alter study populations to get the results they want - as opposed to what actually happens.
These programs sound great to employers and employees - but all too often it's a waste of money for both.The timeframe for the studies is too brief and employees tend to self-select into these programs...and in the case of the Illinois program - it seems like wellness is too broadly defined.
7
The problem with saying that some observational studies stand the test of time is that we don't know which ones in advance. So we end up adding to the scientific record a lot of arbitrary results that get translated into personal action or policy recommendation all the time (omega 3 fatty acids just to name one). Observational studies that don't convincingly address the issue of confounders should be outright rejected. It's draconian, but I don't see another way.
It's time to stop conflating fitness and wellness and health. All the exercise or yoga in the world isn't going to prevent or cure asthma or epilepsy or hypothyroidism or PCOS. Having seen the lives of many people who have lived into their nineties, I can say that the quality of life for most was not all that great. So I think it may also be worthwhile to lessen our efforts to prolong life.
9
@Libby
Possibly, the worst advancement that science can make would be to significantly extend lifespans without improving the health of those who live longer. With more or more diseases being manageable, relatively unhealthy people can live longer.
The other question, which I haven't seen addressed by those working to extend human longevity is "Who pays for people who have 50 year retirements." Many people struggle already to afford 15-20 years in retirement.
4
@Libby You can't cherry pick your diseases. Fitness-related diseases (heart disease, diabetes, etc.) take up the vast majority of healthcare spending, time taken off work, and death in this country.
2
The fact that 60% of all Americans are overweight or obese, and that figure keeps growing, shows that unless one makes people pay for their healthcare according to points as to numbers, there won't be a reduction in costs or even a better health outcome. I know quite a few nurses, and most of them are obese, themselves. It is also a fact that since women starting working full time over 40 years ago, their health has suffered. They tend to eat more junk food, don't have the time to cook healthy meals at night, nor do their husbands. There are more men who are under the age of 40 cooking, than there are women the age. When society doesn't need to manually do lots of tasks like, open the garage, turn on the television, hang out the laundry, use a push mower, garden or rake the leaves, people's health suffers. Besides, most health habits are set by the time a child goes to school, and they become ingrained in their behavior. That is why they need several different options for the school lunch line, as kids today are such picky eaters, some only like pizza, etc. not a variety of foods. Everything that works is daily lifestyle changes if you didn't start out healthy. People don't want to do the exact opposite of what they have been doing for the last 40 or 50 years. People drink more than ever to relax from the stresses of work, and the society we live in. There are 20 million alcoholics, 20 million drug addicts, 45 million smokers, and most are teenagers and adults.
2
Our 25 years of research into the health attitudes of Americans shows that “differences in the people,” as this article points out, distinguishes participants from nonparticipants in corporate wellness programs. The first of the three attitudinal segmentations we have created based on over 25,000 randomly selected individuals shows that our Proactives segment, 38% of the total 18 and older population, are 48 percent of those participating in such programs. Conversely, our Disillusioned segment, 18 percent of the total population, represents just 9 percent of wellness program participants. In studying specific dimensions based on our research we have found that at a significant level participants in wellness programs say they are more willing to practice a healthy lifestyle compared to those not participating. In addition, participants reveal they have a higher trust in doctors, feel they are more self-determined, and say they are more receptive to getting periodic checkups than those not in wellness programs. Finally, those in wellness programs are less likely than nonparticipants to believe they are concerned over the cost of health care. These attitudinal differences demonstrate that "participants differ from nonparticipants in very important ways."
4
@Chiquita42: i used to do segmentation myself, and i'm amused at the way we name segments like characters in opera buffa. but to you point -- so what? the key in any study is fitting the sample to the question. do you want to increase the number of participants in wellness programs? then the disillusioned matter. do you want to find the most effective activities in wellness programs? then the disillusioned don't matter -- unless they're more likely to drop out. either you're designing a wellness program or a marketing campaign for your wellness program or retention measures for your wellness program: choose one. the comments about participation are intended to raise an ethical problem, but i for one would gladly see experimental treatments given to those who want them with fully informed consent, and observe the results. (many "off label" drug uses have been discovered as unexpected drug reactions, that is, reactions inflicted without consent.) it's important to remember that ethical issues shed no light on facts. we can choose to be ignorant for ethical reasons, but it's ignorance all the same.
1
We have a wellness program whereby you earn a $20/month break on your insurance premium by earning points for various activities designed to promote health or financial responsibility.
Some examples? Eating a home-cooked meal once a day for a week, tracking your spending for a week, taking so many steps in a week. It's so easy to game the system that it seems pointless to even offer it.
Yes, they require a health assessment, but it's self-reported with simple questions like, how much red meat do you eat in a week? Again, super easy to game. In three weeks, I've already earned my incentive for the next year.
4
An oft-quoted cancer study combined stats from red meat with “processed meat.” Most “processed meat” is also referred to as “sandwich meat.” That is, folks who eat a lot of processed meat are eating mostly bread.
Glucose, for example from bread, is the prime fuel of glucose-greedy cancer cells. I pointed this out to the authors of the study that conflated red meat and processed meat and asked for their comment. They did not reply.
2
When a question is not even wrong, it's impossible to answer.
Bacon is a processed meat; it's often eaten with eggs. I put smoked sausage in my red beans and rice. No bread there either.
And where did we get the idea that bread is a principal source of glucose in the diet? Not even Gary Taubes, the Elon Musk of nutrition - or was that Steve Jobs? - would beg such a question.
2
@Lorem Ipsum
Bread, pasta, and cereal are major sources of glucose in the average diet.
It would seem that the takeaway for employers in this study is not that wellness programs are worthless, but that their benefits are limited to individual/anecdotal cases. Don't implement a wellness program to save medical costs across the organization, as clearly the data don't support it. But if it contributes to workplace satisfaction or retention for the people who really embrace the program, then perhaps that still counts for something.
My employer offers yoga classes at lunchtime, for example, and the minority of us who attend consider it one of the best things about working here. There are other incentives that motivate other employees in other ways. Employers need data like this study to set realistic expectations of what the real benefits are, but I hope wellness programs and the like don't disappear.
13
I participated in a Workplace Wellness program. Lost 10 pounds. One program focused on eating 5 fruits and vegetables per day. That was great. Basically, knocked cheese and meat out of diet. July program focused on doing a plank for five minutes. I started with 15 seconds, have increased this to 60 seconds. Still working. Wellness program puts health front and center. I love it.
11
With the exception of those comments of distrust for healthcare entities violating HIPAA to share PHI with employers, I share the skepticism of many of the other contributors, as well as that of the author. There is a longitudinal hurdle to overcome with regards to measuring outcomes and those measurements probably can’t include an assessment of medical cost savings. Simple example: my employer has a results-based wellness program. The confounding of results to medical costs simply by the number of transplants in any one year negates any measurable medical cost savings. Longitudinally, two outcomes that could be helpful to measure the success of a wellness program are trends in employee disability and turnover events, including the employer’s transplant population.
1
As a retired Professor of Health Psychology who taught research methods for over 30 years , it is misleading to say "despite their flaws" in describing randomized controlled trials (RCTs). RCTS are the gold standard of inferring causality that eliminate what are often called "threats to internal validity" that plague (that is, bias) non-randomized studies such observational studies that are notoriously subject to "selection bias." The "flaws" with RCTs and all studies concerns the generalizability of their findings to other populations, locations or times (so-called "external validity"). There is no qualifier like "perhaps" in describing the power of causal inference of RCTs--they are "the best method."
8
The purpose of the wellness programs is to attract talent that will take advantage of such programs. These individuals are more proactive, healthier, and less costly for the company in the long term. Thats why they offer it.
5
Some commenters seemed to have missed the point of the article. Ultimately, the researchers conduct the research to address a real-world question: employers want to know if the implementation of a wellness program in the workplace will cause certain benefits (e.g., reduced employee medical spending). By the nature of the question, only a causal mechanism, and not correlation, is meaningful. The problem is that in the social sciences, causality is impossible to prove and can only be inferred.
In an observational study, researchers try to approximate causality by controlling for confounding variables. In doing so, they necessarily assume that the variables they do not control for are not meaningful. If this assumption is correct, then causality can be inferred. However, it is impossible to prove the assumption is correct.
Nevertheless, it is possible to show that the assumption is incorrect. Here is where the randomized control trial comes in. The randomized control trial basically shows that one variable that the observational study does not control for -- employee's motivation for wellness -- is indeed meaningful. Hence, the assumption of the observational study is incorrect, and consequently, causality cannot be inferred.
The correlation between the wellness program and employee benefits still exists, but the RCT trial shows that this correlation is not causation. To employers, the answer to their question is no.
10
@Jason - well put.
1
@Jason
"The randomized control trial basically shows that one variable that the observational study does not control for -- employee's motivation for wellness -- is indeed meaningful."
This is not quite right. The RCT does not explicitly identify the possible confounder (here employee motivation for wellness) as "meaningful," it just controls for it, in expectation, by ensuring that motivated employees exist in roughly equal proportions in both treatment and control groups, independent of participating in the wellness program. Any observed difference in outcomes between these groups could not, therefore, be attributed to employee motivation. Rather, any observed difference in outcomes is attributable to the one thing differing between these groups; here, participation in the wellness program. Of course, for this study, there was no observed difference, which undermines the idea that wellness programs cause better health outcomes.
However, the RCT did not "prove" nor "infer" that this potential confounder was indeed what was doing the work here (although it's plausible), because employee motivation wasn't randomly assigned by the experiment. Indeed, there might be several other things that confound employee motivation, perhaps genetics or other environmental factors etc. A given RCT tries to answer one particular question fairly precisely, but it only speaks to what has been randomly assigned by the experimenter, not the causal effects of other background factors.
1
This article raises an obvious question: Why do companies continue to promote employee programming (health/wellness, diversity, team building, etc., etc.) with no empirical evidence that they work - or even when there's evidence they don't work? In the case of health/wellness, insurers provide discounted rates for participating employees and employers. But then why do insurers discount for something that doesn't necessarily work? Because employees who do not participate pay a higher cost for their insurance. So, instead of raising prices outright, they calculate increased revenue from those who do not participate for any number of reasons (a side door rate increase that looks benevolent). The discount the participators get is only real to the extent that you consider the absence of an increase to be a discount. Many other trainings and programs are CYA efforts by companies to avoid culpability (e.g., sexual harassment) if something goes wrong in the future. So asking whether a program works depends on what the meaning of "works" is and to whom.
16
Workplace wellness programs are incentivized by Obamacare. They were unproven then and have not lived up to expectations. But as long as the incentive exists , they will not go away.
2
@G
Your personal decision to focus on the ACA results in your missing a much larger and longer picture: https://selecthealth.org/wellness-resources/-/media/fbe0d492b06e4935b3fb... . Like other elements of health care, we've been trying to figure out better approaches for a long time.
Some of the limits of randomized controlled trials have been recognized for a while: "Randomized controlled trials vs. observational studies: why not just live together?" ( https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5073487/ ).
Science depends on ignorance and failure (not the political kind), on negative studies, on new questions. Why should wellness programs be any different?
2
A note on terminology: What Mr. Carroll calls "selection bias" is better called "confounding bias". The latter involves the presence of one or more *common causes* of the treatment variable and the outcome variable. There is another kind of bias that is called selection bias or "collider bias" that involves a *common effect* of the treatment and the outcome (or of causes of the treatment and outcome). Randomized treatment assignment provides protection against confounding bias, but not against selection/collider bias.
2
Wellness programs have never been about anything other than increasing profit for a company and inventing a business for the wellness coaches. The questions asked of participants were extremely invasive. Enough that most employees at my job elected to pay more rather than participate. The wellness program was terminated.
17
Randomized Control Trials (RCT) do NOT show causality. They show correlation at a high level. They are better than observational trials, but one cannot conclude causation from one or even many RCTs.
The author shows a major problem in the soft sciences, which includes epidemiology. These are not fully controlled experiments. This cannot be done in human populations due first to ethics, followed by the enormous complexity of organismal design study.
This is why health and wellness studies always show conflicting results - eat this, no do not eat this, etc.
7
@daTulip This comment shows a limited understanding of what RCTs can and cannot do and how science works. To say RCTs "show correlation at a high level" is vague and essentially meaningless. Similarly, it is a mistake to say "these are not fully controlled studies."
As the author writes, random assignment is used to deal with problems of confounders that befall observation or uncontrolled studies. In expectation, randomization does indeed control for things we can measure and things we cannot. What, pray tell, is a fully controlled experiment by your estimation and how do you know what to control for?
It's certainly true that RCTs require assumptions to infer causality, but fewer of them than observational studies. Further, most of these assumptions can be empirically tested. There's a reason why RCTs are used to test the efficacy---i.e., the causal impact--- of new medicines. Does aspirin cause relief headaches? Yes. How do we know that this is more than just a placebo effect? RCTs.
Can one not "conclude causation from many RCTs," even if we have consistent and significant estimated treatment effects? At some point it should be upon you to deal with this burden of proof instead of the other way around.
It's important to develop a sense for what constitutes solid evidence but your skepticism is profoundly misguided. It's the epistemological antecedent to putting naturopathic crystals and modern medicine on equal footing.
5
@mk416
This is not true. I am in the molecular world as a scientist, and a social science RCT type design cannot confirm causality. Only if all variable are controlled. Creating randomized groups cannot control for many factors that may not be accounted for. Social sciences do not equate to hard science, where every variable is controlled in an environment (an in vitro reaction in identical environments with only one variable, e.g. salt concentration) can show causality. An RCT has people out in the world with different backgrounds, genetics, and other factors that prevent a conclusion of causality.
See Jason's post above. He does a great job explaining as well. RCTs can prove a strong correlation if repeated different ways many times. Otherwise I take them with a grain of salt.
@daTulip
"Creating randomized groups cannot control for many factors that may not be accounted for."
That statement is false. A sufficiently large randomized design will balance ALL characteristics among the treatment groups, including unmeasured characteristics. This is a mathematical consequence of randomizing large numbers of people. Assuming procedures to promote internal validity (blinding, concealment, intention-to-treat analysis) observed differences among the treatment groups in a large trial are due to the treatment itself. No correlation here.
You may be referring to the potential mechanisms by which the treatment causes the outcome under study. Trials are typically ill suited to ascertain mechanisms, which must be discovered from other studies, including molecular studies. You may also be referring to very small trials, which are more likely to be impacted by chance.
1
My employer has a "wellness" program that wants me to engage in certain activities and displays some info on doctor visits (thankfully not details from the doctor visits!) including whether or not I have had a colonoscopy it mammogram.
I think this is a huge invasion of privacy, however well intentioned.
I get several emails a week promoting the "wellness" program, too.
I have toyed with the idea of self-reporting fictious activity to get this program to stop pestering me and I suspect there are coworkers who have - which does mean the statistics being reported overall can be very different from the truth.
I have absolutely no interest in participating in sharing my personal activity with any agent of my employer. I don't understand how my concerns can be so ignored by the people trying to determine "value" in this effort.
42
@Margo absolutely, The programme you describe more resembles an optimisation of health coverage incentivised by your employer's health plan. And once they've got the results it's out of your hands. Colonoscopy? Who's to say they wouldn't also sneak a sample for DNA profiling, then alter your coverage accordingly?
So happy I don't have to deal with any of that...
2
I think the focus here is the power of the randomized controlled trial, which is very important. But the takeaway that wellness programs are ineffective is flawed. My company has a wellness program and gym on site and that’s a big attraction for me. If having a wellness program attracts more health-focused employees, then it has achieved its goal. Perhaps we can have another study comparing companies with and without wellness programs and compare the overall health of the employees.
10
Here's a thought: LEAVE PEOPLE ALONE.
Their personal health, exercise and diet are their own damn business.
Do they do this stuff in countries with UNIVERSAL HEALTH CARE, where corporations are not always looking frantically for ways to cut employee benefits?
77
@Concerned Citizen
" .. Do they do this stuff in countries with UNIVERSAL HEALTH CARE, where corporations are not always looking frantically for ways to cut employee benefits?"
Yes. They have workers whose jobs are to lecture patients, about their problems, such as over-eating, alcoholism, smoking, and drugs like cocaine and heroin.
1
@Concerned Citizen
Funny, by supporting Trump you voted to take away health care from tens of millions of Americans, not to mention bringing the federal government into private reproductive decisions, and making it much, much easier for employers to nickel-and-dime their employees.
Don't pretend to care about privacy, health care, or the benefits of employees.
Thanks for a very clear explanation that will really help the general public understand how complicated these questions are and the importance of rigorous research in sorting out the answers.
8
The bias I have going into this article about the difficult problem of studying people comes from my observations of almost 70 years. There are folks who, given a few moments of free time, invent something to amuse and occupy themselves, frequently with some sort of physical activity. That may range from chasing a brother or sister to later in life waiting a few hours after TSA check until boarding, and choosing to get up and walk around rather than sit still reading a newspaper.
The somewhat glib example of looking to see who participated in a 'race' such as running a marathon or 10k or fundraising event was given as a possible measure. Having run a marathon and several races earlier in my life, I can point out that this is not an easy choice. The whole family, friends and yes even workplace setting enables someone to structure the necessary schedule to accomplish this event. To use this as a measure speaks against any random assignment to a class of people. Those who will run a race are far different from those who won't. To differentiate those groups and then think that a study can offer sedentary vs. active as a choice is ludicrous .
Not even mentioned in detail are the other shortcomings, such as the brief length of trial vs. overall health benefits later in life. For example aortic aneurysm risk. 100 or more cigarettes at anytime in your life is a predictor. Such examples show the shortcomings of this analysis.
8
Randomized controlled (RC) trials may be useful for making broad generalizations about workplace wellness programs. For example, such trials aim to determine if there are statistically significant, measurable benefits across diverse programs ON AVERAGE. However, it is not a practical or efficient approach for determining what elements of programs are the "active ingredients" or why some programs are successful and others aren't. The author is correct that RC trials are better at determining measurable effectiveness on average. But rigorous collection and analyses of observational data are much better at generating deep understandings of such programs and determining how an effective program might be designed. It is the proverbial difference between a sledgehammer and a scalpel.
8
The Failure of Evidence Based Medicine
Randomized controlled studies offer the most useful broad brush method to investigate a hypothesis and figure out “what’s true” at least for a little while. Government agencies, pharmacy benefit managers, published clinical guidelines, and young doctors cite them as the truth.
The problem is the problem of applying statistical truths to individual patients. Time and again, I find that individuals don’t conform to the statistically based truth.
20
@John Lipkin M.D.
The fallacy of treating individuals as if they were the mean is indeed a problem. But that problem is not a feature of RCT studies. Unfortunately, it is common in medicine to use the latter to do the former nonsense. It's understandable that practicing doctors would make the incorrect association.
1
I responded to the earlier article. I did my first medical study when 13-15 using rabbits to study induced impacts on the liver. At Yale, I did six studies involving breast cancer using records from 600 hospitals. Every study starts with a pi variable and level of confidence (I note that many studies on nih.gov ignore a study of the study). Other variables are involved so each study group has to be strictly scrutinized. I would not omit outliers since they are unique and later, personal study can lead to breakthroughs.
Very few serious scientists believe that randomly controlled trials are a thing of the past. The problem comes when we rely on them exclusively to make decisions about outcomes and future research. When the outcome, or dependent variable for the quantitative set, can be adequately operationalized (measured), then it can work. When the outcome cannot be measured, it presents problems. For instance, persistent pain, my field of study, cannot be reliably measured. The rating scales one sees everywhere are simply quantifications of self-reports. Other less ubiquitous scales and surveys produce actuarial means that apply to everyone, and no one in particular. In the business world, profit is the ultimate outcome. In person-centered health care, the unique aspects of what a person says is important—and cannot always be operationalized if one wants a more complete understanding. Getting to this more complete understanding, called qualitative research, is a necessary aspect of the goals of science. It is both, not just one or the other.
Matt Vittucci
26
Please, let's get this straight --
For studies of real-world cause and effect, there are FOUR requirements:
This writer points out two of them:
-- randomized
-- controlled.
But he forgot the two other requirements:
-- prospective
-- double-blind.
The gold standard requires these four attributes for any study.
In many cases, the study that one might want to conduct in alignment with these four absolute requirements would be immoral or totally impracticel, so such studies are never conducted. And in those cases, cause-and-effect information is forever beyond our reach.
Example: suppose you want to know the effects of exercise on health. To meet the four absolute, incontrivertible cause-and-effect requirements, you need to:
1. randomize 10,000 subjects for a prospective study ("prospective", i.e. "from now going forward", not retrospective); and 10,000 is for statistical reliability
2. force 5,000 to never exercise, and force 5,000 to exercise many times a week ("controlled")
3. prevent each group from knowing whether they are exercising or not!! ("double-blind")
4. placebo-controlled (have 5,000 control subjects do some sort of sham exercise instead of real exercise).
Good luck trying to do #3 and #4. Tough, but that is what the scientific method requires. Anything short of all four requirements for cause-and-effect information is NOT SCIENCE, not reliable, as Mr. Carroll has clearly stated in his othe writings.
25
@SuPa
Science is the majority opinion agreement ("peer review") of the declared results of methodical observation of the physical world.
Failure to reach the "gold standard" you describe simply introduces uncertainty, but uncertainty clearly is part of science. Perhaps mankind is simply not yet aware of any divination techniques allowing the reliable exploration of the multiverse (time travel, Schroedinger cat heart monitor, etc)? Conclusions are thus not iron clad because the real world is messy and doesn't always fit neatly into perfect methodology, but I assure you, science it remains...
Your definition of science more resembles the "angels dancing on the head of a pin" dogmatic thought experiment, which predated and at one time was thought to be superior to observation of the physical world. The rigid dogma born of a theological examination of the world you describe also coexists with science to this day but most certainly can't be qualified as science.
3
@SuPa I would certainly say that there are a lot of methodologies in terms of quasi-experimental studies where the rigor is less than the gold standard, and that is clearly understood in discussing the results. However, it does allow researchers to make some statements though with more caveats rather than saying nothing at all. Frequently public health intervention programs are evaluated in quasi-experimental studies and allow us to do better than just "throwing money down a rat-hole", to use the coarsest terms.
5
@Andy
Good science has nothing to do with majority opinion. Majorities of scientists have been fundamentally wrong in the past and will be wrong in the future.
Science should encourage critical analysis of data to reach (or not) a rational hypothesis, and should always challenge whatever is the "accepted" dogma.
I'd gladly trade our wellness program for the chance to get to a swimming pool when it's open.
57
Practicality. Yes, swimming in a pool is enjoyable and helpful. Awesome answer
1
Here the randomised controlled trial is advocated to rule out the selection bias where health conscious people may self select the program. Being a better lot, their response variable would be different from the control group. My question is do we need to rule the unmeasurable factors? After controlling for measurable variables such as age, income, drinking/smoking habit etc., if we have left out the factors such as sense of responsibility, punctuality, conscientiousness etc., they should be bunched with the usefulness of the program. The success of any intervention depends not only on the technical aspect of the program but also on certain unmeasurable human aspects of the users. So if proficient people are shown to benefit from the program in an observational study, then that proficiency is integral part of the usefulness of the program and should not be eliminated by randomisation.
6
@CA. That's the beauty of randomization - it allows you to isolate if the treatment caused the effect. Any unmeasured differences between the treatment and control group are random. Make the trial large enough and differences between the treated and untreated group disappear, and the only difference is that one group received the treatment and the other did not.
1
Experimental designs that allow robust inferences about mechanisms are well known, but not as thoroughly taught as they should be. This article makes it sound as if there are a couple of choices, but doesn't really dig into the qualities of robust designs. Scientific training, whether in natural or social sciences, must have a lot.
The challenge the author is really addressing is what one does when the rigorous design is unethical because it harms or kills some of the subjects. This common feature of medical research unfortunately leads to sloppy conclusions (I'd even call them non-science) by some.
2
As a social scientist, the take home message i saw in this report, was very different . I thought that the reported RCT speaks for the problem of RCTs and not the correlational studies or the examined wellness programs. This study shows that RCTs are failing because selection into an intervention group is also an important component in most interventions that is missed during random group assignment. Randomizing people into groups sounds like (it is the gold standard) a great method to allow for drawing causal inferences from the results , yet this practice is artificial, rarely represent how people make choices in life and therefore undermine the validity of the findings....
3
@AO True - randomized studies allow one to make a clear statement that, in a perfect world where everything is implemented according to plan, does this strategy work? Other studies that actually look at and measure the degree of impact self-selection has help us understand the real-life impact when people are left to their own devices to either do something or not.
2
Here's what would improve my health (lower mental stress translating to better eating habits and less alcohol): Create a workplace where the work process for collaboration and teamwork is actually implemented, everyone has the right tools to collaborate, and everyone follows the same process. Instead, employers create "teams" and give lip service to collaboration but in reality, it's unpredictable chaos, uncertainty, and wondering if anyone else on the "team" is actually following the same procedures. Managing people and processes is a tough job and very few people are actually talented at it. For me, anyway, chaotic procedures causes me all sort of stress and physical distress, spending more time and energy trying to figure out how to get something done than actually doing my job.
52
The success of the outcome depends on both the motivation of the participant as well as the type of wellness program. I was a chain smoker, 5 packs a day, when my employer offered a no cost smoking cessation program. I signed up and followed all instructions/behavior modifications during rthe 8 week class. In the end, I stopped smoking forever while those less motivated did not. In my case, I no longer suffered from twice a year upper resipatory infectiions, meaning that I stopped taking the 6 to 7 sick days off from work to recover. Since I remained an employee of this company for many years, I would say the investment they made in the class for me paid off well for them. It did not pay off for those less motivated.
29
Great article Mr. Carrroll. The study of the scientific methodology is perhaps more important than the wellness program findings. I was particularly impressed by "they used machine learning to decide whether to control for even more variables, including (but not limited to) past health, smoking and drinking status; pre-intervention exercise; medication use; and sick days taken."
It would be convincing to hear that observational researchers tested these differences -- after all, don't these differences represent assumed differences between volunteers for a health study and those who decline to participate? But obviously that type of argument is not sufficient...you can't simulate an entire population by subdividing a portion of it. Good stuff!
4
I can give 2 reasons why these programs don't work.
The 1st is the well-known phenomena called the "healthy user" effect. Observational studies have regularly found a benefit that disappeared with a randomized study, with hormone replacement therapy being perhaps the best known. See https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3077477/
The 2nd is the nature of the wellness programs. In an ideal world employees want to make changes and are grateful for a win-win opportunity in which their employer saves money and they get a cut of the savings. In the real world people respond to incentives in ways that suit them. Someone already going to a gym is going to sign up for programs that offer subsidized gym membership. I know people where employers offered $500 or more in incentives to complete online training programs on healthy eating, ergonomics, lifestyle balance, etc. For $500 people were happy to do them, and they did them at work to boot! It was almost like a mini vacation. Then they changed nothing.
14
When my employed did the wellness bit I refused to participate. If I had participated, they would have had documentation of my health issues which would have made my job less secure. The money offered wasn't worth the risk. I already saw my doctor and worked to address my issues so there was little to be gained anyway.
61
@Mary Nebraska's state employee program is considered the worst in the industry. You didn't say whether you were a state employee, but even if you aren't, you probably know some. Ask 'em.
Meanwhile, here is a primer on it. This is a bit "inside baseball-ish," but I'm sure you'll find nuggets of great interest. It is the perfect storm of dishonesty, clueless, and harms to employees.
https://theysaidwhat.net/2016/09/21/the-latest-on-nebraska-ron-goetzel-c...
6
The Illinois study was every bit as flawed as the observational studies. The problem with observational studies is selection bias- people in the wellness group self selected, as did the people who declined to participate. Improvements in outcome could simply be due to having more motivated, proactive, people in the wellness group rather than any actual benefit of the wellness program. The Illinois study was no different. Only a third of the employees chose to participate. Presumably these employees were already the most motivated, and providing a wellness program didn't change that. At best, this shows that a wellness program doesn't change outcome among the most motivated employees. This isn't the same as asking whether a wellness program with generous benefits and a high participation rate reduces overall illness and lowers healthcare costs. It should be no surprise that a stingy program with little benefit and low participation does little to improve health or trim cost. However, it's still an open question as to whether a generous program with a high participation rate might do much better.
7
This entire industry is a scam. "Prying, poking and prodding" employees costs money and harms exceed benefits
https://scholarlycommons.law.case.edu/healthmatrix/vol27/iss1/3/
There is also a $3-million reward for anyone who can show that it isn't a total waste of money.
https://theysaidwhat.net/2017/12/09/does-wellness-work-lets-boost-the-2-...
15
Of course they don’t work! Somebody in HR is sold a wellness program that is “efficient.” The problem with efficient programs is that they simply aren’t fun. Yeah, the boot camp works in the military where you don’t have a choice but in the workplace where you do, that’s a whole different story.
Then there is that fact that most workplaces are, by design, toxic. Long hours, longer commuted, sitting-sitting-sitting in boring meetings, high stress leading to high stress hormone levels, and junk food galore (try serving healthy food at a day-long meeting session- good luck with that.)
Activity that you like to do will cause you to make time for it. Activities that are boring and painful and I’ll find excuses galore. Passing along the stress is one of the major portions of any workday – even beyond the workday in some companies where 2:00 AM e-mails must be answered or else.
The people who run the companies have to make a non-toxic workplace, but they won’t do that and just blame the employees.
38
I believe it was a workplace 'wellness program' for underpaid teachers that was the straw that broke the camel's back to push the teachers to a statewide strike....
15
What kind of wellness programs are they if they change the biochemistry of the participants, promote homeostasis, boost the immune system and reduce frequency of sickness? And what kind of program is that that does not motivate the participants by positive results in the first few sessions? I am afraid everything cannot be well with those wellness programs. If for example I promise the participants that their weight will register at least a 10 percent reduction at the end of the program, it is not rocket science to check if such a program is a success or otherwise: Did the body weight go down or not? Simpler and more objective the evaluations of clinical studies are, the more accurate would be the outcomes. We need independent evaluation systems well-versed in alternative health care systems to look into clinical studies and not only traditional medical evaluations with a silo mindset towards alternative wellness systems.
I don't see the flaw. Doesn't this show that people who choose to participate in a wellness program have better results compared to a random group composed of both?
It seems apparent to me one would not expect the control group to do as well as the observational group. Of course the control group is needed to be sure.
1
@Bob You are misunderstanding the groups.
People were put randomly assigned into two groups - a control group that was not offered any wellness programs and test group that was offered.
Within the test group, there were those who chose to participate and those who did not.
Those within the test group who chose to participate had better outcomes than those who did not chose to participate. However, the test group as a whole, did not have better outcomes than the control group as a whole.
This shows that the people within the test group who were likely to participate, already have something about them that makes them healthier - and there were plenty of people like that in the control group too that never participated in any health program. So healthier people to begin with tend to self-select into these programs.
17
@NH Thanks for pointing out the flawed intent-to-treat analysis even if the people were correctly randomized
You should really participate in the wellness program. . . . No, I don’t think you’re fat.
18
Even in the observational study, "success" was going to the gym 7.8 times per *year*? Wow.
I also wonder what exactly those health spending numbers are. I haven't needed healthcare for anything other than recommended preventative care in a decade, but I guarantee you my insurance still spends far more than $500 for that. Are these people not even getting normal checkups and screening?
4
@Polly I've been to the doctor 5 times since I left home for college in 1991. Not everyone thinks checkups or screenings are necessary / useful
16
@Sharon: even most doctors don't think annual checkups are necessary!
@Polly: I am a study participant and work at the university. The study looked at visits to the campus recreation centers. That's what data they could easily access, because we must use our university IDs to enter. Many employees belong to other gyms in local communities or are active in ways that can't be traced by using a campus ID for access. So this measure tells us that similar numbers of employees in the control and experimental groups already have memberships for the campus gyms. (We have to pay for this, it's not included as an employee benefit.)
Most of these attributes are highly biased, such studies work well in the physical sciences, not so good where humans are involved.
5
Reminds me of the study a few years ago that found the Mediterranean diet to be healthy, and was sponsored by the olive oil industry.
9
@Stephen Spang: and the sugar industry "found out" that sugary drinks and kids cereals were harmless.
The risk FOR ALL STUDIES is inherent bias, and when studies are sponsored by industries or business....the risk is increased.
2
This is an important article that reminds us of the power of randomized controlled studies and to always critically evaluate the methods of any study, especially observational studies that are very prone to bias. We should be especially weary of large observational cohort studies that have lots of study power to detect small differences between exposed / non-exposed groups and which often falsely claim to “adjust for confounders” using multivariate analysis, etc. These studies can still have lots of residual confounding. Possible solutions to these problems include: 1) making sure the NIH fund not one but multiple comparable cohort studies and 2) require confirming results from multiple comparable cohort studies before they are published. This approach has been used with genome-wide association studies which examine
and confirm gene-disease associations in several different populations before results are published. This minimizes the problem of false findings and may better directs future research funding to avoid the problem of sending researchers chasing down blind alleys for years or decades. Unfortunately, our tenure system rewards original research efforts and not replication of other study findings, so there is a disincentive to try and replicate findings. Although they have be carefully interpreted, observational studies are powerful and with some system changes they can be made even more so.
27
@MrStilton Results should always be replicated but this piece points out that there are serious methodological problems, especially selection, that may recur in all observational studies. Duplicating erroneous studies would be a waste of time and money.
I kind of liked it when the conclusion said what I wanted it to prove- that wellness programs are great, and should be expanded! It's a lesson on how difficult it is to interpret results correctly as a layperson reliant on analysis as usually reported.
But even the apparent improved results found in the observational study were less than convincing regarding the efficacy of the program in changing behavior. That was surprising
@cheryl Only surprising to those that don't understand and accept our population. Wellness programs improve somewhat those that like exercise, they do little to nothing to impact those will poor habits. That is human nature in this country.
5
Anything offered for "free" from an employer will turn out to have some kind of cost. Don't take it. "Wellness?" They certainly mean to reduce medical coverage by replacing it with "optional" activities. If they really cared about wellness, there would be regular and predictable work hours, paid vacation and sick time with coverage for absent workers and gym memberships provided. AND employers would pay a living wage so that workers could afford to eat healthily. ALL of these benefits are frequently provided by unions, by the way.
133
@Ellen Tabor: what you describe are the conditions for unskilled workers -- in fast food, retail clerks, etc.
People who are skilled white collar and work in nice big office parks, DO get health insurance -- paid vacation -- sick leave and yes, regular 8 hour days and 40 hour weeks. And virtually NO such workers are unionized.
Unions in business were almost all only BLUE COLLAR workers. Anyways, unions have declined to the point that virtually the only workers in them today are PUBLIC UNION employees, and not most workers outside of that. Only 6% of Americans are in a union today.
@Ellen Tabor
" .. ALL of these benefits are frequently provided by unions, by the way .."
Ever heard of "customers?"
I refused to participate in my company's "wellness" program, because it would give them access to information and data I consider deeply personal (and irrelevant to the quality of my work). Managers were given aggregate stats on said information (e.g., x% of your team is hypertensive). Are you kidding me???
For declining, I was punished with a "voluntary" $25 pay cut each pay period. This is coercive, especially for lower-income workers, and it was one reason I chose to leave the company.
This employer made it a genuine choice only for those who could absorb a hit to the wallet. It's morally indefensible.
166
@KYFilly I don't even see how this is legal, and if it is, it's yet another reason why medical coverage should be separated from employment in every way. I'm so sorry. You got screwed. (I like to think that in unionized work places, this sort of thing doesn't happen...)
13
@KYFilly
I agree, it's very coercive. There are survey questions that ask about workplace stress and dissatisfaction - is anyone convinced that the employer won't/can't see the responses?
34
@KYFilly It is also economically indefensible. See www.theysaidwhat.net -- a website devoted to exposing the scam that is workplace wellness. The only good news is that these wellness vendors (excluding a few that are honest and don't claim phony savings and screen according to guidelines, like www.itstartswithme.com, can't even lie effectively, leading to a great deal of hilarity on that website.
If laughter were the best medicine, the wellness industry would be a boon to society.
11
American culture does not bode well for health and wellness. Most Americans (any age) are either too lazy to work out, or can't get off the couch or from behind their phone to have an active lifestyle. Walking 10,000 steps a day does not keep you fit and healthy. You have to really make an effort to be healthy and fit and most do not want to bother. It will catch up with you eventually.
5
@Anita Unless you have good genes and thus have a massive advantage.
9
I'm retired now but shortly before I left Corporate America my employer started giving free yoga classes 2 or 3 times per week. The HR director told me they were keeping track of who attended so that in the event they needed to do layoffs, they knew who to layoff first... the folks that had enough free time to take the classes!
93
@Sharon
Are you certain the HR director wasn't joking? Such an action would be ridiculous.
The main takeaway: causation and correlation are two entirely different animals.
33
"(RCT is) still, perhaps, the best method for conducting explanatory research."
RTC is the best empirical method for distinguishing between treatments, but it is inappropriate for developing explanations which require more work to merge empirical and logical conceptual accounts.
5
My opinion is that wellness programs are not really wellness programs until they approach the health of the employee from a physical and emotional perspective. I feel that perhaps even observational studies will be more precise in regard to its benefits if the wellness programs involved the components that should.
2
My employer offers such a program in which I am a non-participant (well, I'm not a smoker, so I do fill in the non-smoking affidavit). I don't participate in the free medical assessments, as I don't think my employer needs that information about me (I do get those assessments from my doctor each year). They don't pay for my gym membership, which would be useful to me as I work out regularly, or offer me a discount at the university gyms. Most of the other programs require time on my part, when I'm already time-squeezed. A lot of intrusion into my personal life for little or no benefit. No thanks.
112
I agree that rewarding people who improve their fitness through an objective assessment would be better than what is described in this article.@Leah Shopkow
1
Americans already have enormous financial incentive to keep fit due to our punitive medical costs. Europeans do not. Yet Europeans tend to live longer, healthier lives. This suggests that high deductible health plans and corporate wellness programs don't have a snowball's chance in hell of making any meaningful difference to our state of health.
Reducing Americans' precarious financial state and building cities and towns where it is possible, pleasurable even, to get out and walk, bike and meet neighbors would likely help a good deal more.
Unfortunately for those on the right, currently in power, the answer is to make life ever more miserable for the proletariat - the beatings will continue until morale (and health) improves.
80
@Sean
"Americans already have enormous financial incentive to keep fit due to our punitive medical costs." Have you visited a Walmart lately. Or boarded a flight first and then noticed the "fitness" of a high percentage of the passengers.
As to "Americans" vs. "Europeans" --Such broad brush conclusion with one ill-defined, aggregated independent variable fails all meaningful tests of significance--both statistical and practical.
6
A little too esoteric for me.
The bottom line for me is that wellness programs, self help groups, therapy sessions etc., whatever you want to call them can be very helpful but only if you truly want to be helped.
If you don't it is a waste of time for everybody.
Also programs like above can take time. My step daughter abused alcohol when she was a teenager and continued to after court ordered intervention programs.
It took time but it finally sunk in. Today she is an adult and sober for many yrs.
9
I used to work for a large corporation that introduced a wellness program. In exchange for claiming that we walked certain distances and spent so many minutes a day with our eyes closed, we could earn a minor rebate on our health insurance premiums. The scheme was presented as a cute little game where we earned points for our reported activities.
I suspect that the real goal of the program was to gain information on future insurance costs, which could then be used to fire employees that raised the company’s risk profile.
For obvious reasons it seemed safest to play along, so I dutifully copied and pasted my healthy recipes and logged my mindfulness obligations. Personally, though, I would be very skeptical of any “medical research” based on data from such a program.
136
@Global Charm
If I were offered a program, I would assume that OF COURSE an employer wants to get something out of it. Of course, they want to see if they can estimate and reduce future costs.
The way the idea of wellness programs was sold to employers was that this was a win-win program where each side could gain.
What you were offered was an insufficient reward to induce you to participate.
Also, each side of the equation was, according to your observations, "cheating" - employees were liars, and the company didn't fully inform participants as to how their information would be used.
In the research offered here, researchers focused in on a few facts, not opinions: how many times did the reporter actually go to the gym, how much was spent for medical issues?; did they participate in running events? and did they remain in the job?
If the company went after individuals for medical problems, it was likely they could and would do this without the need for a special program to identify them.
6
@Global Charm
My employer makes us play this points game in order to earn the $700 corporate contribution into our HSA each year. That is significant, but they cap the points for things that healthy people do and add huge bonus points for going to the doctor which provides the health data they can use to decide who needs to stay or go.
45
@Full disclosure
Can they access your actual medical records? if so, how: do you sign a Hippa release to participate? Or do they just take the proof that you had an appointment?
Your company is willing to pay you simply to induce you to take care of your health. Someone working at minimum wage in Montana would have to work over 84 hours for $700. ( or it would be more than 1.3% of the median household income, or 1.7% of the average salary. Significant.)
"Playing the game" seems rational - unless you are convinced that it involves major loss of privacy and there are insufficient legal protections for employees.
Went to the gym 7 times per year versus 3 times per year? Neither is enough to make any possible improvements in health!
76
This seems like the two different approaches to analyzing clinical drug trials.
A) “intention to treat”, where the patient’s who drop out or don,t take the drug are included as if they did,
Vs.
B) actual experience, where the patient’s who took the drug regularly are compared to the control.
It depends whether you wish to define the benefit of the treatment or the actual results prescribers will see including patient’s who don’t (including can’t) take the drug.
You need to say which you wish to define.
If I saw a patient and wanted to motivate them to comply (if possible), it would be nice to have an estimate of the benefit amongst those who took the treatment regularly.
11
@Stephen Rinsler Intent to treat is also the gold standard in measuring marketing campaigns these days. Well at least the analytics people know this - but that means the ad guys have to set up the randomized test(exposed to an ad)/control (not exposed) properly.
1
There is a an incomplete conclusion being drawn here. The control group is in the wellness program, bur shouldn't there be an outside one too? Perhaps this research shows not that wellness programs are useless, but that the quality of them has little impact, since people will be healthier (as shown in the observational impact) regardless.
If anything, this seems to support more numerous, less lavish programs that may simply affirm workers already interested in achieving wellness. Not to mention, the benefits of wellness programs include helping those already fit to maintain their fitness and a perk to not have to offer higher wages.
6
@Ray Martz The thing is -- every study not conducted by the wellness industry shows the same thing -- wellness doesn't save money. These studies are compiled at www.theysaidwhat.net
Even the ones done by the wellness industry often accidentally show total ineffectiveness and often harms. But they still give each other awards, no matter how bad the program is. https://theysaidwhat.net/2016/09/28/this-mornings-koop-award-raising-lyi...
11
Perhaps it is inappropriate to compare to the entire population of employees. Upon challenging many of the awkward incentives offered by our corporate wellness program, the Director quietly admitted that, while the program must be offered to all employees, their unspoken purpose was primarily to activate the subpopulation with the most problematic unhealthy habits. Controlling for selection bias seems inherently impossible in such cases.
17