Close Close Comment Creative Commons Donate Email Add Email Facebook Instagram Mastodon Facebook Messenger Mobile Nav Menu Podcast Print RSS Search Secure Twitter WhatsApp YouTube

Judging Obamacare: How Do We Know If It’s a Success or Failure?

Sign-ups are supposed to formally end today, and attention is shifting from marketing to measuring whether the law is meetings its goals.

One day very soon, the focus on Obamacare will turn from signing up new enrollees to quantifying the law’s success — or failure.

The six-month open enrollment period, during which consumers sign up for health plans under the Affordable Care Act, is supposed to end today. But the U.S. Department of Health and Human Services, as well as many states running their own marketplaces, are giving some extra time to consumers who’ve had trouble signing up.

It’ll probably all wrap up by April 15. Then, the final numbers will be tallied and the pronouncements will begin. Politicians on both sides of the aisle will use the same data to proclaim that they were right about the law.

Last Thursday, the Obama administration said that more than 6 million people have signed up for coverage on the health insurance exchanges, meeting the projections set out by the Congressional Budget Office. Republicans have countered by questioning how many enrollees have paid their first month’s premium, the final step necessary for coverage to be in effect.

Dr. David Blumenthal of the Commonwealth Fund recently told me that any attempt to review the success of the law must go beyond those who sign up for coverage on the exchanges. It should include those who gained coverage through the expansion of state Medicaid programs for the poor, as well as young adults who are now able to stay on their parents’ health plans because of the law.

“I think the real success of the law will be judged over 5 years, not six months,” he said. “In fact, this president, President Obama, has until January 2017 to establish it as a fixture in the American social policy firmament.”

That may well be true, but now seems like a reasonable time to take stock. So, how should success — and ultimately the law itself — be judged? Here’s what some experts are saying about which metrics to use and the problems with each.

What percentage of previously uninsured people are finding coverage under the exchanges?

We can’t answer this question yet because we don’t know whether those signing up for coverage were previously uninsured. In fact, some enrollees, perhaps many, had their insurance plans canceled at the end of 2013 because the plans did not meet the requirements set out by the ACA. Obama administration officials have not released any numbers on this.

That said, a recent report from the Leonard Davis Institute of Health Economics at the University of Pennsylvania and the Robert Wood Johnson Foundation compares enrollment data through the end of February (with one month left to go in the official open enrollment period) to the number of eligible uninsured people in each state.

Here’s what the researchers found:

“Overall, more than 4.2 million people have enrolled and picked a plan through the exchanges, about 14.8 percent of all potential eligibles. The enrollment rate varies from state to state, with a high of 54 percent in Vermont to a low of five percent in Massachusetts. We should note that Massachusetts had the lowest rate of uninsurance in the nation since its health reform in 2006; its previous success might mean that the remaining uninsured population could be especially difficult to reach.”

Here’s a graphic from the report showing the states in which the greatest share of uninsured received coverage (through February).

Did states meet estimates from the U.S. Department of Health and Human Services?

MarketWatch had a story last week comparing enrollment in each state to the HHS projections. By that measure, Connecticut led the pack, signing up 218 percent of its projected enrollment through the end of February. It was followed by Rhode Island, New Hampshire, New York and Maine.

Source: MarketWatch

The problem with this approach is that the goals are “in many cases, based on little more than educated guesswork,” writes Charles Gaba, creator of, which has become akin to the Bible for tracking sign-ups under the law.

He noted that CMS’ state-by-state projections were based on 7 million enrollees nationwide, the original projection of the CBO. That projection has since been revised downward to 6 million because of the problems with, the online sign-up portal for 36 states. In addition, some states provided their own figures while CMS simply sliced up the rest to fit the 7 million projection.

He elaborated in an email: “Ten states out of 50 gave their target numbers to CMS, but those numbers were higher than CMS was figuring, so they had to drop the other 40 states down so that the grand total fit the CBO's 7 [million] total. As a result, you get some absurd numbers — both NY and KY had the same 220K (actually, KY's was 220K, NY's was less at 218K) even though NY's population is much, much higher and so on.”

Gaba suggests an alternate measure more like the one used by the Leonard Davis Institute researchers, which looks at the percentage of eligible enrollees in each state.

What percentage of enrollees are young adults, aka the “young invincibles” who typically are regarded as healthier?

A number of news outlets have focused on the relative dearth of young people choosing plans through the end of February to point out that the insurance companies may not have so-called balanced risk pools, meaning enough young, healthy enrollees to offset the costs of older, sicker ones. The Washington Post noted this month:

“Strong participation by young adults is critical to the program’s success, because they tend to use less medical care. Because they are cheaper to insure, young people offset insurers’ costs of covering the sick, many of whom are eager to sign up for coverage. Under the health-care law, people with preexisting medical conditions can’t be rejected.

Initially, officials had hoped that 40 percent of the sign-ups would be adults under the age of 35, but only about 27 percent of February enrollments were young adults, about the same as in January. On Tuesday, administration officials said they were nevertheless encouraged and predicted more young people would enroll closer to the deadline.”

Drew Altman, president of the Kaiser Family Foundation, is critical of such efforts to equate young enrollees to healthy ones.

“Young people benefit the risk pool because they are healthier, but it’s really the percentage of healthy people that make or break the risk pool,” Altman wrote in a column last week. “Even if enrollment of young adults stays where it is – at about one-quarter instead of 40 percent, which our analysis shows they make up among potential enrollees – premiums would only increase by two to three percent. Though even that isn’t quite right, since many insurers expected this and already built it into their premiums.”

What we really need to know is what percentage of enrollees are healthy vs. sick. That will take time.

What will happen to insurance premiums in 2015?

Some experts are looking beyond this year’s enrollment numbers and are focusing on what the insurance rates will be for those renewing their plans this fall — or selecting plans for the first time. Scott Gottlieb, a resident fellow at the American Enterprise Institute, told me this month that he thinks attention will quickly shift from this year’s enrollment cycle to insurers’ rates for next year.

“The rates are going to come out early spring, so that’s going to be the next big story. And I suspect they’ll go up quite a bit,” he said.

That view was echoed by an anonymous insurance executive who talked to The Hill newspaper.

Dylan Scott at Talking Points Memo isn’t convinced they will uniformly rise. He also notes that insurance rates were increasing before Obamacare and will increase after the law.

“The real data for measuring Obamacare's success aren't in yet, but they eventually will be. At the top of the list: What happens with premiums in 2015? Plus: Do insurance companies leave the market or enter it? And the ultimate barometer: Has the number of uninsured Americans dropped significantly?” he wrote last week. “In simpler terms: Did Obamacare, in year one, create a sustainable insurance market for the long term?”

Another problem with looking at rates is that an insurer’s increase for 2015 may mean that it didn’t set the right price for this year, not that medical costs have increased dramatically.


In the end, some hints of the law’s success — or failure — will be available this year, but it will take longer to assess how much it has reduced the number of uninsured and moderated health care costs (the two key metrics of success).

An article Friday in The New York Times suggests that rather than judging the success of the law nationally, it may make more sense to look at it state by state. “A review of state-by-state enrollment data and other research, as well as interviews with patients, advocates, health policy analysts, elected officials, supporters and critics of the Affordable Care Act, suggest that, for consumers at least, the state of health care under the national law depends almost entirely on where a person lives,” the article said.

At the end of his column, Kaiser’s Altman wondered if the American public would wait for the facts to make up its mind: “The problem is that it will take time to learn if the mix of enrollees is healthier or sicker, and how premium increases vary around the country, and how people feel about their coverage,” he wrote. “Meanwhile Republican politicians will lambast the law and Democratic ones will offer lukewarm support and overall popularity of the ACA probably won’t change very much. Anybody willing to wait for a judgment based on the right metrics?”

Have you tried signing up for health care coverage through the new exchanges? Help us cover the Affordable Care Act by sharing your insurance story.

Latest Stories from ProPublica

Current site Current page