There is an ever-increasing number of hospital rating systems, and many people have come to rely on them when searching for the best hospital. But depending on which system you use, you can find over 1,000 “top 100” hospitals in the country. That's one reason why a panel of prominent doctors, scientists and public health specialists is advising people not to put too much faith in the published hospital ratings used to promote these institutions.

“We found that many of the current hospital quality rating systems should be used cautiously as they likely often misclassify hospital performance and mislead the public,” said lead author, Karl Bilimoria, in a statement. “A lot of the so-called ‘top hospitals’ identified by some rating systems are not places that most physicians would refer their patients to.”

The Centers for Medicare & Medicaid Services' Star Ratings received a C, Leapfrog a C- and Healthgrades a D+.

Four major rating systems were analyzed. While noting that each had their individual strengths and weaknesses, the panel of doctors from Northwestern University, University of Michigan and Washington University in St. Louis, as well as area hospitals and medical groups, emphasized that each system had weaknesses that could lead to flawed ratings. These weaknesses ranged from use of inappropriate data to questionable methodology.

The highest grade given out was a B to U.S. News and World Report Best Hospitals, which was judged least likely to misclassify a hospital. But even for U.S. News and World Report, the panel expressed concerns over aspects of how they collect their data and the fact that U.S. News does not make any of their survey data available for private analysis.

The Centers for Medicare & Medicaid Services' Star Ratings received a C, Leapfrog a C- and Healthgrades a D+.

A major problem for consumers is that current rating systems often give conflicting results. A hospital that ranks best on one system might rank worst on another. What would make these systems better? The panel cites needs for better data, meaningful audits of that data and external peer review of the methods used by these rating systems. In the meantime, they advise skepticism when looking at hospital ratings.

All of the authors have had some relationship with at least one hospital rating system in the past, mostly as unpaid advisors. Evaluators were recused from grading a particular rating system if they had a direct current or recent relationship.

For more details on this and other aspects of the ratings, see the article in the New England Journal of Medicine Catalyst.