U.S. News and World Report's Methodology
Denise S. Gater, TheCenter Reports, 2001
The U.S. News and World Report magazine publication "America's Best Colleges" contains the magazine's controversial annual rankings of the country's best colleges and universities and has received widespread attention since its inception about ten years ago. Members of the higher education community continually debate the methodology used by the magazine, question the data's accuracy, and in general are skeptical of the merit and usefulness of these rankings. Some institutions have refused to participate in U.S. News' annual survey and others have proposed various alternatives to their rankings. Yet, the college rankings publication remains very popular as each year that single issue reportedly outsells any other issue of the magazine. In fact, U.S. News publishes a stand-alone book comprised of these college rankings in addition to publishing the lucrative magazine issue.
The U.S. News rankings appear to remain a popular source of information for prospective students and parents as they pour over a myriad of college guides and rankings in an effort to make informed decisions about which college or university to attend. Within colleges and universities, these rankings are often used for public relations purposes and may be displayed on an institution's web site as well as in various literature promoting the institution. In addition, as accountability and performance indicators continue to be important issues in higher education, some college and university administrators, governing boards, and higher education groups have used U.S. News and similar media rankings as measures of quality for higher education institutions.
One of the arguments commonly made by critics of the U.S News rankings
is that the rankings are not reliable because the measures and methodology change from year to year. They make the point that it does not make sense that an institution can be ranked number nine one year and advance to number one the next, or vice versa. On the other hand, U.S. News downplays the role of changes in methodology as a factor in changes in a college's rank. They state that while it is difficult to pinpoint exactly why changes happen, a college's change in rank can likely be attributed to a change in performance in relation to that of its peer institutions.
A primary research interest is in measuring and improving university performance, as well as in identifying the top public and private research universities in the country. TheCenter's Top American Research Universities project offers an alternative to weighted rank ordered listings such as those produced by U.S. News. This project gives institutions a sense of where they fit into the competitive marketplace of higher education by using reliable and appropriate performance indicators and a clustering methodology to evaluate the nation's top research universities.
As part of the background for TheCenter's research we examined the "America's Best Colleges" rankings over the past ten years to get a sense of their changes in methodology and rankings from year to year. The following tables are based upon a review of the rankings during the period of 1991 through 2000.
Year by year,
the various measures and associated weightings used for the
rankings (1991 through 2000).
Year by year narrative summary of changes in methodology, measures, and weights
(1992 through 2000).
The lists of
schools comprising the U.S. News "Top 50" or "Top 25" for
each year (1991 through 2000).
Comparison of
the U.S. News rankings each year for the group of
institutions included in TheCenter's Top Public Universities
(1991 through 2000).
Comparison of
the U.S. News rankings each year for the group of
institutions included in TheCenter's Top Private
Universities (1991 through 2000).
Our purpose here is not to analyze or debate the issues surrounding the rankings produced by U.S. News and World Report or similar rankings, but merely to provide historical background. These materials were prepared to provide context for The Top American Research Universities, published by TheCenter (July 2000), which gives an in-depth presentation on rankings, characteristics of universities, indicators of performance for research universities, as well as a frame of reference for universities to better measure their quality and achievement.