U.S. News Methodology Changes by Year
Denise Gater, TheCenter, 2000 (revised)
2001
This year, U.S. News adjusted each school's research spending according to the ratio of its undergraduate to graduate student population. This change was in response to the following comments from their advisory board: 1) although large universities with sizable research programs often spend the most, a large portion of those expenditures benefits graduate students rather than undergraduate students; 2) generous spending at universities with expensive medical and engineering schools tends to benefit primarily medical and graduate students; and 3) giving the highest scores to institutions that spend the most ignores the possibility that beyond a certain level each incremental dollar spent no longer has an equal impact on educational quality. For those schools reporting unusually large amounts of per-student spending (outliers), they applied a logarithmic adjuster to all spending values.
A consequence of the above change is that some institutions that are strong in the sciences and thus moved up in the rankings last year, fell back in this year's rankings (for example, California Institute of Technology and Johns Hopkins University).
2000
One key change was the use of a procedure, known as "standardization," that brought U.S. News' calculations more in line with accepted statistical practices. In doing so, they dropped a step that they had previously used for national universities that tended to flatten out large disparities between schools in their performances on each indicator of quality -- commitment to superb teaching, for example, and financial resources, or spending on instruction, research, and related services.
In the past, School A might have spent $200,000 per student, School B might have spent $100,000, and School C might have spent $50,000. If they were the leaders in spending, U.S. News would have recognized them as first, second, and third but not have given them credit for the size of the disparities among them. This year, U.S. News' weighting system takes into account the size of these differences.
In general, the changes in the way they ranked schools this year boosted the rankings of a number of universities with strong science and engineering programs. Also, three public universities moved into the top 50 national universities in large part because their academic reputations are so much stronger than those of many of their peers. These are the University of Texas-Austin, the University of Washington, and the University of Florida.
1999
This year, because of changes in reporting rules for private colleges and universities, U.S. News measured financial resources by the average spending per student on instruction, research, public service, academic support, student services, institutional support, and operations and maintenance during the 1996 and 1997 fiscal years. They stated that generous per-student figures indicate a large endowment, enabling the college to offer a wide variety of programs and services.
In prior years, the financial resources category was broken down into two components: educational expenditures and other expenditures. Education expenditures was calculated as the sum of per-student spending for instruction, student services, academic support, and institutional support. The other component, "other expenditures," included spending on financial aid, research, operations and maintenance, and public service.
The 1997 IPEDS Finance survey introduced a different set of questions for private institutions in order to conform to changes in accounting and financial reporting standards issued by the Financial Accounting Standards Board. As a result of this accounting change, U.S. News revised its calculations for financial resources for this year's guide in order to make comparisons between public and private colleges.
1998
U.S. News did not made any significant changes in its methodology, but there are changes in the way that rankings were presented. Most important, the final score of each ranked school was rounded to the nearest whole number, which created more ties in the rankings. In the past, the score was carried to one place after the decimal point. U.S. News made this change because small statistical differences among institutions are not significant in setting them apart. Also, a school's score in the reputational survey was shown on a scale from 4.0 (the highest) to 1.0 (the lowest). In the past, each school's reputation rank was displayed, such as 164th.
1997
The weighting of the "Retention" measure (consisting of graduation rates and freshman retention rates) decreased from 25 percent to 20 percent (16 percent for graduation rates and 4 percent for freshman retention).
The "Value Added" measure was added to the rankings with a weighting of 5 percent of the total score (the "Value Added" title of this measure has subsequently been changed to "Graduation Rate Performance"). A predicted graduation rate was estimated for each school based on test scores on its 1989 entering class and the school's educational expenditures and this rate was then compared with the actual six-year graduation rate of the same class. Then, U.S. News calculated which universities produced higher than expected, and lower than expected, graduation rates to determine "Graduation Rate Performance."
1996
Because "outcomes" research shows that a school's retention rate -- its ability to retain and graduate its students -- is an important measure of academic quality, U.S. News increased the weight of the Retention measure from 15 to 25 percent. This year, the Retention measure included two components: 1) freshman retention (the school's ability to retain first-time freshmen), and 2) six-year graduation rates. At the same time, the measure of Student Selectivity (described as an input measure) was reduced from 25 to 15 percent.
This year, the schools ranked 26th to 50th (Tier 1) were listed in rank order (with the ranking identified). In previous years, Tier 1 universities were listed in alphabetical order so you did not know what rank they held.
1995
Weighting of the Financial Resources measure was decreased from 15% in 1994 to 10% in 1995. Weighting of Graduation Rates increased from 10% in 1994 to 15% in 1995.
This year's weighting of faculty salaries included a cost-of-living deflator, and this measure also reflected salaries of ALL faculty ranks, not just of full professors as in previous years.
The Alumni Satisfaction measure specified this year that alumni with graduate degrees should be excluded from the data reported for this measure.
1994
The Faculty Resources measure excluded law, medical, dental, and veterinary schools in the FTE counts for the student-to-faculty ratio. Class size was added as a factor in the Faculty Resources ranking (percentage classes of 50 or more and percentage classes under 20). The weighting for Faculty Resources decreased from 25% in 1993 to 20% in 1994.
The weighting for the Financial Resources measure decreased from 18% in 1993 to 15% in 1994.
Graduation Rates were now considered as a separate measure with a weighting of 10% rather than being called a measure of "student satisfaction" (with a weighting of 7% in 1993). This year, six-year graduation rates were used instead of five-year graduation rates as in previous years.
An Alumni Satisfaction measure was added (weighted at 5%), determined by the average percentage of a school's living alumni who gave to its fund drives.
1993
The Faculty Resources measure excluded law and medical schools in the FTE counts for the student-to-faculty ratio.
The Financial Resources measure was changed so that instead of determining just total E&G Expenditures per student, two separate categories were measured: Educational Expenditures per student and Other Expenditures per student. Educational expenditures included the sum spent on instruction, student services, and academic support including libraries. Other expenditures included research, scholarships, and operation and maintenance of plant. The weighting for the Financial Resources measure changed from 20% in 1992 to 18% in 1993.
The Student Satisfaction measure (five-year graduation rates) increased in weighting, from 5% in 1992 to 7% in 1993.
1992
In the Faculty Resources measure, student-to-faculty ratio was determined by FTE students and FTE instructional faculty. Previously, it was determined by the ratio of full-time students to full-time faculty. In addition to percentage of faculty with doctorates, this year the measure included percentage of faculty with terminal degrees in their fields.
This year, the Financial Resources measure was defined by dividing total E&G expenditures by FTE enrollment instead of by total headcount enrollment.