The annual U.S. News & World Report College Rankings are one of the most influential and eagerly anticipated evaluations in the higher education world, affecting everything from the decisions of prospective students to the self-esteem of university administrators.
Yet some critics of U.S. News & World Report argue that members of the higher education community have placed too much value on the rankings and that the rankings make use of flawed methodology.
Although many administrators admit they pay attention to the rankings, some say they are careful not to put too much stock in them. This ambivalence was reflected by a statement released by Princeton University upon learning last month they were rated as the nation's top university for 2003.
In the statement Princeton officials "stressed that the methodology in this report and similar surveys cannot capture the distinctiveness of an institution or whether one or another university might be an appropriate match for any individual student."
In fact, even members of the U.S. News & World Report staff acknowledge that the importance of the rankings have been blown out of proportion.
"There has been a lot of demonizing of U.S. News," said Richard Folkers, a spokesman for U.S. News & World Report. "We say every single year that a ranking is no way to pick a school."
The most heavily weighted and possibly most controversial criterion of the U.S. News & World Report rankings is the peer evaluation section, which counts for 25 percent of a school's overall evaluation.
Some administrators, including University President John T. Casteen III, view the peer evaluation section as unreliable and based mostly on reputation.
"Some of the U.S. News methodology interests me a good bit because it provides yardsticks for comparisons that other rating entities cannot make. Some, such as pure reputation, strikes me (as it strikes everyone else) as too subjective to be trusted," Casteen said in an e-mail.
However, even something as intangible as reputation is not necessarily irrelevant to high school students trying to decide where to go to school.
"Subjective reputation really does matter to students when they make their applications, and popular magazines are no worse a source of information" than anything else, Casteen said.
Daniel Levin, vice president for publications of the Association of Governing Boards of Colleges and Universities, said in a report that problems with the peer evaluation section go further than the subjectivity of the rankings.
The Association of Governing Boards is an organization that represents the nation's college and university presidents and governing board members and seeks to provide guidance to board members.
The peer evaluation section of the rankings is most prone to manipulation because consultants often recommend that schools use costly public relations campaigns to influence evaluators at other schools, Levin said in the report.
"Many schools are spending large amounts of money on such efforts, and boards should question whether this is money well spent," he said.
U.S. News & World Report staff members, however, defend the peer evaluation section as an essential part of their rankings.
"It's a survey of professional standing," Folkers said.
Additionally, in response to critics such as Levin, Folkers claims that public relations campaigns do not have a significant impact on a school's peer evaluation score.
"What virtually none of them get around to saying is what a miniscule factor" public relations campaigns are in determining peer evaluations, he said.
Although many schools may use direct mailings or public relations campaigns to boost their standing with other schools, the University does not, University spokeswoman Louise Dudley said.
"The President's Office has for a long time sent a copy of the President's report" to officials at other schools, but this practice predates the U.S. News & World Report rankings, Dudley said.
Other aspects of the U.S. News & World Report rankings also are perhaps prone to manipulation.
Some schools have tried to encourage more unqualified students to apply or to count incomplete applications as rejections, all to increase the appearance of selectivity, Levin said.
"The perception is out there that some schools are cheating," he added.
The rankings are not supposed to be as much of a competition as they have become, Folkers said.
"The end product here is not to be a horse race but for students to find a school for them," he said. "We produce a ranking for the benefit of providing a good guide for students."
Critics such as Levin also have problems with the rankings' use of per-student-expenditures, which account for 10 percent of a school's overall ranking.
This gives schools an incentive to raise spending even when fiscal prudence dictates doing otherwise, Levin said.
"If officials cut the rate of per-student spending, they risk lowering the institution's ranking," Levin said.
Robert Morse, U.S. News & World Report's database director said, however, that the inclusion of criteria such as per-student spending and faculty resources have been vindicated by recent budget cuts to public schools.
Funding crises at many schools, like the University, have shown the importance of such criteria in evaluating the quality of a school, Morse said.
Casteen agreed the rankings accurately reflect the University's budget troubles that have been caused by state funding cuts.
"Our low rankings on total resources and on support for faculty work certainly reflect realities that we struggle with everyday," he said.
The University dropped two spots in the most recent rankings for national universities, from 21st to 23rd, largely due to budget problems, Dudley said. The University ranked 35th in faculty resources and 66th in financial resources in the most recent rankings, numbers that reflect the state's budget crisis.
Among public universities, the University maintained its second place ranking, behind the University of California-Berkeley.
University officials said they were pleased the University remained as high as it did, given the state budget crunch.
"U.Va. ranks in this top group even though our funding is so much lower," Dudley said.
However, this year's rankings accurately reflect some of the University's strengths, Casteen said.
"Our high rankings on the quality of the student body and of the faculty and on alumni satisfaction match what I see," he said.
Like at most schools, University decision makers do pay attention to the rankings.
Alexander "Sandy" Gilliam, secretary of the Board of Visitors, said the rankings are mentioned "fairly often" at Board meetings.
Raising faculty salaries at the College at Wise marked one specific instance when the rankings influenced Board policy.
"Back a few years ago, the College at Wise got consistently poor rankings because the faculty salaries were so low," Gilliam said. "It really pushed the Board to do something."
Once the Board raised faculty salaries at the College at Wise the school's ranking did improve, Gilliam said.
Policy changes such as the increase in faculty salaries reflect what almost everyone on all sides of the rankings debate agree upon: many of the policy changes the rankings spur are actions schools should be taking anyway.
"Many in higher education acknowledge that the rankings incorporate some important indicators of institutional quality and performance," Levin said in a Washington Post op-ed article earlier this month. "Nor are higher education officials wrong to pursue changes when the rankings reveal an institution's shortcomings."
Both the makers and critics of the U.S. News and World Report rankings also agree that no student should base his or her college choice solely on rankings.
"They can't consider everything a student wants when considering a university," Dudley said.