Apr 22, 2014 at 09:37 AM

How to Pick a College - Don't Use U.S. News and World Report

How to Pick a College - Don't Use US News and World Report

Information in this piece is based on an article published in The New Yorker

Malcolm Gladwell demonstrates the fallacy of the U.S. News and World Report’s annual listings of the “best” colleges. He decodes U.S. News’ methodology for its ranking system and debunks its worth, analyzing the seven weighted variables that determine where an institution ranks on the “Best Colleges” list.

Criteria for Best Colleges, per US News 

  1. Undergraduate academic reputation, 22.5 per cent
  2. Graduation and freshman retention rates, 20 per cent
  3. Faculty resources, 20 per cent
  4. Student selectivity, 15 per cent
  5. Financial resources, 10 per cent
  6. Graduation rate performance, 7.5 per cent
  7. Alumni giving, 5 per cent

These variables generate a score for each institution on a scale of 1 to 100, where Harvard is a 100 and the University of North Carolina-Greensboro is a 22. Gladwell analyzes the factors that are measured in each of these categories and shatters the illusion that these are characteristics of value.

US Government Statistics vs Soft Data that Drives College Rankings

Whereas graduation and freshmen retention rates are published statistics, the measures used to evaluate the most heavily weighted factor (academic reputation at 22.5%) are largely anecdotal and self-referential. The magazine surveys college and university presidents and admissions deans for the listed institutions, asking them to grade all the other schools that appear in the same category as their own. So administrators at the 261 “national universities” are asked to vote on the academic quality of peer institutions (and competitors). But why is it assumed that these judges would have informed insight on such a wide range of complex institutions? Where do they tend to go to research their evaluations? Gladwell writes, “When U.S. News asks a university president to perform the impossible task of assessing the relative merits of dozens of institutions he knows nothing about, he relies on the only source of detailed information at his disposal that assesses the relative merits of dozens of institutions he knows nothing about: U.S. News.” Thus, Gladwell concludes, the convenient tabulations in these rankings become a self-fulfilling prophecy.

How do Class Size and Student-Teacher Ratio Really Count?

The third criteria is “faculty resources,” which accounts for 20% of an institution’s score, and the most heavily weighted factor in this category is faculty salary. Does average faculty pay (plus benefits, adjusted for regional cost-of-living) really signify the quality of teaching? Or “Do professors who get paid more money really take their teaching roles more seriously?”, as Gladwell poses the question. While factors that are heavily promoted in college publicity, such as student-faculty ratio and the proportion of full-time faculty are only weighted at 5% each, more than one-third of the faculty resources score derives from average salary. Class size—another statistic that is touted by many schools—and the proportion of professors who hold the highest degrees in their field are also factored into this score.

Selectivity: Measure of Skill or Game of Chance?

Selectivity, or what percentage of students who apply are offered admission, is a function of the number of applicants a college attracts—and its place on lists of best colleges will drive that number up, another indication of the circular force driving this process. Other factors tabulated in admitted student profiles include scores on the SAT, ACT and other standardized tests, GPA, and class rank, which only compares GPA, not the rigor of the student’s curriculum or the competitiveness of any particular school to a national standard. It is unthinkable that an Ivy League school would rank its own students by grades, but if an applicant to one of these elite schools can’t report being in the top 10% of the class, chances of admission plummet.

How does selectivity impact college experience? It conveys prestige, although the volume of applicants to the schools at the top of the rankings has made admission no more a meritocracy than winning a lottery. What does it really mean when 95% of applicants are rejected?  That with such overwhelming numbers, the process has become arbitrary.

By Trying to be Heterogeneous Rankings of Colleges are Arbitrary

Gladwell gives a sophisticated rendering of the aphorism about lies, damned lies, and statistics. He illustrates how inappropriate criteria create meaningless evaluations by ranking expensive sports cars with measures appropriate to the sedans of everyday transportation. He extrapolates this warning:

A ranking can be heterogeneous, as long as it doesn’t try to be too comprehensive. And it can be comprehensive as long as it doesn’t try to measure things that are heterogeneous. But it’s an act of real audacity when a ranking system tries to be comprehensive and heterogeneous…

If we don’t understand what the right proxies for college quality are, let alone how to represent those proxies in a comprehensive, heterogeneous grading system, then our rankings are inherently arbitrary.

Just as it would be ridiculous to use the specifications for sports cars to buy a minivan, building a college search on these overly broad evaluations will produce an unproductive muddle. The basic flaw in ranking systems is that they overlook the essential fact that every student is unique, and that many factors are at play in each college search. What is best for one student would not be a good fit for another. As the educator Loren Pope put it in his landmark book, Colleges that Change Lives, "A college is not a prize to be won but a match to be made." A college counselor will help the student weigh a range of factors—from majors to selectivity to campus culture to cost—to ensure many good options from which to make the best match.

Affordability is Lacking from Score

Strangely, one measure that is entirely lacking from U.S. News’ list of variables is price. Being affordable doesn’t make a college better, according to this methodology. But as the cost of college becomes an untenable burden and barrier for many Americans, and the amount of student loans has surpassed credit card debt, affordability is a factor that must be weighed carefully by most families. That fact should be reflected in U.S. News methodology, but it isn’t, which is why students must develop personal rankings that reflect their specific resources, needs, and goals. The purpose of college counseling is to assist with that evaluation and help each student find the best fit for their college education.

Learn How You Should Pick a College at Goleta Public Library

On May 5, 2014, at the Goleta Public Library, Tish O’Connor, principal at CollegeConsult, Santa Barbara, will introduce useful tools and helpful guidelines for customizing college search. Her talk, entitled Building a College List, is directed to parents and students currently in junior and sophomore years of high school.

Posted in College Search.