The Perils of the U.S. News Law School Rankings
Published on September 16, 2021
The rankings lead to less-informed decisions
As a tool for making application and enrollment decisions, the U.S. News rankings fail on a number of fronts. First, the rankings pay insufficient attention to what matters most to prospective students: job outcomes. Despite the importance of job outcomes, they account for only 18% of the rank and credit schools for jobs few attend law school to pursue.
Second, the rankings use a national scope, which places schools on the same scale. Only a handful of schools have a truly national reach in job placement. The rest have a regional, in-state, or even just local reach. The relative positioning of California Western and West Virginia in the rankings is virtually meaningless. Graduates from these schools do not compete with one another.
It turns out that ~80% of law schools placed at least half of their employed class of 2019 graduates in one state. The top state destination for each school accounts for 67.9% of employed graduates. A much smaller 8.6% of employed graduates go to a school's second most popular destination, with just 4.5% of employed graduates working in the third most popular destination. Only 19% of employed graduates end up in a state other than the top three. Comparing schools across the country just doesn't make sense.
Third, performance changes over time but year-to-year comparisons are impossible for your average reader. U.S. News will tell you that Stanford knocked Harvard out of the #2 spot in 2012-13, but the swap in rankings does not tell you why. Stanford may have improved while Harvard declined. Or, Stanford may have improved while Harvard's quality stayed the same. Or perhaps both schools saw a decline in quality but Harvard's decline was more severe. In fact, if every single school saw a marked decline in quality the U.S. News rankings would not indicate that this happened. Instead, students can know only relative performance.
U.S. News wants to be the duct tape of law school applications. No matter what an applicant’s career goals, geographic preference, or financial situation are, it wants to be the answer. Not only do the U.S. News rankings measure the wrong things, but they put schools in the wrong order for job outcomes, make meaningless comparisons between schools which are not in competition, and provide no way of telling what a difference in rankings mean.
Rankings are not inherently bad. In fact, they are conceptually quite useful. They order comparable things to help people sort through more information than they know how to or can weigh. However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. As previously described, the U.S. News rankings are victim to many of these flaws.
For these reasons (and more), we created this site to help students sift through volumes of information and make choices about where to apply, where to attend, and how much to pay. We unambiguously measure job outcomes, use a regional scope, and use real terms about the outcomes to help students make educated decisions about not just which school to attend, but whether any school happens to meet their needs.
The rankings make legal education worse
The U.S. News rat race is a story of incentives. These rankings persistently affect decisions made at law schools and by prospective students despite significant changes to the kind and quality of consumer information available to the public because so many stakeholders care so much. Rank jockeying costs not only time and money, but creative spirit. Explicit or implicit, these particular rankings are an X factor in decision-making. Want to make a program or curricular change? Assessing the impact on the school’s U.S. News ranking (and accordingly the dean’s job) plays a part.
Here are just some of the consequences to legal education’s obsession with the U.S. News rankings:
The rankings penalize racial and socioeconomic diversity
This year, for the first time, U.S. News added two student debt metrics to the methodology. Schools with lower average debt and lower borrowing rates perform better than schools with higher debt and more borrowing. While this may sound sensible at first look, the devil is in the details. The easiest way to game the system will not be lowering tuition but enrolling more wealthy students.
Read more here.
The rankings reward burning money (literally)
From the theory that a school that spends more money provides a better quality education, U.S. News adopted a methodological component called expenditures-per-student. A school that spends (and therefore usually costs) more money will outperform a school that spends (and costs) less. In a second expenditures metric, U.S. News rewards schools that increase tuition with scholarship offsets. In other words, a school that charges $40,000 tuition but gives everyone a $10,000 scholarship is better than a school that charges $30,000 without the discount.
Read more here.
The rankings further entrench inequity
While in some senses a reward for scholarship spending is nice, law school scholarships are a reverse-robinhood. The students least likely to complete school, pass the bar, and comfortably repay their debts subsidize those scholarships. Those students are also more likely to be students of color. Both uncomfortable facts stem from schools over-emphasizing LSAT scores in admissions, which highlight disparities by race and socioeconomic status.
Read more here.
The rankings penalize holistic admissions processes
The racial and wealth disparities on the LSAT are troubling on their own, but schools spend considerable resources on gaming their LSAT and GPA medians each year. As summarized above, this affects how schools allocate scholarships. It also affects how much attention schools place on non-gameable factors that are more relevant to who might make a great lawyer or member of the law school community. Ultimately, the rewards for even statistically meaningless improvements on LSAT/GPA medians affect student body composition. The lack of racial diversity in law schools is well publicized, but gaming the rankings over the past 20 years has also led to women attending schools with worse job outcomes and weaker access to the legal profession.
Read more here.
The rankings rat race has no finish line, but does provide a host of incentives that extract a school’s resources rather than enhance its ability to affordably deliver a better education. The negative effects compound yearly due to the structure of contemporary law schools. Moreover, the rat race supports beliefs by students, alumni, and media that these rankings measure something meaningful. If the law school administrators, faculty, and trustees pay attention to the rankings, then the rankings must signal real value. They do not.
By Kyle McEntee