Icechicken's Rankings: What if we only cared about inputs? Forum

[icechicken]

Rankings fascinate me. If we relied completely on GPA/LSAT inputs to evaluate the relative quality of law schools, what would happen?

I think this approach has a lot to commend it. GPA and LSAT percentiles are objective criteria that schools can't fake and do appear to be the strongest indicators we have of law-school performance. This cuts out the noise of statistics which schools can manipulate, such as employment outcomes, and positive-reinforcement mechanisms like peer reputation. A rankings system that focuses only on these statistics rewards schools for being selective in their admissions processes and also for recruiting students with elite profiles. Since law schools already heavily emphasize GPA/LSAT in their admissions criteria, it is a safe bet that their ability to get elite numerical profiles to actually matriculate is a proxy for their attractiveness to applicants in general.

On the other hand, there are problems with this approach. Abandoning any emphasis on outcomes makes it hard to tell which schools are adding the most value. Focusing completely on GPA and LSAT in particular abandons any semblance of holism, and perversely rewards admissions practices which I think are bad (admitting a 4.0/180 who is unlikely to get admitted to the bar due to a criminal record, ignoring the importance of various kinds of diversity). But those concerns only really manifest themselves if people (applicants, admissions officers, employers) start actually putting stock in the input-oriented rankings. This is just a fun exercise.

Here are the results. I took the arithmetic mean of each school's 25th/50th/75th percentile and divided that by the maximum possible value (180 for LSAT, 4.33 for GPA) to create an index value for LSAT and GPA respectively. Then I discounted each index to reflect their relative predictive power according to LSAC's studies - a .36 multiplier for LSAT and .27 for GPA. Reasonable people may disagree that LSAT should be weighted more heavily in this way, but I think that this reflects the extent to which LSAT is a stronger indicator of performance and also more sought-after by schools. Brian Leiter makes a compelling argument for also taking class size into account. I was too lazy, but I agree that it's much more impressive to fill a class of >500 students with talented applicants than a class of <200.

Here's what the T20 (which really is a club of 19) looks like:

[+] Spoiler

Code: Select all

1	Yale University (CT)
2	Harvard University (MA)
3	Stanford University (CA)
4	University of Chicago (IL)
5	New York University
6	Columbia University (NY)
7	University of Virginia
7	University of Pennsylvania
9	University of California–Berkeley
9	University of Michigan–Ann Arbor
11	Duke University (NC)
11	Northwestern University (Pritzker) (IL)
13	Cornell University (NY)
14	University of California–Los Angeles (UCLA)
15	Georgetown University (DC)
16	University of Southern California (Gould)
17	Vanderbilt University (TN)
18	University of Texas–Austin
18	Washington University in St. Louis (WUSTL)

Berkeley does well on this measure. I think they are underrated in USNWR because they have strong 25th percentiles but weak medians compared to their peers. Similar story for USC. Texas suffers somewhat, probably because their cushy home-market position lets them outpunch their weight on statistics I'm ignoring like biglaw placement. No real surprises.

Elsewhere in the T50 (I didn't bother with schools outside of the US News T1), Brigham Young sees a massive jump to #20. W&M, Arizona, and SMU also perform a lot better than their USNWR rankings. Conversely, Iowa, W&L, and Wisconsin see a huge drop. I'm not sure what to make of this. I think my model is much more useful for the top 20 or so schools, which recruit nationally out of roughly the same pool of applicants. Wisconsin and BYU don't compete for very many cross-admits and their graduates don't target the same job markets, so it's less meaningful for the latter to have stronger applicants on paper.

Also interesting: the biggest numerical gaps on the whole chart are the spaces between Yale/Harvard/Stanford/Chicago/NYU and between Cornell/UCLA. I was expecting a big drop at the end of the T13 but I was surprised to see huge differences among the top 4 followed by an smooth plateau from #5 to #13. Not sure what to make of it.

[wmbuff]

At least this time you've given us a methodology?

I do think there are other factors worth considering. Bar passage rate strikes me as a big one that the schools can't manipulate. Biglaw + FC is solid for finding how well the school places in elite jobs, but there isn't really a foolproof way to isolate the rate of graduates who lack full time, long term lawyer jobs. And even more importantly, I'd like to know what graduate debt loads look like.

While there's some merit to finding the quality of students a school starts with, if a school routinely takes those overachievers and leaves them unemployed with debt up to their eyeballs, I want that to be a massive mark against them. If a school takes middling candidates and turns them into highly successful lawyers with low debt, I want that to count in their favor.

And yeah, class size should be a factor - it should amplify how impressed we are with attracting a level of students, but it should also amplify our disapproval when those students are bilked.

[A. Nony Mouse]

If we cared only about inputs we would all be hiko.

[sev]

Looks like you spent a lot of time to essentially reproduce this article: https://papers.ssrn.com/sol3/papers.cfm ... id=2933274

(The only difference seems to be the weight you place on GPA and LSAT based on their predictive power, which seems odd to me--schools may care about that but students don't.)

[TLSModBot]

If we cared only about inputs we would all be hiko.

fuck, you beat me to it

[icechicken]

I do think there are other factors worth considering. Bar passage rate strikes me as a big one that the schools can't manipulate. Biglaw + FC is solid for finding how well the school places in elite jobs, but there isn't really a foolproof way to isolate the rate of graduates who lack full time, long term lawyer jobs. And even more importantly, I'd like to know what graduate debt loads look like.

While there's some merit to finding the quality of students a school starts with, if a school routinely takes those overachievers and leaves them unemployed with debt up to their eyeballs, I want that to be a massive mark against them. If a school takes middling candidates and turns them into highly successful lawyers with low debt, I want that to count in their favor.

And yeah, class size should be a factor - it should amplify how impressed we are with attracting a level of students, but it should also amplify our disapproval when those students are bilked.

Agree with all of this. I definitely don't think anyone should decide what law school to attend based on this (or any similar) methodology.

Looks like you spent a lot of time to essentially reproduce this article: https://papers.ssrn.com/sol3/papers.cfm ... id=2933274

(The only difference seems to be the weight you place on GPA and LSAT based on their predictive power, which seems odd to me--schools may care about that but students don't.)

Thanks for the link! I love stuff like this.

If we cared only about inputs we would all be hiko.

fuck, you beat me to it

I agree that overemphasis on these two inputs is a bit silly, but the fact of the matter is that law schools empirically care a great deal.

It's also silly to judge somebody based on how much money they make, but soccer players' salaries are nevertheless a decent proxy for how good they are at soccer:

1) Ronaldo
2) Messi
3) Neymar
4) Bale
5) Ibrahimovic
6) Rooney
7) Suarez
8) Aguero
9) Rodriguez
10) Pogba

Seems like a defensible list to me, especially the top 5.

[HenryHankPalmer]

If onlys and justs were candies and nuts, then everyday would be Erntedankfest