The Vestigial Reference Letter

The graduate school reference letter is a holdover from a time when academics was much more of a closed off clique than the relatively open network of today. Back then, references served to keep the world of higher education reserved for insiders. Nowadays, these letters are vestigial and purposeless; they only serve to waste the time of applicants, referees, and admissions people.




First, a side note to any students of mine reading this:

Yes, I am still willing to give you a reference. You will have a much better chance of getting a GOOD reference (not just from me, from anyone) if you were in a smaller, more senior class of your referee’s or you did some research with them. Another way to improve the letter you get is it provide a rough draft of what you would like to see in the letter. Common traits that grad schools (in statistics, in Canada, the US, and the UK), look for are the ability to learn advanced material, the ability to conduct independent research, and the ability to communicate in English well enough to teach labs.
 
I will still write you that reference, even a good one if I am able, but I think the whole reference exercise is obsolete.

Now for my gripe:

The potential value of a reference letter comes from two sources:
1)    The relative rarity of having someone of importance vouch for you.
2)    The additional information that could grant or deny admission to an applicant when their other credentials provide insufficient information.

The relative rarity is important because the reference letter is supposed to speak to the unique talents of the applicant. It’s supposed to send a message that “sure, these are the applicant’s grades, experience and test scores, but I, some authority figure, say that THIS applicant in particular has something special.” If every applicant has a reference letter, then all them are special, which means none of them are special. It would take someone of great importance a great amount of effort to produce a letter that had this rarity quality.

Instead, reference letters tend towards being little more than form responses, especially for referees that need to produce many of them each year (hence my advice to provide a rough draft). In a recent round of applications, a colleague of mine complained that they had 200 applicants to sort through, and that all of their letters looked identical.

The referees may as well be identical too. The ideal reference is one that knows the applicant well and has a lot of authority in academia. In schools in the US and Canada, ever more of the teaching load, and hence the student exposure, falls upon sessional staff instead of tenured faculty. The ones that students, who then become applicants, know the best are the same ones that lack the authority for their names to carry any weight as a reference. Internationally, this also holds even for tenured faculty, because its hard enough to keep track of the big names domestically, let alone overseas.

The result is a vast majority of letters coming from non-noteworthy sources and containing non-noteworthy information, which brings that second point about additional information into play.

With very few exceptions, letters of reference will be positive, even if blandly so. If a negative reference letter is ever leaked to an applicant, it’s potential for a libel case or other backlash, which makes positively-worded letters all but mandatory. Furthermore, referees have a vested interest in the applicants from their schools going on to graduate school, so the positively of references tends to be inflated.

This article by Howard Gardner in The Chronicle of Higher Education discusses recommendation inflation: https://www.chronicle.com/article/Recommendation-Inflation/239641 . According to this article “one almost never sees [rank ordering] checklists that are not completely skewed to the positive – so much that checking off ‘top 10 percent’ rather than ‘top 1 percent’ can be the de facto kiss of death for the applicant.” This speaks to the meaninglessness of references, in that a referee’s inflation factor matters far more than an applicant’s actual acumen.

As for the letters themselves, this inflation reduces words like ‘excellent’ to ‘average’, and ‘outstanding’ as ‘slightly above average’, and anything else to sarcasm. Any other unique traits or experiences are best mentioned in the applicant’s personal statement anyways.

Even if a reference wasn’t required, but merely allowed, game theory suggests that nearly every applicant worth considering will include a letter. References need to be removed from the process entirely.