Would you choose a degree based on a newspaper’s ranking?

In mid-June 2014 the Financial Times (FT) will publish its latest rankings for Masters in Finance degree programmes. The Cambridge MFin may be included for the first time.

The FT rankings are divided into exclusively post-experience programmes (where prior work experience is required for entry) and pre-experience (where it is not). Last year there were five programmes in the first (including Singapore Management University’s MSc in Wealth Management) and 40 in the second.

What is the ranking all about? The FT is one of the UK’s best newspapers. It also wants to maximise revenue and web traffic. Like many other magazines and newspapers it has found that rankings are good for business.

The magazine Institutional Investor features polls and rankings in about half of its monthly issues each year, covering the All-American Research Team, the All-China team and so on. Project Finance magazine publishes the Deals of the Year. Euromoney publishes the FX awards. There is some sort of poll, ranking or prize for every category of finance. It seems that readers or perhaps advertisers enjoy these Oscar-like lists and there are dinners at which the winners receive their prizes, though typically without the tearful speeches of thanks that feature in the real Oscars.

MBA programmes have been ranked for many years so it’s not surprising that there are now rankings for other degrees, plus activities such as executive education, which are also based in business schools. Indeed the ranking game is mainly a business school phenomenon. There is nothing equivalent in the economics or history faculty, so far as I’m aware. When I was picking my masters degree in economics (admittedly some years ago) I didn’t consult a ranking.(*) There are rankings of whole universities but nobody really takes these very seriously, including Cambridge, even when it does quite well in them.

But business school rankings are taken seriously by many people. Why? My guess is that the rankings function as a sort of consumer guide and the applicants to business schools, where degree programmes are typically far more expensive than for other subjects, are in some way potential consumers. This creates a demand for consumer information to help applicants decide what to buy – or at least what to apply for.

Consumer information for a potential applicant

What might a potential consumer of a degree want to know? You might want to know about the difficulty of entry, since the credibility of universities is directly related to how hard they are to get into. You might also want to know about the quality of the teaching, meaning the relevance of the syllabus, the skill of the teachers and the general effectiveness of the learning process. You would want to know how the other consumers of university education, the employers, regarded the graduates of the degree programme. You might want to know something about the general experience of the course, including the social side, the opportunities for learning outside the course, the networking possibilities and how enjoyable it is, which would include the architecture, quality of food and accommodation, ease of access, weather and proximity to other interesting places. You would certainly want to know how much the course cost, including the opportunity cost of not working, which means a longer course is more expensive even if it has the same fee as a shorter course. Finally, in most cases business school students are interested in the future career prospects, so you would want to know about the track record of previous graduates, including their level of pay.

The FT ranking would tell you little about most of these except the last. Some 40% of the ranking is determined by the average pay and the increase in pay over three years as reported by the graduates of the degree three years after they completed it (for pre-experience it’s only 20%, for current salary). A further three percent (five percent for pre-experience) is based on “value for money”, which compares the average salary to the cost of the course (including salary foregone). More dubiously the FT calculates a score for employment progression based on the seniority of the person and the size of the company they work for. So presumably if you work for Citi or ICBC  you score more highly than if you work for Goldman Sachs or Renaissance. This accounts for 7% of the total (10% for pre-experience).

This is interesting information but it’s rather incomplete. Since pay is highly correlated with age, you would want to know the average age of the class. You might also want to know the geographic location of the graduates, since richer countries have higher pay (though not necessarily a higher standard of living – just ask somebody paying rent in London or Hong Kong). This data, which would help you interpret the average salary information, is absent, though you can get some of it from individual business school websites.

The FT survey partially compensates for the missing information by asking the graduates a series of subjective questions, including the extent to which their aims were achieved. But this counts for only three percent (five percent for pre-experience).

Another problem is the quality of the data. The FT solicits information from everyone in the class, whose names are provided by the degree programme administrative team. The response rate must be at least 20%, with an absolute minimum of 20 responses. I need not comment on the difficulties of inferring population characteristics from a non-random sample of only 20% of a class. I will just mention the experience of a friend of mine who graduated from a very prestigious and famous US institution. Three years after she completed the degree, she was asked to take part in a survey by an American newspaper, which included questions on her salary. The academic institution concerned contacted her directly to request that she not respond since, as a humble university academic, her salary would drag down the average to the detriment of the institution’s ranking.

I have no idea how common this is, and it’s obviously not the fault of the FT or other organisations that run such surveys. But a low response rate that may in itself be biased (lower income alumni might rationally not reply since they have an interest in a high ranking, all else being equal) makes the results less useful for our potential customer trying to compare different degree programmes.

The FT also includes information on the female proportion of the business school faculty, and of the advisory board and of the class. There are categories for the international mix of the class, proportion of faculty with doctorates and various other items, all of which might be of interest to potential applicants but the survey gives them a fixed percentage of the score. The FT’s method is described here.

Like equity research used to be

As an equity research analyst and then as a manager of analysts, it was frustrating that so much of the perception of success in the equity research industry rested on the results of an American magazine. By the early 2000s we had succeeded in weakening that dependence by conducting detailed and regular surveys of the clients themselves, who had an interest in giving us good information so as to get the service they deserved – or at least were paying for.

It is equally dispiriting that business schools are forced to compete on the basis of rankings done by newspapers. For MBAs there is strong evidence that higher rankings generate higher applications from better qualified applicants. It’s less clear for master of finance programmes since there is no centralised data and the rankings are much more recent. I like to think that finance students are more rational and analytical than MBA students so I hope that they see through the limitations of these surveys and make their decisions on other criteria.

Personally I would prefer not to take part in these rankings but participation may help Cambridge Judge Business School as a whole. The FT ranked the Cambridge MBA at fifth in Europe in 2013 (number 16 globally in 2014). But it ranked CJBS number 48 among European business schools as a whole. Since CJBS is not ranked in any other category than the MBA this is a rather mysterious result. Although it seems obvious nonsense to us, it has some impact on external perception of the School. A ranking for the MFin might improve the overall ranking and make life a little easier for our external affairs colleagues.

And Cambridge?

The Cambridge MFin is eligible this year for the first time to be included because the class of 2010-11 had 33 students, (the minimum class size for inclusion is 30) and the course has run for more than the minimum required four years. We don’t know whether at least 20 alumni replied to the survey so we won’t know if we’re included until June. I can’t think of anything about the course we would change that depends on the ranking. And I would be rather disappointed if applications rose or fell as a result of the ranking, if it happens. If Cambridge is included then I can be confident that we’ll be in the top six, unless a new master of finance programme has been launched somewhere in the last year.

(*) There are of course now rankings of economics masters degrees since there is no barrier to entry to publishing them. How reliable they are is another matter. I found this one which puts the LSE at number one in western Europe, not a controversial finding perhaps. But it has neither University College London nor Cambridge or Oxford in the top 50, which appears a bit odd (it has the Oxford Master of Financial Economics but not the MPhil in Economics). I’m delighted to find that the LSE ignores this ranking on its website. Meanwhile the QS 2013 ratings place Cambridge ninth worldwide in economics and econometrics (on all measures, not just masters degrees). You can find all sorts of stuff on the web and Elvis may indeed still be alive.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.