Are bad exams harder to mark than good ones?

Bored out of my brains marking a large pile (84) of exams, I decided to spice things up a little bit by timing how long it took me to mark each question. I happen to favour the style of marking where you mark every question 1, followed by every question 2 etc.

It seemed clear to me that the speed of marking should depend on the order the paper was marked in: as I become more and more familiar with the kind of weird stuff students write in response to my questions, and as I solidify in my mind how many marks I feel particular answers are worth, it makes sense that I should speed up.

I also wondered whether bad answers are more difficult to mark than good answers.

Looking at this, I can see I’m right about my first assumption:


The duration clearly decreases on average as the marking exercise goes on.

But what about the crucial question: is marking a bad exam answer harder than marking a good one?


The answer is a fairly resounding “no” (Specification 1). The low R2 also makes you think there’s something more important going on here: namely handwriting: I’d guess that’s the big factor.

But like a good social scientist, I wasn’t happy with leaving it there. Maybe there was some kind of nonlinear effect: very, very bad answers are easy to mark (since there’s usually little or nothing written), so are very good ones.

Specification (2) shows that there’s not much evidence that this is the case. (in fact p is less than 0.1 for both marks and squared marks in (2), so in some social science contexts, I’d award myself a little star!)

You might think: this is the most boring result imaginable. Why is this worth a blog post?

Well, you’re right of course…. but the struggle against publication bias just claimed one small victory!


Does your econ department care about new ideas?

The Reteaching Economics network is a group of early-career economics teachers interested in moving the teaching of economics on from nonsense like this:

NAIRU – Not in fact a Pacific island nationImage by Asacarny, distributed under a CC by 2.5 licence.

to something a little bit more resembling the actual world which students expected to be finding out about when they signed up for an Economics undergraduate[1].

They are inspired by the incredible Rethinking Economics, an international student group whose founders have just written an excellent book on the “perils of leaving economics to the experts.” The energy and vision of this group of students makes the average econ department look like exactly the kind of left-behind-with-dust-gathering legacy institution it usually is.

The Reteaching Economics group publishes a list of its members online, so we can see which institutions are most likely to care about teaching new ideas. Here’s the breakdown, including only institutions with more than one member.


There is a fairly long list of lonely singleton Reteachers (full disclosure: I’m one of them) who are the distant outposts of these ideas in otherwise skeptical economics departments. They should be a source of sympathy, greetings cards and reassuring poems/songs/flashmobs.

If you study Economics at an institution other than these, why not make your teacher aware of the new world being created just beyond the walls of their department?



[1] This is excluding the not-inconsiderable minority of econ undergrads who signed up because they want a job at Goldman/KPMG/etc. as soon as possible, who would probably be best served by finding out as little about the real world as possible. go back ^

Does your department care about tax havens?

Oxfam published a press release yesterday containing an open letter to world leaders calling for them to “make significant moves towards ending the era of tax havens” which are “distorting the working of the global economy”.

This seems to me like a pretty important intervention, and it’s a rare opportunity for economists to use their variably-justified reputation as “people who know about the economy” to do something positive to fix the way the system works at a global level.

These kinds of coordinated interventions into the actual workings of the global economic system seem woefully infrequent to me. So when it does happen, I’m anxious to be a part of it in whatever tiny way I can. And the least a bottom-rung economist like me can do is choose to work for the institution which is pulling hardest in the right direction.

Oxfam published the full list of signatories to the letter, so I thought I’d do a teeny bit of analysis, to see which countries and which UK departments care most about bringing an end to the global tax haven system. Perhaps this will help any early-careerists choose an institution whose interests align with theirs: after all, there is more to judging a faculty than counting its number of peer-reviewed publications.

Spoiler alert: the number of signatories from Bristol, the university I’ve recently moved to be a part of? Zero. Bad times…


First, here’s the breakdown of signatories by country:countries

Italy leads the field by a long, long way. From a first glance, it looks like they are from a decent variety of institutions but I haven’t checked this properly. Future work someone? Italy is a known advocate for change in economics, being a big adopter of the CORE project which aims to transform how undergrads are taught economics. The UK comes a pleasing second, I look at this result in a bit more detail below, and it’s pleasing to see the USA being well represented too.

It’s worth pointing out here that not all signatories are equal: although France has only a paltry ten signatories, one is Olivier Blanchard and another is Thomas Piketty. These are both absolute giants of the economics world, and their contribution here, particularly Blanchard who is not exactly known as an iconoclast, is very significant indeed. (One might do an analysis by Google Scholar citation count instead, which would show up these differences.)

Now let’s look at which institutions those 50 UK signatories come from:institutions

The top two, SOAS and Greenwich, are both already on my radar as being slightly more radical than the standard econ department. But this is not a list dominated by such agitators: the LSE is hardly known as an anti-establishment hotbed, and nor are Warwick or Oxford.

Big-name mover for change, Ha-Joon Chang, cuts a lonely figure as the sole signatory from Cambridge. I hope he at least has some researcher staff of his own to go punting with.

And, as I mentioned, zero signatories from Bristol.



Economists should start doing themselves out of a job

The reason why teaching undergrads is the best job I’ve ever done is because interacting with intelligent, energetic people is not the once-in-a-while happy coincidence it is in most jobs, but it’s the central purpose of what you’re supposed to be doing.

Sure, there are the hours of marking, the jocks, the whingers (colleagues that is, not students), the disastrous classroom IT. But the main point and purpose of my being here is to come into regular contact with open-minded people, and try and draw them into interesting discussion. And to do this with no agenda other than to keep everyone involved entertained and intellectually stimulated. It’s like being at a daytime teetotallers’ high-brow cocktail party.

But some conversations are more interesting than others. And a conversation I was drawn into on my first day of teaching after a long, isolated Easter break left me in need of a strong drink and a sit down.

It involved a student of the most memorable Myers-Briggs category: TICL (Talkative; Intelligent; Cheeky; Lazy). TICLs are either a joy or a damnation depending on whether you like the student in question, whether they like you, and how you deal with the inevitable uproar they cause. My managing of this particular TICL has been an up-and-down affair, depending on our respective moods, but on this occasion they were in a particularly restive mode.

After some fairly meandering discussion of an arcane trade theory, the TICL interjected with “when are we going to get to the good stuff?” Foolishly drawn in, I said “This is the good stuff. We’re talking about the actual world here! Trade! You might wish you were back in this class when you’re doing statistical asymptotics in 2nd year econometrics!” To which they smartly replied “Yeah well, I might wish I’d studied something different.”

The basic complaint is that the student had imagined, when applying to do economics, they would learn about what makes the world tick. How do banks make their billions? Why are some countries so seemingly doomed to mishap? Just what is it about China that makes everyone want to talk about it? And come to think of it what ever happened to the US? Or Japan? Or Europe?  In other words, to come to understand the world. To see it as a set of explicable causes and effects, not just a random mishmash of historical happenstance. To know the proposed remedies for the ailments of the world, and why the remedies of the past were no good. To have a sense that somebody, somewhere knows what the heck we should DO!

And I couldn’t completely disagree with this student’s analysis of their undergraduate course. Where were the big ideas? Where were the theories that made all the seemingly insane stuff we see happening actually fall into place? Where was the sense that here, in the Economics department, was a team of geniuses plotting the salvation of global capitalism from its cannibalistic excesses?

But worse than my half-agreeing that we were teaching these undergraduates the wrong stuff was a deeper more gruesome realisation by far. What if there is no explanatory grand theory? What if there is no team of geniuses? Why are we still debating the reasons why people trade with each other, or why they don’t all gamble their savings away on the stock market? Or what governments should do when a recession hits?

It seems to me that 90% of economists are still stuck trying to understand the developed old world, forever peering into the black boxes of the Federal Reserve or poring over UK tax returns, and trying to guess how the various levers work, while studies of development, or the power of Google, or the end of manufacturing, or the dark web of global capital of which Panama is only the tip of the iceberg, are relegated to tiny departmental outposts or, horror of horrors, the social sciences.

Hopefully I have successfully convinced you by the above tirade that I was in a bit of a neg. And so I retreated to the only place I knew I was guaranteed to be undisturbed by man nor beast: the Economics staff common room, a place so lonely and ill-loved that even the old copies of the FT cling and heap together for company.

And so it was that I starting reading this Lucy Kellaway article about how young people are encouraged to think that their shiny graduate jobs at KPMG and Goldman Sachs are going to be fun, important and exciting, only belatedly to discover that they are working for a tax auditor and an investment bank respectively. Needlessly to say they’re pissed off, and feel betrayed:

When the penny drops like this, there are only two possible outcomes. Either you quit… or you silence your doubts and get sucked into the machine.

This seemed to sum up the situation facing my angry, disappointed student. But, of course, these are not the only options. There is a third choice.

You can seek out the parts of your job or course that you do like, and seek out the people you respect and who you would actively like to work and spend time with, then stick to them as tightly as you can and insist without shyness on being in their teams, on their projects or in their classes. And while you’re there learn the ways of the organisation you’re a part of.

And by doing this you can come to know the enemy you seek to undermine. You need to be fluent in the arguments you’re hoping to counter, and to understand the culture you’re so frustrated by before you can become an agent for change within that culture.

So here’s my manifesto: I want to become an economist so I can change what becoming an economist entails, and what being an economist is like. I want to work in a supportive, collaborative department where the common goal of understanding the economy, and hence the world, is palpable all around me. And I want economics to at least start to do itself out of a job, rather than creating the vacancies for a thousand more economists as we did in the years before 2008.

Or, to put it better, in the words of good old JM Keynes:

The day is not far off when the economic problem will take the back seat where it belongs, and the arena of the heart and the head will be occupied or reoccupied, by our real problems – the problems of life and of human relations, of creation and behaviour and religion.

How to make an economist

I’ve often asked myself, in self doubting moments and imposter-syndrome-rich night sweat events, what the difference is really between a person who says they are an economist and, well, just a person. Can I really lay any claim to be something other than the averagely well-informed news media-consuming citizen?

Certainly a lot of what I covered in my MSc was hard to learn but, in terms of sheer insight and understanding of how the world works, I usually think that you can explain all of economics to an attentive listener during one good session in the pub.

But if there is a difference between an economist and our hypothetical well-informed media consumer, it’s perhaps this: it’s an economist’s actual business to read massive, threatening-looking books about the economy. I won’t understand the content of such books differently (or, crucially, retain anything any better) than the average reader. It’s just that I’ll actually take the time to do the reading in the first place rather than tackling, say, an account of the British protectorates in the Middle East or a tome on Cubism. You’ve only got a certain number of reading hours in your life (particularly if you watch as much John Oliver on YouTube as I do) and what you choose to dedicate those hours to ends up shaping the adjectives you feel comfortable using to describe your actual self: I am an economist.

It is in this spirit that I begin what looks like your archetype of the no-fucking-around serious and weighty tome on economics currently threatening to destabilise the centre of gravity of my pretty low-slung coffee table:


Holy shit it’s big. And it’s serious looking. The unbelievably old-fashioned twisted flax motifs bracketing the title are the sleeve-designer’s equivalent of a bow tie on a 35-year-old debate show host: they’re designed to fill you with awe at the seriousness of what’s being presented.

But so far, and it’s very VERY early days by the way, I’m impressed and excited by the tone that’s been set and the direction it seems to be heading. What I hadn’t realised is that the whole thing’s a summary of a load of work Picketty and assorted colleagues did to gather as much historical data as they possibly could about wealth and income (which not the same thing) in as many countries as they could, going back as far in history as they could. The introduction gives some details about the monster database they’ve built, and then the rest of the book is going to be all about the things they’ve actually managed to find out, by looking at this data and asking questions of it.

This is exactly what my whole philosophy of data-driven economics was aiming to be about, although I never actually managed to articulate it in any coherent way. But it’s basically about trying to think of economics as the social science it is, wanting to imitate anthropology or political science, instead of physics as the discipline’s been doing for 120 years. And that involves careful observation of the thing in question, in our case the economy, and conclusions drawn on the basis of those observations.

It also has given me a big grin in the form a wicked (in both senses) quote which sums up perfectly exactly how I feel about the economics establishment I’ve recently become a card-carrying part of:

To put it bluntly, the discipline of economics has yet to get over its childish passion for mathematics and for purely theoretical and often highly ideological speculation, at the expense of historical research and collaboration with the other social sciences. Economists are all too often preoccupied with petty mathematical problems of interest only to themselves.

Boom! It’s reading statements like this that not only make me feel more justified in calling myself an economist, but that actually make me proud to assign myself the title.

The new “improved” UN job application website

Anyone who has every applied for a job at the UN will feel a sweat break out on the back of their neck at the mention of the 90’s-style faux-Latin-named job application site “Inspira”. (Remember when things were all called “Exceptimus” and “Ignitor” and stuff? Ugh.)

Anyway, the site’s been down for a few days for modernisation, and the new improved site came back up on Friday. (They had a FOUR DAY outage! This alone is enough to make me suspect that the people who designed the website didn’t know what they were doing. Outages should measured in minutes or hours, not days.)

Given my experience of large organisations and their ability to procure IT which isn’t grossly overpriced and absolutely terrible (for which see Britain’s NHS, Barclays’ several iterations of terrible online banking websites, any government department in the world ever) it’s no surprise to see that the fundamentally broken Inspira has been given little more than a lick of paint.

Still present are the absence of a “Back” functionality, because the site doesn’t use URLs to control location, visible meaningless Javascript all over the place (hovering over a link to my job opening reveals “javascript:submitAction_win4(document.win4, ‘HRS_CE_WRK_HRS_JOB_LINK$21$$0′” which gives just a hint of the horrors people meeting the website’s code for the first time must experience), Windows 98-style icons and the overall user-experience and look and feel of being dragged by chain up a muddy slope by a slow moving tractor.

But my personal favourite is the built-in spell-check function. (Note to developers: people don’t develop their own spell-check functions any more. Browsers do this now.) My entire cover letter passed the spell-check with flying colours bar one exception: It suggests I replace the word “in” in the opening sentence with the admittedly more emphatic, but perhaps overzealous “IN”.

I’m not even joking.

The UN Inspira spellcheck suggesting "IN" as a replacement for the word "in"

Modelling Africa’s trade routes

People talk a lot about how development aid might be used to improve a country’s attractiveness as a trade partner. (Mostly the World Trade Organisation, but not exclusively!)

“Aid for Trade” is a controversial project because it has a distinctly globalisation-friendly vibe about it, and a fundamental belief in the kind of trickle-down economics so beloved of market-oriented people and organisations.

But one thing that is never discussed when the possibility is raised of improving a country’s export competitiveness, is that in the absence of additional global demand, any increase in export due to an Aid for Trade programme must be accompanied by a reduction in exports for somebody else.

With the global economic model I’ve built as part of my PhD (and some fabulously bold assumptions about how trade works), I can have a stab at modelling which countries stand to gain and which to lose from a particular Aid for Trade project.

This picture, drawn using ESRI’s flashy new ArcGIS Pro shows the modelled results of improving Ethiopia’s export infrastructure. (It’s “inspired” (for which read, pinched) from this nice flight paths visualisation.)

Flows which increase are in blue, and those which decrease are in red.


Here are the boring but very necessary caveats:
– Only those between African countries are shown.
– Because the increased flows are much bigger in magnitude than the decreased ones (at least within Africa) I’ve had to compromise on the line thicknesses, leading to an overstatement of the decreases!! Caveat emptor!!
– This is based on a gravity-type trade model. Their use in predicting trade is controversial.
– The economic descriptions used in this model are based on estimates, since most African countries don’t publish the kind of economic data you’d need to build a proper description.