A Nobel effort?
February 13, 2009
So the changes to Research Council grant application forms made it the front page of THE magazine (and features in a leader article). Or at least the reaction of a small number of senior researchers did. I can’t say I am particularly surprised by the reaction. The RCUK position is admirably set out by Phillip Esler in the article, and here I am setting down some personal views.
A major argument in a letter to the THE criticising the RCUK policy changes is that there has been a decline in the number of Nobel prizes awarded to British scientists, and, the letter argues, this is related to increased regulation of research. While the fall of in numbers of Nobel prizes is a matter of record, does the argument hold water? I think not, for the following reasons:
- There are numerous other factors that will have a strong influence on the number of Nobel prizes, not least of which is the level of research funding. And is is important to remember that there is a considerable lag between when research is done, and the award of a Nobel prize. So the decline in the 80s and 90s could partly reflect previous underinvestment in research funding which has been reversed in the last decade.
- The number of Nobel prizes is fixed, so when the number of UK winners goes down then the number in the rest of the world goes up. Part of the explanation could be a rising standard of research elsewhere rather than a decline in the UK.
- It is also the case that the number of Nobel prize winners is small so will be strongly susceptible to random fluctuations. Of particular note is the fact that Nobel prizes are not awarded posthumously. So however great your scientific discovery if you are unlucky enough to die before it is recognised, there will be no Nobel prize. And if the UK has been unlucky in this regard it could (again partly) explain fluctuations.
Even if you buy into the idea that the reduction in the number of Nobel prizes has been caused by research policy changes in the UK, it does not follow that this is a problem. Don’t get me wrong, Nobel prizes are great and are, in some sense, indicators of excellent research. But they are the tip of a very big iceberg, and would we really want a research policy that maximised the number of Nobel prizes? If you want to look for indicators of the health of UK research than looking at measures that cover a wider range of activity, like bibliometrics, seems more sensible. The most recent independent report on the health of the UK research base shows a steady improvement over the last 10 year in measures like citations per researcher and citation impact. The recent outcome of the Research Assessment Exercise also paints a positive picture. Perhaps things are not as bad as some members of the research community might lead us to believe…
There is also some interesting commentary on this on the Prometheus science policy blog.
Testing hypotheses... 
February 13, 2009 at 5:07 pm
Admirable, indeed. Philip Esler states that RCUK’s position is that
So, all one need do is quote this commitment in the Impact Summary, thereby guaranteeing that it will have no effect on the outcome of the application.
Of course that does then rather beg the question of why we are being made to jump through this tedious bureaucratic hoop in the first place…
February 13, 2009 at 9:54 pm
If the implication here is that only a small number of senior researchers are concerned about the imposition of economic impact criteria in the peer review process, then RCUK is even less in tune with the views of university scientists than I thought possible! I suggest, Steven, that you return to the Prometheus blog cited at the end of your post and read the comment left there by Glazer . Then read it again. And again.
Glazer’s comment is a searingly honest appraisal of the views of very many university scientists. A prevalent view is that we “play the game”, keep our heads down, and do what’s required to get funding. Any academic will be able to churn out two pages of the engaging our stakeholders, going forward vacuous nonsense in the Impact Summary section that RCUK will require of us. I’ve certainly had enough practice in doing it for European Framework Programme applications. But, as Glazer points out, it’s inherently dishonest.
If you want to delude yourself that it’s only a “small number of senior researchers” who are concerned with the imposition of an appraisal of economic impact in the peer review process, then fine. I must admit it wouldn’t be out of line with my expectations. It certainly wouldn’t be the first time RCUK completely ignored the views of a very large number of academics on this subject. Take a moment to visit the Efficiency and Value for Money of Peer Review Project page on the RCUK website. As you no doubt recall, this contains feedback from practically every UK university on a number of topics related to peer review including, of course, the inclusion of economic impact criteria.
Let’s first look at RCUK’s own summary of the responses related to the imposition of economic impact criteria. That reads:
I’d argue that the statement above does not accurately capture the strength of the feedback. Let’s look at some individual responses. I’ll start, if you’ll excuse the “parochialism”, with Nottingham:
Let’s move south. What did the University of Oxford have to say?
And Cambridge…?
I do believe I see a pattern emerging. Does it propagate north of the border? I wonder how Glasgow responded…
So this was a “consultation” with the academic community in tbe best New Labour sense of the word: RCUK asked for advice; it got it; it ignored it.
I’d be interested in reading your response to not only the comments above but to the very important points raised in Ann Mroz’s excellent editorial in yesterday’s THE on the subject of our call for a “modest revolt”.
Philip
P.S. My thoughts on Philip Esler’s repeated claim that what RCUK is trying to promote is not just economic impact but broader societal impact are here .
February 14, 2009 at 8:43 am
@Phillip Moriarty Thanks for taking the time to post a detailed comment. I understand that concerns extend beyond the 20 who sent the letter – apologies if I gave a different impression.
I am not going to comment on the rights and wrongs of the specific policy change. This is a personal blog and for professional matters I prefer to use official channels which is why I refered to Phillip Esler’s comments.
The key point of the post is that the assertion that UK research isn’t as good as it used to be is contestable on the basis of looking at a range of evidence.
February 14, 2009 at 10:34 am
I’m afraid that it is characteristic of people who deal with science policy, rather than science, to believe that the quality of science can be measured by things like “citations per researcher and citation impact”. See, for example, http://dcscience.net/colquhoun-goodscience-jp-version-2007.pdf
The comparison with other countries is, in any case, made harder by the fact that many of them suffer from the same sorts of impediments to original and innovative research as those imposed in the UK by the government and the research councils.
February 14, 2009 at 10:37 am
[…] hypotheses is written by the head the Research Councils UK Strategy Unit. He chooses to concentrate on the secondary matter of Nobel prizes, rather than answer the real questions. Fortunately Philip […]
February 14, 2009 at 12:03 pm
I believe that many of the issues raised in the Braben et al THE letter are about leadership. In my longitudinal work I show that research universities that are led by top scholars improve their later research performance. On average, the best universities in the world — that win the highest number of Nobels — are led by outstanding scholars. Why? I interview 26 heads in research universities, and in short most beleieve that scholars create the best conditions for other top scholars. Top scholars tend to prioritize scholarship. Unfortunately, in the same way I would argue that managers often create the best conditions for other managers. In my forthcoming publication (Princeton U Press) I argue that top scholars should lead research universities, and also, those institutions are that fund research — which in many ways are the most important given that they provide the life-line.
February 14, 2009 at 9:25 pm
Steven,
Why post on issues that are directly related to your professional life if you’re not then going to engage with readers who comment on your posts? I’m disappointed, I must admit.
The key point is whether the amount of innovative, ground-breaking, “world changing” research has been adversely affected by the research councils’ progressively narrow (and increasingly ultilitarian) focus over the years. If you really think that bibliometrics alone provide an accurate measure of the health, novelty, and quality of science produced in the UK, then could I suggest that you read David Colquhoun’s excellent article: How to get Good Science and the associated blog post .
(Two other posts on David’s blog will be of interest:
How to get good science: again and Captain Cook applies for a grant .)
Here’s a study I’d really like to see RCUK commission: Compare the output and impact of pure responsive mode grants against those of the plethora of managed programmes/strategic partnerships/sandpits/Grand Challenges etc… that have been introduced over the years (normalised, of course, to the total amount of funding for responsive mode vs managed/directed programmes).
You might also ask yourself why, for the proposed Research Excellence Framework, there has been a steady back-tracking from the original concept of basing assessment solely on bibliometrics. As the Head of the Strategy Unit for RCUK, I’m sure that you can find the appropriate references.
Philip
February 14, 2009 at 9:32 pm
Steven,
When I logged on to post my comment above, David Colquhoun’s comment wasn’t visible. It was only after I submitted my comment that I saw that David’s had been added. Apologies for the “redundancy” – I make pretty well precisely the same point as David.
Philip
February 15, 2009 at 9:42 pm
Steven,
I’ve just posted this comment at the Prometheus blog. I’d also be interested in your response (from either a personal or professional perspective!).
Best wishes,
Philip
February 16, 2009 at 1:43 pm
Thanks to everyone for the comments – good to see the debate. Here are a few responses:
@David Colquhoun: I agree that there are problems associated with citation based measures of research quality. I guess the real question is how do we measure, objectively, the quality of research. Clearly peer review has an important role. Yours is a thought provoking comment and I will mull it over and post on the topic at a later date.
Take the point about international comparison. But if other countries do suffer from the same ‘impediments’ what is the cause of the change in Nobel prize numbers (if they have gone down here, they must have gone up in the rest of the world)?
I will also re-read some of your earlier writing as recommended by Phillip Moriarty.
@Amanda Goodall: Good point about leadership. Is your research work available to read?
@Phillip Moriarty: You have a pretty thorough answer to your questions over at Prometheus. I will only add to this by giving a personal view. I believe society funds research because of the benefits that arise. These benefits are extremely diverse, and include direct economic benefits, less direct benefits to the well-being of society (solution to climate change anyone?) as well as cultural benefits (like understanding our place in the universe).
February 16, 2009 at 2:05 pm
All my work is available on my website including a draft copy of the book – which is out in September (www.amandagoodall.com).
February 18, 2009 at 11:33 am
To Amanda Godall:
You said: “On average, the best universities in the world — that win the highest number of Nobels — are led by outstanding scholars.”
What do you mean, speaking “outstanding scholar”?
Can you find among universities that won Nobel Prize though one university which is led by Nobel winner?
To all discussants:
There is just other problem – at present, Nobel winners receive their Nobel Prizes not thanks to fact that their universities are led by outstanding scholars, and just despite of fact that their universities are led by non-scientists or non-outstanding scientists.
And such problem is merely reflection of whole critical situation in highest education and science, where the power still resides in bureaucracy and mediocrity.
You can make sure of the truth of my words, checking who run universities, where someone of scientists won Nobel Prize. Here is Nobel website. It contain all info about any Nobel winner, including data about their place of job. Try, please!
March 27, 2009 at 3:58 pm
In this connection it is interesting to note that one of the most famous Nobel prize winners Richard P. Feynman apparently did not have any grants of his own.
December 6, 2014 at 11:22 am
[…] hypotheses is written by the head the Research Councils UK Strategy Unit. He chooses to concentrate on the secondary matter of Nobel prizes, rather than answer the real questions. Fortunately Philip […]