UKH

Aaaaaaaaaaaarrrrrrrrrrrrrrrgggggggggggggggghhhhhhhhhhh!!!!!!!!!!!!!!!!!!!!!!!!!

Just wait for the KEF. You will be begging for a REF impact case study to review.

I used to piss some of my time away on things like writing bullshit about why what I do is so important, for some other people to refine it in to more bullshit, to be submitted in a corpus of bullshit to a large panel of people who will spend months reading and grading bullshit, before producing some bullshit numbers that go on to affect some funding down the line.

Now I do stuff and sell it to people who find it important enough pay me for it, and they use it to change the world for the better.  

There are clear links between what I do and the impact it has on the world beyond universities, and instead of making me piss my time away writing bullshit about it, I JFDI and I get money for it.

I'm so far failing to see a down side.  I was missing the teaching but a wise person on this site note that as things grow and I get minions, training them is very similar to the 1-to-1 part of teaching that I found the most rewarding.  It's just that instead of them paying for it, we pay them.

It's not that the grass is greener on the other side, it's that there's grass instead of bullslhit.

Perhaps in 5 years time I'll look back wistfully and with twinges of regret.

Jadedly Yours,

(ex) Prof WT.

ps - any academic reading this who like data, engineering, hard sums, biotech and so on...

It's a right laugh, isn't it!

I'm currently in a meeting with a presentation on ResearchFish....

Don't forget your Research Fish submissions are due soon also...!

I'm having a dose of IET accreditation next week also which is like some good bullshit icing to top off the cake.

What are you lot on about?  Are you in a Masonic Lodge or summat?

I think they are discussing the French REF at the rugby.

Thanks for clarifying....

It's the most joyous and momentous event that happens every 6 or 7 years that is well summarised by Wintertree's first paragraph (ironic smiley).

It is a way of spending about £250M of taxpayers money to work out how to spend £billions of taxpayers money.

I did some estimates a while ago and I reckon it consumes about 5,000 FTE years across the sector every round.  I thought about posting it before but it was such a preposterous number I thought it couldn't possible be right.   But this will expire with the pub so I'll risk it...

That's before counting the productivity loss from the head space intrusion REF puts on everyone, beaten home by lots of ominous messaging about how everyone eligible is going in, and nothing under 3* is going in, and how there's no stick.  Not that there's a carrot, either...

It will be over soon and then you can lock yourself is a dark room with a cold compress (or bottle of gin or rock, your choice).

.

.

.

.

.

Then you can reclaim your brain, probably with mild impairment some absolutely useless shite permanently lodged there like a 2nd WW shrapnel fragment that couldn't be removed.  It will play up when the weather turns bad but it will be ok (until next time).

5,000FTE years?  That's just my effort, what about everyone else?

I was thinking back along of trying to estimate the FTE effort so thanks for posting.

The bullshit is a fertile compost for the growing of flowers, which look and feel better in the grass..

We've already started planning for the next round. 

130 Unis x 40 depts gives ~5000

If you assume 1 FTE over the lifecyle of REF then it's about 2 person months per year.

That seems about right (depressingly).

Had to look up what it is. Didn't read far. You made the right choice.

Love your photo of the Matterhorn.

I've had the 'pleasure' of being involved in 5, and am on my 6th, of these smoke-and-mirrors exercises.  There are paradoxical trajectories: enthusisam has decreased and cynicism increased with each successive RAE/REF, whilst this relationship is closely paralleled by my increased role responsibility (from Lecturer to Prof).

I look forward to finally ditching this abject waste of everyone's time (and ££) very very soon.

I think its this: https://en.wikipedia.org/wiki/Research_Excellence_Framework

Might be wrong tho

Yep.  An insight into the academic mind from our favourite Hitler parody

youtube.com/watch?v=zF40XzwGA64&

Ah, that’s excellent!

Research Fish really is very embodiment of "busy work"...

I wish I’d not spent the time today reading about what KEF is.

We’re going to need a framework to evaluate the impact of all these frameworks.

It is, and it’s a f*cking nightmare. 

As a soon to be ex academic I agree for similar reasons.

That said, I was able to "sell" my REF submission for a 3 year pt contract, so not all bad.

I was on the REF ( or was it RAE?) panel for my subject in 2008. I recall getting up at 0430 to try to make some headway with the truly impossible amount of reading, and ultimately feeling a great lack of confidence in the end result.

You obviously made the right choice. I too am no longer an academic and oh how thankful I am.

I've got to own up to a bit of involvement in KEF (Knowledge Exchange Framework for the uninitiated) and all I can say is that it is not as bad as it might have been.

The KEF was announced by the Science Minister of the time (Jo Johnson) in a speech which surprised his officials as much as it did the HE community.  I wrote a little piece for one of the HE/research trade rags explaining (in the most constructive and moderate terms, of course) why this might not be a wholly good idea given the existence of other frameworks and some inherent difficulties in measuring the intended outcomes.  

A couple of days later I got a phone call from the official who would have the job of turning the minister's bright idea into an actual bureaucratic process.  "Oh David, you're ringing me up to give me a bollocking about that article, aren't you" I said.  "Not at all", he replied, "the article was great, and now I want you to help me make this thing work".  So I got to chair a group to design the thing, and we insisted on two things that have I think survived all the revisions and consultations, and I think should protect most academics from the worst effects.  

The first was that it should assess whole institutions, not departments or individuals, so any admin burden should fall on research offices/tech transfer offices rather than academics.  The second was that it should only use statistics that were already being collected, so no extra data collection or processing at the institutional level should be required.  Naturally I can't at all rule out the possibility that the thing will morph in the direction of more complication and more work but I thought those were two good principles to start with.

One irony about the last year is that the one person who could and almost certainly would have killed the REF was D. Cummings, whose vision was to be able to visualise all the relevant metrics (probably obtained via a lucrative contract with Elsevier) from his Cabinet Office mission control centre and make all the funding decisions himself.  

Cummings aside, I doubt funding decisions would be much different if  based solely on Google Scholar rather than  REF.

This is an issue that recurs every cycle.  The problem is that different disciplines have very different views on metrics.  The physical and life sciences would be pretty comfortable with just using field-weighted citation scores, the economists would carry on with their ludicrous journal rankings, but the rest of the social scientists would bitterly object, together with their colleagues in humanities. Computer scientists would agree in principle but would insist on the use of an enormously complicated algorithm of their own design.  

Well social scientists dont get any money anyway,  so as I say, no difference 🙂

Awesome!

" The physical and life sciences would be pretty comfortable with just using field-weighted citation scores"

You are wildly mistaken here! did DORA pass you by?

I hope you have something good lined up.  You may find quite a few colleagues just unable to comprehend that you can leave the system and life goes on.

Walking away from a good REF submission and an emerging impact case (these don’t transfer I believe, shame) is painful.  But I’m moving on from the sunk cost fallacy...  

A mental image I am going to struggle to shake.  I imagine Social Media Scores streaming in as well.  

It’s the combination of feeling you’re failing to do a sufficiently good job at something, despite going at it as hard as possible, because of all the other things you have to be doing.  It’s not a good feeling even when you believe the thing you’re failing at is important.  When you don’t buy in to it, it’s soul crushing.

As I age, I find I can context switch less well between lots of different roles.  Many others don’t seem to suffer from this, but I’m cursed or blessed with being sufficiently lazy that I only want to do things that I find interesting.  In the absence of that interest, it takes a lot more than REF to motivate me.

The way I see it, I make a choice and then I make it right.  I was probably a few years too slow in doing so.  A lot of my choice comes down to looking at the direction of travel and just being unable to imagine myself retiring in the job in the 2040s.  

I’m glad you can look back on your choice positively.

Not at all.  DORA says (quite rightly) that you shouldn't use journal based metrics (like impact factors) or judge articles by what journal they're published in, which I completely agree with (hence my comments about the economists and their misguided insistence on using journal ranking lists).  But it doesn't prohibit using appropriate article-level metrics.  I also think there's an important difference between using metrics as a management tool to judge individual researchers (which I am uncomfortable about) and using them at an aggregated level to assess whole departments.  

Of course my sentence was a sweeping generalisation.  But I've now been involved in two REF cycles worth of post-assessment analysis asking exactly this question, can we replace REF by metrics, so I've seen a lot of data and talked to a lot of people about it, and I don't think it was a wild caricature of attitudes held across the sector.

Somewhat unsettlingly, he retweeted me yesterday, so maybe I would have been ok...

I thought the main problem was the England players not adapting to the REF and conceding too many penalties. Itoje in particular. 

Killing the editors does sometimes sound rather attractive....

Anyway, it is ALWAYS reviewer three:

youtube.com/watch?v=-VRBWLpYCPY&

(another Hitler parody, which feels even more familiar)

I had this discussion before here (with Paul I think). 5000 sounds like a massive underestimate to me. I know quite a few departments who were borderline in reaching their institutional submission criteria who spent more time than 1 FTE,  just in meetings. The alternative experience presented  was much better from a successful department in a more fair and self confidently led Russel group institution.

I dealt with the political fallout of research assessment at UK level from the union side, at institutional level (on central committees) and on individual researcher cases. Nationally it was an expensive mess (too often unfair), institutionally good departments could be badly damaged or even wrecked and at the front line individuals could lose all research enthusiasm or at worse be destroyed, if submission decisions were handled badly.

Research assessment was especially damaging alongside the other side of creeping marketisation of the Universities: income from students. The UK was perfectly happy to see world class STEM departments fold; such that for instance when Exeter Chemistry was hit the closest remaining department for students who wanted to remain local or for potential growth of high tech related hubs, would be Bristol. When departments closed in the old days it was planned such that the best was distributed elsewhere. In 'the market' I saw entire physical departments lost to recycled scrap and only a small minority of high quality staff repurposed in other departments, with most being made redundant.

REF is like a tape worm it is massive, it is resource expensive and the host would be better off without it and might become very ill because of it. Even bad metrics would be better as the current system leads to too much crazy internal politics, eats time and money, can be soul destroying work, and it has seriously harmed good departments, and thousands of careers. This has been going on for a long time. The first casualty I saw up close was back in the early 1990s: a saintly star who developed some fantastic and very cheap self contained electrical control mechanisms for micro hydro that made a real global impact. After research assessment humiliation he left academia and set up his own company.  The EEE Engineering panel that round made some rather bizzare public statement about not enough government focus on blue skies engineering.

I thought the idea of KEF might be OK if the bureaucracy could be kept at institutional level as the knowledge transfer function is important for society. We in the UK are arguably up with the best in the world for blue sky research per capita but we underrate knowledge transfer and eventually let everyone else make much more social capital and economic growth from our efforts. It too risks the old 'pig test'. Pigs won't grow by just measuring them better. I remember a Bulgarian colleague telling me a long communist era joke on a similar subject.. the short version is a new train was disappointingly inefficient despite the best efforts of the brightest engineers and scientists because the politburo insisted on diverting so much steam to the whistle.

Oh yes,  good stuff to do.  I am not actually a cynic about HE.  For all the REF and Module Descriptor crap, its actually an unusually good work environment, and well paid despite academics/UCU endless moaning. There will be much less of this outside, but more of other similar rubbish instead.

Well paid? Just before my retirement my office mate left for Oz for the same level of academic job at double the pay. I had some of my best graduates earn more than me within a year.

Even at senior lecturer level salaries are £65k+ very generous pensions +excellent wider benefits.   At Professor level £75k+.  That, on average is more than outside most areas of academia, despite your anecdotes

You must know that's nonsense for the prospects of graduate engineers and be completely ignorant of academic pay rates in the other major economies. Top of SL(pre 92)/PL(post 92)/Reader is in the mid to high £50ks outside the London weighting and what was a generous pension is becoming more expensive by the year.

Bluntly it's not. I know from personal experience at several institutions.  I also know that outside academia, at least in my field, salaries for comparable levels are lower, particularly after pensions have been considered.  You are doing the typical thing of academics who have never stepped outside academia of thinking employer pension contributions of "only" 20% are poor.  Elsewhere anything above 5% is good, and that will be defined contribution.

That's because you are ignorant of pay structures and how they arise and the national variation. The nationally negotiated academic pay scales are here.

https://www.ucu.org.uk/he_singlepayspine

The top three spine points are often tied to strict performance criteria so top of the career grade was for most £56k in 2019. Lecturer grades start at about £27k.

UK institutions are in a bind given their market positions so pay for the top researchers is increasing but all salaries above the pay spine I linked above are on management grades and management contracts (and more insecure because of that). Exeter is a good public example:

https://www.exeter.ac.uk/media/universityofexeter/humanresources/documents/...

I know what my graduate cohort peers do now (very few relate to average engineer pay as most left that simple profession designation for management or running their own company long back, more than half have left engineering in any sense). For my own post 92 graduates. £56k was easily acheivable by the best in a few years, academics in my generation who were not research stars took twenty years from graduation to reach that top of the spine.

Christ you are arrogant. What are you saying - I imagined my salary? 

Those pay-scales aren't complete.  Here's Sheffield, showing Reader/SL runs up to £69k a few years ago.

https://www.sheffield.ac.uk/polopoly_fs/1.864469!/file/Aug19salariesfinal.p...

Those are management pay grades (as I showed for Exeter). Many readers in post 92 institutions are on the top band of the spine (grade 8 on that Sheffield link) and in some, some Profs are.

Wrong again.  Give it up.

You missed my point.  You asserted the physical sciences would be quite happy with FWCI.  My point was DORA pushes back against the use of metrics for research assessment and, as a movement, originated in the physical sciences. I see absolutely no evidence (from the physical sciences) to support your original assertion.

Ignoring the metrics argument, FWCI is a terrible indicator.  It is a noisy number divided by a (subjective) noisy number.  It's been shown to only have weak a correlation to traditional measures of research quality (based on reading papers) and we've measured a weak correlation in some areas.

It has (strike that, had) strong promotion from the commercial pedlars of metrics as a magic bullet but it's bollocks and it does not have the support of the science, medical and engineering communities.

Metrics are a tool, they have an appropriate niche and are unavoidable when aggregating results (almost by definition) but that doesn't mean you can make sweeping assumptions about their quality, suitability or level of support based on your gut feeling - as you well know these arguments are complex and require considered and nuanced dialogue.

I'm tempted to agree with you about the economists' system but I'd argue with you if you asserted that this system had grass roots support because it is as controversial as metrics in other subjects.

Good luck with KEF.

The nationally negotiated spine is, as I linked,  only up to £61k so anything above that is out of national negotiations and so is a locally determined management grade and so a less secure contractual position. Most staff would be happy with the quid pro quo involved in that (at least until something went wrong).  Sheffield can call these grades what they like to deal with their particular market position but pre92 SL in national negotiation terms is band 8 (with three optional increments on top). In post 92 band 8 was traditionally called PL and SL is band 7 (...all very confusing). Various forms of Associate Prof type labels are growing these days.

My colleague who left for Oz was low in your Sheffield band 7 but close to our (band 8 equivalent) Reader criteria (assessed 3* or above papers, above average REFable income for the unit, some research leadership/contribution to impact case studies).

It wasn't entirely gut feeling.  I was on the steering group for this review

https://re.ukri.org/sector-guidance/publications/metric-tide/

so I'm aware of the arguments, I saw the complete article level correlation that HEFCE did of various metrics and the actual REF scores, we interviewed every REF sub-panel chair, we held open meetings across the country, and I think I had a good sense of the views of the membership of the organisation I represented.  I completely endorsed the conclusions of that review (and indeed had a hand in drafting quite a lot of it, especially the one on metrics in HE management).  

Ok - so I think we agree that  "the physical and life sciences would be pretty comfortable with just using field-weighted citation scores" is not supported by the evidence?

There was a typo in must last post which should have read "we've measured a weak _negative_ correlation in some areas".

I find it very Jeckyl and Hyde; for the pay (including pensions) it's a very cushty job in many ways with a level of autonomy that doesn't seem real to people in other sectors.  But there's an opposing side as well.

I gave a very negative take on REF from a personal perspective.

To give some balance, clearly some form of accountability is needed to understand how well public money is, or is not, being spent on research.  This is clearly going to need a Machine to achieve this.  Designing a machine that doesn't end up a net drain on research time and on money is not simple.  

I also worry that fundamentally, the accountability is not compatible with the idea that some research must fail, because if nobody is risking failure, there is no proper blue skies research going on.  The inevitable feedback between the results of the Accountability Machine, the fascination with league tables and management means the outputs of the machine will inevitably loop back to guide what researchers do.  

I don't know what the answer is sector wide, only for myself.  It's not a trivial problem to address by any means, and it's nice to see quality discussion on this thread around what's going on to move this on.

No, we don't agree.  We agree that replacing the REF by metrics is probably a bad idea on balance.  The question is whether that view is widely held in the physical and life sciences community.  I don't suppose either of us has done an opinion poll, and of course there's the question of who speaks for the community and whose views get weighted more, but I've explained the basis for my belief.  I'll also note that this is on a thread where many people have weighed in on what a burden the current process is, what negative side effects it can have, and where a panel member has expressed a lack of confidence in the judgements made.

There is a further point.  A lot of the objections to metrics are when they are used to judge individual performance because of <endless special pleading reasons>.  This is different to judging the performance of a group, let alone a university, where they may be more justified, certainly if it removes endless bureaucracy. 

Rather late to the party but also particle physics would find funding based on citations grossly unfair. Some of my papers have 3000 authors each with roles in building or operating the detector. I moved to a different experiment and now I'm only on the journal output of a community of ~500 so get a lot fewer articles than I used to.

Hint the number where I actually write parts of them rather than contribute to data collection has stayed the same

The key issue is that the job of the REF is fundamentally to do two things - create a fair basis for distributing research money to universities, and to provide accountability to government for what that money is used for.  All the problems arise when people try to use it for other things - to create league tables, and (especially) when university managements use it as an internal research management tool.  What is happening then is that they are using it as a substitute for the judgements they should be making themselves about what's important for their own particular institution, and how to balance the institution's different goals.  It's only if they're clear about that, that they can they properly assess the different contributions individuals make to the institution, without devolving those judgements to an algorithm designed by somebody else for a completely different purpose.

This is a real problem.  And related to that, one of the big problems of citation metrics is that a big part of what they measure is how fashionable a particular sub-field is.  Sometimes areas are fashionable for good reason, lots of rapid progress is being made and worldwide competition is producing loads of great new ideas to follow up.  But sometimes you've got herd-following behaviour and too much handle-turning me-too research in those areas.  On the other side of the coin, you sometimes have quiet persistent work in unfashionable areas one day producing results that are genuinely transformative.  None of this is easy!

I'd say research herding is the norm, not sometimes, and REF makes it worse as the subject 'establishment' who get most of the money have an inevitable interest in ensuring their position. It makes development more of an incremental type with way more publications than needed for the same research output.

Time and time again some of our top stars say they would never have done what they did under the current REF rules. The vast majority of top researchers I have met are incredibly driven people who work so hard that its often a struggle to get them to do anything else, sometimes even to the extent their health suffers. The reality of REF for them is the opposite of their motivation, it's highly time consuming, frustrating, soulless bullshit. The aims of accountability are fair enough but no other country provides anything like this as a solution. You know full well a much cheaper and much less time consuming system could be based on larger scale metrics using best practice from elsewhere: something harder to game and with nothing like the pressure on individual researchers.

Our REF related ills are not usually exclusively the fault of REF but the lack of protection for staff who don't fit the mold is a national scandal. Every other major nation has tenure for senior research staff; we have increasing numbers of such staff on very insecure management contracts, as it's the only way we can compete on anything like reasonable international comparators.  You can't fairly blame the institutional management as they are told to operate in a market. You should expect the institutions (especially in the middle) to obsess and game, given the money and prestige involved. The system stinks.

We were once told subject assessment was vital for accountability in our Universities. When that similar idiocy stopped who lost out as a result?

Thanks for contributing to the discussion as I think it its incredibly important. With brexit, covid, fees, pension stress, IT disruption and the rise of courses taught in English across Europe it's part of a perfect storm our institutions face. We are arguably the top of the world as a sector and it would be tragedy if we lose that down to populist idiocy that claims it wants to celebrate what makes Britain great.

Here I am having promised myself a day off from ranting and you go and publish something like that...  

A couple of years ago my day was spoilt by someone going in to a meltdown because I refused to include negative Kelvin temperatures in to the very first tutorial the students would have.  I'd not heard of them before.  10 minutes later I was pretty pissed off.  There have been several cycles of publications and counter publications on the issue, each inspired by applying a particularly definition of temperature to inversion systems which are non-adiabatic and being held far from an equilibrium state.  The cycle seems to repeat when a new class of inversion system comes along.  It's not just an occasionally trendy field it's basically a load of crap, and the flurry of publications it inspires cycles everyones metrics way up.  

Not for the first time, I thought allowing citations to carry a score of +1 or -1 to the cited paper would go a long way to levelling the field (*).  One thing REF panels can do that citation counts and other data driven metrics can't is to contextualise citations beyond simply the norm for the sub-field they're in...

(*) in reality it would be an awful bloodbath.

At least subject Level TEF was strangled at birth earlier this year.  Though I'm sure some form of it will re-emerge vampire like from its coffin in due course.

Ah TEF, the most dishonest quality assessment system in HE history, of which The Royal Statistical Society said:

"The TEF is in large part a statistical artefact, and we are concerned that it does not meet the standards of trustworthiness, quality and value that the public might expect. Indeed, the statistical issues are so major that, in our view, the TEF is likely to mislead the public and, in particular, mislead students who use TEF to inform their university choices."

https://osr.statisticsauthority.gov.uk/correspondence/concerns-regarding-te...

We might ask ourselves in what kind of banana republic could Universities have allowed such a fundamentally flawed system to be put in place. What happened to the defence of academia by its leaders?

My working theory is that when you go for a job on the management side, you have an unexpected date with a mind control slug.

In my more depressed moods I get a picture in my head of VC Mutley with a government Dick Dastardly holding out the prospect of a knighthood. Sad thing is I've known quite a few very able people who are mainly just guilty of not speaking truth to power. A few braver VCs do post sense sometimes. Other times we get the lack of the first class carriage to Aberystwyth scenario to bring us back down to earth with a bang.

I agree that there's a huge amount to worry about in HE in the UK at the moment - the pension situation, a patently unsustainable student funding system, research only financially viable through a massive cross-subsidy from overseas students, whose continuing presence can't be relied on, and a government whose attitude to the sector has evolved from lack of sympathy to outright hostility. In all this I don't think that REF is anywhere near the biggest worry (or even the worst source of annoyance, we've got ResearchFish for that).

No-one enjoys writing an environment statement, and I'm sure everyone involved will be heartily pleased when March 31st passes.  It's an epic job to be a panel member, and institutions really do need to take that into account when they think about panel members' other workloads.  But I do think you overstate the overall burden of something that happens once every seven years, and I think lots of the ills you attribute to the REF are things that the sector does to itself, as I wrote in yesterday morning's post, and reflect poor and lazy management.

I say this having seen REF both ends of the institutional telescope - I've been submitted in every one since I became an academic in 1989, I had an impact case study in 2014, I've been through one cycle as a head of department, and for the 2014 I had overall institutional responsibility for the submission for a Russell Group university.  Possibly my experience is different to yours in that I've been in bigger institutions with more economies of scale and better thought through support.

I don't "know full well" that a better system could be done on the basis of metrics (and as I said above I've thought about it in quite a lot of detail).  I think you could run a metrics based system that would work for STEM, once.  It would disadvantage the more practical engineering subjects and healthcare research (even further locking in what's already too big an emphasis in the UK system on elite biomedical research), and it would completely disenfranchise non-STEM subjects.  And I think the subsequent management response would create even more distortions.

This is pretty much what the computer scientists wanted.  Yes, we like citations, they said - but we need to assess the quality of the publications that are doing the citing, so we should do a citation analysis on the citing publications, which of course needs to take into account the quality of the publications that the citing publications were themselves being cited by.  I wasn't convinced that the process would converge!

Sounds like they need Google pagerank for papers

This is the problem really; REF has been around long enough that particularly excessive management enthusiasm without thought for individual consequences has been tempered.

Any new system is tantamount to throwing another bucket load of hand grenades in to a room full of monkeys.

Although I do think something better than REF must be possible and I’m glad it gets so much thoughtful consideration.

With a side dose of Twitter analytics.

<“for a Russell Group university”>

Ah - does that bring with it increased workload, responsibility or expectation? What meaning am I supposed to take from this? Or is it some sort of academic willy-waving?

You can take what you like from it but it was intended to convey a sense of the scale of the effort, in terms of numbers of departments and researchers involved, and the range of subjects, from clinical medicine, science and engineering through social sciences and humanities. It's going to give a different perspective to the one you'd get from a more specialised institution or one with a smaller number of research units.

The irony here is that the original Google algorithm was inspired by the citation networks of academic papers. I'm sure Google could do this but their habitual lack of transparency would make them unacceptable for a process whose outcome is distributing a few billions of public money. (Well, that used to be the case, maybe less sure now).

The Metrics Review document I referenced above did explicitly look into what get called "alt-metrics" - which pretty much does amount to Twitter & Facebook analytics. Probably fair to say the conclusion was that the field is immature...

I met the top performing Prof from my old place yesterday on my daily walk; he is doing very well. Apparently nothing has changed in the obsessiveness that you say is bad management but I say such response is inevitable given all the pressures (to do as well on REF as possible given more than a hundred competing Unis) in a market system. There is some real bad management beyond the direct pressure: I was very sad to hear two more great Profs have left in the last six months as they don't fit the research performance criteria of REFable income above average for their unit. Their total income and assessed paper qualities and impact input were excellent.

I respect your efforts on REF but I think we will always disagree: the costs in time and money are too high in too many institutions and dispiriting so many good researchers is a very bad idea. Maybe part of that is I think perfection is the enemy of good/progress (Voltaire/Churchill)....by better I mean better value and minimally disruptive ... if what we do in the UK to account for public spending is such a great idea why is no-one else doing things the same way?

I notice you didn't say anything about my points on TEF...maybe sensible

I can think of nothing positive to say about TEF!

”REFable income“??

is this a whole new angle I am somehow unaware of?

Not all research income counts towards a REF assessment.

Somehow I had not even realised that income had anything to do with it 

Edit: I guess it says something good about my department, that I didn’t know that 

Indeed! To be fair I had more than a hundred meetings either discussing/negotiating policy, or representing/accompanying researchers, since my old place decided to put such staff on REF performance criteria, so I was a bit more familiar than most! Trouble with forcing staff to meet tough targets is many will focus on them to the detriment of other important work, others will say "stuff it.. give me a teaching contract" and some will just leave or retire. The idea that our permanent research staff had to be above mean for most or all criteria for their unit of assessment was pretty grim for a median level uni.. As per the disagreement with MG above, our staff were on average a pay band below those in most Russell group institutions (and had more teaching hours, lower school support budgets and normally lower quality lab access).

So what gets a thread shunted from The Pub to Off Belay?  Bit dodgy that given the number of universities who employ Social Media Police.

For all my grumbling up-thread, I am very lucky that my time has been in a department that treats its staff very well.  The idea of an institution forcing REF inspired targets down to individual staff is very bad in terms of staff welfare and mental health.  Utterly self defeating from an institutional perspective I suspect, but it makes look like We Are Doing Something.

The research from the UK's university sector is a real jewel in the national crown, and I worry about the direction its going in - critically for blue skies research but also the culture changing in a direction that isn't productive for research.

Fun fact: the funding landscape is so distorted in my subject that the Russel Group mean income is higher than the upper quartile boundary. This rather makes a mockery of the university's promotion criteria...

Do you have a link? That sounds slightly odd for average individual income given the Russell group dominates individual submissions... must be more than a quarter of all submitted staff in some units.

Another part of the gaming for some median level institutions was to get large numbers of research active staff not upto the level defined as required, to shift en-masse onto teaching only contracts. I'd hoped everyone permanent in a department was going to be forced to be submitted from news early in this REF cycle, then at least everyone on a permanent research contract, but seemingly neither apply now.

Apologies, I should take that claim back as I can't find the evidence for it. I believe I saw that in an internal presentation, but I can't find any current and public data on the RG mean, only median and upper quartile. And I was also wrong about promotion criteria, which (now, at least) appear to refer to median and upper quartile, not mean and upper quartile as I had recalled.

No worries. I was after such links as further evidence about how things are going wrong, not questioning you.