Jofish Kaye on industry research, having an impact, and values-driven decision making

Jofish_Kaye.jpg

Jofish Kaye is a Principle Research Scientist at Mozilla, and before this he worked at Yahoo and Nokia. Jofish made a deliberate decision not to pursue an academic career after he finished his PhD and it’s interesting to hear how his decision-making criteria evolved from being primarily about the people he could work with to being more values-driven and being able to make an impact. A strong sense of values and having impact are threads in a lot of what he talks about. He also discusses his experiences more generally working in an industry context and also moving into more management/leadership roles.

“I think I’m the only person on the planet who likes job searches because you get to re-invent yourself.”

“I am concerned the way we treat publications as the way to make success in the world.”

“It’s so important and so incumbent upon research as a field to make clear and visible how valuable what it is we do.”

“We need to be taking seriously this call for public outreach.”

A full transcript is coming soon!

Overview:

Jofish discusses (approximate times):

01:38 Getting a PhD at Cornell and moving into an industry job at Nokia and being able to teach at Stanford

09:24 Why he didn’t want to apply for an academic position – the difficulty getting funding vs the freedom to do what he wants in industry, the current Mozilla grant process and research they have supported

19:16 Triggers for moving to different companies, looking at what he really enjoyed doing (CHI4Good), and seeking out a way to do that – the job search as a way to reinvent yourself

25:11 Moving from more of an industry research role to now also being concerned for shipping product to customers and having impact in the world in a different way

30:55 How his thinking about job searching has changed over time, from thinking about the people he would work with, to more values-driven decision making with some additional criteria

36:00 Broader accessibility for young people to universities, and the role of public universities,

38:40 His usual pattern of working now with kids/family; and experiences being in a management role, recruiting people, and the ‘Noah’s Ark’ theory about having people who share the same assumptions

42:00 Being a leader and manager – managing as administration, checking boxes, etc; leading as trying to build a strategic narrative and the difficulty of coordinating with people who have different epistemological assumptions and how you measure impact

50:45 Practical team strategies when people are distributed, combining in-person and online techniques, daily video ‘stand up’ meetings

57:18 Challenges around issues of diversity and inclusion across the industry and in particular how to improve diversity in an open source volunteer community

1:01.40 Challenges for academics moving into industry, getting to actionable insights quickly and how to communicate those in the slide deck (the coin of the realm)

1:07:38 End

Related Links

Phoebe Sengers - http://www.cs.cornell.edu/people/sengers/

Elizabeth Churchill - http://elizabethchurchill.com

Wendy Ju - http://www.wendyju.com

Pam Hinds - https://profiles.stanford.edu/pamela-hinds

Terry Winograd - https://hci.stanford.edu/winograd/

John Tang - https://www.microsoft.com/en-us/research/people/johntang/

Jed Brubaker - https://www.jedbrubaker.com

Allison Druin - https://www.pratt.edu/faculty_and_staff/bio/?id=adruin

Casey Fiesler - https://caseyfiesler.com

Anna Cox podcast - http://www.changingacademiclife.com/blog/2017/3/5/anna-cox

CSCW Medium posts - https://medium.com/acm-cscw

DeleteMe - https://abine.com/deleteme/

TallPoppy - https://tallpoppy.io/

Lindsay Oades on academic wellbeing, connecting to strengths, meaning and purpose, and not taking the system too seriously

Oades.jpeg

Lindsay Oades is a Professor at the University of Melbourne, where he is also the Director of the Centre for Positive Psychology at the Melbourne Graduate School of Education.  He has co-edited the Wiley Blackwell Handbook of the Psychology of Positivity and Strengths-Based Approaches at Work. I caught up with Lindsay in Budapest at the 2018 European Positive Psychology Conference and was keen to talk to him because of his expertise in positive organisations and taking a systems perspective to promoting wellbeing at work. In this conversation we talk about his own experiences of changes in the academic sector, and his key learnings getting to full professor. We also talk about what positive psychology can contribute to academic work environments and wellbeing, covering issues around values, purpose and meaning, strengths, promotion processes, performance reviews, job crafting, and academic leadership. Listen out for his great terms like ‘academic feudalism’ and ‘justificationism’.

We got so caught up in the conversation that neither of us noticed that his microphone had dropped so there is about 5 mins towards the end when he is talking about job crafting. If his distant voice is too difficult to hear, stay on to the end of the podcast where I repeat what he said word for word. The verbatim text is also below for that section.

“Don’t take it too seriously, don’t get sucked into the rumination and the competitiveness that people go through, and the valuing of each other based on the academic gaze.”

“A lot of academics mistake seriousness with excellence.”

“I…coped through…humour, patience, relationships, being in good teams, being quite purposeful…about why I was doing it, so I didn’t have an instrumental view of academia of publications for publication sake, grants for grants sake.”

“Academics love autonomy. The best way to manage academics is to get out of their way.”

Overview:

01:30 Background

09:00 Changing challenges of academic life

16:45 Key learnings getting to full professor

25:30 Values, purpose, meaning and the promotion processes

32:40 Well-being and academia, and how considered academics create to absurd systems

41:00 What Positive Psychology is about, and how it impacts his management role

50:05 Taking a strengths-based developmental approach to performance reviews and job crafting

1:02:57 Final thoughts – towards the positive university

And in more detail, he talks about (times approximate) …

1:30 Lindsay talks about the different phases of his career, from clinical psychology, to doing an MBA and then moving to a business school, and now moving “from negative to positive, from individual to larger system”, an evolution in scale, and what systems thinking offers for him. From health to wellbeing to business to education.

Changing challenges of academic life

9:00 The changes he has seen in academic life over the course of his career – huge. Increased in student numbers, internationalization, reduced funding, more managerial/commercial style, contracting of competitive funding, freezing of PhD scholarship levels and students having to work much more. Quite a different place. What hasn’t changed is the undervalue that the Australian culture places on academics. Anti-intellectualism. Thinks intellectual life valued more in some parts of North America and Europe and popular media. Changes have led to fewer positions, skepticism about ability to develop careers, larger teaching loads, multiple people scrambling for small amounts of money (academic feudalism). “You see these so called good minds spending huge amounts of time to get access to $10K…relatively small amounts of money”. “A lot of academics are very detailed oriented people, what I’d call naive rationalists, they think they are going to get a solution through reasoning and then get frustrated when politics or economics knocks them around.”

13:30 Own experience? His academic vantage point quite different as professor and director of a centre. Reflects on when he was a lecturer, dealing with teaching load and applying for funding, but was doing more applied research so used consultancies as a way of generating funding to side-step the feudalism. A deliberate decision. Institution allowed him to have a slush fund. But not all academics or disciplines are able to do this. Still went for competitive grants but now with a base level source.

Key learnings getting to full professor

16:45 Key learnings getting to full professor? Patience. Not taking the system too seriously because academic life can be very disheartening. A lot of academics would say this, that they feel very undervalued by their own institution and most of the recognition they get is from people they don’t see, from overseas who recognize the quality of your work, yet in your own institution you are told you are not producing enough or teaching enough classes or whatever. So this weird local invalidation and validation from someone a long way away. So don’t seek validation in the wrong place. And remembering what a university is, this incredibly resilient organization. They’re 8 or 900 year old institutions. They do this partly through the slowness of themselves. A lot of academics mistake seriousness with excellence. The constant workload and multiple roles that academics have to cross between teaching, research, community engagement and administration, without a lot of understanding – most think of academics as a teacher. So no real understanding of what academics do. What he learnt was probably a light touch, non-grasping view of what it is, don’t take it too seriously, don’t get sucked into the rumination and the competitiveness that people go through, and the valuing of each other based on the academic gaze. Finds it comical at times. Valuing the absurdity.

21:45 Need to find good mentors, get into good teams. A lot his good research output is from being in good teams. And a healthy skepticism and sardonic humour. When he was younger, he felt academia was ageist. Couldn’t achieve criteria for professor unless you had time. “I’ll keep doing what I’m doing because I’ll get to professor anyway because age will take of it.” So somewhat of an ageism in the way it is structured, the system values declarative knowledge that comes with age. So he probably coped through a bit of humour, patience, relationships, being in good teams, being quite purposeful, “I’ve always had my own purpose about why I was doing it, so I didn’t have an instrumental view of academia of publications for publication sake, grants for grants sake.” So a non-instrumental approach. Care about it. Always been attracted to ideas and learning. Love of learning is one his number one strengths. Conceptually strong. Good with ideas. That comes naturally, easier for him than some other people. That combined with a value and purpose for why I’m doing it, that has buoyed him along.

Values, purpose, meaning and the promotion processes

25:30 In a team at Centre for Positive Psychology at the University of Melbourne, 17 people. A very values and purpose driven group of people. He has some very clear things he is working towards, helping other people, changing systems in service of well-being. So quite purpose, meanings-based initiatives. Keeps those close. And reminds himself. So no surprise he is attracted to ideas like impacts rather than h-indices and metrics of how we stack up against others. One of the frustrating things about that when going for promotion- it is very extrinsically focused. He didn’t like the psychological impact because it took him away from what he valued about what he was doing. But having to report on all the extrinsic things that don’t connect to love of learning or meaningful impact you are trying to have. [27:40]. Lower down the tree it was the external impacts. But now at professor it was about being able to get on committees, have an impact. He calls it rampant justifactionism.

29:07 His ideal promotion process? Prefers whole of career approach, more portfolio-based, less constrained of how you have to fit yourself into a box. Stories would provide more mechanism for people to tell their stories. Using other media to make the case in more variegated and meaningful ways. From a managerial point of view, one of the ways to exploit the workforce where people love their work. It’s a strength of the workforce but also makes it easier to exploit them. It’s a danger for people who love what they are doing.

Well-being and academia, and how considered academics create to absurd systems

32:40 Well-being impacts? Has been involved in surveys of academic and managerial staff. Academic experiences different to other sectors. Has seen in the data academics have high levels of workload and stress but reasonably high levels of job satisfaction. That says there is another variable accounting for that – some value they are getting through their work. Meaning, impact, connection. And not the place to go if money is your key driver. The triggers for the stress? A lot of factors – individual, institution, department. Which institution, which faculty? Different pressures. At the individual level, obsessiveness, narcissism, perfectionism – we see these in academics, we select for these qualities too. Overthinkers, good but if overused it is problematic. All these things play out. “One-on-one I find academics generally very nice people, easy to relate to, usually quite kind and considered people. Yet the systems we create and inherit can be kind of absurd.” And it is at the individual level, the considered academic is good. But put them in committees to make decisions and they can’t make a decision and they develop systems that provide justifications. So the systems they create are not that effective. The effect is that it slows everything down. So one-on-one good people, well-intentioned people, smart people, but not always smart in the sense that they understand organizational life. Some serious problems with that that need re-dressing.

What Positive Psychology is about, and how it impacts his management role.

41:00 Positive Psychology – science of optimal human functioning, taking a strengths-based approach in the service of wellbeing. Historically a re-dressing of a deficits-based focus of psychology.

42:55 Impact of PP on how he plays out his role? All understand the language, have the expertise. But rest of the uni don’t have that language. And still a knowledge-behaviour gap in how they manage their own wellbeing, purpose etc. Everyone in the team has a wellbeing coach, wellbeing in the context of the strategy of the centre. Some take more a physical health approach. Others trying to manage their own perfectionism, change their mental attitude about how much they have to work. Ever since he had kids, he doesn’t work weekends. When he told team members they were shocked because they had themselves in the habit of working weekends. Not a sustainable practice. The critical point for him was having kids.

48:05 Another example: they have 8 people here at the conference, an expense to the centre, his view is that there is a wellbeing component to it. “My problem with my staff is not do they work hard, but do they work too much.” So this is an opportunity for them to have time to get sustained, rejuvenated. Not about reductionist managerialism/ROI.

Taking a strengths-based developmental approach to performance reviews and job crafting

50:05 At performance reviews, ask people what are they really trying to do, where are they trying to go. Have authentic candid conversations about what do people really want to do. What’s in this for them. People are varied. How do we enable different career trajectories? About knowing the people you are working with, and appointing them to match the role you want them to play. A problem though in the way universities appoint. He hasn’t formally done strengths-based recruitment but they have done teams-based strengths assessments with VIA and Realise2. Get individual profiles. And also get a team-based profile. “Academics love autonomy. The best way to manage academics is to get out of their way. If you want to manage a wild beast, give it a large paddock. …Academics love autonomy but they also love a rationale.” What Self Determination Theory tells us about this.  Autonomy doesn’t mean anything goes. Have some external research income targets to hit. Not negotiable. How do we do it. Then let the smart people do it. Don’t tell them they have to have micro-managed parts. They’ll usually find a way.

55:50 [Lindsay’s microphone dropped down here so the audio is not so clear. Here is a word for word transcript as best I could hear]

56:23 You have to do this both individually and as a group and I’ve been trying to push this strategy document so people can see where they fit into where we want to go. And that takes time.

[Turning the lens back to academia?]

It sounds really trite but the evidence bears it out. Fundamentally people at work often feel undervalued, in general or by their immediate boss. So simple things about what do you actually value about your staff and have you told them and in what medium have you told them. So that is number one.  And number two would be the stuff we talked before about strengths. Have you actually had conversations with staff about their role and the job description and how it can be crafted so that they can use their strengths more than they currently are. And that might take time as well because there are organizational constraints, that you have to deliver this or get this class taught or we’ve got to generate that income or we’ve got to get that contract done. So while at this moment we can’t get you exactly fully there at least have that conversation so there is a plan of how it is going to migrate there and those conversations are really important. Because again with academics, if there is a rationale and there has been a conversation, they will probably accept it for a while if there is good intent. So there’s a couple of things there, enabling them to feel valued and enabling them to use their strengths and mould their work, job craft their work from a strengths base.

[Doing that for each other too?]

59:03 I think too if you look at the history of the universities as well, they’ve been gendered so you have rationalist males that might not see the value of some of the stuff I was just talking about.  And […] they might not have had the skills for how to do it. And I don’t mean that in a nasty way. People have different skills. If academic life was originally a very cognitive, individual endeavor, you go into your room and do your work. That was old academia for a lot of people. This new academia, looking after people, many many more women in the academic workforce, also culturally much more externally focused than it used to be, much more community engaged, more demands from students, I wouldn’t say more demands, students have been enabled to give more feedback and they do expect a higher level of teaching quality. So a whole range of things that are different to how they have been.

[Loved job crafting, same job, but control, choices] And by job crafting I don’t just mean offloading your teaching. [Specific example of job crafting?]

1:00:53 Yeah there are a few. In academic life there is obviously research and teaching but the …it may be changing the type of teaching you are doing at a subject level or also gradually doing more research led teaching or face-to-face teaching or particular type of teaching like workshop style, lecture style. Or gradually trying to move to more admin and leadership roles but doing in a way that uses my particular skills or strengths. [end of lost mic – shorter notes continue]

1:02 So there are different types job crafting might look like – tasks, relationships, So different forms of what job crafting can look like. So different ways. Enabling people to take charge of their work life, their career. Academics are sophisticated people. They think a lot and they are willing to work hard. So it’s about capturing that.

Final Thoughts

1:02:57 Currently trying to champion the idea of positive universities. People usually just think of student wellbeing. But it is broader than that – student wellbeing, staff wellbeing, positive organizational practices. How do we take science of wellbeing approaches and apply them to universities? A group of universities around the world currently thinking about it. A bigger picture way of looking at it. He has a paper called “towards positive universities” about how to do it at a tangible level. When people talk about wellbeing, they think it’s the positive experience, feeling happy, but don’t take the functioning bit. Wellbeing from a eudemonic perspective involves positive functioning, growth, virtue. Wellbeing includes good functioning, not just feeling good but functioning well and doing well. That’s where the meaning and purpose part plays a big role. Big changes coming. Universities resilient, they adapt. Not as simple as the commercial arrangement would suggest.

Student wellbeing programs still deficit focused. Working on wellbeing literacy. We don’t have a way to communicate about wellbeing. Positive attributes. More than the absence of anxiety and depression. Wellbeing in the broader sense –where students can communicate about what is self-regulation, what is using strengths, what is wellbeing, what is meaning, what is purpose, and communicate in a way that is meaningful for them. Having senior leaders able to see this relationship between wellbeing and performance and communicate this to staff and students explicitly and implicitly.

01:10:46 Repeat of the content where Lindsay’s microphone dropped

01:14:43 End

Related Links

Lindsay Oades: http://www.lindsayoades.com

Centre for Positive Psychology, Melbourne Graduate School of Education: https://education.unimelb.edu.au/cpp

9th European Conference on Positive Psychology 2018: https://ecpp2018.akcongress.com  

Barbara Frederickson’s Broaden and Build Theory: https://en.wikipedia.org/wiki/Broaden-and-build

VIA character strengths: https://www.viacharacter.org

Strengths Profiler Realise2: http://www.ppquarterly.org/portfolio/realise2-next-generation-strengths-assessment/ Now Capp: the strengths experts https://www.capp.co/Home

Self Determination Theory: https://en.wikipedia.org/wiki/Self-determination_theory; http://selfdeterminationtheory.org

Job Crafting: https://positivepsychologyprogram.com/job-crafting/

Book & Papers:

Oades, Steger, Delle Fave, Passmore (eds), “The Wiley Blackwell Handbook of the Psychology of Positivity and Strengths-Based Approaches at Work”

https://www.wiley.com/en-gb/The+Wiley+Blackwell+Handbook+of+the+Psychology+of+Positivity+and+Strengths+Based+Approaches+at+Work-p-9781118977651

Oades, Robinson, Green & Spence, “Towards a Positive University”: https://www.tandfonline.com/doi/abs/10.1080/17439760.2011.634828?journalCode=rpos20

Oades & Johnston, “Wellbeing literacy: The necessary ingredient in positive education”. https://juniperpublishers.com/pbsij/pdf/PBSIJ.MS.ID.555621.pdf

James Wilsdon on impacts, responsible metrics & evaluation practices

James-Wilsdon.png

James Wilsdon is a Professor of Research Policy in the Department of Politics and Director of Impact and Engagement for the Faculty of Social Sciences, and Associate Director in the Grantham Centre for Sustainable Futures at the University of Sheffield in the UK. He has been involved in many policy and think tank initiatives. Of particular interest here, he chaired an independent review of the role of metrics in the management of the UK’s research system, publishing a final report in 2015 called The Metric Tide. More recently he has chaired an expert panel on Next Generation Metrics for the European Commission. In this conversation we talk about his experiences working in both policy think tanks and in academia, about the increasing focus on research impact for academics and how the UK has created some culture change in this direction. He also discusses issues around metric-based systems of assessments for academics and calls on us not to indulge processes of evaluation that we know empirically are bad science.

“Impact is a team sport.”

“A new breed of brokers and boundary spanners … placing a premium on a skillset that is not the traditional academic skillset.”

“Metrics are a technology and there is nothing intrinsically good or evil in them, it’s all about how they are used.”

“It is incumbent on us not to indulge processes of evaluation that we know empirically are bad science.”

He talks about (times approximate) …

01:40 Introduction of background as professor of research policy, politics of science and research and director of research impact for faculty of social sciences; and working outside of academia as director of science policy for the Royal Society

03:40 Moving from an academic context, working out of academia in policy jobs, and keeping a foot in academia through PhD and collaborations, and then coming back into the academic system proper; not being strategic about PhD and future plans when at the think tank; bridging brokering skills becoming more valued as academia more concerned with impact

06:55 Moving from think tank to university – pluses and minuses of both; pace and speed of think tank, shorter cycles, but can be too swayed by pressures of speaking to think tank audiences; in university time for longer deeper research when you get the funding; just different; think tank more proximate to power and potential to impact policy debates, in university setting harder to earn that seat at the table; impact.

10:30 About having impact as an academic? His role is facilitating academics having impact, part networks, part credibility; for faculty supporting academics at different career stages to strengthen their approach; also in the UK, the Research Excellence Framework (REF) that has 20% of its weighting on impact and needing to think about impact case studies now for next REF cycle; an industry of box ticking around the REF just as much as anywhere else; argues reason to do impact is not the REF but to have real impact, as starting point, so starting with the substance

14:30 Describing REF – institutional assessment done at disciplinary or departmental level, university makes subject-based submissions to a particular panel eg politics that assesses research outputs over 6-7year period of all the politics departments in the country in their area and scores accordingly; 65% on research outputs, primary unit is journal article, 15% about research environment, 20% on impact, here through narrative case studies. Not all academics expected to have an impact case study, usually 1 out of 10. Real money attached to it, as research funding allocated to universities on basis of scores, strategic research funding very valuable to institutions.

17:42 At what costs? Huge debate. Considerable amount of effort. Have just gone through a government review of the exercise, led by Lord Stern. Conclusion was exercise was working effectively and valuable because a trusted accepted mechanism on both sides and provides the accountability for allocation of substantial money. On uni side, while cumbersome and takes a lot of work, a self-governed process. A lot of the debate rests on what’s it purpose is it good value for money; if purpose just to allocate that grant could do it with a lighter touch or purely metric basis. Reason for Metric Tide review.

20:24 REF as it has evolved, now been through successive cycles since the mid 80s and it (REF) has now taken on range of purposes: allocation of funds; accountability mechanism; benchmarking function; driving culture behaviour change through the uni system, affecting wholesale change. In Thatcher times, focus on improving productivity of unis and still has pronounced effect eg UK has most productive research system in the word based on pounds in papers out. Now in part driven by the REF. Productivity a part of that. But in terms of behaviour change, introduction in 2014 of impact as a focus alongside outputs has had a massive cultural effect, positive effect in terms of creating an incentive structure/economy and enabling a more strategic and professional approach to impact, and supportive of a more diverse career paths in the system. China as alternative example, cash bonuses for publications, personal profit, but led to huge problems. In British system, had focus on outputs, now a focus on impacts and by and large a good thing.

25:24 How it now impacts appraisal discussions with staff.  Now have research, teaching, impact. A good thing, good research will have impact. Accepts some areas of research where impacts much longer term e.g., particle physics. Value as part of portfolio of what they do, now system in place to support academics doing it (impact) and doing it better and rewarding them. Now have a body of case studies from the exercise by topic, institution, discipline – a great resource. Means we can be much more strategic of understanding of how impacts arise. Most impact case studies were based on some kind of multi or interdisciplinary research, and often collaborative. Impact is a team sport.

29:20 Funding in UK to support that interdisciplinary emphasis? On the cusp of biggest shake up of funding system in the UK. Since mid 1960s a set of discipline-based research councils (see links below). All are about to be drawn under umbrella of a new mega funding agency, comes into being April 2018. Existing councils will still exist as committees under that body but goal now better support and enable cross disciplinary work. That’s the ambition. A big shift. Other things that have happened alongside that to further incentivise greater inter-disciplinarily are two big new strategic funding sources: global challenges research fund from aid budget, development money so research relevant to needs of developing worlds and in collaboration with partners in eligible countries, starting with their problems, and more global impact; and other is around industrial strategy, pump priming commercial realisation, not been as good at that e.g., as Germany to do the translational funding, more immediate commercial impact with industry partners.

34:45 Also opening new career paths. He talks about this as a new breed of brokers and boundary spanners that the system now demands and placing a premium on a skillset that is not the traditional academic skillset. Has flow on effects for how we think about doctoral training, early career research. But how does a boundary spanner submit e.g., to the politics panel? An inbuilt tension in the system over time. If you push the system towards more interdisciplinary work should you come back and evaluate people in the politics department? A question for the REF in 2027. Now is the time to start thinking about this. If you push all the incentives in the system towards new ways of working design, how do we design the assessment system in 10 years time? Incentives drive behaviour so how do we have complementary incentives systems. Two schools of thought on the new mega structure, negative is its terrible monolithic and inhibit diversity in the system, positive is it allows us to be more strategic and more collective intelligence to arise. By and large he is focussed on the positive.

38:00 What are the issues around metrics? The Metric Tide (report) was commissioned by the minister on role of metrics in management of university system. REF is by peer review over a year, it is labour intensive not metric driven. Looked at whole system for the REF. Committee had mix of great people, and did consultation, workshops, etc, a big process. Conclusion was that in the narrow context of the REF, more negatives in going hard to a metric-based systems than positives, in that yes you might remove some of the burden of the exercise but you shake off a lot of what was good about the REF. Current allows for a whole diversity of different outcomes, journal articles a part but can also put in books or arts-based outputs. Metrics tend only to cover journal articles. In politics area, about a quarter of the outputs were books and monographs but you don’t get metrics for those. Another reason is concern for diversity e.g., gendered nature of citation practices. Also re impact, currently recording through narrative case studies and can’t easily convert that to a metric. New metrics coming up e.g., social media measures but again could unleash perverse behavioural consequences like twitter bots if included in REF.

42:55 Interpreted mission more broadly though and in the broader sense of how metrics are interpreted and used in the university context, they expressed a serious concern about rising pressure of quantification on academic culture and how to manage that sensibly. Argued for scope to govern and manage systems of measurement much more sensibly, intelligently, and humanely in terms of their effects. A lot of that is about being responsible in the way you design and use metrics. Metrics are a technology and there is nothing intrinsically good or evil in them, it’s all about how they are used. Came up with set of principles for how metrics should be used eg diversity of indicators. More awareness now than 2-3 years ago, not just their review but growing chorus of voice gathering in volume and intensity internationally eg San Francisco Declaration on Research Assessment that came out 2012 or 13, pushing hard against emphasis on journal impact factors, the Leiden Manifesto for Research Metrics which was closely aligned with what they were doing.

Seeing in the UK more universities adopting policies and statements of good practice in terms of how they will use bibliometrics and altmetrics. Also having an impact on the REF in not going to bibliometrics.

46:50 Impact on own CV and presenting academic persona? Would never use journal impact factors and h-indices to make decisions, would look very bad. Wouldn’t use it in a panel because he thinks there are better ways of dealing with filtering applicants. “I think to simply look and say they’ve published in Cell therefore they’re better than this person… is the worst kind of sloppy practice. And we know this is statistically illiterate…. A very hard-edged reason why this is bad practice. It is incumbent on us not to indulge processes of evaluation that we know empirically are bad science.” All sorts of subtle signifiers we use and academia is full of these. “All we can do if you’re on an interview panel or evaluating stuff at a departmental level is try to be very conscious of what you’re doing, being quite reflexive about it and do stamp explicit bad practices.” Hasn’t experienced resistance to this where he is. “It’s my friends who are the hard-core scientists and who have looked at this and realised what utter bullshit it is.”

50:33 End

Related Links

James Wilsdon - https://www.sheffield.ac.uk/politics/people/academic/james-wilsdon

UK research funding councils – Higher Education Funding Council - http://www.hefce.ac.uk

Research Excellence Framework (REF) - http://www.ref.ac.uk/2014/

The Metric Tide report – https://responsiblemetrics.org/the-metric-tide/

San Francisco Declaration on Research Assessment - http://www.ascb.org/dora/

Leiden Manifesto for Research Metrics - http://www.leidenmanifesto.org