A Need for Better Metrics
Released in February 2022, this report is the result of 18 months work by the HuMetricsHSS initiative, funded by the Mellon Foundation and conducted in conjunction with member institutions of the Big Ten Academic Alliance (BTAA). As noted in the Executive Summary, the white paper focuses on the reappointment, promotion and tenure (RPT) process, offering “a set of recommendations for making wide-scale change to address systematic injustice, erasure, and devaluation of academic labor in order to strengthen the positive public impact of scholarship.
Describing the current research-teaching-service triangle as heavily imbalanced, the executive summary notes that “The responsibility for making change in systems of RPT and evaluation more generally seems at once to belong to everyone and no one. Departments bow to disciplinary convention or the bylaws laid out by deans and provosts; deans and provosts swear that the departments themselves set the standards by which they are evaluated. A refusal of agency for fear of failure — and of consequences for the attempt (such as plummeting university rankings, loss of grant money, and so on) — results in a foundering machine aware of its defects but unwilling and unable to fix them...”
Among other recommendations, the report calls for institutions to:
- Shift the categories of the tenure and promotion process from the means to the ends toward which they are directed. Research, teaching, and service are not meaningfully distinct categories of modern scholarly labor and should not be treated as such by systems of evaluation. Instead, evaluation should focus on the ends those means serve: sharing knowledge, expanding opportunity, and mentoring/stewardship.
- Collaborate with provosts to revise university-level statements on promotion and tenure. The process by which promotion and tenure guidelines are set varies from university to university. However, campus-level expectations must work in tandem with the department-level governing documents mentioned earlier to realign institutional values with the practices and policies of tenure.
- Increase opportunities for disciplinary leaders to experience evaluation practices and procedures from a wider diversity of disciplines across the mission of the university. Leadership training programs supported by college deans and managed through offices of associate provosts for faculty affairs would provide opportunities to explore and share disciplinary values and practices, with the goal of expanding perspectives on the assessment process.
As articulated elsewhere, there is a recognition that “current systems of evaluation in academia tend to rely on a set of proxy measures, often drawn from science, technology, engineering, and mathematics (STEM) fields and mapped poorly onto HSS disciplines, that fail to recognize and reward the many dependencies upon which a healthy scholarly ecosystem relies. These proxy measures privilege certain kinds of scholarship — and scholars — over others and reinforce historical biases and barriers for scholars of marginalized backgrounds and identities. There is a growing sense among scholars across all disciplines, but particularly those within HSS, that they are being evaluated on what can be easily measured rather than on a holistic and textured understanding of their work and the impact it has on communities within and outside of the academy.”
Much of what is included in this white paper was touched upon during a 2021 NISO Roundtable Discussion on the topic of metrics. Rebecca Kinnison, one of the authors of this publication, was a part of that discussion and addressed some of the emerging issues with which this report is concerned.
Those interested in reading the report can find it online in both HTML as well as PDF file formats. The 225-page report contains an extended set of end notes, references, and appendices, offering very real value to those working in this arena. HuMetricsHSS invites feedback through their Twitter account or by email.