NISO Alternative Assessment Metrics (Altmetrics) Project
In June 2013, the Alfred P. Sloan Foundation awarded NISO a grant to undertake a two-phase initiative to explore, identify, and advance standards and/or best practices related to a new suite of potential metrics in the community. This initiative was a direct outgrowth of a breakout discussion group during the altmetrics 12 meeting in Chicago, IL. This project is an important step in the development and adoption of new assessment metrics, which include usage-based metrics, social media references, and network behavioral analysis. In addition, this project will explore potential assessment criteria for non-traditional research outputs, such as data sets, visualizations, software, and other applications. After the first phase, which will expose areas for potential standardization, the community will collectively prioritize those potential projects. The second phase will be to advance and develop those standards/best practices prioritized by the community and approved by the membership.
Original proposal (public version)
First In-person Meeting
We're currently working on plans to host the first in-person meeting in support of this work, which will take place on Wednesday, October 9, 2013 in San Francisco. With generous support from the Alfred P. Sloan Foundation, the objectives of this one-day meeting will include a short opening keynote on the topic of assessment, lightning talks on related projects, brainstorming for identification of topics for discussion, and prioritizing proposed work items.
More details on location, agenda, and speakers will be posted shortly.
The meeting is free for all attendees. Please RSVP here, which will assist in planning and logistics. In-person registration for this event closes on Friday, October 4 at 5:00 p.m. (ET). Virtual attendance registration closes on Tuesday, October 8 at 5:00 p.m. (ET).
NISO will live-stream the event, and those recordings will be available on this page after the close of the event.
Future In-person Meetings
To accomodate as broad collaboration from the community as possible, there will be two more meetings to accomodate increased attendance, input and participation from all stakeholders. These meetings are free for all attendees.
Washington, DC: Wednesday, December 11, 2013
RSVP here
Philadelphia, PA: Thursday and Friday, January 23-24, 2014
RSVP here
More details, including location and agendas to follow.
Output from the Project
NISO anticipates several outputs from this project. The first major output will be a white paper describing the outcomes, recommendations, and action items identified during Phase One. This paper will be developed from the in-person meetings, circulated and vetted publicly by the research community and experts, and will identify the Recommended Practices and/or standards that need to be developed by NISO (possibly in conjunction with others) in order to speed the adoption of altmetrics. The white paper will be used as the basis for a draft New Work Item Proposal within NISO for the Phase Two work.
With the draft and community input in hand, NISO will be positioned to leverage the community used in developing the paper to create the working groups needed to develop those practices and standards. Presuming the approval of the NISO membership and leadership committees, the final output of this initiative will be a published standard(s) or recommended practice(s) regarding altmetrics. There will also be training and resources produced about how to implement and adopt the recommended practice(s)/standard(s).
A Steering Committee has been established. All documents (including MP3 recordings of conference calls) and email related to this project are planned to be publicly available via this page.
Draft Timeline
A high-level, tentative timeline (from the proposal) for project activities that will be accomplished is as follows:
-
July 2013 – Negotiate contract with identified consultants to moderate Phase One work and to write mid-term whitepaper. Make arrangements concerning in-person meetings and issue invitations to contributors.
-
August 2013 – Set up wiki, online workspace, and e-mail lists. Announce fist meeting and availability of informational resources about the project.
-
November/December 2013 – Hold first face-to-face meeting, possibly to coincide with CNI. Build initial list of issues and concerns. Schedule and announce two follow-up webinars.
-
January 2014 – Hold the second in-person meeting in conjunction with ALA-Midwinter meeting in Philadelphia. Consultant begins drafting paper based on in-person sessions.
-
February/March 2014 – Public webinars to engage larger community and to discuss meetings and draft paper.
-
April/May 2014 – Consultant finalizes draft of white paper and first draft is released. Presentation of results and discussion at Altmetrics14 (TBD), CNI, and elsewhere.
-
June 2014 – Presentations at ALA Annual Conference in Las Vegas (June 27-July 1) and session held at Joint Conference on Digital Libray (JCDL), or similar venues, where the initial draft of the paper is presented, along with an outline of areas where recommended practices and standards are needed. Gather public feedback.
-
July 2014 – Consultant finalizes the whitepaper and draft new work item proposal for standards/best practices based on the study. Paper released. Proposal vetted by NISO leadership and members. WG members identified.
-
August/September 2014 – Proposal approved and working group formed.¹
-
September 2014 – August 2015: Working group develops recommended practices/standards. Public draft for comment or draft standard for trial use released. Discussion/input at webinars to report project outputs.
-
Fall/Winter 2015 – NISO publishes standards/recommended practices and related training materials.²
-
November 2015 – NISO submits report to the Sloan Foundation.
¹ The second phase standards/recommended practices project is subject to approval of the the Business Information Topic Committee as well as the NISO Voting Membership. Approval is not guaranteed.
² Final publication is subject to the approval of the the Business Information Topic Committee as well as the NISO Voting Membership. Approval is not guaranteed.
Additional information regarding this project, including calls for participation, will be forthcoming.