In areas where the need for standards is known but the specific areas and requirements are ill-defined, NISO often undertakes pre-standards work to identify and prioritize the standards or recommended practices that should be developed. In the past year and a half, NISO has undertaken two such initiatives. The first, started in December 2012 with a grant from The Andrew W. Mellon Foundation, was the Bibliographic Roadmap Project to develop a community roadmap for extending the usability of the new bibliographic framework into the global networked information environment. The second, begun in June 2013 with funding from the Alfred P. Sloan Foundation, was the Alternative Assessment Metrics (Altmetrics) Initiative to explore, identify, and advance standards and/or best practices related to a new suite of potential metrics.
Both of these projects completed their initial phase of community input and assessment and produced reports with recommendations of further actions NISO should take. This article summarizes the work of these initiatives and the recommendations.
Bibliographic Roadmap
In the current landscape of bibliographic exchange, most libraries are still creating and managing their extensive bibliographic data in MARC format. MARC, the lingua franca in libraries for over forty years, is often described as an outdated format, but its biggest liability in the modern web world is that it is unknown and unused outside of libraries. This uniqueness thus dooms library materials described with it to a siloed existence available within only library-oriented systems. The vast majority of library users today, who no longer consider libraries as the first point of entry for most of their information needs, prefer accessing information via the larger networked world, which demands approaches to data that can be more easily shared, indexed, and linked.
Recognizing the need to advance bibliographic exchange, the Library of Congress (LC) initiated a community discussion on the Future of Bibliographic Control in 2006 and the report of its recommendations was published in January 2008. Since that report was issued, libraries have begun to embrace the concept of the Semantic Web and linked data and have implemented specific projects that are elements of a new paradigm for bibliographic exchange. Resource Description and Access (RDA), a structure developed by the Joint Steering Committee that is meant to replace the Anglo- American Cataloguing Rules, 2nd edition revised (AACR2), was published in June 2010 to provide a model for mapping some of MARC data into web resources, but the processes, workflows, and systems to support a full conversion to RDA are not yet in place. The Library of Congress in an announcement in October 2011 stated that the MARC standard as a carrier of bibliographic records is not sufficient in the web-based world.
Given the diverse community that is impacted by bibliographic exchange as well as the tremendous investments made in existing MARC-based library systems and records, NISO proposed developing a roadmap for the high-level coordination of activities to help avoid duplication and fragmentation of the bibliographic exchange community.
The work began with a two-day meeting in Baltimore in April 2013 attended in person and virtually by over 100 experts and participants, including librarians, system vendors, publishers, and consultants and vendors providing services around these. Eight major general areas to address were identified in that meeting:
» Business models
» Prototyping
» Provenance/Authority » Rules
» Users
Each of these themes was discussed in greater depth and over 40 ideas for potential actions to address them were collected and posted in the NISO Ideascale idea-sharing website. The Ideascale tool was discussed in a follow-up webinar and publicized to the community to encourage feedback on prioritizing the ideas. The two most highly- ranked ideas from Ideascale were taken forward to an open discussion session held at ALA Midwinter, January 2014, where specific projects that NISO could undertake were proposed. The two ideas and the proposed projects are:
- Work to make vocabularies work across systems
» Work specifically to bring related vocabulary efforts together to take better advantage of expertise, tools, and existing best practices.
» Explore existing stores of vocabulary information (the Linked Open Vocabularies project is a good start) to identify problems, gaps, and potential for collaboration.
» Ensure that NISO’s own published vocabularies are in a machine-accessible form and take advantage of advancing knowledge in vocabulary expression and management
- Improve the ability of our data to be consumed and manipulated
» Create a recommended practice or an informational document around the use of linked data and associated rights and their implications.
» Create a community recommended practice specifically for data contribution for corporate entities to utilize as a justification for their contributions and potentially to use as a shield, or partial shield, in regard to liability questions.
» Organize, evangelize, and manage an authority file as an additional/alternative Registration Agency for ISNI to expose the ISNI to communities not familiar with the standard.
The activities that were determined, through community discussion to be part of the NISO Bibliographic Roadmap in large part aim to be applied to existing efforts and maximizetheir usability as much as possible. It was recognized in many discussions that though the larger library community overall may seem to be hesitant in moving forward amid a fair amount of uncertainty in the lack of a solid technical framework, there is already much experimentation and many projects under way in diverse spaces. Further practical exploration of existing vocabularies, linked data tools, and methods for data contribution can help to reassure the community that the transition forward will not be endless and the value of what libraries already do will be enhanced.
NISO’s leadership, via the Content and Collections Management Topic Committee is examining these prioritized Roadmap work items—as well as the other ideas generated throughout this process—for future action during 2015.
Altmetrics
Since Eugene Garfield’s pioneering work in the 1960s, much of the research assessment work has been based upon citations. As a metric, citation reference counts have been an exceptionally rich source of accessible data upon which to draw conclusions about the quality of scholarship and will remain an important component of research assessment. The JIF (Journal Impact Factor), which measures journals’ average citations per article, is one of the most used assessment measures, but such citation-based metrics are not keeping pace with the expanded scope of forms and usage that are presently available. Online reader behavior, network interactions with content, social media references, and online content management—all important indicators of scholars’ interaction with research outputs—are not reflected in today’s measures. Newer forms of scholarly outputs, such as datasets posted in repositories, software tools shared in GitHub, and algorithms or molecular structures are now commonplace but they are not easily—or if at all—assessed by traditional citation metrics.
The goal was to identify specific action items that NISO could pursue, particularly for the development of standards or recommended practices, to advance the use of altmetrics in the community.
These are among the many concerns the growing movement around alternative metrics, sometimes called altmetrics, is trying to address. In developing and applying new forms of altmetrics, many issues come up, such as:
» What exactly gets measured?
» How do we decide what the criteria are for assessing the quality of the measures?
» At what granularity should these metrics be compiled and analyzed?
» How long a period should altmetrics cover?
» What is the role of social media in altmetrics?
» What is the technical infrastructure necessary to exchange these data?
» Which metrics will prove most valuable and how do we decide?
» What types of assessment criteria could and should be applied to these new metrics to best assess the value of the analysis?
» How do we ensure consistent quality across providers?
In the first phase of NISO’s Altmetrics Initiative, input from relevant stakeholders about these and other issues surrounding altmetrics was obtained through three in-person meetings and 30 in-person interviews. Recordings, documents, and other output from these meetings are archived on the Altmetrics Initiative webpage. The goal was to identify specific action items that NISO could pursue, particularly for the development of standards or recommended practices, to advance the use of altmetrics in the community.
The input received was summarized in a white paper, which identified a total of 25 action items in nine categories.
- Definitions
» Develop specific definitions for alternative assessment metrics.
» Agree on proper usage of the term “Altmetrics,” or on using a different term.
» Define subcategories for alternative assessment metrics, as needed.
- Research Outputs
» Identify research output types that are applicable to the use of metrics.
» Define relationships between different research outputs and develop metrics for this aggregated model.
» Define appropriate metrics and calculation methodologies for specific output types, such as software, datasets, or performances.
- Discovery
» Agree on main use cases for alternative assessment metrics and develop a needs-assessment based on those use cases.
- Research Evaluation
» Develop statement about role of alternative assessment metrics in research evaluation.
» Identify specific scenarios for the use of altmetrics in research evaluation (e.g., research data, social impact) and what gaps exist in data collection around these scenarios. -
Data Quality and Gaming
» Promote and facilitate use of persistent identifiers.
» Research issues surrounding the reproducibility of metrics across providers.
» Develop strategies to improve data quality through normalization of source data across providers.
» Explore creation of standardized APIs or download or exchange formats to facilitate data gathering.
» Develop strategies to increase trust (e.g., openly available data, audits, or a clearinghouse).
» Study potential strategies for defining and identifying systematic gaming of new metrics.
-
Grouping and Aggregation
» Identify best practices for grouping and aggregating multiple data sources.
» Identify best practices for grouping and aggregation by journal, author, institution, and funder.
» Define and promote the use of contributorship roles. - Context
» Establish a context and normalization strategy over time, by discipline, country, etc.
- Stakeholders’ Perspectives
» Describe main use cases for the different stakeholder groups.
» Identify best practices for identifying contributor categories (e.g., scholars vs. general public).
- Adoption
» Identify organizations to include in further discussions.
» Identify existing standards to include in further discussions.
» Prioritize further activities.
» Clarify researcher strategy (e.g., driven by researcher uptake vs. mandates by funders and institutions).
Due to the number of potential action items, a follow-up survey was conducted to obtain further feedback on prioritizing the proposed actions. The top three “very important” items were:
» Promote and facilitate use of persistent identifiers in scholarly communications. (59.5%)
» Develop specific definitions for alternative assessment metrics. (54.3%)
» Develop strategies to improve data quality through normalization of source data across providers. (41.7%)
The NISO Business Information Topic Committee with input from the Altmetrics Steering Committee is evaluating the white paper, the comments received on it, and the prioritization survey and will be recommending one or more Working Groups for start-up by year-end 2014.
