Report on the NISO Forum on
Performance Measures and Statistics for Libraries

February 15-16, 2001
Washington, DC

Sponsored by
National Information Standards Organization

 

Acknowledgements

NISO is a small organization whose success depends on the talent and energy of a tremendous group of dedicated volunteers. NISO extends its thanks and appreciation to the members of the Forum Planning Committee:

Committee co-chairs:

Denise Davis, National Commission on Libraries and Information Services (NCLIS) and

Patricia A. Wand, American University.

Committee members:

Michael Gorrell, EBSCO

Martha Kyrillidou, Association of Research Libraries

Karen Motylewski, Institute for Museum and Library Services

Barbara Perry, World Bank/IMF Library

Pat Stevens, OCLC;

J.D. Waggoner, West Virginia Library Commission

Peter Young, Library of Congress

This Committee took a complex issue and synthesized it into a workable agenda and combined their skills to bring together a remarkable group of Forum participants.

The success of the Forum also relied on the talents of two key Forum staff: Octavia Seawell, Forum facilitator, assisted in all stages of agenda development and provided leadership throughout the event. Judy Luther served as the Forum recorder to document the event and capture the good thinking that emerged.

This event could not have occurred without the strong support of these partners who helped fund the Forum: the Mellon Foundation, the U.S. National Commission on Libraries and Information Science, and the Association of Research Libraries.

Pat Harris
Executive Director
National Information Standards Organization
April 9, 2001

 

 

TABLE OF CONTENTS

 

I. Executive Summary
  1. Conclusion
  2. General Themes
  3. Recommendations

II. Background

III. Framework of the Forum

IV. Presentations

V. General Themes, Discussions and Comments from Participants

VI. Environmental Scan

VII. Panel presentations (I/II), PowerPoint slides

VIII. Webography

IX. Forum Agenda and List of Attendees

 

NISO Forum on Performance Measures and Statistics for Libraries

I. Executive Summary

On February 15-16, 2001 the National Information Standards Organization (NISO) organized an invitational Forum to gather information from the library community and key vendors about the best approach to evaluate the NISO standard on Library Statistics. Due for review in 2000, ANSI/NISO standard Z39.7 details and defines significant library input and output measures as they were collected by various organizations and agencies in the early 1990s.

Sixty-five participants representing academic, public, school, government, and special libraries, associations, publishers, vendors, integrated library systems and the research community gathered in Washington D.C.

Conclusions

This Forum enabled a broad, diverse group of stakeholders to explore their requirements and vision for describing, measuring, and showing the significance of contemporary library services.

Forum discussions confirmed that libraries and related organizations have a critical need to demonstrate concretely the value they bring to the communities they serve.

As a whole, Forum participants felt that NISO played a critical role in bringing all constituencies together. NISO was seen as a rare "honest broker" – an objective agent for sharing information across traditional boundaries to enable each constituency to build on the work of others and reduce the duplication of effort.

General Themes

Throughout the Forum, common themes emerged around the need to demonstrate the value that libraries contribute to their communities.

  • There is critical need for systemic data collection. With all sectors of the industry represented in the Forum, discrepancies in the availability of accurate, current, national data for essential library services within key segments became rapidly apparent. The lack of information about school libraries may be the type example.
  • There is a pressing need for guidelines for collecting qualitative and performance data. The community is ready to move beyond statistics into measures that tie the value of libraries more closely to the benefits they create for their users. Changes in the technology of library services and the competitive climate for resources requires libraries, librarians, and even NISO, to understand better how to market themselves to a variety of audiences.
  • Different methodologies are evolving to measure network performance (NCLIS), service quality (e.g. ARL’s LibQUAL+), impact, and economic value. Additional research is needed to develop new tools and approaches for practical application by the library community.
  • NISO is well-positioned to develop a "data dictionary" of terms in frequent use by different constituencies. The definition and meaning of terms can vary greatly, and a common vocabulary is necessary to understand the data being collected and presented. NISO can also serve as a clearinghouse for guidelines, standards, benchmarks, and best practices. In a rapidly changing environment, a variety of flexible tools are needed to facilitate coordination of developments throughout the industry.

 

Recommendations

NISO Executive Director Pat Harris observed that guidelines and best practices are more flexible than standards, and may be better suited to coordinate the development of evolving common practices. Many of the discussion groups agreed that NISO is well positioned to play a bigger role to:

  • Facilitate cross-constituency information sharing, enabling specialized stakeholders to build on the work of others and to avoid duplication of effort.
  • Serve as a clearinghouse for standards, guidelines, and other tools across diverse communities with closely related interests.
  • Move beyond defining data elements to provide methodologies for qualitative as well as quantitative measures of library service.
  • Support the development of surveys/measurements for service quality and outcomes.
  • Offer training for the industry on what to measure and how to measure it.
  • Explore tools such as ISO 9000 and the Baldridge Award for application to the library and information industry.
  • Create a best practice publication series.

 

II. Background

Patricia A. Wand, NISO Forum Planning Committee co-chair, welcomed the sixty-five information professionals attending the NISO Forum on Performance Measures and Statistics for Libraries on February 15-16, 2001, in Washington, D.C. and provided background information for Forum discussions. Participants represented every type of library: academic, government, public, school, and special, as well as associations, government agencies, vendors, and the library research community. The participants, all of whom are involved in collecting, aggregating, contributing, or studying library data, were invited because of their demonstrated professional expertise and commitment.

Every NISO standard is re-evaluated five years following approval. The NISO standard on Library Statistics, ANSI/NISO Z39.7-1995, came up for review in 2000. The primary goal of the Forum was to advise NISO about the review of Z39.7Participants were asked to evaluate the utility of the current standard for library statistics and to help build a common vision of what is needed in assessing libraries and their contributions to their communities.

The Forum was designed around certain assumptions, including:

  • The library community has a deep respect for standards and a long history of cooperation.
  • Although the current standard for library statistics primarily counts inputs and outputs, the profession is ready to measure other indicators of productivity and quality. No standards exist for these other measurements.
  • Successful standards start small and may begin as pilot projects, best practices, or guidelines that can be tested. A successful action plan will encompass phases with short-term and long-term activities.
  • Many different constituents comprise the information profession and many different needs must be addressed.

Participants considered a variety of approaches, including ideas for research; topics that need to be addressed; areas where standards, guidelines, or best practices are appropriate; and opportunities for NISO to collaborate with other organizations.

Pat Harris, NISO Executive Director, provided background on NISO and reviewed the history of the Standard for Library Statistics, Z39.7-1995.

Harris noted that NISO's members and supporters are brought together under the NISO umbrella by a common vision: To find practical solutions to the problems they share.

Sometimes those problems are unique to one professional segment, but more often, especially in our networked world, the same problem will be shared across communities. NISO's view is that it is to our mutual benefit to try to reach solutions that all can use.

NISO uses a number of approaches to support its standards development work:

  • NISO sponsors educational events, conferences, executive briefings, publications, white papers, and studies.
  • As an ANSI-accredited standards developer, NISO supports the development of technical standards when a standard is viewed as the best solution. Harris stressed that NISO advocates a practical approach to standards development and that standards should not be viewed as "the perfect document."

NISO's approach to standards development has changed over the years. NISO Procedures demand that NISO revisit each standard five years after its approval. This "five-year review" is a chance to determine if the standard is still used. In the process NISO considers if it would benefit from revision; if it does the job as is, it can be reaffirmed.; If the standard is no longer used, we consider whether it should be withdrawn or maintained as a legacy document;

For many years, NISO tradition limited this discussion to its members, but it finds far more value in opening discussion to the community at large and engaging a variety of knowledgeable and interested persons to advise on next steps. NISO has used this approach successfully to explore emerging standardization issues such as the DOI; advances in thesaurus construction and design; , knowledge management; and the challenges and complexities of reference linking.

All Standards are expected to change to address new needs and technologies. The Library Statistics Standard was first released in 1968, and revised in 1983 and again in 1995. With each revision the Standard has grown and changed. When the 1995 edition of the Library Statistics Standard was released the committee that developed it acknowledged that the Standard did not address two important emerging areas: measurement of electronic resources and performance measures. They recommended that these issues be examined at the next five-year review point.

In concluding, Harris reminded participants that the Forum's goal was learning: NISO wanted to hear from all the participants, as advice distilled from the Forum will inform NISO’s decisions about what directions to take. NISO also hoped that the Forum participants would learn from one another.

III. Framework of the Forum

Throughout the Forum discussion was channeled through a mixture of panel presentations, whole-group discussions, and structured break-out groups. The goal was to maximize cross-disciplinary information-sharing around broad issues and to use discussion in homogeneous groups (called affinity groups) to capture the viewpoint of specific segments, for example school libraries, researchers, and vendors.

The Forum was structured around a concept map comprised of four categories of measurement and three key management functions supported by those measures. The four categories were:

  • Statistics – What we are measuring - how much we do (input/output)

Counting activities and resources that quantify library materials, services, staff, operations and users.

  • Performance indicators – How well we do.

Collection and interpretation of quantitative and qualitative data pertinent to local goals and objectives as well as values and/or characteristics in order to assess service quality, performance efficiency and customer satisfaction within a specific context.

  • Economic value – How much we’re worth in $$.

Collection and interpretation of expenditures data in relation to output production to document costs, unit costs, and economic value derived, and assign a dollar value to library products and services within a specific environment characterized by various environmental constraints and opportunities.

  • Outcome measures – What good we do.

Evaluation of the results of program activity compared to its intended purpose. Measures relate to observable changes in knowledge, skills, behavior, attitudes, status or life condition. Quantitative measures are based on test scores, and qualitative measures are based on observations of indicative behaviors.

The three functions were:

  • Internal decisions – to assist with management decisions and internal specifications and policies
  • Local advocacy – to justify budgets and demonstrate that local goals and objectives have met the needs of the community
  • Broad advocacy – to identify trends, understand the larger environmental factors, promote the value of information organizations and professionals such as library and information professionals

This perspective was identified by a presenter and recognized as a useful distinction by many of the participants. During Forum discussions a fourth function emerged. There was significant discussion of the importance of marketing in library management.

IV. Presentations

Two panel presentations set the stage for small heterogeneous or homogeneous group discussions. The first panel, representing academic, school, special, and public librarians, and the vendor community suggested significant current issues for measurement in libraries.

The second panel presented current research and development focused on measuring the performance and value of libraries.

Panel I: Issues for Libraries – Measuring the Information Age. Moderated by Peter Young, panelists were asked to address the question "Given the speed of change, how should libraries measure today what they will need to report on tomorrow?"

  • Eileen Hitchingham, Virginia Polytechnic Institute, asked participants to consider whether data was being collected "to show" administrators for budget justification or "to know" – for use by library management.
  • Ann Weeks, College of Information Studies, University of Maryland, noted that no national data about school libraries is being collected. She pointed out that it is difficult to make compelling arguments for school libraries without data. Weeks also noted that information literacy skills begin with school libraries, so life-long information literacy and effective library use is dependent on the success of this building block.
  • Susan Fifer-Canby, National Geographic Society Library, suggested that library staff should be treated as assets rather than expenses. She pointed out that special libraries are well positioned to facilitate organization wide conversations.
  • Mark Smith, Riverside (California) County Library System, suggested that libraries have not been fundamentally changed by technology . Rather, he noted, it is users’ expectations and patterns of using library information resources that have changed dramatically.
  • Michael Gorrell, EBSCO Publishing, emphasized the importance of statistics. His early work in engineering demonstrated that "without data, you are just another person with an opinion."


Panel II. Current Research. Denise Davis moderated the panel on research conducted in academic, public, and school libraries to develop tools that provide data on a variety of measures of library services and their value.

  • Fred Heath, Texas A&M University, described the development and testing of LibQUAL+, a protocol adapted from the business community, to assess service quality in academic libraries. LibQUAL+, funded in part by the Fund for the Improvement of Post-secondary Education (FIPSE), U.S. Department of Education, maps how well the level of service delivered matches user expectations, based on what users find important.
  • Denise Davis gave the presentation of John Bertot and his colleagues at Florida State University. Bertot is the lead researcher for two studies. The first project is jointly sponsored by IMLS, NCLIS, and NISO. This work defines electronic network performance measures that can be collected at the national level for public libraries. The second project, sponsored by ARL, explores e-metrics for research libraries , suggests measures for the use of electronic resources, and develops a framework of outcomes assessment in relation to the use of electronic resources
  • Karen Motylewski delivered a presentation for Keith Lance, State Library of Colorado. Funded in part by IMLS, Lance has documented the positive relationship of student performance and strong school library/school media center programs in Alaska, Colorado, Pennsylvania and Oregon.
  • Christie Koontz, Florida Resources and Environmental Analysis Center, Florida State University, described her research in collecting in-library use statistics. In the most recent project, Keith Lance joined Ms. Koontz and Dean Jue of Florida State University as co-principal investigators in developing a model for using portable technology (PDAs) to assess use of the public library in majority, minority, and low-income settings. This project updates "Counting on Results" and is funded by a grant from IMLS.
An informal, impromptu update came from Jeffrey Owings, Associate Commissioner, National Center for Education Statistics (NCES). Owings provided information about the national Library Statistics Program and a longitudinal study of 10th graders that includes data about library media centers.
 
V. General Themes, Discussions and Comments from Participants

Throughout the Forum, common themes emerged around the topic of how to demonstrate the value that libraries contribute to their communities.

  • There is critical need for a national initiative to develop a systemic data collection effort, especially for school libraries. With all sectors of the industry present in the discussions, discrepancies in the availability of timely local and national information on essential library services in some industry segments became rapidly apparent.
  • There is a need for an action plan to develop guidelines on the collection of qualitative and performance measures. The community is ready to move beyond statistics to add measures that demonstrate the value of libraries in the context of the mission of organizations and benefits to users.
  • Librarians need tools to collect and interpret qualitative data to "tell their story" more effectively. There is recognition that libraries need more than input/output data. New tools are needed to convey the value of libraries to their users and funding agencies.
  • Libraries, librarians, and even NISO need a better understanding of how to market themselves and their organizations to their communities.
  • The dramatic increase of technology in library functions requires different skill sets than were necessary a decade ago. Systematic training needs to be available to working librarians to develop the skills to assess, manage and promote their organizations.
  • Different methodologies are evolving to measure network performance, service quality, library impact, and economic value. Additional research is needed to develop new tools and approaches to document library productivity, excellence, and value.
  • The definition and meaning of terms can vary greatly due to technology factors (such as timeouts in "sessions"). A common vocabulary is necessary to understand the data being collected and presented. NISO is well positioned to develop a "data dictionary" of terms in use by different constituencies.
  • Given its pivotal role in the industry, NISO can facilitate the cross-sector collaboration necessary to evolve the most useful working documents. Given the number of related initiatives, NISO could greatly reduce duplication of effort.
  • NISO can serve as a clearinghouse for guidelines, standards, benchmarks, and best practices. In a rapidly changing environment, a variety of flexible tools are needed to support and facilitate coordination of developments throughout the industry.
  • The last time the library statistics standard was revised, the focus was on documenting or labeling input and output data consistently. The Forum identified a need for action, and a consensus that a vision that moves libraries past the measurement of data and into assessment of performance, impact, and user wants and needs is desirable. A summary of discussion and comments from Forum participants follows. Comments ranged from minority opinions that "people don’t care about libraries," and its opposite, "everyone values the library and there is no need to collect data," to strong demands for the development of tools and models to give libraries frameworks to engage in meaningful assessment activities.
    • Caution was offered in data interpretation. For example, a decrease in number of searches conducted may reflect increased efficiency and higher user satisfaction rather than less use and lower value.
    • The user’s universe is changing in fundamental ways that affect how information is located and used.
    • Libraries are beginning to develop customer-oriented assessment tools focused on user satisfaction, service quality, and outcomes.
    • Libraries need new tools and best practices to assess user behavior.
    • Expectations for customer service standards can be operationalized as "every user has the right to…."
    • The psychographics of subgroups (understanding how/why certain segments behave) can offer insights in developing customized services to meet specialized needs. (e.g. "minit markets")
    • With the advent of electronic resources, physical collections are less apparent but stable while digital collections may be more apparent, yet not stable Measurement might more accurately focus on the activity of librarians rather than the activity of the library.
    • Workload measures (e.g., how many ILLs can be handled in an hour) are needed to establish parameters for staffing.
    • Guidelines are needed to determine when to collect whole-universe data vs. sampling.
    • The more libraries work with unified standards, the more likely they are to be heard.
    • Guidelines and tools are needed to assess users’ needs, satisfaction and competency.
    Pat Harris, Executive Director of NISO, observed that guidelines and best practices are more flexible than standards and may be better suited to supporting evolving common practices. The discussion groups agreed that NISO might play a bigger role to:
    • Bring constituencies together to learn abut the work of other stakeholders, to build on that base, and reduce duplication of effort.
    • Serve as a clearinghouse for standards and guidelines across diverse communities with related interests.
    • Move beyond defining data elements to provide guidelines and methodologies for qualitative as well as quantitative measurement.
    • Support the development of surveys, measurements, and other resources for measuring service quality and outcomes.
    • Offer training for the industry on what to measure and how it can be measured.
    • Explore tools such as ISO 9000 and the Baldridge Award program for application to the library and information industry.
    • Create a publication series, with best practices identified as one desirable focus.
    Attendees’ comments about the Forum were positive. Many noted their own discovery process, learning from colleagues and developing an appreciation of the need for guidelines within the industry. General comments included:
  • "Interesting discussions. Well organized. I hope something tangible comes out of this Forum in the near future"
  • Will share with colleagues and advocate for future developments
  • "Committee members did outstanding work"
  • "Found this to be a very interesting and worthwhile two days"
  • "Thanks for the invitation. I learned from others, especially activities underway in other types of libraries.
  • "…There wasn’t adequate time to examine any problem beyond a superficial level that could tap the knowledge base (represented).
  • "As a resource provider, I found it useful to participate in the "max-mix" groups where I could learn more about the needs of libraries."
  • "Overall very thought provoking. Very interesting mix of people"
  • "I cannot tell you how valuable it was to have representatives from the school library community…at this forum."
  • "Additional focus group forums would be great."
  • "Discussion was good but time structure didn’t enable us to get enough in-depth."
  • "Good to have representation from diverse parts of the library community."
  • "There were many comments about the need for tools,– whether software products, training, etc. In addition to common standardized units of measurement."
  • Enthusiasm to support future efforts was indicated by the number of volunteers who offered to continue working on this topic.

    VI. Environmental Scan

    In the mid-1990’s the United States benefited from the services provided by more than 8,900 public library systems. These comprised more than 16,000 service sites; 50 state library agencies (of which 48 have libraries servicing residents or state government employees); 3,400 higher education institutions with libraries; and 98,400 public and private schools with library media centers. In total, residents of the United States benefit from the services and resources of more than 117,700 libraries. Although the numbers of libraries remain largely unchanged, the shifts in the services, materials; and expenditures have been dramatic. When comparing the status of libraries in the mid-1990s to today, we discover some interesting shifts in library acquisitions, the technological infrastructure required to perform basic tasks and provide basic services, and the organizational structures in which libraries operate. Libraries are major contributors to the United States economy. They expended more than $10.8 billion annually in the mid-1990s–expenditures are expected to exceed $13 billion in 2000. To emphasize this point, state library agencies reported operating expenditures of $218.2 million in 1994-95. In 1998-99 operating expenditures had risen to $949.5 million. 68% of this amount provided financial assistance to individual libraries. Capital outlay rose from $334,000 to $6.95 million during this same time period. http://nces.ed.gov/surveys/libraries/

    Journal pricing has skyrocketed in the last decade. ARL Statistics, 1998-1999 reported a unit cost increase of 207% between 1986 and 1999. Monograph expenditures rose by 34% during this same period. The ARL Supplementary Statistics, 1998-1999 reported an increase in expenditure for electronic serials from $11.8 million in 1994-1995 to $67 million in 1998-1999, a five-fold increase. Libraries are making information resources more widely available, but their ability to continue to do that is at stake. Libraries cannot afford to purchase the breadth and depth of information resources they used to buy, as library budgets are not keeping up with either the growth of information resources or their associated costs. http://www.arl.org/stats/index.html

    The 1980s and 1990s saw the integration of microcomputers into the routine day-to-day operations for library staff and their customers. In the 1990s the Internet became the telecommunication network for all libraries. Internet connectivity in public libraries rose from 20.9% in 1994 to 95.7% in 2000. Graphical Internet access rose from 11.8% to 94.5% during the same period. Fifty-three percent of public libraries are in rural areas. In 2000, 93% were connected to the Internet. The 2000 Internet Connectivity in Public Libraries study revealed that 87.7% of public libraries used operating funds from local government for Internet-related technology and infrastructure expenses, 29.6% received state grants for a portion of their Internet-related technology and infrastructure expenses, and 31.4% benefited from special grants for technology and infrastructure, including the Gate’s Library Program. http://www.nclis.gov/statsurv/statsurv.html

    In 1994 few libraries, regardless of type, made graphical workstations available to their customers. There are now, on average, 8 graphical public access Internet workstations per public library branch. 81% of public libraries provide public access to magazine, newspaper and other databases; 36% provide access from locations outside the library. 91% of academic libraries reporting provide access to electronic indexes and reference tools within the library and 70% reporting provide access to full text periodicals within the library. 62% of public libraries and 75% of academic libraries provide Internet-related training services.

    1.7% of Americans required adaptive technology devices in 1999. 28.8% of public libraries and 42% of academic libraries provide special hardware or software for accessing the Internet by individuals with disabilities

    Unfortunately little information on expenditures or target resources is available for K-12 school media centers, but four studies report related data. In aggregate, these studies found that 51% of public school library media centers reported that inadequate funds for technology were a "major barrier" to gaining or maximizing access to the Internet; 42% of school library media centers identified insufficient staff time as a major barrier; and only 42% of school library media centers considered "access to a network connection provider" no barrier. Other "major barriers" to Internet access reported by school library media centers included telephone lines (42%), and telecommunications equipment (38%). If these barriers were removed, and if adequate funds were made available, school media centers could be expected to join other types of libraries as major sources of access to educational materials and information, and as markets for resources in electronic format.

    VII. Panel presentations (I/II), PowerPoint slides

    Presentations from each panelist are included below. For additional information about presenters, please refer to Section IX for the list of attendees and presenters, or the NISO website at http://www.niso.org.

    Forum Introroduction Slides

    Remarks from Ann Weeks

    Remarks from John Carol Bertot

    Slides from Panel Two

    Slides from Keith Lance

    Remarks from Fred Heath (LibQUAL+ bibliography hot link: http://acs.tamu.edu/~bbt6147/servqbib.htm)

    Remarks from Glen Holt

     

    VIII. Webography

    Forum on Library Statistics and Performance Measures

    Webography of Related Resources

    Prepared by Linda Ferguson
    Relicorp, Inc.

    Since 1968 NISO has maintained the national standard for Library Statistics, ANSI/NISO Z39.7. In conjunction with the five-year review process, and in support of the Forum on Library Statistics and Performance Measures, the following "webography" has been prepared as a guide to those organizations, library groups, and resources impacting and impacted by the need to "Measure the Information Age". NISO encourages the review and use of the resources linked through this webography.

    Key Organizations and Resources

    American Library Association
    List of ALA-reviewed standards and guidelines publications. See also Standards and Guidelines.

    Association of Research Libraries (ARL)
    ARL Homepage. Has information on ARL-related projects and publications.

    British Library Board
    Library and Information Statistics Tables for the United Kingdom (LIST). Loughborough University, Leicestershire, UK, 1998.
    Includes statistical tables for all types of libraries in the UK.

    Centre for Research in Library & Information Management (CERLIM)
    EQUINOX: Library Performance Measurement and Quality Management System
    International project to develop performance measures for electronic environments in libraries, and to develop a quality management and measurement tool for same.

    Corporation for National Research Initiatives
    D-Lib Working Group on Digital Library Metrics
    Links to documents by the Working Group.

    Council on Library and Information Resources (CLIR)
    Links to the Digital Library Federation (administered by CLIR), publications, projects, and events.

    European Commission, Telematics Applications Programme, Telematics for Libraries
    CAMILE: Concerted Action on Management Information for Libraries in Europe
    Projects include EQLIPSE, a 24-month joint project to develop a quality management and performance measurement system for all types of libraries; DECIDE; DECIMAL; and MINSTREL. The CAMILE site reports on the results of the four projects.

    Florida State University, School of Information Studies
    Information Use Management and Policy Institute.
    Projects and publications from the Institute.

    Institute of Library and Museum Services
    Provides federal funding for libraries and museums. A set of recommended outcome-based measures and a bibliography is available.


    International Coalition of Library Consortia (ICOLC).
    Information about the coalition members. See also Guidelines for Statistical Measures of Usage of Web-based Indexed, Abstracted, and Full Text Resources.

    International Organization for Standardization.
    International federation to promote the development of standardization. See also SC 8, the subcommittee "responsible for the standardization of library and publishing statistics and the evaluation of library performance".

    LibEcon2000
    Project to create an international statistics database on library activities and their associated costs. Funded by DG13 of the European Commission within the Telematics Applications Programme.

    Library and Information Commission
    Value and Impact Research Programme. London, UK.
    Descriptions of current projects with links to researcher sites.

    Library Research Service
    Partnership between the Colorado State Library and the Master of Library and Information Sciences Program in the College of Education at the University of Denver to conduct research projects for all types of libraries and related associations.

    Loughborough University, Library & Information Statistics Unit (LISU). Leicestershire, UK.
    Homepage of LISU. Includes Library and Information Statistics Tables (L.I.S.T.) tables for all types of libraries from 1997-1999.

    National Center for Education Statistics
    National Center for Education Statistics Library Statistics Program.
    Library statistics surveys for "academic libraries, federal libraries and information centers, public libraries, school library media centers, state library agencies, and library cooperatives".

    National Commission on Libraries and Information Science (NCLIS)
    Information about the program, presentations, and links to statistics resources. See also NCLIS Library Statistics Program (LSP).

    National Information Standards Organization (NISO)
    NISO homepage. Includes standards publications, news and events.

    National Library of Canada National Core Library Statistics Program [PDF format].
    Statistics and analyses of findings for Canadian public, academic and special libraries from 1994 through 1996.

    Online Computer Library Center, Inc. (OCLC)
    OCLC Office Of Research, Web Characterization Project.
    Statistics and publications from the Office of Research's project to sample and analyze the Web. See also O'Neill, Edward T., McClain, Patrick D. & Lavoie, Brian, 1997:A Methodology for Sampling the World Wide Web.


    Library Statistics & Performance Measures
    Comprehensive, bibliography of print and web resources related to statistics and performance measures.

    Titus, Elizabeth McKenney
    LIBSTATS. Statistics from U.S. and some international libraries of all types.

    <Performance Measurement and Metrics: the International Journal for Library and Information Services. Aslib, ISSN 1467 8047. Articles available online from August 1999 and April 2000 issues.

    University of Bath, UK Office for Library and Information Networking
    eLib supporting studies and workshops managed by UKOLN
    Hyperlinked listing of studies managed by UKOLN, including "Management information systems and performance measurement for the electronic library" by Peter Brophy and Peter M. Wynne. See also eLib homepage.

    University of Northumbria at Newcastle, Information Management Research Institute (IMRI)
    Includes research by Indah Sidek, "The Impact of Implementing ANSI".

    Urban Library Council (ULC)
    Homepage of the ULC, including information on research and projects. See also Fast Fact Surveys, a collection of survey reports.

    Academic and Research Libraries

    Association of College and Research Libraries
    Information Literacy Competency Standards for Higher Education. ACRL, 2000.
    Standards, performance indicators, and outcomes as approved by the ACRL.

    Association of College and Research Libraries

    Academic Library Statistics. ACRL, 1999.
    Summary reports from the 1998 and 1999 surveys.

    Association of College and Research Libraries, Standards and Accreditation Committee
    Sources of Information on Performance and Outcome Assessment.
    Annotated bibliography on performance and outcome measures.

    Association of Research Libraries
    ARL Statistics and Measurements Program.
    Top-level site for ARL programs, initiatives, and publications.

    Association of Research Libraries
    ARL New Measures Initiative
    Links to New Measures Imitative projects supported by ARL Statistics and Leadership Committees, including LIBQUAL+ and Measures for Electronic Resources (E-Metrics)

    Association of Research Libraries
    Measurements of Research Libraries
    Bimonthly reports from ARL, CNI, and SPARC.

    Association of Research Libraries
    ARL, Special Issue on Measures. Issue 197, April 1998.
    Articles and issues on performance measurements in research libraries. Several focus on electronic resources.

    Australian Academic and Research Libraries
    Performance indicators for academic libraries. 28, (4): December 1997 Entire issue covers this topic.

    Bibliostat.com
    Scholarstat Libraries.
    Software automates data from the ACRL and IPEDS surveys.

    Cook, Colleen
    SERVQUAL: A Bibliography.
    Comprehensive bibliography of articles as of October 1999 written about the SERVQUAL tool.

    Cook, Colleen, Heath, Fred and Thompson, Bruce.
    LibQUAL+: One Instrument in the New Measures Toolbox. ARL, 212, October 2000.
    Status of LibQUAL+ as of October 2000.

    Cook, Colleen and Heath, Fred.
    The ARL "LibQUAL+" Pilot Project: An Update. ARL, 211, August 2000.
    Status of LibQUAL+ as of August 2000.

    Cook, Colleen and Heath, Fred
    SERVQUAL and the Quest for New Measures. ARL, 207, December 1999.
    Description and background of project to test SERVQUAL for use in research libraries.

    Indiana State Library
    Indiana Academic Library Statistics.
    Data collected from 1995, 1996, and 1998.

    Kyrillidou, Martha
    Trends in ARL Libraries. Introduction to ARL Statistics, 1998-99. ARL, 1999.
    Discussion of service, access, and expenditure trends in 121 ARL libraries.

    Library Research Service
    Academic Library Statistics on the Web
    Academic library statistics on state agency websites as of September 1998.


    Assessing the Academic Networked Environment: Strategies and Options. Washington, DC: Coalition for Networked Information, 1996.
    The results from "Assessing the Academic Networked Environment," conducted from 1994-1995.

    National Center for Education Statistics
    Integrated Postsecondary Education Data System (IPEDS).
    A system based on a collection of surveys to collect institution-level data from postsecondary education providers.

    Oregon State Libraries
    Academic Library Statistics.
    Statistics from Oregon academic libraries. Includes data from 1994-1995 and 1996-1997.

    Shim, Wonsik, McClure, Charles R. and Bertot, John Carlo
    Data-Gathering Practices in the Networked Environment. ARL, 213, December 2000.
    Project to develop and test strategies for networked information sources in ARL libraries. Phase I completed in 2000. Phase II to be completed in 2001.

    Troll, Denise
    How and Why Are Libraries Changing? [Draft]. Digital Library Federation, 2001.
    Addresses the need to examine changes in libraries in order to develop a new set of methods for measuring libraries.

    University of California, San Diego (UCSD)
    UCSD Libraries User Survey.
    Performance-based survey of UCSD users conducted in 1996.

    Public Libraries

    Bassman, Keri; Lacompagne, Carole; Korb, Roslyn and Chute, Adrienne.
    Statistics in Brief: How Does Your Public Library Compare? Service Performance of Peer Groups [PDF format]. National Center for Education Statistics, September 1998.
    Suggest public libraries should use peer comparisons based on size for the purpose of gathering and analyzing statistics.

    Bertot, John Carlo, McClure, Charles R. and Eppes, Francis.
    Developing National Data Collection Models for Public Library Network Statistics and Performance Measures 2000 National Leadership Grant from the U.S. Institute of Museum and Library Services (IMLS).
    Project works towards the development of a national data collection model in conjunction with U.S. National Commission on Libraries and Information Science and the National Information Standards Organization.

    Bertot, John Carlo & McClure, Charles R.
    Developing National Public Library Statistics and Performance Measures.
    Funded by the Institute of Museum and Library Services. Project to develop statistics and performance measures for the use of electronic resources in public libraries and library-based statewide networks. Completed May 2000.

    Bertot, John Carlo, McClure, Charles R. and Ryan, Joe
    Developing national network statistics and performance measures for US public libraries: issues, findings and recommendations [PDF format].
    Performance Measurement and Metrics: the International Journal for Library and Information Services, vol. 1, no. 1, April 2000.
    Interim report on this project.

    Bertot, J.C., McClure, C.R.
    Public Libraries and the Internet 2000: Summary Findings and Data Tables [PDF format]. 2000: Washington, DC: National Commission on Libraries and Information Science (NCLIS).
    Preliminary findings and data from the study of the use of the Internet in public libraries.

    Bertot, J.C., McClure, C.R., and Ryan, J.
    The Importance of California Public Libraries in Increasing Public Access to the Internet". 1999: Sacramento, CA: California State Library.
    Evaluation of InfoPeople project and users, and recommendations.

    Cooper, Sandra M., Bolt, Nancy, Lance, Keith Curry and Webster, Lawrence Community Analysis Methods and Evaluative Options: The CAMEO Handbook. Richmond, VA: Virginia State Library & Archives, 1993.
    Guide for public libraries in selecting and using tools to look at libraries and their communities.

    Hennen, Thomas J.
    Great American Public Libraries: HAPLR 2000 Ratings [PDF Format]. American Libraries, November 2000.
    Most recent edition of Hennen's public library rating index.

    Hiott, Judith, presenter
    Public Library Perspectives on Statistics in the Digital Age. Houston Public Library, Houston, TX, 1999
    Overview and slides of the presentation.

    Holt, Glen E., Elliott, Donald and Moore, Amonia
    Placing a Value on Public Library Services
    Project by the St. Louis Public Library to "study to apply economic measurements to value public investment in library services".

    Holt, Glen.
    Panelist, Summary Report of the 1996 Forum on Library and Information Services Policy on Impact of Information Technology and Special Programming on Library Services to Special Populations. National Center for Education Statistics and Co-sponsored by the U.S. National Commission on Libraries and Information Science (NCLIS), 1996.
    Holt's presentation to this forum on cost-benefit analysis.

    Indiana Public Libraries
    Indiana Public Library Statistics. <
    Data from 1994-1999.

    Koontz, Christine; Jue, Dean K. and Lance, Keith Curry

    <Market-based adult lifelong learning performance measures for public libraries serving lower income and majority-minority markets: Final Performance Report September 1, 1996-August 31, 1999.
    Addresses the need for national performance measures and surveys of library usage for libraries in majority-minority and lower income markets regions.

    Library Research Service
    Public Library Statistics on the Web.
    Public library statistics on state agency websites.

    Ontario Public Library
    Ontario Public Library Statistics.
    Annual surveys of Ontario public libraries. Data from 1994 through 2000.

    Oregon State Libraries
    Public Library Statistics.
    Statistics from Oregon public libraries. Includes data from 1994-1995 and 1996-1997.

    Public Library Association (PLA)
    Homepage of the PLA, a division of the ALA. Includes links to PLA publications and other resources.

    Urban Libraries Council
    Impacts of the Internet on Public Library Use.
    Telephone survey (in English and Spanish) of adults during 2000. Funded by a grant from the Institute of Museum and Library Services.

    School Libraries American Association of School Librarians
    Division of ALA. Site contains facts and figures on school media centers, and standards and guidelines resources.

    California Department of Education
    School Libraries.
    Information on statistics, assessment measures, and other data for California public school libraries. Upcoming: school library survey.

    Lance, Keith; Rodney, Marcia J. and Hamilton-Pennell, Christine
    Measuring Up to Standards: The Impact of School Library Programs and Information Literacy in Pennsylvania Schools [PDF format].
    Harrisburg, Pennsylvania Citizens for Better Libraries for the Office of Commonwealth Libraries, Pennsylvania Department of Education, 2000. Study based on the 1993 "Colorado Study" to measure outcomes of school library programs - including use of electronic resources - on academic achievement.

    Library Research Service Statistics and Input-Output Measures for School Library Media Centers in Colorado 1999.
    Results of the 1999 LRS Colorado survey. Data includes technology and electronic resources statistics.

    Library Research Service
    School library media impact studies.
    Performance-based research conducted on the impact of school media centers in Alaska, Colorado and Pennsylvania.

    Maryland School Assessment Performance Program (MSPAP)
    Library Media Skills, K-8, Learner Outcomes.
    Outcome-based evaluation program used by the state of Maryland to test students on the usage of public school media centers.

    Special Libraries

    Association of Specialized and Cooperative Library Agencies (ASCLA)
    Library Standards for Adult Correctional Institutions.
    Division of ALA. Excerpt from the publication.

    Cram, Jennifer
    "Six impossible things before breakfast": A multidimensional approach to measuring the value of libraries. Keynote address, 3rd Northumbria International Conference on Performance Measurement in Libraries and Information Services, 27-31 August, 1999.
    Describes a methodology for measuring service in corporate libraries.

    Indiana State Library
    1995 Special Library Statistics.
    Includes operating expenditures and holdings data.

    Janke, Eberhard
    Revising the German Special Library Statistics: A Quantity-to-Quality Initiative [PDF format]. INSPEL 31:4, 1997.
    Project to revise measurement of statistics in special libraries. Emphasizes the importance of statistics gathering and analysis in the special library sector.

    Oregon State Libraries
    Special Library Statistics.
    Statistics from Oregon special libraries. Includes data from 1994-1995 and 1996-1997.

    Special Libraries Association
    Information Services Panel Surveys.
    Series of surveys conducted by SLA. Data elements include technology usage and trends.

    Spiller, David
    UK special library statistics: the challenge of collecting and analysing data from libraries in the workplace. 64th IFLA General Conference, 1998.
    The difficulties of data gathering and surveying in special libraries; recommended methodologies.

     

    IX. Forum Agenda and List of Attendees