Search

The Open University (OU) is the UK’s largest academic institution dedicated to distance learning, with over 120,000 students. The Library, based in Milton Keynes, was established alongside the University in 1969 to provide academic staff with access to the latest research in their field. The print library was designed to meet the needs of campus-based staff. The design of the teaching materials posted to students meant that they did not require access to an academic library service. At the turn of the millennium, as electronic publishing and home internet access grew, we commenced offering services to students and sought to establish a library that met their needs.\ud \ud We have a strong culture in performance measurement and assessment, which helps to ensure that strategy and service improvement is underpinned by evidence. Since 2012, this effort has been helped significantly by our Student Library Research Panel, a dedicated community of students who work with us in partnership as co-creators (Dick and Killick, 2016). A rolling recruitment effort attracts 500 students annually who agree to work with us for a year to contribute to our programme of continual improvement. Although there is a natural self-selection bias to panel membership, many students who join have had limited or no experience of the library when they agree to become part of the community due to the nature of distant learning. This inclusive approach to our student insight research ensures that our service development takes into consideration both user and nonuser needs. \ud \ud As our assessment culture has grown, one concern about our approach has continued to surface: the panel membership has been restricted to distance learning undergraduate and taught postgraduate students but these are not the only user groups that the library serves. When we developed the panel in 2012, a key objective was to ensure that we captured the views and experiences of these members of our community. They are the largest community we serve, and by their very nature difficult to gain feedback from. While some user experience research was conducted with the predominantly campus-based research students, insight from academic staff was provided only through anecdotal feedback. Feedback from our 9,000 associate lecturers, who teach our students directly and who are predominantly employed on a part-time distance basis, was even scarcer. We were concerned that we were planning service improvements based on student needs alone and not seeing a fuller picture.\ud \ud In February 2018 we started a Library Needs project, seeking to gain a more detailed understanding of current and future needs and expectations of the Library service from the whole OU community by undertaking focussed qualitative research with:\ud • Academic staff based on campus and in other parts of the UK\ud • Associate lecturers\ud • Research students on campus and in other parts of the UK\ud • Students (undergraduate and taught postgraduate)\ud \ud A total of 33 interviews were conducted as part of the project, using a directed storytelling approach to gain rich insights into the needs of our community. The conversations were recorded and transcribed before conducting Conversation Analysis (CA) and Thematic Analysis (TA) to draw out the themes in our very rich data. The findings were used by the library’s extended leadership team, coupled with their professional expertise, to develop the departmental operational plan and objectives for the forthcoming academic year. \ud \ud While the insights gained from the students (and, to some extent, from the research students) corresponded with the research previously undertaken with the student panel, some wider insights from the other community groups surprised us. Positively, we are seen as a prized resource that is central to the work of the University. Participants spoke about the value of the Library in their work, research, and study. In line with our continual improvement culture, however, one area was singled out as in need of improvement: the physical library. Since commencing offering services for student the Library placed a strong focus on electronic resources and we moved increasingly towards a virtual presence. Understandably, this suits our largest community, the distant learning students, but in the process the Library lost some relationship capital with campus-based academic staff and research students.\ud \ud In response to these findings, the Library has partnered with the Estates department to develop a new vision and strategy for the physical library. In line with our assessment culture this has been based on insight. Through observation studies, love letters and break up letters, and guerrilla interviews with both users and nonusers of the building we have identified the needs and expectations of the physical library. This insight, combined with our professional expertise has resulted in the new vision and strategy which we are now working to deliver.\ud \ud The wider learning from this process, however, is the importance of capturing the views of our whole community, including the users and nonusers, from all user groups. To take this forward we are in the process of extending and enhancing the Library Student Research Panel which has served us so well, to become the Library Research Panel, to ensure we continue to gather insight from our whole community. \ud \ud References\ud Dick, S. and Killick, S. (2016) ‘Delighting Our Customers : Building Services Collaboratively with Learners at a Distance’, in Baughman, S., Hiller, S., Monroe, K., and Pappalardo, A. (eds) 2016 Library Assessment Conference. Arlington, VA: Association of Research Libraries, pp. 543–548. Available at: http://oro.open.ac.uk/id/eprint/57807.

Objective – Gothenburg University Library has implemented a process with the goal to combine quality assurance and strategic planning activities. The process has bottom-up and top-down features designed to generate strong staff-involvement and long-term strategic stability. Methods – In 2008 the library started implementing a system in which each library team should state a number of improvement activities for the upcoming year. In order to focus the efforts, the system has gradually been improved by closely coupling a number of assessment activities, such as surveys and statistics, and connecting the activities to the long-term strategic plan of the library. Results – The activities of the library are now more systematically guided by both library staff and users. The system has resulted in increased understanding within different staff groups of changing external and internal demands, as well as the need for continuous change to library activities. Conclusion – Library assessment and external intelligence are important for tracking and improving library activities. Quality assurance and strategic planning are intricate parts in sustainable development of better and more effective services. The process becomes more effective when staff-driven and built upon systematic knowledge of present activities and users.

Objective – Patron counts are a common form of measurement for library assessment. To develop accurate library statistics, it is necessary to determine any differences between various counting devices. A yearlong comparison between card reader turnstiles and laser gate counters in a university library sought to offer a standard percentage of variance and provide suggestions to increase the precision of counts. Methods – The collection of library exit counts identified the differences between turnstile and laser gate counter data. Statistical software helped to eliminate any inaccuracies in the collection of turnstile data, allowing this data set to be the base for comparison. Collection intervals were randomly determined and demonstrated periods of slow, average, and heavy traffic. Results – After analyzing 1,039,766 patron visits throughout a year, the final totals only showed a difference of .43% (.0043) between the two devices. The majority of collection periods did not exceed a difference of 3% between the counting instruments. Conclusion – Turnstiles card readers and laser gate counters provide similar levels of reliability when measuring patron activity. Each system has potential counting inaccuracies, but several methods exist to create more precise totals. Turnstile card readers are capable of offering greater detail involving patron identity, but their high cost makes them inaccessible for libraries with lower budgets. This makes laser gate counters an affordable alternative for reliable patron counting in an academic library.

This paper examines the processes taken to design and administer a collaborative ethnographic study of humanities doctoral students within an inter-institutional, collaborative framework. Project organization and management, including the creation of instruments and analysis of results across two local research teams and institutional cultures is discussed. Effective communications, among and between project teams, and time management were identified as critical factors for success. Benefits resulting from the project included an improved understanding of the needs of a key user group, a heightened interest in user assessment and data-driven decision-making among staff within the partner organizations, and a deeper engagement with academic administrators on both campuses.

A Review of: Passoneau, S., & Erickson, S. (2014). Core competencies for assessment in libraries: A review and analysis of job postings. Library Leadership & Management, 28(4):1-19. https://journals.tdl.org/llm/index.php/llm/article/view/7080 Abstract Objective – The authors sought to determine whether existing definitions of assessment agree with assessment-related skills sought in job postings, and to identify key assessment-related skills, needs for training, and trends in assessment. Design – Content analysis. Setting – Job postings from six library-specific websites: the American Library Association, the Library & Information Technology Association, the Society of American Archivists, the Council on Library and Information Resources, the Association of Research Libraries, and Library Assessment job announcements at http://libraryassessment.info/?cat=13. Subjects – Job titles and descriptions published during an 18-month period between Summer 2012 and Winter 2013 that met the inclusion criteria (n=231). Methods – Job postings were searched and analyzed in two separate sets whose inclusion criteria is as follows: First, job postings with the term assessment in the position title or as the main focus of the position (n=44) were retrieved; of these, three postings were too old to contain descriptions, so were excluded from analysis. Second, job postings were retrieved with the terms assessment, evaluation, metrics, and strategic in the descriptive text of postings with position titles that did not specifically mention assessment (n=187). The full text of both sets was downloaded to ATLAS.ti software for analysis using a grounded theory approach. Mutually exclusive terms emerging from the coding process were documented and defined; from this analysis, networks of code “families” or co-relational groupings helped to create categories and sub-categories. The context of terms was closely examined to understand the meaning of assessment-related terms in job descriptions. Following this step, Microsoft Excel was used to generate tables and pivot tables, aiding understanding and illustrating data. Main Results – All 44 job posts containing the term assessment as part of the position title were from research universities or four year colleges; of these, most were ARL member libraries. For these postings, the concept of assessment was more clearly aligned with definitions of assessment as an ongoing process. The positions described, requiring a minimum of three years’ of experience, ranged from entry-level to administrative in nature. In the second set (187 postings), the interchangeable use of the terms “assessment” and “evaluation” was particularly evident in job postings unrelated to library instruction. No library types other than academic were recruiting for assessment librarians, but related skills, usually referred to as evaluation in public and special libraries, were mentioned in all areas of library practice including instruction, administration, public services, user behavior, and to a lesser extent, access services, archives, information technology, cataloging, and more. While less prominent, these less often mentioned areas of practice also appear to be increasing their awareness of assessment. Key skills and knowledge areas needed for assessment in libraries emerged from content analysis of the job postings. These were grouped under eight main areas of competency and were augmented by the authors’ own experiences as assessment librarians: background in library assessment, research methods, statistical and analytic skills, visualization and presentation skills, and project management and people skills. Conclusions – Based upon analysis of this set of documents, a culture of assessment in libraries appears to be emerging, demonstrating a possible upward trend when contrasted with the earlier research of Walter and Oakleaf (2010). Overall, assessment related skills and knowledge were increasingly evident across all library types and positions. Suggestions for aiding the development of an emerging culture of assessment include fostering liaisons between ALA divisions and library schools to persuade the schools of the need for related coursework, workshops focused on assessment-related skills, certification programs, and a proposed minor in library assessment. Opening avenues for discussion between library types could enhance the growth of an assessment culture beyond academic librarianship. Additional research to better understand the diffusion of assessment culture and practice into non-academic libraries is also recommended.

No abstract.

As Booth notes in a recent commentary on the conceptual and practical links between performance measurement and evidence based library and information practice (EBLIP), there has been a discernable creep among segments of the library community that seemingly existed as two solitudes: those in evidence based librarianship (EBL) circles and those in the library assessment practitioners group (“Counting What Counts” 63). Beginning in 2005, individuals from one group have been showing up at the others’ conferences and events to discuss their methods, frameworks and processes. Are these separate movements within librarianship forming theoretical bridges? Is some sort of merger, fusion or takeover in the future? Or are these simply collegial discussions about our evidence‐based leanings in librarianship? Is all evidence based practice in librarianship, that is, some form of research‐derived data guiding the decision‐making of practitioners, subject to the theoretical framework proposed by the EBL movement? If so, are the tools and practices of library assessment rigorous enough, by EBL theory standards, to afford equal participation in these evidence‐based practice circles or will assessment practitioners forever be relegated to wallow as devotees of the lowest cells of Eldredge’s(2002) exploratory research evidence chart? If we are just now coming to understand our similarities, will our differences be enough that we wish never to be one movement and therefore forever remain as two solitudes in evidence‐based practice?