University Library
University of Illinois at Urbana-Champaign
June 2008
Section 1: Past Assessment Results
University Library has undertaken a number of assessment activities that have impacted student learning. This unit plan represents an opportunity to report on some of those activities while also formalizing a plan for the coming years.
“Information literacy” is the overall, encompassing phrase that describes student learning outcomes in the field of academic librarianship and, specifically, in the University Library at the University of Illinois at Urbana-Champaign.
Though necessarily not a comprehensive list, what follows are examples of assessment activities that have resulted in changes and improvements to the University Library’s information literacy instructional activities.
- The Undergraduate Library has developed a robust instructional program for sections of Rhetoric 105, a composition course for first-year students. Each section receives a one class period instructional session, which utilizes a mix of lecture, demonstration, and hands-on activities for the students. Some classes also include video clips and a short interactive game. In 2004, discussions with the course coordinators indicated dissatisfaction with the instruction program currently in place and the curriculum of the session was re-designed to be more responsive to course needs and to utilize more interactive and student-centered pedagogical approaches. Though much of the assessment is anecdotal, changes have been made based on evaluative comments and e-mail received from the instructors, comments from students during and after the classes, and observations of students as they engage in hands-on activities. Recently, a brief assessment exercise was incorporated into one section at the end of the semester that asked two questions. The questions are: (1) What is the one thing you learned in class today that will be most helpful to you as you do research for your paper? and (2) Is there something about the library or academic research you still find confusing or have questions about? What is it? Analysis of this data will be used as a guide for similar assessment during the coming academic year and be the basis of a more formal assessment project in the future.
- The Reference Library has for many years taught instructional sessions for graduate-level English as a Second Language (ESL) courses. An assessment project for the instruction for ESL 500 focused on gathering feedback from students in order to improve the instructional session but also provide additional information to students on topics which were still confusing for them after the instruction. The Cephalonian Method, a highly interactive instructional method using music, query-and-response, and visual images, is used for instruction, which also includes a demonstration of the Library’s online catalog. The Cephalonian Method enables integrated observation of student understanding based on questions asked and is complemented by an email-based follow-up method that asks students to send one question they still have about the Library after the session. Questions are compiled anonymous and answers are to the entire class. The questions sent by email help the instructor understand the skills and information that the students come into the class with and what types of information they seek from an introductory library instruction session so that changes can be made to the instructional program.
- The Library and Information Science (LIS) Library provides an intensive set of instruction sessions for students in LIS 502LE, which is an intensive, short-term required course during “boot camp” orientation for students in each annual cohort MSLIS distance education degree program (a.k.a. the LEEP program). The four sessions include three hands-on instruction sessions and a tour of library facilities and services and each has its own specified learning outcomes. Assessment for this series of instruction sessions includes an online evaluation form, which students are asked to fill out at the end, that combines feedback on the value and content of the sessions with self-reporting of usage of resources introduced in the sessions; informal gathering of instructor reactions to students’ work; and informal use of the participant observation method when helping students with course-based reference questions at the library or during office hours. Over time, session content and presentation approaches have been altered in response to student and instructor feedback. Examples include adding a web page for the Information Policy Tracking Assignment specifically to address concerns raised by students about the timing of the workshop vis-a-vis individual work styles. Four years of student feedback were also the topic of an article in Public Services Quarterly devoted to the assessment of information literacy instruction. The librarian teaching the sessions commented that she herself “was surprised to see how strongly the assessment had influenced the shape of the instructional design over just four years!”
Additional examples of library faculty scholarship on student learning assessment are incorporated in the Assessment @ UIUC Library website in section Assessment Bibliography – Related Library Faculty Publications(http://www.library.illinois.edu/assessment/biblio.html#pubs)
Section 2: Revised Assessment Plan
(a) Process
The Coordinator for Information Literacy Services and Instruction serves as the lead faculty member for the development and implementation of the University Library’s student learning outcomes and assessment activities. This position was created in 2002. The User Education Committee is the Coordinator’s advisory committee and provides guidance and advice for all aspects of the University Library’s instructional offerings. This assessment plan was a standing agenda item during its 2007-2008 monthly meetings. Input from the University Library faculty was solicited through a presentation and discussion at a faculty meeting and through a survey distributed through LibFac-L, the University Library’s faculty listserv. The Dean of the University Library provided oversight for the entire process through her standing bi-monthly meetings with the Coordinator for Information Literacy.
The Coordinator for Information Literacy and the User Education Committee developed the University Library’s Statement on Learning Goals in 2002-2003, which was endorsed by the University Library’s Executive Committee in June 2003. This document provides a framework for developing and assessing student learning in the University Library’s instructional programs. General questions related to information literacy were included in the general surveys conducted by the University Library (http://www.library.illinois.edu/assessment/surveys.html) in 2004-2006 and are an even larger component on the LibQUAL+ survey (LibQual) conducted in spring 2008.
The Coordinator for Information Literacy and the User Education Committee will continue to oversee information literacy student learning assessment in the coming years. They will work in conjunction with the University Library’s Library Assessment Working Group, a Library committee with the charge of coordinating all of the Library’s assessment activities and on which the Coordinator for Information Literacy serves.
(b) Student Outcomes
The University Library’s Statement on Learning Goals was developed in 2002-2003 and endorsed by the Library’s Executive Committee in June 2003. TheStatement is aligned with and reflective of disciplinary standards for information literacy in higher education.
(c) Measures and Methods Used to Measure Outcomes
Benchmarking to Professional Standards/Guidelines
The Association of College and Research Libraries has promulgated three sets of standards/guidelines that have been used for planning and then informal assessment and evaluation of the Library’s instruction programs and instructional staff:
- Characteristics of Programs of Information Literacy that Illustrate Best Practices: A Guideline (2003)
- Guidelines for Instruction Programs in Academic Libraries (2003)
- Standards for Proficiencies for Instruction Librarians and Coordinators (2007)
Reported Instructional Goals
Most measures of student learning in use are customized to the particular instruction session as described in examples in Section 1 of this unit plan. On a more programmatic level, as librarians report their instructional activity, they are provided with the opportunity to report generally which learning goals and instructional content they are including in their instruction. This reporting is voluntary and approximately 37% of sessions report learning goals (many more sessions are known to have learning goals but the reporting form is cumbersome and so not all instructors take the time to enter the data). An analysis of the goals reported indicate that most session focus on information structure, organization and access, which are traditionally the focus of library instruction. (Note: The most recent complete year of data reported is 2006-2007 and is the basis for the data complied here.)
| Goals Taught | Sessions |
|---|---|
| How information sources are physically organized and accessed. | 62 % |
| How information sources are accessed and used intellectually. | 61 % |
| How information sources are structured. | 53 % |
| To use information effectively to accomplish a specific purpose. | 46 % |
| How information is identified and defined by experts. | 44 % |
| To determine nature and extent of information needed. | 34 % |
| To evaluate information and its sources critically. | 33 % |
| To access and use information ethically and legally. | 20 % |
| Understanding the history of books and/or printing | 7 % |
User Surveys
The Library has also conducted a number of user surveys that have incorporate questions about how students learn about the library and their preferences for doing so as well as faculty preferences for the library’s instructional modes. The LibQUAL+ TM survey captured perceptions of the library’s contributions to people’s abilities. These findings have been useful in understanding how people learn about the library as well as areas for possible programmatic development.
Undergraduate Students Survey (2005)
10. Which of the following methods for learning about the UIUC Library have you experienced, and how useful was that experience? (4780 responses)
| Very Important (5) | (4) | Somewhat Important (3) | (2) | Not Important (1) | Not Applicable | MEAN | |
|---|---|---|---|---|---|---|---|
| I was taught by a friend or classmate | 552 (11.5%) | 799 (16.7%) | 1295 (27.1%) | 248 (5.2%) | 99 (2.1%) | 1787 (37.4%) | 3.5 |
| I was taught by a classroom instructor | 586 (12.3%) | 950 (19.9%) | 1104 (23.1%) | 364 (7.6%) | 163 (3.4%) | 1613 (33.7%) | 3.5 |
| A librarian met with my class | 511 (10.7%) | 527 (11.0%) | 646 (13.5%) | 279 (5.8%) | 172 (3.6%) | 2645 (55.3%) | 3.4 |
| I attended a library workshop | 161 (3.4%) | 211 (4.4%) | 337 (7.1%) | 186 (3.9%) | 138 (2.9%) | 3747 (78.4%) | 3.1 |
| I talked with a librarian | 735 (15.4%) | 655 (13.7%) | 752 (15.7%) | 252 (5.3%) | 107 (2.2%) | 2279 (47.7%) | 3.7 |
| I taught myself | 1533 (32.1%) | 1107 (23.2%) | 1229 (25.7%) | 335 (7.0%) | 161 (3.4%) | 415 (8.7%) | 3.8 |
| I learned from an online tutorial | 126 (2.6%) | 236 (4.9%) | 413 (8.6%) | 248 (5.2%) | 219 (4.6%) | 3538 (74.0%) | 2.8 |
Faculty and Academic Professional Survey (2006) – 1377 Responses
12. Faculty have indicated that it is important for students to find, evaluate, and use information correctly for their assignments. Which of the following could the University Library provide that would be useful to your courses?
| Most Useful (5) | (4) | (3) | (2) | Least Useful (1) | Not Applicable | No Answer | MEAN | |
|---|---|---|---|---|---|---|---|---|
| Librarian presentation during class session | 184 (15.2%) | 146 (12.1%) | 139 (11.5%) | 63 (5.2%) | 70 (5.8%) | 608 | 167 | 3.5 |
| Librarian presentation outside class attendance required | 107 (8.9%) | 156 (13.0%) | 158 (13.1%) | 76 (6.3%) | 90 (7.5%) | 615 | 175 | 3.2 |
| Web tutorial completed outside class as assignment | 182 (15.0%) | 175 (14.4%) | 153 (12.6%) | 55 (4.5%) | 58 (4.8%) | 591 | 163 | 3.6 |
| Library produced materials that instructor presents in class | 104 (8.7%) | 185 (15.4%) | 168 (14.0%) | 90 (7.5%) | 63 (5.3%) | 589 | 178 | 3.3 |
| Librarian-taught credit course on general library research | 102 (8.5%) | 121 (10.0%) | 164 (13.6%) | 105 (8.7%) | 103 (8.5%) | 610 | 172 | 3 |
| Librarian-taught credit course on subject-specific library research | 118 (9.8%) | 130 (10.8%) | 161 (13.4%) | 85 (7.1%) | 103 (8.6%) | 605 | 175 | 3.1 |
LibQUAL+ TM Faculty, Student, and Staff Survey (2008)
Please indicate the degree to which you agree with the following statements:
(1 = Strongly Disagree; 9 = Strongly Agree) – 703 Responses
| Mean | SD | |
|---|---|---|
| The library helps me stay abreast of developments in my field(s) of interest. | 6.59 | 1.93 |
| The library aids my advancement in my academic discipline or work. | 7.18 | 1.77 |
| The library enables me to be more efficient in my academic pursuits or work. | 7.18 | 1.81 |
| The library helps me distinguish between trustworthy and untrustworthy information. | 6.05 | 2.09 |
| The library provides me with the information skills I need in my work or study. | 6.41 | 1.98 |
Section 3: Plans for Using Results
(a) Plans
In the development of this Unit Plan, one theme emerged on a recurring basis: the need for professional development for librarians to increase familiarity, comfort, and facility with assessment methods and processes. To this end, the Coordinator for Information Literacy Services and the User Education Committee’s Professional Development Working Group have begun to develop a series of professional development opportunities around assessment, primarily workshops on different topics and tools. Discussion forums will also be scheduled. The Instructional Services Librarian in the Undergraduate Library has also been nominated to participate in the Association of College and Research Libraries’ Assessment Immersion Program. The program will only have an acceptance rate of 50% so it is not yet known if she will be admitted. Librarians attending conferences have been asked to look for sessions on assessment and to share their notes with their colleagues. Librarians have expressed interest in having “clickers” available for in-session formative assessment and the Coordinator for Information Literacy Services is pursuing necessary funding to acquire these through either Library Facilities or Library Information Technology. Finally, more systematic benchmarking of the Library’s instructional programs using Professional Standards/Guidelines will begin on a regular basis as part of the Library’s Unit Annual Report process.
(b) Timeline for Implementation
Professional Development – Workshops to begin in Fall 2008
Benchmarking – Beginning with 2007-2008 annual report
This was a plan submitted as part of the campus accountability self-study in 2008.