Library Committee Handbook

Executive Committee



Scoring Guidelines: FRC Annual Merit Review for 2004 Annual Reports

The UIUC Library Faculty Review Committee is charged with peer evaluation of Library Faculty for merit. The guidelines below are used by the committee to score the librarianship, research, and service sections of faculty annual reports.

 

Under current procedures committee members read each individual Library faculty member’s Annual Report. Using the criteria listed below committee members assign a score of between 0 and 10 for each of the three areas of the annual review, allowing for .5 increments within and between categories. The scores for the three areas of evaluation are weighted with librarianship receiving a 50% weight, research receiving a 30% weight, and professional service receiving a 20% weight. The end result is that each faculty member receives an aggregate score of between 0 and 100. FRC members do not score their own annual reports, nor those of faculty members related by blood or marriage.

 

Committee members score each report on individually on its own merits using these guidelines and do not consult with other committee members during that process. Scores are entered into a database by the committee member. After this the scores are cumulated and all members meet to discuss the scoring process.  During the committee discussion each faculty member is discussed in turn and members of the committee can adjust their score if they choose to do so. Once the scoring process is completed the scores are forwarded to the University Librarian.

 

Scoring in all categories is based on the criteria below. In all of the categories a score of 10 is intended to recognize unusually high achievement and will be awarded only rarely.

 

LIBRARIANSHIP: 50%

 

Evaluation factors:

Librarianship assessments are based on each faculty member’s self reports. With little externally verifiable criteria, and with a wide variation of faculty jobs being reviewed, the standards for evaluating librarianship are necessarily vague. Evidence of truly exceptional leadership, innovation, exceptional productivity, and creativity would contribute to scores of 8 to 10.

 

Please note:  The librarianship section of annual reports is restricted to three pages. Beginning with the 2004 annual reports 1 point will be deducted for reports that exceed three pages. This limit will be strictly enforced.

 

0-3       Unsatisfactory performance in multiple areas

 

4-5       Needs improvement; performance may be satisfactory in one or more areas but unsatisfactory in others

 

6          Satisfactory performance in most parts of the job

 

7          Good—Satisfactory performance in most parts of the job

 

8          Very good—Better than satisfactory performance in most parts of the job

 

9          Excellent performance

 

10         Outstanding performance. A score of 10 reflects an extremely unusual set of achievements and will be awarded only very rarely.  

 

RESEARCH: 30%

 

Evaluation factors:

The scoring criteria below are a rough guide for committee scoring but can be affected by a variety of factors. Scoring involves a more nuanced assessment of the reported work than simply counting items on the report. Factors that affect the scoring of research activities include the prestige of the publication, quality of the research, and whether the work is in print. For books the committee considers type of book including the amount and nature of writing involved. For articles the type of article is considered including length and the amount of research underlying the article. If an article in a journal is also published as a chapter in a book it will be counted only once. Committee members will have the prerogative of assessing the relative importance and prestige of peer-reviewed journals, conference proceedings, and other venues for publication. Examples listed for each score range are not exhaustive or exclusive and are intended only as a guide to assessing research activities. 

 

0          No visible research activity in the past three years

 

1-3       Some research activity in the past three years (for example, presenting a paper at local or state conferences, a book review, an article submitted or accepted for publication; books in progress without a contract, or books with contract with little or no evidence of progress).

 

4-5       Evidence of a focused research agenda and the promise of its completion. (As an example, the faculty member will have at least one article in print and at least one other in progress during the past three years, subject to the evaluation factors listed above.)

 

6-7       Indication of a developing and active research program including publications in the last three years (for example at least two articles, conference proceedings, or book chapters during the past three years; evidence of progress on books with contract; or support through library research grants), subject to the evaluation factors listed above.

 

8          Evidence of a maturing and ongoing research agenda. For example, three or more articles in major/peer reviewed journals, conference proceedings, book chapters or book in print for the past three years, or support through campus research grants, all subject to the evaluation factors listed above.

 

9          Highly productive research work over the last three years. This could include more than three articles (in major or peer reviewed journals), plus conference proceedings, or the publication of a book during the last three years or support through outside campus research grants, subject to the evaluation factors listed above.

 

10         Highest level of scholarship and an active on-going research program during the last three years. To achieve a score of ten, it is expected that the faculty member would have some combination of the following activities including invited papers from respected organizations or sponsors, peer-reviewed and/or prestigious publications, heavily cited publications or seminal works, and significant outside support through honors or significant research grants. A score of 10 reflects the very highest achievements in research will be awarded only rarely.  

 

SERVICE: 20%

 

Evaluation factors:

The scoring criteria below are a rough guide for committee scoring but can be affected by a variety of factors. Scoring involves a more nuanced assessment of the reported service than simply counting items on the report. Factors that affect the scoring of service include the prestige of the organization in which the service takes place, the prestige of the position or activity within the organization, the process of appointment to or of obtaining the position, the amount of work involved, the impact of the service on the organization, and the impact of the service on the profession. Examples listed for each score range are not exhaustive or exclusive and are intended only as a guide to assessing service activities.

 

0          No service activity.

 

1-3       Items in this category assume a minimal amount of involvement and/or leadership. Examples include Membership in a regional/national/international professional organization, attendance at a national/international conference, participation in local workshops, coursework or workshop taken to enhance work-related professional development, or community activities related to professional service.

 

4-6       Items in this category assume a more than minimal amount of involvement and/or leadership. Examples include participation in a national/international committee, participation in a local/state/regional committee, or membership on a campus committee.

 

7-8       Evidence of ongoing and progressively more responsible service. Items in this category assume recognition of the individual involved as an expert in a particular field. Examples include appointment to a prestigious campus committee; reviewer for grant proposals (e.g., National Endowment for the Humanities); formal consulting, paid or unpaid; presenting workshops, demonstrations, or lectures; serving as an outside evaluator or referee (e.g., promotion and tenure); service on thesis or dissertation committees; papers read at a conference; or significant community activities related to professional service.

 

9          Higher level contributions to service. Items in this category assume contributions beyond mere membership. Examples include being chair of a state, regional, national, or international committee; significant contribution to the work of a committee (e.g., preparation of bylaws or reports for committees or other professional groups); editorships of newsletters or columns in newsletters; papers at a major conference; or invited papers.

 

10         Highest levels of leadership. The score reflects the highest level of service activities and an active on-going service record during the last three years. This score is awarded only rarely.