Balanced Scorecard
Metric Results for FY 2003-04
University of Virginia
USER PERSPECTIVE
Metric U.1.a. Overall rating in student and faculty surveys.
Target1: We should attain a score of at least 4.00 (out of 5.00) from each of the major constituencies: undergraduates, graduates, humanities faculty, social science faculty, science faculty.
Target2: Score of at least 3.90 from each of the major constituencies.
Separate Tallies: The ratings for faculty and students will be kept and reported separately.
Result for FY2004: Target1 achieved for faculty. / Student survey is out-of-date.
2004 faculty survey—overall rating:
Humanities = 4.41
Sciences = 4.25
Social Sciences = 4.42
Metric U.1.b. Customer service rating in student and faculty surveys.
Target1: Customer services should receive a score of at least 4.25 (out of 5.00) from each of the major constituencies: undergraduates, graduates, humanities faculty, social science faculty, science faculty.
Target2: Score of at least 4.00 from each of the major constituencies.
Result for FY2004: Target1 achieved for faculty. / Student survey is out-of-date.
2004 faculty survey:
Humanities = 4.70
Sciences = 4.51
Social Sciences = 4.61
Metric U.1.c. ARL index ranking.
Target1: The UVa Library should rank in the top 20.
Target2: The UVa Library should remain ranked in the top 25.
Result for FY2004: Target2 achieved.
UVa Library is ranked 25th in the 2003 ARL Index.
http://fisher.lib.virginia.edu/arl/index.html
Metric U.2.a. Overall instruction rating in user education participant survey.
Target1: An overall rating of 4.00 on a scale of 1 to 5.
Target2: An overall rating of at least 3.80.
Result for FY2004: Target2 achieved.
3.94 overall score
4.04 for upper level instruction
3.73 for 1st year instruction
Metric U.2.b. Use of Special Collections materials.
Target1: Increase by 10% annually the number of materials used or circulated in Special Collections.
Target2: Increase by 5% the number of materials used or circulated.
Result for FY2004: Targets not met.
Collection use for FY2003 = 14,634
Collection use for FY2004 = 11,252
Use decreased by 23.1%
Metric U.3.a. Circulation of new monographs.
Target1: 60% of all newly cataloged monographs (excluding reference books and other non-circulating materials) should circulate within two years of the time of cataloging. Usage of newly acquired or newly digitized electronic monographs will be measured separately.
Target2: 50% of all new titles should circulate within the first two years. Usage of newly acquired or newly digitized electronic monographs will be measured separately
Separate Tallies: The numbers for print material and digital materials will be kept and reported separately.
Result for FY2004: Target1 achieved.
Books in sample: 26,807
Books circulated: 16,639
% circulated: 62.1%
Measure for digital materials was suspended for FY2004.
Metric U.4.a. Turnaround time for processing and filling user requests.
Target1: Satisfy the turnaround targets 90% of the time.
Target2: Satisfy the turnaround targets 80% of the time.
Categories:
Searches: Notify patron of results of search within 4 days of request.
ILL: Fill requests in 7 days.
LEO: Fill requests in 7 days.
ISS: Fill requests by agreed upon date.
New Books: Fill requests in 7 days.
Result for FY2004-Searches : Target1 achieved.
For 89.6% of items requested, patrons were notified in 4 days.
Result for FY2004- ILL: Targets not met.
74% of ILL requests were filled in 7 days.
Result for FY2004-LEO: Target1 achieved.
96.8% of LEO requests were filled in 7 days.
Result for FY2004-ISS: Target1achieved.
Summer2003 = 99.12%
Fall2003 = 98.75%
Spring2004 = 96.98%
Average = 98.28% of requests were filled by agreed upon date
Result for FY2004-New Books: Targets not met.
34% of new books requested by patrons were received in 7 days.
Metric U.4.b. Reduce the number of recalls placed each year.
Target1: Reduce the number of recalls by 5%.
Target2: Reduce the number of recalls by 0%.
Result for FY2004: Targets not met.
FY2003 = 43,272
FY2004 = 46,708
Recalls increased by 6.1%.
INTERNAL PROCESS PERSPECTIVE
Metric I.1.a. Processing time for routine acquisitions.
Target1: 90% of in-print books from North America will be processed within one month.
Target2: 80% of in-print books from North America will be processed within one month.
Result for FY2004: Target2 achieved.
Total processed = 11,053
Processed within 1 month = 9,880
Percent processed within 1 month = 89.3%
Metric I.1.b. Turnaround time for reshelving/refiling.
Target1: Accurately shelve/refile 85% of all materials by the end of the day following discharge regardless of where the item was discharged.
Target2: Accurately shelve/refile 75% of all materials by the end of the day following discharge regardless of where the item was discharged.
Result for FY2004: Target2 achieved.
Items shelved correctly within one day = 81.1%.
Metric I.2.a. Staff survey rating of internal service activities.
Target1: Composite rating of 4.00 of all internal service activities in annual staff survey, with no units rated below 3.50.
Target2: Composite rating of 3.50, with no units below 3.00.
Result for FY2004: Suspended for FY2004.
Metric I.2.b. Internal communications rating.
Target1: A score of 4.00 on internal communications in the annual staff work life survey, with no rating below 3.50 for a single question.
Target2: A score of 3.75, with no rating below 3.25 for a single question.
Result for FY2004: Targets not met.
Score = 3.11, with 40% agreeing or agreeing strongly with communications statements. In the future, we will use the percentage agreeing rather than the rating score for this metric.
Metric I.3.a. Process improvement.
Target1: Complete at least one major process improvement each year.
Target2: Complete at least one major process improvement every two years.
Result for FY2004: Target1 achieved.
Improvement project for FY2004: Special Collections looked at its acquisitions and cataloging processes as well as its organizational structure.
FINANCE PERSPECTIVE
Metric F.1.a. Non-state expenditures as a proportion of total University Library expenditures.
Target1: The target for FY2002, our initial year for Balanced Scorecard, was 10.0%. The target will increase by one half of one percent each following year, i.e., the target for FY2003 will be 10.5%, for FY2004 it will be 11.0% etc.
Target2: The partial success indicator for each following year is .5% less than the Target1 indicator.
Result for FY2004: Target2 achieved.
Total expenditures for FY2004 = $21,700,316
Expenditures from non-state sources = $2,272,785 or 10.5%.
Metric F.1.b. Library expenditures as a proportion of University expenditures.
Target1: The University Libraries should continue to account for at least 4.00% of the E&G (educational and general) share of University expenditures.
Target2: The University Libraries should account for at least 3.75% of E&G.
Result for FY2004: Target2 achieved.
University of Virginia E&G total expenditures for FY2003: $715M
University of Virginia Libraries expenditures for FY2003: $26,867,504
Library budget proportion of total E&G: 3.76%.
Note: The University has been required to adopt a new financial accounting reporting system (GASB 34/35) that does not report an explicit E&G total. The $715M figure is an estimate, derived from the GASB report (Total Operating Expenditures less Depreciation and Auxiliary Enterprises). For future years we may want to adjust this metric and use the GASB model directly.
Metric F.1.c. Amount of development receipts that are unrestricted.
Target1: Using FY2001 as a baseline, increase unrestricted (or minimally restricted) giving by 10%.
Target2: Using FY2001 as a baseline, increase unrestricted (or minimally restricted) giving by 5%.
Result for FY2004: Target1 achieved.
Unrestricted donations total of $354,000 (all from annual giving).
Note: Includes any funds donated (endowment, annual giving, planned gifts, etc.) for which the library may make the decision for its use and which could be used to replace state funds if needed. The baseline of FY 2000/01 was $232,000, meaning that Target1 for FY04 was $309,000, and Target2 was $269,000.
Metric F.2.a. Unit cost of electronic serial use.
Target1: 0% increase in unit cost per use each year.
Target2. Less than 5% increase in cost per use each year.
Result for FY2004: Target1 achieved.
Cost is $1.64 per article used; FY03 cost was $1.64 per article used.
Note: This calculation included resources funded by VIVA as well as those funded directly by UVa. In FY05 we should use only UVa expenditures for this metric. The FY04 unit cost per search for UVa funded databases was $2.34. This will be our baseline in determining if we meet our target in FY05.
Metric F.2.b. Unit cost of online databases.
Target1: 0% increase in unit cost each year.
Target2: Less than 5% increase in unit cost each year.
Result for FY2004: Targets not met.
Cost is $0.59 per search for FY04 and for FY03 was $0.52 per search, a 12% increase.
Note: This calculation included databases funded by VIVA as well as those funded directly by UVa. In FY05 we should use only UVa expenditures for this metric. The FY04 unit cost per search for UVa funded databases was $1.05. This will be our baseline in determining if we meet our target in FY05.
Metric F.2.c. Unit cost of processing library materials.
Target1: Annual percent change in unit cost will be equal to or less than annual percent change in overall library operating expenditures.
Target2: Annual percent change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
Separate Tallies: The unit costs for Acquisitions, Cataloging (including Preservation), and Digital Library Production Services will be tallied and reported separately.
Result for FY2004: Suspended for FY2004.
Metric F.2.d. Unit cost of public service transactions.
Target1: Annual percent change in unit cost to be equal to or less than annual percent change in overall library operating expenditures.
Target2: Annual percent change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
Separate Tallies: The unit costs for ILS (including Instructional Scanning Services), circulation, and reference will be tallied and reported separately.
Result for FY2004: Suspended for FY2004.
LEARNING/GROWTH PERSPECTIVE
Metric L.1.a. Impact of training.
Target1: A rating of 4.00 for overall satisfaction with training activities in the annual work-life quality survey, with no individual training measures below 3.50.
Target2: Overall rating of 3.75, with no measures below 3.25.
Result for FY2004: Targets not met.
Score = 3.52, with 55% agreeing or agreeing strongly with training statements. In the future, we will use the percentage agreeing rather than the rating score for this metric.
Metric L.1.b. Diversity of staff.
Target1: A net increase of at least 4 in faculty/staff diversity, with a net increase of at least 2 in faculty diversity.
Target2: A net increase of at least 2 in faculty/staff diversity, with a net increase of at least 1 in faculty diversity.
Result for FY2004: Target1 achieved.
2 diverse faculty hired, none lost
4 diverse classified hired, one lost
Net increase of 5 with 2 in faculty
Metric L.2.a. Job satisfaction among Library employees.
Target1: A score of 4.25 for overall job satisfaction in annual staff work-life survey, with no rating below 3.50 for a single question.
Target2: A score of 3.75, with no rating below 3.25 for a single question.
Result for FY2004: Targets not met.
Score = 3.60, with 60% agreeing or agreeing strongly with job satisfaction statements. In the future, we will use the percentage agreeing rather than the rating score for this metric.
Metric L.2.b. Retention rate of commended employees.
Target1: 95% of commended employees remain on the staff.
Target2: 90% of commended employees remain on the staff.
Result for FY2004: Target2 achieved.
Classified staff: 64 commended, 62 retained = 97%
Faculty: 40 commended, 36 retained = 90%
94.2% overall
Metric L.2.c. Compare staff salaries (faculty and classified) to peer groups.
Target1: Library faculty salary rating in the upper 40% of all ARL institutions and 60% of classified employees above the mean salary for similar jobs in the State of Virginia.
Target2: Library faculty salary rating in the upper 50% of all ARL institutions and 50% of classified employees above the mean salary for similar jobs in the State of Virginia.
Result for FY2004: Target2 achieved.
Faculty ranked 50 among 114 ARL institutions = upper 44%.
Salary data for positions comparable to classified staff positions in Virginia is incomplete and unreliable.
Metric L.3.a. Reliability of equipment.
Target1: Staff and user workstations (computers, scanners, video, audio, microform, and copy equipment, etc.) shall function properly at least 95% of the time.
Target2: Equipment shall function properly least 90% of the time.
Result for FY2004: Target1 achieved.
The percent of equipment functioning properly was 96.6%.
Metric L.3.b. Renovation of major staff and public areas.
Target1: Three or more major renovations per year of which at least one is a staff area and one is a public area.
Target2: Two major renovations per year of either staff or public areas.
Result for FY2004: Target1 achieved.
The following projects were completed in FY2003/04:
1. Clemons was recarpeted.
2. Clemons reserves room was built.
3. Alderman circulation desk was renovated and combined.
4. Workstation furniture was acquired for Associate University Librarian, Music Library offices, and Acting Director of Cataloging.
Metric L.4.a. Expenditures for digital materials.
Target1: UVa should be in the top quarter of all ARL libraries in our percentage of collections dollars expended on digital materials.
Target2: UVa should be in the top third of all ARL libraries in our percentage of collections dollars expended on digital materials.
Result for FY2004: Targets not met.
For FY2003, UVa ranked 44 out of 111 or in the top 40%.
Metric L.4.b. Usability testing of library websites.
Target1: At least 75% of all websites under development or review should receive heuristic evaluation and at least 50% of all such sites should receive usability testing.
Target2: At least 50% of all websites under development or review should receive heuristic evaluation and at least 25% of all such sites should receive usability testing.
Result for FY2004: Target1 achieved.
10 websites were under development in FY2004.
7 usability tested + 2 heuristics = 90%
Note: The original intent of this metric was to measure whether websites under revision receive either heuristic OR usability testing, not both. The metric is being revised accordingly for FY05. We are measuring according to that standard for FY04 as well.
Metric L.4.c. Increasing the quality of digital access.
Target1: 100% of identified priority content will be produced per DCRT (Digital Contents Review Team) standards.
Target2: 90% of identified priority content produced per DCRT standards.
Result for FY2004: Target1 achieved.
3 of 3 projects planned were completed for 100%.
Metric L.5.a. Develop a culture of assessment within the Library.
Target1: 75% of respondents score 12 or more affirmative responses on the ARL Culture of Assessment IQ.
Target2: 50% of respondents score 12 or more affirmative responses on the ARL Culture of Assessment IQ.
Result for FY2004: Targets not met.
30.4% of library managers scored 12 statements "true" on the ARL Culture of Assessment IQ instrument.