This paper derives from research commissioned by the European Commission and undertaken in 1994 by De Montfort University in partnership with Essex County Libraries and the Library & Information Stat istics Unit at Loughborough University. Publication of the full report of that work by the E.C. is scheduled for Spring/Summer 1995. Informally it was known as the Toolbox study. Publication of the findings here is by agreement of the E.C. The very substantial contributions of colleagues to the text of this paper is acknowledged - particularly that of Suzanne Ward, who will be named as joint a uthor in any published version of this paper.
This paper is in three parts. After the Introduction Part I considers shortcomings in current performance indicators - as revealed by the recent study - and proposes a general strategy to make best use of present computer power. Particular attention is drawn to the weak area of Cost:Output ratios. Part II outlines proposals to improve and initiate particular performance measures. Part III con tains some observations on the applicability of these techniques to libraries of different type and size.
Performance Indicators are considered in a broad context. As an important management technique there is not intended to be any rigid distinction between them and the I.T. framework for management pr ovided by Management Information Systems (MIS) or Decision Support Systems (DSS) or later developments. Their purpose covers:
The aim of the Toolbox study has been to act as a stimulus to demonstrate not only how performance measurement can be carried out, but how it can be executed by taking advantage of the advanced autom ated environment which will soon be commonplace in all but the smallest libraries.
Having examined the extent of Performance Indicators in practice - and their shortcomings - the study then proceeded to put together a Toolbox of performance indicators and measures. This was design ed both to be comprehensive and innovative. All the main indicators already developed were intended for inclusion, with reference to existing descriptions as appropriate. New and extended indicator s were added to meet general and particular shortcomings identified - and to employ latest computing facilities. This article considers only the most significant and innovative features.
There are two levels of development called for. The first - and the longest - is to raise the overall standard of performance measurement to that already practised by pre-eminent library managers. In each country, and between countries, there are large and crucial gaps between the average and the exemplary.
The second level involves work at the leading edge to produce more effective and flexible systems for the leading institutions. There is substantial scope here to utilise the power of contemporary so ftware.
Most of these weaknesses can potentially be overcome; the ways in which this can be achieved are discussed in the rest of this paper. However there is one shortfall relating to Cost:Benefit ratios t hat is so significant that it calls for special explanation.
The main problem can be presented quite simply. The obvious indicator is the Cost per Loan calculated as Total (Net) Expenditure divided by Total Issues. This is extremely c rude and is mostly to be avoided - since it makes no allowance for the extent of non-lending use of the library. Increasingly this is significant where there are:
In broad terms there are two approaches to resolve this problem - both in theory and as evidenced in practice.
Solution 1 concentrates on the output measure by replacing ‘loan’ in the ‘Cost per loan’ formula with a composite measure totalling either all the principal uses provided or all types of document delivery.
A ‘total activity’ example from a Swedish public library comprises:
Solution 2 is the Functional Cost Analysis approach where costs involved in every principal service are estimated to produce a separate Cost per loan calculation for each main service provided.
The main feature is the analysis of staff costs according to staff time spent on main functions in each location. The main functions are:
It should not be necessary to install any sophisticated or routine time clocking procedures. Annually each member of staff should be asked to apportion the percentage of time spent on principal func tions - this to be checked and agreed by management. These percentages are then applied to annual staff cost figures and grossed up by an allowance for indirect and overhead charges.
It is more straightforward to allocate other costs. Materials are conventionally charged to adult lending, adult reference, children’s audio/visual, etc. and/or analysed as books, periodicals, CD-ROM s, music, maps, audio/visual, etc. Premises costs are allocated according to space used for different functions. Automation and other costs are allocated as appropriate - typically in proportion eith er to space or to staff time employed on each function.
The results of this approach are cost:output ratios which are more exact and meaningful than any large composite measure can be. In a large library they are vital for informed allocation of resources and to inform decisions on service expansion.
Why has this not happened already? Factors that have inhibited such a functional cost analysis in the past include:
The fourth factor should be overcome by modern relational database software on platforms now within a feasible price range. The fifth factor has been considered above: it is essential not to confuse this system with that required to monitor and control spending levels. The first three factors are - in effect - the subject of this study: there are many decision areas where managers are currently navigating in financial shadows.
Large organisations need to estimate costs in four dimensions:
The requirement (for large units) is not to provide extensive weekly or monthly reports covering all levels of measure and indicator. The requirement is to provide facilities for cost:output calculat ions, or cost allocation ratios, on an infrequent enquiry basis as and when these are required to support particular investigations or decision steps. For this purpose the requirement is for data to be readily accessible and conveniently manipulated for special reports and for ad hoc enquiries.
The requirements derive from the expressed needs of library managers - and from the logic of the situation. It is not a case of putting new technology to use because it is there, but rather that tech nology will now permit what has been a requirement for decades. The library manager needs to be empowered, not inhibited or impoverished. This is the strategic approach that underlies the innovative aspects of the Toolbox.
Working this out represents a major change in approach. Generally systems analysts aim to hold and present only data which is specifically known to be in demand. The proposal here is for data to be h eld for those occasions when it might be required - often in several years’ time.
In this concept data are entered and held with regard to their ultimate not their immediate use. For example to hold date of birth, date of registration and post code area will allow not only correc t calculation of fines and privileges but also analyses of lapsed users and comparisons of users:age/sex profile with residents in areas served. Studies can be made of library use by students by yea r and by faculty/department - to compare with previous years’ experience.
How long is “the recent past”? Practitioners consulted in the course of our study focused on the last ten years rather than the last five. For such a database the “future” should at least cover the budget for the present year and a forecast for the year following. Beyond that lies strategic or medium/long-term planning. There will be important connections between actual performance indicators a nd longer-term plans.
For how many indicators and measures should historic data be held? This is not easy to decide, but the following guidelines should be helpful:
We give here examples to illustrate the technique rather than the comprehensive set of applications.
To determine the time taken to acquire items and to make them available to users the following steps are undertaken:
For more detailed analysis each individual stage through ordering, checking, cataloguing and processing can be analysed separately, if data is collected at each point. This is an important measure to identify bottlenecks and delays at different stages in the supply process.
This sampling technique should also be applied to the Overall speed in satisfying requests - the time taken to satisfy all types of request for material not immediately available on site. This covers time taken for delivery of material to be supplied through acquisition , inter-library loan, fetching from other sites/service points , and material recalled where it is already on loan. Each of these is also an indicator in its own right.
To establish the proportion of the population using the library we need to know the number of people from the target population who have used the library during the last year. This can be obt ained by one of two methods: either (1) In a survey of the target population, people are asked whether or not they have used the library during the last year; or (2) Establish from a questionn aire survey the proportion of users who never borrow material and apply this percentage to the number of active borrowers .
12260 ------------ = 14,256 (1-0=0.86)
To overcome this weakness a modification to the A.L.A. Needs Fill Rate questions is proposed by including a question:
If you were looking for information (i.e. to find something out), were you successful? YES / NO / PARTIALLY
Did you consult a member of staff? YES / NO
The results assess:
(i) ‘Library Services’ includes space used for reading, studying, information delivery, computers and any other services delivered to users by library staff.
(ii) Library operations includes receipt of materials, bindery, acquisitions, cataloguing, computing, and management.
(iii) Materials storage includes all areas principally devoted to materials whether open access, closed access, special collections or reserve stock.
(iv) ‘Special events’ includes seminar and meeting rooms, space for groups to meet, and exhibition space.
(v) Miscellaneous includes cafe’s, toilets and staff recreation areas.
(vi) Access measures space required only for access to other areas, ie. corridors and gangways.
(v) and (vi) need not be separated; in small libraries categories (i), (ii) and (iii) will be sufficient.
Measurement of space is useful to review allocation of the space provision and to establish or avoid the need for new building. A further analysis by principal services delivered will also be valuabl e in many situations, eg. space taken up by special collections, local history, special information services, etc. To optimise space use is an important objective.
Measurement of floor areas can often be taken from data used for other purposes, eg. cleaning contracts, insurance quotations. Space measurements should be checked for an annual review, but there is no need for frequent recalculation. Precision in the allocation of space between categories is not essential. This measure should not include space occupied by theatres, museums, concert halls where these are not used for ‘library’ purposes but may be physically on the same premises.
There is more scope for Title counts in sets of Performance Indicators than is indicated in recent literature. To assess the staleness/appropriateness of stock a count of titles that have issued duri ng the year (or quarter) can be compared with the number that have stayed on the shelves. This can be particularly revealing for separate sections of stock, although it does not, of course, cover use on the premises.
Titles added per capita, Copies added per title added, and Titles issued per capita are all indicators revealing the extent to which multiple copies of bestsellers or textbooks conflict with objectives of stock variety and depth of interest.
Multiplying Service Points by Hours Open (to give Service point hours open) is an improvement, but Total service point hours can give a very misleading picture where there are many service poi nts of variable size open for different total hours per week.
To overcome these problems a new measure is proposed:
Method: Opening hours for each service point are weighted according to the size/use of that service point to produce an average figure. Weighting factor can be either Issues, Floor Space, St aff numbers, Stock, items On Loan, Readers' Seats or Visits - whichever is most appropriate or critical. For public libraries Issues or items On Loan will be favoured. For academic libraries it may w ell be Readers' Seats (student bias) or Stock (research bias).
For the purpose of illustration we choose bookstock for the weighted average calculation, which then gives these results for 1993 and 1994.
1993 1994 Bookstock OpeningHours Mult'n Bookstock OpeningHours Mult'n '000s '000s Central Library 50 x 60 = 3000 50 x 48 = 2400 Large Branch 30 x 50 = 1500 32 x 48 = 1536 Medium Branch 20 x 44 = 880 18 x 42 = 756 Small Branch 6 x 24 = 144 5 x 24 = 120 New Small Branch - - 4 x 24 = 96 106 178 5524 109 186 4908 Weighted Average 5524 4908 ---- = 52.1 ---- = 45.0 Opening Hours 106 109In this example the Service Point Hours count would have shown an increase of 186/178 = +4 per cent, whereas the more realistic Weighted Hours Open shows a reduction of 45.0/52.1 = 14 per cent!
This measure refers to hours services are physically available to users on library premises. Distinction can be made between (a) full services, and (b) partial services. In academic libraries separat e counts are needed for term time and vacations.
Some of the most significant variations are these:
The principles underlying the toolbox apply in all these situations, but the appropriateness of particular indicators will vary greatly.
School libraries were not considered as part of this project; special libraries and college libraries have received less consideration than university and public libraries. National libraries or regi onal co-operatives have also not been considered.