66th IFLA Council and General
Jerusalem, Israel, 13-18 August
Code Number: 112-126-E
Division Number: III
Professional Group: Public Libraries
Joint Meeting with:
Meeting Number: 126
Simultaneous Interpretation: No
Quality management and self assessment tools for public libraries
Margaret Kinnell Evans
Information Strategy and Services
Library and Information Statistics Unit
Loughborough University, UK
E-mail: : M.Evans@lboro.ac.uk
QUALITY EXPECTATIONS IN THE UK PUBLIC SECTOR
Pressures to demonstrate value for money and also to prove their worth, especially in making a measurable impact on their community, have become more intense for all public sector services in the UK. These can be seen in the following ways:
- External inspections and audits by government agencies such as the Audit Commission, OFSTED, the Further and Higher Education funding councils;
- Measures of service effectiveness used to produce 'league tables';
- Market testing of public sector services;
- Quality and Value for Money initiatives such as Charter Mark and Best Value.
Quality management in public library services
Local authorities, government agencies and the National Health Service have therefore adopted 'total quality' and marketing-oriented approaches as a cornerstone of their management principles and practices. (Kinnell and MacDougall, 1997). Quality management, using various methods, has been tested by public and academic library services: ISO 9000 and its derivatives, Charter Mark, Investors in People and other elements of Total Quality Management. Performance indicators and measures have been developed, with special interest being shown in measuring the impact of services as well their take-up by users. (Linley and Usherwood, 1998).
However, there has been poor sustained take-up by library and information services. This is the case across the library and information sector, but is especially pronounced in public libraries. (Brockman, 1997). The reasons have been identified as:
- quality models are too commercially oriented
- the language and concepts of quality do not embody public sector values
- the ethics of information provision are not addressed in quality models.
There is therefore a need for quality approaches that librarians can accept as being much more relevant to library and information services.
The public sector benchmarking project
One of the most significant initiatives to take quality forward for the public sector in the UK using a distinctive, but complementary, approach to existing quality models has been the public sector benchmarking project. This was launched by the Government Cabinet Office in April 1996 as a pilot with 30 organisations, and a second phase, which ran from April 1997 to January 1998 was pronounced a 'runaway success'. (Samuels, 1998). The third phase, from April 1998 to 2001, aims to support the Government's continuing drive to achieve best value in the delivery of public services, by exploiting the value of benchmarking to support agencies and other public sector organisations, in meeting the challenge of improving the management of their operations within tight financial constraints.
The Business Excellence Model of the European Foundation for Quality Management was used in the project. This model had been developed from 1989-1991 and used in around 200 private sector organisations across Europe. It built on other quality models and used the concept of self assessment, so that managers could assess their own organisation against nine criteria, each weighted to account for their significance. The self assessment against the Model provided a diagnostic framework which enabled organisations to:
- measure their performance, identifying strengths as well as areas needing improvement
- prioritise improvement activities
- measure their effectiveness.
Prior to the use of self assessment the measurement of an organisation's achievements had tended to be subjective. If managers were prepared to apply for certification or awards such as Charter Mark or ISO 9000 then external validation was available. The Malcolm Baldrige National Quality Award and the European Quality Award have both been attempts to increase the competitiveness of companies. (Easton, 1995). Emphasis in the public sector benchmarking project on aligning management with the best commercial practice has therefore been influential in encouraging managers similarly to provide external evidence for achievement.
Those organisations who took part in this Government project were positive about its benefits - they saw considerable scope for the Business Excellence Model as a tool to help achieve real improvement in managing their activities. Researchers at Loughborough and Sheffield universities decided subsequently to explore the potential to develop quality management through self assessment for library and information services. The results of their study provide the means for library and information service managers to align services with best practice across both the public and private sectors.
SELF ASSESSMENT FOR THE LIBRARY AND INFORMATION SECTOR
The Library and Information Sector Improvement Model (LISIM)
The motivation for undertaking the Self Assessment project for library services was that there was an urgent need to develop more effective quality assurance for library and information services. A number of studies (many of them summarised in the Aslib Review of public library services (Aslib, 1995)), have shown that public libraries need to move forward in developing their management, especially the planning of services. The Department of Culture, Media and Sport (the Government Department responsible for public library services) requirements for annual library plans as part of a three-year strategic planning cycle were an important stimulus to developing a more rigorous - and at the same time more user-friendly - quality model.
The British Library Research and Innovation Centre two-year study to examine the potential of self-assessment for public library services was directed by Professor Margaret Kinnell Evans (Loughborough) and Professor Bob Usherwood (Sheffield). Dr Kathryn Jones was the researcher for the project. Earlier work at Loughborough and Sheffield Universities had shown that, despite the need for quality management models to be implemented by public library managers, there was a lack of understanding of the range of tools available to them and how these could be adapted for library services. (Milner, Kinnell and Usherwood, 1997).
Three approaches were identified as having potential. Each shared the following characteristics:
- they were applicable to the public sector
- they stressed the importance of human resource needs and customer satisfaction, both key issues for public libraries
- they enabled library services to integrate other quality programmes already in place, initiatives such as:
- Customer Satisfaction Surveys
- Charter Mark
- Investors in People
- Quality Standards and Specifications.
Of the three self-assessment approaches to quality management selected for consideration the EFQM Business Excellence Model was highlighted at an early stage because it offered the most structured approach through its nine assessment criteria and 32 sub-criteria. However, it was essential to take on board the criticisms that had been levelled at previous attempts at implementing quality management in library services, especially the lack of a public service orientation in some of the approaches. The two other models examined therefore influenced the way in which the Business Excellence model was interpreted for use by library services.
The approaches which formed the basis for the investigation, were:
1. The Business Excellence Model (EFQM, 1997)
This model had been adopted by Government and applied to public sector services. However, it had been developed initially for the commercial sector and was focused less on social impacts than on business effectiveness.
2. The Quality Framework (Stewart and Walsh, 1989)
This also offered a foundation for those who wanted to begin developing quality management in the public sector. It was argued that public services operate within a context which requires special consideration, with emphasis needed on relationship building and service surroundings, as well as the core service to be delivered.
3. The Democratic Approach (Pfeffer and Coote, 1991)
This offered a further understanding of quality management in the context of the modern welfare state, with an emphasis on the key ideological differences between the public and the private sectors. The main issue was identified as that of the public sector serving the interests of the community as a whole, as well as meeting the needs of individuals within the community.
Developing and using the LISIM
Three local authorities were selected by the project team as case study demonstrator services, in order to test the applicability of each of the approaches: a shire county, London borough and metropolitan authority. Links were established with German and Swedish library authorities who tested some of the self-assessment tools and the draft self-assessment tool kit.
There were two phases to the project, which mirror how a library service should implement a self assessment programme:
1. an initial decision to commit to self assessment
2. an implementation phase which takes the process forward and levers out the benefits for the service.
In Phase 1 a quality audit was undertaken in each of the local authority public library services. The planning, communication, review and assessment of quality initiatives were identified and assessed. Issues relating to customers, the management of people, stakeholder involvement and the planning and evaluation of service initiatives were then discussed in feedback sessions and a workshop was held. This provided further feedback and validation of the interim analysis.
In Phase 2 the quality approach or model the library services wished to use were identified through collaborative working between the research team and senior managers. A process of detailed iteration was the principal method. At this stage the core values of the 3 approaches were introduced to the managers and they then identified those values which were a priority for their organisation. The core values identified by library managers were:
- customer focus
- equity in service
- an open system
- public participation
- visible and visionary leadership
- employee development
- involvement and satisfaction
- continuous improvement
- management by fact
- partnership development
- consistency of purpose
- process management
- public responsibility
- stakeholder consideration
A gap analysis of the fit of the demonstrator services against the approaches was then undertaken. This analysis formed the basis of the self-assessment model that was selected as the most appropriate to all the services. The model was then used by two of the demonstrator services (the third had organisational problems and was unable to complete the work) to undertake self-assessments.
Any library service seeking to implement self assessment should undertake a similar examination of its core values as part of the self assessment process. Self assessment can then be owned by all of the stakeholders.
Finally, the model which emerged - The Library and Information Sector Improvement Model - was sent out for feedback and comments more widely in the LIS sector, in order to produce a tool-kit and training pack that would achieve wide acceptance. A selection of academic and special librarians were asked to evaluate the model in terms of their organisations' quality management needs as it was considered that the model had value for LIS in a range of contexts.
A number of issues emerged, which meant that the Business Excellence model had to be refined for use by our demonstrator authorities. These were:
- defining quality in the public information sector (what does it mean?)
- defining the users of public sector information services (who are the customers?)
- providing adequate performance measures ('whose quality is it anyway?')
- ensuring long-term commitment when quick returns are required.
Interestingly, the EFQM was conducting a revision of the Business Excellence Model in tandem with our research. Their resulting Organisational Excellence model took on board many of the issues which we had shared with other organisations, including a police authority, local education authority, county council and National Health Service trust in a public sector network group.
The LISIM: a summary
Library and information service managers were involved at every stage in the development of the final model. They wanted it to:
- offer a supportive framework or structure for understanding the often disparate management activities in library and information services
- offer guidance on how LIS can improve their current management practices and achieve excellence, through a staged approach to continuous improvement
- help ensure that the LIS retain a high degree of autonomy in the way they approach self-assessment.
The principles of the LISIM
The following table summarises the principles established with the demonstrator authorities, and which underpin the model.
||Whilst the model presents criteria for assessment and offers examples of good practice, it does not prescribe the approach which the library service should undertake. It offers a broad spectrum of issues which might be considered, however it is up to the library service in question to determine how relevant these are in the context of their own plans, policies and strategies.
|All plans, policies and strategies should be deployed in a structured and systematic way across the whole organisation and all its activities are co-ordinated and aligned to them.
|Should be the focus of all work practices and procedures and should be embedded in the culture of the library service. Assessment measures should be aligned to goals, targets and objectives in order to facilitate a structured and systematic approach to continuous improvement. Excellent organisations are expected to provide evidence of year-on-year improvements in key results areas. Continuous improvement is also about using the review and assessment process to drive improvement.
||Excellent organisations are expected to benchmark key result areas and be able to provide evidence of an improving trend when compared to good practice organisations. Excellent organisations are also expected to provide evidence of how they have used process benchmarking to drive improvements.
|Relevant and accurate information should be the basis of planning and improvement decisions within the library service.
|The commitment of senior management is vital to the success of self-assessment. They drive the planning and improvement activities of the organisation.
|Meeting the needs and expectations of external stakeholders is inherent in the model. Instead of focusing inwards, the library service should be addressing the management of customer-facing services. The stakeholders of a library service are those people or organisations who have a stake in the service. 'Stakeholders do more than simply use the library: they care about its success, they promote its activities, and they are lobbyists on its behalf' (Weingand, 1997, 58). These might include: staff, customers, councillors, council departments, funding councils, book suppliers, electronic information providers, library networks e.g. EARL, JANET, SELPIG.
|The delivery of quality library services is dependent upon motivated and committed employees. Therefore systems should be put in place to ensure that they are supported in their role.
A full explanation of the model, together with a training pack which can be used to implement it in any library and information service, is available. (Kinnell, Jones and Usherwood, 1999).
CONCLUSION: USING SELF ASSESSMENT IN LIBRARY AND INFORMATION SERVICES
The research undertaken by Loughborough and Sheffield universities therefore had a very practical outcome: the design of a model for implementing a holistic quality improvement programme. The programme would reflect best practice in the commercial sector, but imbue quality management with the values needed to deliver a public library service to meet both individual and community information needs. How to set about delivering a self assessment programme which will improve overall effectiveness, is set out below. Essentially, they are the steps required for any successful change management activity.
1. Identify the role of self assessment
The library service should identify the role or purpose of the self assessment. These are likely to include:
- to review the management practices of the service
- to provide a focus for implementing total quality in the service
- to identify what the service is achieving against models of best practice.
2. Commit to the process
The route to self assessment begins with the initial decision to commit to the process. All involved in the assessment need to be fully informed of their duties and responsibilities in carrying out the task. This requires:
- the involvement and consent of key staff to ensure that the exercise runs smoothly
- the cooperation of all staff to maximise the potential benefits to the service
- the necessary resources to implement self-assessment
- the motivation to undertake post-assessment improvements.
3. Identify the self-assessment team
There are four clear issues:
- who will have the responsibility for administering the self-assessment
All of the recent work on quality management for library and information services shows that the key issue is not just how to implement self-assessment - which can be achieved using the tailored LISIM model - but how to sustain it as part of mainstream planning activity. The following critical success factors for maintaining self assessment are:
- regarding self-assessment not as a static or one-off project, but acknowledging the need for acting on the outcomes with improvement plans and cycles;
- aligning self-assessment with the planning structure of the library and information service: making it an integral part of the data gathering process;
- having marketing strategies in place to create an awareness of what the library can do thereby ensuring the support of key stakeholders for the process.
4. Choose the self assessment model/approach
While we recommend the LISIM, there is a range of other approaches, including the new EFQM Organizational Excellence model. The LISIM is unlikely to need tailoring for content, but the way it is implemented can be varied. The toolkit provides three methods, each offering different levels of thoroughness and accuracy.
- A matrix approach - the library service can match its current position on the matrix and identify where it fits good practice.
- A simple questionnaire which can raise awareness and facilitate discussion of performance issues.
- A pro-forma, which offers a comprehensive self assessment, including a robust scoring mechanism and the means of identifying future improvement actions.
Any library service considering self assessment must decide whether or not to pilot the scheme. Training is not optional, whether or not piloting is undertaken. Three levels of training are essential:
Planning the assessment, including the kinds of outcomes expected (e.g. scoring, or not?), and the time and human resource which will be allocated, must also be undertaken.
- awareness-raising for those staff not formally involved in the process
- assessment team training
- lead-officer training.
6. Undertake the self assessment: manage the process
Understanding what will constitute evidence (e.g. plans, reports, statistical data, user surveys) is essential before commencing the process, as well as ensuring a common approach to scoring by the self assessment team. Achieving consensus is an important part of the process. In order that the programme is effective the overall project should be carefully managed by a key person in the organisation, and should be integrated with the planning for the service. Self assessment provides the opportunity to:
- perform critical analyses of progress
- identify causes of deviations from original plans
- verify the state of capabilities in relation to new goals.
7. Identify priorities for improvement/plan actions/implement actions
All quality management programmes should have actions as their outcome. It is important to distinguish between long- and short-term objectives, and to align improvement activities to the current implementation plan. Quick returns from some activities will need to be balanced with daunting but important longer-term goals. It is common that very many areas for improvement are identified in a self assessment and consensus has therefore to be gained on the most significant.
The final stage is a review of what has been achieved, i.e.:
Evidence of improved results will be looked for in the next self assessment and the programme can only benefit the organisation fully if it is seen as an ongoing, continuous improvement initiative.
- whether the objectives have been reached
- whether the performance targets have been met
- whether the planned timescales have been achieved.
'This was not just a once-and-for-all project but an ongoing process of monitoring, auditing and managing change for the benefit of the organisation and its stakeholders'. (Kinnell, Jones and Usherwood, 1999, 159).
Brockman, J. (ed) (1997). Quality management and benchmarking in the information sector. London: Bowker Saur.
Easton, G. (1995). A Baldrige examiner's assessment of US total quality management. In: R.E. Cole (ed) The death and life of the American quality movement. Oxford: Oxford University Press.
EFQM (1997). Guide to self-assessment 1997. Brussels: European Foundation for Quality Management.
Kinnell, M., Jones, K. and Usherwood, B. (1999). Improving library and information services through self-assessment. London: Library Association Publishing.
Kinnell, M. and MacDougall, J. (1997). Marketing in the not-for-profit sector. Oxford: Butterworth Heinemann.
Linley, R. and Usherwood, B. (1998). New measures for the new library: a social audit of public libraries. Sheffield: Centre for the Public Library in the Information Society, Department of Information Studies, University of Sheffield. (British Library Research & Innovation Centre Report 89).
Milner, E., Kinnell, M. and Usherwood, B. (1997). Quality management and public library services. In: Brockman, J. (ed). Quality management and benchmarking in the information sector. London: Bowker Saur.
Pfeffer, N. and Coote, A. (1991). Is quality good for you? London: Institute of Public Policy Research.
Samuels, M. (1998). Towards best practice: an evaluation of the first two years of the Public Sector Benchmarking Project, 1996-98. London: Cabinet Office (Office of Public Service), 1.
Stewart, J. and Walsh, K. (1989). The search for quality. Luton: Local Government Training Board.