61st IFLA General Conference - Conference Proceedings - August 20-25, 1995
The role of assessment of services in planning future developments in
Jennifer Tanfield, Librarian of the House of Commons, Westminster, United Kingdom
Given the pressure on all parliamentary libraries to make the best use of theresources available to them, it is increasingly important for these libraries to be confident
that they understand their users' needs and priorities. They may therefore need to consider methods of assessing and evaluating the services they provide. This paper lists
and discusses some of the feedback techniques used by parliamentary libraries, with a view to building up a reference resource of reports, questionnaires, feedback forms, etcetera, which could be use
d by any parliamentary library to initiate or modify assessment procedures.
Probably every legislative library considers information obtained by observation of its users when making decisions about the development of services. Observation can provide simple data such as the
level of use of reader spaces.
2. Committees of Members
At Westminster five domestic select committees have been set up to "representMembers' interests as consumers in determining the provision of services to the House." Library services are conside
red by the Information Committee, which is consulted about new services, changes to existing services, or any proposal to stop a service. Some Members of Parliament have always channelled their sugg
estions or complaints about the
Library through the chairman of the Information Committee or its predecessor, the Library Sub Committee.
3. Satisfaction surveys
A number of legislatures conduct annual or regular satisfaction surveys of the whole range of services to Members; this is the case, for example, in New Zealand and
Queensland. In other parliaments, such as the Federal Parliament in Australia, the Library conducts its own survey. The surveys are usually undertaken using questionnaires and may be conducted by a
n independent agency.
A high level of general satisfaction with the services of parliamentary libraries is usually revealed by these regular surveys. The 1993 94 Report of the New Zealand Parliamentary Service says that
"Library based research services received quality ratings of very good or better from 95 percent of the respondents and overall library services were rated very good or better by just over 80 percent
of the respondents. In terms of amount of services a rating of adequate or better was achieved from 100 percent of respondents." In Queensland's 1994 survey more than 84 percent of respondents gav
e an average score of "high" or "very high" to the services offered by the Library. 98 percent of the respondents to the 1993 confidential survey of the services of the Parliamentary Library in Canb
erra rated overall services as either "very good" or "good". This high level of overall satisfaction might cast doubt on the value of such surveys but they do allow comparison to be made over time a
nd, provided that the parliamentary library aims to achieve a level of satisfaction approaching 100 percent, can be a useful
indicator of change.
These surveys also present the opportunity to question users in more depth about particular services or proposed new ones. Queensland's 1994 questionnaire sought users' views on whether a service pr
oviding typed transcripts of recorded material should be instituted, as well as asking for assessments of the quality of certain aspects of the
Research and Reference Section and satisfaction with the Current Awareness Service.
4. Other questionnaires
Audit or value for money studies often involve surveys of user satisfaction and the needs which users perceive; this was the case in the Canadian Library of Parliament in 1991 and in the House of Com
mons Library in Westminster in 1994. Parliamentary libraries themselves also commission such studies, one of the best known of which was conducted by Ed Parr in 1990 91 for the Department of the Par
liamentary Library in Australia. Other detailed studies include the 1990 survey in Chile, and surveys of theResearch Branch of the Parliamentary Library in Ottawa in 1988 and 1991
Questionnaires may be used to test Members' responses to a new service as was done in Westminster in connection with a proposal to store press material electronically. The findings helped to reach t
he decision to include the project in the Library's development plan, but almost as useful was evidence (as in the Australian Ed Parr study)
that a number of Members' staff were not aware of existing Library services.
5. Interviews with individual Members and their staff
Interviews allow a more targeted, in depth approach than is possible by use of questionnaires alone and are often used in conjunction with surveys based on questionnaires. The Congressional Research
Service (CRS) reports that it obtains good feedback from interviews. They may be the best way to learn, for example, why a parliamentarian does not use some or all of the parliamentary library's se
rvices. Recently the Head of the Canberra Parliamentary Research Service "instituted a program of visits to Senators and Members...to seek feedback and suggestions on future directions." It will be
interesting to learn about the value of this initiative.
There can be some difficulties in obtaining representative feedback using the interview method. Legislators are very busy people who are reluctant to devote time to interviews. They also tend to ha
ve individual ways of working, which makes essential the selection of a sample reflecting a wide spectrum of approaches to the job.
6. Focus groups
A focus group is a group of users with common characteristics who are interviewed as a group. CRS uses focus groups with success, obtaining honest and useful assessment. However, when the House of
commons Library considered the use of focus groups it was warned, both by interviewers with experience of Westminster parliamentarians and by the Members of the Information Committee, that MPs could
not be relied upon to keep
such appointments. The benefits to be derived from carefully selected focus groups could be mitigated by failure to attend on any significant scale.
7. Feedback forms attached to written reports
Returnable feedback forms with boxes to tick and space for comments have been attached to House of Commons Library Research Papers since January 1994. Only a small number of forms are returned (on a
verage less than one per paper). Rather surprisingly, more of the forms have been returned by Members than by their staff. To date 63 percent of forms returned have given the paper maximum box mark
comments are constructive and may include criticism. This response is in contrast to the experience of CRS which dropped the use of returnable forms more than ten years ago because those who tended
to send back questionnaires generally had very high opinions of CRS work. The Research Branch of the Canadian Library of Parliament includes evaluation forms with each response, apparently with a si
milar level of satisfaction to that experienced by CRS.
8. Telephone follow up of responses
The concept of selecting a sample of responses and telephoning parliamentarians to seek feedback shortly after they have received the information is an attractive one. It was used by the value for m
oney team who investigated the research services of the House of Commons Library. They reported on 31 responses but did not say on how many occasions they had failed to contact the Member. Poor res
ponse rate is likely to be a problem with this method of obtaining feedback bearing in mind the pressures on
parliamentarians and the difficulty of contacting them by telephone.
9. Monitoring of complaints or suggestions
Given the high level of overall satisfaction with the services of parliamentary libraries, the number of complaints is likely to be small, but can provide useful
management information. Westminster library staff are now required to record all complaints, comments, or suggestions, whether these are made orally or in writing. Results are collated and consider
ed twice a year.
10. Analyzing information about use of services
Information about the extent to which Members use particular parts of the parliamentary library service is important in allocating resources to maximum effect. With the coming of computers this sort
of analysis has become much easier to conduct. Requests put to the research service in Westminster are now logged into an computerised ENQuiries database and considerably more detail is readily ava
ilable about, for example, which Members use which part of the service and the deadlines imposed.
Most computer databases routinely produce information about usage data on the number of times particular passwords have been used, time of use, and the part of the system interrogated, as well as pe
rformance data like speed of response. Modern telephone systems can also provide data on the time taken to answer a call and the number of occasions on which a caller hangs up. The problem is to de
fine the user reports so that they show important trends. It is easy for managers to be overwhelmed by reports that are too numerous and too detailed. Future methods of evaluation
It is clear that parliamentary libraries are rapidly adopting electronic methods of accessing and delivering information and in this area the emerging legislatures are often ahead because they have g
one directly to information technology in the absence of alternative systems. To avoid the danger the managers will be overwhelmed with usage information generated by library systems it is important
that specifications provide for reports that contribute to performance measurement and assessment. Westminster's ENQuiries database allows for the calculation of elapsed time between an enquiry bei
ng placed and its completion so that performance indicators can be monitored.
A "help desk" is a common feature of computer systems made available to library users. Requests placed by legislators and their staff, both to help desks and to those providing user training, often
do not seek technical help with hardware and software, but related to problems with documentation and indexing. These are areas where the parliamentary library could make improvements by amending pr
ocedures or developing more useful menus and explanations.
It is technically possible to put feedback forms or questionnaires addressed to users on computer networks, perhaps linked to particular databases, but users might be irritated by this method of seek
ing feedback. Speed is one of the great advantages of using information technology to obtain information and users may be resistant to anything that makes it slower for them to access, or exit from
The House of Commons Library is in the process of moving from reliance on observation and assumptions about Members' needs to more systematic collection offeedback. Our 1994 questionnaire to Members
and their staff on use of press articles has given us confidence to seek money and staffing to introduce a networked electronic storage system in 1997/ 98. In the early summer of 1995 a firm of con
sultants will be asking a sample of our Members and their staff about how well the Library's services meet their needs. We expect to follow up this survey in future years and will use the informatio
n gained to prioritise the distribution of resources.