Formal methods of evaluating services have been a recent feature of community care, and local authorities are now required to complete Best Value Reviews of all their services.
How do service users define ‘quality’ and evaluate the services they receive? What do they consider ‘best value’? Users’ Best Value is based on experiences in Wiltshire, where disabled people demonstrated the effectiveness of designing and executing the Best Value process themselves. Building on a strong tradition of local user involvement, disabled people worked within the Best Value framework to evaluate the service from their perspective.
They reviewed the direct payments service, which linked into their particular understanding of independent living and quality-of-life issues. But this step-by-step guide will be of use to all who wish to involve users fully in service evaluation.
All Best Value Reviews include a requirement to consult with service users about the service under review. This innovative review demonstrated how service users can play a more central role in designing and undertaking much of the review themselves. A project group of disabled people was established to review direct payments in one local authority using the statutory Best Value framework. The following lessons for good practice were revealed:
On the review process:
- The user-designed review demonstrated an innovative approach to collecting information including survey work, user-led interviews, diary work, cost comparisons and literature reviews.
- Everyone involved in the review felt they had gained from it in a number of ways. Service users were unanimous in saying that it had been a worthwhile learning experience for them and that they had gained new skills and confidence.
- The service users highlighted that an essential part of good practice built into the review project was the recognition of, and commitment to, meeting the range of access and support needs of the individual group members, throughout the project.
- The user-led review was supported by senior social services management and could not have succeeded without this support.
- The user-led review also had an impact on the authority's approach to user involvement outside the social services department.
On direct payments:
- Developing more appropriate independent support mechanisms and improving internal management and financial processes within the Social Services Department would improve disabled people's experience of Direct Payments use and therefore the quality of their lives.
- The knowledge and confidence of care managers is crucial to the information and messages given to service users about their choice of Direct Payments and therefore its uptake.
- Further work needs to be done by social services departments to enable disabled people to explore the potential benefits of direct payments to purchase equipment and to develop jointly funded packages of care through Direct Payments with Primary Care Trusts.
Implementing Best Value Reviews of services is a new policy requirement for local authorities which are gradually developing the ways in which their staff undertake the reviews. All reviews include a requirement to consult with service users about the service under review and often this is the only way for users to be involved. This review, in contrast, was innovative as it sought to demonstrate how service users themselves can play a far more central role in Best Value Reviews and, as this project demonstrates, design and undertake much of the review.
The research project was conceived by the Wiltshire & Swindon Users' Network and developed with the Wiltshire Social Services and University of Bath Research & Development Partnership using Wiltshire County Council as the fieldwork site. The project directors from Wiltshire & Swindon Users' Network and the University Partnership acted as managers of the review for Wiltshire County Council providing links between the user-controlled project group and the County Council's Best Value process. A disabled researcher was employed as project co-ordinator to facilitate the involvement of the project group in the review and undertake some of the review tasks.
The Wiltshire & Swindon Users' Network was set up by long-term service users in 1991 as an independent user-controlled organisation to support users to empower themselves and influence change. It has become an example of good practice nationally and places emphasis on demonstrating innovative examples of user controlled involvement in social care.
Wiltshire Social Services Department, led by its senior management, has a long tradition of supporting and resourcing users' involvement from members of the independent Users' Network. Therefore users in Wiltshire were in a good position to explore a lead role for service users in the Best Value Review process. A review of direct payments provided a promising opportunity to demonstrate a user-controlled Best Value Review as it is an area in which service users have considerable knowledge and expertise.
The approach to reviewing direct payments
The project, carried out as a Wiltshire County Council Best Value Review, was required to undertake activities within the focus on the Best Value '4Cs' framework of challenge; compare; compete and consult. Wiltshire County Council added a fifth 'C' - collaborate. This built on previous work recognising the importance of the authority working collaboratively with the Health Service, other agencies and citizens who use their services. In order to compare direct payments with other kinds of services, the group adopted the definition of 'independent living' defined by disabled people and based upon the principles of the social model of disability and of disabled people having choice and control over their lives.
Direct payments users and other interested disabled people were invited to participate in a project group to design and oversee the service review. The project group consisted of disabled people who were users and potential users of direct payments facilitated by a co-ordinator with expertise in this area as a disabled researcher. They responded to a democratic invitation to users to participate and met regularly to design and implement their research activities. In addition, the project co-ordinator and directors acting as the Best Value Review managers collated data collected, carried out a literature review and wrote the review report.
The group met frequently throughout the twelve-month process and regularly reported their progress to the Best Value Project Board and other relevant personnel in the Best Value Framework. The report was received and accepted by the Best Value Senior Managers Panel and elected members of the Scrutiny Committee. The social services department developed an action plan in consultation with the project group from the report recommendations. The social services department has agreed to fund the project group to monitor the action plan implementation.
The project group employed a range of methods in order to collect the information they required for the review they had designed:
- A survey of users of the Support Service for Independent Living (SSIL) to evaluate how well the SSIL was meeting service users' needs in respect of providing support to people purchasing their own care;
- A survey of social services staff, to explore the internal process of setting up and managing direct payments and to highlight areas of concern;
- A small number of one-to-one interviews carried out by members of the project group with team managers in social services, to provide the opportunity for other stakeholders to contribute their perspective;
- A cost comparison with direct services completed by officers was commented on by the project group;
- Some members of the project group kept a diary for six months, to record their experiences of using either direct payments or directly provided services and to reflect on the process;
- A survey, including costings, of five independent organisations in other local authorities providing support to direct payments users to make comparisons with the local Support Service for Independent Living and to look at competitiveness with other support mechanisms;
- A literature review to provide evidence relating to independent living, user empowerment and policy issues nationally; and to provide comparisons with other direct payment schemes and the barriers for disabled people.
Lessons about user involvement in Best Value
An infrastructure of an established user-controlled organisation, which had given disabled people expertise and experience in participation, provided a group of volunteers interested in forming the project group. They consisted of disabled people and older people and did not think it necessary that the Best Value Project Board see the group as representative of all service users but the project co-ordinator sought out potential users with dual sensory impairments and learning difficulties to join the group to ensure more marginalised voices were heard. A representative from the Wiltshire Race Equality Council joined the Project Board in the absence of any users from minority ethnic communities being available.
The project also had the support of the social services senior management for the innovative approach to user involvement. This made it possible to use the Best Value Framework flexibly in, for example, the role played by the Best Value Project Board. The Best Value external challenger was a disabled person working as director of the Centre for Independent Living in a neighbouring authority. As the project developed it gained recognition across other sections of the County Council. The project was invited to contribute to a corporate Best Value Review training programme and the County Council asked to put the project's Best Value Review Report Executive Summary with a commentary on its website.
Everyone involved in this review felt they had gained from it in a number of ways. The local authority felt their Direct Payments service had been reviewed thoroughly. When asked for their evaluation of the process, the service users who undertook much of the work were unanimous in saying that it had been a worthwhile learning experience for them and that they had gained new skills and confidence. They also commented that they felt they had been in 'privileged' positions, because they were able to see internal documents and examine practices which service users would rarely have access to. Whilst this meant that there were increased responsibilities and a greater need to respect confidentiality, the service users commented that this experience contributed to a growing confidence in their own abilities and self-esteem. A number of the service users said they now had a better understanding of how the care management process worked which would be of help to them when dealing with their own assessments. Some service users said that they had found the whole review experience of great interest and would like to be involved in further research projects in the future.
The service users highlighted that an essential part of good practice built into the review project was the recognition of, and commitment to, meeting the range of access and support needs of the individual group members, throughout the time of the project. This included making arrangements for transport, pre-meeting briefings for people with learning difficulties and communication needs, the provision of BSL and other interpreter services alongside the speedy reimbursement of expenses and fee payments. The costs of this and independent facilitation would need to be built into any similar work.
Lessons relating to direct payments
"The quality and flexibility I now have is because I have individually designed a package to suit not just myself but also my family and lifestyle and it is worth every second of any stress." (Diary abstract from a project group member)
The findings of this review supported the limited evidence available nationally, which suggests that the attitude of care managers is a key element in determining whether people take up direct payments and how well care managers work in partnership with disabled people. The review also acknowledges the complexities involved which necessitate both practical operational changes and a shift in approach to concepts of risk and control, and the challenge it presents to the culture of direct service provision.
The project group found that the internal social service systems for payment and monitoring of direct payments also affected disabled people's experience as users. Sometimes, for example, the late receipt of direct payments funding to pay personal assistants' wages could jeopardise their receipt of care. In addition, the independently provided Support Service needed to provide more responsive information and peer support.
No provision of equipment through direct payments was available in Wiltshire and research showed only two local authorities were currently doing this despite the requests from disabled people. There was also little evidence of professionals collaborating to develop jointly funded direct payments packages with Health Authorities. The Best Value Review Report also identified a number of barriers to independent living, affecting the project group members' lives though not directly controlled by the local authority.
About the project
This project took place within Wiltshire with Louise Brown of the University of Bath and Clare Evans of Wiltshire & Swindon Users' Network acting as project directors, and Best Value Review managers. Angie Carmichael was employed as researcher and project co-ordinator of the Project Group of disabled people carrying out the evaluation. The Project Group met regularly over one year before the Best Value Review Report on Direct Payments in Wiltshire was presented to Wiltshire County Council.