Evaluating a community-based prevention programme
In recent years, research has identified a number of 'risk factors' for children that may lead anti-social or problem behaviour in later life.
This has lead to an increased interest, by policy makers and practitioners, in developing targeted preventive work with children and families that may reduce risks and increase protection. In 1998 the Joseph Rowntree Foundation developed a new community-based programme that aimed to reduce risks for children and families. Communities That Care (CTC), a programme originally developed in the United States, aims to create neighbourhoods where children and young people would be less at risk of school failure, school-age pregnancy, youth offending, and drug misuse.
This study evaluates the impact of the first stages of this process. It details how three CTC projects were set up in the UK, and what lessons have been learnt from their early experience. It looks particularly at the set-up period when areas analysed the problems they faced and set about devising action plans aimed at addressing these problems.
Communities That Care (CTC) is a community-based prevention programme, designed to ensure that children and young people grow up in safer and more caring communities. This interim evaluation report traces the early development of the three, original demonstration projects that are now part of a larger programme of 23 projects operating in sites across England, Scotland and Wales. The CTC process provides a blueprint by which communities can identify the particular local risks that face their young people and work with others to implement evidence-based projects targeting those risks. This report, by Alan France and Iain Crow at the University of Sheffield, covers the first stages of this process and is part of a longer-term evaluation, due for completion in 2003. The evaluation found that:
- The CTC process created opportunities for local people and professionals to identify risk factors within their community.
- Local CTC projects managed to engage a core group of local people and professionals in the implementation of the programme.
- Project coordinators were critical to any success. Bringing coordinators in before the process became established also improved the success of involving local people and professionals.
- By bringing together professionals and local people to assess 'community problems' the CTC process created opportunities for local people to influence what was being defined as the 'problem' and how it should be tackled.
- Creating partnerships between professionals and the community created opportunities for local people to have a better understanding of how and why decisions were made in their community. Local professionals were also required to account for previous and future actions.
In the mid-1990s, the Joseph Rowntree Foundation (JRF) became interested in identifying whether early intervention programmes with children and families could reduce future offending. One such approach was an American-based programme called Communities That Care (CTC). This early intervention and risk reduction programme showed promising signs that it might be successful in the long-term reduction of problem behaviour, including youth crime. The JRF decided to support a British-based CTC programme that would be fully evaluated. The programme included four components:
- Anglicising the USA working tools;
- The setting up of an independent company responsible for the running of the CTC programme;
- Funding of three demonstration projects in deprived areas of the UK;
- The commissioning of a longitudinal evaluation of the long-term impact of the CTC programme.
The Communities That Care approach
A theory of prevention
Two main theoretical strands underpin the CTC approach. First, it is claimed that certain risk factors associated with particular types of problem behaviour can be identified. In the USA a wide range of risk factors have been identified through research within at least two international longitudinal studies. These factors include: lack of discipline in families; academic under-achievement in primary school; lack of neighbourhood attachment; and friends involved in problem behaviour.
The second theoretical strand suggests that reducing risks requires intervention and leadership by adults. To achieve positive behaviour, children need to be given clear standards about acceptable behaviour and to have social bonding with those adults who can give clear standards. To aid this process children and young people should:
- be given the opportunities to be involved and valued in their lives;
- have opportunities to gain social and learning skills;
- be given recognition and praise.
The importance of evidenced-based approaches
Underpinning both the theory and practice of CTC is a belief that prevention should be guided by scientific evidence:
- First, longitudinal social research has shown that risk and protective factors exist as predictors of social problem behaviour.
- Second, risk can be identified and measured using quantitative data collected through self-report surveys, national data and administrative information.
- Third, once risk levels have been identified, programmes that evaluation and research have shown to reduce risks are implemented.
- Finally, the overall programme is measured for its success by comparing levels of risk and protective factors before and after the interventions have been made.
Involving the community
CTC sees the involvement of the local community in the process as essential. In the CTC model, the 'community' is not just focused on local people who live in the area where the programme is to be run, it also includes professional workers who have either managerial responsibility for services in the area or who are working practitioners providing front-line services. Community 'mobilisation' is seen as increasing the impact of interventions by reducing social disorganisation, promoting strong community norms against anti-social behaviour, and creating community ownership and investment in prevention activities.
CTC is a structured and systematic process containing a number of critical components that help participants systematically to assess the levels of risk and protection within their community. It also helps communities design approaches that will improve the overall management of local resources and target particular risk factors with specialised, evidence-based programmes. The CTC process can be defined in three phases:
- Phase One: Community Readiness.
- Phase Two: Community Mobilisation and Action Planning
- Phase Three: Implementation
This report concentrates on Phases One and Two.
After two years of hard work by the demonstration projects the CTC programme had produced some valuable findings. These aid understanding about how communities and their service providers can be brought together in partnership to create evidence-based local preventative approaches to tackling social problems.
Delivering community programmes with community support
It is clear that the CTC process can engage and maintain levels of active involvement. Over the two years of the demonstration projects, a core group of local people and professionals stayed with the programme and became active partners. They were recruited to the programme by a structured training process where members were introduced to the CTC approach and asked to join a 'Community Planning Team'. Membership of this group aimed to be wide, encouraging both local professionals working in the area and local people to be involved from the onset. Once engaged this group became the core decision-making body responsible for the day to day running of the project. While the numbers involved over the life of the projects were low, the durability of core members indicates that the CTC model does create opportunities to actively involve local communities and their service providers in partnership working. The CTC process was reasonably effective in maintaining engagement of those already involved but not so successful in recruiting less engaged people or new professionals. Getting people in early helped create a core group but new entrants found it difficult to become members of a programme that was established and had generated a strong commitment and knowledge of the process amongst others. Increasing membership therefore became difficult.
The CTC process introduced professionals to a new way of working which requires them to actively engage and forge partnerships, not only with other professionals but also with community members. The CTC approach also gave local people the opportunity to become more informed about their community (and the services that serve it) and to be more active partners in making recommendations for the future use of resources.
Part of the effectiveness of the CTC programme is having a structured process in place. This guides and informs but is also task-orientated, requiring active involvement by members of the Community Planning Teams. For example, having to work together on analysing data or designing questionnaires for interviewing other professionals and service providers brought professionals and local people into a different set of relationships. This meant that all parties had to agree what their working relationship would be. Furthermore, individuals were able to gain a fuller understanding of the needs and pressures that came with either working in a professional environment or living in a disadvantaged community.
But the evaluation made it clear that achieving these outcomes requires 'mobilisation' to be central from the beginning, with attention being paid to how people can be brought into the programme at each stage of the process. The role of the coordinator was fundamental to this process. Coordinators have core responsibility for overseeing the implementation of the CTC approach within the local areas. Their role is to bring together partners and ensure that participants draw upon the materials provided by CTC in making their decisions. Having someone in place who could help oversee mobilisation at all stages of the project was critical. For example, coordinators could use resources to support active involvement of local people by providing creche support, free taxis and flexibility of meetings. Without effective coordinators, participants are likely to drift and become disengaged from the programme.
Using evidence-based approaches with communities
Involving communities and professionals in 'evidence-based approaches' to tackling social problems has had a number of added benefits. The CTC approach created dialogue and debate between professionals and local people over what was really happening in the area. In the past, professionals would base their decisions upon 'professional knowledge' of the problem. Some of this would be informed by evidence but it could be fragmented and greatly influenced by their own values and professional judgements. In the CTC approach, professionals became better informed because they were introduced to a broader and fuller knowledge of the community. Evidence became the focal point for decision-making, with data being provided from risk audits that were constructed from self-report surveys and local data sources. This information was then used to underpin recommendations made in locally based Action Plans.
This also created the opportunity for professionals to share responsibility and accountability as they were not making decisions alone or without consultation. Accountability is built in to the CTC model because professionals take collective responsibility for the final outcomes with both other professionals and local people. Having local people involved also increased the levels of accountability for professionals: they had continually to justify how and why certain decisions were made.
Previously, local people would not have been party to the process of decision-making about resources or services. The CTC approach clearly created opportunities for local people to be more central to this process. For example, in the development of a locally based Action Plan local people were critical players, being actively involved in the interpreting of data and construction of recommendations for service providers. Not only did those who did get involved have an influence on the outcomes but they themselves also became more knowledgeable about how decisions in their communities were made and how professionals worked with each other.
How far the final proposals from the demonstration areas were 'evidence based' remains unclear. This arose because of the problems associated with data collection and the production of risk audits. CTC UK had difficulty getting the audits produced on time and in a format that made assessment easy. Other difficulties existed over the assessment of local services in that local people had problems in being able to make judgements about services and their effectiveness.
The researchers conclude that the CTC programme provided invaluable opportunities for assessing how communities can be engaged in evidence-based approaches to preventative solutions to locally based social problems. While its success has varied on a number of fronts, such as mobilisation and evidence-based approaches, it is clear that opportunities exist within this model to create new ways of working which challenge traditional professional practices and encourage greater community involvement.
About the study
In March 1998 Alan France and Iain Crow from the Centre for Criminological and Legal Research Centre (CCLR) at the University of Sheffield were commissioned by JRF to undertake a five-year evaluation of the CTC programme. The evaluation is looking at both the process and the outcomes. This dual approach is intended to measure not only the success of the programme but also to identify the mechanisms that have influenced the results. The process evaluation is complex and requires the use of various diverse investigative methods. These include: interviewing key stakeholders; observation; and analysing management data.