CONTINUOUS LEARNING, CONTINUOUS IMPACT

IMPACT STUDY OF THE ‘STILL SEPARATE, STILL UNEQUAL’ CAMPAIGN IN SLOVAKIA

executive SUMMARY

introduction

This document presents the key findings of the impact study of Amnesty International’s Still Separate, Still Unequal campaign, which aimed to reduce the segregation of Romani children in Slovakia and to increase their access to a quality education. Launched in November 2007, the campaign was implemented in three phases and is currently under way. This impact study covers the first two phases of the campaign, which took place between 2007 and 2010.

The purpose of this study was to further our understanding of the effectiveness and impact of the campaign, and to provide lessons relevant to the Amnesty International movement on issues including Active Participation and Economic, Social and Cultural Rights (ESCR). The study aimed to establish whether change had taken place as a result of Amnesty International’s contribution to the issue. This was accomplished by monitoring changes in the context of the campaign mainly from the perspective of those communities and individuals most affected by the segregation of Romani children in the Slovak educational system.

As the campaign is still active, the findings and recommendations presented here were taken into account when reviewing the campaign strategy in 2010. This led to changes both in the approach and the direction of the campaign going forward, and the lessons learnt have now been integrated into the plan for the current phase.

This impact study was conducted through a joint effort between the Learning and Impact Unit (LIU) and the Europe and Central Asia (ECA) Programme at the International Secretariat (IS), and Amnesty International Slovakia.

Methodology

The Impact Assessment Team was composed of the IS Regional Campaign Coordinator in the ECA Programme, the Director of Amnesty International Slovakia, and the Head of LIU at the IS. The evaluation methodology consisted of the following main steps:

• Developing the campaign’s Theory of Change and using Amnesty International’s Dimensions of Change framework to identify changes in lives, policy, accountability and activism.

• Mapping the stakeholders in the campaign, grouping stakeholders into clusters.

• Creating a sample through the snowball method (whereby contact is initially made with a small group of participants, who in turn suggest others) to build a list of participants for the study. Stakeholders external to Amnesty International were prioritized.

• Developing interview frameworks for different stakeholder groups with open and closed questions. The team interviewed 47 people across several locations in Slovakia, ensuring a balance between different groups of stakeholders, including Romani communities and families, and interviewing individuals from different representative groups.

• Analysing findings according to the Dimensions of Change. Lessons learnt were drawn from patterns emerging from the interviews.

• The report was complemented with findings from ongoing monitoring of the project by the Slovakia team, through communication with authorities, NGOs, school administrators and Romani individuals, as well as monitoring of government data and school enrolment statistics by the Ministry of Education.

1. CHALLENGES AND LIMITATIONS

The Impact Team was confronted with several challenges, some of which included:

• Introducing Amnesty International and explaining the organization’s work in a clear and concise way. The team prepared a script to explain the organization’s work using specific examples that people in communities could relate to, an approach that proved to be effective.

• Unwillingness from certain individuals, mostly civil servants and teachers in special schools, to talk to the team.

• Limitations and risks of using semi-structured interviews, including open questions, as a survey method. For instance, an interviewer’s attributes (gender, age, role) can influence a participant’s replies; some participants may choose answers they feel are desired rather than feeling free to give a full and open account.

KEY FINDINGS

The study concluded that the campaign had been successful in leading to change, including:

• Changes in the lives of Romani pupils and improvement in their access to education on an equal basis with Slovak children (in Pavlovce nad Uhom, the particular town which was a focus of Amnesty International’s campaign).

• A contribution to legislative change, and to some changes in the policy and accountability of government officials.

• Some indications of empowerment of the parents of Romani children, and of education professionals (in Pavlovce nad Uhom).

• Raising government, public and media awareness through activism and mobilization.

The study also evidenced both limitations and successes in the approach and results of the campaign, for example:

• The response of Slovak authorities and the policy and accountability measures taken, as well as the empowerment and engagement of communities affected by discrimination in access to education, were found to be limited.

• Letter writing by activists in both phases of the campaign, accompanied by media work and high-level advocacy, were found to be effective tools to pressurize the Slovak authorities to take action.

• The study showed that in order to ensure sustainability and systemic change, traditional membership action, combined with a long-term strategy that encompasses the roots of discrimination, is needed.

2. CHANGES IN POLICY

The study demonstrated that the campaign secured changes in legislation, public policy, awareness and public perception.

The campaign, which included letter writing combined with domestic advocacy, secured a major success of legislative change: a prohibition against discrimination and segregation was included in a revised School Act passed in 2008, abolishing the provision of “diagnostic stay” (the temporary placement of a child in a special school to identify his or her needs). The campaign also succeeded in changing awareness and public perception by focusing government, public and media attention on the situation of Romani children. Finally, the campaign succeeded in prompting the Slovak Government, including the Ministry of Education, to introduce procedural guidelines to be followed by Psychological Counselling Centres responsible for the placement of children in schools and classes for pupils with “mild mental disabilities”.

3. CHANGES IN ACCOUNTABILITY

Some steps towards accountability and the implementation of legal and policy changes were taken. However, these largely affected the situation of a special school in one town rather than producing systematic change in the Slovak education system. The main outcome was the assembly of a team of experts by the Deputy Prime Minister for Human Rights to monitor changes in one of the key special schools. In response to the campaign, some direct accountability was taken for the situation of Romani pupils in the school’s town.

4. CHANGES IN PEOPLE’S LIVES

The testimonies gathered showed that some changes have occurred in the lives of stakeholders as a result of the Amnesty International campaign. These included the transfer of almost two dozen children from the special to the mainstream school in Pavlovce nad Uhom, with signs of successful reintegration into mainstream education and no apparent evidence of negative impacts; a reduction in the total number of pupils attending the town’s special school, from 237 in the 2007/8 school year to 175 in 2009/10; and fewer transfers of Romani children from the mainstream to the special school. While the number of Romani children in mainstream schools throughout Slovakia increased, this could not be attributed to Amnesty International’s work.

However, the campaign has not been able to end the practice of segregation of Romani children in the education system, nor persistent discrimination against them, and Romani children still represent the majority of pupils attending special education in Slovakia.

5. CHANGES IN ACTIVISM AND MOBILIZATION

There were several indications of the empowerment of stakeholders resulting from Amnesty International’s work:

• Two Romani mothers were included onto the board of the mainstream school in Pavlovce nad Uhom, and there appeared to be more general awareness among Romani parents around issues of discrimination.

• There were signs that teachers have become more empowered to raise issues related to segregation within the remit of their work.

• The campaign was successful in mobilizing Amnesty International members, with tens of thousands of letters and postcards received by national and regional authorities in Slovakia, including the Minister for Education. In addition to the “traditional” Amnesty International mobilization, other actions included: creative ways of educating and mobilizing 100,000 children in Belgium in one day; e-mobilizing thousands of people in Austria, Spain and the USA; thousands of postcards signed by festival-goers in Croatia, and constructive lobbying meetings held in several European countries. More than 20 Sections and Structures, groups and coordinators participated in this effort and displayed great creativity in mobilizing activists and contributing to the campaign.

• The campaign engaged with local Romani groups and NGOs, and with professional associations and individual experts in the educational field. It was successful in supporting strengthened activism by local NGOs and increasing their focus on the issue of education.

• As a result of letter writing, and also of advocacy by the IS and Amnesty International Slovakia, many foreign embassies in Slovakia requested meetings with Amnesty International Slovakia and Romani civil society organizations, which led to a field visit to special schools organized for staff of foreign embassies.

Despite these positive outcomes, the study also highlighted some significant limitations:

• Awareness of the campaign and involvement in activism among key stakeholders (Romani pupils and communities) was more limited than expected. Very few of those interviewed knew about the campaign, and were therefore disconnected from the international activism undertaken on their behalf.

• Only a few community members were aware of changes to the school assessment and placement process.

• The majority of Romani parents interviewed expressed a preference for sending their children to the special school due to their belief that children were better treated in the special school, that it was less costly and that it allowed them to receive better financial support.

• Some Romani parents believed that it was natural for Romani children to attend special school. Some even felt that Romani children were less able to learn.

Lessons learnt

1. THE CHALLENGE OF SUSTAINABILITY

• Where failures of policy and practice in institutional provision are systemic, and reflect deep-rooted discrimination, intervention requires long-term investment from Amnesty International. While the Amnesty International campaign clearly led to local and, to a much lesser extent, national change, the results are not yet sufficient to sustainably end systematic discrimination of Romani children in access to education . Measures taken by the government were insufficient to address existing policies and practices in the education system, and were insufficient to overcome deep-rooted prejudice within the education system, government and wider Slovak society.

• The main lesson learnt is that traditional membership action works if combined with a long-term strategy that takes into account the wider roots of the problem, and exposes publicly the failure of the government to address entrenched discrimination. The study showed that stopping a violation in a particular case does not necessarily lead to sustainable change. A more sustainable approach, that includes Romani communities and other spheres of civil society as part of the solution, would be needed. More sustained work with local NGOs also needs to be part of the picture.

• Where government policy and practice is influenced and shaped by discrimination in wider society, addressing institutional policy and process alone may not be enough. Broader measures beyond the education system may also be needed (and may not necessarily be undertaken by Amnesty International). Addressing structural exclusion and discrimination, which require comprehensive positive action by governments, requires a range of measures beyond specific institutions.

1. ACTIVE PARTICIPATION AND ESCR ISSUES

• A key area of learning relates to active participation and participatory approaches that seek to empower primary stakeholders and to include them as agents of change. Due to its reactive character, and the lack of an Amnesty International or a community-based organization presence on the ground, the intervention in the case of the local community was not aimed at active participation of its members and, as a result, did not include an element of working with Romani communities on the ground beyond the research stage.

• The study found that awareness of the campaign among affected communities was limited, as well as their active engagement for change; some key stakeholders did not agree with the campaign aims that children’s access to mainstream schooling is a priority.

• The model of change designed for the campaign is similar to those used for civil and political rights as well as other more traditional Amnesty International concerns and human rights abuses. The element of active participation and the empowerment of communities and NGOs was included, but was not on the critical path for achieving changes in either policy or law. Other critical paths towards change might have involved working first to empower and educate communities and to give them the resources to create solutions.

• The limited Amnesty International presence on the ground restricted our impact. Availability on the ground, as well as prioritizing this as an important part of the strategy, would have enabled Amnesty International to support the local campaign and to work with the community.

• Lessons about the challenges of active participation have been applied to the third phase of the campaign which is now under way. Amnesty International is now focusing on policy change at the national level, rather than on campaigns for specific cases. This recognizes the fact that effective active participation work requires extensive engagement with the community and perhaps investment in human rights education, as well as additional resources.

• Work on ESCR issues sometimes requires specialized technical expertise to complement internal Amnesty International capacity, for example on issues of education and disability.

• Amnesty International campaigns need to include clear exit strategies which can promote sustainability. The end of Amnesty International campaigns has implications for NGO partners and other stakeholders, including primary stakeholder communities.

1. EVALUATION METHODOLOGY

• Some stakeholders refused to speak to Amnesty International or provided conflicting views on the issue. The parameters and purpose of missions and individual meetings need to be made clear, and teams need to be ready to answer any questions transparently.

• The impact study team found it difficult to explain Amnesty International’s work to Romani communities affected by discrimination. This emphasizes the need for Amnesty International to dedicate sufficient effort and, if necessary, to generate the right materials, to ensure that stakeholders can understand what the organization is and does.

• The use of open questions during the impact study proved effective in discussing change and making sure that interviewees thought of unintended consequences.

• Combining a follow-up mission using traditional Amnesty International research methodologies and impact assessment proved beneficial to the project in terms of cost and human resources. It also allowed the integration of impact assessment into ongoing work to be tested.

Other key lessons learnt include:

• STRATEGIC CHOICES AND THE HORIZON FOR CHANGE. When working on ESCR issues the horizon for change is long term, and the vision for change differs among stakeholders. In such contexts, there will be different options for Amnesty International’s engagement and overall strategy.

• PLANNING FOR SUCCESS AND MANAGING TRANSITION. The campaign in Slovakia raises the issue of what Amnesty International’s responsibilities are towards the individuals on whose behalf it calls for change on ESCR issues. Changes in service delivery have profound social implications and, where poverty prevails, they can lead to high levels of demand and bring new challenges. Transition and change should be managed to protect rights holders.

• ANALYSIS OF DUTY BEARER INSTITUTIONS. The limited response of the government suggests the value of more detailed political and institutional appraisal of government bodies. This might be undertaken prior to campaign design and could provide understanding of potential drivers of change within government bodies.

• INCREASE RESEARCH AND THE USE OF GOOD EXAMPLES. There is sufficient evidence to suggest that Amnesty International should increase the use of good examples of human rights change when attempting to influence governments.

• PLANNING FOR REACTION. The rapid mobilization of the Slovakia team to work on the case of the focus community demonstrated that timing is crucial for effective campaigning, and that swift action may lead to campaigning or political opportunities.

• WORKING AT THE LOCAL LEVEL. The focus on the community at Pavlovce nad Uhom was very effective in securing the attention of the authorities and bringing them into dialogue with Amnesty International. However, while the focus on Pavlovce nad Uhom brought benefits, the government responded to the problems specific to that town, rather than admitting that they were systemic and nationwide.

• PARTNERSHIPS. The study highlighted the importance of partnership for Amnesty International in achieving its goals. Continued engagement with NGOs and working with greater collaboration can provide a means to greater sustainability. Amnesty International needs to consider civil society capacity and presence when it makes decisions about working on issues, and where to work on local cases.

• ORGANIZATIONAL ISSUES. Tension between impact on human rights issues and priorities for section growth emerged during the implementation of the campaign. The issue of discrimination against Romani communities remains unpopular and challenging to work on in Slovakia, and is not necessarily conducive to membership growth. Amnesty International should look for alternative approaches to growth, that maybe do not relate to official membership through financial contribution, but rather focus on building activist groups with a particular interest in discrimination. A demand for the accountability of Amnesty International from civil society was also noticeable during the impact study. It emphasizes the need for effective processes of evaluation, monitoring and impact assessment, which enable the organization to report transparently on how, and with what results, it is using its resources.

Recommendations

• Amnesty International should further explore and, whenever relevant, pilot research on positive examples of bringing about human rights change to be used as models to be followed by governments, civil society organizations or individuals. These models could also be useful when influencing governments, to help convince them that solutions are possible and have been implemented by their peers. Positive examples can also be a tool for empowering rights holders as they reinforce the idea that change is possible.

• Amnesty International campaigns and projects should include clear and considered exit strategies in their design, in particular those which are designed to include active participation of communities and those subject to human rights abuses. This should be made compulsory and integrated into the systems for planning, reporting and approvals. Similarly, sustainability of campaign interventions should be made a priority when developing and implementing campaigns. In particular, the role of partners on the ground should be taken into account to ensure that the change process initiated by the Amnesty International campaign continues, even if organizational priorities shift.

• Amnesty International needs to take steps to ensure that campaigns are constantly monitored throughout implementation, and that mid-term and end-of-campaign studies are performed. Lessons learnt from these exercises need to be taken into account, and management approaches to campaign delivery should allow for flexibility and adjustment of directions in real time. Adequate resources in the operational budget need to be allocated for this.

• Scoping missions at the development phases of campaigns should expand research on situation analysis and prospective impact assessment. Follow-up missions should include stronger aspects of monitoring of the impact of Amnesty International’s interventions.

• Amnesty International needs to improve the matching of ambitions with reality, and realistic planning has to be improved.

• Amnesty International should place more emphasis on coordinating and aligning growth with human rights priorities. This is particularly the case for International Mobilization Trust funded sections such as Amnesty International Slovakia