By Nicola Popovic;
Training sessions, workshops and courses form part of the professional environment in the development cooperation world. Not only cognitive knowledge but also attitudinal and behavioral changes form part of the learning agenda when working on human rights issues, sustainable peace and gender equality. Gender trainings in particular address soft skills and socio-emotional changes. These factors and shifts are difficult to measure. How can we evaluate whether training sessions really help to break glass ceilings, close pay gaps, prevent trafficking and violence through institutional reforms, and aid in statistical data collection?
Gender training is often defined as a long term, transformational process that strives to foster greater gender equality within societies through the transmission of knowledge, the enhancement of skills, and attitude as well as behavioral change.[1] Such definition implies various levels of assumptions, namely that a) the materials, presentations and exercises presented in the training contribute to a change in knowledge, attitude and skills; and b) that these new or changed knowledge, attitude, behavior and skills will translate into actions that contribute to greater gender equality. The justification to develop and conduct training for gender equality is based on this theory behind the change that it claims to bring.
The need to prove that gender training really does link to both these smaller and larger objectives and goals is especially relevant considering the large funding cuts for gender equality and sustainable development projects. The most common evaluation tools in trainings are satisfaction surveys or tests handed out to participants after a training session. An analysis of these tools shows us immediate reactions and desired knowledge curves, but do not tell us anything about the ways the training´s content will translate in actual behavioral, institutional or social change nor if it actually has an impact.
Security sector actors receive gender training in order to reform the sector towards more inclusive, participatory and sensitive provision of security for all: men, women, boys and girls. Mostly male dominated and driven by an agenda that fosters traditional gender roles, the security sector can be more resistant to training for gender equality.
Nonetheless, once open to change, security actors can be receptive and engaged participants in gender training sessions. If received as an order, the mandate to provide security for all can be taken very seriously if received as an order, particularly in the military sector. It can also be seen as an operational necessity, such as in the area of evidence finding for the police or crime prevention. The effects of gender training and gender mainstreaming measures may be even more visible in this sector than elsewhere. The importance of gathering evidence and documenting changes in attitudes and behavior is important to prove the effectiveness and impact of gender training.
A very specific and most commonly used evaluation method when it comes to training is the Kirkpatrick’s four level model.[2] After the training has been conducted, this method measures different levels of comprehension to estimate how the learned knowledge and skills will be applied in contexts relevant to participants. It analyses the immediate reaction, learning, behavior and impact of the participants. While the first two levels are relatively easy to evaluate and gain data on using questionnaires and observation, levels 3 and 4 may be more difficult to assess and may require more resources. Levels 3 and 4 may be the most relevant to track progress participants have made and how effective the training has been for them, and for meeting the overall learning objectives.
It is important to recognize that training initiatives are only effective if the trainee has a chance to implement the learned content and skills in the environment in which he or she works in an effective manner.
The higher levels at the Kirkpatrick scale are more difficult to evaluate and require longer term observation, analytical detail and resources. However, they are also a lot more significant when it comes to the relevance and impact of gender training. Behavioral change does not solely depend on theoretical knowledge or even emotional awareness about certain issues, but also on the practicality and individual disposition of the participants to express and perform tasks and actions in a way coherent with the theoretical knowledge and awareness learned during a training for gender quality. Very often that means breaking out of routines, stereotypes, and comfort zones. It may question values and modus operandi of one’s own culture, institution or family. A behavioral change and other long term changes may be complex goals to achieve and can be even more complex to prove and measure.
For example, a UN peacekeeper may have a positive reaction to the gender training he has received prior to his mission in country X. He may have scored high on the final test and feel that he has learned quite a lot. Nonetheless, he may fall back into the same behavior he meant to avoid once in mission due to the environment and conflict situation he is in, which may also be fostered by comrades. Additionally, the infrastructure in mission has not changed in a way that any gender issues can be integrated into operations. Therefore, results may not prove very different from the previous mission he participated in. In this case, gender training has made a change on the first two levels on the Kirkpatrick Scale but failed to have long term impact and results. This is also due to the fact that the infrastructure surrounding the trainee does not permit the application of the learned content and skills.
The tools and resources institutionally available often do not allow follow-up with individual training participants, nor observation and analysis of their behaviors. Instead only anecdotal data, tests, or interviews may indicate long term changes or even impacts. “Most tools assume that change occurs in a linear way, but social change is a complex process that cannot be reduced to neat sets of causal relationships. What often ends up happening when organizations use this kind of linear framework is that their focus is on the tool itself – how to meet the expected achievements – rather than on the larger picture.”[3]
However, the larger picture often has culturally specific gender codes, roles and norms that dynamically relate to each other. “Gender roles, identities and expectations are socially, culturally and politically constructed through the power relations between men, women and those identifying with other gender categories – as well as through the power relations within these groups.(…) In order to bring meaningful gender dimensions into these methods and processes, DME should integrate local understandings of gender identities of different social classes, religious beliefs, economic resources, education and ages (…).“[4] Therefore, such understandings of gender roles and norms need to be unpacked and contextualized before being able to assess the level of behavioral, institutional or societal change that may be realistic and worth achieving in each context. In addition, these roles and norms need to be considered in relation to other social and political factors such as ethnicity, age, economic status and religious beliefs which may influence power dynamics and discriminatory practices within a given society, organizational context or institution. In the above example of the example of the UN peacekeeper, the context is a) the specific security sector he works in –a military unit- and b) the mission country.
In addition, the background, experiences and perceptions of the various participants taking the same gender training and course can be very diverse. Particularly in international settings, a heterogenic audience with different background knowledge, attitudes and skills can pose challenges in the design, implementation and evaluation of a training’s success towards the creation of a more gender sensitive audience and subsequently greater gender equality overall.
Apart from evaluation approaches that focus on quantitative and conventional data collection methods and analysis, there are also alternative methods that can provide a deeper understanding of issues surrounding gender perceptions and social change. This is particularly important when specific attitudes and behavioral change shall be challenged through the training. Most significant change focuses on a central question: “Looking back, what do you think was the most significant change in (particular domain of change)”? Significant change stories are collected and selected to provide qualitative input about the interventions contribution to change through individual narratives. The stories collected are then chosen by a selection committee to complement quantitative data. This method allows the contextualization of other data and achievement of a deeper understanding of the different levels the individual can benefit from in the given training. In this way, a logical link between a gender training and an observed change, even if subjective and anecdotal, is established.
Universities and training institutes try to follow up with their former students through alumni networks on a much larger and more systematic scale. Nonetheless, these networks are no guarantee for any systematic or reliable data collection that might lead to any significant statements or conclusions about the impact of gender training.
For peacekeeping in particular, the changes that gender training contributes can be particularly complex and constantly changing. An evaluation of the impact of gender training on each UN peacekeeper in their different military units is almost impossible to fully measure. In addition, an investment in the evaluation of an initiative or activity must always be proportionate to the investment in the activity itself. For example, the impact of a one-hour gender training should not be evaluated and analyzed for months since there is likely to be only very limited change in the participants and environment.
The effects of the gender training compete with so many other social, economic, cultural and spiritual influences. A lot of times changes show themselves in an anecdotal, one-off and unpredictable manner. Social change is not a linear process and is not only guided by rational judgement but also strongly by socio-emotional interaction, especially when it comes to social power dynamics and gender relations. Correlations between changes in behaviors, social dynamics or even practices or policies can be correlated but not necessarily caused by a given training. The link needs to be established when drawing conclusions about the impact and effects of gender training.
There is a dilemma between the need to balance how effective the different training units are on the one hand, with the complexity of and resources needed for a scientifically valid analysis. Overall, there are methods and ideas for how to measure the effects and changes gender trainings bring, but they are often too narrow to really demonstrate significant conclusions and correlations. While the development world would benefit from more in-depth analysis of what really works for training participants and the context they live and work in to make gender training more effective, the resources, time and expertise needed to do so are still not invested and may not even be worth the extensive effort for each short term training given.
A solution can be to make gender training less short term and randomly applied but more sustainably and dynamically integrated into the curricula at police academies and military schools, for example. At the same time an evaluation of the training should have a longer term perspective to match the proportionality of efforts between gender training and evaluations. The role of security sector is essential for establishing gender equality, sustainable peace and human rights protection. Because of this, training for gender equality in the security sector should be mainstreamed in other training units and implemented in a manner so that its effects are more realistic and visible. Institutional changes towards a culture of equality, respect and sensitivity within the security sector go hand in hand with an effective training unit as well as its evaluation.
[1] UN Women: http://www.unwomen.org/en/how-we-work/capacity-development-and-training
[2] Kirkpatrick Partners: http://www.kirkpatrickpartners.com/OurPhilosophy/tabid/66/
[3] Kinoti, Kathambi and Sanushka Mudaliar. The Pitfalls of Monitoring and Evaluation: Do Current Frameworks Really Serve Us? A Summary of Part I ‘Capturing Change in Women’s Realities: The Challenges of Monitoring and Evaluating Our Work’ a Paper by Srilatha Batliwala and Alexandra Pittman. Association For Women’s Rights in Development. Ottawa: 2010
[4] Khattab, Lana. Myttinen, Henri. Popovic, Nicola. Measuring Gender in Peacebuilding. International Alert. Londond: 2016 http://www.international-alert.org/resources/publications/measuring-gender-in-peacebuilding#sthash.52tbvEzm.dpbs