This is a really difficult part of the work, and one that many voluntary and community organisations find so difficult that they put the evaluation work away and never complete it! Sometimes you get to feel overwhelmed by the data- working with someone else or going away from it for a few days can help.

It is important to look at every piece of evidence, preferably with people from the key group of organisation's stakeholders, jointly analysing and making sense of information collected; and in light of the learning, the group making decisions about change needed in the organisation. Often you have to do the analysis on your own but really early on check out themes and conclusions with key informants- have I got this right? You may be met with anger and defensiveness but persist as this is part of the learning process. You may have got it wrong and have to be willing to make changes.

You start this stage with looking at every piece of evidence:

e.g. a summary of questionnaire responses, summaries of notes of interviews from users with all the things that could identify individuals removed, some drawings, some monitoring data, a piece of observation, a summary of telephone interviews with funders or Board members........

and asking yourselves:

• What is this telling us?
• What are the key points?
• What are the range of issues raised?
• How does what people in this group say vary across the group?
• How does what this group is saying "fit" or conflict with what others are saying? What issues recur? What are special to this group?
• How important are the different issues?
• What kinds of patterns, or themes are emerging?
• Are there contradictory views about the organisations‘s purpose, processes or outcomes/ results?

• Search for explanations: why do things occur or why are they done the way they are?
• Look for surprises: was there a 'dog who did not bark in the night?'
• Search/ look for what needs changing
• Search/ look FOR WHAT CAN BE CHANGED

This is detective work- Raymond Chandler rather than Agatha Christie!

It involves a cycle of discovery, reflection and review.

Some suggestions about how to move from initial analysis to further analysis and developing your report.

There is not much written about this, but this is how I and others tackle the job:

• Get really familiar with the data- read it, read it aloud, listen to it if on tape.
• Identify recurrent themes or issues as you see them.
• Index ( using numbers or coloured pens) the material according to themes - OR photocopy your notes/ summaries etc and cut them up put into theme piles and stick them on sheets OR type onto a computer, cutting and pasting into themes.
• Chart out the material in each theme and pull out core elements; search for patterns and relationships in the data ( You may find some themes getting too big and subdividing, and other themes becoming so small that you realise it is your theme rather than what the evidence is telling you! ).

This is where your interpretations can be different from others: you need to be able to justify how you are coming to conclusions, constantly checking out "Have I the evidence to draw this conclusion? Does my story fit the data?" and being quite systematic and rigorous in your approach. We return again to ethical issues- being aware of our bias and prejudices, and unwilling for example to suppress data that does not fit our conclusions.

It can be useful to draw diagrams or trees to see how themes relate or connect. I nearly always find I make many changes in the order and development of themes as I go along; I collapse headings and then re order.

It can be useful to count instances ( For example the number of people from different groups who told me that a particular project had been set up to fail.) On the other hand just because only one person told us something that does not mean we discount it: it may be very important when seen against other pieces of evidence. ( eg a parent told me about a particular member of staff throwing a child across the room: taken with other evidence about the fear of this staff member, and the supervisor‘s concerns, a picture began to emerge).

Use people‘s quotes - they are much more powerful evidence than our assertions

Tie in qualitative and quantitative data

Writing the report- or identifying other ways of communicating the learning.

If the members of the stakeholder group do not write up the report (and usually they do not) the report writer needs to work through with them a number of drafts( or videos, drawings, cartoons...) consulting wider group of participants; the stakeholder group then needs to disseminate the learning widely, including feedback to all participants.

Some suggestions about how to write your report.

• Be very clear about your audience. What do they want/ need to know? What do you want to tell them?
• Either make it brief so people will read it - put information you do not want to loose into appendices - or have a good summary and use that as your main report with other materials as back up.
• Consider producing more than one report- internal and external, produced in the appropriate formats.
• Consider producing a very simple feedback summary for users and other participants.
• Present findings and recommendations clearly so people will understand it - beware of jargon!
• Use visuals- graphs, pie charts, bar charts, diagrams, stories in boxes. photos, cartoons.......
• Avoid criticism or praise of individuals where possible, and deal with sensitive or controversial issues with 'appropriate honesty and tact'.

For suggested Headings, after title and contents pages, consider:

1. Summary: including recommendations and acknowledgments- mainly bullet points ( One/ two pages )
2. Introduction- the Context:
   • The issues and national and local context in which the organisations works.
   • The organisations' purpose, values, work/ activities, funding, organisational ‘map‘ and a short history.
   • ( Ask yourselves- would someone not knowing anything about the organisations have enough of a picture from this section of the report in order to understand the rest?)
3. The Evaluation:
   • The purpose of the evaluation- WHY did we do it?
   • The focus of the evaluation- WHAT questions did it need to answer?
   • WHO was involved in the evaluation?
   • Values and Methods- HOW was it done?
4. The Findings
5. Conclusions and recommendations
6. Appendices inlcuding
   • List of people interviewed ( excluding users ) and other sources of evidence.
   • Topic guide or interview questions.
   • reports and publications consulted etc