Featured

Evaluation #hacks

20191119_152448 cropWhat a super day!  2 years in the making (don’t ask) and then yesterday I met up with 15 of the region’s keenest and we had a splendid day mucking about with outcomes frameworks, logic chains, question banks and love them/loathe them survey tools.

The fulfilment of a private ambition really – West Midlands Museum Development team had invited me to devise and deliver an Evaluation Training Day at the Hereford Museum of Cider.

My ambition for the day was to provide museum professionals with an overview of how to plan for and structure an evaluation programme that is embedded in project activity from start to finish. All were encouraged to base the ‘hypothetical’ planning work on live projects in their own organisations.

I took the group through the planning process from formative enquiry, carrying out monitoring and transitional assessment research through to summative evaluation and reporting.

Yes, it was as nerdy as it sounds but we had enormous fun and there was a great buzz in the room as people worked together to share their knowledge and experience and banish their fears of EVALUATION.  It’s a beast, but the course feedback suggests I’ve helped people to tame it!

We also

  • created our own evaluation framework and question banks
  • considered the levels of resource (money, time, staffing, training) required to carry out evaluation work
  • discussed the pros and cons of various approaches to monitoring and survey
  • finished with a good natured debate over whether to tackle the work in-house or commission an external consultant for the job.

As I said, it was fun with information attached.  Actually, we were having so much noisy fun that a member of the Cider Museum team asked what kind of party we were having and could he join in!

Keep reading below the picture for links to evaluation guidance and resources…

20191119_152447 crop

Here are some of links to information and tools we talked about in the session which might well be of use to you, my darling readers.

What types of evaluation are there? Here’s a handy guide from the V&A Learning Team to some of the terms used by consultants!

National Lottery Heritage Fund:

Arts Council England:

Measuring Learning

Association for Independent Museums (AIM): measuring economic, social and environmental impact

Measuring Wellbeing

Head Heart Bag Bin (pdf) – a visual survey tool for evaluating the learning and wellbeing impact of your activity

Collecting Birmingham: evaluation of a Prize Winner!

Much as I love all my clients, it’s a tremendous buzz when their work wins prizes!

Birmingham Museums Trust commissioned me to evaluate their terrific Collection Birmingham project during 2017-2018, and then used elements of my report to help them win the Museums Association’s Museums Change Lives Award 2018.

The project was funded through the HLF Collecting Cultures programme and focussed on bringing people from some of Birmingham’s diverse communities together with artists and curators to shape the future of Birmingham Museum Trust’s collecting policies.

Over 1,800 objects were acquired through consultations with local people including The Rivers of Birminam, a series of 100 black and white photographs by Vanley Burke chronicling 40 years of Caribbean heritage in Birmingham. The photographs are by far the largest and most significant collection of work by the celebrated photographer in any public collection and were displayed at Birmingham Museum & Art Gallery earlier this year.

I feel like the photos belong to the people of Birmingham as so many people feel connected to the stories and experiences documented in them. I’m very pleased they will be part of the city’s collection.

Vanley Burke

As an evaluator, it was exciting to see how eager the Collections Team of curators, conservators and cataloguers were to get involved with community engagement activity more usually carried out by the Learning & Engagement team members.

As the project team worked across the whole Trust to bring many permanent staff members’ skills to bear on creating networks, it is clear that Collecting Birmingham has significantly increased the network of communities and local experts with whom Birmingham Museums Trust hopes to work in future.

The very public nature of the project and the open consultation processes employed have gone a long way to demonstrate goodwill to and build trust with communities who had previously had reason to suspect the institution’s intentions. The team have also supported external partners to ensure a more representative process of collecting and presenting culturally significant objects to wider audiences in future years.  As a result, participants have demonstrated changes in attitude towards BMT that augur well for future collaboration.

There is no doubt of the impact that Collecting Birmingham will have on BMT’s Collections Development Policy and on the organisation as a whole. As Rebecca Bridgman, Project Manager and Curator said:

As a result of Collecting Birmingham, I don’t think we’ll ever go back to the method whereby only curators decide what new objects to acquire for the collection.

You can download the report pdf here: Collecting Birmingham Evaluation Report 2018

Art & reminiscence work with dementia patients

Last Autumn, it was my pleasure to research and evaluate the contribution that structured Art & Reminiscence activity can provide to the daily life and care of dementia patients in residential homes.

The full report can be downloaded here: Arts and Reminiscence in Wychavon Care Homes report 2016-02-15

2015-11-23  Bricklehampton 2
Commissioned by Museums Worcestershire and Wychavon District Council Arts Development Officer, the creative and reminiscence activities took place over 5 weeks in 6 different residential care homes across the Wychavon area.  The activity was intended as a pilot – although both organisations have carried out one-off activities in care homes over the last few years, this was their first concerted attempt at providing more in-depth and structured activity over a period of weeks.

 

The work was very challenging, given the participants’ varied capacity to take part.  In some homes, the groups sizes were comparatively small (up to 6 participants) making it easier for the facilitators to focus on each individual and encourage them to take part.  In other homes, large group sizes or the profound state of some participants’ dementia made it far more difficult to make a meaningful connection.  However, as the sessions ran over a series of weeks, the facilitators were able both to build a relationship with the more able participants, and to make slow roads toward connection with the more profoundly disabled residents.

Both facilitators spoke of the work as being emotionally laborious, but also of the rewards they felt they had received when a silent participant suddenly smiled in recognition, or reached for a handshake before leaving.

The evaluation methodology is one that I used in 2013 on the Memories in the Making work for Wolverhampton Art Gallery, and is based upon the Warwick Edinburgh Mental Wellbeing Scale (WEMBWS) and Dementia Mapping techniques.  This Worcestershire project enabled me to test the methodology further and to continue to build upon my experience and knowledge in this rewarding field.

 

 

New report published: MEDIEVAL ABBERLEY REVEALED

Researching and writing evaluation reports is fascinating and enjoyable part of my work.  It is a privilege to be able to follow project teams and volunteers as they deliver complex community activity and then to be able to reflect back to them the impact their work has had.

Excavating for Abberley Castle. An added bonus of evaluating the Medieval Abberley Revealed project!

Excavating for Abberley Castle. An added bonus of evaluating the Medieval Abberley Revealed project!

This week, I finished work on the evaluation report for MEDIEVAL ABBERLEY REVEALED, an HLF-funded Medieval Abberley Revealed community archaeology project which took place over 12 months, and built upon the enthusiasm for local history sparked by the popular Abberley Lives 20th century history project of 2012-2013.

Participation levels on this project were really high, with over 13% of the village either volunteering or attending an event.  The social and well-being outcomes were very strong – even long-time residents expressed how much they had enjoyed meeting new neighbours and volunteering with friends.

The activities seemed to generate a genuine sense of occasion, drawing volunteers aged from 9 to 90 years.  As a local resident and member of the AHPS said later:

‘We joined Abberley Hills Preservation Society but it went dormant for a while.  This activity has revived it all and we have met a lot of people we had never met before. It’s brought us together to fight against [the proposed] houses.  I would absolutely do it again.’

This was the first project evaluation where I was able to use multi-media sources to provide evidence of activity.  Several of the participants have very active social media feeds, so I was able to collate real-time responses during activity and save them using Storify.  I also trained project volunteers to be able to blog so that they could encourage each other to share their reflections online via the Abberley Lives website.

Furthermore, with the advice and assistance of Clear Picture Productions, I have also recorded and edited my first short films for the project website that capture the excitement and curiosity generated during live events.

Thanks very much to Abberley Hills Preservation Society for hiring me to do this lovely piece of work, and for giving me the chance to explore new means of gathering evidence.

Download the report: Medieval Abberley Revealed evaluation

Because you’re worth it!

heritagetourismhlfThe updated Economic Impact Toolkit: West Mids DIY version is here

In a time of financial uncertainty and the ‘downgrading’ of the value of culture, we all need to be able to say with certainty why we are ‘worth it’.

West Midlands Museum Development Officers (WMids MDOs) led the first region-wide survey in 2012 and commissioned me to crunch up the data and see how we had all done.

18 museums replied and demonstrated that in 2011-2012, they contributed over £59 million to the tourism economy. Over Christmas this year WMids MDOs did a quick numbers version, 88 sites responded and showed that museum contribute over £208 millions to the economy in 2012-2013!

Now AIM (Association of Independent Museums) has updated their formulae using government GreenBook statistics.  The WMids MDOs have commissioned me to translate the revised formulae for the West Mids into the attached spreadsheets so that all you need to do is put in the raw numbers, letting Excel do the rest!

With just 4 sets of numbers you will be able to calculate your  impact both regionally and in your local area on:

  • local tourism income
  • local employment figures
  • the economic impact of your expenditure on goods  services in the local area
  • the value of the effort contributed by your volunteers

You are free to use these spreadsheets at ANY time, for your own use…. 

However, as a BIG FAVOUR, I would ask that as many of you as possible complete the relevant spreadsheet by April 18th and send it to back to me.  I shall be collating it all in order to create printed pieces of advocacy for the region and each county …don’t let your impact be left out .

The spreadsheet should be self explanatory, please try to fill in all sheets, save it and send it it me by email.  It has been designed with museums in mind, but there is no real reason why archive or built heritage organisations should not use it.

If you are a large museum, with 50,000  visitors per annum, please use this worksheet: CALCULATE YOUR IMPACT_LARGE museums toolkit_2014_FINAL

If you are a small or medium size museum, with fewer than 49,999 visitors per annum, please use this worksheet: CALCULATE YOUR IMPACT_SMALL-MEDIUM museums toolkit_2014_FINAL

I know this will be a really valuable tool for you all – if you have any queries please contact me.