Setting out and setting up
Across the wealth of activity within the Europeana Initiative, choosing where to focus and conduct our impact assessments was not easy. We tried to find a good balance across the Initiative’s activities in terms of partners, communities and format. We also had to think practically - where could we get enough data within the timeframe - and materially - where do we anticipate substantial impact to occur?
In the end, we looked at the impact of Europeana Network Association membership, and considered participatory initiatives like Transcribathon (Enrich Europeana Generic Services project) and the Europe at Work campaign. We looked into the impact of EuropeanaTech and the Europeana Initiative on the promotion and adoption of IIIF within Europe, and the social return on investment of the Europeana Education MOOCs. We investigated capacity development initiatives like our National Workshops programme with our aggregator partners, our digital programming (webinars and online workshops) and the biannual European presidency events. Impact assessments of the Europeana annual conferences in 2019 and 2020 will also be published shortly.
How we did it
We used a mixed methods approach to assess the economic, social, innovation and organisational impact of Europeana’s services. Each case study has its own discrete approach and methodology; some are ‘light-touch’ and others are more in-depth because we had more opportunity and flexibility in terms of data collection and methods. Some incorporate the views of multiple stakeholders, some just one main stakeholder.
The research was collaborative from the outset. Collaboration, combined with the enthusiasm of colleagues for contributing to it, suggests that impact methodology is becoming more embedded in processes and mindsets. It doesn’t mean our processes are perfect: we are still learning and improving. Yet embedding impact in how we design and deliver our work is already giving us tangible value. Thanks to our findings, we have more data to support our reporting to the European Commission and we are learning and creating better events and programmes for the sector.
What we learned
Each of the case studies offers learning about the specific service, tool or event that they examined, and we encourage you to read the individual reports and their summary pages. However, we also drew out some high-level insights that we’d like to share. We learned that:
We need to think holistically about impact. Although we know, for example, about the social return on investment of our Europeana Education MOOCs, we still need to understand what value the knowledge the educators gained had for the quality of their students’ education.
Europeana has added value when addressing socially relevant themes. We know from our Impact Assessment of Europeana 2019 that the fact we were proactively working to minimise negative environmental impact was important to conference attendees. But whether it’s the diverse stories we want to tell through our editorial, or our participatory campaigns like Europeana Migration or Europe at Work, digital heritage is most relevant and can have the greatest impact when it speaks to the language - and the challenges - of contemporary life.
Europeana creates impact as a convenor of sector professionals. The strongest outcomes of our annual conferences are the networks they create. While harder to measure, we think that this can create impact through the generation of new knowledge through collaborative projects or increased sustainability of the sector through levered funding.
Depth of experience leads to stronger impact. In initiatives like Transcribathon, increased depth of connection and interaction with the tool is likely to lead to those longer-term outcomes we want to see. Similarly, with our Network, we need to encourage greater depth of involvement and experience.
Barriers to digital transformation cannot be solved by Europeana alone: it’s about partnership and collaboration. This was also one of the strongest themes that emerged from the sense-making workshops held last summer. How do we convene the sector to co-create, learn together, support each other, and benefit from economies of scale, in order to deliver impact in a period of accelerated digital transformation?
(Digital) change takes time. IIIF is a huge innovation but its implementation takes a lot of time (and this has implications for impact assessment, too). The digital transformation of the sector won’t happen overnight, and we need to start thinking now about how to support, monitor and measure this change.
Where do we go from here?
These case studies, and the ones that follow, are designed to be replicable models for others to build and improve on. They document our learning and help us grow the data to help us answer the question, ‘what is the value of digital cultural heritage?’. The challenges faced by cultural heritage institutions in measuring their impact are likely to be even more acute as a result of the current COVID-19 crisis. There is even more urgency to understand and measure the value of cultural heritage in order to advocate for continued financial, political and social support. Furthermore, we need to continue to experiment, learn and innovate in order to harness the full impact that digitisation and digitalisation - in short, digital transformation - can have on a cultural heritage instution’s mission.
In our Impact Community webinar series we are seeing a need for practical training in impact assessment. To meet this, the Impact Lite Task Force is thinking about ways to turn observers of impact assessments into practitioners. Training materials will be released as part of their work and also as part of the continued development of the Europeana Impact Playbook. Phase three of the Playbook is due to be published in the Autumn of 2021, and further materials to support Phase two, including a standardised question bank, will soon be available. Ongoing work to develop a Europeana Initiative Theory of Change will further guide focus on key areas where we want to demonstrate impact.
Get involved
You can find each report on our Impact case studies page on Europeana Pro. Take a look through the summaries, see if anything strikes you, and then delve into the main report. We want you to learn from and build on the approach that we are taking and we want to hear about your experiences, too.
In the midst of social justice action, a global health emergency, the climate catastrophe and ongoing political upheaval, cultural heritage has an important role to play and we want to make sure the Impact Playbook is there to support and catalyse change. If you’re interested in being part of our next steps developing the Playbook, get in touch or join the Europeana Impact Community to stay up to date with new initiatives!