AI Powered Toolkits
SHIFT will enrich the public experience of interacting with cultural heritage using a series of tools grouped within themed toolkits. These include a visual toolkit that will transcribe cultural heritage content automatically using AI and an auditory toolkit that will deliver an emotional response to the audience.
What does this mean in practice? A few months ago, the auditory toolkit team (headed by AUDEERING) provided an example of this work using a Text-to-Speech tool (TTS). The TTS transcribed heritage videos provided by partners in Romanian and Serbian and then automatically revoiced them in English with emotional affect, moving away from the stilted automatic voices of the past. Improvements are constantly being made to this tool but you can watch a taster of the pilot version on the SHIFT YouTube channel.
Recognising the sector's need for inclusivity and accessibility, SHIFT focuses on generating targeted textual descriptions for artworks, ensuring diverse audiences — such as blind and partially sighted individuals — can engage with cultural assets. To achieve this, the project has developed a tool that enables curators to upload images and metadata, leveraging AI and Vision Language models to craft user-specific narratives.
A haptics toolkit developed by FORTH, Greece’s largest research centre, will allow users to interact using also their sense of touch to explore 3D and 2D objects in museums and other cultural settings. A haptic glove will enable sensory feedback in this novel approach. In this video you can check out an early version of the haptics tool.
The project outputs will also include an Intellectual Property Rights toolkit designed to protect native content by adopting international copyright and digital protection of derivative copyrights. This will be shared in the coming months.
The role of the Europeana Initiative
Europeana has played a pivotal role in supporting SHIFT’s mission through its tools and resources. For example, through the integration of the Europeana API, the project automates the retrieval of rich metadata for artworks — such as creator information and descriptions — which enriches the contextual input for the AI models. This integration not only accelerates metadata collection but also enhances the tool’s ability to deliver accurate descriptions.
This collaboration exemplifies how the Europeana’s Initiative’s resources can fuel innovation and accessibility in cultural heritage and highlights the transformative potential of its API in fostering inclusivity. Together, we are shaping a future where cultural treasures are accessible to all.
Coming up next
To ensure we deliver the right tools, team SHIFT has launched an extensive set of workshops, focus groups and use cases to assess user requirements and test solutions across partner sites in the museums and libraries of Germany, Greece, Hungary, Romania and Serbia. The project is now focused on integrating this feedback to develop more inclusive and accessible technology solutions for all users of museums and libraries. Soon the first results of this work will be trialled in a number of exhibitions hosted across the partner sites.
Meanwhile, a policy brief produced by SHIFT and the other five projects (MuseIT, PERCEIVE, MEMORISE, PREMIERE, and MEMENTOES) that make up the EU-funded cluster for advanced technologies in cultural heritage is currently being finalised and will be published in the coming weeks.
Get involved
Visit the SHIFT project website to find out more, sign up to our newsletter and discover opportunities to contribute to the development of our tools and toolkits by subscribing. We welcome your views and feedback on SHIFT as results appear on our website!
This post was edited on 9 December 2024 to add details about a tool that enables curators to upload images and metadata, leveraging AI and Vision Language models to craft user-specific narratives; and the section on the Europeana Initiative.