AI4EU Cafe: Challenges for AI in the description of cultural heritage paintings

-

This AI4EU Café will present Joaquim Moré. He is working actively in the BSC for the Oficina Técnica de Gestión for the Plan Nacional de Impulso a las Tecnologías del Lenguaje, sponsored by the Spanish Ministerio de Asuntos Económicos y Transformación Digital.

Description

Current AI systems that describe images are mostly trained with photographs that depict present-day scenes. The talk will be centered on the challenges that the Saint George on a Bike project has faced when describing paintings that depict scenes of the past and entities that belong to the supernatural. First, the challenges related to the differences between photographs and paintings will be exposed, as well as topics such as to what extent the differences in the identification of objects are based on technical matters (e.g: differences in textures) or rather on the presence of different objects and actions altogether, or even the different disposition of objects in present day images compared to past images. Secondly, we will discuss aspects related to transfer learning, mainly when identifying supernatural beings and entities that are not likely to be found in present day photographs, and we will report on how Saint George on a Bike prevents the identification of entities that are anachronistic. Lastly, the talk will deal with the challenges in collecting training data for describing cultural heritage paintings. We will discuss the availability of images of paintings with their descriptions and will show how these descriptions, especially when taken from art collections, must be first processed to select only the scene content, while leaving out content that introduces noise in the training (biographical data of the painter, for example).

CV

Dr. Joaquim Moré López is a senior researcher and expert in Computational Linguistics. He has a Ph.D. in Knowledge and Information Society at the Open University of Catalonia. His major areas of expertise are Machine Translation, Information Extraction, Text Mining, Natural Language Processing, Knowledge Engineering and Opinion Mining. He is working actively in the BSC for the Oficina Técnica de Gestión for the Plan Nacional de Impulso a las Tecnologías del Lenguaje, sponsored by the Spanish Ministerio de Asuntos Económicos y Transformación Digital, to use HPC to exploit the possibilities of Natural Language Processing for Public and Private institutions. He provides solutions to issues related to natural language processing in the Saint George on a Bike project.

 

Joaquim Moré
Taggar AI4EU Café