[Apologies for multiple postings]
ImageCLEFaware (3rd edition) Registration: https://www.imageclef.org/2023/aware Run submission: May 10, 2023 Working notes submission: June 5, 2023 CLEF 2023 conference: September 18-21, Thessaloniki, Greece
*** CALL FOR PARTICIPATION *** Images constitute a large part of the content shared on social networks. Their disclosure is often related to a particular context and users are often unaware of the fact that, depending on their privacy status, images can be accessible to third parties and be used for purposes which were initially unforeseen. For instance, it is common practice for employers to search information about their future employees online. Another example of usage is that of automatic credit scoring based on online data. Most existing approaches which propose feedback about shared data focus on inferring user characteristics and their practical utility is rather limited.
We hypothesize that user feedback would be more efficient if conveyed through the real-life effects of data sharing.
The objective of the task is to automatically score user photographic profiles in a series of situations with strong impact on her/his life. Four such situations were modeled this year and refer to searching for: (i) a bank loan, (ii) an accommodation, (iii) a job as waitress/waiter, and (iv) a job in IT. The inclusion of several situations is interesting in order to make it clear to the end-users of the system that the same image will be interpreted differently depending on the context.
The final objective of the task is to encourage the development of efficient user feedback, such as the YDSYO Android app https://ydsyo.app/.
*** TASK *** Given an annotated training dataset, participants will propose machine learning techniques which provide a ranking of test user profiles in each situation which is as close as possible to a human ranking of the test profiles.
*** DATA SET *** This is the third edition of the task. A data set of more than 1,000 user profiles with 100 photos per profile was created and annotated with an appeal score for a series of real-life situations via crowdsourcing. Participants to the experiment were asked to provide a global rating of each profile in each situation modeled using a 7-points Likert scale ranging from strongly unappealing to strongly appealing. An averaged and normalized appeal score will be used to create a ground truth composed of ranked users in each modeled situation. User profiles are created by repurposing a subset of the YFCC100M dataset.
*** METRICS *** Participants to the task will provide an automatically ranking of user ratings for each situation which will be compared to a ground truth rating obtained by crowdsourcing. The correlation between the two ranked list will be measured using Pearson's correlation coefficient. The final score of each participating team will be obtained by averaging correlations obtained for individual situations.
*** IMPORTANT DATES *** - Run submission: May 10, 2023 - Working notes submission: June 5, 2023 - CLEF 2023 conference: September 18-21, Thessaloniki, Greece (https://clef2023.clef-initiative.eu/)
*** OVERALL COORDINATION *** Jérôme Deshayes-Chossart, CEA LIST, France Adrian Popescu, CEA LIST, France Bogdan Ionescu, Politehnica University of Bucharest, Romania
*** ACKNOWLEDGEMENT *** The task is supported under the H2020 AI4Media “A European Excellence Centre for Media, Society and Democracy†project, contract #951911 https://www.ai4media.eu/.
On behalf of the Organizers,
Bogdan Ionescu https://www.AIMultimediaLab.ro/