Media and Journalism Students verifying with a WeVerify component – Context Aggregation and Analysis

By Olga Papadopoulou on December 19th, 2019 in News

On 12 December 2019, the MeVer team of MKLab CERTH-ITI launched a user study during a  Media Informatics Lab session in collaboration with Professor Andreas Veglis and Dr. Dimitrios Giomelakis of the Media and Journalism department of the Aristotle University of Thessaloniki. The aim of the user study was to assess the value of the Context Aggregation and Analysis (CAA) tool for journalistic verification tasks. The tool is already part of the InVID-WeVerify plugin which is used by more than 17.000 users from mainstream media organisations, social media users and fact-checking teams.

The Media Informatics Lab session was attended by 28 students, of which nine were male and nineteen female. Students are in their 3rd or 4th year of their studies. They attended the presentation of the CAA tool first and were then asked to verify a set of fake and real videos. 

Presentation of Context Aggregation and Analysis tool.

The user study has been split in three stages, and the students were asked to verify a number of videos in each of them. In total, 70 videos, 35 real and 35 fake were selected by the Fake Video Corpus and other sources. During the first stage, the students had to verify 20 videos without the use of the CAA tool, following best practices that they have already studied and were aware of. For the next two stages, the students had to use as their primary means of verification the CAA tool. The results that they reported are the fake/real label and the degree of certainty which quantifies how certain the student was about his/her label, the list of verification cues that contributed the verification (during the first stage, where the CAA tool was not used, the verification cues were not reported), a free text description of the analysis and the time that the student needed to verify the video.

Screenshot of the tool which was used for reporting the results.

Students in action. Photos by Olga Papadopoulou and Dimitrios Giomelakis.

The study will run until 13 January 2020. When it has been completed, the collected data will be analyzed and reported.

Overall, the study so far has shown that the prospective journalists are interested in the tool and willing to use it again. As a final step, a discussion between CERTH’s researcher Olga Papadopoulou and the students will be held on 16 January 2020, after the completion of the user study, where the observations and comments of the students about the tool and its usage will be collected and taken into account for its next release. 

Author: Olga Papadopoulou (CERTH). Editor: Jochen Spangenberg (DW)

Image credits: respective persons named. Usage rights have been obtained by the authors named above for publication in this article. Copyright / IPR remains with the respective originators.

Note: This post is an adaptation of the Media and Journalism Students Play with MeVer Tools blog post, which was originally prepared for the CERTH Media Verification team (MeVer)  website.