Deconstructing Zuck’s deep fake

By on June 21st, 2019 in Blog Posts, News

The recent so-called “deep fake” circulating on Instagram involving Mark Zuckerberg allegedly telling the ‘truth’ about privacy on Facebook was another episode of the fearful future of disinformation. Potentially, any celebrity can be caught in an awkward situation or say any kind of unexpected statements, through artificial intelligence (read machine learning) manipulated media content.

An animated gif made with ffmpeg with 30 frames between the 2nd and 3rd seconds, with a x2 upscaling

So far, most of the “deep fakes” (also called synthetic media) that made the news were fabricated on purpose for advertising, parody or simply to draw attention. It is the same case here: the manipulated audio track does not look like Zuckerberg’s voice so the manipulation remains easy to spot.

While we are currently testing some detectors of the several AI techniques available to manipulate video content, we investigated that video with our verification plugin.

On any fake video, keyframe similarity search remains so far the best method to try to identify if those images are already known and if they have been previously indexed by some reverse image search engine. Our Instagram video here is no exception.

First, the plugin allows, through its launcher, to retrieve the mp4 video link so that you can download it. Then you can use the keyframes fragmentation tool with that video as a local file. Once fragmented, a reverse video search on some keyframes will allow you to get back to several similar videos.

One of those videos, on YouTube, is clearly very close, with a similarity score of above 0.97. The 18 seconds segment of the “deep fake”, with the same CBSNews chyron, is a duplicate of the above YouTube video fragment from 3:48 till 4:06. It took us quite a bit of time to match manually both sequences mostly by looking at the video background. This fastidious and time-consuming task could be automated and we are working on better identifying and matching images and videos.

At the end, our “deep fake” was mainly a a decontextualized 18 sec video footage from a previous September 2017 video, apparently tampered with some face re-enactement. It can be seen between the 2nd and 3rd second if you look Zuckerger’s left ear movement on the right side from the viewer perspective (see the above animated gif made with 30 frames of the Instagram video).

  • Share:

Leave a Comment

sing in to post your comment or sign-up if you dont have any account.