Description: Build a model that determines whether a video is a deepfake​

The team performed the following activities:

Data Preprocessing​- Extracted sample frames from each video (pulled from a pre-established data set in Cloud Storage)​, created bounding boxes using Multi-task Cascaded Convolutional Networks (MTCNN), which is the standard pre-trained tool for facial recognition. Used bounded boxes to crop images​ and extract features from cropped frames​

Data Storage- Stored data using Google Cloud Platform architecture; Stored Frames and Features Metadata in Google Cloud Datastore​

Model Build/Training-​ Used VertexAI, which is Google CloudPlatform’s unified machine learning platform, and used TensorFlow as the ML language.  Model was trained using EfficientNetV2. EfficientNet is a newer convolutional neural network that is pretrained for image classification which ramps up its training speed significantly.

Push Processed Data to Nearline in Cloud Storage​

Results: Analyzed 500gb of video data resulting in approximately 95 percent accuracy in detecting deepfakes.

‍