How to Build a Deepfake Detection System using Python/ML: A practical coding tutorial for video and audio verification.

CyberDudeBivash ThreatWire

How to Build a Deepfake Detection System Using Python/ML: A Practical Tutorial for Video and Audio Verification

By CyberDudeBivash Pvt Ltd
Independent, practitioner-led security engineering for modern media risk

Executive context

Deepfakes are no longer a “research-only” threat. They now show up in:

• Executive impersonation (voice + video) for fraud and extortion

• Recruitment scams and social engineering

• Brand abuse and reputational attacks

• Evidence manipulation and disinformation campaigns

What makes this risk operationally difficult is that detection is not a single model problem—it’s a pipeline problem: ingest, preprocess, feature extraction, scoring, decision thresholds, and human review.

This edition provides a practical blueprint to build a Python-based deepfake verification system for both video and audio, designed for real workflows (SOC, trust & safety, investigations, media verification).

System design overview (what you’re building)

A robust deepfake detection system is best implemented as two parallel detectors plus a fusion layer:

1. Video detector (frame-level and temporal cues)

2. Audio detector (voice authenticity + artifacts)

3. Fusion/scoring (combine signals, calibrate thresholds, produce a decision)

4. Explainability layer (return reasons: low confidence, face swap traces, voice artifacts, mismatch)

5. Human review workflow for borderline cases

Your final output should not be “real vs fake” only. It should be a risk score + rationale.

Part A — Video Deepfake Detection (Python workflow)

1) Extract frames and faces

Most modern video deepfake detectors work on face crops rather than full frames.

Install

pip install opencv-python facenet-pytorch torch torchvision

Face detection + cropping

import cv2
from facenet_pytorch import MTCNN
import torch

mtcnn = MTCNN(keep_all=True, device="cuda" if torch.cuda.is_available() else "cpu")

def extract_face_crops(video_path, every_n_frames=5, out_size=224, max_faces=1):
    cap = cv2.VideoCapture(video_path)
    frame_id = 0
    crops = []

    while True:
        ret, frame = cap.read()
        if not ret:
            break
        frame_id += 1
        if frame_id % every_n_frames != 0:
            continue

        rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        boxes, _ = mtcnn.detect(rgb)

        if boxes is None:
            continue

        # Pick the largest face (common in verification use-cases)
        boxes = sorted(boxes, key=lambda b: (b[2]-b[0])*(b[3]-b[1]), reverse=True)[:max_faces]
        for b in boxes:
            x1, y1, x2, y2 = map(int, b)
            face = rgb[max(0,y1):y2, max(0,x1):x2]
            if face.size == 0:
                continue
            face = cv2.resize(face, (out_size, out_size))
            crops.append(face)

    cap.release()
    return crops

Why this matters: deepfakes typically manipulate facial regions; clean face-crops improve signal and lower noise.

2) Choose a model approach

There are three practical approaches:

Approach 1: Use a pretrained deepfake detector
Best for fast deployment, good baseline.

Approach 2: Fine-tune a general vision backbone (EfficientNet/ViT) on deepfake datasets
Best balance between performance and engineering effort.

Approach 3: Add temporal modeling (CNN + LSTM/Transformer)
Best for attacks that look good per-frame but fail across motion/consistency.

For most teams, Approach 2 is the practical default.

3) Fine-tuning a simple video-frame classifier

This example shows the training skeleton (you’ll adapt to your dataset loader).

import torch
import torch.nn as nn
from torchvision import models, transforms

device = "cuda" if torch.cuda.is_available() else "cpu"

model = models.efficientnet_b0(weights=models.EfficientNet_B0_Weights.DEFAULT)
model.classifier[1] = nn.Linear(model.classifier[1].in_features, 2)  # real vs fake
model = model.to(device)

transform = transforms.Compose([
    transforms.ToPILImage(),
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]),
])

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.AdamW(model.parameters(), lr=2e-4)

def train_one_epoch(dataloader):
    model.train()
    total_loss = 0.0
    for faces, labels in dataloader:
        faces = faces.to(device)
        labels = labels.to(device)

        logits = model(faces)
        loss = criterion(logits, labels)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    return total_loss / max(1, len(dataloader))

Operational advice: do not stop at accuracy. Evaluate precision/recall and calibrate thresholds for your business risk.

4) Video scoring strategy (don’t classify one frame)

Use multiple face crops and aggregate probabilities:

• Score each crop: p_fake

• Aggregate: median/trimmed mean

• Output: risk_score + confidence

import numpy as np
import torch.nn.functional as F

@torch.no_grad()
def score_video_faces(face_crops):
    model.eval()
    scores = []
    for face in face_crops:
        x = transform(face).unsqueeze(0).to(device)
        logits = model(x)
        p = F.softmax(logits, dim=1)[0,1].item()  # probability fake
        scores.append(p)

    if not scores:
        return {"risk_score": None, "reason": "no_face_detected"}

    risk = float(np.median(scores))
    return {"risk_score": risk, "frames_scored": len(scores)}

Key point: deepfake detection is probabilistic. A single frame can be misleading.

Part B — Audio Deepfake Detection (Python workflow)

Audio deepfakes are often detected by:

• Spectral artifacts (phase inconsistency, over-smoothing)

• Model fingerprints

• Speaker mismatch (claimed speaker vs observed speaker)

1) Convert audio to mel-spectrogram

pip install librosa soundfile numpy

import librosa
import numpy as np

def audio_to_melspec(audio_path, sr=16000, n_mels=128, hop_length=160, n_fft=512):
    y, _ = librosa.load(audio_path, sr=sr, mono=True)
    mels = librosa.feature.melspectrogram(y=y, sr=sr, n_mels=n_mels, n_fft=n_fft, hop_length=hop_length)
    mels_db = librosa.power_to_db(mels, ref=np.max)
    return mels_db.astype(np.float32)

2) Train a lightweight CNN classifier

This is a minimal CNN for spectrogram classification (real vs fake). In production, you would likely use a stronger architecture, but the pipeline is similar.

import torch
import torch.nn as nn

class AudioCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.AdaptiveAvgPool2d((1,1)),
        )
        self.fc = nn.Linear(64, 2)

    def forward(self, x):
        x = self.net(x).flatten(1)
        return self.fc(x)

Practical note: audio models are sensitive to dataset diversity and microphone conditions. Validate across devices and compression levels.

Part C — Fusion: combine video + audio into one decision

A practical scoring method:

• If both scores exist:

  • final_risk = 0.6 * video_risk + 0.4 * audio_risk (tune by environment)

• If only one exists: use that score with lower confidence

• Add policy thresholds:

  • risk < 0.35: likely authentic

  • 0.35–0.65: review required

  • > 0.65: likely synthetic/manipulated

Return structured output:

• final_risk

• video_risk, audio_risk

• confidence_reason

• recommended action

Part D — What separates “toy projects” from real systems

To make this operational, you must add:

1) Dataset strategy

Use representative data:

• Different lighting, cameras, compression, languages, and codecs

• Real calls, meeting audio, user-generated video styles

• Evaluate against unseen manipulation methods

2) Calibration and false-positive management

Deepfake detection at scale fails if false positives are high. Use:

• threshold calibration on a clean validation set

• “review queue” design (human-in-the-loop)

3) Adversarial resilience

Attackers can:

• re-encode video to destroy artifacts

• apply post-processing to hide traces

• mix real audio with synthetic segments

Defend by:

• using ensembles (multiple detectors)

• including compression augmentations during training

• evaluating on “hard negatives”

4) Evidence integrity

If you’re verifying content for investigations:

• hash inputs

• preserve originals

• log model version and score metadata

CyberDudeBivash ecosystem

CyberDudeBivash Pvt Ltd supports organizations building verification and fraud-resilience programs through:

• Deepfake risk assessments and workflow design

• Media verification pipelines (SOC/trust & safety/investigations)

• Security awareness programs for executive impersonation threats

• Cloud, identity, and incident readiness services

Explore our Apps, Products & Services:
https://www.cyberdudebivash.com/apps-products/

Recommended by CyberDudeBivash

For teams operationalizing detection programs:

• Endpoint protection for analysis workstations and responder laptops (Kaspersky)

• Hands-on security and DevSecOps training for analysts and engineers (Edureka)

(Partner links support the CyberDudeBivash ecosystem at no extra cost.)

#cyberdudebivash #CyberDudeBivashThreatWire #CyberDudeBivashPvtLtd #DeepfakeDetection #AIForSecurity #MachineLearning #Python #ComputerVision #AudioForensics #VideoForensics #DFIR #ThreatIntel #FraudPrevention #IdentitySecurity #SocialEngineering #SecurityEngineering #CyberSecurity #CISO