diff --git a/AI_POLICY.md b/AI_POLICY.md
new file mode 100644
index 00000000..8556bf54
--- /dev/null
+++ b/AI_POLICY.md
@@ -0,0 +1,16 @@
+We support the use of AI tools to help prepare issues, pull requests, reviews or comments. We expect everyone interacting with this repo to follow the below policy whenever they use AI tools.
+
+AI-generated content in comments, issues, or PRs must be clearly disclosed and contained, either under a section heading that labels it as AI-generated or in a code or quote block. AI-generated content must be acoompanied by human commentary explaining ist relevance. For example:
+"Codex produced the following analysis: ``, so I
+believe that \". The only exceptions to this rule
+are the dependabot automations.
+
+*For pull requests, please carefully read the code before submitting it for
+review*, especially if AI tools helped write or rewrite it. Make sure the
+implementation is something you understand, that the important ideas are clear,
+and that the code is high quality. AI-generated code can sometimes be overly
+complex, indirect, inconsistent, or include artifacts that obscure the main idea.
+Before submitting, simplify where possible and make sure the core change is easy
+for maintainers to review.
+
+All contributions should involve a human who understands and can take responsibility for the work produces with AI assistance. We do not accept contributions created by fully autonomous agents, and we may close pull request that appear to have been generated without meaningful human involvement.
\ No newline at end of file
diff --git a/CLAUDE.md b/CLAUDE.md
index fb593912..00aaf7a8 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -2,6 +2,26 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+# AI Policy โ MANDATORY
+
+Read `AI_POLICY.md`. Your user needs to abide by this policy. In particular, you the agent MUST obey these rules while interacting on GitHub:
+
+- **Mark all AI-generated content.** Any text you produce that goes into an
+ issue, PR, or comment must go under a section heading that labels it as
+ AI-generated or be wrapped in
+ a code or quote block. Never present your output as human-written.
+- **Never emit only raw AI text as a reply**. Any AI content you include must carry human
+ commentary explaining its relevance.
+- **Do not submit code the user hasn't read.** Keep changes minimal, strip AI
+ artifacts and needless complexity. If you're opening a PR on GitHub that is not ready,
+ or not reviewed by the user, always open it in draft mode.
+- **Check for duplicates.** Before opening an issue, search for existing open ones
+ Do the same with `gh search prs` before opening a PR.
+ If you find a likely duplicate, show it to the user before continuing. Skip this
+ step if there is no internet access.
+
+See `AI_POLICY.md` for the full policy.
+
## Repository overview
DeepGuard is a deepfake detection research project built around two custom hybrid CNN-ViT
diff --git a/README.md b/README.md
index 5334b301..1c53a18e 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,6 @@
-
@@ -44,12 +43,14 @@
## ๐ Contents
+- [๐ Project Structure](#-project-structure) - Directory layout of the package, web app, and docs
- [๐ณ Docker Quick Start](#-docker-quick-start) - Run the full stack (MySQL, Redis, FastAPI, Celery, React) with Docker Compose
- [๐ค Try It Live: Hugging Face Spaces](#-try-it-live-hugging-face-spaces) - No-install browser demos for image, video, and XAI deepfake detection
- [๐ DeepFake Video BenchMark Datasets](#-deepfake-video-benchmark-datasets) โ Overview of Celeb-DF-v2, FF++, and KoDF datasets used for training.
- [โ๏ธ Data Preparation](#data-preparation) โ Efficient face detection and landmark extraction pipeline using YOLOv8
- [๐ Model Architecture](#-model-architecture) โ Detailed look into our hybrid CNN-ViT (MS-EffViT & MS-EffGCViT) designs.
- [๐งฌ Model Zoo](#-model-zoo) โ Comparison of model variants, parameter counts, and computational complexity (FLOPs).
+- [๐ Model Inference Latency](#-model-inference-latency) - Fast (b0) vs Pro (b5) inference speed on a single Tesla T4
- [๐ Training](#-training) - Step-by-step training scrips with Goolge Colab and W&B experiment tracking
- [๐ Model Evaluation](#-model-evaluation) - Benchmarking results
- [๐ป Model Usage](#-model-usage) - Load pretrained models via `pip install deepguard` or straight from the Hugging Face Hub
@@ -60,6 +61,29 @@
- [๐ Reference](#-reference) - Libraries, datasets, and prior work this project builds on
- [โ๏ธ License](#-license) - MIT license
+## ๐ Project Structure
+
+```
+DeepGuard/
+โโโ deepguard/ # Core package (models, layers, data pipeline, optimizer, scheduler, utils)
+โโโ preprocess/ # Face detection, cropping & landmark extraction pipeline
+โโโ inference/ # Image / video prediction
+โโโ explainability/ # Dual-branch CAM-based XAI
+โโโ App/ # FastAPI backend (excluded from PyPI package)
+โโโ client/ # React frontend
+โโโ docker/ # MySQL / Redis Docker images and configs
+โโโ tutorials/ # Colab notebooks (prediction & XAI visualization)
+โโโ docs/ # README assets (architectures, ablation, benchmarks, samples, XAI results)
+โโโ .github/ # CI workflows and repository settings
+โโโ docker-compose.yml # Full stack (MySQL, Redis, FastAPI, Celery, React)
+โโโ train_eff_vit.py # Training entrypoint for MS-EffViT
+โโโ train_eff_gcvit.py # Training entrypoint for MS-EffGCViT
+โโโ pyproject.toml # Package build config
+โโโ requirements.txt # Dev environment dependencies
+โโโ README.md, README_KR.md, README_JP.md
+โโโ LICENSE
+```
+
## ๐ณ Docker Quick Start
Run the full stack (MySQL + Redis + FastAPI + Celery + React) with Docker Compose โ no local Python/Node setup required.
@@ -223,10 +247,19 @@ We utilizes two distinct types of self-attention to capture both long-range and
## ๐งฌ Model Zoo
-| Model | Resolution | # Total Params(M) | # Backbone(M) | # L-ViT(M) | # H-ViT(M) | FLOPs (G) | Model Config |
-| ----- | ---------- | -------------- | ----------- |------------- | ------------- | -------------- | ------- |
-| โก ms_eff_gcvit_b0 | 224 X 224 | 8.7 | 3.6(41.4%) | 1.7(19.5%) | 3.3(37.9%) | 0.87 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
-| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.3 | 27.3(54.3%) | 6.6(13.1%) | 16.1(32.0%) | 13.64 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| Model | Resolution | # Params(M) | # FLOPS(G) | Model Config |
+| ----- | ---------- | -------------- | ----------- | ------- |
+| โก ms_eff_gcvit_b0 | 224 X 224 | 8.70 | 0.85 | [spec](deepguard/config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
+| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.34 | 13.43 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+
+## ๐ Model Inference Latency
+
+Latency is measured with batch size 1 on a single Tesla T4 and video (num_frames=20).
+
+| Model | Mode | Target Deployment | Latency (s) | Ratio |
+| ----- | ---- | ----------------- | ----------- | ----- |
+| โก ms_eff_gcvit_b0 | Fast Mode | Edge / on-device deployment | 0.051 | 1.00ร |
+| ๐ฅ ms_eff_gcvit_b5 | Pro Mode | Enterprise deployment | 0.479 | 9.41ร |
## ๐ Training
@@ -240,13 +273,6 @@ pip install -r requirements.txt
We provide training scripts for both `ms_eff_vit` and `ms_eff_gcvit`. We recommend using **Google Colab** for free GPU access and **Weightes & Biases(W&B)** for experiment tracking
-#### ๐ Weight & Biases Experiments
-
-* **ms_eff_vit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_kodf)
-* **ms_eff_vit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_kodf)
-* **ms_eff_gcvit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_kodf)
-* **ms_eff_gcvit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_kodf)
-
```python
!python -m train_eff_vit \ # train_eff_gcvit
--root-dir DATA_ROOT \
@@ -601,6 +627,9 @@ The jupyter notebooks themselves can be found under the tutorials folder in the
| 8 | [`Pytorch-Grad-Cam`](https://github.com/jacobgil/pytorch-grad-cam) | Advanced Visual Explanations for PyTorch Models |
| 9 | [`DeepfakeBench`](https://github.com/SCLBD/DeepfakeBench) | A Comprehensive Benchmark of Deepfake Detection by SCLBD |
| 10 | [`Effort-AIGI-Detection`](https://github.com/YZY-stack/Effort-AIGI-Detection) | Official Implementation of Effort (ICML 2025 Oral) by YZY-stack |
+| 11 | [`SIA`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/sia_detector.py) | An Information Theoretic Approach for Attention-Driven Face Forgery Detection (ECCV 2022), implemented in DeepfakeBench |
+| 12 | [`CORE`](https://github.com/niyunsheng/CORE) | Official Implementation of CORE: COnsistent REpresentation Learning for Face Forgery Detection (CVPRW 2022) by niyunsheng |
+| 13 | [`IID`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/iid_detector.py) | Implicit Identity Driven Deepfake Face Swapping Detection (CVPR 2023), implemented in DeepfakeBench |
## โ๏ธ License
diff --git a/README_JP.md b/README_JP.md
index ef7dea9a..f0a3a17e 100644
--- a/README_JP.md
+++ b/README_JP.md
@@ -25,7 +25,6 @@
-
@@ -44,12 +43,14 @@
## ๐ ็ฎๆฌก
+- [๐ ใใญใธใงใฏใๆงๆ](#-ใใญใธใงใฏใๆงๆ) - ใใฃใฌใฏใใชๆงๆใฎๆฆ่ฆ (ใใใฑใผใธใWebใขใใชใใใญใฅใกใณใ)
- [๐ณ Docker Quick Start](#-docker-quick-start) - Docker Compose ใงใใซในใฟใใฏ(MySQL, Redis, FastAPI, Celery, React)ใ่ตทๅ
- [๐ค ไปใใไฝ้จ: Hugging Face Spaces](#-ไปใใไฝ้จ-hugging-face-spaces) - ใคใณในใใผใซไธ่ฆใใใฉใฆใถใ ใใง่ฉฆใใ็ปๅ/ๅ็ป/XAI ใใฃใผใใใงใคใฏๆคๅบใใข
- [๐ ใใฃใผใใใงใคใฏๅ็ปใใณใใใผใฏใใผใฟใปใใ](#-ใใฃใผใใใงใคใฏๅ็ปใใณใใใผใฏใใผใฟใปใใ) โ ๅญฆ็ฟใซไฝฟ็จใใ Celeb-DF-v2, FF++, KoDF ใใผใฟใปใใใฎๆฆ่ฆใ
- [โ๏ธ ใใผใฟๆบๅ](#ใใผใฟๆบๅ) โ YOLOv8 ใ็จใใๅน็็ใช้กๆคๅบใใใณใฉใณใใใผใฏๆฝๅบใใคใใฉใคใณใ
- [๐ ใขใใซใขใผใญใใฏใใฃ](#-ใขใใซใขใผใญใใฏใใฃ) โ ใใคใใชใใ CNN-ViT (MS-EffViT & MS-EffGCViT) ่จญ่จใฎ่ฉณ็ดฐใ
- [๐งฌ Model Zoo](#-model-zoo) โ ใขใใซใใชใขใณใใใจใฎใใฉใกใผใฟๆฐใใใณๆผ็ฎ้(FLOPs)ใฎๆฏ่ผใ
+- [๐ Model Inference Latency](#-model-inference-latency) โ Tesla T4 1ๅบใซใใใ Fast(b0)ใจ Pro(b5)ใฎๆจ่ซ้ๅบฆใฎๆฏ่ผใ
- [๐ ๅญฆ็ฟ](#-ๅญฆ็ฟ) - Google Colab ใใใณ W&B ใๆดป็จใใๆฎต้็ใชๅญฆ็ฟในใฏใชใใใ
- [๐ ใขใใซ่ฉไพก](#-ใขใใซ่ฉไพก) - ใใณใใใผใฏ็ตๆใ
- [๐ป ใขใใซใฎไฝฟใๆน](#-ใขใใซใฎไฝฟใๆน) - `pip install deepguard` ใพใใฏ Hugging Face Hub ใใๅญฆ็ฟๆธใฟใขใใซใใญใผใ
@@ -62,6 +63,29 @@
---
+## ๐ ใใญใธใงใฏใๆงๆ
+
+```
+DeepGuard/
+โโโ deepguard/ # ใณใขใใใฑใผใธ (ใขใใซใใฌใคใคใผใใใผใฟใใคใใฉใคใณใใชใใใฃใใคใถใในใฑใธใฅใผใฉใใฆใผใใฃใชใใฃ)
+โโโ preprocess/ # ้กๆคๅบใใฏใญใใใใฉใณใใใผใฏๆฝๅบใใคใใฉใคใณ
+โโโ inference/ # ็ปๅ / ๅ็ปใฎไบๆธฌ
+โโโ explainability/ # ใใฅใขใซใใฉใณใ CAM ใใผใน XAI
+โโโ App/ # FastAPI ใใใฏใจใณใ (PyPI ใใใฑใผใธใใ้คๅค)
+โโโ client/ # React ใใญใณใใจใณใ
+โโโ docker/ # MySQL / Redis Docker ใคใกใผใธใจ่จญๅฎ
+โโโ tutorials/ # Colab ใใผใใใใฏ (ไบๆธฌใจ XAI ๅฏ่ฆๅ)
+โโโ docs/ # README ็จใขใปใใ (ใขใผใญใใฏใใฃใablationใใใณใใใผใฏใใตใณใใซใXAI ็ตๆ)
+โโโ .github/ # CI ใฏใผใฏใใญใผใจใชใใธใใช่จญๅฎ
+โโโ docker-compose.yml # ใใซในใฟใใฏ (MySQL, Redis, FastAPI, Celery, React)
+โโโ train_eff_vit.py # MS-EffViT ๅญฆ็ฟใจใณใใชใใคใณใ
+โโโ train_eff_gcvit.py # MS-EffGCViT ๅญฆ็ฟใจใณใใชใใคใณใ
+โโโ pyproject.toml # ใใใฑใผใธใใซใ่จญๅฎ
+โโโ requirements.txt # ้็บ็ฐๅขใฎไพๅญ้ขไฟ
+โโโ README.md, README_KR.md, README_JP.md
+โโโ LICENSE
+```
+
## ๐ณ Docker Quick Start
Docker Compose ใงใใซในใฟใใฏ(MySQL + Redis + FastAPI + Celery + React)ใ่ตทๅใใพใ โ ใญใผใซใซใซ Python/Node ็ฐๅขใ็จๆใใๅฟ
่ฆใฏใใใพใใใ
@@ -221,10 +245,19 @@ DATA_ROOT/
## ๐งฌ Model Zoo
-| ใขใใซๅ | ่งฃๅๅบฆ | ็ทใใฉใกใผใฟ(M) | ใใใฏใใผใณ(M) | L-ViT(M) | H-ViT(M) | ๆผ็ฎ้(FLOPs, G) | ่จญๅฎใใกใคใซ |
-| ----- | ---------- | -------------- | ----------- |------------- | ------------- | -------------- | ------- |
-| โก ms_eff_gcvit_b0 | 224 X 224 | 8.7 | 3.6(41.4%) | 1.7(19.5%) | 3.3(37.9%) | 0.87 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
-| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.3 | 27.3(54.3%) | 6.6(13.1%) | 16.1(32.0%) | 13.64 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| ใขใใซๅ | ่งฃๅๅบฆ | ใใฉใกใผใฟ(M) | ๆผ็ฎ้(GFLOPS) | ่จญๅฎใใกใคใซ |
+| ----- | ---------- | -------------- | ----------- | ------- |
+| โก ms_eff_gcvit_b0 | 224 X 224 | 8.7 | 0.85 | [spec](deepguard/config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
+| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.3 | 13.43 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+
+## ๐ Model Inference Latency
+
+ๆจ่ซ้ๅบฆใฏใTesla T4 1ๅบใงใใใใตใคใบ 1ใๅ็ป(num_frames=20)ใๅฏพ่ฑกใซๆธฌๅฎใใฆใใพใใ
+
+| ใขใใซๅ | ใขใผใ | ๅฑ้ๅฏพ่ฑก | ๆจ่ซ้ๅบฆ(s) | ๆฏ็ |
+| ----- | ---- | -------- | ----------- | ----- |
+| โก ms_eff_gcvit_b0 | Fast Mode | ใจใใธ / ใชใณใใใคในๅฑ้ๅใ | 0.051 | 1.00ร |
+| ๐ฅ ms_eff_gcvit_b5 | Pro Mode | ใจใณใฟใผใใฉใคใบๅฑ้ๅใ | 0.479 | 9.41ร |
## ๐ ๅญฆ็ฟ
@@ -238,13 +271,6 @@ pip install -r requirements.txt
`ms_eff_vit` ใจ `ms_eff_gcvit` ใฎไธกๆนใซใคใใฆๅญฆ็ฟในใฏใชใใใๆไพใใพใใ็กๆใฎ GPU ็ฐๅขใจใใฆ **Google Colab** ใใๅฎ้จใฎ่จ้ฒใใใณใใฉใใญใณใฐใจใใฆ **Weights & Biases(W&B)** ใฎไฝฟ็จใๆจๅฅจใใพใใ
-#### ๐ Weight & Biases ๅฎ้จ็ตๆ
-
-* **ms_eff_vit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_kodf)
-* **ms_eff_vit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_kodf)
-* **ms_eff_gcvit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_kodf)
-* **ms_eff_gcvit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_kodf)
-
```python
!python -m train_eff_vit \ # ใพใใฏ train_eff_gcvit
--root-dir DATA_ROOT \
@@ -593,6 +619,9 @@ Jupyterใใผใใใใฏใฏใgitใชใใธใใชใฎtutorialsใใฉใซใๅ
ใซ
| 8 | [`Pytorch-Grad-Cam`](https://github.com/jacobgil/pytorch-grad-cam) | PyTorch ใขใใซใฎใใใฎ้ซๅบฆใช่ฆ่ฆ็่ชฌๆใใผใซ |
| 9 | [`DeepfakeBench`](https://github.com/SCLBD/DeepfakeBench) | SCLBD ใซใใๅ
ๆฌ็ใชใใฃใผใใใงใคใฏๆคๅบใใณใใใผใฏ |
| 10 | [`Effort-AIGI-Detection`](https://github.com/YZY-stack/Effort-AIGI-Detection) | YZY-stack ใซใใ Effort๏ผICML 2025 Oral๏ผใฎๅ
ฌๅผๅฎ่ฃ
|
+| 11 | [`SIA`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/sia_detector.py) | An Information Theoretic Approach for Attention-Driven Face Forgery Detection๏ผECCV 2022๏ผใDeepfakeBench ๅ
ใฎๅฎ่ฃ
|
+| 12 | [`CORE`](https://github.com/niyunsheng/CORE) | niyunsheng ใซใใ CORE: COnsistent REpresentation Learning for Face Forgery Detection๏ผCVPRW 2022๏ผใฎๅ
ฌๅผๅฎ่ฃ
|
+| 13 | [`IID`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/iid_detector.py) | Implicit Identity Driven Deepfake Face Swapping Detection๏ผCVPR 2023๏ผใDeepfakeBench ๅ
ใฎๅฎ่ฃ
|
## โ๏ธ ใฉใคใปใณใน
diff --git a/README_KR.md b/README_KR.md
index decec284..ab3c44a9 100644
--- a/README_KR.md
+++ b/README_KR.md
@@ -25,7 +25,6 @@
-
@@ -44,12 +43,14 @@
## ๐ ๋ชฉ์ฐจ
+- [๐ ํ๋ก์ ํธ ๊ตฌ์กฐ](#-ํ๋ก์ ํธ-๊ตฌ์กฐ) - ๋๋ ํฐ๋ฆฌ ๊ตฌ์กฐ ๊ฐ์ (ํจํค์ง, ์น ์ฑ, ๋ฌธ์)
- [๐ณ Docker Quick Start](#-docker-quick-start) - Docker Compose๋ก ์ ์ฒด ์คํ(MySQL, Redis, FastAPI, Celery, React) ์คํ
- [๐ค ์ง๊ธ ๋ฐ๋ก ์ฒดํํ๊ธฐ: Hugging Face Spaces](#-์ง๊ธ-๋ฐ๋ก-์ฒดํํ๊ธฐ-hugging-face-spaces) - ์ค์น ์์ด ๋ธ๋ผ์ฐ์ ์์ ๋ฐ๋ก ์ฒดํํ๋ ์ด๋ฏธ์ง/๋น๋์ค/XAI ๋ฅํ์ดํฌ ํ์ง ๋ฐ๋ชจ
- [๐ ๋ฅํ์ดํฌ ๋น๋์ค ๋ฒค์น๋งํฌ ๋ฐ์ดํฐ์
](#-๋ฅํ์ดํฌ-๋น๋์ค-๋ฒค์น๋งํฌ-๋ฐ์ดํฐ์
) โ ํ์ต์ ์ฌ์ฉ๋ Celeb-DF-v2, FF++, KoDF ๋ฐ์ดํฐ์
๊ฐ์.
- [โ๏ธ ๋ฐ์ดํฐ ์ค๋น](#๋ฐ์ดํฐ-์ค๋น) โ YOLOv8์ ์ด์ฉํ ํจ์จ์ ์ธ ์ผ๊ตด ๊ฒ์ถ ๋ฐ ๋๋๋งํฌ ์ถ์ถ ํ์ดํ๋ผ์ธ.
- [๐ ๋ชจ๋ธ ๊ตฌ์กฐ](#-๋ชจ๋ธ-๊ตฌ์กฐ) โ ํ์ด๋ธ๋ฆฌ๋ CNN-ViT (MS-EffViT & MS-EffGCViT) ์ค๊ณ ์์ธ.
- [๐งฌ ๋ชจ๋ธ ์ฃผ(Model Zoo)](#-model-zoo) โ ๋ชจ๋ธ ๋ณ์ฒด๋ณ ํ๋ผ๋ฏธํฐ ์ ๋ฐ ์ฐ์ฐ๋(FLOPs) ๋น๊ต.
+- [๐ Model Inference Latency](#-model-inference-latency) โ Tesla T4 1๋ ๊ธฐ์ค Fast(b0) vs Pro(b5) ์ถ๋ก ์๋ ๋น๊ต.
- [๐ ํ์ต](#-ํ์ต) - Google Colab ๋ฐ W&B๋ฅผ ํ์ฉํ ๋จ๊ณ๋ณ ํ์ต ์คํฌ๋ฆฝํธ.
- [๐ ๋ชจ๋ธ ํ๊ฐ](#-๋ชจ๋ธ-ํ๊ฐ) - ๋ฒค์น๋งํฌ ๊ฒฐ๊ณผ.
- [๐ป ๋ชจ๋ธ ์ฌ์ฉ๋ฒ](#-๋ชจ๋ธ-์ฌ์ฉ๋ฒ) - `pip install deepguard` ๋๋ Hugging Face Hub๋ฅผ ํตํ ์ฌ์ ํ์ต ๋ชจ๋ธ ๋ก๋
@@ -62,6 +63,29 @@
---
+## ๐ ํ๋ก์ ํธ ๊ตฌ์กฐ
+
+```
+DeepGuard/
+โโโ deepguard/ # ํต์ฌ ํจํค์ง (๋ชจ๋ธ, ๋ ์ด์ด, ๋ฐ์ดํฐ ํ์ดํ๋ผ์ธ, ์ตํฐ๋ง์ด์ , ์ค์ผ์ค๋ฌ, ์ ํธ)
+โโโ preprocess/ # ์ผ๊ตด ๊ฒ์ถ, ํฌ๋กญ ๋ฐ ๋๋๋งํฌ ์ถ์ถ ํ์ดํ๋ผ์ธ
+โโโ inference/ # ์ด๋ฏธ์ง / ๋น๋์ค ์์ธก
+โโโ explainability/ # ๋์ผ ๋ธ๋์น CAM ๊ธฐ๋ฐ XAI
+โโโ App/ # FastAPI ๋ฐฑ์๋ (PyPI ํจํค์ง์์ ์ ์ธ)
+โโโ client/ # React ํ๋ก ํธ์๋
+โโโ docker/ # MySQL / Redis Docker ์ด๋ฏธ์ง ๋ฐ ์ค์
+โโโ tutorials/ # Colab ๋
ธํธ๋ถ (์์ธก ๋ฐ XAI ์๊ฐํ)
+โโโ docs/ # README ์๋ฃ (์ํคํ
์ฒ, ablation, ๋ฒค์น๋งํฌ, ์ํ, XAI ๊ฒฐ๊ณผ)
+โโโ .github/ # CI ์ํฌํ๋ก ๋ฐ ์ ์ฅ์ ์ค์
+โโโ docker-compose.yml # ์ ์ฒด ์คํ (MySQL, Redis, FastAPI, Celery, React)
+โโโ train_eff_vit.py # MS-EffViT ํ์ต ์ํธ๋ฆฌํฌ์ธํธ
+โโโ train_eff_gcvit.py # MS-EffGCViT ํ์ต ์ํธ๋ฆฌํฌ์ธํธ
+โโโ pyproject.toml # ํจํค์ง ๋น๋ ์ค์
+โโโ requirements.txt # ๊ฐ๋ฐ ํ๊ฒฝ ์์กด์ฑ
+โโโ README.md, README_KR.md, README_JP.md
+โโโ LICENSE
+```
+
## ๐ณ Docker Quick Start
Docker Compose๋ก ์ ์ฒด ์คํ(MySQL + Redis + FastAPI + Celery + React)์ ์คํํฉ๋๋ค โ ๋ก์ปฌ์ Python/Node ํ๊ฒฝ์ ๋ฐ๋ก ์ค์นํ ํ์๊ฐ ์์ต๋๋ค.
@@ -221,10 +245,19 @@ DATA_ROOT/
## ๐งฌ Model Zoo
-| ๋ชจ๋ธ๋ช
| ํด์๋ | ์ด ํ๋ผ๋ฏธํฐ(M) | ๋ฐฑ๋ณธ(M) | L-ViT(M) | H-ViT(M) | ์ฐ์ฐ๋(FLOPs, G) | ์ค์ ํ์ผ |
-| ----- | ---------- | -------------- | ----------- |------------- | ------------- | -------------- | ------- |
-| โก ms_eff_gcvit_b0 | 224 X 224 | 8.7 | 3.6(41.4%) | 1.7(19.5%) | 3.3(37.9%) | 0.87 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
-| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.3 | 27.3(54.3%) | 6.6(13.1%) | 16.1(32.0%) | 13.64 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| ๋ชจ๋ธ๋ช
| ํด์๋ | ํ๋ผ๋ฏธํฐ(M) | ์ฐ์ฐ๋(GFLOPS) | ์ค์ ํ์ผ |
+| ----- | ---------- | -------------- | ----------- | ------- |
+| โก ms_eff_gcvit_b0 | 224 X 224 | 8.70 | 0.85 | [spec](deepguard/config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
+| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.34 | 13.43 | [spec](deepguard/config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+
+## ๐ Model Inference Latency
+
+์ถ๋ก ์๋๋ Tesla T4 GPU 1๋์์ ๋ฐฐ์น ํฌ๊ธฐ 1, ๋น๋์ค(num_frames=20) ๊ธฐ์ค์ผ๋ก ์ธก์ ํ์ต๋๋ค.
+
+| ๋ชจ๋ธ๋ช
| ๋ชจ๋ | ๋ฐฐํฌ ๋์ | ์ถ๋ก ์๋(s) | ๋น์จ |
+| ----- | ---- | -------- | ----------- | ----- |
+| โก ms_eff_gcvit_b0 | Fast Mode | ์ฃ์ง / ์จ๋๋ฐ์ด์ค ๋ฐฐํฌ์ฉ | 0.051 | 1.00ร |
+| ๐ฅ ms_eff_gcvit_b5 | Pro Mode | ์ํฐํ๋ผ์ด์ฆ ๋ฐฐํฌ์ฉ | 0.479 | 9.41ร |
## ๐ ํ์ต
@@ -238,13 +271,6 @@ pip install -r requirements.txt
`ms_eff_vit` ๋ฐ `ms_eff_gcvit` ๋ชจ๋์ ๋ํ ํ์ต ์คํฌ๋ฆฝํธ๋ฅผ ์ ๊ณตํฉ๋๋ค. ๋ฌด๋ฃ GPU ํ๊ฒฝ์ ์ํด **Google Colab**์, ์คํ ๊ธฐ๋ก ๋ฐ ํธ๋ํน์ ์ํด **Weights & Biases(W&B)** ์ฌ์ฉ์ ๊ถ์ฅํฉ๋๋ค.
-#### ๐ Weight & Biases ์คํ ๊ฒฐ๊ณผ
-
-* **ms_eff_vit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b0_kodf)
-* **ms_eff_vit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_vit_b5_kodf)
-* **ms_eff_gcvit_b0:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b0_kodf)
-* **ms_eff_gcvit_b5:** [Celeb-DF-v2 ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_celeb_df_v2) | [FaceForensics++ ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_ff++) | [KoDF ๐](https://wandb.ai/origin6165/ms_eff_gcvit_b5_kodf)
-
```python
!python -m train_eff_vit \ # ๋๋ train_eff_gcvit
--root-dir DATA_ROOT \
@@ -593,6 +619,9 @@ result = explainer.display_heatmap_bbox_on_image(
| 8 | [`Pytorch-Grad-Cam`](https://github.com/jacobgil/pytorch-grad-cam) | PyTorch ๋ชจ๋ธ์ ์ํ ๊ณ ๊ธ ์๊ฐ์ ์ค๋ช
๋๊ตฌ |
| 9 | [`DeepfakeBench`](https://github.com/SCLBD/DeepfakeBench) | SCLBD์ ํฌ๊ด์ ์ธ ๋ฅํ์ดํฌ ํ์ง ๋ฒค์น๋งํฌ |
| 10 | [`Effort-AIGI-Detection`](https://github.com/YZY-stack/Effort-AIGI-Detection) | YZY-stack์ Effort(ICML 2025 Oral) ๊ณต์ ๊ตฌํ์ฒด |
+| 11 | [`SIA`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/sia_detector.py) | An Information Theoretic Approach for Attention-Driven Face Forgery Detection(ECCV 2022), DeepfakeBench ๋ด ๊ตฌํ |
+| 12 | [`CORE`](https://github.com/niyunsheng/CORE) | niyunsheng์ CORE: COnsistent REpresentation Learning for Face Forgery Detection(CVPRW 2022) ๊ณต์ ๊ตฌํ์ฒด |
+| 13 | [`IID`](https://github.com/SCLBD/DeepfakeBench/blob/main/training/detectors/iid_detector.py) | Implicit Identity Driven Deepfake Face Swapping Detection(CVPR 2023), DeepfakeBench ๋ด ๊ตฌํ |
## โ๏ธ ๋ผ์ด์ ์ค
diff --git a/deepguard/MS_EffGCViT.md b/deepguard/MS_EffGCViT.md
index 169740e9..3c064367 100644
--- a/deepguard/MS_EffGCViT.md
+++ b/deepguard/MS_EffGCViT.md
@@ -15,7 +15,7 @@ This Repository presents the PyTorch implementation of **Multi Scale Efficient G
This model is a **frame-level** and **spatial-domain** architecture, designed to perform classification tasks on both **static images** and **video sequences**
-
+
## ๐ฅ News ๐ฅ
- [**09.05.2026**] ๐ฅ Released **HIGH-Eff-GCVIT-B5** (384X384) weights for **Celeb DF(V2)**, **FaceForensics++**, **KoDF**
@@ -32,36 +32,42 @@ This model is a **frame-level** and **spatial-domain** architecture, designed to
**MS-EFF-GCViT achieves state-of-the-art (SOTA) results across three DeepFake benchmarks.**
The model ships in two variants from a single architecture โ **Fast (b0)** for real-time / edge
-deployment and **Pro (b5)** for enterprise-grade accuracy. Notably, **Fast** matches or exceeds
-much larger SOTA models while using a fraction of the parameters and compute.
+deployment and **Pro (b5)** for enterprise-grade accuracy.
-
-
-
+**๐ Model Performance Results on Celeb-DF (v2) Dataset**
-> On **Celeb-DF(v2)**, Pro reaches **0.9981 Acc** (rank #1) and Fast **0.9842** (rank #3) among 20 architectures.
-> On the **KoDF competition** leaderboard, Pro ranks **#1** and Fast **#4** out of 49 entries.
+| Model | Test@Acc | Test@Auc | #Params(M) | Params Ratio | #FLOPS(G) | FLOPS Ratio |
+| --- | --- | --- | --- | --- | --- | --- |
+| **ms_eff_gcvit_b0** | 98.4% | 99.7% | 8.7 | 1x | 0.85 | 1x |
+| sia (eccv,2022) | 99.0% | 99.8% | 17.57 | 2x | 2.05 | 2.4x |
+| core (cvpr,2022) | 99.6% | 99.7% | 20.81 | 2.4x | 8.40 | 9.9x |
+| **ms_eff_gcvit_b5** | **99.8%** | 99.8% | 50.34 | 5.8x | 13.43 | 15.8x |
+| iid (cvpr,2023) | **99.8%** | **99.9%** | 65.5 | 7.5x | 12.7 | 14.9x |
+| effort (icml, 2025) | **99.8%** | 99.7% | 303.4 | 34.9x | 155.9 | 183.4x |
-
-๐ Celeb-DF (v2) โ Accuracy & Efficiency
-
-
-
+**๐ Model Performance Results on FaceForensics++ Dataset**
-
-๐ FaceForensics++ โ Accuracy & Efficiency
-
-
+| Model | Test@Acc | Test@Auc | #Params(M) | Params Ratio | #FLOPS(G) | FLOPS Ratio |
+| --- | --- | --- | --- | --- | --- | --- |
+| **ms_eff_gcvit_b0** | 98.1% | 99.7% | 8.7 | 1x | 0.85 | 1x |
+| sia (ECCV,2022) | 95.0% | 97.5% | 17.57 | 2x | 2.05 | 2.4x |
+| core (CVPR,2022) | 98.3% | **99.8%** | 20.81 | 2.4x | 8.40 | 9.9x |
+| **ms_eff_gcvit_b5** | **98.5%** | 99.7% | 50.34 | 5.8x | 13.43 | 15.8x |
+| iid (CVPR,2023) | 97.3% | 99.6% | 65.5 | 7.5x | 12.7 | 14.9x |
+| effort (ICML, 2025) | 98.4% | **99.8%** | 303.4 | 34.9x | 155.9 | 183.4x |
-
+**๐ Model Performance Results on KoDF Dataset**
-
-๐ KoDF Competition โ Accuracy Ranking
-
-
+| Model | Test@Acc | Test@Auc | #Params(M) | Params Ratio | #FLOPS(G) | FLOPS Ratio |
+| --- | --- | --- | --- | --- | --- | --- |
+| **ms_eff_gcvit_b0** | 96.6% | 97.9% | 8.7 | 1x | 0.85 | 1x |
+| sia (ECCV,2022) | 97.0% | 98.8% | 17.57 | 2x | 2.05 | 2.4x |
+| core (CVPR,2022) | 96.3% | 99.2% | 20.81 | 2.4x | 8.40 | 9.9x |
+| **ms_eff_gcvit_b5** | 97.9% | 98.3% | 50.34 | 5.8x | 13.43 | 15.8x |
+| iid (CVPR,2023) | 96.5% | 97.6% | 65.5 | 7.5x | 12.7 | 14.9x |
+| effort (ICML, 2025) | **99.0%** | **99.5%** | 303.4 | 34.9x | 155.9 | 183.4x |
-
## Model Indroduction
@@ -109,28 +115,20 @@ Modern DeepFakes can leave very localized forgery region. To Capture this, we ad
## ๐ Model Zoo
-| Model | Resolution | # Total Params(M) | # Backbone(M) | # L-ViT(M) | # H-ViT(M) | FLOPs (G) | Model Config |
-| ----- | ---------- | -------------- | ----------- |------------- | ------------- | -------------- | ------- |
-| โก ms_eff_gcvit_b0 | 224 X 224 | 8.7 | 3.6(41.4%) | 1.7(19.5%) | 3.3(37.9%) | 0.87 | [spec](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
-| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.3 | 27.3(54.3%) | 6.6(13.1%) | 16.1(32.0%) | 13.64 | [spec](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| Model | Resolution | # Params(M) | # FLOPS(G) | Model Config |
+| ----- | ---------- | -------------- | ----------- | ------- |
+| โก ms_eff_gcvit_b0 | 224 X 224 | 8.70 | 0.85 | [spec](./config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
+| ๐ฅ ms_eff_gcvit_b5 | 384 X 384 | 50.34 | 13.43 | [spec](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
-## ๐ Model Variants
+## ๐ Model Inference Latency
-**โก ms_eff_gcvit_b0 (Fast Mode / Mobile)**: Efficiency at the Edge
-- Optimized for **real-time inference** and mobile deployment.
+Latency is measured with batch size 1 on a single Tesla T4 and video (num_frames=20).
-**๐ฅ ms_eff_gcvit_b5 (Pro Mode / Enterprise)**: Uncompromising Precision
-- Engineered for high-fidelity analysis and enterprise-grade accuracy.
-
-## โ๏ธ Model Weight Initialization
-The model incorporates a hybrid initialization strategy to leverage pre-trained features while ensuring stable convergence of the transformer components
-
-> **Backbone**: **ImageNet-1K Pretraiend Weights**(EfficientNet)
-
-> **L-GCViT / H-GCViT / Head**: Truncated Normal( `std=0.02` )
-
-> **No Weight Decay**: `relative_position_bias_table`, `bn`, `norm`
+| Model | Mode | Target Deployment | Latency (s) | Ratio |
+| ----- | ---- | ----------------- | ----------- | ----- |
+| โก ms_eff_gcvit_b0 | Fast Mode | Edge / on-device deployment | 0.051 | 1.00ร |
+| ๐ฅ ms_eff_gcvit_b5 | Pro Mode | Enterprise deployment | 0.479 | 9.41ร |
## DeepFake Video Benchmarks
@@ -143,24 +141,24 @@ The model incorporates a hybrid initialization strategy to leverage pre-trained
**Celeb DF(v2) Pretrained Models**
-| Model Variant | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| Model | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
| ------------- | -------- | -------- | ---------- | -------- | ----- |
| ms_eff_gcvit_b0 | 0.9842 | 0.9965 | 0.0283 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b0_celeb_df_v2.bin) | [recipe](./config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
| ms_eff_gcvit_b5 | 0.9981 | 0.9984 | 0.0089 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b5_celeb_df_v2.bin) | [recipe](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
**FaceForensics++ Pretrained Models**
-| Model Variant | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| Model | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
| ------------- | -------- | -------- | ---------- | -------- | ------ |
-| ms_eff_gcvit_b0 | 0.9808 | 0.9969 | 0.0637| [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b0_ff++.bin) | [recipe](./config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
-| ms_eff_gcvit_b5 | 0.9850 | 0.9974 | 0.0492 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b5_ff++.bin) | [recipe](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| ms_eff_gcvit_b0 | 0.9808 | 0.9969 | 0.0637| [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b0_ff++.bin) | [recipe](./config/ms_eff_gcvit_b0/ff++.yaml) |
+| ms_eff_gcvit_b5 | 0.9850 | 0.9974 | 0.0492 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.1.0/ms_eff_gcvit_b5_ff++.bin) | [recipe](./config/ms_eff_gcvit_b5/ff++.yaml) |
**KoDF Pretrained Models**
-| Model Variant | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| Model | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
| ------------- | -------- | -------- | ---------- | -------- | ------ |
-| ms_eff_gcvit_b0 | 0.9655 | 0.9792 | 0.1237 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.2.0/ms_eff_gcvit_b0_kodf.bin) | [recipe](./config/ms_eff_gcvit_b0/celeb_df_v2.yaml) |
-| ms_eff_gcvit_b5 | 0.9850 | 0.9974 | 0.0492 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.2.0/ms_eff_gcvit_b5_kodf.bin) | [recipe](./config/ms_eff_gcvit_b5/celeb_df_v2.yaml) |
+| ms_eff_gcvit_b0 | 0.9655 | 0.9792 | 0.1237 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.2.0/ms_eff_gcvit_b0_kodf.bin) | [recipe](./config/ms_eff_gcvit_b0/kodf.yaml) |
+| ms_eff_gcvit_b5 | 0.9792 | 0.9831 | 0.0692 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v0.2.0/ms_eff_gcvit_b5_kodf.bin) | [recipe](./config/ms_eff_gcvit_b5/kodf.yaml) |
## Usage
diff --git a/docs/ablation/ablation_celeb_df_v2.webp b/docs/ablation/ablation_celeb_df_v2.webp
new file mode 100644
index 00000000..2dde6eca
Binary files /dev/null and b/docs/ablation/ablation_celeb_df_v2.webp differ
diff --git a/docs/ablation/ablation_feature_map.png b/docs/ablation/ablation_feature_map.png
new file mode 100644
index 00000000..53424f32
Binary files /dev/null and b/docs/ablation/ablation_feature_map.png differ
diff --git a/docs/ablation/ablation_ff++.webp b/docs/ablation/ablation_ff++.webp
new file mode 100644
index 00000000..081ec75f
Binary files /dev/null and b/docs/ablation/ablation_ff++.webp differ
diff --git a/docs/ablation/ablation_kodf.webp b/docs/ablation/ablation_kodf.webp
new file mode 100644
index 00000000..26bf888e
Binary files /dev/null and b/docs/ablation/ablation_kodf.webp differ
diff --git a/docs/benchmarks/celeb_df_v2_gcvit.png b/docs/benchmarks/celeb_df_v2_gcvit.png
deleted file mode 100644
index 77190479..00000000
Binary files a/docs/benchmarks/celeb_df_v2_gcvit.png and /dev/null differ
diff --git a/docs/benchmarks/celeb_df_v2_gcvit_2.png b/docs/benchmarks/celeb_df_v2_gcvit_2.png
deleted file mode 100644
index 5cc437f2..00000000
Binary files a/docs/benchmarks/celeb_df_v2_gcvit_2.png and /dev/null differ
diff --git a/docs/benchmarks/ff_gcvit.png b/docs/benchmarks/ff_gcvit.png
deleted file mode 100644
index 9a70a73b..00000000
Binary files a/docs/benchmarks/ff_gcvit.png and /dev/null differ
diff --git a/docs/benchmarks/gcvit_params_acc.png b/docs/benchmarks/gcvit_params_acc.png
new file mode 100644
index 00000000..a1095322
Binary files /dev/null and b/docs/benchmarks/gcvit_params_acc.png differ
diff --git a/docs/benchmarks/gcvit_summary_bars.png b/docs/benchmarks/gcvit_summary_bars.png
deleted file mode 100644
index ee8d30b4..00000000
Binary files a/docs/benchmarks/gcvit_summary_bars.png and /dev/null differ
diff --git a/docs/benchmarks/kodf_gcvit.png b/docs/benchmarks/kodf_gcvit.png
deleted file mode 100644
index d163b389..00000000
Binary files a/docs/benchmarks/kodf_gcvit.png and /dev/null differ
diff --git a/explainability/explainer/base_explainer.py b/explainability/explainer/base_explainer.py
index bb815106..da4e93b4 100644
--- a/explainability/explainer/base_explainer.py
+++ b/explainability/explainer/base_explainer.py
@@ -11,11 +11,17 @@
class BaseExplainer:
def __init__(
self,
- model_name: Literal["ms_eff_vit_b0", "ms_eff_vit_b5", "ms_eff_gcvit_b0", "ms_eff_gcvit_b5"],
+ model_name: Literal[
+ "ms_eff_vit_b0", "ms_eff_vit_b5", "ms_eff_gcvit_b0", "ms_eff_gcvit_b5",
+ # ablation models
+ "eff_b0", "eff_b5",
+ "low_eff_gcvit_b0", "low_eff_gcvit_b5",
+ "high_eff_gcvit_b0", "high_eff_gcvit_b5",
+ ],
dataset: Literal["celeb_df_v2", "ff++", "kodf"],
margin_ratio: float = 0.2,
conf_thres: float = 0.5,
- branch_level: Literal["low", "high"] = "high",
+ branch_level: Literal["low", "high", "both"] = "high", # "both"๋ ms_eff_* ๋ชจ๋ธ๋ง ๊ฐ๋ฅ
l_stage_idx: int = -1, # gcvit low branch stage index (0~3)
block_idx: int = -1, # vit, gcvit block index
model: Optional[torch.nn.Module] = None,
@@ -32,8 +38,15 @@ def __init__(
# Parse Model Metadata
self.model_variant = model_name.split("_")[-1] # b0, b5
- self.transformer_type = model_name.split("_")[-2] # vit, gcvit
-
+ # vit, gcvit, cnn (eff_b0/eff_b5: transformer ์๋ EfficientNet baseline)
+ if "gcvit" in model_name:
+ self.transformer_type = "gcvit"
+ elif "vit" in model_name:
+ self.transformer_type = "vit"
+ else:
+ self.transformer_type = "cnn"
+ self._validate_branch_level()
+
# Model & Tools setup
self.model = model if model is not None else timm.create_model(
model_name, pretrained=True, dataset=dataset
@@ -54,8 +67,34 @@ def _get_reshape_fn(self) -> Callable[[torch.Tensor], torch.Tensor]:
Returns:
Callable: 3D/4D ํ
์๋ฅผ GradCAM ์ฐ์ฐ์ ์ ํฉํ ํํ๋ก ๋ณํํ๋ ํจ์.
"""
- return reshape_transform_vit if self.transformer_type == "vit" else reshape_transform_gcvit
-
+ if self.transformer_type == "cnn":
+ return None # backbone feature map์ ์ด๋ฏธ (B,C,H,W)
+ return reshape_transform_vit if self.transformer_type == "vit" else reshape_transform_gcvit
+
+ def _validate_branch_level(self) -> None:
+ """Ablation ๋ชจ๋ธ์ด ์ค์ ๋ก ๊ฐ์ง branch์ branch_level์ด ์ผ์นํ๋์ง ๊ฒ์ฆํ๋ค.
+
+ Raises:
+ ValueError: low_/high_ ๋จ์ผ branch ๋ชจ๋ธ์ ์กด์ฌํ์ง ์๋ branch๋ฅผ ์์ฒญํ ๊ฒฝ์ฐ ๋ฐ์.
+ """
+ if self.transformer_type == "cnn" and self.branch_level != "high":
+ raise ValueError(f"{self.model_name}์ ๋ง์ง๋ง feature map๋ง ์ฌ์ฉํ๋ฏ๋ก branch_level='high'๋ง ๊ฐ๋ฅํฉ๋๋ค.")
+ if self.model_name.startswith("low_") and self.branch_level != "low":
+ raise ValueError(f"{self.model_name}์ low-level branch๋ง ๊ฐ์ง๋ฏ๋ก branch_level='low'๋ง ๊ฐ๋ฅํฉ๋๋ค.")
+ if self.model_name.startswith("high_") and self.branch_level != "high":
+ raise ValueError(f"{self.model_name}์ high-level branch๋ง ๊ฐ์ง๋ฏ๋ก branch_level='high'๋ง ๊ฐ๋ฅํฉ๋๋ค.")
+
+ def _get_backbone_layer(self, feature_idx: int) -> torch.nn.Module:
+ """EfficientNet backbone์์ feature_idx ๋จ๊ณ์ ์ถ๋ ฅ ๋ชจ๋์ ์ฐพ๋๋ค."""
+ backbone = self.model.feat_extractor.backbone
+ name = backbone.feature_info[feature_idx]["module"] # e.g. "blocks.6"
+ modules = dict(backbone.named_modules())
+ # timm FeatureListNet์ ์ด๋ฆ์ '.'๋ฅผ '_'๋ก ๋ฐ๊พธ๋ ๊ฒฝ์ฐ๊ฐ ์์ด ๋ ๋ค ์๋
+ for key in (name, name.replace(".", "_")):
+ if key in modules:
+ return modules[key]
+ raise ValueError(f"backbone์์ '{name}' ๋ชจ๋์ ์ฐพ์ ์ ์์ต๋๋ค.")
+
def _resolve_target_layers(self) -> list[torch.nn.Module]:
"""์๊ฐํ(XAI)์ ๋์์ด ๋ Target Layer๋ค์ ์ ํํ๋ค.
@@ -67,14 +106,18 @@ def _resolve_target_layers(self) -> list[torch.nn.Module]:
"""
layers = []
- if self.transformer_type == "gcvit":
- n_stages = len(self.model.l_gcvit.stages)
- stage_idx = self.l_stage_idx if self.l_stage_idx >= 0 else n_stages + self.l_stage_idx
-
- if not (0 <= stage_idx < n_stages):
- raise ValueError(f"l_stage_idx={self.l_stage_idx} ๋ฒ์ ์ด๊ณผ (stage ์: {n_stages})")
+ if self.transformer_type == "cnn":
+ # eff_b0/eff_b5: transformer๊ฐ ์์ผ๋ฏ๋ก backbone์ ๋ง์ง๋ง feature map์ ์ง์ ์ฌ์ฉ
+ layers.append(self._get_backbone_layer(self.model.h_meta["feature_idx"]))
+ elif self.transformer_type == "gcvit":
if self.branch_level in ("low", "both"):
+ n_stages = len(self.model.l_gcvit.stages)
+ stage_idx = self.l_stage_idx if self.l_stage_idx >= 0 else n_stages + self.l_stage_idx
+
+ if not (0 <= stage_idx < n_stages):
+ raise ValueError(f"l_stage_idx={self.l_stage_idx} ๋ฒ์ ์ด๊ณผ (stage ์: {n_stages})")
+
n_blocks = len(self.model.l_gcvit.stages[stage_idx].blocks)
blk = self._resolve_block_idx(self.block_idx, n_blocks, f"l_gcvit stage[{stage_idx}]")
layers.append(self.model.l_gcvit.stages[stage_idx].blocks[blk].norm2)
diff --git a/inference/Ablation.md b/inference/Ablation.md
new file mode 100644
index 00000000..b5788f39
--- /dev/null
+++ b/inference/Ablation.md
@@ -0,0 +1,78 @@
+# Ablation Study
+
+
+
+
+
+- ** (_Subtle Artifacts_)**: High-Resolution feature maps are extracted from early backbone blocks(`l_block_idx`) to capture like skin texture or boundary artifacts
+
+- ** (_Global Features_)**: Low-Resolution feature maps are extracted from deeper blocks(`h_block_idx`) to analyze overall lighting, shadows, and structural consistency.
+
+- **Feature Fusion**: The Outputs from both branches (`L-GCViT and H-GCViT`) are fused to make a comprehensive decision based on both local and global context.
+
+## Ablation Model Zoo
+
+| Model | Branch | Resolution | # Params(M) | #FLOPs(G) |
+| :--- | :---: | :---: | :---: | :---: |
+| eff_b0 |  | 224 ร 224 | 3.65M | 0.76G |
+| eff_b5 |  | 384 ร 384 | 27.42M | 13.79G |
+| low_eff_gcvit_b0 |  | 224 ร 224 | 5.30M | 0.79G |
+| low_eff_gcvit_b5 |  | 384 ร 384 | 33.92M | 10.70G |
+| high_eff_gcvit_b0 |  | 224 ร 224 | 6.94M | 1.08G |
+| high_eff_gcvit_b5 |  | 384 ร 384 | 43.51M | 18.59G |
+
+## Ablation Feature map
+
+
+
+
+
+# Ablation Evaluation
+
+### Celeb DF(V2)
+
+
+
+
+
+| Model | Branch | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| ------------- | -------- | -------- | -------- | ---------- | -------- | ----- |
+| eff_b0 |  | 0.9643 | 0.9754 | 0.0977 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b0_celeb_df_v2.bin) | [recipe](../deepguard/config/eff_b0/celeb_df_v2.yaml) |
+| eff_b5 |  | 0.9768 | 0.9837 | 0.0676 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b5_celeb_df_v2.bin) | [recipe](../deepguard/config/eff_b5/celeb_df_v2.yaml) |
+| low_eff_gcvit_b0 |  | 0.9788 | 0.9837 | 0.0658 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b0_celeb_df_v2.bin) | [recipe](../deepguard/config/low_eff_gcvit_b0/celeb_df_v2.yaml) |
+| low_eff_gcvit_b5 |  | 0.9865 | 0.9987 | 0.0150 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b5_celeb_df_v2.bin) | [recipe](../deepguard/config/low_eff_gcvit_b5/celeb_df_v2.yaml) |
+| high_eff_gcvit_b0 |  | 0.9807 | 0.9884 | 0.0343 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b0_celeb_df_v2.bin) | [recipe](../deepguard/config/high_eff_gcvit_b0/celeb_df_v2.yaml) |
+| high_eff_gcvit_b5 |  | 0.9942 | 0.9977 | 0.0101 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b5_celeb_df_v2.bin) | [recipe](../deepguard/config/high_eff_gcvit_b5/celeb_df_v2.yaml) |
+
+
+### FaceForensics++
+
+
+
+
+
+| Model | Branch | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| ------------- | -------- | -------- | -------- | ---------- | -------- | ------ |
+| eff_b0 |  | 0.9202 | 0.9497 | 0.1523 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b0_ff++.bin) | [recipe](../deepguard/config/eff_b0/ff++.yaml) |
+| eff_b5 |  | 0.9660 | 0.9831 | 0.0842 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b5_ff++.bin) | [recipe](../deepguard/config/eff_b5/ff++.yaml) |
+| low_eff_gcvit_b0 |  | 0.8033 | 0.9257 | 0.4148 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b0_ff++.bin) | [recipe](../deepguard/config/low_eff_gcvit_b0/ff++.yaml) |
+| low_eff_gcvit_b5 |  | 0.8700 | 0.9486 | 0.3365 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b5_ff++.bin) | [recipe](../deepguard/config/low_eff_gcvit_b5/ff++.yaml) |
+| high_eff_gcvit_b0 |  | 0.9783 | 0.9886 | 0.0767 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b0_ff++.bin) | [recipe](../deepguard/config/high_eff_gcvit_b0/ff++.yaml) |
+| high_eff_gcvit_b5 |  | 0.9808 | 0.9956 | 0.0606 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b5_ff++.bin) | [recipe](../deepguard/config/high_eff_gcvit_b5/ff++.yaml) |
+
+
+### Kodf
+
+
+
+
+
+| Model | Branch | Test@Acc | Test@Auc | Test@log_loss | Download | Train Config |
+| ------------- | -------- | -------- | -------- | ---------- | -------- | ------ |
+| eff_b0 |  | 0.9180 | 0.9520 | 0.2010 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b0_kodf.bin) | [recipe](../deepguard/config/eff_b0/kodf.yaml) |
+| eff_b5 |  | 0.9601 | 0.9770 | 0.1120 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/eff_b5_kodf.bin) | [recipe](../deepguard/config/eff_b5/kodf.yaml) |
+| low_eff_gcvit_b0 |  | 0.8684 | 0.9481 | 0.2843 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b0_kodf.bin) | [recipe](../deepguard/config/low_eff_gcvit_b0/kodf.yaml) |
+| low_eff_gcvit_b5 |  | 0.8808 | 0.9596 | 0.2381 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/low_eff_gcvit_b5_kodf.bin) | [recipe](../deepguard/config/low_eff_gcvit_b5/kodf.yaml) |
+| high_eff_gcvit_b0 |  | 0.9462 | 0.9696 | 0.1761 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b0_kodf.bin) | [recipe](../deepguard/config/high_eff_gcvit_b0/kodf.yaml) |
+| high_eff_gcvit_b5 |  | 0.9745 | 0.9911 | 0.0844 | [model](https://github.com/HanMoonSub/DeepGuard/releases/download/v1.1.0/high_eff_gcvit_b5_kodf.bin) | [recipe](../deepguard/config/high_eff_gcvit_b5/kodf.yaml) |
+
diff --git a/inference/README.md b/inference/README.md
index 2ee59dd9..6edf569b 100644
--- a/inference/README.md
+++ b/inference/README.md
@@ -1,23 +1,29 @@
-# ๐ต๏ธโโ๏ธ Inference Pipeline & Strategy
-The inference pipeline of this project is designed for real-world deepfake detection scenarios. Our strategy focuses on **capturing subtle forgery artifacts while ensuring highly stable and reliable predictions.**
+# Inference Strategy
-## ๐ฏ Label
+## Label
Label Mapping: **`Fake: 1`** | **`Real: 0`**
-In the deepfake detection domain, misclassifying a fake video as real (False Negative) poses a significantly greater threat than misclassifying a real video as fake (False Positive).
-
## Test Time Augmentation(TTA)
-Deepfake artifacts often consist of subtle, pixel-level manipulations. Applying heavy augmentation can inadvertently destroy these crucial forensic clues. So We applied a **`simple Horizontal Flip (p=0.5).`**
+Deepfake artifacts often consist of subtle, pixel-level manipulations. Applying heavy augmentation can inadvertently destroy these crucial forensic clues. So We applied a **`simple Horizontal Flip (p=0.5).`** TTA is controlled by `tta_hflip` in `get_test_transforms` (`deepguard/data/transforms.py`):
+
+```python
+from deepguard.data import get_test_transforms
+
+# Horizontal Flip TTA (p=0.5)
+transforms = get_test_transforms(img_size=[224, 224], tta_hflip=0.5)
+```
+
+Internally, `get_test_transforms` resizes the image, appends `A.HorizontalFlip(p=tta_hflip)` only when `tta_hflip != 0`, and then applies normalization.
## Number of Frames & Face Crop
-To ensure consistent and fair evaluation across diverse benchmarks (including Celeb-DF-v2, KoDF, and FaceForensics++), we standardized the frame extraction and face cropping parameters during inference:
+To ensure consistent and fair evaluation across diverse benchmarks (including **_Celeb-DF-v2_**, **_KoDF_**, and **_FaceForensics++_**), we standardized the frame extraction and face cropping parameters during inference:
-- **`Fixed Number of Frames`**: We uniformly sample exactly `20 frames` per video. This guarantees a balanced computational load while maintaining sufficient temporal coverage across all datasets.
+**`Fixed Number of Frames`**: We uniformly sample exactly **`20 frames`** per video. This guarantees a balanced computational load while maintaining sufficient temporal coverage across all datasets.
-- **`Cropping Margin Ratio`**: A `20% margin` (margin_ratio=0.2) is applied around the detected face bounding box. This optimal margin ensures that blending boundaries and facial contoursโcrucial areas for deepfake artifactsโare fully preserved without including excessive background noise.
+**`Cropping Margin Ratio`**: A **`20% margin`** (margin_ratio=0.2) is applied around the detected face bounding box. This optimal margin ensures that blending boundaries and facial contoursโcrucial areas for deepfake artifactsโare fully preserved without including excessive background noise.
## Frame Aggregation Strategy
@@ -55,38 +61,47 @@ def heuristic_aggregation(preds: np.ndarray) -> float:
else:
return float(np.mean(preds))
```
-## ๐ DeepFake Video Benchmark Datasets
-To evaluate the generalization and robustness of our deepfake detection model, we utilize three large-scale, widely recognized benchmark datasets. Each dataset presents unique challenges and covers different types of forgery methods.
-| Dataset | Real Videos | Fake Videos | Year | Participants | Description (Paper Title) | Details |
-| :--- | :---: | :---: | :---: | :---: | :--- | :---: |
-| **Celeb-DF-v2** | 890 | 5,639 | 2019 | 59 | *A Large-scale Challenging Dataset for DeepFake Forensics* | [๐ Readme](../preprocess/celeb_df_v2/README.md) |
-| **FaceForensics++** | 1,000 | 6,000 | 2019 | 1,000 | *Learning to Detect Manipulated Facial Images* | [๐ Readme](../preprocess/ff++/README.md) |
-| **KoDF** | 62,166 | 175,776 | 2020 | 400 | *Large-Scale Korean Deepfake Detection Dataset* | [๐ Readme](../preprocess/kodf/README.md) |
+# In-domain Evaluation
-> **Note:** For detailed information on data preprocessing, frame extraction, and formatting for each dataset, please refer to the respective `Readme` links in the table above.
+Each model is trained and evaluated on the same dataset: it is tested on the held-out test split of the dataset it was trained on.
-## ๐ฌ Celeb DF(V2) Video Evaluation
-
-
-โ
Test Data Evaluation
+### Celeb-DF-v2
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_vit_b0** | 20 | conf | 0 | 0.9742 | 0.9877 | 0.0625 |
| **ms_eff_vit_b5** | 20 | conf | 0 | 0.9900 | 0.9900 | 0.0408 |
| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.9842 | 0.9965 | 0.0283 |
| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.9981** | **0.9984** | **0.0089** |
-
+### FaceForensics++
+
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **ms_eff_vit_b0** | 20 | conf | 0 | 0.9517 | 0.9860 | 0.1334 |
+| **ms_eff_vit_b5** | 20 | conf | 0 | 0.9842 | **0.9977** | **0.0477** |
+| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.9808 | 0.9969 | 0.0637 |
+| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.9850** | 0.9974 | 0.0492 |
+
+### KoDF
+
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.9655 | 0.9792 | 0.1237 |
+| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.9792** | **0.9831** | **0.0692** |
-
-๐ Cross Evaluation
-#### ๐ Dataset: `celeb_df_v2` | ๐ง Model: `ff++`
+# Cross-domain Evaluation
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+Each model is trained on one dataset and evaluated on a different, unseen dataset. Each table is titled `Train โ Test`: the dataset the weights were trained on, followed by the dataset used for evaluation.
+
+### Celeb-DF-v2
+
+### Train: `ff++` โ Test: `celeb_df_v2`
+
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_vit_b0** | 20 | conf | 0 | 0.7317 | 0.6949 | 0.6816 |
| **ms_eff_vit_b5** | 20 | conf | 0 | 0.7143 | 0.7266 | 0.7662 |
@@ -94,34 +109,18 @@ To evaluate the generalization and robustness of our deepfake detection model, w
| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.7722** | **0.7309** | **0.6657** |
-#### ๐ Dataset: `celeb_df_v2` | ๐ง Model: `kodf`
+### Train: `kodf` โ Test: `celeb_df_v2`
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.5579 | 0.4719 | 1.2605 |
| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.5946** | **0.5400** | **1.0078** |
-
-
-## ๐ฌ FaceForensics++ Video Evaluation
-
-
-โ
Test Data Evaluation
-
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
-| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
-| **ms_eff_vit_b0** | 20 | conf | 0 | 0.9517 | 0.9860 | 0.1334 |
-| **ms_eff_vit_b5** | 20 | conf | 0 | 0.9842 | **0.9977** | **0.0477** |
-| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.9808 | 0.9969 | 0.0637 |
-| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.9850** | 0.9974 | 0.0492 |
-
+### FaceForensics++
-
-๐ Cross Evaluation
+### Train: `celeb_df_v2` โ Test: `ff++`
-#### ๐ Dataset: `ff++` | ๐ง Model: `celeb_df_v2`
-
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_vit_b0** | 20 | conf | 0 | 0.5241 | 0.6860 | 1.1241 |
| **ms_eff_vit_b5** | 20 | conf | 0 | 0.5533 | 0.7299 | 0.9987 |
@@ -129,94 +128,45 @@ To evaluate the generalization and robustness of our deepfake detection model, w
| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.5825** | **0.7307** | **0.8897** |
-#### ๐ Dataset: `ff++` | ๐ง Model: `kodf`
+### Train: `kodf` โ Test: `ff++`
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_gcvit_b0** | 20 | conf | 0 | **0.4875** | 0.5341 | 1.6178 |
| **ms_eff_gcvit_b5** | 20 | conf | 0 | 0.4525 | **0.5902** | **1.5321** |
-
-
-## ๐ฌ Kodf Video Evaluation
-
-โ
Test Data Evaluation
-
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
-| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
-| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.9655 | 0.9792 | 0.1237 |
-| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.9792** | **0.9831** | **0.0692** |
+### KoDF
-
+### Train: `celeb_df_v2` โ Test: `kodf`
-
-๐ Cross Evaluation
-
-#### ๐ Dataset: `kodf` | ๐ง Model: `celeb_df_v2`
-
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_gcvit_b0** | 20 | conf | 0 | **0.4935** | **0.7258** | **1.3459** |
| **ms_eff_gcvit_b5** | 20 | conf | 0 | 0.4832 | 0.7160 | 1.4897 |
-#### ๐ Dataset: `kodf` | ๐ง Model: `ff++`
+### Train: `ff++` โ Test: `kodf`
-| ๐ค Model | ๐ผ๏ธ Frames | โ๏ธ Agg Mode | ๐ TTA (H-Flip) | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
+| Model | Frames | Agg Mode | TTA (H-Flip) | Test@Accuracy | Test@AUC | Test@Log Loss |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **ms_eff_gcvit_b0** | 20 | conf | 0 | 0.7544 | 0.862 | 0.8829 |
| **ms_eff_gcvit_b5** | 20 | conf | 0 | **0.7695** | **0.8821** | **0.7635** |
-
-
-## ๐งฉ Branch-Ablation Study
-
-To quantify how much each branch โ the **Low-level Branch** and the **High-level Branch** โ contributes to MS-EffGCViT's performance, we trained and evaluated 3 variants (`Eff`, `Low-Eff-GCViT`, `High-Eff-GCViT`) in both `b0`/`b5` sizes across all three benchmark datasets.
-
-## ๐ฌ Celeb DF(V2) Ablation Evaluation
-
-
-โ
Test Data Evaluation
-
-| ๐ค Model | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
-| :--- | :---: | :---: | :---: |
-| EfficientNet B0 | 0.9643 | 0.9754 | 0.0977 |
-| EfficientNet B5 | 0.9768 | 0.9837 | 0.0676 |
-| low_eff_gcvit_b0 | 0.9788 | 0.9837 | 0.0658 |
-| low_eff_gcvit_b5 | 0.9865 | **0.9987** | 0.0150 |
-| high_eff_gcvit_b0 | 0.9807 | 0.9884 | 0.0343 |
-| **high_eff_gcvit_b5** | **0.9942** | 0.9977 | **0.0101** |
-
-
-
-## ๐ฌ FaceForensics++ Ablation Evaluation
-
-
-โ
Test Data Evaluation
-
-| ๐ค Model | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
-| :--- | :---: | :---: | :---: |
-| EfficientNet B0 | 0.9202 | 0.9497 | 0.1523 |
-| EfficientNet B5 | 0.9660 | 0.9831 | 0.0842 |
-| low_eff_gcvit_b0 | 0.8033 | 0.9257 | 0.4148 |
-| low_eff_gcvit_b5 | 0.8700 | 0.9486 | 0.3365 |
-| high_eff_gcvit_b0 | 0.9783 | 0.9886 | 0.0767 |
-| **high_eff_gcvit_b5** | **0.9808** | **0.9956** | **0.0606** |
+# Inference Latency Comparison
-
+Latency is measured with batch size 1 on a single Tesla T4 and video (num_frames=20).
-## ๐ฌ Kodf Ablation Evaluation
+| Model | Mode | Params (M) | FLOPs (G) | Latency (s) | Ratio |
+| ----- | ---- | ---------- | --------- | -------- | ----- |
+| **ms_eff_gcvit_b0** | Fast Mode | 8.70 | 0.85 | 0.051 | 1.00ร |
+| **ms_eff_gcvit_b5** | Pro Mode | 50.34 | 13.43 | 0.479 | 9.41ร |
-
-โ
Test Data Evaluation
-| ๐ค Model | ๐ฏ Accuracy | ๐ AUC | ๐ Log Loss |
-| :--- | :---: | :---: | :---: |
-| EfficientNet B0 | 0.9180 | 0.9520 | 0.2010 |
-| EfficientNet B5 | 0.9601 | 0.9770 | 0.1120 |
-| low_eff_gcvit_b0 | 0.8684 | 0.9481 | 0.2843 |
-| low_eff_gcvit_b5 | 0.8808 | 0.9596 | 0.2381 |
-| high_eff_gcvit_b0 | 0.9462 | 0.9696 | 0.1761 |
-| **high_eff_gcvit_b5** | **0.9745** | **0.9911** | **0.0844** |
+| Model | Params (M) | FLOPs (G) | Latency (s) | Ratio |
+| ----- | ---------- | --------- | ----------- | ----- |
+| **ms_eff_gcvit_b0 (Ours)** | **8.7** | **0.85** | **0.051** | **1.0ร** |
+| sia (ECCV 2022) | 17.57 | 2.05 | 0.094 | 1.8ร |
+| core (CVPR 2022) | 20.81 | 8.40 | 0.153 | 3.0ร |
+| iid (CVPR 2023) | 65.5 | 12.7 | 0.181 | 3.5ร |
+| effort (ICML 2025) | 303.4 | 155.9 | 1.041 | 20.4ร |
-
\ No newline at end of file