| --- |
| license: mit |
| datasets: |
| - Bingsu/Gameplay_Images |
| language: |
| - en |
| metrics: |
| - accuracy |
| - precision |
| - recall |
| - f1 |
| - roc_auc |
| - confusion_matrix |
| base_model: |
| - google/efficientnet-b0 |
| pipeline_tag: image-classification |
| tags: |
| - game-detection |
| - image-classification |
| - efficientnet |
| - hashtag-generation |
| - computer-vision |
| - gaming |
| --- |
| |
| # Game_Detection |
| |
| ### Automated Video Game Recognition for Hashtag Suggestion on Live Streaming Platforms |
| |
| A 10-class image classifier that identifies which video game is being played from a gameplay |
| screenshot. Built on a fine-tuned [`google/efficientnet-b0`](https://huggingface.co/google/efficientnet-b0) |
| backbone, trained at a custom, aspect-ratio-preserving **180×320** input resolution (instead of the |
| standard 224×224 square crop) on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images) |
| dataset. |
| |
| This model was built as part of a university course project (AI Lab, SE334) — *"Automated Video Game |
| Recognition and Hashtag Suggestion for Live Streaming Platforms Using Image Classification"* — and |
| powers the [GameSense](https://gamesense-h456.onrender.com/) demo app. |
| |
| **Authors:** S. M. Nihal Ahmed, Afrim Hossen Khan |
| |
| ## Model Details |
| |
| - **Base model:** `google/efficientnet-b0` |
| - **Task:** Multi-class image classification (10 classes) |
| - **License:** MIT |
| - **Architecture:** EfficientNet-B0 backbone (ImageNet-pretrained), fine-tuned end-to-end with the |
| final classifier layer replaced for 10 output classes. Trained at a custom **180×320** input |
| resolution — half of the source dataset's native 640×360, preserving the true 16:9 aspect ratio — |
| made possible without architectural changes since EfficientNet's `AdaptiveAvgPool2d` head is |
| resolution-agnostic. |
| - **Fine-tuning objective:** Cross-entropy loss with label smoothing (0.1), `sklearn` balanced class |
| weights applied in the loss (the source dataset is already perfectly balanced at 1,000 images/class) |
| - **Training regime:** Mixed-precision (AMP) training on dual CUDA T4 GPUs, AdamW optimizer with a |
| OneCycleLR schedule, up to 25 epochs with early stopping (patience = 6, monitored on validation loss) |
| |
| ## Classes |
| |
| `Among Us, Apex Legends, Fortnite, Forza Horizon, Free Fire, Genshin Impact, God of War, Minecraft, |
| Roblox, Terraria` |
| |
| ## Intended Use |
| |
| This model is intended for identifying which video game is shown in a gameplay screenshot. Example use |
| cases: |
| |
| - Auto-generating hashtags/tags for gameplay clips, stream thumbnails, and social posts |
| - Categorizing or organizing gameplay footage/screenshots by game on a content platform |
| - A component in a larger stream metadata or content-tagging pipeline |
| - Research and coursework on multi-class visual classification |
| |
| **Out of scope:** This model only recognizes the 10 games listed above — any other game will be forced |
| into one of these 10 labels rather than correctly rejected. It has been evaluated on one dataset only, |
| and has not been validated against real-world production streaming footage, unusual camera angles, |
| menu/loading screens, or extensive in-game cosmetic content (e.g. crossover skins) that may visually |
| resemble a different game in the label set. |
| |
| ## How to Use |
| |
| This model is distributed in two formats — pick whichever fits your stack. |
| |
| ### Option A: ONNX (lightweight, CPU-friendly) |
| |
| Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the |
| `.onnx.data` file alongside it at runtime: |
| |
| - [`efficientnet_b0_gameplay.onnx`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay.onnx) — the ONNX graph |
| - [`efficientnet_b0_gameplay.onnx.data`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay.onnx.data) — the external weights file |
| |
| Install dependencies: |
| |
| ```bash |
| pip install onnxruntime huggingface_hub pillow numpy |
| ``` |
| |
| #### Single-image prediction |
| |
| ```python |
| import numpy as np |
| import onnxruntime as ort |
| from PIL import Image |
| from huggingface_hub import hf_hub_download |
| |
| REPO_ID = "nihal4/Game_Detection" |
| IMG_SIZE = (320, 180) # PIL resize takes (width, height) |
| IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32) |
| IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32) |
| CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire', |
| 'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria'] |
| |
| # Downloads both files into the same local cache folder — required, since the |
| # .onnx graph references .onnx.data by relative path at load time. |
| onnx_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx") |
| hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx.data") |
|
|
| session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"]) |
| input_name = session.get_inputs()[0].name |
| output_name = session.get_outputs()[0].name |
| |
| def preprocess_pil(img: Image.Image) -> np.ndarray: |
| img = img.convert("RGB").resize(IMG_SIZE) |
| arr = np.asarray(img, dtype=np.float32) / 255.0 # HWC, [0,1] |
| arr = (arr - IMAGENET_MEAN) / IMAGENET_STD # normalize, same stats as training |
| return arr.transpose(2, 0, 1) # HWC -> CHW |
| |
| def softmax(x: np.ndarray) -> np.ndarray: |
| e = np.exp(x - x.max(axis=1, keepdims=True)) |
| return e / e.sum(axis=1, keepdims=True) |
| |
| def predict(image_path: str): |
| image = Image.open(image_path) |
| x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32) |
| logits = session.run([output_name], {input_name: x})[0] |
| probs = softmax(logits)[0] |
| top_idx = int(probs.argmax()) |
| return CLASS_NAMES[top_idx], probs |
| |
| label, probs = predict("path/to/screenshot.jpg") |
| print(f"Prediction: {label}") |
| for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]): |
| print(f" {name:<16} {p*100:5.1f}%") |
| ``` |
| |
| #### Batch prediction |
| |
| ```python |
| image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"] |
|
|
| batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32) |
| logits = session.run([output_name], {input_name: batch})[0] |
| probs = softmax(logits) |
| preds = probs.argmax(axis=1) |
|
|
| for path, pred, p in zip(image_paths, preds, probs): |
| print(f"{path}: {CLASS_NAMES[int(pred)]} ({p[int(pred)]*100:.1f}%)") |
| ``` |
| |
| > For GPU inference, install `onnxruntime-gpu` instead and pass |
| > `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session. |
| |
| ### Option B: PyTorch (.pth checkpoint) |
| |
| Download the checkpoint: |
| |
| - [`efficientnet_b0_gameplay_final.pth`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay_final.pth) |
| |
| Install dependencies: |
| |
| ```bash |
| pip install torch torchvision huggingface_hub pillow numpy |
| ``` |
| |
| #### Single-image prediction |
| |
| ```python |
| import torch |
| import torch.nn as nn |
| import numpy as np |
| from torchvision import models, transforms |
| from PIL import Image |
| from huggingface_hub import hf_hub_download |
| |
| REPO_ID = "nihal4/Game_Detection" |
| IMG_SIZE = (180, 320) # (H, W) — torchvision transforms convention |
| CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire', |
| 'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria'] |
| |
| ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay_final.pth") |
| checkpoint = torch.load(ckpt_path, map_location="cpu") |
| |
| device = torch.device("cuda" if torch.cuda.is_available() else "cpu") |
| |
| model = models.efficientnet_b0(weights=None) |
| in_features = model.classifier[1].in_features |
| model.classifier[1] = nn.Linear(in_features, len(CLASS_NAMES)) |
| model.load_state_dict(checkpoint["model_state_dict"]) |
| model.to(device).eval() |
| |
| transform = transforms.Compose([ |
| transforms.Resize(IMG_SIZE), |
| transforms.ToTensor(), |
| transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), |
| ]) |
| |
| @torch.no_grad() |
| def predict(image_path: str): |
| image = Image.open(image_path).convert("RGB") |
| x = transform(image).unsqueeze(0).to(device) |
| logits = model(x) |
| probs = torch.softmax(logits, dim=1)[0] |
| top_idx = int(probs.argmax()) |
| return CLASS_NAMES[top_idx], probs.cpu().numpy() |
| |
| label, probs = predict("path/to/screenshot.jpg") |
| print(f"Prediction: {label}") |
| for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]): |
| print(f" {name:<16} {p*100:5.1f}%") |
| ``` |
| |
| #### Batch prediction |
| |
| ```python |
| from torch.utils.data import Dataset, DataLoader |
|
|
| class ImageListDataset(Dataset): |
| def __init__(self, paths, transform): |
| self.paths = paths |
| self.transform = transform |
| |
| def __len__(self): |
| return len(self.paths) |
| |
| def __getitem__(self, i): |
| img = Image.open(self.paths[i]).convert("RGB") |
| return self.transform(img), self.paths[i] |
| |
| image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"] |
| loader = DataLoader(ImageListDataset(image_paths, transform), batch_size=8) |
| |
| model.eval() |
| with torch.no_grad(): |
| for images, paths in loader: |
| images = images.to(device) |
| logits = model(images) |
| probs = torch.softmax(logits, dim=1) |
| preds = probs.argmax(dim=1) |
| for path, pred, p in zip(paths, preds, probs): |
| print(f"{path}: {CLASS_NAMES[int(pred)]} ({p[int(pred)]*100:.1f}%)") |
| ``` |
| |
| ## Training Data |
|
|
| The model was fine-tuned on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images) |
| dataset — 10,000 gameplay screenshots (1,000 per class) at native 640×360 resolution, PNG format. |
|
|
| - **Labels:** 10 classes (see [Classes](#classes) above) |
| - **Splits:** Stratified 70 / 15 / 15 train / validation / test (the source dataset ships a single |
| `train` split only; the split above was carved out manually, preserving per-class balance) |
| - **Preprocessing:** Resize to 180×320 (custom, aspect-ratio-preserving resolution), ImageNet |
| normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`) |
| - **Training augmentation:** Random horizontal flip, color jitter, random rotation (±8°), random |
| erasing |
| - **Class balancing:** The dataset is already perfectly balanced (1,000 images/class); `sklearn` |
| balanced class weights are still computed and applied in the loss as a safeguard |
|
|
| ## Training Procedure |
|
|
| <!-- PLACEHOLDER: training curves (loss/accuracy per epoch) — image to be uploaded --> |
|
|
|  |
|
|
|
|
| - **Framework:** PyTorch |
| - **Hardware:** Kaggle free-tier T4 x2 GPUs |
| - **Loss:** Cross-entropy with label smoothing (0.1) |
| - **Mixed precision:** Enabled (AMP) |
|
|
|
|
| ## Evaluation |
|
|
| Evaluated on the held-out test split (n = 1,500) at a decision threshold of 0.5. |
|
|
| ### Classification Report |
|
|
| | Class | Precision | Recall | F1-score | Support | |
| |----------------|:---------:|:------:|:--------:|:-------:| |
| | Among Us | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Apex Legends | 1.0000 | 0.9933 | 0.9967 | 150 | |
| | Fortnite | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Forza Horizon | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Free Fire | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Genshin Impact | 0.9934 | 1.0000 | 0.9967 | 150 | |
| | God of War | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Minecraft | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Roblox | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | Terraria | 1.0000 | 1.0000 | 1.0000 | 150 | |
| | **accuracy** | | | **0.9993** | 1,500 | |
| | macro avg | 0.9993 | 0.9993 | 0.9993 | 1,500 | |
| | weighted avg | 0.9993 | 0.9993 | 0.9993 | 1,500 | |
|
|
| **Test ROC-AUC:** 1.0000 (macro average; per-class AUC is also 1.0000 across all 10 classes) |
|
|
| ### Confusion Matrix |
|
|
| <!-- PLACEHOLDER: image to be uploaded --> |
|
|
|
|
|  |
|
|
| ### ROC Curve |
|
|
| <!-- PLACEHOLDER: image to be uploaded --> |
|
|
|
|
|  |
|
|
| ## Limitations |
|
|
| - Performance is reported on a single dataset; generalization to other capture sources, image |
| qualities, camera angles, or game versions/UI updates is not guaranteed. |
| - The classifier is closed-set — it will always assign one of the 10 trained classes, even to games or |
| content it has never seen, rather than rejecting out-of-distribution input. |
| - Confidence can be lower on visually ambiguous content, such as games with extensive cosmetic/skin |
| systems whose art style can resemble another class in the label set. |
| - The model has not been evaluated as a standalone production guardrail; low-confidence predictions |
| should be handled with a confidence threshold or human review rather than trusted outright. |
|
|
| ## Citation |
|
|
| If you use this model, please cite this repository and reference this course project: |
|
|
| ``` |
| @misc{game-detection-classifier, |
| title = {Automated Video Game Recognition and Hashtag Suggestion for Live Streaming Platforms |
| Using Image Classification}, |
| author = {S. M. Nihal Ahmed and Afrim Hossen Khan}, |
| year = {2026}, |
| note = {Course project, AI Lab (SE334), Daffodil International University} |
| } |
| ``` |
