README.md

---
license: mit
datasets:
- Bingsu/Gameplay_Images
language:
- en
metrics:
- accuracy
- precision
- recall
- f1
- roc_auc
- confusion_matrix
base_model:
- google/efficientnet-b0
pipeline_tag: image-classification
tags:
- game-detection
- image-classification
- efficientnet
- hashtag-generation
- computer-vision
- gaming
---

# Game_Detection

### Automated Video Game Recognition for Hashtag Suggestion on Live Streaming Platforms

A 10-class image classifier that identifies which video game is being played from a gameplay
screenshot. Built on a fine-tuned [`google/efficientnet-b0`](https://huggingface.co/google/efficientnet-b0)
backbone, trained at a custom, aspect-ratio-preserving **180×320** input resolution (instead of the
standard 224×224 square crop) on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images)
dataset.

This model was built as part of a university course project (AI Lab, SE334) — *"Automated Video Game
Recognition and Hashtag Suggestion for Live Streaming Platforms Using Image Classification"* — and
powers the [GameSense](https://gamesense-h456.onrender.com/) demo app.

**Authors:** S. M. Nihal Ahmed, Afrim Hossen Khan

## Model Details

- **Base model:** `google/efficientnet-b0`
- **Task:** Multi-class image classification (10 classes)
- **License:** MIT
- **Architecture:** EfficientNet-B0 backbone (ImageNet-pretrained), fine-tuned end-to-end with the
  final classifier layer replaced for 10 output classes. Trained at a custom **180×320** input
  resolution — half of the source dataset's native 640×360, preserving the true 16:9 aspect ratio —
  made possible without architectural changes since EfficientNet's `AdaptiveAvgPool2d` head is
  resolution-agnostic.
- **Fine-tuning objective:** Cross-entropy loss with label smoothing (0.1), `sklearn` balanced class
  weights applied in the loss (the source dataset is already perfectly balanced at 1,000 images/class)
- **Training regime:** Mixed-precision (AMP) training on dual CUDA T4 GPUs, AdamW optimizer with a
  OneCycleLR schedule, up to 25 epochs with early stopping (patience = 6, monitored on validation loss)

## Classes

`Among Us, Apex Legends, Fortnite, Forza Horizon, Free Fire, Genshin Impact, God of War, Minecraft,
Roblox, Terraria`

## Intended Use

This model is intended for identifying which video game is shown in a gameplay screenshot. Example use
cases:

- Auto-generating hashtags/tags for gameplay clips, stream thumbnails, and social posts
- Categorizing or organizing gameplay footage/screenshots by game on a content platform
- A component in a larger stream metadata or content-tagging pipeline
- Research and coursework on multi-class visual classification

**Out of scope:** This model only recognizes the 10 games listed above — any other game will be forced
into one of these 10 labels rather than correctly rejected. It has been evaluated on one dataset only,
and has not been validated against real-world production streaming footage, unusual camera angles,
menu/loading screens, or extensive in-game cosmetic content (e.g. crossover skins) that may visually
resemble a different game in the label set.

## How to Use

This model is distributed in two formats — pick whichever fits your stack.

### Option A: ONNX (lightweight, CPU-friendly)

Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the
`.onnx.data` file alongside it at runtime:

- [`efficientnet_b0_gameplay.onnx`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay.onnx) — the ONNX graph
- [`efficientnet_b0_gameplay.onnx.data`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay.onnx.data) — the external weights file

Install dependencies:

```bash
pip install onnxruntime huggingface_hub pillow numpy
```

#### Single-image prediction

```python
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download

REPO_ID = "nihal4/Game_Detection"
IMG_SIZE = (320, 180)  # PIL resize takes (width, height)
IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
IMAGENET_STD  = np.array([0.229, 0.224, 0.225], dtype=np.float32)
CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire',
               'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria']

# Downloads both files into the same local cache folder — required, since the
# .onnx graph references .onnx.data by relative path at load time.
onnx_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx")
hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx.data")

session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name

def preprocess_pil(img: Image.Image) -> np.ndarray:
    img = img.convert("RGB").resize(IMG_SIZE)
    arr = np.asarray(img, dtype=np.float32) / 255.0        # HWC, [0,1]
    arr = (arr - IMAGENET_MEAN) / IMAGENET_STD              # normalize, same stats as training
    return arr.transpose(2, 0, 1)                           # HWC -> CHW

def softmax(x: np.ndarray) -> np.ndarray:
    e = np.exp(x - x.max(axis=1, keepdims=True))
    return e / e.sum(axis=1, keepdims=True)

def predict(image_path: str):
    image = Image.open(image_path)
    x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32)
    logits = session.run([output_name], {input_name: x})[0]
    probs = softmax(logits)[0]
    top_idx = int(probs.argmax())
    return CLASS_NAMES[top_idx], probs

label, probs = predict("path/to/screenshot.jpg")
print(f"Prediction: {label}")
for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]):
    print(f"  {name:<16} {p*100:5.1f}%")
```

#### Batch prediction

```python
image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"]

batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32)
logits = session.run([output_name], {input_name: batch})[0]
probs = softmax(logits)
preds = probs.argmax(axis=1)

for path, pred, p in zip(image_paths, preds, probs):
    print(f"{path}: {CLASS_NAMES[int(pred)]}  ({p[int(pred)]*100:.1f}%)")
```

> For GPU inference, install `onnxruntime-gpu` instead and pass
> `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session.

### Option B: PyTorch (.pth checkpoint)

Download the checkpoint:

- [`efficientnet_b0_gameplay_final.pth`](https://huggingface.co/nihal4/Game_Detection/resolve/main/efficientnet_b0_gameplay_final.pth)

Install dependencies:

```bash
pip install torch torchvision huggingface_hub pillow numpy
```

#### Single-image prediction

```python
import torch
import torch.nn as nn
import numpy as np
from torchvision import models, transforms
from PIL import Image
from huggingface_hub import hf_hub_download

REPO_ID = "nihal4/Game_Detection"
IMG_SIZE = (180, 320)  # (H, W) — torchvision transforms convention
CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire',
               'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria']

ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay_final.pth")
checkpoint = torch.load(ckpt_path, map_location="cpu")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = models.efficientnet_b0(weights=None)
in_features = model.classifier[1].in_features
model.classifier[1] = nn.Linear(in_features, len(CLASS_NAMES))
model.load_state_dict(checkpoint["model_state_dict"])
model.to(device).eval()

transform = transforms.Compose([
    transforms.Resize(IMG_SIZE),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

@torch.no_grad()
def predict(image_path: str):
    image = Image.open(image_path).convert("RGB")
    x = transform(image).unsqueeze(0).to(device)
    logits = model(x)
    probs = torch.softmax(logits, dim=1)[0]
    top_idx = int(probs.argmax())
    return CLASS_NAMES[top_idx], probs.cpu().numpy()

label, probs = predict("path/to/screenshot.jpg")
print(f"Prediction: {label}")
for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]):
    print(f"  {name:<16} {p*100:5.1f}%")
```

#### Batch prediction

```python
from torch.utils.data import Dataset, DataLoader

class ImageListDataset(Dataset):
    def __init__(self, paths, transform):
        self.paths = paths
        self.transform = transform

    def __len__(self):
        return len(self.paths)

    def __getitem__(self, i):
        img = Image.open(self.paths[i]).convert("RGB")
        return self.transform(img), self.paths[i]

image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"]
loader = DataLoader(ImageListDataset(image_paths, transform), batch_size=8)

model.eval()
with torch.no_grad():
    for images, paths in loader:
        images = images.to(device)
        logits = model(images)
        probs = torch.softmax(logits, dim=1)
        preds = probs.argmax(dim=1)
        for path, pred, p in zip(paths, preds, probs):
            print(f"{path}: {CLASS_NAMES[int(pred)]}  ({p[int(pred)]*100:.1f}%)")
```

## Training Data

The model was fine-tuned on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images)
dataset — 10,000 gameplay screenshots (1,000 per class) at native 640×360 resolution, PNG format.

- **Labels:** 10 classes (see [Classes](#classes) above)
- **Splits:** Stratified 70 / 15 / 15 train / validation / test (the source dataset ships a single
  `train` split only; the split above was carved out manually, preserving per-class balance)
- **Preprocessing:** Resize to 180×320 (custom, aspect-ratio-preserving resolution), ImageNet
  normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`)
- **Training augmentation:** Random horizontal flip, color jitter, random rotation (±8°), random
  erasing
- **Class balancing:** The dataset is already perfectly balanced (1,000 images/class); `sklearn`
  balanced class weights are still computed and applied in the loss as a safeguard

## Training Procedure

<!-- PLACEHOLDER: training curves (loss/accuracy per epoch) — image to be uploaded -->

![training_curves](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/lBckExVfa7nctNBbRq36B.png)


- **Framework:** PyTorch
- **Hardware:** Kaggle free-tier T4 x2 GPUs
- **Loss:** Cross-entropy with label smoothing (0.1)
- **Mixed precision:** Enabled (AMP)


## Evaluation

Evaluated on the held-out test split (n = 1,500) at a decision threshold of 0.5.

### Classification Report

| Class          | Precision | Recall | F1-score | Support |
|----------------|:---------:|:------:|:--------:|:-------:|
| Among Us       | 1.0000    | 1.0000 | 1.0000   | 150     |
| Apex Legends   | 1.0000    | 0.9933 | 0.9967   | 150     |
| Fortnite       | 1.0000    | 1.0000 | 1.0000   | 150     |
| Forza Horizon  | 1.0000    | 1.0000 | 1.0000   | 150     |
| Free Fire      | 1.0000    | 1.0000 | 1.0000   | 150     |
| Genshin Impact | 0.9934    | 1.0000 | 0.9967   | 150     |
| God of War     | 1.0000    | 1.0000 | 1.0000   | 150     |
| Minecraft      | 1.0000    | 1.0000 | 1.0000   | 150     |
| Roblox         | 1.0000    | 1.0000 | 1.0000   | 150     |
| Terraria       | 1.0000    | 1.0000 | 1.0000   | 150     |
| **accuracy**   |           |        | **0.9993** | 1,500 |
| macro avg      | 0.9993    | 0.9993 | 0.9993   | 1,500   |
| weighted avg   | 0.9993    | 0.9993 | 0.9993   | 1,500   |

**Test ROC-AUC:** 1.0000 (macro average; per-class AUC is also 1.0000 across all 10 classes)

### Confusion Matrix

<!-- PLACEHOLDER: image to be uploaded -->


![confusion_matrix](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/GgkCX5ir4A12Iip96hrQy.png)

### ROC Curve

<!-- PLACEHOLDER: image to be uploaded -->


![roc_auc_curves](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/Q0J0IoHtaHhkq_IaTP-Qf.png)

## Limitations

- Performance is reported on a single dataset; generalization to other capture sources, image
  qualities, camera angles, or game versions/UI updates is not guaranteed.
- The classifier is closed-set — it will always assign one of the 10 trained classes, even to games or
  content it has never seen, rather than rejecting out-of-distribution input.
- Confidence can be lower on visually ambiguous content, such as games with extensive cosmetic/skin
  systems whose art style can resemble another class in the label set.
- The model has not been evaluated as a standalone production guardrail; low-confidence predictions
  should be handled with a confidence threshold or human review rather than trusted outright.

## Citation

If you use this model, please cite this repository and reference this course project:

```
@misc{game-detection-classifier,
  title  = {Automated Video Game Recognition and Hashtag Suggestion for Live Streaming Platforms
            Using Image Classification},
  author = {S. M. Nihal Ahmed and Afrim Hossen Khan},
  year   = {2026},
  note   = {Course project, AI Lab (SE334), Daffodil International University}
}
```