quang/ai
VI Hire me
Computer VisionResearchCapstone GSU26AI07

Inference-Efficient ConvMAE

Can a masked-autoencoder backbone be cheaper at inference without losing accuracy? Four hybrid designs, one fair protocol, measured under TensorRT rather than guessed from parameter counts.

hybrid backbones
4
pre-training data
IN-1K
face fine-tune seeds
3
task × model × backend demo checks
30/30

Question

ConvMAE is a strong self-supervised backbone, but deployment cost matters as much as accuracy. My FPT University capstone (GSU26AI07) asked which hybrid design gives the best inference efficiency for universal visual recognition tasks.

Method

  • Four hybrid backbones — Ghost, Transformer, Mamba-2 and Bi-Mamba — each MAE-pre-trained on ImageNet-1K with multi-GPU DDP and mixed precision.
  • Linear probe, plus face fine-tuning over three seeds.
  • Bias-controlled PyTorch vs ONNX → TensorRT benchmarks so the latency comparison is actually fair.

Result

The Ghost arm was faster and lighter than ConvMAE-Base under TensorRT at comparable face-ID accuracy.

Live demo

The GhostConvMAE Face Demo runs ConvMAE-Base and Ghost+ConvMAE side by side on CelebA, CASIA, SCface, LFW and an ImageNet-1K linear probe, with a live PyTorch ⇄ TensorRT switch on a 4 GB GTX 1650 and per-forward latency and peak-VRAM accounting. Its automated self-test harness passes 30/30 task × model × backend combinations — and caught a CASIA RecordIO label-corruption bug.

→ More work

Other case studies

BTS Digital Twin — Viettel AI Race 2026

Novel view synthesis of telecom (BTS) towers from drone captures with 3D Gaussian Splatting — then large urban scenes under an inference-time limit on an NVIDIA H200 in the final round.

 iPBMS — real-time patient monitoring

Fall and seizure detection on live RTSP hospital camera streams — from pose model to on-premise production on two cameras, with alerts to care staff in under two seconds.

 Face-Aware MAE

Identity-aware masked autoencoding: instead of masking random blocks, point MAE pre-training at the eyes, nose, mouth and central facial structure — and change nothing else.

 Vietnamese License Plate Recognition Benchmark

A reproducible benchmark for Vietnamese automatic license plate recognition: three YOLO generations, three OCR pipelines, real RTSP traffic-camera data.
esc