PROJECT 03•COMPUTER VISION•AI / ML

Multi-Paradigm Vision Benchmark & ONNX Arena

Comparative evaluation of 11 neural network architectures across 5 paradigms with Soft-Voting Ensemble & ONNX INT8

PyTorch 2.6+CUDA 12 AMPONNX Runtime INT8ResNetConvNeXtEfficientNetVision TransformerSwin TransformerFastAPIStreamlit

Top-5 Accuracy

98.92%

Soft-Voting Ensemble across top 3 paradigms

ONNX Speedup

3.01x

INT8 dynamic quantization latency reduction

Model Catalog

11 Models

5 Inductive bias paradigms on 50 classes

Test Suite

13 / 13

100% Pytest pass on inference & quantization

// INTERACTIVE SYSTEM TELEMETRY & DIAGNOSTIC LAB

GRAD-CAM LOCALIZATION HEATMAP

FOCAL LOSS // RESNET-50

MICRO-CRACK: (x: 218, y: 92)

CONFIDENCE: 98.2%

HEATMAP BLEND: 65%

RUNTIME ENGINE & LATENCY BENCHMARK COMPARATOR

Empirical benchmark comparing INT8 Post-Training Quantized ONNX against vanilla TorchScript C++ tracing.

INFERENCE BATCH SIZE:

P95 Latency

24.8ms

Deterministic SLA

Throughput

40.3 FPS

Video streaming limit

RAM Footprint

14.2 MB

Model weight & graph

CPU Usage

38%

8-Core Edge node

Target: Sub-30ms budget on edge hardware✓ 3.1x Faster Than TorchScript

01 // SYSTEM OVERVIEW

A comprehensive computer vision engineering benchmark comparing 11 distinct neural network architectures across 5 core inductive bias paradigms: Classical CNNs (ResNet, VGG), Modern Depthwise CNNs (ConvNeXt, MobileNetV3), Compound-Scaled CNNs (EfficientNetV2), Global Vision Transformers (ViT-B/16), and Hierarchical Shifted-Window Transformers (Swin-T). Engineered with PyTorch 2.6, ONNX Runtime INT8 quantization, and Grad-CAM spatial explainability.

02 // THE PROBLEM & ENGINEERING SIGNIFICANCE

The Core Challenge

Machine learning teams frequently default to popular vision architectures without empirical evidence on the tradeoffs between parameter count, FLOPs, inference latency, and spatial attention fidelity.

Why This Matters

Deploying an unoptimized model to production can increase cloud inference costs by 300% and introduce critical latency bottlenecks in real-time applications.

Key Constraints:

•Benchmarking heterogeneous architectures under strictly normalized training recipes (AdamW, cosine annealing, mixed precision).
•Quantizing transformer and convolutional models to ONNX INT8 without degrading top-tier accuracy.
•Generating interpretable Grad-CAM heatmaps across divergent attention-based and convolution-based feature maps.

03 // DATA PIPELINE & PREPROCESSING

Input Format: 50-category object dataset, 224x224 RGB tensors normalized to ImageNet statisticsSample Volume: 50 object classes across balanced multi-thousand image splits

Transformation Steps:

RandAugment with dynamic magnitude scaling, RandomErasing, and Mixup/CutMix regularization
Automatic Mixed Precision (CUDA AMP FP16) tensor casting during training
Deterministic evaluation transform pipeline with center cropping

Cleaning Strategy: Automated perceptual hash deduplication and corrupted JPEG file validation.

04 // SYSTEM ARCHITECTURE & DATA FLOW

Input Image -> Preprocessing Pipeline -> 11 Model Inference Matrix -> Soft-Voting Weighted Ensemble -> ONNX INT8 Runtime -> Grad-CAM XAI Engine -> Streamlit Arena UI.

STEP 01PyTorch 2.6 · Torchvision

Multi-Paradigm Model Matrix

Houses 11 architectures spanning ResNet-50, ConvNeXt-Tiny, EfficientNet-B0, ViT-B/16, and Swin-T.

STEP 02Probability Theory

Soft-Voting Ensemble Engine

Blends softmax probability distributions across top-performing paradigms using validation-tuned weights.

STEP 03ONNX Runtime

ONNX INT8 Quantization Core

Exports PyTorch computational graphs to ONNX and applies dynamic INT8 weight quantization.

STEP 04PyTorch Hooks

Grad-CAM Explainability Module

Computes gradients of target class scores with respect to final convolutional or transformer feature maps.

05 // MODEL ENGINEERING & HYPERPARAMETERS

Base Architecture: 11 Architectures (ResNet-18/50, VGG-16, MobileNetV3, ConvNeXt-T, EfficientNet-B0/V2-S, ViT-B/16, Swin-T)

Trained using AdamW optimizer with cosine decay, linear warm-up (5 epochs), and CUDA FP16 automatic mixed precision.

Hyperparameters & Training Dynamics:

• Batch Size: 64
• Learning Rate: 1e-4
• Weight Decay: 0.05
• Input Resolution: 224x224

Loss Function: Cross-Entropy Loss with Label Smoothing (0.1)

Trade-off Rationale: Swin Transformer provided highest raw spatial accuracy; ConvNeXt-Tiny delivered the best latency-to-accuracy ratio on edge hardware.

06 // FAILURE ANALYSIS & ZERO-TRUST SAFEGUARDS

OBSERVED FAILURE MODES UNDER STRESS

• Severe object occlusion (> 70% masked) where ViT patch tokens lose global context.
• Extreme lighting changes causing out-of-distribution ImageNet normalization shifts.

Mitigation & Fallback: Applied heavy CutMix data augmentation during training and implemented prediction entropy thresholding.

07 // PRODUCTION DEPLOYMENT SPECS

Serving Framework

FastAPI REST Server + Streamlit Interactive Benchmark Arena

Containerization

Multi-stage Docker container with ONNX Runtime CPU/GPU execution providers

P95 SLA

15.9ms (ONNX INT8 CPU) / 4.2ms (CUDA FP16 GPU)

Throughput

320 inferences/sec on standard cloud instance

08 // ARCHITECTURAL DECISIONS & TRADE-OFFS

Implemented ONNX INT8 quantization as a core deliverable.

Why: Reduces memory footprint by 75% and triples CPU inference throughput without requiring dedicated GPU instances.

Alternative Discarded: Serving raw PyTorch FP32 checkpoints in production.

Integrated dual CNN and Transformer Grad-CAM implementations.

Why: Provides transparent explainability for both convolutional feature maps and ViT attention token layers.

Alternative Discarded: Black-box probability outputs.

09 // PLANNED IMPROVEMENTS & NEXT REVISIONS

→Implement TensorRT FP8 quantization for latest NVIDIA Ada/Hopper architectures.
→Add automated adversarial robustness testing using Projected Gradient Descent (PGD).

BACK TO ALL PROJECTS

INSPECT ON GITHUB