Cross-Domain Generalization Failure in Lightweight Intrusion Detection Models for IIoT Networks

Authors: MD Azizul Hakim, Md Shihab Uddin, Talha Ibne Anis

arXiv ID: 2607.00553

Problem: Lightweight ML intrusion detection models achieve near-perfect accuracy within their training network but fail catastrophically when deployed on unseen IIoT networks.

Key Methodology:

Trained four lightweight architectures (DecisionTree, SmallMLP, Small1DCNN, SmallLSTM) on Edge-IIoTset and evaluated zero-shot on Gotham and WUSTL-IIoT-2021 using a minimal 16-dimensional common feature schema (port buckets, protocol, TCP flags)
Applied SHAP explainability across two top-performing models to diagnose feature reliance, and compared performance under both artificially balanced and naturally imbalanced class distributions
Tested adversarial robustness (HopSkipJump attack) and few-shot recovery by fine-tuning on 1–25% of target-domain labeled data

Key Results:

In-domain F1 was ~0.97 across all models; cross-domain F1 collapsed to 0.09–0.28 under natural class distributions (≤29% of in-domain performance)
The most influential port-bucket feature (dst_port_none) appeared in source-domain attack traffic at 96× the rate in Gotham and 435× the rate in WUSTL-IIoT-2021
Balanced evaluation reversed which target appeared harder: DecisionTree scored 0.53 (WUSTL) vs 0.17 (Gotham) under balanced sampling, but 0.13 vs 0.18 under natural distribution
Adversarial robustness did not correlate with cross-domain generalization - SmallMLP/SmallLSTM were strongest cross-domain but most fragile under attack (accuracy drop of 0.88)

Applied Context: Builders should not trust in-domain accuracy as a proxy for deployment readiness - cross-network evaluation under realistic (imbalanced) class distributions is necessary, and port-bucket shortcuts must be explicitly addressed (not just coarsened) for models to transfer across IIoT environments.