LLM / Training / Evaluation
Cross-Domain Generalization Failure in Lightweight Intrusion Detection Models for IIoT Networks
** MD Azizul Hakim, Md Shihab Uddin, Talha Ibne Anis
Cross-Domain Generalization Failure in Lightweight Intrusion Detection Models for IIoT Networks
Authors: MD Azizul Hakim, Md Shihab Uddin, Talha Ibne Anis
arXiv ID: 2607.00553
Problem: Lightweight ML intrusion detection models achieve near-perfect accuracy within their training network but fail catastrophically when deployed on unseen IIoT networks.
Key Methodology:
- Trained four lightweight architectures (DecisionTree, SmallMLP, Small1DCNN, SmallLSTM) on Edge-IIoTset and evaluated zero-shot on Gotham and WUSTL-IIoT-2021 using a minimal 16-dimensional common feature schema (port buckets, protocol, TCP flags)
- Applied SHAP explainability across two top-performing models to diagnose feature reliance, and compared performance under both artificially balanced and naturally imbalanced class distributions
- Tested adversarial robustness (HopSkipJump attack) and few-shot recovery by fine-tuning on 1–25% of target-domain labeled data
Key Results:
- In-domain F1 was ~0.97 across all models; cross-domain F1 collapsed to 0.09–0.28 under natural class distributions (≤29% of in-domain performance)
- The most influential port-bucket feature (
dst_port_none) appeared in source-domain attack traffic at 96× the rate in Gotham and 435× the rate in WUSTL-IIoT-2021 - Balanced evaluation reversed which target appeared harder: DecisionTree scored 0.53 (WUSTL) vs 0.17 (Gotham) under balanced sampling, but 0.13 vs 0.18 under natural distribution
- Adversarial robustness did not correlate with cross-domain generalization - SmallMLP/SmallLSTM were strongest cross-domain but most fragile under attack (accuracy drop of 0.88)
Applied Context: Builders should not trust in-domain accuracy as a proxy for deployment readiness - cross-network evaluation under realistic (imbalanced) class distributions is necessary, and port-bucket shortcuts must be explicitly addressed (not just coarsened) for models to transfer across IIoT environments.