Malware continues to evolve, exposing weaknesses in conventional detectors and motivating realistic adversarial evaluations. Prior RL-based evasion methods often rely on partial model access or feature-level perturbations, limiting realism under strict black-box constraints. We propose xPriMES, a dual-environment reinforcement learning framework that generates functionality-preserving binary mutations for malware evasion in black-box settings. A LightGBM surrogate provides continuous confidence feedback for dense reward shaping, while the real target detector supplies binary feedback—used both for episode termination and for issuing the final reward—ensuring learning remains grounded in real evasion outcomes. The agent employs Thompson sampling and SHAP-guided prioritized replay to focus exploration on feature-relevant mutations and accelerate convergence. Experiments on multiple static detectors (LightGBM, RF+CNN, MalConv, CNN, KNN) demonstrate up to 97.4% evasion success, surpassing PSP-Mal under equivalent conditions. Further tests on VirusTotal confirm the transferability and real-world impact of the adversarial samples. These findings show that integrating explainable guidance with surrogate-assisted RL yields interpretable and effective black-box evasion while preserving functionality. We conclude with implications for defensive hardening and discuss limitations related to surrogate fidelity and the focus on static detection.
