NEWS

SECRECY PERFORMANCE ANALYSIS OF RIS-ASSISTED RSMA SYSTEMS UNDER IMPERFECT ARTIFICIAL JAMMING CANCELLATION AND IMPERFECT SIC


(Received: 13-Aug.-2026, Revised: 29-Sep.-2026 , Accepted: 4-Oct.-2026)
This work investigates a secrecy communication for a novel reconfigurable intelligent surface (RIS)-assisted rate- splitting multiple access (RSMA) downlink transmission system using the artificial jamming (AJ). To better reflect practical scenarios, this work considers the effects of imperfect successive interference cancellation (SIC) and residual artificial jamming (AJ) interference at legitimate users. The mathematical analysis is derived by using the statistical characteristics of the cascaded channel using the Gaussian-Chebyshev quadrature and the incomplete Gamma function. Closed-form expressions are derived for four key performance metrics: outage probability, intercept probability, secrecy reliability trade-off, and average secrecy capacity. Moreover, the Golden Section Search (GSS) is separately applied to determine the optimal values of individual system parameters through one-dimensional optimization. The analytical expressions are verified against Monte Carlo simulation, which verifies that the proposed jammer-assisted RIS-RSMA scheme achieves significantly better performance than RIS-assisted non-orthogonal multiple access (NOMA), and RIS-assisted Non-jammer benchmark schemes in terms of secrecy performance. The results further reveal that the proposed jammer-assisted RIS-RSMA scheme achieves the most favorable security-reliability trade-off among all considered schemes, outperforming both the RIS-assisted NOMA and non-AJ counterparts.

[1] M. H. Khoshafa, T. Ngatched, M. H. Ahmed, Y. Gadallah and D. Niyato, "RIS-empowered Rate-splitting Multiple Access Toward 6G and Beyond Wireless Communication Networks: A Comprehensive Survey," IEEE Internet Things J., vol. 13, no. 3, pp. 3913-3937, 2026.

[2] H. Niu, Y. Xiao, X. Lei, J. Chen, Z. Xiao, M. Li and C. Yuen, "A Survey on Artificial Noise for Physical Layer Security: Opportunities, Technologies, Guidelines, Advances and Trends," IEEE Commun. Surv. Tutor., vol. 28, pp. 341-381, 2026.

[3] L. Cai et al., "Secure Physical Layer Communications for Low-altitude Economy Networking: A Survey," IEEE Commun. Surv. Tutor., vol. 28, pp. 2497-2530, 2025.

[4] N. Q. Sang, T. C. Hung, T. T. Duy, M. Tran and B. S. Kim, "Securing Wireless Communications with Energy Harvesting and Multi-antenna Diversity," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 11, no. 2, pp. 197-210, 2025.

[5] B. Clerckx et al., "A Primer on Rate-splitting Multiple Access: Tutorial, Myths and Frequently Asked Questions," IEEE J. Sel. Areas Commun., vol. 41, no. 5, pp. 1265-1308, 2023.

[6] Q.-N. Van, "Multiple RISs for Enhancing the Secrecy Performance of NOMA Systems over Realistic Nakagami-m Fading Channels," IEEE Trans. Veh. Technol., vol. 73, no. 5, pp. 6584-6599, 2024.

[7] P. T. Tran et al., "Enhancing secrecy performance of full-duplex relaying systems using IRS and RSMA," IEEE Trans. Mobile Comput., vol. 25, no. 1, pp. 878-891, 2026.

[8] Z. Lin et al., "Supporting IoT with rate-splitting multiple access in satellite and aerial-integrated networks," IEEE Internet Things J., vol. 8, no. 14, pp. 11123-11134, 2021.

[9] Z. Lin, Z. Feng, K. Guo, A. Nauman, D. Niyato, J. Wang, "AI-driven Seamless and Massive Access in Space-air-ground Integrated Networks," IEEE Wireless Commun., vol. 32, no. 3, pp. 72-79, 2025.

[10] Z. Lin et al., "Refracting RIS-aided Hybrid Satellite-terrestrial Relay Networks: Joint Beamforming Design and Optimization," IEEE Trans. Aerosp. Electron. Syst., vol. 58, no. 4, pp. 3717-3724, 2022.

[11] Z. Feng, Z. Lin, H. Wang, R. Ma, K. An and Y. He, "Toward Secure and Reliable SAGIN: Learning-Driven Multi-dimensional Resource Scheduling for Multi-RIS-assisted OTFS Transmission," IEEE J. Sel. Areas Commun., vol. 44, pp. 4848-4860, 2026.

[12] T. Bao and M. Erol-Kantarci, "Heuristic Deep Reinforcement Learning for Phase Shift Optimization in RIS-assisted Secure Satellite Communication Systems with RSMA," Proc. of the IEEE Int. Conf. on Communications (ICC), pp. 3900-3905, 2025.

[13] P. Huang, S. Pala, K. Singh and C.-P. Li, "RIS-assisted Multiple Access for Secure Multiuser MISO Non-terrestrial Networks: A Deep Reinforcement Learning Approach," Proc. of the IEEE Conf. on Computer Communications Workshops (IN-FOCOM WKSHPS), pp. 1-6, London, UK, 2025.

[14] B. Wang et al., "Secure Beamforming Design for RIS-aided Rate-splitting Systems with Untrusted User Cooperative Relay," IEEE Trans. Veh. Technol., vol. 75, pp. 1557-1562, 2026.

[15] Z. Ali et al., "Secure Sum-rate Optimization in RIS-assisted RSMA HAP Systems with Untrusted Users," IEEE Trans. Aerosp. Electron. Syst., vol. 62, pp. 5167-5182, 2026.

[16] C. Zhang et al., "Robust Secure Beamforming Design for Downlink RIS-ISAC Systems Enhanced by RSMA," IEEE Wireless Commun. Lett., vol. 14, pp. 1271-1275, 2025.

[17] H. Ke et al., "Rate Splitting and Beamforming Design for RIS-assisted RSMA-enhanced ISAC Networks," IEEE Wireless Commun. Lett., vol. 14, pp. 1738-1742, 2025.

[18] H. Liu et al., "Joint Resource Allocation and Secure Beamforming for Active RIS-aided mmWave-RSMA with Dual-identity User," IEEE Internet Things J., vol. 13, no. 4, pp. 6626-6639, 2026.

[19] M. Guo et al., "RIS-empowered Secure and Reliable Satellite Communications with RSMA," Proc. of the IEEE/CIC Int. Conf. on Communications in China (ICCC), pp. 1-6, 2025.

[20] Y. Gao, Q. Wu, W. Chen and D. W. K. Ng, "Rate-splitting Multiple Access for Intelligent Reflecting Surface-aided Secure Transmission," IEEE Commun. Lett., vol. 27, no. 2, pp. 482-486, 2023.

[21] D. Wang, J. Li, Q. Lv, Y. He, L. Li, Q. Hua, O. Alfarraj and J. Zhang, "Integrating Reconfigurable Intelligent Surface and AAV for Enhanced Secure Transmissions in IoT-enabled RSMA Networks," IEEE Internet Things J., vol. 12, no. 8, pp. 9405-9419, 2025.

[22] X. Guo, R. Yuan, J. Zhang, Y. Song and S. Chen, "RSMA-enhanced Secure Transmission Scheme for Active STAR-RIS-aided Networks," IEEE Wireless Commun. Lett., vol. 15, pp. 2699-2703, 2026.

[23] H. Lei, S. Zhou, K.-H. Park, I. S. Ansari, H. Tang and M.-S. Alouini, "Outage Analysis of Millimeter Wave RSMA Systems," IEEE Trans. Commun., vol. 71, no. 3, pp. 1504-1520, 2023.

[24] H. Lei, S. Zhou, X. Chen, I. S. Ansari, Y. Li, G. Pan and M.-S. Alouini, "On secure mmWave RSMA Systems," IEEE Internet Things J., vol. 11, no. 10, pp. 18917-18929, 2024.

[25] Q. Dan, H. Lei, K.-H. Park and G. Pan, "Beamforming for Secure RSMA-aided ISAC Systems," IEEE Trans. Cogn. Commun. Netw., vol. 11, no. 5, pp. 2970-2983, 2025.

[26] H.-N. Nguyen and P.-C. Ngo, "On the Effect of Keyhole Channel in RSMA Networks: A Theoretical Outage Analysis," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 12, no. 2, pp. 201-211, 2026.

[27] V. Ta Ty, T. Trung Duy and P. Ngoc Son, "Enhancing the Secrecy Performance and Secure Energy Efficiency of NOMA Systems Using RIS with Artificial Jamming," Int. J. Commun. Syst., vol. 39, no. 7, p. e70474, 2026.

[28] M. H. Khoshafa, T. M. N. Ngatched and M. H. Ahmed, "Reconfigurable Intelligent Surfaces-aided Physical Layer Security Enhancement in D2D Underlay Communications," IEEE Commun. Lett., vol. 25, no. 5, pp. 1443-1447, 2020.

[29] Y. Pei et al., "Secrecy Outage Probability Analysis for Downlink RIS-NOMA Networks with on-off Control," IEEE Trans. Veh. Technol., vol. 72, no. 9, pp. 11772-11786, 2023.

[30] N. H. Kiem et al., "Outage Probability and Ergodic Capacity of RIS-assisted RSMA Communication System," Digit. Signal Process., vol. 172, Article no. 105900, 2026.

[31] C. Ruan, Z. Zhang, H. Jiang, J. Dang, L. Wu and H. Zhang, "Approximate Message Passing for Channel Estimation in Reconfigurable Intelligent Surface Aided MIMO Multiuser Systems," IEEE Trans. Commun., vol. 70, no. 8, pp. 5469-5481, 2022.

[32] Z. Wang, L. Liu and S. Cui, "Channel Estimation for Intelligent Reflecting Surface Assisted Multiuser Communications: Framework, Algorithms and Analysis," IEEE Trans. Wireless Commun., vol. 19, no. 10, pp. 6607-6620, 2020.

[33] X. Wei, D. Shen and L. Dai, "Channel Estimation for RIS Assisted Wireless Communications Part I: Fundamentals, Solutions and Future Opportunities," IEEE Commun. Lett., vol. 25, no. 5, pp. 1398-1402, 2021.

[34] I. Trigui, W. Ajib, W. P. Zhu and M. Di Renzo, "Performance Evaluation and Diversity Analysis of RIS-assisted Communications over Generalized Fading Channels in the Presence of Phase Noise," IEEE Open J. Commun. Soc., vol. 3, pp. 593-607, 2022.

[35] C. Gong et al., "Intelligent Reflecting Surface Aided Secure Communications for NOMA Networks," IEEE Trans. Veh. Technol., vol. 71, no. 3, pp. 2761-2773, 2021.

[36] C. Hung Dinh et al., "Transmit Antenna Selection for Full-duplex Relay Millimeter-wave Communications Employing Rate-splitting Multiple Access," Int. J. Commun. Syst., vol. 38, no. 3, p. e6113, 2025.

[37] Z. Mo, Y. Ni, Y. Liu, H. Zhao, L. Li and Y. Cai, "Active RIS-assisted Communication Systems with Discrete Phase Shifts," IEEE Commun. Lett., vol. 29, no. 4, pp. 674-678, 2025.

[38] T. T. Nguyen, T. M. Hoang and P. T. Tran, "Secrecy Performance Optimization for UAV-based Relay NOMA Systems with Friendly Jamming," Comput. Commun., vol. 235, p. 108086, 2025.

[39] W. Guo, H. Zhao and Y. Tang, "Testbed for Cooperative Jamming Cancellation in Physical Layer Security," IEEE Wireless Commun. Lett., vol. 9, no. 2, pp. 240-243, 2020.

[40] Q. Dan, H. Lei, K.-H. Park, G. Pan and M.-S. Alouini, "Two Birds with One Stone: Beamforming Design for Target Sensing and Proactive Eavesdropping," IEEE Trans. Veh. Technol., vol. 75, no. 8, pp. 17590-17603, 2026.

[41] S. N. Quoc et al., "Enhancing Secrecy Performance in Multi-user Multi-antenna Systems Using Rate-splitting Multiple Access," Digit. Signal Process., vol. 173, Article no. 105943, 2026.

[42] Y. Zou, X. Wang, W. Shen and L. Hanzo, "Security versus Reliability Analysis of Opportunistic Relaying," IEEE Trans. Veh. Technol., vol. 63, no. 6, pp. 2653-2661, 2014.

[43] B. Li, Y. Zou, T. Wu, Z. Zhang, M. Chen and Y. Jiang, "Security and Reliability Trade-off of NOMA Based Hybrid Satellite-terrestrial Network with a Friendly Jammer," IEEE Trans. Veh. Technol., vol. 74, no. 2, pp. 3439-3444, 2025.

[44] V. T. Ty, P. N. Son and T. T. Duy, "Secrecy Performance of RIS-assisted Wireless-powered Systems with Artificial Jamming Generation," Phys. Commun., vol. 69, Article no. 102592, 2025.

[45] B. C. Nguyen et al., "Secrecy Performance of Multi-RIS-assisted Wireless Systems," Mob. Netw. Appl., vol. 28, no. 3, pp. 1206-1219, 2023.

[46] F. Xiao, P. Chen, S. Xu, X. Pang and H. Liu, "Physical Layer Security of STAR-RIS-aided RSMA Systems," Phys. Commun., vol. 61, p. 102192, 2023.

[47] M. V. Katwe, R. Deshpande, K. Singh, M.-L. Ku and B. Clerckx, "RSMA-enabled Aerial RIS-aided MU-MIMO System for Improved Spectral-efficient URLLC," IEEE Trans. Veh. Technol., vol. 74, no. 2, pp. 3110-3127, 2025.

[48] C. Pan et al., "Intelligent Reflecting Surface Aided MIMO Broadcasting for Simultaneous Wireless Information and Power Transfer," IEEE J. Sel. Areas Commun., vol. 38, no. 8, pp. 1719-1734, 2020.

[49] H. Chang, H. Yang, S. Xu, X. Pang and H. Liu, "Covert Communications in STAR-RIS-aided Rate-Splitting Multiple Access Systems," Phys. Commun., vol. 64, p. 102342, 2024.

[50] B. Li et al., "Securing Multiuser Communications via an Energy Harvesting Node: Jammer or Relay?," IEEE Trans. Veh. Technol., vol. 72, no. 7, pp. 8755-8769, 2023.

[51] T. Q. Duong, T. T. Duy, M. Matthaiou, T. Tsiftsis and G. K. Karagiannidis, "Cognitive Cooperative Networks in Dual-hop Asymmetric Fading Channels," Proc. of the 2013 IEEE Global Communications Conference (GLOBECOM), pp. 955-961, Atlanta, USA, 2014.

[52] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables, US Government Printing Office, 1968.