Nonlinear-Precoded Multiuser Secure Transmission with Cooperative Jamming and Adaptive Limited Feedback

This paper studies secure transceiver design for multiuser multiantenna systems with an external passive eavesdropper and a cooperative jamming helper. Due to the finite-rate constraint of feedback channels, only quantized channel state information (CSI) of the legitimate users is available at the transmitter and the helper. A nonlinear-precoded secure transmission strategy is proposed using Tomlinson-Harashima precoding at the transmitter and null-space beamforming at the helper based on the quantized CSI. The accurate closed-form expression of an approximation for the ergodic rate of each legitimate user is obtained using quantization cell approximation for random vector quantization of the channels. Assuming the quantized CSI of the legitimate channels and the helper's channel at the eavesdropper, closed-form expression of an upper bound of the ergodic rate of each user's message at the eavesdropper is also derived. Then, a closed-form expression of an approximation for the worst-case ergodic secrecy sum rate follows. We also theoretically show that, besides the advantage in ergodic rate over the linear precoding scheme, when the quantized CSI is not available at the eavesdropper, Tomlinson-Harashima precoding is also more effective in degrading the received signal quality at the eavesdropper, and, thus, is more capable to enhance the secrecy of the systems compared with the linear precoding scheme. Given the total bandwidth constraint of CSI-feedback channels, an adaptive feedback bit allocation algorithm is proposed for each legitimate user to optimize the ergodic secrecy rate performance. Numerical results illustrate that the proposed nonlinear-precoded secure transmission strategy outperforms the corresponding linear precoding scheme, and significant advantage can be achieved by adaptively allocating the total available feedback bits.