Transceiver design for multi-user multi-antenna two-way relay cellular systems

In this paper, we design interference free transceivers for multi-user two-way relay systems, where a multi-antenna base station (BS) simultaneously exchanges information with multiple single-antenna users via a multi-antenna amplify-and-forward relay station (RS). To offer a performance benchmark and provide useful insight into the transceiver structure, we employ alternating optimization to find optimal transceivers at the BS and RS that maximizes the bidirectional sum rate. We then propose a low complexity scheme, where the BS transceiver is the zero-forcing precoder and detector, and the RS transceiver is designed to balance the uplink and downlink sum rates. Simulation results demonstrate that the proposed scheme is superior to the existing zero forcing and signal alignment schemes, and the performance gap between the proposed scheme and the alternating optimization is minor.