Inter-channel nonlinearity compensation in long-haul optical fiber communication using overlapped-dense LSTM network: Complexity reduction and visual understanding.

The nonlinear distortion resulted from the Kerr effect in optical fibers limits the transmission capacity in long-haul optical fiber communications. In wavelength-division multiplexing (WDM) systems, the inter-channel nonlinear distortion is generally stronger than the intra-channel one, and also more difficult to be compensated, despite that there exist many reports on the nonlinearity compensation methods. Recently, recurrent neural networks have emerged as a potential solution without relying on detailed transmission link information. However, these networks often suffer from high computational complexity and lack interpretability due to their black-box nature. In this paper, we propose a novel overlapped-dense long short-term memory network (OD-LSTM). It can track the correlation of cross-phase-modulation (XPM) and consequently is capable of compensating the inter-channel nonlinear distortion. For a lower complexity, the overlapped-dense structure reduces the hidden units at time steps of different regions, since the correlation of XPM is reduced for the more distant symbols. To evaluate the effectiveness of our proposed scheme, we conduct a numerical simulation on a nine-channel WDM coherent optical transmission over a 1600-km standard single-mode fiber (SSMF) with DP-16QAM at 40 Gbaud. At the optimal optical power of −3 dBm, we observe a 0.52 dB Q-factor gain with the OD-LSTM equalizer, outperforming the digital back-propagation (DBP) with three steps per span and the Co-LSTM regressor. In terms of complexity, our proposed approach achieves a 13.5 % reduction compared to the Co-LSTM classifier, and its complexity is slightly lower than that of the ultralow complexity Co-LSTM regressor. We also discuss the visual understanding of the OD-LSTM, which provides insights into the internal mechanisms of the network. [ABSTRACT FROM AUTHOR]