Hardware-Efficient and Performance-Enhanced Joint Pulse Shaping and Dispersion Compensation for Coherent Data Center Interconnects

1

Hardware-Efficient and Performance-Enhanced Joint Pulse Shaping and Dispersion Compensation for Coherent Data Center Interconnects

arXiv:2605.25818v1 [cs.IT] 25 May 2026

Yukun Zhang, Xiaoxue Gong, Weigang Hou, Xu Zhang, Lei Guo

Abstract—With the explosion of data traffic triggered by 5G/6G and Generative artificial intelligence, coherent optical communication is moving towards higher baud rates and more complex modulation formats. This leads to a significant increase in the computational complexity and power consumption of digital signal processing (DSP) at the transmitter and receiver ends, especially in the chromatic dispersion(CD) Compensation and low roll-off shaping filter modules. We propose a joint shaping filtering and CD compensation (JFS-CD) algorithm. This algorithm moves the CD compensation to the transmitter side and utilizes the characteristics of discrete fourier transform and the spectral features of shaping filtering for integrated processing. Aiming at the high peak-to-average power ratio (PAPR) problem caused by chromatic dispersion pre-compensation, we propose a low-complexity square boundary clipping algorithm(SBC). Simulation results show that, under the premise of maintaining unchanged performance, JFS-CD can reduce the real multiplication complexity by about 46%. Meanwhile, benefiting from the suppression of the effects of system nonlinearity and receiver IQ imbalance, the joint JFS-CD and SBC scheme improves the Q-factor by about 0.3 dB in experiments compared to the traditional post-chromatic dispersion compensation scheme. This research provides a highly potential transmitter DSP solution for next-generation low-power and high-performance data center interconnects (DCI). Index Terms—Coherent optical transmission

I. I NTRODUCTION

D

RIVEN jointly by 5G/6G communication, large-scale cloud computing, ultra-high-definition video streaming, and generative artificial intelligence large models, global data traffic is experiencing explosive growth. As the core foundation of information infrastructure, coherent optical communication systems are being forced to evolve towards higher rates, wider spectrum, and more complex modulation formats to support this massive demand. However, this increase in signal rate directly leads to a sharp rise in the load of coherent digital signal processing (DSP) chips. In DSP demodulation algorithms, the CD compensation occupies a large amount of power consumption and chip area. This work was supported in part by the National Key Research and Development Program of China under Grant 2023YFB2905900, in part by the National Natural Science Foundation of China under Grants U24B20134, 62222103, in part by the Chongqing Municipal Education Commission under Grants KJZD-K202400608. Yukun Zhang, Xiaoxue Gong (Corresponding author), Weigang Hou, Lei Guo are with the School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China (e-mail: [email protected]). Yukun Zhang, Xiaoxue Gong, Weigang Hou, Xu Zhang, and Lei Guo are with the Institute of Intelligent Communications and Network Security, Chongqing University of Posts and Telecommunications, Chongqing 400065, China. Weigang Hou, Xu Zhang, and Lei Guo are with the School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China.

Traditional frequency domain equalization (FDE) technology usually adopts the Overlap-Save method. This method highly relies on large-point Fast Fourier Transform (FFT) and inverse transform (IFFT). In high-baud-rate transmission scenarios, FFT/IFFT operations not only consume huge logical resources, but also bring high dynamic power consumption. In order to alleviate this difficult problem, researchers have carried out extensive research on low-complexity algorithms in recent years. Reference [1] proposed a Roots of Unity Equalizer. By approximating the filter coefficients as equally spaced phase shifts, it relies on shifts and adders to complete chromatic dispersion compensation. Reference [2] utilizes Chirp-Filtering technology. While ensuring the compensation performance of high-order QAM signals, it reduces the computational complexity by nearly half. These schemes reduce partial signal performance in exchange for the reduction of complexity. On the other hand, the computational complexity of shaping filtering in high-baud-rate systems is also not to be ignored. In the time domain, a low roll-off factor will cause the impulse response of the filter to present an extremely slow attenuation characteristic. Therefore, in order to maintain the high quality of the signal, the shaping filter needs to use a Finite Impulse Response (FIR) filter with dozens of taps. A large number of taps brings more Multiply operations. This similarly increases the power consumption of the transmitter-side DSP. In this paper, we propose a joint shaping filtering and CD Compensation (JFS-CD) algorithm to reduce the computational complexity required for shaping filtering and chromatic dispersion compensation. In our scheme, the CD compensation module is moved to the transmitter side. It is combined with the shaping filtering to simultaneously achieve signal shaping filtering and CD compensation. By utilizing the scale transformation characteristics of the Discrete Fourier Transform and the spectral features of shaping filtering, the computational complexity required for shaping filtering and CD compensation is reduced. CD pre-compensation will lead to an increase in the PAPR of the waveform. We propose a low-complexity square boundary clipping algorithm(SBC). Experimental/simulation results show that JFS-CD can achieve shaping filtering and CD compensation without performance loss. The real multiplication complexity is reduced by about 46%. Moreover, in experiments, compared to the post-CD compensation scheme, the joint application of JFS-CD and SBC algorithms can improve the Q-factor by about 0.3 dB. II. W ORKING PRINCIPLE Figure 1 shows the principles of JFS-CD and SBC.Assume that the time-domain impulse response of the pulse shaping

2

results are combined as the final output and sent into the SBC module. Assume that the frequency-domain representation of the original baseband signal x(t) is X(ω) = |X(ω)|ejθ(ω) . θ(ω) is the modulation phase jointly determined by the signal sequence and the shaping filter. The transmitted signal after pre-compensation is s(t). Z ∞ β2 2 1 |X(ω)|ej (ωt+θ(ω)+ 2 ω L) dω (4) s(t) = 2π −∞ Fig. 1. Schematic diagram of JFS-CD and SBC.

filter of the system is h(t). Its corresponding frequencydomain transfer function is Hp (ω). The chromatic dispersion effect of the single-mode fiber can be modeled as an all-pass filter in the frequency domain. Its transfer function is   β2 HCD (ω) = exp −j ω 2 L (1) 2 β2 represents the chromatic dispersion coefficient. L represents the signal transmission length.The cascade of linear time-invariant systems is equivalent to the multiplication of frequency-domain responses. Therefore, the transfer functions of pulse shaping and CD pre-compensation are multiplied in the frequency domain to construct the simplified joint transfer function Hjoint (ω) = Hp (ω)HCD (ω)    β2 2  ω L , |ω| ≤ π(1−α) T exp j s  2 Ts  π(1+α) (2) Hjoint (ω) = 0,  |ω| > Ts    β C(ω) exp j 2 ω 2 L , otherwise 2

C(ω) =

    Ts Ts π(1 − α) 1 + cos |ω| − 2 2α Ts

(3)

α is the roll-off factor. T s is the symbol period. In the process of implementing JFS-CD, the transmitted symbol sequence is first divided into blocks. Each block contains N symbols. An N point FFT is performed on each block. According to the scale transformation property of the Discrete Fourier Transform (DFT), the N point frequency-domain data is periodically copied and concatenated according to the target oversampling rate. This generates a frequency-domain signal with a length of 2N. Then, this 2N length frequency-domain signal is multiplied point by point with Eq.2.It should be pointed out that when |ω| > π(1+α)/Ts , Eq.2 is 0. Therefore, the product of this part of the coefficients and the signal can directly be 0. This reduces the computational amount of multiplication. Finally, a 2N point Inverse Fast Fourier Transform (IFFT) is performed on the multiplied frequencydomain signal. This generates an oversampled time-domain waveform. In the IFFT stage, there are a large number of zero elements in the frequency-domain signal. Therefore, the computational complexity can be further reduced. Assume that Xm represents the time-domain signal of the m-th block after JFS-CD. After completing the IFFT, the first No points and the last No points of the Xm signal are polluted by circular convolution. Therefore, they need to be discarded. The remaining data blocks also undergo the same operation. The final

Let it be Ψ(ω, t) = ωt + θ(ω) + β22 ω 2 L. This integral term has a quadratic phase term. According to the principle of stationary phase, the main energy of this integral comes from the frequency band where the phase changes slowly with frequency. Let ∂Ψ(ω, t)/∂ω = 0 ∂θ(ω) − β2 ωL (5) ∂ω When a sufficiently long signal sequence is transmitted, some special data sequences will randomly exhibit a specific quadratic phase distribution θ(ω) in their spectrum. The derivative of this quadratic phase generates a linear term that cancels out the linear dispersion group delay −β2 ωL, thereby eliminating the variable ω. As a result, within a wide frequency range, it satisfies R− ∂θ(ω) ∂ω − β2 ωL ≈ t0 . Eq.4 can be rewritten 1 jC s(t0 ) ≈ 2π e |X(ω)| dω. At this time, ignoring the ∆ω constant phase factor, the instantaneous signal amplitude is Z 1 |s(t0 )| ≈ |X(ω)|dω (6) 2π ∆ω t=−

This means at time t0 , a high waveform peak is generated. Therefore, there are two types of ideas to reduce the PAPR of the signal. One type is to destroy the formation conditions of phase consistency at the peaks, suppressing the generation of high peak values from the source. The other type is to clip the signal. Due to the latter does not add extra system overhead. Therefore, we propose a square boundary clipping algorithm suitable for CD pre-compensated signals. The principle of the SBC algorithm can be expressed as  xAth max(|Q{x}|) , max(|Q{x}|) > Ath y= (7) x, otherwise Ath represents the threshold. Q(·) reprent real or imaginary parts. When the waveform peak exceeds the threshold, the complex signal x is scaled by the same ratio. SBC performs simple threshold comparisons through the real and imaginary values of the signal.It avoids multiplication and square root operations. This reduces the computational complexity required for SBC. III. S IMULATION SETUP AND R ESULTS In order to study the performance of our proposed JFSCD and SBC, we use 36 Gbaud signals with 2 samples per symbol(SPS) to perform simulations and experimental tests. The system simulation setup and DSP flow are shown in Fig. 2. The signal use a PRBS sequence with a length of 218 . The ROF is 0.2. The transmission laser and the LO laser are operating at 193.1 THz with a 100 kHz linewidth and a 1 GHz frequency offset. The CD parameter is 16 ps/(nm · km). The

3

DPIQ. Bias control

ICR

CPE

FOE

Resample CD Emulator DGD Emulator Set OSNR

SBC

Resample

JFS-CD

PRBS

16QAM Mapping

Laser

RV-MIMO

RX DSP

TX DSP

ADC×4

Laser

Fig. 2. System setup and DSP flowchart

Fig. 5. (a) CR performance curve test under different ROFs (b) System OSNR performance curve test under different CRs

In Fig. 4, we use a PRBS sequence with a length of 226 to conduct the CCDF performance curve study. Clipping ratio (CR) is a relative threshold setting metric based on power normalization. The calculation of CR can be expressed as ! Ath (8) CR(dB) = 20log10 p E[|x|2 ]

Fig. 3. JFS-CD Performance Test Results

Fig. 4. Test results of the CCDF performance curve for SBC

transmission distance is set to 100 km. A high-order PMD model is used to simulate the DGD effect. The OSNR is defined assuming a bandwidth of 12.5 GHz and is set to 23 dB. In order to verify the reliability of JFS-CD, Fig.3 evaluates the Q-factor performance of the system under different ROF. The evaluation compares the cascaded architecture (pulse Shaping + Pre CD compensation) and JFS-CD. As shown in the figure, under ROF=0.01, 0.1, and 0.2, the performance curves of the traditional architecture and the JFSCD architecture have a high degree of overlap. Fig.3 shows that embedding the shaping filter into the frequency-domain chromatic dispersion operator can achieve a lossless equivalent transformation. It does not introduce any additional system penalty.

As shown in Fig. 4, with the gradual decrease of CR, the CCDF curve presents a regular leftward-shifting trend. When CR=8.52 dB is adopted, it reduces the PAPR to 10.4 dB (about 0.5 dB gain) at a probability of 10−3 . When CR=6.72 dB is adopted, it obtains a peak suppression gain of 1.3 dB. In order to quantitatively evaluate the distortion cost introduced by SBC, Figure 5 shows the response curve of the SBC. As shown in the Fig.5(a), all curves present a highly consistent asymptotic behavior. Fig.5(a) shows the robustness exhibited by the SBC algorithm towards different ROF. By comparing the attenuation slopes of different ROF curves, it can be found that the Q penalty introduced by SBC is not significantly amplified with the decrease of ROF. In systems with low ROF, the signal easily generates peaks with huge amplitudes. ecause the SBC algorithm is robust to ROF changes, using the SBC scheme can bring greater benefits after CD pre-compensation in low-ROF systems. As shown in Fig.5(b), the response curves of the Q-factor varying with OSNR present significant interval differences under different CR. In the low OSNR region (<21 dB), the distortion introduced by SBC is basically masked by the noise. Therefore, the difference in the Q-factor penalty caused between different CR is relatively small. However, as the OSNR gradually climbs to 26 dB, the limiting effect of noise is greatly weakened. At this time, the in-band crosstalk caused by SBC becomes the bottleneck restricting signal quality. Therefore, in the high OSNR condition, the gap of the Qfactor penalty caused by different CR expands accordingly. Therefore, under low OSNR conditions, a low CR is used to reduce the PAPR to a greater extent. Under high OSNR, a high CR is used to reduce the in-band crosstalk brought by SBC. IV. E XPERIMENTAL SETUP AND R ESULTS We investigate the performance of the proposed scheme through experiments. The experimental setup is the same as the simulation setup shown in Fig.2. Two DFB lasers with a

4

Fig. 6. (a) Measured launched optical power under different CRs (b) Measured Q factor under different CRs

linewidth of <100 kHz and a wavelength of 1551.12 nm are used as the signal carrier and the local oscillator, respectively. An Arbitrary Waveform Generator (Keysight M8196A) at 93.4 GSa/s is adopted to generate 36 GBaud DP-16QAM signals. The ROF is 0.01. Subsequently, a dual-polarization optical IQ modulator (FTM7977)is used to modulate the signal onto the optical carrier. The signal is transmitted in 100 km of standard single-mode fiber. The optical signal is amplified by erbium-doped fiber amplifiers (EDFA) at 0 km and 60 km, respectively. At the receiver end, an Integrated Coherent Receiver (ICR, CPRV1225A) is adopted to demodulate the optical signal. Finally, an oscilloscope (DSOZ594A) with a sampling rate of 80 GHz is used to sample the signal. In the TX DSP, in order to prevent the resampled signal from entering the nonlinear region of the devices, it is necessary to perform normalization processing on the signal. Figure 6 shows the relationship between the EDFA drive current and the actual launched optical power(LOP). As shown in Fig. 6(a), compared to the signal without pre CD compensation(w/o pre comp.), the power of the CD pre-compensated signal without using SBC decreases by about 2 dBm. After using the SBC algorithm, with the decrease of the CR, the LOP of the signal is significantly improved. The improvement of power in Fig. 6(a) helps to improve the Q-factor of the signal. Fig. 6(b) shows the performance of the Q-factor under different CRs. For the signal without adopting SBC, its average power is too low. This causes its Q-factor to be at the lowest level throughout the entire testing interval. Using SBC can effectively improve the Q-factor of the signal. When CR=6.92 dB, the transmitted power is lower than that of the signal without pre-CD compensation. However, it shows a higher Q-factor quality than the signal without pre-CD compensation There are two reasons for this. First, the signal without pre-CD compensation has a larger PAPR when it reaches the receiver. This brings a stronger nonlinear effect. Second, because the signal use CD pre-compensation, the receiver IQ imbalance does not have a mixing effect with CD [3]. The receiver IQ imbalance can be compensated by RV-MIMO. The above two reasons jointly lead to the fact that the scheme proposed in this paper is better than the traditional post-CD compensation scheme in performance. V. C OMPLEXITY A NALYSIS We evaluate the computational complexity of the algorithm by the number of real multiplications required

per symbol. When the ROF is 0.01 and N is 128 points (128 symbols). JFS-CD requires a total of 8/N [N/2log2 N + (1 + α)N + N log2 (2N )] ≈ 101 real multiplications. In the shaping filter cascaded with CD compensation scheme, the shaping filter uses a 21-tap FIR filter. Each symbol requires 42 + 16log2 (2N ) + 16 = 186 real multiplications. Our proposed JFS-CD reduces the computational complexity by about 46%. When N = 64, the JFS-CD reduces the computational complexity by about 51%. Due to a small number of symbols need to be adjusted using the SBC algorithm, the increased number of real multiplications can be ignored. VI. C ONCLUSION Aiming at the DSP power consumption and computational complexity bottlenecks faced by high-speed coherent optical communication systems, this paper proposes a JFSCD algorithm. Furthermore, it introduces a SBC algorithm to suppress the high PAPR caused by chromatic dispersion precompensation. Research results show that the algorithm fusion is performed by utilizing the characteristics of Discrete Fourier Transform at the transmitter end. Under the premise of not losing system performance, this scheme successfully reduces the real multiplication complexity by about 46%. Experiments further confirm that the joint application of JFS-CD and SBC can effectively suppress the effects of system nonlinearity and receiver-side IQ imbalance. Compared to the traditional postcompensation scheme, it achieves a Q-factor improvement of about 0.3 dB. In summary, while significantly reducing the hardware computational overhead, the scheme in this paper also takes into account the optimization of signal quality. It provides a highly competitive DSP architecture reference for realizing next-generation low-power and high-performance coherent DCI. R EFERENCES [1] G. Gomes, P. Freire, J. E. Prilepsky, and S. K. Turitsyn, “Fpga implementation of low-power multiplierless pre-processing free chromatic dispersion equalizer,” in 2025 Optical Fiber Communications Conference and Exhibition (OFC), 2025, pp. 1–3. [2] A. Felipe and A. L. N. d. Souza, “Chirp-filtering for low-complexity chromatic dispersion compensation,” Journal of Lightwave Technology, vol. 38, no. 11, pp. 2954–2960, 2020. [3] E. P. da Silva and D. Zibar, “Widely linear equalization for iq imbalance and skew compensation in optical coherent receivers,” Journal of Lightwave Technology, vol. 34, no. 15, pp. 3577–3586, 2016.