The performance of optical communication system is mainly determined by the various modulation formats in L band at different bit rates. For analyzing these conditions  Optisys software is used for simulation of  optical communication using WDM (Mux.) and WDM (Demux.) by comparing with Return-to-zero(RZ), Non-Return-to-zero(NRZ) modulation formats. It has been investigated that in L band  the distance covered with more data rates in RZ formats gives better results as compare with NRZ. The high quality factor is obtained in case of RZ format. The study of optical communication systems is multidisciplinary involving a wide range of area including: optical communication design, channel modelling, communications and information theory. The value of wavelength with RZ at 35 Gbps is 1625 nm  gives a bit error rate of 5.187e-010.