With widespread abuse of antibiotics, bacterial resistance has increasingly become a serious threat. Acinetobacter baumannii has emerged as one of the most important hospital-acquired pathogens worldwide. Bacteriophages (also called “phages”) could be used as a potential alternative therapy to meet the challenges posed by such pathogens. Endolysins from phages have also been attracting increasing interest as potential antimicrobial agents. Here, we isolated 14 phages against A. baumannii, determined the lytic spectrum of each phage, and selected one with a relatively broad host range, named vB_AbaP_PD-6A3 (PD-6A3 for short), for its biological characteristics. We over-expressed and purified the endolysin (Ply6A3) from this phage and tested its biological characteristics. The PD-6A3 is a novel phage, which can kill 32.4% (179/552) of clinical multidrug resistant A. baumannii (MDRAB) isolates. Interestingly, in vitro, this endolysin could not only inhibit A. baumannii, but also that of other strains, such as Escherichia coli and methicillin-resistant Staphylococcus aureus (MRSA). We found that lethal A. baumannii sepsis mice could be effectively rescued in vivo by phage PD-6A3 and endolysin Ply6A3 intraperitoneal injection. These characteristics reveal the promising potential of phage PD-6A3 and endolysin Ply6A3 as attractive candidates for the control of A. baumannii-associated nosocomial infections.