Background Left ventricular hypertrophy (LVH) is an independent prognostic factor for cardiovascular events and it can be detected by echocardiography in the early stage. In this study, we aim to develop a semi-automatic diagnostic network based on deep learning algorithms to detect LVH. Methods We retrospectively collected 1610 transthoracic echocardiograms, included 724 patients [189 hypertensive heart disease (HHD), 218 hypertrophic cardiomyopathy (HCM), and 58 cardiac amyloidosis (CA), along with 259 controls]. The diagnosis of LVH was defined by two experienced clinicians. For the deep learning architecture, we introduced ResNet and U-net++ to complete classification and segmentation tasks respectively. The models were trained and validated independently. Then, we connected the best-performing models to form the final framework and tested its capabilities. Results In terms of individual networks, the view classification model produced AUC = 1.0. The AUC of the LVH detection model was 0.98 (95% CI 0.94–0.99), with corresponding sensitivity and specificity of 94.0% (95% CI 85.3–98.7%) and 91.6% (95% CI 84.6–96.1%) respectively. For etiology identification, the independent model yielded good results with AUC = 0.90 (95% CI 0.82–0.95) for HCM, AUC = 0.94 (95% CI 0.88–0.98) for CA, and AUC = 0.88 (95% CI 0.80–0.93) for HHD. Finally, our final integrated framework automatically classified four conditions (Normal, HCM, CA, and HHD), which achieved an average of AUC 0.91, with an average sensitivity and specificity of 83.7% and 90.0%. Conclusion Deep learning architecture has the ability to detect LVH and even distinguish the latent etiology of LVH.