The diagnosis of liver cancer is one of the most attractive fields in clinical practice for its high mortality. Accurate segmentation of liver and tumor has been publicly accepted to be an effective method to assist doctors in determining the disease condition and planning the subsequent treatments. Recently, deep learning based methods have been widely used in tumor segmentation and provided good performance. However, current methods cannot fully reflect the differences between tumor, inside-liver tissues and outside-liver organs simultaneously, while the extraction of features reflecting axial changes of liver and tumor is always discounted by the heavy computational burden, resulting in limited learning effects and efficiencies. To solve these problems, in this paper, we propose a novel framework to segment liver and tumors in abdominal CT volumes, which consists of two parts: 1) we propose a multi-branch network where an up-sampling branch for liver region recognition and a pyramid-like convolution structure for inner-liver feature extraction are integrated into the back-bone Dense UNet structure for better extracting intra-slice features of liver and tumors; 2) we simplify the traditional 3D UNet by using the convolutional kernels with the fixed size 3 × 3 in x-y plane and apply it as a 3D counterpart for aggregating contextual information along the z-axis from the stacked, filtered CT slices, with the advantages of inhibiting the influence from neighboring pixels and alleviating the computational burden greatly. The above two parts are formulated as a unified end-to-end network so that the intra-slice feature representation and the inter-slice information aggregation can be learned and optimized jointly. Furthermore, we novely define a loss function combining a modified dice loss and a contour-detection based loss, so that the region features and contour features of the predicted segmentation of liver and tumors are jointly considered for network training and parameters optimization. Experimental results on the MICCAI 2017 Liver Tumor Segmentation Challenge dataset and 3DIRCADb dataset have demonstrated that the proposed method can provide superior performance to the state-of-the-art methods with respect to the certain benchmarks for liver and tumor segmentation. [ABSTRACT FROM AUTHOR]