In a post-COVID-19 world, extensive study of deep learning-based biometric authentication techniques prompts the need to secure them. Further, the biometric data is assumed to be largely immutable; thus, if it is compromised, it is lost forever. Hence, reliable and secure biometric authentication is of utmost importance. In this paper, we address the security and privacy concerns of biometric templates generated via deep networks. We propose a cancelable biometric authentication approach. The framework consists of a lightweight Convolutional Neural Network (CNN) with a few-shot enrollment for generating biometric templates. Further, to enhance biometric templates’ discriminative power and to provide revocability, biometric templates are projected onto a random subspace (based on the user-specific key). Later projected biometric templates are mapped onto robust n - bit unique codes (using a KNN classifier) and protected via. SHA-3 hash digest. Moreover, a real-world biometric authentication system is always dynamic (users keep on changing). Thus we have also integrated phase-wise incremental learning within a deep learning-based cancelable biometric authentication framework. This is the first work in which deep cancelable templates are generated incrementally to the best of our knowledge. We analyze the proposed scheme for its performance and privacy preservation on three benchmarks constrained iris data-sets and over one unconstrained iris data-set along with one publicly available knuckle data-set. Furthermore, it has been demonstrated that the proposed cancelable incremental framework strictly follows the four fundamental properties of cancelability viz. non-invertibility, unlinkability, revocability, and usability.