Let ‖ A ‖ max : = max i , j | A i , j | denote the maximum magnitude of entries of a given matrix A. In this paper we show that max { ‖ U r ‖ max , ‖ V r ‖ max } ≤ ( C r ) 6 r N , where U r and V r are the matrices whose columns are, respectively, the left and right singular vectors of the r-th order finite difference matrix D r with r ≥ 2 , and where D is the N × N finite difference matrix with 1 on the diagonal, −1 on the sub-diagonal, and 0 elsewhere. Here C is a universal constant that is independent of both N and r. Among other things, this establishes that both the right and left singular vectors of such finite difference matrices are Bounded Orthonormal Systems (BOSs) with known upper bounds on their BOS constants, objects of general interest in classical compressive sensing theory. Such finite difference matrices are also fundamental to standard r th order Sigma-Delta quantization schemes more specifically, and as a result the new bounds provided herein on the maximum l ∞ -norms of their l 2 -normalized singular vectors allow for several previous Sigma-Delta quantization results to be generalized and improved.