We analytically and numerically study quasinormal frequencies (QNFs) of neutral and charged scalar fields in the charged anti-de Sitter (AdS) black holes and discuss the stability of the black holes in terms of the QNFs. We focus on the range of the mass squared μ2 of the scalar fields for which the Robin boundary condition parametrized by ζ applies at the conformal infinity. We find that if the black hole of radius r+ and charge Q is much smaller than the AdS length ℓ, the instability of the charged scalar field can be understood in terms of superradiance in the reflective boundary condition. Noting that the s-wave normal frequency in the AdS spacetime is a decreasing function of ζ, we find that if |eQ|ℓ/r+ is greater than , where e is the charge of the scalar field, the black hole is superradiantly unstable irrespectively of ζ. On the other hand, if |eQ|ℓ/r+ is equal to or smaller than this critical value, the stability crucially depends on ζ and there appears a purely oscillating mode at the onset of the instability. We argue that as a result of the superradiant instability, the scalar field gains charge from the black hole and energy from its ambient electric field, while the black hole gives charge to the scalar field and gains energy from the scalar field but decreases its asymptotic mass parameter. [ABSTRACT FROM AUTHOR]