With the growing efforts in isolating solid-state qubits from external decoherence sources, the origins of noise inherent to the material start to play a relevant role. One representative example are charged impurities in the device material or substrate, which typically produce telegraph noise and can hence be modelled as bistable fluctuators. In order to demonstrate the possibility of the active suppression of the disturbance from a {\em single} fluctuator, we theoretically implement an elementary bang-bang control protocol. We numerically simulate the random walk of the qubit state on the Bloch sphere with and without bang-bang compensation by means of the stochastic Schr\"odinger equation and compare it with an analytical saddle point solution of the corresponding Langevin equation in the long-time limit. We find that the deviation with respect to the noiseless case is significantly reduced when bang-bang pulses are applied, being scaled down approximately by the ratio of the bang-bang period and the typical flipping time of the bistable fluctuation. Our analysis gives not only the effect of bang-bang control on the variance of these deviations, but also their entire distribution. As a result, we expect that bang-bang control works as a high-pass filter on the spectrum of noise sources. This indicates how the influence of $1/f$-noise ubiquitous to the solid state world can be reduced., Comment: 4 pages, 3 figures; update of references, slight changes of figure layout