It's Only a Theory

Don't you get fed up with people when they say; "It's only a theory," when referring to a scientific theory, and usually in reference to the theory of evolution?