Gauge symmetry is not a symmetry?

I have read before in one of Seiberg's articles something like, that gauge symmetry is not a symmetry but a redundancy in our description, by introducing fake degrees of freedom to facilitate calculations. Regarding this I have a few questions: Why is it called a symmetry if it is not a symmetry? what about Noether theorem in this case? and the gauge groups U(1)...etc? Does that mean, in principle, that one can gauge any theory (just by introducing the proper fake degrees of freedom)? Are there analogs or other examples to this idea, of introducing fake degrees of freedom to facilitate the calculations or to build interactions, in classical physics? Is it like introducing the fictitious force if one insists on using Newton's 2nd law in a noninertial frame of reference?