Why is nuclear waste more dangerous than the original nuclear fuel?

I know the spent fuel is still radioactive. But it has to be more stable than what was put in and thus safer than the uranium that we started with. That is to say, is storage of the waste such a big deal? If I mine the uranium, use it, and then bury the waste back in the mine (or any other hole) should I encounter any problems? Am I not doing the inhabitants of that area a favor as they will have less radiation to deal with than before?