What benefits do governments receive from not eliminating debt?

Most major economies have substantial government debt; for example among the OECD countries the lowest debt rate is Estonia with around 6% of GDP. Now we may be in a global recession at the moment, but even during the more prosperous times of the twentieth century most countries maintained substantial public debt. Here is a graph of the US federal debt: And here is one of the UK's national debt: (Both graphs via Wikimedia.) What benefits do governments of advanced economies receive from not eliminating their debt?