The worth of the currency is roughly a fraction of the GDP (see
Fiat Money). So the value of the money grows or diminishes, depending on the supply of currency.
The more dollars you print, the less each dollar is worth. However, you can print more dollars and push them into the economy, where they represent new income chasing after goods and services produced by businesses.
...which increases GDP, and the sub-conclusion that "The more dollars you print, the less each dollar is worth" remains true only for
constant GDP.
If the dollars you "print" are spent in a way that increases GDP by
less than their face value, the value of the dollar falls (inflation).
But most government spending increases GDP by
more than the face value of the money that spending creates. This increases the value of money (deflation).
Deflation is bad. So governments avoid the deflationary effect of the GDP boost that comes from spending, by destroying some money. This destructive process is called "taxation".
Spending (particularly on infrastructure in the broadest sense) implies growth; Growth is good; Deflation is bad; Taxes prevent deflation; Taxes are good.
Austerity - low tax, low spending - is a recipe for disaster. It leads to inadequate and shoddy infrastructure, and the entire economy depends on infrastructure.
Ideally, taxes should be lower than spending, in proportion to the rate of growth of the economy. Or to look at it another way, growth will occur proportional to the size of the defecit, as long as that spending is directed at things that sustainably grow the economy. And it's quite hard to find something to spend money on that
doesn't cause sustainable growth.
Of course, there is another source of money - borrowing. And that money creation is offset by debt repayment. So the money supply can also be "tuned" to GDP growth by manipulating interest rates to encourage or discourage the lending of money. So it's more complex than that ideal. But a balanced budget (at the level of the currency issuer - ie in the case of the USA, the federal budget) remains a bad thing for a growing economy.
State and city budgets are more like business budgets, and balancing those is generally a good idea, though as income to such sub-national entities can come from the federal budget, as well as from local taxation, the total spend can still be higher than the local tax receipts, with a budget that is nevertheless "balanced".
Limiting spending to tax receipts is a terrible idea, and will fuck up your economy in short order. But it is a
simple strategy. And so simpletons love it, despite it's being a total disaster.