Yep, that's correct. That's what would happen if we were declared stateless overnight. However, scaling back the state shouldn't be out of the question. It should be gradual and reasonable. Some of the things you've mentioned are already provided on private models. In some cases it's not entirely possible to provide them on private models and therein lies the value of the state and taxation. Public transport, especially in places like Sydney is a prime example. It's not possible to run a transport system profitably in our case. Would I prefer we spend a bunch of money on making new trains (inb4 autism) and buses and assisting people get from point A to point B than on useless stuff like giving political parties money for winning elections? You bet. I think you would too.
I'm one of the advocates for limiting government on this forum, but I do see uses for the state. I do think taxation is theft and is therefore intrinsically obtained immorally, but that doesn't mean we can't be pragmatic about this stuff.