Militia must be deleted or at least should be avaible by respawn every (for example) 1hour, not via buy from sheriff.
Yes this would be for the best, even if it's just 10 minutes, much better than spamming them out.
Or even, every 10 minutes you can buy a new one sort of thing, or it build up, so in an hour you can buy 6.
Having the guards protect whoever is being attacked is weird, why would someone buy town guards that might attack them?