You have like 5 seconds after first warning (in red) to holster weapon, that's not the problem. I think it's the same in a started TB fight, you have the same time but no warning. And as you can't move or do any action they will consider you are hostile and shoot.