Sorry to break this to you, you would not be safe. Maybe safer from sexual assault and/or rape, but you would be more than 3 times more likely to die from homicide from a stranger. Every time a man who you don't know intercedes on your behalf from another man, he's risking his life.

If he's smaller or weaker than the man who is harassing you, he's no different than you. The bigger man will over power him. Being male is not the suit of armor you think it is.