First of all, I like the way you write/reason things. You have a natural way of thinking out loud that enables your reader to follow your train of thought. It’s very reasonable. However, I do believe the notion that men should be giving up this privilege is indeed what he is disagreeing with. If you read Peggy McIntosh’s essay, she does describe not being singled out because of race or gender as an unearned privilege.