To be fair, it's not just men. And it's not necessarily driven by men either.
https://medium.com/sexography/more-women-than-men-want-to-make-abortion-illegal-c8c4187f23bc
It's driven by religious fundamentalists and supported by members of both sex that are part of that group. We need the support of both men and women to vote people into office that will give women back their federally protected right to an abortion. Why alienate men?