A lot of trans men and other transmasc individuals claim that there is a systemic hatred and oppression of them in society alongside transphobia, and have coined the term transandrophobia to describe this phenomenon. The term mimics the existing term transmisogyny, which describes the particular oppression that transfem individuals experience.
It goes without saying that transandrophobia is hogwash, a completely useless term that presupposes that men are in any way systemically oppressed for being men as the "androphobia" part suggests. Some men are marginalized under racism, classism, homophobia, and other forms of bigotry and intersections thereof, but not for the sole reason of being men, because the patriarchy elevates all men over women on every conceivable axis, and men who are oppressed on a particular axis still hold privilege over women on that same axis. Androphobia/misandry exists only in the form of the justified contempt for a privileged social class, and is not a systemic force like misogyny is.
Julia Serano defines transmisogyny in her seminal work, Whipping Girl, describing how trans women are often excluded from women-only spaces, are dismissed as targets of domestic and sexual violence, and are categorically excluded from conversations about transness and feminism. Trans men are often gladly welcomed into these spaces, because it's not transness alone that excludes trans women. This is the primary form of oppression that specifically affects trans women, and trans men and transmascs are exempt from. While non-transfeminine individuals may experience some intersection of transphobia and misogyny, this is different from transmisogyny, which is its own axis of marginalization.
Trans men, leveraging either their maleness or cissexism (by dismissing their own identity to align with cis women), hold power over trans women, who have no avenues to exercise social power over anyone (and especially so if they're not white). Despite what bigots try to claim, trans women do not have male privilege in any form, because their femininity and rejection of masculinity both deny them access to it. Cis women, particularly white cis women, have essentially claimed stewardship over rape victimhood, which they use to reject and defame transfeminine victims of sexual violence, again especially non-white ones. This in turn makes trans women even more prominent targets of sexual violence. In situations where a community has turned against a trans woman for any reason, her only option to avoid further harassment and denigration is to simply leave.
This issue overlaps with a broader problem where the anger expressed towards a privileged social class like men, white people, heterosexuals, or even Christians, is erroneously interpreted as oppression. Oppression is systemic power, not just being mean to someone, and non-men, people of color, queer people, and non-Christians don't have the systemic power to oppress their oppressors. Misogyny is present in queer spaces just the same as outside them, and a lot of openly queer people are going to be openly critical of the patriarchy and men. Anyone who sees expressions of hatred of men in queer spaces and attributes it to an oppression of masculinity within the community has deluded themselves into thinking that queer culture is in some way uniquely positioned outside of the patriarchy and that their privileges don't carry over, or that they're somehow absolved of being men because they aren't personally and actively responsible for someone's marginalization. This victim mindset is simply another expression of patriarchy through devaluing the words of women in order to shield oneself from criticism of masculinity. A lot of trans men try to separate themselves from cis men to weasel themselves out of the same situation, seemingly not understanding that they're engaging in the clearest form of transphobia possible.
Sadly, trans men are men, and are deserving of the same reproach that targets all other men. And y'all cry about it just the same.