The two teams are rarely ever in the same reality, and this might be a result of an odd anomaly.

Thanks to recent trends that have been established in the Marvel Cinematic Universe, it seems like diverse Marvel projects, those produced by other studios before and parallel to the MCU are part of the same multiverse. Given that it has been established that the multiverse functions on a ‘What If’ mechanism (basically there is a norm that things deviate from rather than being radically different from the get-go), it begs the question of why some concepts exist in certain universes, but are absent from others.

HYDRA in the MCU
For example, it seems that whichever universe has the X-Men, usually does not have the Avengers, and the universes that have a version of the Avengers, usually end up without a unified team of mutants. This has been speculated about by fans, and the conclusion that has been reached might be a little more out there than what most would expect. The line that keeps the universe from uniting Avengers and the X-Men, might just be the organization that is out to get Captain America’s blood: HYDRA.

HYDRA’s presence in the timeline is instrumental to the creation of the Avengers

The original six Avengers in 2012’s The Avengers
The theory on Reddit goes something like this: Key members of the Avengers can all trace their origins back to HYDRA. Because of the creation of the organization, the Allied forces were forced to create the SSR which gave Steve Rogers his powers. Further, the Hulk was a direct result of recreating the super soldier serum that gave Captain America his abilities. Thanks to the existence of SHIELD, the successor to SSR, Black Widow had an organization to defect to along with Hawkeye.

Above all else, the SSR gave Howard Stark the tech he needed to lay the groundwork for future technology, the kind that would directly allow Tony Stark to create the Iron Man suit, hence rounding up the Avengers. HYDRA is also responsible for messing with the tesseract, that gets Loki and the Asgardians directly involved, hence allowing Thor to enter the fray and round out the original Avengers roster.

Further, HYDRA deposes Hitler before his plans of genocide are carried out, making it so that mutants like Erik Lehnsherr and Charles Xavier never meet and never discover their powers, therefore removing the formation of the X-Men from the universes where Hydra exists.

Without HYDRA, the X-Men rise to become the premiere heroes of Earth

X-Men ’97
Without HYDRA, Hitler goes ahead with his genocidal plans, causing mutants like Magneto to come into his abilities, which suddenly puts mutants on the radar of governments around the world. This allows for mutants to organize themselves in the face of government scrutiny, creating the X-Men under Charles Xavier. Of course, a more radical faction forms under Magneto, bringing the mutants to the forefront of the conflict.

As mentioned above, the absence of HYDRA makes it difficult for the Avengers to come into play, which allows characters to coexist, but not function as superheroes in the same timeline. While there are exceptions to this, as Earth 616 sees the two populations co-exist, it seems to be true for most other Universes, as the two teams rarely ever share a continuity together.