My best friend's husband is a sports fanatic. He can rattle off stats, he holds season tickets to a certain team's games, he collects sports memorabilia, and he dresses up for home games - and yet he derides her for being a fan and attending Trek conventions, because that's just "weird". Any attempts to draw the parallels between his behavior and hers is met with complete stonewalling. He absolutely refuses to see it, because hey, at least sports are real, not silly stuff with aliens.

So yeah, there's some kind of wall that prevents a lot of them from seeing that being fanatical about whatever it is they're fanatical about makes them no different from us.