I don't know, it seems to me like Marvel movies and Star Wars are both so mainstream and popular that it should be okay for someone-- or even most people-- to just have a passing interest in them. /shrug??

Of course. But I figure, it's like sports. There are plenty of people who are only casual fans of any particular sport. But fans of that sport are often shocked that there are people who AREN'T hardcore fans. It's so mainstream that they assume that everybody knows and loves it. People who hate sports they can maybe, sometimes, understand. But their default is "everyone knows and loves this and thinks it's important." So someone who's only a casual fan with a passing interest ... does not compute.