This is the point where things start to get a little confusing. First off, I want to make it clear that in any given story, none of these things are exclusive. You can have the Gross-out, the Horror, and the Terror all in the same story, sometimes even in the same scene.
The Horror is different from the other two because it's trying to play on our fears and elevate them to higher levels than normal. For instance, many people are afraid of snakes and lizards. The thought of something scaly just unnerves them. So, what does it do to people who are already afraid of reptiles when they see something like this?
Whereas the Gross-out is going for something to make your stomach turn, the Horror is trying to activate your fears. That's one reason people gravitate towards certain writers and not others. The monster horror grips them in ways the crime thriller doesn't. Or vice-versa if the person is more afraid of who their neighbors might be than of snakes, or spiders.
This definitely isn't just about creepy crawlies though. It can be any fear. The fear of being alone, the fear of drowning, even the fear of being in a crowd. The Horror is about finding what scares you and shoving it right up into your face.
That makes the Horror very different from the Gross-out. It's not going for the gut-punch that makes you queasy or makes you turn away in disgust. This is going for actual fear.
I think this is the level that most people actually enjoy. It's the fear that's embodied by movies such as Pacific Rim and Godzilla, where there's the "fun" fear of being crushed by a giant monster, or even of some unstoppable predator that's trying to eat us, a fear that used to be very based on reality but that has dwindled over the years.
It's the fear of natural disasters such as tsunami's and earthquakes. The movies that are Horror movies if you actually stop to think about it, (seriously, imagine the sheer numbers of dead people in Pacific Rim), but that Hollywood usually labels as Action or Thrillers.
Aliens has a lot of the same things as Alien had. The claustrophobic setting, the monsters, and yet, while this is undoubtedly a Horror movie, it gets listed as an Action movie first. If anything the sequel is even more of a Horror movie, because it shows all our preparations are for naught.
Still, it's more fun to think we can fight the Horror, than to just sit back and be terrified of it, even if we fail miserably.
And that's why I think this kind of horror is generally the most popular.
But that's just my opinion, I could be completely off base. What do you think?