Atmospheric sounds, moody imagery and building tension really do it for me.
I've played a number of horror games in my time, some of which I found boring that other people love (Silent Hill). But I've almost never been as freaked out as I have been in Thief 3, during the Shalebridge Cradle mission (called Robbing the Cradle). Basically the Cradle used to be an insane asylum, and before then it was an orphanage. It caught on fire, and was abandoned. Garrett describes it as a 'house with bad dreams', which turns out to be quite right when you play through the mission.
This kinda stuff is what creeps me out. It's used quite a bit in Amnesia as well - building tension, when there really aren't any (or only few) enemies close by. The audio is also great in both games - distant screams, babies crying, voices with no visible speaker, etc.
I will admit though, that jump scenes really *do* scare me. I have a fear of loud sudden noises, so it's not the imagery that freaks me out. It's the loud noise usually accompanying it. But that's why I also play games like FEAR and Dead Space