I've only had problems with threshers in one of the side missions. It had a time limit and I got ambushed by a ton of them at one of the locations I had to go, not even my turret could help me there.
Now, for annoying enemies, nothing beats rats IMO, there's always a ton of them, they can deal a ton of damage, steal all the loot before you even get to it and then there's the lab rat which goes all cyclops on you when you let it get close and that attack can easily kill you.