How is he annoying?

when i fought him the first time i didnt do much damage so it took me a loooong time to kill him.

What level did you beat the game at? I think I was a DEX build at level 80 when I did and Gwyn was one of the easier bosses imo. His attacks are very easy to parry and he's not a very challenging boss. I don't see why you were doing so little damage to him though. It's the final part of the game. By then you should probably have a well built character with fully upgraded weapons and gear so that you'd do tons of damage.