First thing is you have to kill the servitors that spawn on the left or right to weaken him. You have to do a bunch of damage and then Skolas loses his buff. Also make sure you are at least level 33. If it is not the actual mechanics of the fight just internet problems have 1 person stay in the lobby while everyone else goes to orbit so you can rejoin and try to fix your connection.