Those happen when you take to long to get away from him.
Every video I saw that had this happening he went for a running swing at the sword bearer and slid off the side.
I haven't had this happen since the first week.
We just make sure to have the same few players host it.
If you have a good host the bugs go away.
Even the best host can have a bad connection to some people so finding the right host is key.