Man I swear none of us were on the center plate and I was solo over on the other side, as soon as I got the ogres to spawn out the totems activate. You have to periodically run in and out of the totems while killing the ogres.