You realise if you stand at the far back of the swordbearer's spawn room that raises the bridge too, right? Just stay there for 10 seconds and everything will despawn and just proceed as normal. Easy.