When using the controller "Swap to Power Weapon" bind (long press Y) you always go into 3rd person when trying to go from your Special to Power weapon (if you have a sword equipped in the other slot). Not sure if this can be fixed, but figured I'd provide the feedback that it can be very jarring, especially when you're switching back and forth quickly (when you don't want to actually equip the sword it's just kinda motion sickness-inducing).