It's impossible to unlink the account without the account credentials. It is, unfortunately, stuck on your account. Bungie cannot remove this account linked as well. It's third party and Bungie has no access to view your information only to receive information. Sorry. You could go through a rather length process of using a new forum account and linking your PS4 account to that forum account instead, and leaving the Xbox account behind in that regard.