Allo there, If Xbox is where your main save file is then removing it from your linked accounts will just stop you accessing them on steam and you'd need to start fresh on steam or just play those characters on Xbox as cross save doesn't transfer save data between accounts.