To my knowledge, as Silver is not purchasable from the JP Store and you're using a JP Account (which means you'd have to purchase it from the JP store), you only have one real option if you wish to continue to use your JP PSN ID. You'll need to set up Cross-Save with your JP PSN ID and either a Steam ID (if you have access to Steam) or an XBL ID (if you have access to an Xbox One). Once you have Cross-Save set up, you can download the game on that platform. Please make sure you use a Steam ID or an XBL ID that can purchase Silver, or you'll still have the same issue. Once done, you can also purchase Silver on that platform. When you log into the game on that platform, your Silver purchase will be applied to the Characters, and then you can purchase the Season Pass. Once the Season Pass has been purchased for the characters and applied, the Season Pass will transfer with those characters through Cross-Save for when you go back and play on the PS4. Unfortunately, this does mean you'll need either a PC or an Xbox One to install and log into the game after your Silver purchase to get it applied and then use it to purchase the Season Pass in-game, but once you've done so, you're good. I would highly recommend purchasing the Beyond Light Deluxe on your JP Account in order to gain the Season Passes to that account upon logging in in order to avoid having to repeat the above steps to get future Season Passes applied, because Beyond Light will work the same as Shadowkeep (Only the actual owning account will gain the included Season Pass unlocks, no other shared accounts will).