That sounds like a NAT/port sharing issue. If you don't have certain things in place, the router won't know which console to send the return packets to from the internet. It might take long enough to figure out which device needs it, that the destiny servers assume host is offline or not on a good enough connection and kicks them. I had this problem too when people would bring their console over and we would play zombies or ME3 multiplayer. It works better with UPnP and port range triggering, but usually we need one of our other friends to host the game.