But they travel through time
Via few gateways. If the vex gateways don't exist in that time then they can't travel to it. They can Only travel as far back as when the gateways they use were built.
It's actually identical to a time travel device being created right now in the real world. The man wanted to go back in time to save his dad but he knows with the device he couldn't go further back Then when he created it. Everything has it's limit, even time travel.