I am led to believe that the traveler might have visited the fallen before humanity because numerous times in the grimoire a "great machine" is referred to. Along with hints to the traveler in grimoire in the house of wolves mission "the ruling house" at the boss battle in the center is a hologram of the traveler but not the city. Since the traveler visited them first they might have thought that it was a god and attempted to follow it but instead came upon us. After seeing us they thought we stole it and decided to attack us in defense of their God not to wipe us out just because. While on this topic I'd I like to address that the servitor's resemble the traveler quit a lot if you ask me. This might have been their way of worship of some sort it exalted respect.