If I am correct the armory was there before we moved in to the new tower, so it makes sense that the vanguard never decided to bust down an apparently normal wall in their new home.