The reason you're waiting so long isn't because you're further down in the Queue, it's because you're actually one of the first in line, there's just very few guides out there doing the guided games that you'd be able to join. You'll stay waiting in the queue until a guide joins to host a raiding party, and until then, you wait. It's extremely difficult to get a good raid party going through the guided raids, especially with the seemingly random disconnects that some people get that cause the debuff and make it unjoinable for a certain amount of time. As the other poster said, you'd have much more luck with the raid by setting up a fireteam ahead of time. The Guided Raid and Guided Nightfall thing was a nice idea, but because of how they have it limiting the way that it does, with more downside than upside, most people seem to avoid it. It sucks, but right now, that's how it is. Good luck to you!

Just curious...why not use the #fireteam forum on here? Will be quicker & easier, plus you can play while waiting/looking for a group.