I think a way to fix this is to make it clearer to other players what activity you are interested in doing BEFORE you load it. So you could look through your roster and see someone is in orbit around Dragon Strikes, or someone is in orbit around the Crucible - and they need one or two more players and whatnot. That way you can see who is doing things that are people you have already played with and can delete if they are a jerk.