Speaking from personal experience, buttons are a pain when you [b][u]really[/u][/b] have to go.
Zipper is faster, as long as you don't go commando you're fine.
I like zippers because of the convenience, but buttons are far more comfortable. I'm big down there and lemme tell ya, button fly jeans got hella dick room.