I have all male characters because I like the male model more than that of the female. If I was female, it'd probably be the other way around. Does that mean I care if one gender uses a character of the opposite? Absolutely not, do what you want.