Why's it matter, anyway? Don't people, at least in America, have the right to determine the direction and content of their own lives, even if they do make mistakes?
Sure. A diabetic also has the right to eat as much sugar straight from the bag as he could want. A blind man has the right to drive (well, the same entitlement to the privilege of driving as a sighted man; driving on public roads isn't a right) A woman has the right to be a priest.
American jurisprudence may have a few things to suss out, but it's not the hook we hang this meat on. We all have certain capacities, certain charisms, and lack others. A mildly diabetic man could get by without medication if he didn't want to eat more sugar than he could stand, but he's not capable of reducing to the significantly different question of tooth decay like a man who happens to have a healthy pancreas. A blind man, through extraordinary hearing, might do a better job of keeping a car between the curbs than the many people who deposit their own vehicles in ditches, around trees, and all over other vehicles every day, but they're in no position to make an attempt under common circumstances. A woman may be highly capable of being a spiritual shepherd, but it's not an accident that Rome considers itself lacking the authority to extend Orders to them. In fact, we all lack the authority to circumvent the circumstances of our being by philosophical or legal fiat. Rights and privileges aren't a factor, it's the metaphysical authority, not simply the permission to exercise temporal power in an apparently similar way.
What about adoption, as long as we're here?
Homosexual couples might do a better job of raising kids than some heterosexual parents, estranged or together, but they're incapable of demonstrating the fullness of humanity in the way that a man and a woman as the primary parental figures in a child's life can.
Being gay isn't hurting anyone else, so why should I care, right? Okay, it doesn't really impact my life if you're gay, unless one of us chooses to make it so, but the same argument is made for smoking (I refuse to hold the marginal increase in my insurance rates over the heads of smokers), and no one really pretends that people make it to show that it's not actually harmful. It doesn't show it is harmful, either, but you don't get to use that argument for raising children until you show it isn't.
Oh, you say you have the right to raise your children as you see fit? Yes, you do, within other limits you won't disparage, but they're not your children.
Marriage isn't about having kids anyway, remember?