Colin and Mia had been together two years when they hit their first dry spell. "We hadn't had sex in a few weeks, and one night before bed she asked me how I was dealing with it," says Colin (not his real name), a 38-year-old vice president at a Web start-up in Boston. "I told her I was masturbating," he says. "She asked me where and how." So Colin took a deep breath and told her: at the office, once a week or so, to Internet porn.

Mia (not her real name) freaked out. "She was fuming mad," Colin says. "She couldn't handle me leaving her out—it made her feel insecure." Mia adds, "Just imagining him there in his office on his own, it seemed like the ultimate act of desperation, something you'd only do if you didn't have a partner."

Mia was shocked that Colin was rubbing one out when he should have been crunching numbers, but she wouldn't have been if she'd looked at the stats: Most guys in long-term relationships continue to masturbate—even when they're having regular sex. A 1994 study found that nearly 85 percent of men living with a sexual partner masturbate, compared with only 45 percent of women.

And while the cliché is that it's the sex-starved husband who is driven to furtively spending some "me" time in the bathroom, research shows that most men's masturbation habits have nothing to do with how often they have sex.

Even so, when your wife finds out she'll hear a singular message: Our sex life isn't up to scratch. "Many women assume that if they catch their partner masturbating, it means they aren't doing their part to keep him happy. 'Why would he want to jerk off if I am right here?'" says Searah Deysach, owner of the Chicago sex shop Early to Bed.

What may bother your partner most of all, though, is just whom you're keeping virtual company with—an ex-girlfriend, the new girl in the office, Natalie Portman in Closer—while the bathroom door is locked. Which is why most women settle on a don't-ask-don't-tell policy regarding their partners' masturbatory fantasies.

Tony (not his real name), a 35-year-old small-business owner in Chicago, doesn't discuss with his wife what he fantasizes about. "If it's not about her, then it doesn't seem very helpful to anyone to bring that up," he says. "'Hey, honey, I jerked off while thinking about a hot woman I saw yesterday.'"

While some guys store everyday images and encounters to fuel their imaginations, many go straight for the porn. In a 2005 study, 25 percent of all men (and only 4 percent of women) reported having visited a pornographic website in the previous 30 days. Some women find this kind of fantasy easier to handle—you're not likely to bump into adult-movie stars at the office, after all. But others hate the idea of their partners' lusting after other women, even if it's just virtually.