So, first, I don’t think Dolokhov was trying to make anyone believe anything. We have no indication of that. He was definitely trolling Pierre at the club, but this sounded far more like him taking advantage of things other people had already done (i.e. spread rumors) and kinda shrugging and turning them to his advantage. It was all very “wink-wink nudge-nudge” not like he was seriously trying to make Pierre believe or not believe anything.
As for whether or not they actually had a n affair. Ha. See, I think it’s completely wide open in canon. I don’t think there’s actually any “canonically right” answer here. (And part of me thinks this is almost an intentional authorial choice.) I don’t think the text supports anything other than maybe a flirtation. (But is open enough that it could have happened.) If I had to choose just one scenario to headcanon that I personally believed was most likely, I would say there was a flirtation but nothing more, no actual affair.
First, why I think a flirtation did happen. Mainly this is based on the fact that the rumors were real and clearly widespread enough to penetrate various social groups. Dolokhov knows what’s up; Anna Mikhaylovna has heard the word on the street. This suggests that the rumors weren’t just a few people fabricating stories for some purpose, e.g. dislike of Helene, wanting to troll/hurt Pierre. There must have been something to make people go, “yea that seems plausible.” Dolokhov goes out of his way to tell Pierre about Helene’s beauty, so even if he’s trolling or teasing, it’s a behavior that could easily be misinterpreted by a third party.
Apparently, people also took notice of the fact that Dolokhov “followed” Pierre and Helene to Moscow from St. Petersburg, but I’m actually not sure if this proves anything significant. Dolokhov was living with Pierre in St. Petersburg. It appears that Anatole wasn’t around at the time and Dolokhov doesn’t seem to have a place of his own in Petersburg. It’s possible that most people from his friendship circle were still with their regiments so finding a new place to stay would have possibly been a pain in the ass. So why not go to Moscow where he has family? Not to mention that they’ve likely missed each other and would like to spend some time together before he goes back to war. (FWIW, I don’t think we ever see Dolokhov in Petersburg other than when he was living with Anatole or that brief period with Pierre. Nor does he seem to have a lot of close friends, generally.) In any case, I think a flirtation is likely, but it doesn’t look like a serious pursuit even, not to mention an full-on affair.
Next, why do I personally feel it’s more likely that there was no actual affair? Well, most importantly, it’s what makes the most sense, IMO, with Helene’s character. Is she pretty free about sex? Appears so, but she’s not Anna Karenina, falling passionately in love and losing her mind, nor is she quite a Marquise de Merteuil, just doing it out of boredom. I don’t think we ever see Helene seduce anyone just for fun. (In the book, the UST with Natasha is nowhere near as high as in various adaptations, and anyway, she’s helping her brother there.) I think the closest we get is the implied stuff with Boris. But again, it’s really unclear how far that goes. We just know that Boris because a frequenter of the Bezukhov household and that Helene treats him with a lot of attention but also patronizingly, almost like a child. I’m not really convinced, by how their explicit interactions go, that she’s sleeping with him. Sounds more like another flirtation that stops short of an affair.
The thing with Helene is that she’s a social climber, and I think this goes to her sexual relationships too. This is why I don’t think she’s quite a Marquise de Merteuil. Helene, other than just enjoying fun, is clearly in the business of rising as high as she can in society by any means she’s got. She’s totally on bored with marrying Pierre for the money, even though it’s clear she would get no emotional anything out of that marriage. She runs a very successful salon and this seems to take up a good chunk of her attention. Both of Helene’s lovers during 1812 are influential people who could potentially help her in her endless social climb.
Let’s also talk about babies. Not that it wasn’t possible to avoid pregnancy back then while having sex, but…it would have been hard, and dangerous. No contraceptives, no birth control that a woman would need to either try to time intercourse only when she was very unlikely to get pregnant (right before or after her period) or use some remedy (not the safest method) to prevent or early-terminate pregnancy. As we see historically, this didn’t work great for most women as they kept getting pregnant despite not wanting to be pregnant (see: Tolstoy’s own wife.) Helene is very adamant about not having children – she tells Pierre this from the very start. I doubt she’d play around with this. (Even more so given that she is clearly averse to scandal, as we see in her reaction to the duel.) Given that it is implied that Helene’s death was the result of a botched abortion attempt, either her “contraception” methods failed her after many years, or she just doesn’t actually have a lot of experience in this. It’s arguable, but I lean toward the latter.
The point of all this being that I don’t think Dolokhov – with his lack of funds and connections – is the sort of person she would risk getting intimate with. (Especially given that at that time she had not secured the amount of social capital and success that she had even during her flirtation with Boris.) Even if she was enjoying flirting with him.
I think it’s also notable how the rumors characterize the supposed Dolokhov/Helene relationship. Anna Mikhaylovna gives us the most information by noting the circumstance that Dolokhov followed Pierre and Helene to Moscow and that Dolokhov had “compromised” Helene. She refers Dolokhov as a “daredevil” but also seems to express some lighthearted sympathy for him. Overall the impression she gives is a sort of “wink-wink nudge-nudge” recounting of a flirtation where one party is infatuated with the other party and the other party isn’t completely stonewalling. But it doesn’t seem like she is describing a full-on affair. The note Pierre receives is more explicit, referring to it as a “connection.” But I’m more tempted to believe Anna Mikhailovna at this point, because she has no reason to lie to the Rostovs about this and she’s so prone to exaggeration that it’s unlikely that she’s purposefully underselling the rumor mill consensus on what’s happening.
Of course, all of this is arguable. But, for the last thing, I just don’t see that much evidence to say that the affair certainly happened. The only “evidence” is that people are spreading rumors and one (1) person, whose identity we know nothing about, wants Pierre to believe they’re true or maybe just to tease/mock him about them by sending him an anonymous, somewhat insulting, note. I think it’s pretty obvious why this is a very suspect source of information. (Like a million reasons. There’s also the circumstances discussed above – Dolokhov following the Bezukhovs to Moscow (he’s on leave and his family lives in Moscow. It’s not exactly weird for him to go there) and Dolokhov complimenting Helene’s beauty (an innocuous remark in and off itself). Dolokhov doesn’t deny the rumors but Dolokhov also doesn’t deny the rumors of being a shaper (which are more likely to be true, tbh). Dolokhov isn’t really in the business of denying rumors about himself, even when they’re potentially damaging. (Also, if Dolokhov was actually having an affair with Helene, why would he force the issue with Pierre? It would be more to his advantage to keep a low profile on it. This way, there would be no reasonably way for the affair to continue.)
The fact that Pierre ends up believing the rumors doesn’t really mean anything. He’s a notably unreliable narrator and his POV was full of bias against Helene (nor is he of the best opinion of Dolokhov) before he even married her. And his opinion of Helene back then? Was also mostly based on nasty gossip. Tell Pierre anything with enough conviction, and if it fits his general worldview, he’ll roll with it.