The obvious line people always cite is sleeping with someone, but I don't think it's actually that clear cut. I suspect it is possible to be emotionally/heart unfaithful without anything physical happening, and it is possible to sleep with someone and yet not be unfaithful to the emotional relationship with your spouse or partner. And it is different from couple to couple, whose conditions and requirements of their intimate relationship vary. Which on the one hand doesn't seem to help at all, and indicates that each new couple discuss and stipulate what they personally would see as a transgression, a violation, of their intimacy.
Or, do we think, that bar the usual exceptions in particular, individual couples who stray from the norm, that there are general, objective standards of decent, 'faithful' behaviour? Or is it actually that the rule of thumb is the basic one by which to guide all our actions? That is: is my act loving or un-loving, both to myself and to all touched by the act? The 'to myself' and not just others is crucial.