Is Emotional Infidelity a Bigger Sin?

The question, is emotional attachment to someone else a more blatant betrayal than a physical adulterous affair has been engaging the attention of people from time immemorial. Sadly, despite some heavy duty thinking no clear-cut answer has emerged.
I think it is one of those, which came first question. My opinion is emotional or physical a betrayal is a betrayal and there is no point in refining the issue.

What is your take on this question?