Joe left Norma not because of Betty, but because he knew he was going to lose his soul, his self-respect, his identity if he stayed. Besides he wasn't in love with either woman.