don't worry bout him, he just wants to see more of your gorgeous face.
anyway, i'd say end it, as you said it seems to hard.
Isn't there a saying; "If you love somebody, let them go. If they return, they were always yours. If they don't, they never were."
Hmm it sounds like you have something serious but if both of you aren't prepared to move (either him to here or you to there) then I think it should end. LDR = Epic Fail.