His wife was not his patient, she got sick after they were married. And I don’t think there was any love between he and Kate, in fact, IIRC, I think they were only together a couple of times, and to me, they were more like "f***ing buddies." (Sorry to be so crass but no other phrase describes it as well.)
As for Chelsea, I think there was strong attraction but I don’t think it could have actually been called "love" yet.
So that really only leaves Chloe. I think he was kind of portrayed as a swinger when he came to town but I remember him meeting a woman for a date early on where he ended it because he couldn’t forget about his wife. He also turned down an attractive doctor who was pursuing him, so I don’t believe he "got around" as much as people were led to believe.
But then, I really like Daniel (especially with Jen) so, YMMV.