We’re two years out and

We’re two years out and really doing well, but I DO have those sleepless nights where what I call “the junk” plays over and over in my head. Without the distractions of the day, my brain digs those reminders out of the dark recesses.
I tried to talk to my husband about this but it didn’t go so well. I told him that during those times I start wondering if I’d be better off alone, without him. After all, he’s the biggest reminder of what happened. I tried to explain that I love him and I don’t want to leave, but the fear part of me wants to run away and hide. Love is risky. Loving AGAIN after infidelity even riskier. So yes, the fear does overtake me at times and I question if I’d simply be happier in a little apartment somewhere, living a relatively quiet pain-free life.
He doesn’t get it when I express this. All he hears is that I want to leave and it makes him disheartened because we’ve both worked so hard. I tried to explain that I don’t want to leave, I’m not leaving, it just seems attractive at times because it feels like the less hurtful, painful choice.
My logical side argues that I’d never escape the pain of what he did to us, it will always be present, until the day I die. Moving out and living alone won’t make it magically disappear. What he chose to do is now something I’m forced to live with forever. That’s the part I hate. I can’t go through a single day without this thing disturbing my peace, destroying my joy. I DO have happy moments and times of forgetfulness, but then it pops into my head to rob and steal from my good times.
I long for a day when this fades to insignificance, I just can’t imagine that day very well. I’m weary if it all. I just want it to go away.