From Page Six, so take it for what you will (though they were the first to accurately report he quit):

But sources told Page Six that Abbott ("Charlie") and Dunham began butting heads as the critical-darling series entered production again. “They’ve just started work on Season 3, and Chris is at odds with Lena,” a source said. “He didn’t like the direction things are going in, which seems a bit odd since the show put him on the map.”