Yeah, Haytham wasn't thinking very clearly when he did that, but, to be fair, Charles can be very persuasive. Haytham was originally going to keep Connor at his temporary townhouse, but then Charles was begging for a second chance and made some legitimate points about how Haytham's townhouse was not as secure as Charles's permanent house...

Haytham is totally facepalming now.

:)