I actually once read a hard-boiled crime novel by T. Jefferson Parker called "Silent Joe," which explored a similar theme - - the protagonist had a horribly scarred face and used to pay models to simply have a cup of coffee with him so that he could look at their faces. I found the character so compelling and sad that I actually stopped reading the book a third of the way through. If you're intrigued, it's available at Amazon.
Via.