Some people are just emotional cactuses with reasons of their own for being afraid, and all he do is remind himself it's their own challenge to bear, depriving themselves of his friendship!!

!!1

Seriously tho, I love the idea of him having some early on antagonistic cr not just from the locals who go "ew necromancy" at him, but even the other abducted by mirrors. And this is a pretty fundamental divide of mutual fear with some merit to it, which isn't easy to resolve...