Kaneda-san without his jacket isn't Kaneda-san at all.

[He's embarrassed by the smile, and it shows in the way he looks down. His hair flops down over his eyes and obscures all but the curl of his own lips. He feels... fuzzy, inside.]

So you have to take care of it. Or else people will stop recognizing you.