Charles cooley's concept of the looking-glass self hypothesizes that:

I believe the answer is: People base their images on how they think other people see them.

Most people would not directly tell us on what is their true opinion about us due to the fear of offending us. But, as a social animal, we crave to be positively accepted by other people. Because of this, we often create a mental argument in our head on how other people see us and form our identity based on that image (even though it might not be true)