Do not sweat about trying to justify yourself. Its very difficult, if at all possible, to change others’ firm opinions about you. It is not unusual to be misunderstood or misjudged. Unfortunately once another person decide to perceive you in a negative way, no matter how much you try to impress them it never works. My take is lets not waste energy on things we can do nothing about. Lets continue to do good, to improve what have been invested on us, to hold a positive attitude and a clear conscience towards everyone.
Rather focus your energy on the bigger picture, on things that can make you or break you. On your calling, your vision and on doing good. The bible encourages us not to stop doing good, especially to those of the household of faith, I don’t think it matters whether you do get good from them in return.
If at some point certain people do realize that they had wrong conclusions about you well and good, if not its still okay, life is too short to stress about such issues. If you are a believer rest assured, when they get to heaven they will know they were wrong, which is still fine. What are you worried about today? If you cannot do anything about it just roll it off to God and keep your peace. Remember, you need to do what you were born for, while it is still today.