How Do We Judge Statements Based on the Speaker

Lately I've been thinking about perceptions a lot. The way we see people obviously changes how we view their words and actions. For example, I have believed that religious beliefs should have no bearing on politics. When I identified myself as someone who was extremely religious, this view was taken with a lot of weight. My opinion counted. Now I still have the same view, but people say "Well of course you think that. You don't understand what it means to have the law challenge your morals". And even more often they don't say that, they just think it.

If you were previously in a position of privilege and move to one without it do your thoughts and opinions become worth less? What about your actions, previous and present, do they count differently? In some ways it seems like they shouldn't, but it is much easier for an atheist to say that religion is irrelevant to the way we govern ourselves than a theist. It doesn't feel right to give more weight to one than the other. Yet it does make sense in some ways.