I think that as long as you're not abusing the flagging system, you should be allowed to do whatever you want with your vote weight. I have zero rights (nor control) to tell anyone else what to vote for, and that's a good thing :)