Whether it’s a flag or downvote button is not going to change this, the discussion is solely UI. 

As far as the culture of current downvotes - If a large stakeholder is using downvotes unjustly, then many times in the past the community has stepped forward to counter.

 If the community agrees with the initial downvote or possibly the fact that it has turned into some sort of childish fight on both sides, they most likely will not. 

Which is why I feel normalizing downvotes and changing the culture is the only way to fix this.. what it appears like on the UI just makes it easier to understand, it will not change the underlying issues you are mentioning though.