I have a mod who is a real pain, who likes to down vote me and who generally makes my life (or tries to) a misery. Surely there should be some mechanism or community vote on this, or are they above the law?
There is no direct mechanism to remove a mod that I am aware of.
A down vote is not considered an abuse of power. People can use them as they please.
If someone serial down votes there is a script that runs and will back it out. But I seriously doubt any mods here are doing that.
When a mod comes up for election you can comment and not vote for them. But mods can stay in position for years.
I can tell you from experience that complaining about down votes will only get you more down votes.
Argue with a comment will do you no good. Just ignore the comment if you don't agree. If it is rude or not constructive you can flag it.
This site has a narrow definition of right and wrong. Mods and users are going to enforce it with comments and down votes. Play by their rules or be frustrated.
You may use the contact us link (it's in the footer of every page on the site) if you wish to report potential moderator misconduct. Moderators were elected by the community in the first place, and are not above the law. If we find mistakes we've made, we'll do our best to fix them, and if we egregiously misbehave, SE will take appropriate action.
- downvoting is normal, and the best response is to try to improve your posts and accept that you'll get a few downvotes despite your best efforts. (And unless other mods have been downvoting you a ton, I'm pretty sure you've gotten more downvotes from non-mod users than mods. I believe all three mods have also given you more upvotes than downvotes - I definitely have, anyways.)
- I do not try to make anyone's life miserable, and neither do the other mods. (I'm pretty sure I know them well enough to speak that much on their behalf.) I also don't like downvoting - I'd rather all posts be worth an upvote, or at least not worth a downvote - but downvoting is part of how we make sure better answers show up higher, so I do it when I feel it's justified and don't feel guilty about it. Same goes for having to close questions, or point out various rules or issues: it'd be nice not to have to, but it's part of the site and part of my job so I do it.
So if you have specific concerns that you'd like community input on, I would encourage you to post them, rather than a more general accusation like this.
In that case, you would also be able to easily gauge the community's view on issues. You could also discuss with the moderator in question or the other two - and we can also create private chat rooms if you wish. Assuming everyone approaches things in good faith, with the goal of working things out for the users involved and the site, this tends to be better than immediately escalating to the top.
See also the canonical question on network-wide meta for this: What recourse do I have if I believe a moderator has abused his/her privileges?