Note: Since the response is mostly (but not entirely) positive to this proposal, I'm accepting the answer: I'll be using this policy, erring on the side of not deleting anything of significant value. (See the full answer for discussion of which kinds of situations likely merit deletion.)
What do we do?
This is a real issue (see the second section), so mods should eventually delete answers in comments, after:
- sufficient time has passed or the comment has little value (note that there is room for discretion here, see the third section); and
- they or another user have asked the comment author to consider writing an answer. (Linking to this post may be helpful.)
Users should thus feel free to ask comment authors to write answers, or to custom-flag answers in comments. If mods forget to follow up with a deletion, it's probably appropriate to flag as obsolete.
Another option is for users to post answers containing what was mentioned in comments, and flag the comments as obsolete. (It's probably best to write it in your own words and/or credit the user, though.) I know might feel a bit like cheating, but you're doing the site a service by writing up a good answer, and the user who posted the comment made a decision not to write an answer. If you wish you can always mark the answer community wiki in order to not get the reputation, encourage others to edit, and make it even clearer that it's not totally yours. But it's totally fine to earn reputation off it; the commenter missed their chance when they decided to post as a comment. They may even have intended for users to take the ideas, as Joe points out.
Why is it a problem?
Answers in comments really are problematic:
- they can't be edited
- they can't be downvoted
- they can't be properly responded to (comment replies are mixed with other question comments)
- they aren't as easily seen
- they aren't ranked against answers
- they are ranked against comments (if enough are present they may be hidden)
- they can't be truly upvoted (users deserve reputation for good answers!)
Bottom line, comments aren't answers, answers are answers, and unsurprisingly answers have better support for the kinds of things answers need than comments do.
So it really is worth discouraging them, and any response that doesn't involve eventually deleting them doesn't actually do much to discourage them. I'm not in favor of immediate deletion (with exceptions for patterns of low-quality answers in comments) since it prevents @-notifying that user that they should post a real answer, and it prevents the user from copy-pasting their comment as a start for their answer.
The end goal here is to get more actual answers. Yes, it sometimes takes a bit more effort to flesh out an answer. But it's okay to post brief answers; not every answer needs to be your best answer ever. Most of the decent answers in comments are already long enough to be an answer, so they really could just be posted as answers. Another option might be to suggest your partial answer as an addition to another existing answer, letting the author know that they can edit it in.
When exactly do we delete?
With all that in mind, some rough ideas I have in mind when considering whether it's time to delete:
- If ideas in the comment have been incorporated into an answer, it's safe to delete.
- If well-received answers have been posted which don't use those ideas, and it's been a bit (say a day or two), it's likely safe to delete. (That suggests that people aren't interested in adopting what's in the comment, so it's up to the comment author to post a real answer.)
- If the comment is attracting discussion (especially disagreement) on the question, it probably should be deleted. This is a case where it's directly causing issues as described above due to being in the wrong place, so it potentially means deleting things relatively quickly.
- If it hasn't been long (~hours), and it has merit and isn't causing a ton of discussion, it's probably not safe to delete.