I don't think there's a single answer. It's definitely never bad to ask for clarification, and it's good to edit the question in obviously-okay ways. Beyond that it gets a little trickier.
Roughly, I'd try to decide like this:
- are we fairly confident we'll get the OP what they need? if so, edit as necessary (perhaps toward a general "why could this happen?"), and write comprehensive answers, not speculative comments. (If people are only willing to speculate in comments, this may not apply - it suggests no one is confident they're getting it right.)
- otherwise, can we find out quickly? if so, just find out, edit, and carry on (don't speculate too much in the meantime)
- otherwise, do we think the OP has abandoned the question? if so, we can possibly edit it into a suitable question
- otherwise, vote to close, and if desired, ask a new question that avoids the issues (whether general or specific)
To elaborate a bit on that..
If we're pretty confident we can give the OP a good answer, we shouldn't be closing, even if things are a bit vague. We should probably just accept that the question is a little general (possibly editing it), and strive for a comprehensive set of answers. General questions can be a very good thing for future readers, so this can even be a good thing! Note in particular that answers can present various possibilities: "if your recipe includes A, you might try adding B as well, while if it involves doing X, you might try Y instead." We can still ask for clarification, to make sure we cover the right things, but I'd avoid telling the OP outright that we can't answer. And finally, the answer bit is important: if we're speculating in comments and no one is confident enough to wrap things up into answers, we are not in this situation.
If we're not totally sure we know what we need to in order to help the OP, then the next question is whether we think we can get it quickly. If we have just a simple question, and there's a good chance of the OP replying, we can just ask. There's really no sense doing the close-open dance unless things are in so bad a state that we're getting horrible answers already. We should probably not write speculative answers in the meantime, though.
That leaves questions where we don't know enough to provide decent answers (not even by covering obvious common possibilities). A lot of the time this happens when the OP has just disappeared. We shouldn't be too impatient, but if they really have abandoned it, we may want to lean toward editing into a suitable question even if we're not totally confident it matches the OP's original situation; we might as well salvage something out of it.
But if we really can't write answers and we're not ready to co-opt the question, yes, that's when we should be voting to close and telling the OP we need help before we can answer, and not trying to write any sort of answers. We can always reopen if things improve - and we moderators are more than happy to help quickly reopen questions.
Finally, when in doubt, we can always post new questions. This can often be a really good compromise, for when we aren't confident enough to edit the OP's question, but we also know what would make a good question.