Is cooking too subjective to work?

Question

I received an email which specifically cited cooking as an example of a Stack Exchange site that "is awful". This person said:

Meta-judgement is the ability to assess the quality of someone else's contribution. Different people have different meta-judgement skills in different areas. Wikipedia has been largely successful because they've carefully circumscribed out the area of knowledge in which people's meta-judgement skills are reasonably good (no original research, neutral point of view, verifiability etc.). Stack Overflow has also been successful because objective programming questions are also an area where people's meta-judgement abilities are good.

Once you move out of these carefully circumscribed areas, I think you run into trouble with your philosophy.

I've been casually lurking on the cooking stack exchange for a while. It's awful. The people there don't know [anything] and they don't even know enough to recognize the people who do know [anything]. As someone who is a damn good cook, I look around for a bit and I can't see a compelling reason for me to contribute in such a community. There's a thousand other places I could go to throw my pearls of wisdom into an indifferent crowd, the cooking stack exchange is no different from any of the rest of them.

I am not saying I agree with this, I am presenting it for your feedback.

It's true that we explicitly disallowed subjective questions for a long time, and for good reason. But we have learned that it is possible to have constructive, useful Q&A around totally subjective topics. To that end, we came up with the Six Guidelines for Great Subjective Questions.

It's hard to address this argument in detail without specific examples of these bad cooking questions and answers, so in Wikipedia parlance, {{citation needed}}.

But I guess the crux of it, as stated, is that cooking is a topic where this "meta-judgement" ability is in particularly short supply.

Your thoughts? And let's fill in the missing citations: what are the best and worst examples of cooking questions on the site, right now?

worst as in "most highly upvoted but objective wrong answer" and best as in "most downvoted/argued against but actually objectively correct" answer.

Answer 1

I don't want this to sound like thoughtless retaliation, and I will respond to each point in kind, but first of all, assuming you've reproduced this e-mail verbatim, the fact that the author not only feels the need to define metacognition but also incorrectly refers to it as "meta-judgment" gives me the impression that this person just read the original Dunning-Kruger paper and has decided that s/he is seeing the pattern everywhere. ^{[citation needed]} indeed.

Anyway, I said I'd respond to each point, so here goes:

1. I reject the notion that Wikipedia succeeds at this in the way that the author implies. A quick look at their list of controversial issues should quickly lay to rest any beliefs that every article is consistently unbiased and high-quality. Wikipedia is not bad by any means, but there is a good reason why academia continues to refuse to allow it as a research source.

2. I also reject the notion that Stack Overflow is exclusively the domain of those whose metacognitive ability backs up each and every one of their votes. Beginner questions do well, of course, but the more advanced/obscure the topic, the less reliable the information becomes. Security is one area that I've noticed is often extremely weak. Take, for example, the question about how to store recoverable passwords; there is an answer with 47 upvotes advocating an approach that is completely insecure and making a completely invalid analogy to a system that actually is secure. As with Wikipedia, Stack Overflow is not bad by any stretch of the imagination, but neither is it perfect even in its limited domain.

3. I accept the assertion that in order to survive in the long haul, we will need far more people from the cooking community as opposed to the programming community. That is one reason why I asked a long time ago if we could get "crossover" statistics, i.e. find out how many members are new to SE and are not coming from SO or similar. It goes without saying that as long as we have a fairly steady influx of users from SO and only a trickle from Google and WOM marketing, this will be a problem. However, I have yet to see a QA thread so egregiously bad that it could seriously classify this effort as a failure or even anything close to it.

4. Generally speaking, there are actually many telltale signs of inadequate metacognitive ability and/or groupthink within a community. The most obvious is a flurry of open-ended, loosey-goosey questions that anybody can participate in. The Software Engineering proposal was stymied on Area 51 by this; the questions were so obviously bad that it was clear that the wrong people were participating. That's why the name was changed to ACM/IEEE, to say "hey, this might be outside your area of expertise." Not meaning to harp on the SE proposal here but what I'm trying to say is that it's blindingly obvious when the unskilled outnumber the skilled, and if you have to cherry-pick examples to "prove" it then it's probably not a real problem.

5. Cooking is subjective but not as subjective as some people make it out to be. Many claims are testable, and presumably if somebody asked the question then they are going to test it. There are several questions on the front page now with answers that are either obviously correct, easy to verify, or backed up by a source. For example:

   • What are the requirements for a dish to be kosher? - backed up with Wikipedia source and would be easy to verify even without that.

   • My question the other day, Why did my “Turkish Delight” turn into a horrible goopy mess?, is about a relatively obscure ingredient in North American cooking (agar). Michael's theory was exactly correct and his suggestion to fix it worked. I saw it happen; I was there!

   • How do I control the temperature of burnt sugar?, another "advanced" question as far as typical home cooks are concerned, has an easily-testable answer and I can verify first-hand that the approach works.

   • One of the older, highly-voted questions is what is the difference between white and brown eggs? As usual, the answer which cannot be verified in the kitchen is backed up with a source.

   • In How can you reduce the heat of a chili pepper?, the best answer outvoted the others by an entire order of magnitude, and although it is not sourced, it is blindingly simple to test. If that is a subjective question then I am the president of France.

   And so on and so forth. I admit that not every single answer actually has a citation, but to demonstrate that the site has a metacognition problem one has to show answers that are actually wrong, not just unsourced.

6. Even if our community does not have any more expertise than the myriad of other communities, as this anonymous e-mailer suggests, the engine gives it a major advantage over most other sites which are based on forums. So if this person really believes that the site is collectively "ignorant", then that would be all the more reason to join, to help fix some of these issues, and recruit other knowledgeable people to do the same. Everybody benefits from having a smarter community participating on a site that doesn't suck.

   What kind of person sends off an e-mail to say that a site is lousy and even goes so far as to admit that they've just lurked, instead of actually doing something constructive and making it better? And why should we listen to what they have to say?

Anyway, to sum it up - to the question Is cooking too subjective to work? My answer is a big fat

No.

If it were, we'd never have come this far.

Answer 2

Ah, prognosticating on The Internet - what could be more fun! As kids we pretended to be Matt Dillon; now we play Nostradamus.

By way of credentials, I shall reveal the following: I have predicted the Teletubby phenomenon, the rise of Google, the success of the iPhone, the victory of Hillary Clinton over Barack Obama in the 2008 US Presidential Primary Elections, and The Terrible Hummingbird Plague of 2009.

And I can confirm that Mr. Shalmanese is absolutely, incontestably correct in what he predicts:

• Ignorant brutes will descend upon the site, men of low pedigree will fill it with base answers and more will follow to greedily consume this vile offal, never mindful of its origin or the inauspicious name recommending it.

• The informed and enlightened will strive against this barbarousness, and find their words lumped in with barbecue stories and tips for making "eggs over easy". Bitter tears of frustration will flow like vinegar as they reflect on how, in another time, their pearls of wisdom might have lined bird cages or pressed flowers.

• Name-dropping will continue to be seen as a poor substitute for practical advice.

In addition, I predict that something not common to every public forum in every medium ever will occur. But I'll leave the exact nature of this to your imagination...

Answer 3

Since the author of that e-mail has come out and, among other things, suggested that we only do well for questions that are easily answered by Google already, allow me to provide a few counter examples:

1. Why can applesauce be used in place of oil?

   • I answered this one, and gave the actual (correct) explanation of why the substitution sometimes works, and heaped on disclaimers about the limitations of this substitution. Perhaps it is not the most comprehensive answer on the web, but at least it tries to lead readers in the right direction. The top 3 google results are full of inaccuracies.

   • The first result is a chowhound thread. I've found that site to be very useful in the past, but the answers are all over the place. The first two answers suggest a 1:1 substitution and that's almost never what you should do, even in a cake. Only much later on (and who reads that far down?) do you start to hear that, actually, it really does change the taste and texture, and doesn't work in all baking recipes, and doesn't work so well when you've made the recipe from scratch as opposed to using a mix with all sorts of stabilizers, and that maybe you should go half-and-half or at least add a little bit of oil, and so on and so forth. Not very authoritative at all.

   • The next result, from eHow, is slightly better in that it does mention some (not all) of the limitations. However, it repeats the mantra that a 1:1 substitution is just fine without any other modifications. It also adds some misleading statements at the bottom: "Unsweetened applesauce will reduce the amount of sugar in your recipe." I think I understand what the author was getting at, but the way it's worded is backwards. Adding even unsweetened applesauce may increase the total amount of sugar in a recipe, since apples are naturally sweet. The more important point, which is never mentioned, is that using sweetened apple sauce will almost certainly require you to reduce sugar in other parts of the recipe to compensate. A casual reader stumbling upon that eHow article would not take away that message.

   • And in third place we have an Answers.com response, arguably the worst of them all, which suggests that you can substitute applesauce for butter or margarine (although, confusingly, it also says "in oil-based baked goods"). It does mention using some oil, so it gets a point for that, but mentions none of the other potential problems (sweetness, shape, etc.), and suggesting that applesauce would be a good substitute for butter in bread dough is wrong wrong wrongity wrong.

2. Just try using Google to find the answer to Why do my muffin tops lean?

   I'm sure that you consider this a basic question, but it is not so basic to everyone. I've actually known a few people with this problem. Using Google for this will get you nowhere, because both the word "lean" and the phrase "muffin top" are associated with body fat and weight loss, which, as you might expect, are several orders of magnitude more popular than muffin baking.

3. Why do tomatoes get so hot?

   I love this one because on what other cooking site would you get a physicist arriving to weigh in on the issue of heat distribution, conduction, and the difference between a food simply being hot and burning your mouth? Perhaps you are looking for examples that are more about culinary techniques/issues, and I will get to those, but this is really an awesome question for demonstrating the usefulness of the whole SE "network" as opposed to individually-focused cooking forums.

4. Why do some recipes recommend Kosher salt?

   I've used kosher salt plenty of times in recipes but never knew half the stuff that Darin put in his answer. That is, by and large, the most definitive post I've ever read on kosher salt. It's way more useful than the Wikipedia page on it.

5. Pasta: Simmering Water or Rolling Boil?

   This is another one that I like, because it's a bit of a tricky issue. We have two competing answers that are actually both sort of correct. The accepted answer repeats the conventional wisdom but includes a proper explanation of all the terminology (including specific temperatures). The other, higher-voted answer challenges the conventional wisdom and provides a source to back it up. Both of these answers received many more votes than other answers.

   Now you might say "A-ha! The accepted answer is wrong! You see!" Except it's not wrong. The issue is that pasta does not require a huge pot or a rolling boil to cook perfectly. But it's also true that especially for certain types of pasta, such as spaghetti, those factors do help if you tend toward leaving the pot unattended. And that's exactly what the top answer says: "Help prevent sticking." Nowhere does it repeat the erroneous folk wisdom that a simmer will turn the pasta into mush.

   Together these two answers provide a lot of insight that you just won't find on Google. Most of the top results just parrot the myths.

6. Frozen veggies meets gumbo = flavorless. How to add flavor?

   I wouldn't even know where to begin searching for something like this on Google, and although Michael's answer seems obvious in hindsight, I think many if not most of us would not have thought of it soon enough to fix it (it's important to note that this answer came in less than half an hour).

So, do I need to go on, or is this sufficient to explain why our community and site is better than a Google search?

Obviously, like any site, there's going to be a long tail. We're always going to have our share of prosaic or even mind-numbing questions, and sometimes they get voted up more than they should because they're easy to understand and people want Electorate badges. But these kinds of questions are the staple of any cooking site. Many home cooks have years of experience but are also self-taught and have gaps in their knowledge that a professional wouldn't have; many of these people also have valuable insights to contribute, and will be much more willing to do so if they can get their naïve questions answered without the smarmy attitude that so often accompanies the more "advanced" cooking communities.

And I would really like to see more advanced, "haute cuisine" questions here, but I think that will only come with time. As I stated in my other answer, even on a site with many professionals such as Stack Overflow, the truly expert-level questions tend to get the least attention. The only way to prevent that is to have a completely closed system that only allows recognized experts in the field, and then it starts to look more like an industry journal than a Q&A site. We want to be useful to regular people too.

Answer 4

It'd help if the author of the email could point to specific examples of

I've been casually lurking on the cooking stack exchange for a while. It's awful.

"Everything sucks!" Is neither constructive nor actionable. I am still waiting for actual URL examples and explanations of what specifically is "awful" about cooking.stackexchange.com.

Provide URLs. Briefly explain what about them, exactly, is wrong.

Beyond "it's awful", even with examples, let's talk about positive examples: you have to be able to point to other sites that you do think do things correctly, or maybe aspects of different sites that work well.

The author, in his answer here, did cite http://forums.egullet.org/ and specifically http://forums.egullet.org/index.php?/topic/116617-sous-vide-recipes-techniques-equipment/ as examples of

information resource that cannot be mimicked by any online and very few real world expertise sources

Though it's a bit of an appeal to billionaire authority due to the participation of Nathan Myrvold in the thread.

sad for me seeing Quora turn from an absolutely amazing resource slowly into yet another internet community, exactly as I predicted when I interviewed with them ~6 months ago .. Joining the site and actively contributing will not be helpful if I believe the ultimate structure of the site is not geared for success.

Based on your Quora participation history, with friends like that, who needs enemies?

Still, it's useful to consider http://forums.egullet.org as an example of a cooking Q&A (ish) site that is working.

Answer 5

@Shalmanese: There's a critical piece of the philosophy and functionality that has eluded you: Stack Exchange's "archive of knowledge" is nowhere near finished in neither breadth nor depth. If the "depth" does not suit you, it will only continue to improve as new answers are contributed, existing answers are improved, and the whole collection is vetted and undergoes peer review.

You said that Stack Exchange is no different from the rest of them. That's high praise considering the Cooking Stack Exchange has barely existed for 90 days. So what if — at least in your opinion — Cooking.SE didn't snap into instant, unassailable genius in its first 90 days. That develops with time, hard work and continuing contribution.

But you chose to sit on the sidelines and watch and condemn.

In contrast, the forums you celebrate are constantly watered down and mired in irrelevance. Throw your "pearls of wisdom into an indifferent crowd"; Who's to know? Some of those questions are hundreds of pages long. What good are those pearls when there is so much noise to drown them out? How am I to find them?

If you fixed just two questions on this site, you would have improved this archive of knowledge by over 1%, single-handedly. That's quite an accomplishment for so little effort. —Small units of incremental impromement.

Answer 6

I have trouble seeing the e-mail author's point at all. In general the community has worked extremely hard to carefully craft a site definition that encourages good subjective questions which can be accurately judged by professionals and experienced amateurs and discourage the bad subjective. For example we carefully defined what recipes might be objective enough that they could be correctly judged "the" recipe and therefore were acceptable on the site. We've looked over what makes a list of X question acceptable and not many times, carefully debating who can verify whether or not a flavor pairing is good or not and other details, with our professional members and top contributors weighing in. I think this has been largely successful in helping to define a very useful community where members seek answers that explain the why and how and information is shared in a generally constructive, informative, and fair tone.

I went looking for downvoted answers that were actually correct. The closest thing I could find was this response to changing a quick bread mix's sweetness. While the answer is technically correct, the discussion about the downvote gives a pretty clear defense of why the answer was voted down. One could make a fairly good argument that it is not more correct than any higher voted answer.

I'm marking this CW - if you see anything else that could qualify for a worst answer as defined in the question, please fill it in.

Answer 7

I know that at this late stage there's little point in adding another opinion to this, but still...

To me, the problem with Shalmanese is captured in this comment of his:

In cooking, providence matters deeply for Subject Matter Experts. That's something inherent in the process. Because cooking contains so much subjectivity and taste, I need to know where you're coming from to even be able to interpret your answer.

First off, I think he means "provenance," but in any case this just amounts to an appeal to authority. The people posting here don't meet his definition of established experts, therefor nothing they post can possibly be useful. I strongly disagree with this attitude.

Yes, there's lots of subjectivity, but I don't rely on an expert to tell me what's good and bad, I can see it for myself. I'm not a professional baker, but I've done enough and read enough books that I can make a pretty good cake and I can certainly tell a cake that tastes good from one that sucks w/out an expert to explain it to me.

I don't think cooking.se is that much different from Serverfault in that respect: most of my experience is w/ Windows, but I've used Unix and Linux enough that I can recognize a good answer from someone who is a Linux expert.

But basically Shalmanese thinks the site can't thrive with experienced amateurs posting, only if accredited professional chefs post.

Answer 8

I am the author of that email and I welcome constructive debate. It's been a long time since I used the Stack * sites so excuse me if I fail to follow some point of social etiquette. The email thread arose out of a conversation around an essay I recently posted entitled "The Evaporative Cooling Effect in Online Communities".

In response to Aaronut's post:

• Metacognition and Meta-judgement are two separate things. Metacognition is awareness of your own reasoning processes and relies on introspection. Meta-judgement is the assessment of someone else's contribution. These are two separate skills.

• I completely agree with you that Wikipedia has many, many problems. However, overall, they are largely successful. This is not what I'm arguing.

• I also completely agree that Stack Overflow is not perfect. This is also not what I'm arguing.

• You argue that Google/WOM is important and it obviously is but you've failed to articulate how, after you've reached out to people beyond the SO family, how you convert them into active participants. This is the crux of the issue.

• My concern is not with the level, it's with the first derivative. The question is can cooking exchange be built into a site that gets better over time? I don't see a mechanism for this to happen.

• Of the 5 examples Aaronut listed, 3 of them can have the exact same answer found as one of the first 3 hits in Google. This, IMO, is fine but not an especially compelling use of a Q&A site. The other two are genuinely good answers but they're not the type of great answers that make something an addicting resource. For an example of truly great answers, check out http://forums.egullet.org/index.php?/topic/116617-sous-vide-recipes-techniques-equipment/ for example. One of the most active contributors in that thread is Nathan Myhrvold, author of the new mega-tome on Molecular Gastronomy. In that thread, you see real, genuine Q&A going on and it's an information resource that cannot be mimicked by any online and very few real world expertise sources.

• As to what kind of person sends off an email complaining about a site, I'm a person who genuinely is invested in the SE mission and wants to see it succeed but I believe the best way of doing so is via informed critique. Joining the site and actively contributing will not be helpful if I believe the ultimate structure of the site is not geared for success. The other factor is that I'm one person with a limited number of hours in the day and I've already invested in a cooking site. I'm Shalmanese on eGullet and you probably recognize me there if you've spent any time there. If you imagine I or any other person in the world is going to see the site and immediately jump in with gusto, then that's a fundamentally flawed view. Instead, you should be asking, what it is about cooking exchange that could convince someone like me to want to make it a great resource.

Also, my main arguments have been on the downfall of Quora as they are coping with issues of growth and Cooking Exchange only incidentally came into the conversation as another community I expect to go down the same path. It's been extraordinarily sad for me seeing Quora turn from an absolutely amazing resource slowly into yet another internet community, exactly as I predicted when I interviewed with them ~6 months ago. The tricky issue is that, by the time you recognize the problem occurring, it's too late for the fix. What I'm arguing is not so much the state of cooking exchange now but what I see as it's inevitable state 12 months down the road if the same fundamental design is kept to.