These are some of the most common mistakes I have seen in cupping labs in my career. These are the problems that pop up again and again. Even when you warn people to prepare for the problem, they often underestimate the problem.
These are concerns for you to address whether you are setting up a one-time cupping on the road, or whether you are building out or improving your own permanent lab.
I have a friend who is a professional taste-tester and product-tester. She works in all kinds of foods, like crackers, chocolate, wine, TV-dinners, soft drinks, and yes, even coffee. A few years back she was in charge of training some experts for a panel she was putting together.
To test her subjects, she put together a little trick, making cherry gelatin with yellow coloring. When she served the yellow gelatin to the panel and asked them to write down what flavor it was, most of the tasters wrote "lemon," a couple wrote "banana," and only one of them wrote "cherry."
Think this is a one-time incident? Check this out:
In France, a decade ago a wine researcher named Fréderic Brochet served 57 French wine experts two identical midrange Bordeaux wines, one in an expensive Grand Cru bottle, the other accommodated in the bottle of a cheap table wine. The gurus showed a significant preference for the Grand Cru bottle, employing adjectives like "excellent" more often for the Grand Cru, and "unbalanced," and "flat" more often for the table wine. — Wall Street Journal
If you think you are immune to this effect, you're wrong. No one is immune. If you know ahead of time what you are tasting, your opinion will be influenced by your preconceptions.
And yet, people still put together coffee cuppings ignoring this principle. It's okay to do an open cupping if it's just for fun, obviously. But if you are seriously trying to get useable data from your cupping, IT MUST BE BLIND.
There is no way around this. And no, you are not an exception to this rule.
Just to be clear what I mean by "blind": the identity of each coffee should be hidden to as great a degree as possible. Most times, of course, you will know you are cupping 20 Panamas, or 8 coffees from the same roaster, or whatever. This level of knowledge if usually unavoidable. But you should have no idea which is which. To achieve this level of ignorance, use codes for each coffee and make sure to order them randomly.
Ideally, you won't even know what category of coffee you are tasting. That's always the most interesting. But it's pretty rare to not have at least some idea what the category is.
The easiest way to make a cupping blind is, of course, to have most or all of the cuppers out of the room when the organizer sets up the cupping and gives the coffees a code. But many times you will be the one who is setting things up and at the same time you are the one hoping to do some evaluation.
There's a simple way to get around this. Just double-code the coffee. Imagine I am setting up a cupping with one other friend, and we both want to score blind. I will write down one set of codes. So a coffee from the farm "Finca Alfa" becomes 001 and "Finca Beta" becomes 002, etc.
Then I leave the area and my colleague comes in and re-codes the samples. "001" becomes "282" or some other random number. "002" becomes "991" etc. She makes sure to mix the order of the coffees.
When I come back in, I have no idea what is what, and neither does she, since she never saw the original list. We both cup completely blind and our data is therefore much more reliable. Then, at the end, we trace back through our double-code and re-establish the identity of each coffee. Easy peasy.
If you are not doing this, you might be having fun cupping, but you're not learning anything about your coffee.
Incidentally, I have noticed that 99% of all espresso-tasting that goes on in the specialty coffee world isn't even remotely blind. Everyone who tastes the different espressos knows exactly what he or she is tasting. Why people think this is a maximal way to evaluate their products is beyond me. (In order for it to be even half-way useful, the taster would have to be someone other than the barista, and he or she would have to be sitting somewhere away from the espresso machine so as not to have any hints about what's going on. Really you need at minimum three people: one barista, one "server" to act as liaison, and one taster... ideally you should have several tasters). This is really a topic for another post as espresso-tasting presents unique difficulties.
But in straight cupping it's very simple and straightforward. If you are not doing it blind, you're not really doing it! So now, are you a professional, or aren't you?
In this series: Common Cupping Mistakes — Hot Water!
In this series: Common Cupping Mistakes — Smelly Cups
In this series: Common Cupping Mistakes — Not Really Blind