Control Your Variables

I've been doing a lot of taste tests lately, and I've had a few interesting experiences that have reminded me of the importance of strict protocols when doing taste tests.

The whole reason for the ritual of cupping, of course, is to eliminate the variables from cup to cup, so that you know you are tasting the difference between the beans themselves (or, alternatively, the roasts). If more than one variable changes, you lose control of your experiment very quickly. Unfortunately, in my experience, most people have two or sometimes many more variables out of control when they are evaluating flavor.

Controlling your variables is important not just in cupping, but in evaluating your own espresso, deciding which roasted coffee beans to buy, choosing your brew parameters for your home coffee, buying coffee equipment... everything! One must avoid blanket statements that aren't backed up by the real data. The mind is very good at focusing in on one particular factor, to the exclusion of everything else.

For example, you may have a relatively rare coffee, such as a Timor or a Burundi. Someone makes a cup in a drip machine that is under-temperature and which uses a paper filter. Earlier in the day, you had some Guatemalan coffee from a properly prepared French press. The Guat tasted full-flavored, rich, and sweet. The Timor (let's say) tastes watery and flat. "I hate Timor coffees... they are too thin and flavorless!" Big mistake.

In general, in all different preparation and testing methods (from home brewing, to sampling coffee shops, to evaluating your own espresso, to cupping), here are the variables you have to be aware of:

Temperature: All coffee is very sensitive to the temperature at which you brew it. Many home brew coffee is too cool. Aim for 200 Fahrenheit. In espresso, a few degrees one way or another can make a night-and-day difference.

Grind: Inconsistent grinding can force you to choose between very weak coffee on the one hand or bitter coffee on the other, with no middle ground.

Season: If you compare an El Salvador to a Rwanda in September, you are giving the El Salvador a huge advantage (assuming its current crop). Be aware of when certain coffees are most fresh.

Roast: As everyone knows, roast has a huge influence on taste. Don't mix and match light roasts with dark when making comparisons.

Age of Roasted Coffee: As little as a few days can make a tasteable difference as coffee loses its aromatic gasses and the oils start to go bad. And many people are comparing coffees that are months apart in roast date.

Time Since Grinding: If I gave you coffee ground yesterday versus coffee ground 5 minutes ago, I could convince you they were two different kinds of beans.

Dose/Strength: Here's an obvious one... are you comparing coffees that were brewed to the same strength?

Dwell Time: Don't compare a pot made in 4 minutes to one made in 7.

Extraction: Extraction rate is actually highly dependent on all of the above factors, and from a certain perspective is not in the same category as these other variables. But I'm referring here to the evenness of the extraction. That is... if you make two Chemexes with the same grinder, water, dose, etc. but you pour one straight down the sides and the other one in a careful spiral motion, you will have two different tasting brews.

Non-coffee Factors: There are so many other things that can influence your experience that it would be impossible to list them all here. But pay attention to how long it's been since your last cup of coffee (first one in the morning versus after a long day of sampling coffee), what you last had to eat (spicy food? fudge? cheese?), the environment you are in, expectations you had for a given coffee (due to branding, preconceived notions, etc), and other smells or stimuli around you.

I'm sure I'm missing something in this list. My point is: Be careful! Make sure you are evaluating what you think you are evaluating.

Comments

[Nick Cho]

Turbulence. Slurry percentage (i.e., how much of the Chemex is full of water and how much is wet grounds?). Flow rate (relates to your "Extraction"). Espresso extraction ratio (a.k.a. measuring shots by mass vs. volume).

Good stuff, S.D.

[Daniel Humphries]

Good points, Nick!

Also thought of:

Pressure (for espresso)
Time since brewing
What kind of pretty design is on the cup?