Master Blend: Ingredients of an Effective Cigar Review
By Rob Gray, Ph.D.As cigar smokers, we are constantly searching for crucial information about cigars that might help us with our cigar purchasing decisions. We will likely come across many different sources in our search: cigar magazines and newsletters, web sites, blogs, podcasts, and discussion boards. We are also exposed to a wide variety of different methods for evaluating cigars. Each of the people or organizations that employ different methods of reviewing will have their own assumptions as to what makes a good cigar review and will often differ in opinion as to why and how cigars should be evaluated. Some of the ways that cigar evaluations can differ include:
1) Level of detail – Some reviewers will provide a highly detailed description while others prefer a short synopsis
2) Number of samples used for evaluation – Certain methods involve smoking the cigar once vs. smoking the cigar several times before doing the final evaluation.
3) Inclusion of numerical ratings – Some methods provide only a description of the cigar while others provide a description plus a rating score.
4) Number of different evaluators used – Some evaluations are based on experiences of one smoker, while others evaluate a cigar based on the combined experiences and observations of a few, and sometimes many, smokers.
5) Timeline of sampling – Some evaluations are based on smoking a cigar once or a few times in relatively short time period (e.g., 2 weeks) while other evaluations are based on smoking a cigar several times over a relatively long time period (e.g., 2 years).
6) Awareness of cigar identity – Using blind vs. non-blind reviews.
Which combination of these characteristics makes for the most effective cigar review for the purpose of helping us to best decide which cigars we should buy? In this article, I will examine points #1-5 in light of what is known about how we perceive and evaluate things in our environment. For detailed discussion of point #6 please see: “Great Expectations: What Do Blind Cigar Reviews Really Tell Us?”
First, I would like to define some of the terminology that I will be using in this article. A cigar “Review” is a qualitative description of the experience one has while smoking a cigar. The description might include: flavors detected, straightness of burn line, and aromas picked up in smoke. A cigar “Rating” is an attempt to quantify this experience by assigning it a numerical value, such as 92 on a 100-point scale or 18 on a 20-point scale. The current cigar literature is replete with examples of rating scales with different criteria and point structures.
A subjective cigar evaluation is one that is influenced by personal opinions, biases and preferences. This might include a person’s preference for mild vs. full-bodied cigars, their opinion about a particular cigar manufacturer, and their mood at the time of smoking the cigar. An objective cigar evaluation is one that is independent of the characteristics of the person doing the evaluation and is based only on the accurate observation of phenomena such as taste, smell, wrapper color, quality of construction, and other things in the outside world.
An absolute judgment involves making an accurate assessment about the properties of a single object at one point in time. For example, “The cigar’s ring gauge looked to be about 52” and, “This cigar scores 92/100” are absolute judgments. A relative judgment involves accurately comparing the properties of two objects at one point in time. As an example, “The Oliva Serie V is more full-bodied than the Oliva Serie G” is a relative judgment. A relative judgment can also compare the properties of one object at two different points in time: “This cigar is much less harsh after one year of aging as compared to when it is smoked right off the truck”. In theory, all combinations of objective/subjective and absolute/relative are possible. For example, if person employed by a particular company stated, “Our cigar is much smoother than those made by our competition,” this would be a subjective/relative judgment.
Now that I’ve discussed the terminology, let’s look at what we know about our abilities to evaluate and compare things in our environment. First, it is clear that completely objective evaluations are impossible to achieve. Though any given cigar evaluation may address observable phenomena such as cigar flavors and wrapper colors, the outcome of any evaluation will depend on the person conducting it. Therefore, all cigar evaluations are ultimately subjective to the experience of the reviewer/evaluator.
Perhaps the most striking example of this comes from quantum physics where it has been shown that when an experimenter attempts to observe an electron he/she will alter the particle’s path of movement by the very act of measurement. A more common everyday example is that of assessing the temperature of an object with your hand (e.g., a child’s forehead or food just out of the microwave). When we place our hand on the object, heat will transfer from our hand to the object or vice versa resulting in a change in the temperature of the object. So the perceived temperature of the object will depend on the body temperature of the particular person touching it. If we can’t measure basic object properties such as these objectively then how can we possibly expect to objectively assess something as complex as the flavor profile of a cigar? The answer is we can’t! Cigar reviewers can never take themselves out of the equation. Their previous experiences, preferences, and emotional states, will always influence the outcome of the review and/or rating.
A second important point to make, relevant to cigar ratings, is that relative judgments are much more generalizable (i.e., widely applicable) and meaningful as compared to absolute judgments. For example, imagine you are given a dish of Mexican food and asked to rate how spicy it is on a scale of 1 to 10 (an absolute judgment). What score should you give a dish that makes sweat break out when you eat it? How about a dish that has no spice at all? Since the 1-10 scale has no universal meaning, outside of the subjective experience of the evaluator, and since the perception of “spiciness” varies greatly from person-to-person, what does it mean when a particular dish scores a ‘7’? Should a person that likes their food “medium hot” order it? A single rating score is of very little use for guiding our purchasing decisions. Now imagine you are given two dishes of Mexican food (a mild one and a hot one made with red chili) and asked to judge which one is spicier (a relative judgment), would you be able to correctly assess the difference? In this case, while individuals may still vary in their perception of how “hot” each individual dish is; most people would give the same answer for which one is spicier. Therefore, the relative judgment will provide useful and generalizable information to the consumer.
What does this difference between absolute and relative judgments mean for cigar ratings? When we are asked to assign a single rating score to one particular cigar (an absolute judgment) this score will be inconsistent and unreliable across different raters. Relative judgments about cigars (e.g., the same reviewer comparing two different cigars or comparing the same cigar at different times) will be much more generalizable and meaningful.
Characteristics of Cigar Evaluations
In this next section I examine the different characteristics of cigar evaluations in light of the two main points discussed above: (i) our inability to make objective evaluations and (ii) the lack of generalizability and universal meaning of absolute judgments.
1) Level of detail
Given that cigar reviews will always be limited to the personal experiences, characteristics, and moods of the person doing the review and, thus, will not necessarily be generalizable to anyone else, a higher level of detail will be more useful to the person(s) who read the review. For example, a short synopsis such as, “This cigar was full-strength with rich, maduro flavors” would not be very useful because what is meant by “full-strength”, “rich”, and “maduro flavors” will depend on the particular person doing the review. Remember, it is not possible to take the reviewer out of the equation when assessing a property such as a cigar’s strength...it will depend on his/her preferences, smoking experience, current mood, and other factors. A review that stated, “This cigar was full-strength and made me feel slightly light headed. It had flavors of dark chocolate, coffee, and leather that I typically get from a maduro” is more descriptive and would be more helpful in helping the reader to find common ground with the reviewer and the review. The more detail that is given, the better the reader will be able to determine if the cigar will have the characteristics that match their own particular preferences and if the reviewer has similar tastes in cigars. If there is an underlying assumption that whatever the reviewer says about the cigar is their own experience and not necessarily generalizable, then the reader can begin to associate their own experiences with those of the reviewer, but only after trying a few of the same cigars. In other words, they may start to find out that they share very similar experiences with the reviewer and can more often identify with their reviews.
2) Number of samples used for evaluation
Since cigar reviews and ratings are subjective, it is much better to make one’s evaluation of a particular cigar after smoking the cigar a few times rather than passing final judgment after smoking it just once. A single instance of smoking a particular cigar can be influenced by numerous “reviewer factors” including their emotional state (happy vs. sad), time of day, drink pairing (see my article “The Science of Cigar-Drink Pairings”), food that was eaten before or during smoking, and the smoking environment. In science we refer to such factors as noise because they interfere with the factors that you are really interested in when smoking a cigar (called the signal) such as flavor, burn quality, and smoke aroma.
Imagine you receive a phone call from a friend that is speaking to you from a very loud and boisterous environment. In this example, the signal (the message your friend is trying to give you) is masked by the noise in the background (e.g., other people talking, traffic, etc.), so the signal to noise ratio is very low. How do we improve this situation? One way would be to have your friend call you several times and give the same message each time. Every time he/she calls the signal will be the same while the noise will be different. For example, if on his first call a police siren masked the last word in the message it is highly unlikely that the siren will be there again the next time he/she calls. By using multiple samples we increase the signal to noise ratio. The same is true of cigar reviewing. If the evaluation of a particular cigar is based on smoking it once, the signal (flavor, aroma, body, etc.) and noise (mood, food eaten, etc.) will contribute about the same to the smoking experience. But if the cigar is smoked a few times the signal will stay roughly the same while the noise will change each time you smoke the cigar. For example, it is unlikely that a reviewer would be in a bad mood each time he/she smoked the cigar so the effect of mood will be lessened if the cigar is smoked more than once. By using multiple samples we increase the signal to noise ratio and get a better evaluation of the cigar. Note that no matter how many samples are used we can never completely remove noise effects—another reason why it is impossible to achieve purely objective evaluations.
3) Inclusion of numerical ratings
Given that absolute judgments are almost always inaccurate and not generalizable beyond the person making the judgment, a single numerical rating for one particular cigar will be practically useless even if the score is based on smoking the cigar several times. Is a cigar rated ‘92’ a great cigar? On the surface it would seem that “yes it is” except that the meaning of a ‘92’ score will vary greatly from person to person.
Cigar ratings are only useful when two ratings made by the same reviewer are contrasted (e.g., a relative judgment). If the same reviewer gives a cigar a rating of ‘90’ (based on several samples to ensure a high signal to noise ratio) in 2007 and then gives the same cigar a ‘92’ rating in 2008 (again based on several samples) we can confidently infer the cigar improves with age.
What about comparing ratings for different cigars made by the same reviewer? For example, a particular reviewer gives “Cigar A” a score of ‘93’ and “Cigar B” a score of ‘89’. While we can infer something from this comparison it is not what most people would think. Because two different cigars can vary on many dimensions (e.g., flavor, draw, body, aroma) it is not possible to infer from this outcome that “Cigar A” is better than “Cigar B” because that depends on the subjective preferences of the reviewer. For example, “Cigar B” might have been given a lower rating because it is milder in body and the reviewer prefers strong cigars. For a reviewer that prefers milder cigars the ratings might have been completely reversed. The only thing this score will tell us is that, “for this particular reviewer, Cigar A is better than Cigar B”. Whether this information is useful to any given cigar smoker will depend on how closely their own preferences, and tastes match those of the reviewer.
4) Number of different evaluators used
Given that the experiences one has while smoking a cigar are highly subjective (depending on “reviewer factors” such as mood and food eaten prior to smoking) and that cigar ratings are highly inaccurate (due to the inconsistency of absolute judgments from person to person), it makes no sense whatsoever to combine the reviews and/or ratings of multiple smokers to create a single overall evaluation. The following example can illustrate the above point. This is an actual example from a cigar publication where there were 4 different people reviewing and rating a particular cigar. Here was the outcome:
If we combined these 4 reviews and ratings into a single composite evaluation we would get something like: “This cigar was Very Good with a score of 8.9. This cigar had flavors of earth, nuts, spice, chocolate, and mocha, that tended to blend together (‘muddy’) and were not complex.” It is clear that this combined evaluation does not give a good representation of the cigar at all. The average score (8.9 ‘Very Good’) is only consistent with the experiences of one of the reviewers since two found it ‘Excellent’ and one thought it was just ‘Good’. Further, not all of the reviewers noted all 5 of the flavors listed in the composite review. Reviewer #1 said the flavors were “muddy” and indistinguishable, while the other three reviewers could clearly identify particular flavors. And, how do we resolve the fact that some reviewers noted a flawed wrapper (light spots), while one reviewer said it was dark and “attractive”? This is a good example of the absurdity of trying to combine highly subjective and variable cigar evaluations. The more responses you add to a composite cigar review, the more fragmented the descriptions become and the less valid the results to each individual cigar smoker.
5) Timeline of sampling
In cases where reviewers attempt to evaluate the effects of age on a particular cigar, typically a single review/rating of an aged cigar is presented with no comparison made to an evaluation of the same cigar with a different amount of age. As discussed in detail above, a single review and rating is an attempted absolute judgment, which is prone to a great deal of reviewer noise and is likely to be unreliable outside of the reviewer’s own experience. Similarly, due to the subjective nature of reviews, it will also be ineffective to compare the rating for the aged cigar with ratings for un-aged versions of the same cigar made by different reviewers; a practice currently used by some cigar media. The only way to determine the effect of age on a cigar is to have the same person review and rate the cigar (based on multiple samples) at several different points in time as is done with the Stogie Fresh Cigar Reviews.
How to get the most out of cigar reviews
Based on this discussion, here are a few tips that will help you get the most benefit from the cigar reviews you read:
Get to know your reviewer – By far the best way to use cigar reviews to help you with your future purchases is to a find a particular reviewer that has preferences similar to your own. Read several reviews from one particular person to get a feel for their preferred cigar strength and body, their favorite type of wrapper, the drinks they like to pair with their cigars, and the flavors they like. Then, smoke several of the same cigars and see how well your experience matches the reviewer.
Look for reviews with a lot of detail – It will be much easier to determine whether you have similar preferences to a particular reviewer when they give more detail about their cigar experiences. Plus, if the reviewer has done a capable job of researching the background of a cigar, you will benefit from more detailed knowledge about the cigar.
Avoid reviews that combine observations and ratings from several people (or hundreds) and/or base their evaluations on smoking the cigar only once –Evaluations with these characteristics are clearly not very useful for the reasons discussed above.
Understand the values and limitations of numerical cigar ratings – As discussed above, numerical cigar ratings are only useful in two cases: (i) for evaluating the effects of age on a cigar, and (ii) for making comparisons between cigars if, and only if, the reviewer has similar preferences to your own.
Look for non-blind reviews and ratings – for more information on why please see: “Great Expectations: What Do Blind Cigar Reviews Really Tell Us?”
Rob Gray is Stogie Fresh’s resident “Cigar Science Guy.” He is an educator, researcher and writer. Rob conducts research in Human Factors Psychology at Arizona State University.
Rob is a regular contributor to the Stogie Fresh Cigar Journal.
