Location:Home > NEWS > Blind Tasting Methodology

Blind Tasting Methodology

Time:2019-11-26 18:18wine - Red wine life health Click:

tasting Blind Methodology

Our Industry Leading Methodology

We have a deep commitment to impartiality, open-mindedness, and fairness. All of our tastings, ratings, and evaluations are done by trained panelists using our proprietary blind tasting methodology and custom-built software. Tastings powered by the Beverage Testing Institute will never accept remuneration for a favorable review; we believe in reviews based on objective, benchmark-based criteria, not subjectivity and bias. We do not require that products adhere to strict style guidelines. Our panels evaluate each product monadically, on its own individual merits. How do we ensure the fairest, most consistent beverage analysis methodology available? Let us explain.

The Tasting Laboratory

Blind Tasting Methodology

One of our blind tasting panels using BTI's custom software

To ensure consistency and accuracy, the Beverage Testing Institute uses a dedicated tasting lab located in our offices in Chicago. The room is specially designed to minimize external factors and maximize our panelists' concentration. We taste in the morning, under the same ideal conditions—yielding far more consistent results than working out of a suitcase, a trade show, or at a producer's facility. We only taste 25-30 wines, 25-35 beers, or 12-18 spirits in our morning sessions to prevent palate fatigue. Tasting more than these amounts in a single day, as most other competitions and many magazines do, negatively affects the accuracy, quality, and consistency of alcoholic beverage evaluations. And of course, we always use a spittoon.

Our Expert, Highly Trained Panels

We conduct daily invite-only judging panels that include highly experienced, professional tasters. Many of our panelists have completed certifications of their own such as SWE, CSW, CSS, MS, Cicerone or others from credible institutions. Our panelists are industry buyers and experts such as beverage directors, educators, retailers, restaurateurs or prominent writers. All panelists are rigorously screened and audited and then trained in our proprietary blind tasting methodology. They follow strict protocols for maintaining the tasting room environment and priming their palate prior to analysis. For further information on our panelists click here: Meet our Judges

The Analysis

Blind Tasting Methodology

All products are tasted in Riedel crystalware

A basic tenet of qualitative evaluation is the shorter the scale, the easier it is for panelists to repeat their scores — and repeatability is the essence of accuracy. Our panelists taste in two rounds. We use a simple scale with four bands in the first round:

1 — Not recommended   (less than 80 points)

2 — Of sound commercial quality, though not overly exciting   (80-84)

3 — Shows style and character, yet probably not of the highest merit   (85-89)

4 — Highest quality   (90+)

With this scale, panelists do not concern themselves with the difference between an 88 and an 86, or a 90 and a 91. Products with two or more judges scores of "4" are sent to the "merit round," whose range is as follows:

3 — Very good, but not of the highest merit   (88-89 points)

4 — Truly excellent in style and distinction   (90-92)

5 — Outstanding, though not quite one of the world's finest   (93-95)

6 — Provides a world-class experience   (96-100)

This method allows our tasters to consider general quality, without the distraction of adding points. Products breaking the critical 90-point gold medal barrier are tasted twice without exception, virtually guaranteeing that they deserve their accolades.

The scores are then converted to a modified consumer-friendly 100-point scale. The below five quality bands of score more accurately reflects the quality of products in today's market. It roughly corresponds to a five-star system:

96-100 — Superlative   (Platinum Medal)

90-95 — Exceptional   (Gold Medal)

85-89 — Highly Recommended   (Silver Medal)

80-84 — Recommended   (Bronze Medal)

less than 80 — Not Recommended   (No Medal)

To help translate the panels' scores, we use a different statistical method, relying on the mode (the most frequent score) rather than the mean (the average score). This helps reduce the impact of maverick scores; the mode is much closer to what the panelists are really saying. If, for instance, a product in the first round receives three scores of "3," it is placed in the upper center of its band (85-89) and given a final score of 88 points. Should the third score be a 4 or a 2, the product in question would be given an 89 or an 86, respectively. The third score is used to move the total score up or down within the same band. Again, controversial permutations are re-tasted. Those falling in the lowest band (less than 80) are simply listed as "79/NR" (Not Recommended) and no specific score is assigned.

Editorial Retastes and Crosschecks

Blind Tasting Methodology

Editorial re-taste session in action

Copyright infringement? Click Here!