What’s Wrong with the AIS Awards System

by Tom Waters

One of the most important functions of the American Iris Society (AIS) is to carefully evaluate new irises as they grow in gardens and decide which are worthy of commendation and can be recommended to the gardening public. This is done through a system of trained garden judges working in all geographical regions, who evaluate the irises and vote them awards.

I’ve been growing irises on and off since the 1970s, and served as a judge for many years. There have always been grumblings about the award system, from simple shaking of the head (“What were the judges thinking?”) to tales of secret regional cabals working to subvert the process. I’ve not taken much heed of such complaints, attributing them to a combination of sour grapes and the ubiquitous human inclination to complain and gossip. Although there are exceptions, I’m sure, judges I have known personally have all been honest, conscientious, and reasonably skilled and knowledgeable. They do their very best to vote for irises they deem truly worthy of recognition.

Nevertheless, I think there is a fundamental structural problem with the process of voting for AIS awards that keeps some good irises from being recognized and elevates some mediocre ones to unearned fame.

The awards system asks judges to vote following the model of a political election: an assortment of eligible candidates are placed on the ballot, and the judges are to vote for the one(s) they deem best. For this system to identify the best irises, judges need to be familiar with all or most of the candidates on the ballot. The rules state that you should not vote for an iris unless you have seen it growing in a garden (or gardens) over more than one year. Ideally, the judges should grow the irises themselves. The ideal of judges intimately familiar with all the candidates is not usually met. Often, judges have seen only a smattering of the eligible irises (particularly for early awards, such as honorable mention). They may select the best of those they are familiar with, but if they are only familiar with 10%, what of the other 90%?

When there are many names on the ballot, but only a few are actually seen and evaluated by the judges, the system is very vulnerable to a particular sort of bias. Not an intentional bias on the part of judges, but a systemic bias built in to the process: the more widely grown an iris is, the more likely it is to win awards.

Consider this hypothetical. Assume there are about 400 judges voting. Iris A is bred by a famous hybridizer that many iris growers order from. It is thus widely distributed and widely grown. 350 of those judges have seen it growing in a garden. It is a nice iris, but only 10% of the judges who have seen it think it should win the award. 10% is still 35 judges! Now consider iris B, introduced through a smaller iris garden that sells only a few irises each year. Maybe only 20 judges grow iris B. But iris B is extraordinary! It is so good in every way that 90% of the judges who grow it think it should win the award! But 90% of 20 judges is just 18, so iris B gets only about half the votes of iris A, although it is clearly a much better iris.

Note that this undesirable result is not a consequence of anyone making bad choices, being unethical, or doing anything wrong. The hybridizers, growers, and judges are all doing their best; it’s just the way the numbers play out.

Another way to look at this phenomenon is to consider the meaning of a judge voting for an iris or not voting for an iris. Clearly, a vote for an iris means the judge thought it was the best among those seen. But what does a judge not voting for an iris mean? It can mean two very different things: it can mean the judge has evaluated the iris and found it wanting, or it can simply mean the judge has not seen the iris. These are two very different circumstances, and treating them the same is a very bad idea.

In 2019, 378 judges voted for the Dykes Medal, and the iris that won received only 29 votes. That’s less than 8%. This is nothing new, it is typical of recent years. What does that mean? It is difficult for the public to be confident that this is the best iris of the year, when we don’t know what the other 349 judges thought of it. Did they love it, but just slightly preferred another iris over it? Did they think it was bad? Did they just not see it? Such ambivalent results are a direct consequence of using an election model with a long list of candidates, many of which are not familiar to most of the judges.

There is a way to address this structural bias. If we moved from an election model to a rating model, we could much more accurately identify the worthiest irises. A rating model is what is commonly used for reviews of products, businesses, restaurants, and so on. Everyone who is familiar with the product gives it a rating, and the average of those ratings is what helps future consumers decide whether the product is worthy or not.

How would a rating system for irises work? It would not have to be as elaborate as the 100-point scoring systems presented in the judges’ handbook. A rating from 1-10 would do just fine, or even a scale of 1-5 stars, like you often see in other product ratings.

Consider our two hypothetical irises again. Assume that judges who vote the iris worthy of the award rate it at 5 stars, and those who have seen it but do not vote for it rate at 3 stars. Iris A, which 350 have seen but only 10% vote for, would have an average rating of (315 x 3 + 35 x 5)/350 = 3.2. Iris B, which only 20 judges have seen but 90% vote for, would have an average rating of (2 x 3 + 18 x 5)/20 = 4.8. Iris B is the clear winner, as I think it should be.

In this system, judges would enter a rating for every iris they have evaluated. They would not have to pick the single best one to receive an award. They could rate any number of irises highly, and if they saw some with serious faults, they could give them low ratings, which would bring the average rating down and make it much less likely for these poorer irises to win awards, no matter how widely grown they are.

Judges would not enter a rating for irises they had not evaluated. So their not having seen it would not penalize the iris, since it would not affect its average rating at all. A non-rating (from not having seen the iris) would have a very different consequence from a low rating (the judge evaluated the iris and found it unworthy).

If such a system were implemented, some additional considerations would probably have to come into play. We might want the iris to be rated by some minimum number of judges before we would trust the average and give it an award, for example. We could also use this system to check for consistent performance in geographical areas, if that were deemed desirable. We could also demand a certain minimum average rating (say 4, perhaps), so that if no candidate iris were rated very highly, no award would be given.

Under the current system, I think the training and skill of the judges is largely wasted. They evaluate many irises over the course of the year, and form opinions about each one. That information is lost when they are instructed to simply vote for the best one. Every time a judge rates an iris favorably, its chance of receiving an award should go up; every time a judge rates an iris unfavorably, its chance should go down. Not being seen should not be a penalty.

A rating system would also encourage new hybridizers, as it would give us a way to recognize really exceptional irises that aren’t introduced through the big growers. It would allow hybridizers to build their reputation by receiving awards for quality work, rather than receiving awards because of an established reputation. Established hybridizers would not be much hurt by such a change; they still have the advantage of large, extended breeding programs and experience in recognizing quality seedlings. They don’t need the additional advantage of distribution bias to have a fair chance at awards.

I hope this post stimulates some discussion on the topic of our awards system and the consequences of structuring it as we have. I see the potential to improve the system in a way that makes it more fair to all new irises, more useful and credible with the gardening public, more supportive of new hybridizers, and more conscientious in reflecting the careful evaluation work of our judges.

On the Road Again: The Vaughn Garden in Salem

By Bryce Williamson

After a too short visit to the Keppel garden—it would be possible to spend days there watching the bloom unfold, it was on the road again this time to head south of Salem to the garden of Kevin Vaughn.

Vaughn T-18-1
A tetraploid MTB voted  best seeding at the 2018 Region 14 Spring meeting.

When Kevin retired from the USDA job, he found 3 acres south of Salem. Just as Lynda Miller had provided a short cut to get to her garden, Kevin had told me a quicker way to get to his garden by skirting the east side of Salem before cutting over to his place on

River Road.

Kevin brings a wealth of scientific knowledge to irises and he as been a frequent and useful contributor to The World of Irises blog. While working for the USDA, he published over 160 scientific papers and recently he wrote Beardless Irises A Plant for Every Garden Situation and is now working on a book about median irises. His Louisiana irises have won awards and in 2019, his ‘Lemon Zest’ was one for the winners of the Mary Swords Deballion Medal for Louisiana irises.

He has been raising Siberians and out of the selections below, several will be introduced when stock allows.

His hybridizing interests in irises ranging from dwarfs through tall bearded and including in addition to the above Siberian seedlings, Louisianas and Spurias, and that interest is match also  by his interest in hybridizing other types of plants too. In his twenties, Kevin was a bright star of hosta breeding and the American Hosta Society have honored Kevin's contributions to the development of the genus Hosta by establishing in 2001 the Kevin Vaughn Award, which is given to the entered sport that is chosen as Best Overall by the AHS Judges.

With a new garden, Kevin has returned to hosta hybridizing something that he could not do during his years stationed in the South for USDA.

From his early teens, he was interested in breeding sempervivums and continues to do so. To find out more about his creations in that area, follow the this link. Daylilies and daffodils have experienced his touch; most recently he has seed from miniature gladiolus. A trip to this garden is always rewarding.

In the last two years, one of the most interested bearded irises in the yard is a Witt seedling—Kevin is growing the last of Jean Witt’s irises and the first of the selections, ‘Just A Dusting’, will be introduced by Aitken’s Salmon Creek in 2020. The most interesting diploid iris in the garden is the reddest beard iris any of us have seen. It will never be introduced since it is too big for the MTB class, but some of us are growing it in the hopes of using tetraploid pollen on it and get—if we are very lucky—a tetraploid from those crosses that should bring new a new source of red pigments into tall bearded irises.

To make sure he does not get stale in retirement, he is also an accomplished musician playing wind instruments, mainly oboe, in Salem Symphonic Winds, Salem Orchestra and Winds of the Willamette wind quintet.  From time to time he also plays for musicals and other groups.

In between checking out plants and great conversation that continued over dinner at Roberts Crossing Restaurant on River Road (why can't we have a high quality, reasonable priced eatery in my neighborhood like this?), Kevin and I made a dash to Larry Lauer's garden nearby garden and that will be the subject of my next 'On the Road Again' post.

Did We Give Up on the Recessive Amoenas too Early?

By Kevin Vaughn

Amoenas and variegatas have long been favorites of iris growers.  The early amoenas and variegatas were all derived from I. variegata and had many problems associated with that species, chiefly very veined hafts, and a pattern of striped falls rather than solid ones.  Breeders were persistent in their work, despite poor germination of crosses involving amoenas. Cultivars like the Dykes Medal winner dark purple amoena ‘Wabash’ and variegatas like ‘Mexico’ and ‘Pretender’ were very popular irises in their day and had covered up most of the problems in this breeding line.  In fact, ‘Wabash’ topped the AIS popularity poll for many years. Crosses of these recessive amoenas to yellow amoenas resulted in the unusual green amoena ‘Frosted Mint’ (one of my childhood favorites) and red amoena ‘Repartee’. (Editor's note: Kevin sent images of seedlings as examples and I do not have numbers for any of these photos, but they give an idea of what he is getting out of these lines.)

A revolution happened in the 50’s when Paul Cook found that I. reichenbachii had a dominant inhibitor of the TB anthocyanins in the standards only, resulting in amoenas, neglectas, and bicolor patterns only dreamed about in the past.  Paul’s ‘Whole Cloth’ won the Dykes Medal and most deservedly so as it was not only a beautiful iris but an important breeders’ iris as well. Almost all breeders made a few crosses with ‘Whole Cloth’ and generally with good results.  Moreover, these dominant amoenas had seed that germinated well and the flowers had none of the veined hafts so typical of the recessive amoenas of the past. People flocked to these amoenas, basically dropping the old recessive amoenas, although Jesse Wills, Catherine and Kenneth Smith and a few other brave souls kept the recessive amoenas going for a while. Catherine Smith’s ‘Bread and Wine’ may be the last of these, in the 1970’s.

In the MTBs, recessive amoenas and variegatas are alive and well!  In fact, if you breed MTBs you can’t help but getting them in spades.  As I looked through my patch of MTB seedlings this past spring, I saw many colors and patterns that I have not seen much or at all in the TBs.  The variety of variegatas is quite staggering, including those with patterning of colors on the falls, stitching on the standards, and several just in STRIPES.  In the opposite direction, my ‘Booyah’ and many of its seedlings had nearly solid falls with minimal haft marks. Amoenas followed the same patterns, with some striped variations in delicate stripes and some all-over versions as well as nearly solid colored falls.  When Rick Tasco, Roger Duncan and Keith Keppel stopped by one day, they were equally impressed in the variety of colors and patterns that were occurring in these recessive amoenas and variegatas.

When these recessive amoenas and variegatas are crossed into the plicatas and then recombined the variety of plicata patterns also increased with lots of strong bitoned and bicolor plicatas plus unusual distributions of the dots and spots.  I inherited a number of Jean Witt’s seedlings and she had explored these variegata-influenced plicatas a good bit. Crosses with them give even further variations. Some of these plicatas have the looks of things that Rick Ernst got out of his ‘Ring Around Rosie’ lines.  These tetraploid lines may have some variegata influence too.

Catherine Smith sent me a plant of ‘Repartee’ when I was a kid. I was impressed with how red the falls of that iris are and especially so when it bloomed in my garden in 1968!  Amazingly it still looks very red to me. ‘Repartee’ is a cross of the purple neglecta ‘Grosvenor’ with a yellow amoena and that overlay of strong purple with an inner yellow layer gives a very red effect.  Admittedly, ‘Repartee’ has some problems. The stalk is awful with buds toed in and the form of the flower is a bit “blobby”, although I have seen worse.

Fortunately yellow amoenas have improved a LOT since 1968 and I crossed several of them onto ‘Repartee’.  The F1 crosses of ‘Repartee’ X yellow amoenas gave almost all red amoenas/ pale variegatas. The stalks on these were much better than ‘Repartee’ and the forms began to approach the yellow amoenas.  Rather than sibbing these seedlings, I crossed them once again to yellow amoenas, hoping to obtain even better form. Those seedlings bloomed in 2018 and the improvements were noticeable. Much better forms and quite red falls were the norm.  Interestingly, several of these seedlings didn’t have solid falls but rather a splashed phenotype. Yellow amoenas have the dominant inhibitor I that suppresses anthocyanin production but the anthocyanins in amoenas are not fully inhibited by this gene.  It is possible that these anthocyanin-free sectors are due to some partial inhibition by I.  None of these is quite a finished product but they are interesting and with quite good color and very vigorous plants.  I’ll see ~50of the best of these red amoenas sib crossed (best form X best color) in the spring. I don’t expect any introductions from this line in the near future, but it’s been fun to see what the recessive amoenas can do.

This makes me wonder out loud whether we were wise in giving up on recessive amoenas too early despite their many problems.  Hopefully there will be a few more brave souls out there to use them in crosses.

IRISES: The Bulletin of the AIS - Fall 2019 Edition

By Andi Rivarola

A warm welcome to those who are seeing IRISES, the Bulletin of The American Iris Society for the first time. If you are a member of The American Iris Society I hope you enjoy this new issue.

The Fall issue of the AIS Bulletin will be available online soon, accessible via the Emembers section of the AIS website. The print copy has been mailed via the U.S. Post Office. On the cover, The 2019 Dykes Medal Winner 'Bottle Rocket’ (Michael Sutton 2009, TB).

Note: to access this area of the website you must have a current AIS Emembership. (AIS Emembership is separate from the normal AIS membership.) Please see the Electronic Membership Information area of the AIS website for more details.

Enjoy a full list of all the AIS Award for 2019, including beautiful 'Bottle Rocket,' above by Mike Sutton. Pages 10 - 17, then images of these glorious winners on pages 2-3, 17, and then 62 - 63.

The registration information for the 2020 Centennial National Convention to be held in Newark, New Jersey is on pages 32 - 33.

International news on pages 34 - 35. 

A wonderful article filled with tips by Australian hybridizer Barry Blyth called, Barry Blyth Reveals His Iris Photo Secrets, on pages 36 and 37.

A very interesting article on the genetics of diploids versus triploid irises called From Triploid Bridge to Diploid Pink by Dan Meckenstock on pages 40 - 45. 

There's a lot more to see and read in this edition of IRISES, either in digital or print formats.

Not a member of The American Iris Society? Please see our website for information about becoming one: http://irises.org/

Happy Gardening!