What is survivor bias?

What is “survivor bias”, and why is it important in the study of historical artefacts?

When historical items of any sort are preserved in a collection of any kind, they can give us information about the time period from which they originated. They can tell us more about that kind of item, or the kind of people who would make it or who would use it. Such artefacts are an important element in the study of history.

However, items often end up in a collection for a particular reasons; collectors rarely buy just anything and everything. Therefore, sometimes the items in collections only tell part of the story, or may even give us details that are not representative of the typical example from history.

“Survivor bias” is the name given to these biases (whether negative or positive) that are attached to the survival or an object in a collection. We must consider any survivor bias when we study items in a museum and draw conclusions from a small sample of items.

Something like a sword can end up in a collection for a variety of reasons. It could be collected because it is pretty, or it could be collected because it is an unusually ugly example. What is “ugly” to one person can be “interesting” to another person;[1] for example, some of the 15th and 16th century helmets with fantastical faces could easily be deemed “ugly”, and I certainly find few of them to be “pretty”, but they could easily hold a fascination for someone who wanted to collect unusual helmets.

The piece could end up in a collection of very fine swords, or the collection could be a curation of bad examples. For example, the Arthur and Janet Freeman Collection of Literary and Historical Forgery (also known as the “Bibliotheca Fictiva”) at the the John Hopkins University is a famous collection of over 1500 books and manuscripts, all of which are forgeries and fakes;[2] it is an entire collection of forgeries, with that as the theme and subject of the collection.[3]

The sword could exist in the collection because the theme of the collection is items from the 15th century, or it could be in the collection because it was the only 15th century sword that the collector could acquire and he wanted to add such a thing to the collection.

It could be in the collection because the collector wanted that sword in particular (maybe due to its provenance, its looks, or some other reason), or because the collector just wanted an example of that type of sword and that particular sword happened ot be for sale at the time when the collector was looking for such a piece. For example, if a 9th century Ulfberth sword came up for auction, several collectors would bid on it in order to possess that particular piece, because of its history; if the next item under the hammer was a generic 19th century British cavalry sword of no special provenance, fewer collectors would be interested in that specific sword, although some would still bid on it just to add such an example to a collection of 19th century British cavalry swords.

The sword could have been collected because it had seen use in battle, or it could have remained in the collection precisely because it never saw battle and was therefore never damaged. For example, R.L Scott built his collection of “the real fighting stuff” and was less interested in pieces only for show.[4]

When I taught at the Fechtschule York event in 2013, I attended a handling session at a collection in a museum, and many of the items in that collection were heavy, ungainly, and definitely not agile. They probably remained unused in the collection for exactly these reasons, and therefore survived relatively unused. Taking the small number of samples at this handling session, and inferring that clearly all medieval swords were heavy and unwieldy, would obviously be wrong, because other collections include many finer and more agile examples. Similarly, going to another collection of fine and well-balanced swords, inferring that all medieval swords were perfectly balanced would be equally incorrect, because other collections show worse examples.

When I visited the Royal Armouries in Turin and the collection of arms and armour in the Hofburg Palace in Vienna, I saw many exceptionally fine swords and suits of armour, including the stunning fluted Gothic armour of the Archduke Sigmund of Tyrol; yet it would be a mistake to believe that every single sword and every single suit of armour ever produced were of such standard and quality. It might be sensible to understand that the items on display in a museum called something like the “royal armouries” would probably have been collected due to their ties with the royal house or due to the high quality of their construction, and that what is bought and worn and used by princes and dukes is rarely representative of what the common man used over the centuries.

Conclusion

Whenever you see an item in a collection that stands out for you, for whatever reason, consider the survivor biases that could have led to the inclusion of that item in that collection. Is it truly representative of that kind of item in history? Or is it included in the collection exactly because it is an oddity?

Becoming aware of survivor bias and the range of reasons why items may be included in collections is an important step in being able to use physical items found in collections to support an argument or hypothesis about historical artefacts.

Footnotes

[1] Susan Holloway Scott. “The Horned Helmet of Henry VIII, 1514.” Two Nerdy History Girls, 23rd August 2011, accessed 18th April 2017. http://twonerdyhistorygirls.blogspot.co.uk/2011/08/horned-helmet-of-henry-viii-1514.html

[2] Bret McCabe. “Johns Hopkins shows off its extraordinary collection of fake texts.” John Hopkins University, September-October 2014, accessed 18th April 2017. http://hub.jhu.edu/gazette/2014/september-october/datebook-arts-rare-books/

[3] Isabelle Kargon. “What is the Bibliotheca Fictiva?” The Sheridan Libraries Blog, 26th October 2012, accessed 18th April 2017. http://blogs.library.jhu.edu/wordpress/2012/10/what-is-the-bibliotheca-fictiva/

[4] Glasgow Museums. “Arms and Armour.” Glasgow Museums, accessed 18th April 2017. https://www.glasgowlife.org.uk/museums/kelvingrove/about/collection-highlights/Pages/Arms-and-Armour.aspx