The Defense Dilemma: The Inherent Flaws of MLB's Defensive Statistics (Part 1)

Evaluating MLB player performance, or even more daunting, predicting it, is an endeavor requiring its own science in Sabermetrics. To accomplish this feat, analysts have placed impetus on a player’s offensive capabilities to make their evaluations, largely leaving defensive statistics behind.  Why?

Defensive statistics are inadequate descriptors of player performance for many good reasons. Basic defensive statistics and are largely influenced by human judgement: too subjective to incorporate into any meaningful (data-driven) decision making. Advanced statistics can hide behind fancy acronyms, but they aren’t any better at evaluating how well a player defends their position. Furthermore, defensive statistics are inherently flawed. Comparing players at different positions, playing in various parks, and facing the unpredictability of each play makes it nearly impossible to gather actionable conclusions.  Defensive Statistics were not “born on 3rd base," so to speak: Let’s take a look at how we got here.

Before the founding of either of today’s American or National leagues, Henry Chadwick publicized a box score in 1859. Chadwick, inspired by scorekeeping in cricket, sought to introduce a box score to baseball to grow the sport’s audience, keep fans engaged, and most importantly for students of today’s game, judge player performance. Chadwick notes, “In order to properly judge of a player's work, it is necessary to keep a record of his doings, and to tabulate them so that a summary can be made, showing the full measure of his ability”. Chadwick's aim laid the groundwork for early defensive statistics like putouts and errors.

An early box score, showing plate appearances, runs, singles, putouts, assists, and errors from left to right.

               Putouts are credited to a fielder who physically records the act of completing an out, such as a caught fly ball or a runner tagged out. Errors are charged to fielders who mishandle plays which should have resulted in an out, allowing a baserunner to reach base safely and/or take extra bases. If these definitions seem acceptable, consider the following: A fly ball dropping between two outfielders. Which outfielder should have caught the ball? Which should be charged an error for letting it drop? Another scenario: An infielder smothers a sharply hit ground ball destined for the outfield, springing up to make a difficult, off-balance throw missing its target. Is this an error, because the runner was not put out, or is it not, because the play required an above-average effort from the infielder? Each scenario could be scored differently depending on who is watching the game.

               While putouts and errors do provide value, all early statistics were calculated by an “official scorekeeper,” introducing heavy subjectivity. In the early days of baseball, there was typically one official scorekeeper monitoring each game. These scorekeepers did not travel with the team due to financial and logistical constraints of the time. Consequently, the scorekeeper was usually affiliated with the home team, introducing bias and potential pressure from the club’s staff to ensure the game record positively reflected the team’s performance. Unlike umpires, who required extensive training to officiate games, scorekeepers did not undergo formal training and certification. The lack of standardized training resulted in scorekeepers having varied levels of understanding of the rules and criteria for scoring. Additionally, the technology available to scorekeepers and managers in more modern eras was nonexistent in the mid-late 1900s. As a result, the scorekeeper had to rely solely on their judgement, often with limited perspective on fast-paced and complex plays.

Over time, defensive statistics have seen significant improvements due to advancements in training and technology. Scorekeepers have more tools and guidelines today, reducing biases and errors over time. The official rules of baseball were codified in the early 20th century. In 1980, the scorekeepers were employed by the commissioner’s office, relieving the scorer of biases towards the teams paying them. Beginning in the 90s, digital scoring systems allowed for real-time data entry and automatic calculations of rate stats like OBP. During the early-mid 2000s, scorekeeper training was standardized to improve their function. In 2008, instant replay began allowing for more accurate assessments of close plays, and multiple camera angles cover the field to provide better perspectives. Recent updates include Statcast, a system comprised of multiple high-speed cameras and pieces of radar equipment to automatically calculate statistics in every area of play. To this day the lexicon of baseball statistics grows, perhaps more rapidly now than ever before. Despite these technological advancements, inherent flaws in defensive statistics persist, making them poor indicators of a player’s past or future performance.

Part two will discuss how modern defensive statistics obscure what a player has done on the field to contribute to their team’s success, and what to keep in mind when discussing defensive metrics.