Monday, May 17, 2010

Moneyball: The SCIENCE of Winning an Unfair Game

This is an email I sent to my boss this morning. Baseball really can be used to achieve everything you want in life.
(Dr. M is the Chief Medical Officer and I'm the Regional Director of Medical Education.)

Dear Dr. M,
I went to LA this weekend and listened to audiobooks on the drive. Yesterday I listened to Moneyball: The Art of Winning an Unfair Game, by Michael Lewis. I can't believe I hadn't read it before -- it's been on my list for a long time. It's about how Billy Beane, the General Manager of the Oakland As, put together consistently winning teams despite having the lowest payroll in baseball. The concepts described can be applied to medical education, the students being the baseball players. In my analogy, you are the team owner, I'm the GM, Dr. W is the assistant GM, the attendings are the coaches, and the students are the players.

First of all, the title of the book is wrong. It should be The SCIENCE of Winning an Unfair Game, not the ART. This is what I call Evidence Based Baseball. We are trying to practice Evidence Based Medicine when it comes to patient care, but I believe we should try to find a way to apply Evidenced Based Baseball principles to medical education. We should pick medical students the way Billy Beane picks baseball players.

The Billy Beane approach was not just unique in baseball, it was nearly heresy. For over a century, baseball has been run by tobacco-spittin old guys who did things because their coaches did it that way because their coaches did it that way. Just like in Medicine. We do it because our attendings did it because their attendings did it. Both in execution (baseball strategies/patient care) and personnel selection (players/students). Baseball was run by tradition and habit. Beane and his Harvard-educated-geeks-with-laptops wanted to look at the statistics, the evidence, and do things in a new way.

I'll try to distill this down.

The Oakland As had the lowest payroll in Baseball. The Yankees and Red Sox had the highest payroll. Billy Beane was the GM of the As. How could he get his team to compete? Conventional Wisdom said you: have a lot of money, you buy the best players, you win. But what exactly is "the best player"?

First: define your goal. The ultimate goal is to win the World Series. The practical goal is to get to the playoffs every year. How do you get to the playoffs? You win games. How many wins does it take to get to the playoffs? 95. Not a number picked out of the air, but by looking at statistics. Now, how do you win 95 games? Conventional Wisdom said it had to do with the team's ERA or batting average or home runs or steals or defense. The evidence said it was about one thing: run differential. At the end of the season, if your team scores a certain number of runs more than your opponents, you will win 95 games. (I don't remember the number, but it was a specific number.) Great. Now, how do you get that run differential? The statistical evidence showed that it's about On Base Percentage, Slugging Percentage, and the OPS, but primarily about OBP. How important is pitching and defense? Statistically, not as important as has traditionally been thought.

I'm skipping A LOT here for brevity, but let's talk about OBP. Beane didn't have much money, so he wanted to get the "best" player for his money. He wanted undervalued players, players with a high OBP who weren't considered top prospects by the old tobacco-spittin scouts that utilized their eyes to evaluate players and had no knowledge of sabermetrics.

Now: who has a high OBP? Specifically, what kind of player consistently has a high OBP? It all boils down to the strike zone and the players ability to see the strike zone, see the pitches, and make good choices at the plate. It's about PLATE DISCIPLINE.

Now back to medicine. I have all these ideas floating around that I need to organize and settle. We need to decide what our ultimate goal is. What is our equivalent of getting to the playoffs and winning games? Do we want to produce doctors who will score highest on the Boards? Do we want to produce doctors who can churn out a large volume of patients? Do we want to produce specialists? Do we want to produce doctors who will return to our health center to practice primary care medicine? We have to define our goal.

The next part will be harder: how to achieve the goal we've set. In baseball, there are statistics for everything going back to the 1800s. We'll have to do some research on what kind of medical student becomes the doctor we want to produce. Is it the kid with the highest MCAT scores and GPA? Probably not. Is it the kid who just barely made an acceptable MCAT score or GPA? Maybe not that kid, either, but we don't know that yet. Is it the kid who speaks Spanish or spent a summer in Honduras or who did research as an undergrad?

Back to baseball, the As trained their minor league coaches to teach plate discipline to their players in the farm system. It turns out, the player who naturally has plate discipline will always have it. It's easier in the long run to find a player with plate discipline than to create one.

We need to figure out the medical student equivalent of On Base Percentage (the result we're looking for) and Plate Discipline (the QUALITY that gets us to that result).

When we took a look at the first year medical students who wanted to be placed at our site, I made up a list of qualities I wanted in a student to choose from among them - our version of the Draft. We got 90% of the students we wanted. We'll see how that works out. By next year, I hope to work that out with more evidence and less "gut."

No comments:

Post a Comment