Blayne Sucks

Malcom Gladwell’s recent New Yorker article compares “the quarterback problem” to the challenge of finding a good teacher. It’s an interesting article, but it is, perhaps, too narrow in its focus.

For those who don’t know, the quarterback problem is defined as the extremely difficult task of selecting a quarterback to play in the NFL from the pool of college football quarterbacks. It’s deceptively challenging to do this because there’s so much data available and so many ways to rank college football quarterbacks. However, the college football game is so different from the NFL game that success at the college level seems to have very little correlation, or perhaps no correlation, to success at the professional level.

One of the best examples of this is the comparison of Peyton Manning and Ryan Leaf, who were both extremely successful college quarterbacks drafted first and second in the 1998 NFL draft. Most experts thought it was a toss-up as to which of these two would have a better career. Of course, it’s clear to everyone now that Peyton Manning is a lock for the Hall of Fame while Ryan Leaf is famous for being a complete bust in the NFL.

The key element of the quarterback problem is that past results simply aren’t useful in predicting future success. Gladwell argues that this is also true of selecting good teachers. He claims that the usual metrics used to measure hiring and promotions for teachers, such as master’s degrees, teaching certifications, and other cognitive standards, are just as useless in attempting to determine a good teacher as college football statistics are in trying to determine a successful NFL quarterback.

Another important element of the quarterback problem is that the difference between ‘good’ and ‘bad’ is extremely large, potentially several orders of magnitude. There are very few ‘good’ quarterbacks in the NFL. There aren’t even very many quarterbacks decent enough to serve as an emergency backup. The quarterback position in the NFL may be the single toughest position to play in all of professional sports. Gladwell argues that the same is true of teachers:

Suppose that Mrs. Brown and Mr. Smith both teach a classroom of third graders who score at the fiftieth percentile on math and reading tests on the first day of school, in September. When the students are retested, in June, Mrs. Brown’s class scores at the seventieth percentile, while Mr. Smith’s students have fallen to the fortieth percentile. That change in the students’ rankings, value-added theory says, is a meaningful indicator of how much more effective Mrs. Brown is as a teacher than Mr. Smith.

It’s only a crude measure, of course. A teacher is not solely responsible for how much is learned in a classroom, and not everything of value that a teacher imparts to his or her students can be captured on a standardized test. Nonetheless, if you follow Brown and Smith for three or four years, their effect on their students’ test scores starts to become predictable: with enough data, it is possible to identify who the very good teachers are and who the very poor teachers are. What’s more—and this is the finding that has galvanized the educational world—the difference between good teachers and poor teachers turns out to be vast.

It follows that if you want a school system filled with good teachers, then you have to be willing to identify the poor teachers and get rid of them. This is the only solution to the quarterback problem. It’s a brutal process for both the teachers and the administration. Time Magazine recently had a cover story on Michelle Rhee’s unusual approach to improving schools in the nation’s capital, which is attempting to implement this brutal process.

Rhee wants to solve the quarterback problem the only way possible: by mitigating its effects. Simply put, if you can’t identify good teachers without seeing how they perform in the classroom, then you have to hire a bunch of teachers, watch their classroom performance, identify those that are succeeding, and reward them. Similarly, you have to identify teachers that are failing and eliminate them. Time’s article does a good job explaining why doing these two things is extraordinarily complicated in the teaching industry.

Of course, this is exactly how they solve the quarterback problem in the NFL. On-field performance is everything. Many of the best quarterbacks were identified as such by their play in real NFL games as backups for injured quarterbacks. Matt Cassel is a great example. In college, he never started a game and served as a backup for Carson Palmer and Matt Leinart. He was drafted into the NFL and played as a backup for Tom Brady, who suffered a season-ending injury in the first game of the 2008 season. Brady’s injury made Matt Cassel a starting quarterback for the first time since high school, which would undoubtedly determine his future in the NFL. If he played well, he would likely be rewarded with a starting role for another team during the off season. If he played poorly, he would fall into the nameless abyss of all the other failed NFL quarterbacks.

After reading Gladwell’s article, I had to wonder, how many other professions are like that? Surely the quarterback problem isn’t just limited to teachers and NFL quarterbacks. The first thing that came to my mind was a Paul Graham essay about great programmers, which is really a must-read for anyone in the software industry. In it, Graham talks about the nature of great programmers, and summarizes the problem of identifying them by saying, “The problem is, if you’re not a hacker, you can’t tell who the good hackers are.” Fred Brooks also talks about the vast difference between a great programmer and an average programmer in The Mytical Man-Month. Here’s Fred Brooks on great software designers:

The differences are not minor – it is rather like Salieri and Mozart. Study after study shows that the very best designers produce structures that are faster, smaller, simpler, cleaner, and produced with less effort. The differences between the great and the average approach an order of magnitude.

Clearly, selecting a software engineers fits the definition of the quarterback problem. It would be very interesting to study how the extreme challenge of creating a start-up company performs as a system for identifying great programmers.

Apparently, some people believe there’s a quarterback problem in selecting good lawyers. Although, I have no particular experience with this, I think the environment in which lawyers at big law firms operate is strikingly similar to the ideal solution to the quarterback problem. There’s an incredibly small percentages of lawyers who end up making partner at a big law firm, which indicates to me that there’s a quarterback problem in trying to hire a big law firm partner.

I’ve also seen the suggestion that selecting a mate is a version of the quarterback problem, but I personally think that’s taking things too far. I don’t think that people really have an objective idea of what a good mate is, let alone what metrics to use in measuring potential mates. Furthermore, the role of being a “mate” really isn’t the same thing as having a job.

I believe the quarterback problem is potentially much more prevalent than people currently recognize. I also think that the solution to the quarterback problem is clearly defined. The two important lessons to learn and apply from the quarterback problem:

• Don’t be afraid to give people a chance. They might surprise you.
• Don’t be afraid to make a change when things aren’t working out.

These two steps are the best known solution to the quarterback problem. What other fields could benefit from implementing them? If you have any suggestions for other areas where this problem seems to occur, please mention them in the comments.

If one were to take a national poll on the most important year in United States history, 1776 would easily be a top contender. Thomas Paine published Common Sense in January. Although not an American, Adam Smith published The Wealth of Nations, which had (and continues to have) extraordinary influence on American financial history. The Continental Congress signed the The Declaration of Independence that summer. Perhaps most importantly, George Washington held together the young and inexperienced “rabble” that was the Continental Army throughout the course of that entire year and eventually through the conclusion of the war in 1783. This is the story told by David McCullough’s novel 1776.

Although 1776 is the first book by David McCullough that I have ventured to read, I had most certainly heard of the author before I began reading it. McCullough is one of the most decorated living historians in America, and I would be astonished if more than a few American readers haven’t heard of him. He has twice won the Pulitzer Prize and twice won the National Book Award. He has also received the highest honor bestowed to American civilians: the Presidential Medal of Freedom.

As a result of the author’s fame and previous success, I had extremely high expectations for 1776 in several areas. First, McCullough is known as a fastidious researcher, so I was expecting the book to provide details on the inner workings of the fledgling American Revolution. Second, McCullough is often praised as an exemplary story teller, and with the material he has to work with in the American revolution, I was expecting something amazing. Third, McCullough used many of his notes originally compiled in writing his biography of John Adams, so I was expecting much of the book to deal with the political relationship Washington maintained with the Continental Congress.

In regard to my first expectation, I was quite pleased with the details found throughout the novel. If anything, these details were the most redeeming aspect of the book to those who already know the story quite well. McCullough goes out of his way to drop some now-famous names of people who were at the time merely rising stars, including Alexander Hamilton, James Madison, and Aaron Burr. I was also quite pleased to see many misconceptions dispelled. For example, I grew up learning that the Hessians in the Battle of Trenton were drunk or hungover at the time of the attack. Of course, this has been dispelled in many accounts, as McCullough dispels it here, but it was through this book that I learned the truth of the matter stated plainly.

As to my second expectation, McCullough is certainly a master story teller. Even knowing this beforehand did not prepare me for the fluidity of the book. I was truly surprised at how easy to read it is. It flows so quickly and effortlessly that I found my self reading it in large chunks. One of the primary reasons that I even picked the book up was to find some respite from the large amount of technical reading I have been doing recently, and I am pleased to say that it did not disappoint in the least, even though I was already quite familiar with the major events of the year.

Lastly, I was expecting a focus on Washington’s political discourse with the Continental Congress over the course of the year. This expectation was simply not met. Although there is some discussion of this important relationship, it was certainly not a highlight of the book. Each discussion of the relationship feels unbalanced because the book does not introduce or develop most of the important characters in the Continental Congress. I was left wondering if that is due to the coverage it received in his biography of John Adams, which I will endeavor to read.

My one major disappointment with the book was the artificial framing of the subject. Although 1776 was an extremely important year, it was just one of eight years (1775-1783) that comprised the American Revolution. Quite a few important events are not covered by this book including the winter in Valley Forge, enlisting the aid of France, signing the Treaty of Paris, and George Washington stepping back into the role of a civilian. Perhaps feeling a bit cheated at the missing events was somewhat inevitable. The entirety of the American Revolution itself has been the subject of many books, so I can understand why McCullough would not want to plow the same field all over again. However, I have no doubt that his talents would have made the remaining years and events of the Revolution exceedingly enjoyable stories to read.

On the whole, I can heartily recommend 1776. Owing to its easily accessible nature, it is perhaps best geared towards the average American, but there are certainly details that may interest those more deeply involved in studying history.

I know many readers of this blog also follow ThePrivacyPlace.org, but I wanted to ensure that those who simply follow this one where aware that there is a research survey currently being conducted at ThePrivacyPlace.org. I encourage everyone to participate as this is an excellent way to contribute to academic research and our understanding of online privacy concerns.

Cross posted from ThePrivacyPlace.org:

ThePrivacyPlace.Org Privacy Survey is Underway!

Researchers at ThePrivacyPlace.Org are conducting an online survey about privacy policies and user values. The survey is supported by an NSF ITR grant (National Science Foundation Information Technology Research) and was first offered in 2002. We are offering the survey again in 2008 to reveal how user values have changed over the intervening years. The survey results will help organizations ensure their website privacy practices are aligned with current consumer values.

The URL is: http://theprivacyplace.org/currentsurvey

We need to attract several thousand respondents, and would be most appreciative if you would consider helping us get the word out about the survey, which takes about 5 to 10 minutes to complete. The results will be made available via our project website (http://www.theprivacyplace.org/).

Prizes include $100 Amazon.com gift certificates sponsored by Intel Co. and IBM gifts.

On behalf of the research staff at ThePrivacyPlace.Org, thank you!

I have talked about some of the things I would like to see in electronic book readers before. I have a feeling that this is a list that is fundamentally incomplete simply because I haven’t sat down and actually analyzed the problem from an engineering standpoint. However, I read something on Cult of Mac recently that was truly depressing.

It appears that analyzing electronic book readers from an engineering standpoint is probably fruitless because they are fundamentally crippled from a marketing standpoint. Cult of Mac cites a New York Times blog post about Steve Jobs passion for computers and electronic devices. In that post Steve Jobs is quoted as saying the following about the Amazon Kindle:

“It doesn’t matter how good or bad the product is, the fact is that people don’t read anymore. Forty percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don’t read anymore.”

His opinion on this area is probably more important than the average person might realize. One of the things that Jobs has demanded mercilessly has been incredible user interfaces. This goes back a long way to his interest in calligraphy. If anyone were to design an electronic reader and do it right, it would be Apple. If I were to bet, I would bet that they have considered making a product like this in recent years and ruled it out based on their findings.

Of course, Cult of Mac also posted a link to a recent study on reading by the National Endowment for the Arts. The full study is morbidly fascinating and about 100 pages long, many of which are charts on reading trends. It seems that at every level reading is in decline.

In particular, I thought that the statistics on those who were college educated or above were frightening. The report also makes some general conclusions that the decline of reading skills will affect our democracy. This makes sense if you consider the fact that our government was designed based on the idea that the citizenry would be educated and actively engaged in the process. Sort of an ominous thought to consider…