Sunday, November 11, 2012

Nate Silver & the Rise of Math Geeks

"Nate Silver is God."

At least that's the sentiment of people like Jon Stewart and Democrats across the country who have been calmed and reassured by Nate Silver for the past year that Barack Obama had a 70-90% chance of being re-elected, despite all the press and the abysmal first debate in Denver.  Much of the rest of the country is just now waking up to the significance of this man and his particularly astute mathematical models for prediction and probability.  So, who is Nate Silver?  And how does he know what everyone else didn't?

Nate Silver is a writer and statistician who publishes his insights on election probability on a the blog for the New York Times.  He rose to prominence among the geeks and nerds - and Democrats who read the Times - during the 2008 election when he accurately predicted all fifty states within percentages of 1%.  The key, according to Silver, is his aggregation of the results of numerous polls without bias or prejudice.  Though Silver is clearly a Democrat, he confidently asserts his predictions are simply about algorithms, and the rest of the country - most notably people like Karl Rove - simply don't understand the difference between probability and prediction.

Nate Silver caught the nation's attention the final weeks before the election as many Republicans began to publicly criticize his predictions that Mitt Romney only - at his best polling - had a 25% chance of winning the election.  In fact, after MSNBC conservative commentator Joe Scarborough publicly chastised Nate Silver for "claiming this is anything but a toss-up," Silver responded by crossing the line of journalistic integrity by publicly betting Scarborough $1000 (to the charity of the winner's choice) that his calculations were correct.  While Scarborough didn't - as far as we know - follow through on the bet, the clear winner in this battle is Nate Silver and statisticians.

Nate Silver, who has heavyweight math degrees from both the University of Chicago and the London School of Economics, entered the career of mathematical probability and statistics by honing his craft with number crunching in the world of major league baseball.  This approach of sabermetrics, known to the masses now from the movie and film Moneyball by Michael Lewis, who publicized the approach used by Billy Beane with the Oakland A's and Bill James with the Boston Red Sox.  The phenomenal success of the Oakland A's again this year, despite a low payroll, has continued to validate the role math can play in the unlikeliest of places.  Of course, Nate Silver is the current rock star of mathematical probabilities, and for those who seek to understand it better, he has published a engaging and surprisingly readable book The Signal and the Noise: Why So Many Predictions Fail but Some Don't.    As a testament to Nate Silver's newfound significance, Silver's book sales jumped nearly 800% in the days after the election, as the nation sought to catch up with the "boy who knew."

Mathematics - and geekiness - is definitely basking in some glory right now, as the press shines a spotlight on not only Nate Silver but also the mathematical models that were able to accurately predict the coming threat of Hurricane Sandy.  The role math can play in election prediction or baseball recruiting or even gambling in Vegas - which was glamorized in Ben Mezrich's fascinating "realistic fictional novel" Bringing Down the House: The Inside Story of Six MIT Students Who Took Vegas for Millions - is great publicity for math and the promotion of STEM emphasis in education these days. Certainly, this is a logical development of the rise of people like Bill Gates and Steve Jobs twenty years ago.  Being smart - really, really smart - can be cool.

No comments: