Disturbing revelations about climate science

on Nov 25 in Climate Change, Software tagged by Trevor Hicks

I’m not talking about the hardball or dirty pool revealed in the leaked emails and data from the Climatic Research Unit at East Anglia University.  I never had any reason to suspect that academics were nicer to each other than, say, investment bankers.  Nope, what has me bothered is actually much more relevant to the results of the research.  Some of the email and source code that was leaked also indicates some rather shockingly poor software engineering processes that call into question the validity of any results produced by the standard climate models.

As far as I can tell, there are three legs of support for the notion that increased concentrations of greenhouse gases in the atmosphere such as CO2 resulting from emissions from human activity are warming the planet.  First, there is the theoretical science based on the physics of the gases and water vapor, the energy of the sun’s rays and how they are affected by passing through or reflecting off various materials, etc.  The leak shows that the global warming side has been very aggressive about suppressing academic dissent, but it certainly may be the case of the police framing a guilty man.  That is, their dirty tricks don’t, in and of themselves, discredit the science.  Next there is the empirical data, both from modern measurements and from other sources such as tree rings and ice cores that allow us to track and correlate the changes over time in temperature and greenhouse gas concentrations.  And third there are the computer models that are intended to apply the theoretical science to the accumulated data and make predictions about what will be observed in the future.  Unfortunately this leak should cause us to doubt the validity of both the data and the models.

First with respect to the data, it’s clear that there was no source control on the data.  Thus there is no plausible chain of custody if you will that can give us confidence that the raw data being fed into the model is a faithful representation of what was actually recorded by the various sensors or other data collection methods.  This wouldn’t be a huge problem in isolation, but it’s clear that some files of raw data were lost and subsequently ’synthesized’  or reverse engineered from secondary sources.  Also, there is no controlled taxonomy or metadata leaving the interpretation of the various bits of data in question.  The emails and comments in the leak also indicate that some liberties were taken in data input routines to massage or alter the data to ensure that the models would run correctly.  That is potentially really scary.  And I say potentially on purpose, none of this yet amounts to any smoking gun that the data is wrong, it just means we should have a lot less trust in its fidelity.  What we don’t know is the sensitivity of the model to the possible problems in the data.

But we have the same sets of problems with the source code for the models as well.  No source or revision control processes appear to exist.  There was clearly a ‘band-aid’ approach to debugging and no systematic review of the architecture of the model at any time.  And worst of all, testing appears to be a completely foreign notion to this project.  There’s no evidence of any kind of comprehensive test plan and no regression testing which means there is no reason to have any confidence that the model is an accurate implementation of the science.  And unlike similarly complex modeling systems like I’ve worked on throughout my career, there isn’t a good way to really ‘field test’ the predictions of the model against what is observed in actual operation because the time scales are so long.  The approach to making the model’s predictions fit out-of-sample real-world measurements appears to be to apply arbitrary tweaks to either the code or the data to get the pre-ordained ‘correct’ answer.

This is a very similar software system to what would be used by a hedge fund using a quantitative trading strategy.  A hedge fund with a similar approach to quality in its software would probably go out of business very rapidly unless it was just spectacularly lucky.

I’ve tried to be careful to point out that what I’ve seen so far is certainly not a definitive refutation of global warming, far from it.  It’s quite possible that all of these faulty processes and errors in the code or data make very little difference in the accuracy of the published results and forecasts.  But it also forces us to consider that it’s quite possible that the data and climate models are utter garbage.  With so many people around the world now poring over the emails and source code I think we will get a clearer picture on the trustworthiness of the data and models soon.

Personally, I’m still willing to state that I think man-made global warming is probably real and even if the models are (or ought to be) just thrown away I expect that opinion will not immediately change.  I certainly have a lot less confidence in that opinion than before.  In fact, it gives me enough pause to think that we probably ought to have a thorough review and possibly a rewrite of the data and models before we undertake any expensive emissions reductions policies though.

Conservative decision costs Belichick

on Nov 16 in Decision Making tagged by Trevor Hicks

If you don’t follow American football or know the rules, just skip this post.  As someone who likes to read and think about cognitive bias and decision making processes, I’m fascinated by the reaction today to Bill Belichick’s decision to go for it last night on fourth and two from his own 28 yard line with 2 minutes to go and a 6 point lead.  The conventional wisdom in this case is to punt.  Here’s a very typical quote from former coach, now analyst Tony Dungy:

On NBC, former Indy head coach-turned studio analystTony Dungysecond-guessed his onetime rival, saying “You have to punt the ball in that situation. As much as you respect Peyton Manning and his ability, and as much as you may doubt your defense, you’ve got to play the percentages and punt the ball.”

Read more:http://sportsillustrated.cnn.com/2009/writers/don_banks/11/16/colts.pats.insider/index.html#ixzz0X4BmydR5
Get a free NFL Team Jacket and Tee withSI Subscription

What’s really ironic about that quote is that Belichick probably was playing the percentages by going for it in that situation.  The Fifth Down blog at the New York Times has the math.  Some computer guys have a simulator that figured a better chance of winning the game all things considered by going for it rather than by making the conventional decision.

Most of the media seems happy to pile on the decision, calling it arrogant or whatever.  Now, I’m happy to concede that the above referenced computer simulation can not be considered the final word, but I think it shows that at worst the decision was 50-50 with respect to maximizing his team’s chance to win.  In that case, the decision comes down to believing your offense can get those two yards and that your defense can stop the Colts if they don’t.  The fact that there was a negative outcome after this decision should absolutely not taint your assessment of it.  When you are making decisions under uncertainty, the outcome is only relevant regarding how it may motivate you to reassess the information that went in to the decision.  Good decisions can have adverse outcomes and bad decisions sometimes work out just fine.  If your financial advisor was showing positive returns for your portfolio by gambling the money in Vegas, you might be happy with the outcome but you probably wouldn’t think much of the decision-making process.

If you really think that Belichick screwed up, then I challenge you to work out the math honestly yourself and do the Bayesian analysis.  I’ll even help you out here.  First, you need to consider the scenarios and assign probabilities to various outcomes.

A: Probability of a first down when going for it.  In this case, we can assume a 100% chance of victory, it’s probably only slightly less, but will simplify the math.

B: Probability Colts a score touchdown from the New England 30.

C: Probability Colts score a touchdown after a New England punt.

Bear in mind there are all sorts of possible outcomes, there can be fumbles, interceptions, long runbacks, whatever.  But you just have to figure the probability that the Colts get that winning touchdown to assess the New England decision.

What did you get for your answers?  I’ll say A is 60%, B is 60% and C is 40%.  YMMV.  Here’s the math, the probability of a New England victory when they go for it is:

60% * 1 that is, they win if they make the first down + 40% * 40% that is, they still win 40% of the time even turning the ball over on downs. This works out to .6 + .16 = 76% chance of winning the game if they go for it.  Now, what are the chances of winning if they punt?  Well, I assumed they had a 60% chance of winning by punting (40% chance Indy drives down and scores).  So under my assumptions New England raised its chances of winning from 60% to 76% by going for it.  It didn’t work out for them, even if there were some strategy that could have given them a 99% chance of winning it still might not have worked out for them.  But plug your own numbers in, I promise you that you have to put in what I would consider to be some pretty crazy numbers to make this an “obviously” bad decision just looking at the percentages.  Bear in mind that real decision making also includes an intuitive assessment of the situation on the ground that really is not amenable to quantification.  Figuring probabilities like this is an aid to decision making, not a replacement for it.

The main points are to accept that good decisions sometimes don’t pan out.  Don’t shoot the decision maker in that case!  If you manage managers like I do please bear this in mind when running your performance evaluations at the end of the year.  Next, note that following the conventional wisdom is generally the safest route personally, but might not be in the best interest of your team.

The lighter side of Peak Oil

on Nov 12 in Peak Oil tagged by Trevor Hicks

The Hubbert Peak Theory of Rock, or, Why We’re All Out of Good Songs | Overthinking It.

On a blog called Overthinking It, there’s this uncanny correlation on the production of Rolling Stone’s greatest songs of all time and production of oil in the US:

Nick Anderson | The Houston Chronicle’s editorial cartoonist | Chron.com - Houston Chronicle

on Nov 11 in health care tagged by Trevor Hicks

Nick Anderson | The Houston Chronicle’s editorial cartoonist | Chron.com - Houston Chronicle.

Nice cartoon.  I just wish those seeking to increase government control and regulation would realize that this is the inevitable consequence as entrenched corporate interests always co-opt new government powers and programs for their own competitive advantage.  And trying to circumvent this natural outcome with even more restrictions on our freedom like McCain-Feingold just don’t work.  The only solution is to lower the stakes of the game by making government less powerful.

That doesn’t mean we abandon worthwhile initiatives like universal access to basic health care, it just means we should try to do it in simple and efficient ways like with vouchers for the needy, whether that need is economic or from a pre-existing condition.

Where do you want to go today?

on Nov 10 in Globalization tagged by Trevor Hicks

Gallup surveyed people from around the world the last few years and created what it is calling the Potential Net Migration Index (PNMI) which shows the aspirations of people regarding where they would like to live, scaled to the population of the destination.  So even though the US has the largest number of potential migrants to its shores, it ranks just outside of the top 10 in the index rankings because of its large population.  Take a guess what you think will be #1 before clicking the link.

A couple of things I find curious, first, Iraq ranks pretty high relative to my expectations, it’s about the median country in fact.  Even Afghanistan ranks ahead of some places I didn’t think were so bad such as Morocco and Honduras.

Second, this really warmed up the statistician in me.  I’d be very curious to see a multi-factor regression to see what elements might predict these results.  Per-capita income?  Per-capita income of immigrants?  What about some measure of personal safety?  How about the WHO health system rankings?  Well, it made me curious enough that I would be interested in reading such analysis, not to actually do it myself.  C’mon, I got football to watch or something.

Cult of Personality

on Nov 09 in Economics tagged by Trevor Hicks

The New York Times has an interesting profile on Dick Armey, the former Republican congressman turned think tank head. I found a lot to agree with Mr. Armey, such as this:

“We are a wealthy nation, and there is not much reason that I can justify for anybody who lives within our borders doing without essential health care, and I’m happy to tell you that very few people do,” he said. “If there’s 15 percent who are in that category, then let’s construct a 15 percent solution so they can have it. To me, that’s a wonderfully generous act.”

And then, of course, there’s this idiotic statement:

“President Obama is a talented person who showed up at exactly the right time, but I don’t believe the man has ever been exposed to a serious economic idea, and I’m not sure anyone around him ever has, either.”

I pointed out that Obama’s circle of advisers includes some decorated economists, including Larry Summers, the former president of Harvard — a controversial figure but not one generally regarded as an intellectual slacker. “I don’t consider Larry Summers a serious economist,” Armey said. “You can get a Ph.D. from Harvard without ever having seriously considered the subject.”

Beware putting too much stock in the opinions of anyone, even me!

Price information

on Nov 02 in Uncategorized tagged by Trevor Hicks

Ezra Klein, who is really an excellent source for following health care reform, followed up on my article from last week regarding the importance of price information.

There is a simple explanation for why American health care costs so much more than health care in any other country: because we pay so much more for each unit of care. As Halvorson explained, andacademics and consultancies have repeatedly confirmed, if you leaveeverything else the same — the volume of procedures, the days we spend in the hospital, the number of surgeries we need — but plug in the prices Canadians pay, our health-care spending falls by about 50 percent.

OK, obviously he wasn’t following up on my comment. Ezra is right to point out that prices have received short shrift in the national discussion over reform, he fails to mention unfortunately that the Democrat proposal appears to insulate decisions from prices even further. This reform is going the wrong direction and will make our system both cost more and perform worse.

Oil supply

on Oct 30 in Oil Exploration, Peak Oil tagged by Trevor Hicks

I love this article, it’s an excellent little narrative about just how difficult deep water drilling is.  It’s valuable insight when you think about the future of oil supply.

I want to comment briefly on Peak Oil, I enjoy reading about the future of oil supplies, but I find it kind of curious to see so much effort expended trying to prove one way or the other when or if there will be a peak in oil production capacity.  I’m not sure why that really matters.  It’s kind of like how our government and media seem obsessed over GDP numbers.  What we really care about is the well being of the population, not some accounting metric that is heavily influenced by international capital flows and whether consumption is is current production and inventory.  Same for peak oil, I don’t care whether 2005 was really the peak or not, what affects our lives and economy is the price of oil. The prevailing price today would have seemed absurdly high even 10 short years ago. Is this an anomaly, a new normal or a portent of further increase?  I’d like to see the debate focused more on those terms.

While I’m on the topic, I also want to revisit a comment on the terminology that is so frequently abused in the media regarding oil issues. Don’t get me wrong, I’m no pedant, but I think the repeated misuse of terms like oil supply and demand really hinder the public’s understanding of the underlying issues. First, note that supply and demand are NOT numbers - they are curves showing the relationship between quantity and price. We cannot measure supply or demand, they are inferred from the movements in other data. What is measurable are oil production, price and inventory. Consumption is derived very simply from those other numbers.

The problem with being imprecise about these terms is that it obscures just how central price and expectations about future prices are to decisions regarding investments in additional capacity, in tradeoffs between capital costs and efficiency and consumption patterns for governments, businesses and individuals.

Treating price as an arbitrary number imposed on the world by greedy producers leads to really poor policy from government across the spectrum from emissions control, ‘price’ gouging rules and decisions to subsidize various fuel sources. Individuals underestimate their financial risks regarding their commitments to consume energy at certain levels when they buy cars and choose how far to live from work and how large of a house they want to heat and cool.  This leads to irrational demands on government to insulate people from the consequences of their own decisions and punish the ‘greedy’ entities who failed to maintain some price equilibrium or status quo that should never have been expected in the first place.

Price is so central to the proper functioning of any market that government should fiddle around with it at its great peril. The root cause of most problems in American health care today is the fact that 90% of all consumption decisions are made completely independent of any thought to price. My biggest frustration with the reform in the air is that the Democrats seem determined to even further insulate Americans from price information and the consequences of their individual decisions. This reform will make our system much, much worse for that reason.

OK, I drifted a bit on the topic here, sorry about that.

Simplicity vs Efficiency

on Oct 09 in Economics, Employment tagged by Trevor Hicks

I write a lot about how much I prefer government policies to have the virtues of simplicity and efficiency.  But often times, these virtues are at odds with each other. Indeed, much of the complexity that creeps in to policy is an attempt to wrangle the maximum possible efficiency out of whatever regulation or subsidy is on offer. Of course it is also often an attempt to hand out political favors, but I’m going to stick with an “ideal” version of Congress for this post that is genuinely trying only to benefit public wellbeing.

The source of the complexity is a distrust that the individuals involved will make optimal decisions. Bill Clinton once famously articulated his preference for government spending programs over tax breaks when the budget was in surplus in 1999 because “you wouldn’t spend it right.” Recipients of food stamps have to endure all sorts of byzantine rules about what can and cannot be purchased to make sure they buy the “right” foods. And now we have a proposal before Congress to do another round of economic stimulus, but this time via a payroll tax refund for creating new jobs.

Now we are talking about a stimulus plan I can get behind. There are huge problems in our existing stimulus plan that it avoids. The benefits are immediate, not dribbled out over several years. A blanket credit does not try to target winners or losers, if you reduce taxation on employment, employment should increase.

Unfortunately, the pursuit of maximum efficiency is introducing quite a bit of complexity which is, ironically, an enemy of efficiency. That is, I believe the attempt to get the tax credit just exactly perfectly right is counterproductive. The plan goes to great lengths to try to target the tax breaks for newly created jobs. Not only will this system be ripe for gaming, as quite convincingly illustrated by Greg Mankiw, but it seems superfluous. Obama talks a lot about jobs his program creates or saves. A payroll tax credit will be more effective if it simply applies to all legal workers, it won’t be targeted towards “new” jobs, but isn’t saving existing ones also a worthy policy goal? The efficiency loss with a blanket credit is that companies will also get tax breaks for all of the millions upon millions of jobs that weren’t in jeopardy. That is, a perfectly efficient policy would target the margin, credit would only be given for the jobs that are created or saved as a direct result of the policy in question.

But there is another rationalization for the blanket credit. Even with a big efficiency leak by crediting all the jobs that were not in jeopardy, the money is not gone to waste. Indeed, if the tax credit is temporary you would not expect it to impact worker’s wages. Employers would not pass the benefits through if they know that they will be on the hook to pay those higher rates without the tax credit next year. Stay with me a minute because this is a feature, not a bug. With a temporary tax break, the money drops down to corporate profits. In an age of tight credit, investment, expansion and growth have to be funded from profits. It should be noted here that I’m a weirdo that would prefer to abolish all corporate taxes - in a revenue neutral fashion by raising progressive income tax rates of course*. So the tax break would have a large stimulative effect beyond its direct subsidy of increased employment. And it’s just exactly the right kind of stimulus that addresses the biggest problem in the economy which is business’s inability to fund growth plans through debt.

So getting back to the larger point, what do you do when efficiency and simplicity are opposed? I think though efficiency is the more important goal, simplicity is usually the optimal choice. The pursuit of efficiency is often counterproductive when the additional complexity creates systems that are easily co-opted by rent-seeking corporate interests. Choosing simplicity often means that some ideal level of efficiency cannot be achieved, but I think will usually result, paradoxically, in a more efficient outcome than a policy that makes a futile aim for perfect efficiency.

*Regarding corporate taxation, if it could always and everywhere be simply a confiscation of profits it would achieve some of the progressive goals on which it is based. However, the reality is that corporations allocate their tax burden by some combination of reducing profits, reducing wages and raising prices. This allocation depends on the particular competitive situation in which it exists and is neither uniform nor easily predictable. So the reality of corporate taxation is that its burden is shared by shareholders, employees and customers in a completely unpredictable fashion. And even if it could be perfectly tuned to fall only on shareholders, note that more than half our population owns stocks and bonds if not directly, through mutual funds with no way to control the progressivity of its impact. So corporate taxation, in impact, is worse than regressive, it’s random. Better to simply do away with it and collect the money through higher normal income taxes which do a decent job of spreading the impact in a progressive fashion.

Martin Feldstein reads my blog

on Oct 08 in health care tagged by Trevor Hicks

OK, now he doesn’t of course. But his recent Op-Ed in the Washington Post echoes and improves up on the vague idea I sketched out last week for the government to get in the business of providing blanket catastrophic coverage for everyone and pushing individuals to bear the direct cost of their smaller decisions regarding health care and provide a loan facility to help those with costs beyond what they can easily afford and less than the catastrophic deductible. We borrow for houses, cars and education, I see nothing wrong with borrowing for health care when you think of it as an investment in your future ability to enjoy life.  Here’s Feldstein:

A good system should not try to pay all health-care bills. That would lead to excessive demand, wasteful use of expensive technology and, inevitably, rationing in which health-care decisions are taken away from patients and their physicians.

Such a system also keeps us on our currently unsustainable fiscal path. I think the Democrats don’t seem to understand that as much as we’d like to extend better health care and coverage to the poor, our government is already paying for more health care than we can afford. Adding to this burden without slashing benefits elsewhere just doesn’t make any sense. Her’s more:

Here’s a better alternative. Let’s scrap the $220 billion annual health insurance tax subsidy, which is often used to buy the wrong kind of insurance, and use those budget dollars to provide insurance that protects American families from health costs that exceed 15 percent of their income.

I like the 15% threshold rather than a hard dollar figure, we are already reporting our income to the government. This is a simple and efficient way to ensure that the benefits of this program are progressively means-tested. Skipping to the big finish:

The combination of the 15 percent of income cap on out-of-pocket health spending and the credit card would solve the three basic problems of America’s health-care system. Today’s 45 million uninsured would all have coverage. The risk of bankruptcy triggered by large medical bills would be eliminated. And the structure of insurance would no longer be the source of rising health-care costs. All of this would happen without involving the government in the delivery or rationing of health care. It would not increase the national debt or require a rise in tax rates. Now isn’t that a better way?

Simple and efficient usually leads to fair. This idea makes a real reform to the system rather than simply entrenching the existing status quo as the Baucus bill does. It achieves the primary goals of any reform which is to make health care available to all and to protect people from financial catastrophe caused by doctor bills. And it looks pretty low risk.  I’m on board.