Bogus Studies & Unfounded Data: A Warning

Repeat the same mistaken data enough times and people start believing it. If you’re active in policy or a reporter and think any of these are accurate, please email me for primary sources. I’m writing this quick note because two respected groups just sent to me stuff I know is wrong.

There is a large, measurable cost to Internet shutdowns. If you can send your birthday gift a few days later or catch up two days of missed lessons, the Brookings report falls apart.

Connecting more people to the Internet has large economic benefits. Senior economists, including one now at Harvard, debunked this in 2009 in D.C. 

The Internet has no economic downsides. Tell that to the store owner who went broke because he couldn’t compete with giant Amazon. About a third of the world’s newspaper reporters lost their jobs. Almost all claims about “economic benefits” leave out the negative.

False Claims about Internet Benefits

If any of this is wrong, please email me. In particular, if you find solid primary data that says otherwise, please send it to I make many mistakes, as the butler said.

Cost of economic shutdowns. “Internet shutdowns cost countries $2.4 billion last year,” writes Darrell West of Brookings. He takes (inflated) estimates of the economic contribution of the Internet and divides them by the number of days the net is out. As he acknowledged when I asked, he includes activities that could be performed just as well the next day, saying he doesn’t know how to measure them. I would guess his figure should be reduced by 70-90%. At the minimum, the work should be corrected to prominently indicate his figure is not an accurate measure of the cost. He hasn’t corrected the error despite three requests. The Deloitte work has a similar problem.

Connecting more people to the Internet has large economic benefits. Ask yourself: Is checking Facebook, glancing at the news, and watching porn, or looking up a recipe likely to have a big economic effect. Those are the kind of things most people do on the net. Those get computer science degrees 8,000 miles away from Georgia Tech are a small minority.

Video calling your mother 12,000 miles away is a good thing. So is relieving the tedium of housebound elderly with video and things to read. Organizing everything from bowling leagues to political protests is much easier. Maybe reading Net Policy News is a good thing. 

The total economic impact seems to be too small to measure in the most careful studies. I’ve earned my living since 1999 writing about the Internet and wish there was a big economic benefit. It’s not in the data.

Anyone with common sense can see that growth has fallen most places despite a remarkable increase in Internet users. Productivity plummeted after 2005 as the worldwide Internet growth accelerated. Just one example: India, one of the fastest growing economies, has lagged far behind in Internet use until the last year or two.

The many claims otherwise, when examined, range from statistically weak to total garbage. Nearly all the positive claims do not account for a likely confounding variable: investors in broadband networks are usually not stupid. They will build where the economic prospects are most attractive, such as areas likely to grow fast.

The dozen+ I’ve checked ignore basic statistical principles. This shouldn’t be surprising. Stanford Professor Ioannidis founded about half the work in leading medical journals didn’t hold up. His now classic 2005 paper, Why Most Published Research Findings Are False, is an inspiration for evidence-based medicine. We need evidenced-based policy as well. 

Shane Greenstein, now at Harvard, did a study that concluded then prominent claims of benefit had to be reduced by about 90%. He presented the work at a Columbia event in D.C. where Bob Crandall & Scott Wallsten generally agreed. A 2016 World Bank report should have put this to rest. “The Internet has not resulted in successful development. … The broader benefits of higher growth, more jobs, and better services have fallen short.” 

I could go on nearly forever and have at great length over a decade. The will to believe – and financial interests – keeps this bs going.

There are no downsides to the Internet. Most of us agree that on balance the Internet is a good thing. I strongly support expanding it. But when you try to measure the impact, you must include both the positive and many negative factors. Store clerks lose their jobs to e-commerce; reporters to news outlets on the web; radiologists in the U.S. report an income drop as cheaper doctors in India read more xrays. 

Columbia Professor Raul Katz is one of the few to have incorporated both upsides and downsides in his work. In a 2009 work, he began by acknowledging we don’t have accurate measures for many factors. He built seven different scenarios for the return on investing in broadband infrastructure based on different assumptions. 

One of the seven scenarios actually found a economic decline with broadband because so much commerce and so many jobs left the territory. I think one of the positive scenarios is more likely but neither Raul nor I have seen enough data to be confident.

Leave a Reply

Your email address will not be published. Required fields are marked *

Scroll to top