Browsed by
Tag: data collection

No, “Big Data” Can’t Predict the Future – Article by Per Bylund

No, “Big Data” Can’t Predict the Future – Article by Per Bylund

Comments 0 Comment
The New Renaissance HatPer Bylund
******************************

With Google’s dominance in the online search engine market we entered the Age of Free. Indeed, services offered online are nowadays expected to be offered at no cost. Which, of course, does not mean that there is no cost to it, only that the consumer doesn’t pay it. Early attempts financed the services with ads, but we soon saw a move toward making the consumer the product. Today, free and unfree services alike compete for “users” and then make money off the data they collect.

Data has always been used, but what’s new for our time is the very low (or even zero) marginal cost for collecting and analyzing huge amounts of data. The concept of “Big Data” is taking over and is predicted to be “the future” of business.

There’s a problem here, and it is the over-reliance on the Law of Large Numbers in social forecasting. Statistical probabilities for events may mathematically converge to the mean, but is it applicable in the real world? The answer is most definitely yes in the natural sciences. Repeated controlled experiments will weed out erroneous explanations or causes to phenomena, at least assuming we’re good enough at separating and controlling those causes.

What about the social sciences? In this age of scientism, as Hayek called it, we’re told “Big Data” will completely transform production, logistics, and sales. The reason for this is that vendors can better target customers and even foresee what they might want next. Amazon.com does this on their web site in crude form, where they make suggestions based on your purchase history and what others with similar purchase histories have searched for. Sometimes it works, and sometimes it doesn’t.

There is some regularity to our interests and behavior. All of us are, after all, human beings — and we’re formed in certain cultures. So one American with interests x, y, and z may have other interests similar to another American who also has an interest in x, y, and z.

Human Behavior Is Unpredictable
But similarity is not the same thing as prediction. Amazon.com’s suggestions or the highly annoying ads following you around web sites are useful methods for sellers because they can somewhat accurately identify what not to offer. Exclusion of very low-probability interests increases the probability for suggesting something that the person behind the eyeballs focusing on the computer screen may be interested in.

To use as prediction, however, exclusion of almost-zero probability events is far from sufficient. Indeed, prediction requires that we are able to accurately exclude all but one or a couple highly probable outcomes. And we have to be able to rely on that these predictions turn out to be true. Otherwise we’re just playing games, and so we’re making guesses. Sure, they’re educated guesses (because we’ve excluded the impossible and almost-impossible), but they’re still games and guesses.

Where Big Data Fails
Speaking of guesses, Microsoft’s Bing search engine, which powers the Windows digital assistant Cortana among other things, has produced a prediction engine with the purpose of predicting sports and other results. They rely on very advanced algorithms and huge amounts of collected data.

Amazingly, they did very well initially and predicted the outcomes of the World Cup perfectly. So maybe we can use Big Data to get a glimpse of the future?

No, not so. The Bing teams are learning a lesson only Austrians and, more specifically, Misesian praxeologists, seem to be alone in grasping: that there are no constants in human action, and therefore that predictions of social phenomena are impossible. Pattern predictions, as Hayek called them, may not be impossible, but predictions of exact magnitudes are. For instance, we can rely on economic law (such as “demand curves slope downward”) to estimate an outcome such as “the price will be lower than it otherwise would have been,” but we can’t say exactly what that price will be.

When it comes to sports, reality shows and other competitions between individuals or teams, the story is exactly the same. The team with a better track record doesn’t always win. Why? They have objectively performed better than the other team, perhaps exclusively so, but this doesn’t say anything about the future. We’re not here referring to the philosophical doubt as in “will the sun shine tomorrow?” (maybe something changes completely the sun’s ability to shine during the night).

The Social Sciences Are Different
In the social sciences we’re dealing with complex phenomena. Action and, especially, its outcome is the result of a complex system of social interaction, psychology, and much more. Are the players in both teams as motivated and focused as they were before? Did anything in their personal lives affect their mindsets or psyches? How do the players within their teams and players in other teams react on each other before and during the game? A team with a poor track record can upset a team with an objectively better track record; this happens all the time. Sometimes for the sole reason that the better team underestimates the worse team, or because the underdog feels no pressure to perform and therefore plays less defensively.

Bing’s prediction engine struggles with this, just as we would predict. As Windows Central reported recently, the prediction engine had its “worst week yet” picking only four of fourteen winners in the NFL. Overall, its track record was approximately two-thirds right and one-third wrong (95–53). It’s definitely better than tossing a coin, but pretty far from actually predicting the results.

In other words, if you’re placing bets you may want to use the Bing prediction engine. That is, unless you have the type of tacit, implicit understanding of what’s going on that the engine is missing. Maybe you can beat it, or maybe not. In either case, you cannot count on coming out a victor each and every time.

The reason for this is that the outcome simply cannot be predicted perfectly — or even close to it. Even the players themselves cannot predict who’ll win a game, but they may have inside information about whether their own team seems motivated and focused. It is not a perfect method, however, and it certainly cannot be scientific.

Even with Big Data there’s no predicting of social events — there’s only guessing. Yes, guessing with access to huge amounts of data is easier, at least if the data is reliable and relevant. But a good guess is not the same thing as a prediction; it is still a guess, and it can be wrong. Winning every time requires luck.

Per L. Bylund, PhD, is Assistant Professor of Entrepreneurship and Records-Johnston Professor of Free Enterprise in the School of Entrepreneurship at Oklahoma State University. Visit his website at PerBylund.com.

This article was originally published by the Ludwig von Mises Institute. Permission to reprint in whole or in part is hereby granted, provided full credit is given.

Why The 2,776 NSA Violations Are No Big Deal – Article by Ron Paul

Why The 2,776 NSA Violations Are No Big Deal – Article by Ron Paul

Comments 0 Comment
The New Renaissance Hat
Ron Paul
August 18, 2013
******************************
Thanks to more documents leaked by Edward Snowden, this time to the Washington Post, we learned last week that a secret May 2012 internal audit by the NSA revealed 2,776 incidents of “unauthorized” collection of information on American citizens over the previous 12 months. They are routinely breaking their own rules and covering it up.The Post article quotes an NSA spokesman assuring the paper that the NSA attempts to identify such problems “at the earliest possible moment.” But what happened to all those communications intercepted improperly in the meantime? The answer is, they were logged and stored anyway.

We also learned that the NSA routinely intercepts information from Americans while actually targeting foreigners, and that this is not even considered a violation. These intercepts are not deleted once discovered, even though they violate the US government’s own standards. As the article reports, “once added to its databases, absent other restrictions, the communications of Americans may be searched freely.”

The Post article quotes an NSA official explaining that the thousands of unauthorized communications intercepts yearly are relatively insignificant. “You can look at it as a percentage of our total activity that occurs each day. You look at a number in absolute terms that looks big, and when you look at it in relative terms, it looks a little different.”

So although the numbers of Americans who have had their information intercepted in violation of NSA’s own rules seems large, it is actually miniscule compared to the huge volume of our communications they intercept in total!

Though it made for a sensational headline last week, the fact is these 2,776 “violations” over the course of one year are completely irrelevant. The millions and millions of “authorized” intercepts of our communications are all illegal — except for the very few carried out in pursuit of a validly-issued search warrant in accordance with the Fourth Amendment. That is the real story. Drawing our attention to the violations unfortunately sends the message that the “authorized” spying on us is nothing to be concerned about.

When information about the massive NSA domestic spying program began leaking earlier in the summer, Deputy Attorney General James Cole assured us of the many levels of safeguards to prevent the unauthorized collection, storage, and distribution of our communications. He promised to explain the NSA’s record “in as transparent a way as we possibly can.”

Yet two months later we only discover from more leaked documents the thousands of times communications were intercepted in violation of their own standards! It is hardly reassuring, therefore, when they promise us they will be more forthcoming in the future. No one believes them because they have lied and covered up continuously. The only time any light at all is shone on these criminal acts by the US federal government is when a whistleblower comes forth with new and ever more disturbing information.

Americans are increasingly concerned over these violations of their privacy. Calls for reform grow. However, whenever Washington finds itself in a scandal, the federal government responds by naming a federal-government panel made up of current and former federal employees to investigate any mistakes the federal government might have made. The recommendations invariably are that even more federal government employees must be hired to provide an additional layer or two of oversight. That is supposed to reassure us that reforms have been made, while in fact it is just insiders covering up for those who have hired them to investigate.

Let us hope the American people will decide that such trickery is no longer acceptable. It is time to take a very serious look at the activities of the US intelligence community. The first step would be a dramatic reduction in appropriations to force a focus on those real, not imagined, threats to our national security. We should not be considered the enemy.

Ron Paul, MD, is a former three-time Republican candidate for U. S. President and Congressman from Texas.

This article is reprinted with permission.

The Eyes Watching You: “1984” and the Surveillance State – Article by Sarah Skwire

The Eyes Watching You: “1984” and the Surveillance State – Article by Sarah Skwire

Comments 0 Comment
The New Renaissance Hat
Sarah Skwire
June 19, 2013
******************************

George Orwell. 1984. New York: Plume, [1949] 2003. 323 pages.

In the kind of horrifying coincidence that surely would have prompted one of his more acerbic essays, the news that various U.S. government surveillance agencies have been gathering data from millions of citizens’ phones, email accounts, and web searches broke during the week of the 64th publication anniversary of George Orwell’s 1984. As the news reports poured in, and as sales of 1984 surged by an astonishing 6,884 percent, a friend asked me whether the PRISM story strikes me as more Orwellian or more Kafkaesque.

My response? We’d better hope it’s Kafkaesque.

No one wants to inhabit a Franz Kafka novel. But the surveillance states he describes do have one thing going for them—incompetence. In Kafka’s stories, important forms get lost, permits are unattainable, and bureaucrats fail to do their jobs. Like the main character in Kafka’s unfinished story, “The Castle,” if you were trapped in Kafka’s world you could live your whole life doing nothing but waiting for a permit. But at least you could live. Incompetence creates a little space.

What is terrifying about Orwell’s 1984 is the complete competence of the surveillance state. Winston Smith begins the novel by believing he is in an awful, but Kafkaesque world where there is still some slippage in the state’s absolute control, and still some room for private action. Winston says that Oceania’s world of telescreens and Thought Police means that there are “always the eyes watching you and the voice enveloping you. Asleep or awake, working or eating, indoors or out of doors, in the bath or in bed—no escape.” But he follows that by saying, “Nothing was your own except the few cubic centimeters inside your skull.” He also believes that while the diary he keeps will inevitably be discovered, the small alcove in his apartment where he writes his diary puts him “out of the range of the telescreen.”

The feeling that some tiny space for private thought and action can be found leads Winston into his relationship with Julia. Though they know they will inevitably be discovered, Winston and Julia believe that, for a time, their relationship and their meeting place will remain secret. They could not be more wrong.

One day after making love to Julia in their clandestine room, Winston, prompted by a singing thrush and a singing prole woman who is doing laundry, has a vision of a future that “belongs to the proles.”

The birds sang, the proles sang. The Party did not sing. All round the world, in London and New York, in Africa and Brazil, and in the mysterious, forbidden lands beyond the frontiers, in the streets of Paris and Berlin, in the villages of the endless Russian plain, in the bazaars of China and Japan—everywhere stood the same solid unconquerable figure, made monstrous by work and childbearing, toiling from birth to death and still singing. Out of those mighty loins a race of conscious beings must one day come. You were the dead; theirs was the future. But you could share in that future if you kept alive the mind as they kept alive the body.

 

In this very moment, just as Winston comes alive to what feels like hope and possibility and the dream of some kind of a future for humankind, the telescreen that has been hidden in the room all along speaks to Winston and Julia. The Thought Police break down the door. The couple is taken off to be imprisoned, tortured, and broken.

There has never been any private space for Winston or Julia—not in their “secret” meeting places, not in their sexual rebellion, not even in the few cubic centimeters inside their skulls. “For seven years the Thought Police had watched him like a beetle under a magnifying glass. There was no physical act, no word spoken aloud, that they had not noticed, no train of thought that they had not been able to infer.” Winston should have taken more seriously the description of Oceania he read in the forbidden book The Theory and Practice of Oligarchical Collectivism, by Emmanuel Goldstein:

A Party member lives from birth to death under the eye of the Thought Police. Even when he is alone he can never be sure that he is alone. Wherever he may be, asleep or awake, working or resting, in his bath or in bed, he can be inspected without warning and without knowing that he is being inspected. Nothing that he does is indifferent. His friendships, his relaxations, his behaviour towards his wife and children, the expression of his face when he is alone, the words he mutters in sleep, even the characteristic movements of his body, are all jealously scrutinized. Not only any actual misdemeanour, but any eccentricity, however small, any change of habits, any nervous mannerism that could possibly be the symptom of an inner struggle, is certain to be detected.
***

The Orwellian surveillance state is terrifying not because—as in Kafka—you might be arrested because of a rumor or a mistake, or because despite your innocence you might be caught in the surveillance state’s unnavigable maze. It is terrifying because it never makes mistakes. It does not need to listen to rumors. And it knows that no one is ever innocent.

Sarah Skwire is a fellow at Liberty Fund, Inc. She is a poet and author of the writing textbook Writing with a Thesis.

This article was originally published by The Foundation for Economic Education.