Hidden Danger of Big Data

[SearchingforPeace]

The Hidden Danger of Big Data by Carlo Ratti and Dirk Helbing
- Project Syndicate

I have a lot of concerns with the Big Data model. It really only can answer questions that are presented. Big Data seems to poor substitute for intuition and judgment.

In game theory, the “price of anarchy” describes how individuals acting in their own self-interest within a larger system tend to reduce that larger system’s efficiency. It is a ubiquitous phenomenon, one that almost all of us confront, in some form, on a regular basis.

For example, if you are a city planner in charge of traffic management, there are two ways you can address traffic flows in your city. Generally, a centralized, top-down approach – one that comprehends the entire system, identifies choke points, and makes changes to eliminate them – will be more efficient than simply letting individual drivers make their own choices on the road, with the assumption that these choices, in aggregate, will lead to an acceptable outcome. The first approach reduces the cost of anarchy and makes better use of all available information.

.....

But here we should consider what is lost when we reduce the level of anarchy. The most meaningful book you should read after those previous ten is not one that fits neatly into an established pattern, but rather one that surprises or challenges you to look at the world in a different way.

Contrary to the traffic-flow scenario described above, optimized suggestions – which often amount to a self-fulfilling prophecy of your next purchase – might not be the best paradigm for online book browsing. Big data can multiply our options while filtering out things we don’t want to see, but there is something to be said for discovering that 11th book through pure serendipity.

....

But with centralized algorithms coming to manage every facet of society, data-driven technocracy is threatening to overwhelm innovation and democracy. This outcome should be avoided at all costs. Decentralized decision-making is crucial for the enrichment of society. Data-driven optimization, conversely, derives solutions from a predetermined paradigm, which, in its current form, often excludes the transformational or counterintuitive ideas that propel humanity forward.

....

A certain amount of randomness in our lives allows for new ideas or modes of thinking that would otherwise be missed. And, on a macro scale, it is necessary for life itself. If nature had used predictive algorithms that prevented random mutation in the replication of DNA, our planet would probably still be at the stage of a very optimized single-cell organism.

....

Decentralized decision-making can create synergies between human and machine intelligence through processes of natural and artificial co-evolution. Distributed intelligence might sometimes reduce efficiency in the short term, but it will ultimately lead to a more creative, diverse, and resilient society. The price of anarchy is a price well worth paying if we want to preserve innovation through serendipity.

Big Data, as discussed in this article, seems to be the ultimate SiTe endeavor. It can find maximium efficiency, but only as long as it holds to past endeavors. It misses out on change and evolution.....

 

[Cellmold]

 

[Mvika]

It comes down to the famous Hummel Vs Goodsell debate on systemization, centralizing control and hierarchical structuring of social institutions, and human lives themselves. Hummelian Bureaucratic experience is a cry in the dark for the deeply personal, the spiritual, the individualistic soul that is the biggest casualty of the giant bulldozing machinery of standardized, impersonal, bureaucratic organizing force. Goodsell, on the other hand, warns that on the other side lies anarchy and chaos. The argument is that those of us who complain about being brought under the giant umbrella are actually fighting modernization itself. That if we eschew being on call all the time with a cell phone and state expects you to carry one all the time, then we are just luddites.

Recently, I was told that only way I could buy a particular item was if I liked them on their facebook page! When I said I didn't have a Facebook account, I was asked to open one on the spot using my iPhone. When I said that I don't own one, I was told that there was no other way to buy the product, despite having the cash to pay for it. I cannot meet my doctor or take a book out of the library without making my birthdate known to the disinterested clerk. I am waiting sadly for my Fi to break and surrender to my Te one of these days and join the microchipped citizenry of the future.

 

[Fluffywolf]

Big Data is an extremely useful and powerful phenomenon regarding very specific questions.

Worrying about 'missing' out on something because big data tunnelvision seems a bit contrived to me. As the name suggests, there is too much data to consume, so all it does it attempt to show the relevant data. Of course it might miss something, but it's not like you can just try and consume it all. Life is too short to try and consume it all.

Here's where the money lies though. And that is in writing smart interpreter software that can sift through it all quick outputs relatively small and reliable usable data. If AI will ever have a place in society, it is for big data interpretation.

The flaw you see in big data, isn't big data itself but the interpreter software developers not doing a good job of it.

 

[Mole]

It comes down to the famous Hummel Vs Goodsell debate on systemization, centralizing control and hierarchical structuring of social institutions, and human lives themselves. Hummelian Bureaucratic experience is a cry in the dark for the deeply personal, the spiritual, the individualistic soul that is the biggest casualty of the giant bulldozing machinery of standardized, impersonal, bureaucratic organizing force. Goodsell, on the other hand, warns that on the other side lies anarchy and chaos. The argument is that those of us who complain about being brought under the giant umbrella are actually fighting modernization itself. That if we eschew being on call all the time with a cell phone and state expects you to carry one all the time, then we are just luddites.

Recently, I was told that only way I could buy a particular item was if I liked them on their facebook page! When I said I didn't have a Facebook account, I was asked to open one on the spot using my iPhone. When I said that I don't own one, I was told that there was no other way to buy the product, despite having the cash to pay for it. I cannot meet my doctor or take a book out of the library without making my birthdate known to the disinterested clerk. I am waiting sadly for my Fi to break and surrender to my Te one of these days and join the microchipped citizenry of the future.

Even more important than using modern electronic technology is understanding it. A good place to start is Understanding Media - the Extensions of Man by Marshall McLuhan, click on http://robynbacken.com/text/nw_research.pdf

 

[Coriolis]

Recently, I was told that only way I could buy a particular item was if I liked them on their facebook page! When I said I didn't have a Facebook account, I was asked to open one on the spot using my iPhone. When I said that I don't own one, I was told that there was no other way to buy the product, despite having the cash to pay for it. I cannot meet my doctor or take a book out of the library without making my birthdate known to the disinterested clerk. I am waiting sadly for my Fi to break and surrender to my Te one of these days and join the microchipped citizenry of the future.

You have this backwards. Use Te to support what Fi knows to be right: these requirements are stupid and needless. You can lie about your birthdate in many places, though I would not do this with medical providers. They need to know your age. Also, never give your social security number to anyone for purposes other than taxes, which includes government entities, employers, and financial institutions. The rest don't need to know it. Really.

Big Data is an extremely useful and powerful phenomenon regarding very specific questions.

Worrying about 'missing' out on something because big data tunnelvision seems a bit contrived to me. As the name suggests, there is too much data to consume, so all it does it attempt to show the relevant data. Of course it might miss something, but it's not like you can just try and consume it all. Life is too short to try and consume it all.

Here's where the money lies though. And that is in writing smart interpreter software that can sift through it all quick outputs relatively small and reliable usable data. If AI will ever have a place in society, it is for big data interpretation.

The flaw you see in big data, isn't big data itself but the interpreter software developers not doing a good job of it.

As with any inquiry, big data can answer only the question posed to it. So, the first problem is asking the right question. The second problem is avoiding the tunnel vision that can come from fixating on that question, or simply accepting the given answer uncritically.

 

[Bush]

I was involved with a particular project for a particular agency, which recognized this and sought to bring humans into the loop. The purpose was essentially to marry human intuition with analytical tools. I was elated that someone recognized the value of both.

If you want randomness in your Big Data processing, too, you can look toward methods like genetic algorithms. Every once in a while, some "child" will have an oddball "mutation" that serves as a total outlier -- that 11th book. Any data handling method worth its salt is going to recognize that, for many purposes, that's a thing to incorporate to some degree.

Oddball things can very well lead to.. well, unexpected exploration. That "What if?" question. Unknown unknowns.