Skip to content →

Month: May 2014

Social Physics V: Data-Driven Society

Quick overview: Sandy Pentland subscribes to the kind of ecological view that a lot of my recent sources have espoused- an emphasis on the relations between objects, instead of on the objects themselves. He argues for a “computational theory of behavior”, using Big Data and a system of collection/observation that he calls “reality mining”: the point is to grow the fullest, richest models of social behavior possible in order to capture all of the details, the micro-patterns, that traditional methods would overlook in favor of averages and stereotypes, and to capture them in “real life” as it is lived, instead of in a sanitized environment. Causation can be worked out through “quantitative, time-synchronized predictions” to make other explanations implausible, and then we might seek supporting quantitative lab experiments.

Pentland demonstrates how he has applied social network incentives (as opposed to traditional, individualistic market incentives) to construct and improve complex organizational behavior often with a relatively light nudge.

He also discusses, in the previous installment, particular methods and thoughts on designing and diagnosing prosperous data-driven cities, cities with balances of engagement and exploration that allows for improved idea flow, and innovation among its constituents.

Some more specific applications of this information can be found in the previous installment of notes.

This is part five of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

  • Part 0: Some immediate personal/social context, what I’m thinking about before reading.
  • Part 1: A broad summary of the book, the methods involved, etc.
  • Part 2: Heavier details on the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
  • Part 3: On Social Network-Incentivized Organizations
  • Part 4: On “Data-Driven Cities”
  • This part will be about “Data-Driven Society”, policy and design thought.

 

I. A Simple Draft of a New Deal on Data

I think that you, reader, can figure the benefits to sharing all of the personal data needed for deep, continuous “Reality Mining”, and the justifiable apprehension that the people generating the data might feel about it. Pentland offers what he calls “A New Deal on Data”.

 It has long been recognized that the first step to promoting liquidity in land and commodity markets is to guarantee ownership rights so that people can safely buy and sell. Similarly, the first step toward creating greater idea flow (“idea liquidity”) is to define ownership rights. The only politically viable course is to give individual citizens the rights over the data that are about them, and in fact, in the European Union these rights flow directly from the constitution. We need to recognize personal data as a valuable asset of the individual that is given to companies and government in return for services.

  • “You have the right to possess data about you.” (Data collectors “play a role akin to a bank”, managing data that you can access at any time).
  • “You have the right to full control over the use of your data.” (Terms of use are opt-in, and clearly explained. You reserve the right to remove your data from a company’s use.)
  • “You have the right to dispose of or distribute your data.”

“Note that these ownership rights are not exactly the same as literal ownership under modern law”, but was meant to be a simpler framework for resolving disputes.

Pentland introduced this idea to the World Economic Forum in 2007. He claims credit for helping to shape the 2012 Consumer Data Bill of Rights in the USA, and the declaration on Personal Data Protection in the EU. “These new regulations are intended to accomplish the combined trick of breaking data out of the silos they are currently held in, thus enabling public goods, while at the same time giving individuals greater control over data about themselves.”

Enforcement is an issue. The current best practice, Pentland claims, is “a system of data sharing called trust networks.” The terms labeled to the data on what can(not) be done with it are matched to legal contracts between the parties. Permissions allow automatic auditing of data use. Pentland’s research group, in conjunction with the Institute for Data Driven Design, have helped to build “openPDS” (Personal Data Store), a consumer version of this system, and are “now testing it with a variety of industry and government partners.”

“Soon sharing personal data could become as safe and secure as transferring money between banks.”

Large internet entities are being pressured to conform to standards on data ownership and portability(?).

Challenge: The presence of safely shareable data will enable a new class of data-driven governance and policies. The biggest barrier here isn’t really data size/speed or privacy/accountability, but “learning how to build social institutions based on the analysis of billions of individual connections.” The scientific method is not very effective to a data-richsociety, because the sheer number of connections can lead to spurious results. We can’t form a limited, testable number of hypotheses. “Using live data to design institutions and policies is outside of our normal way of managing things.”

Again, returning to the importance of expanding on the “living lab” method I described in Part 1. Pentland recently opened the “open data city” in Trento, Italy. Participants know and consent to the massive experiment.

Challenge: Another major challenge is to human understanding of the complexities we will need to grapple with in order to produce good, understandable, consentable policy. “We need to build a human-scale, intuitive understanding of social physics. A new language and a new interface are necessary to make these ideas accessible and communicable.

Continue reading Social Physics V: Data-Driven Society

Leave a Comment

Social Physics IV: Data-Driven Cities

Quick overview: Sandy Pentland subscribes to the kind of ecological view that a lot of my recent sources have espoused- an emphasis on the relations between objects, instead of on the objects themselves. He argues for a “computational theory of behavior”, using Big Data and a system of collection/observation that he calls “reality mining”: the point is to grow the fullest, richest models of social behavior possible in order to capture all of the details, the micro-patterns, that traditional methods would overlook in favor of averages and stereotypes, and to capture them in “real life” as it is lived, instead of in a sanitized environment. Causation can be worked out through “quantitative, time-synchronized predictions” to make other explanations implausible, and then we might seek supporting quantitative lab experiments.

He has also demonstrated, as my previous installment notes, how he has applied social network incentives (as opposed to traditional, individualistic market incentives) to construct and improve complex organizational behavior often with a relatively light nudge.

Some more specific applications of this information can be found in the previous installment of notes.

This is part four (of presumably five?) of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

  • Part 0: Some immediate personal/social context, what I’m thinking about before reading.
  • Part 1: A broad summary of the book, the methods involved, etc.
  • Part 2: Heavier details on the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
  • Part 3: On Social Network-Incentivized Organizations
  • This part will be about “Data-Driven Cities”
  • Part 5 will be about Policy

I.

 …We need to radically rethink our approaches. Rather than static systems that are separated by function- water, food, waste, transport, education, energy, and so on- we must consider them as dynamic and holistic systems. We need networked, self-regulating systems that are driven by the needs and preferences of citizens instead of ones focused only on access and distribution.

In other words, we need mobile sensing to create a “nervous system for cities, enabling them to become more healthy, safe, and efficient.”

This is obvious and I won’t dwell on it.

“Many of the sensing and control elements required to build a digital nervous system are already in place. What is missing, though, are two critical items: the first is social physics, specifically dynamic models of demand and reaction that will make the system function correctly [this chapter], and the second is a New Deal on Data, an architecture and legal policy that guarantees privacy,stability, and efficient government [next chapter].”

Already, this is a very different approach than the centralizing, “market-oriented” [not an oxymoron, folks] philosophy that Against the Smart City railed against.

Continue reading Social Physics IV: Data-Driven Cities

Leave a Comment

Social Physics III: Organizations

Quick overview: Sandy Pentland subscribes to the kind of ecological view that a lot of my recent sources have espoused- an emphasis on the relations between objects, instead of on the objects themselves. He argues for a “computational theory of behavior”, using Big Data and a system of collection/observation that he calls “reality mining”: the point is to grow the fullest, richest models of social behavior possible in order to capture all of the details, the micro-patterns, that traditional methods would overlook in favor of averages and stereotypes, and to capture them in “real life” as it is lived, instead of in a sanitized environment. Causation can be worked out through “quantitative, time-synchronized predictions” to make other explanations implausible, and then we might seek supporting quantitative lab experiments.

The two most important concepts in social physics are Idea Flow and Social Learning.

  • Idea Flow exists within social networks, and can be “separated into exploration (finding new ideas/strategies) and engagement (getting everyone to coordinate their behavior)”.
  • Social learning is “how new ideas become habits”. Learning can be “accelerated and shaped by social pressure.”

Some more specific applications of this information can be found in the previous installment of notes.

This is part two (of presumably five?) of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

  • Part 0: Some immediate personal/social context, what I’m thinking about before reading.
  • Part 1: A broad summary of the book, the methods involved, etc.
  • Part 2: Heavier details on the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
  • This part will be about Organizations (a favorite topic on this blog!)
  • Part 4 will be about “Data-Driven Cities” (a recent favorite topic on this blog!)
  • Part 5 will be about Policy (mostly avoided on this blog…)

This time around, I will probably be a little sparser with references to the studies that backup the results.

 

I.

Organizations are sustained systems of information flow. Group intelligence, which we will measure by idea flow, is “as important a factor in performing group performance as IQ is in predicting individual performance.”

“The largest factor in predicting group intelligence was the equality of conversational turn taking.” The second largest factor was the social intelligence of the group’s members.

Briefly, on method: “The badges used in this experiment, and in other research studies from my lab, produce detailed, quantitative measures of how people interact. Typical variables measured include: the tone of voice used; whether people face one another while talking; how much they gesture; and how much they talk, listen, and interrupt each other. By combining data from individuals within a team and comparing it with performance data, we can identify interaction patterns that make for successful teamwork.”

What this sociometric data showed was that the pattern of idea flow by itself was more important to group performance than all other factors and, in fact, was as important as all other factors put together [eg. individual intelligence, personality, skill, etc.]

Pentland (with colleague Wen) identifies three simple patterns that account for “approximately 50 percent of the variation in performance access groups and tasks.” Highest performing groups include:

  1. Large number of ideas (many short contributions instead of a few long ones)
  2. Dense interactions: a continuous, overlapping cycling between making contributions and very short (<1 second) responsive comments (“good”, “aight”, “what?”) to validate and invalidate ideas and build consensus.
  3. Diversity of ideas: everyone contributing ideas and reactions, with similar levels of turn taking among the participants.

 

Two exceptions:

  1. The decision needs to be made now. Discussion may not be available. We need a tyrant.
  2. High-running emotions, socially broken team. We need a facilitator.

The interventions to resolve these exceptions should take as little time as possible.

Pentland speculates an evolutionary basis for why our ideal social dynamic is the way that it is. Essentially, language is a new layer of communication in species-time and older methods of signaling were visual, quick consensus-building activities to get basic things coordinated. No need to dig too deep into that bit.

(Social) engagement is the central predictor of productivity. Study example: Pentland et al. improved call center metrics by allowing whole teams to go on break at the same time, instead of scheduling breaks on an individual basis. The result was more inter-team informal communication, and higher productivity during work.

 

Continue reading Social Physics III: Organizations

Leave a Comment

Social Physics II

This is part two (of presumably five?) of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

  • Part 0: Some immediate personal/social context, what I’m thinking about before reading.
  • Part 1 was on the basic premise of the book
  • This part is about the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
  • Part 3 will be about Organizations (a favorite topic on this blog!)
  • Part 4 will be about “Data-Driven Cities” (a recent favorite topic on this blog!)
  • Part 5 will be about Policy (mostly avoided on this blog…)

Each part may not follow on the heels of the previous. I reserve the right to get distracted by some other thing.

Finally, all of the math is in the back of the book. Smart choice, Sandy. (Although this sometimes obfuscates how new some of these conclusions are- they smell like social science consensus, making the book feel like a retread when the real innovation is about the method, the computation, that brought about the reliable result).

Finding Good Ideas

Pentland agrees that the “singular genius” construct is not useful for creating more good ideas/decisions. He instead suggests that seekers of different (good) ideas/views, who benefit from many disparate streams of thought, are more likely to find serendipitous connections. They can then socialize that new idea, bouncing off of people from various backgrounds, before publicly proclaiming that new idea. Science, art, and leadership are all about “developing a compelling story about the world and then deciding to test it against reality.”

So now we ought to make this process a little more concrete. This exploration of ideas through people “is fundamentally a search of one’s social network”, so “a good place to start […] is by investigating the role of social interactions in how we find new ideas and use them to make decisions.”

The most productive people are constantly developing and testing a new story, adding newly discovered ideas to the story and then trying it out on everyone they meet. Like sculpting raw clay into a beautiful statue, over time their story becomes more and more compelling. Finally they decide it is time to act on it, to bring it into the light and test it against reality. To these people, the practice of harvesting, winnowing, and sculpting with ideas feels like play. In fact, some of them call it “serious play” [note: link mine]

It is known that the “wisdom of crowds” is most effective when the humans involved are giving their own judgment without social influence (that is, all of the judgments are independent). On the other hand, social influence can allow for the apparently-best ideas to spread through a population in the right doses. Periods of “idea harvesting” and “expert evaluation” are needed for the smartest groups- there must be idea diversity and pragmatic tests in order to develop the best ideas.

Continue reading Social Physics II

Leave a Comment

Social Physics I

Pulling heavily from the introduction to Social Physics.

Frankly, skimming below for emphasized words and block quotes will get you the gist, but I like to shore up with quotes directly from the source. And sometimes, I like to bloviate.

I.

If I were preparing to create a PBS documentary of this book, I’d probably start with a campy scene with a telescope or microscope, and I’d talk about the interplay between the development of technologies that extend the senses, and new concepts and sciences, notations and theories, that the new observations support. The technologies that enable “Big Data”, I might argue, are the new telescopes.

“Our ways of understanding and managing the world were forged in a statelier, less connected time. Our current conception of society was born in the late 1700s during the Enlightenment and crystallized into its current form during the first half of the twentieth century. […] When we think of how to manage our society, we speak of “markets” and “political classes,” abstractions that events move slowly, so everyone has pretty much the same information and so people have time to act rationally.”

These assumptions are outdated. Groups can form and dissolve much faster thanks to technology and globalization. We need to develop more useful concepts for the new world- “we can no longer think of ourselves as only individuals reaching carefully considered decisions; we must include the dynamic social effects that influence our individual decisions and drive economic bubbles, political revolutions, and the internet economy.”

The goal of this book is to develop a social physics that extends economic and political thinking by including not only competitive forces but also exchanges of ideas, information, social pressure, and social status in order to more fully explain human behavior. To accomplish this we will have to explain not only how social interactions affect individual goals and decisions but, more importantly, how these social effects produce Adam Smith’s otherwise mysterious invisible hand. Only once we understand how social interactions work together with competitive forces can we hope to ensure stability and fairness in our hyperconnected, networked society.

Continue reading Social Physics I

Leave a Comment