Quick overview: Sandy Pentland subscribes to the kind of ecological view that a lot of my recent sources have espoused- an emphasis on the relations between objects, instead of on the objects themselves. He argues for a “computational theory of behavior”, using Big Data and a system of collection/observation that he calls “reality mining”: the point is to grow the fullest, richest models of social behavior possible in order to capture all of the details, the micro-patterns, that traditional methods would overlook in favor of averages and stereotypes, and to capture them in “real life” as it is lived, instead of in a sanitized environment. Causation can be worked out through “quantitative, time-synchronized predictions” to make other explanations implausible, and then we might seek supporting quantitative lab experiments.

Pentland demonstrates how he has applied social network incentives (as opposed to traditional, individualistic market incentives) to construct and improve complex organizational behavior often with a relatively light nudge.

He also discusses, in the previous installment, particular methods and thoughts on designing and diagnosing prosperous data-driven cities, cities with balances of engagement and exploration that allows for improved idea flow, and innovation among its constituents.

Some more specific applications of this information can be found in the previous installment of notes.

This is part five of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

  • Part 0: Some immediate personal/social context, what I’m thinking about before reading.
  • Part 1: A broad summary of the book, the methods involved, etc.
  • Part 2: Heavier details on the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
  • Part 3: On Social Network-Incentivized Organizations
  • Part 4: On “Data-Driven Cities”
  • This part will be about “Data-Driven Society”, policy and design thought.


I. A Simple Draft of a New Deal on Data

I think that you, reader, can figure the benefits to sharing all of the personal data needed for deep, continuous “Reality Mining”, and the justifiable apprehension that the people generating the data might feel about it. Pentland offers what he calls “A New Deal on Data”.

 It has long been recognized that the first step to promoting liquidity in land and commodity markets is to guarantee ownership rights so that people can safely buy and sell. Similarly, the first step toward creating greater idea flow (“idea liquidity”) is to define ownership rights. The only politically viable course is to give individual citizens the rights over the data that are about them, and in fact, in the European Union these rights flow directly from the constitution. We need to recognize personal data as a valuable asset of the individual that is given to companies and government in return for services.

“Note that these ownership rights are not exactly the same as literal ownership under modern law”, but was meant to be a simpler framework for resolving disputes.

Pentland introduced this idea to the World Economic Forum in 2007. He claims credit for helping to shape the 2012 Consumer Data Bill of Rights in the USA, and the declaration on Personal Data Protection in the EU. “These new regulations are intended to accomplish the combined trick of breaking data out of the silos they are currently held in, thus enabling public goods, while at the same time giving individuals greater control over data about themselves.”

Enforcement is an issue. The current best practice, Pentland claims, is “a system of data sharing called trust networks.” The terms labeled to the data on what can(not) be done with it are matched to legal contracts between the parties. Permissions allow automatic auditing of data use. Pentland’s research group, in conjunction with the Institute for Data Driven Design, have helped to build “openPDS” (Personal Data Store), a consumer version of this system, and are “now testing it with a variety of industry and government partners.”

“Soon sharing personal data could become as safe and secure as transferring money between banks.”

Large internet entities are being pressured to conform to standards on data ownership and portability(?).

Challenge: The presence of safely shareable data will enable a new class of data-driven governance and policies. The biggest barrier here isn’t really data size/speed or privacy/accountability, but “learning how to build social institutions based on the analysis of billions of individual connections.” The scientific method is not very effective to a data-richsociety, because the sheer number of connections can lead to spurious results. We can’t form a limited, testable number of hypotheses. “Using live data to design institutions and policies is outside of our normal way of managing things.”

Again, returning to the importance of expanding on the “living lab” method I described in Part 1. Pentland recently opened the “open data city” in Trento, Italy. Participants know and consent to the massive experiment.

Challenge: Another major challenge is to human understanding of the complexities we will need to grapple with in order to produce good, understandable, consentable policy. “We need to build a human-scale, intuitive understanding of social physics. A new language and a new interface are necessary to make these ideas accessible and communicable.


II. Distinguishing Social Physics from concepts of Markets and Class

Challenge: Some people react negatively to the non-human focus of social physics (“humans as automatons”). “The social physics I envision, though, acknowledges our human capacity for independent thought but does not try to account for it.” It is population-level, it is about tendencies.You can also still be a person. Pentland also makes an interesting claim that “Adam Smith’s markets” and “Karl Marx’s classes” are both more dehumanizing, roughly as dehumanizing as each other, and that social physics allows much more breathing room in its assumptions about human behavior. He also argues that in our fervor to support an individualistic view of humanity, we fail to recognize that our commonality, and the patterned nature of our lives is a good thing, a benefit to all of us.

This will strike most of us as obvious but bears repeating.

Today’s market-based model for ideologies has its roots in the natural law notions of the eighteenth century (“the idea that humans are self-interested and self-commanded and that they relentlessly seek to gain from the exchange of goods, assistance, and favors in all social transactions”). Open competition is a serious engine of social life and progress in general, and that if costs are properly accounted for, such open competition would develop an efficient society. Pentland argues that it took some time for the Yang to this ideological Yin to catch back up: social norms are an important bonding force. Cooperation is not an exception. Coordination is just as deeply natural.

“Peer groups with shared norms are different from the traditional idea of class, because they are not defined only by standard features such as income, age, or gender (e.g. traditional demographics), their skills and education (per Max Weber), or their relationship to the means of production (per Karl Marx). Instead, group members are peers in the context of a particular situation.” This is also notable in that it rejects an “ideal form” or a typology- a given peer group is always defined by its constituents, and by their difference from other peer groups at whatever level of magnification we’re looking at.

“Just as classes are oversimplified stereotypes of a fluid and overlapping matrix of peer groups, the idea of a market is a similarly flawed idealization, in this case in which it is imagined that all the participants can see and compete evenly with everyone else.” In real life, the ground is never even and flat, and the information isn’t all public, the rules are not fair and not all the fighters are armed. Compare the average person to the professional stock trader, and him to the computerized HFT. The ideal market is far more symmetric than the exchange networks that are “real markets”.

…The Ford Motor Company went to the US Congress to argue for rescuing his fiercest competitor, General Motors. Why? Because Ford and GM depend on many of the same suppliers: If GM went broke, then the suppliers would go broke, and then Ford would be unable to manufacture its own cars. This sort of cooperation between competitors is definitely not something we would expect if we relied on only classic market thinking. [The] basic assumption withing classic market thinking- that there are many sellers and buyers that can be substituted for each other easily- does not apply to much of the US economy. Instead, we need to think of the economy as a complex network of specific exchange relationships.


III. Design Criteria

Pentland then spends time discussing studies of early human society, deeming that they were not necessarily organized the way that our classical state-of-nature arguments suggested they would be. And we have not changed that much, really- our number of trusted peers remains pretty static, for example.

“How are we to use these insights about human nature- the importance of social learning and social pressure, as well as the idea that human society is more of an exchange network than an open market- to design a society better suited to human nature?” Pentand identifies three design criteria for “our emerging hyperconnected societies”:

  1. Social Efficiency: Optimal distribution of resources throughout society- once thought to occur in Smithian markets via the invisible hand (which “doesn’t work unless everyone is engaged in the same social fabric so that peer pressure can ensure that everybody will follow the same set of rules.”). See also: trust networks- (again, one-to-one exchange with controls, inherent to exchange networks instead of “markets”). “The open market and strong personal control models are but two approaches to social efficiency.” Blends (such as a limited data commons, for instance for Health Care) are also possible.
  2. Operational Efficiency: “The infrastructure of our society should work quickly, reliably, and without waste if our society is to thrive in our modern, resource-limited world.” Our financial, transportation, energy, health, and political systems have an long way to go here, as they were designed three centuries ago based on “rigid, centralized control” models “because the primary sensing and data systems of the time were literally people riding around in horse-drawn buggies.” Public data commons (aggregated, anonymous data) that allowed for real-time “big picture” views would be a good step forward here. That would be enough to set broad policy, perhaps, with more local tweaking, tuning, and exploration based on private data through agreed-upon exchanges. Real-time monitoring and continuous exploration are the key to better systems.
  3. Resilience: Long-term stability of social systems. “Today’s social systems- finance, government, and work- seem to periodically seize up, fall apart, or crash and burn.” There must be smart, responsive, independent disaster management for core systems, perhaps using the Red Balloon method and some reliable communications technology. We need a diverse set of systems rather than one Best System, for familiar experimental (“Exit/Voice/Loyalty”) and evolutionary (“Anti-Fragile”) reasons. Independent but spreadable technique and technology sharing should be possible.

“We are beginning to see such design principles in military and first-responder systems.” There is a growing realization that not only can the central command be disrupted, it can be wrong, too far from the field to assess as well as people closer to the action. “Distributed leadership” is the buzzword.

The “Data for Development” (D4D) initiative is spawning over 90 projects using the above design criteria. Pentland rushes through several flagship projects focusing on each of the three design criteria in different fields, using aggregated anonymous data to improve various efficiencies on large scale projects and infrastructures. “Each of these D4D research projects has demonstrated the great potential of a big data commons for improving society.”

Pentland ends on an aspirational note, calling the methods and frameworks “Promethean Fire”. The appendices were interesting but unnecessary to summarize, I think.