Novel Opinion Filtering and Social Attractors

Opinion Filtering
In limited information space decisions are difficult without emotions for guidance, whether we call it instinct or values. We also find it easier to trust decisions made on an emotional basis because there is (we hope) a clear understanding of a sustained agenda, the echo of the future. We find it hard to trust machines to make any more than the most basic decisions because we do not share empathy with them. The act of seeking recommendations or advice from a person is shaped by our expectation of their values. Even though we may be able to follow publication bots providing automated content filtering, it is more difficult to extend this to opinion filtering, which is what we rely on people for.
 
Novelty Index
In this context, a novel article is by definition introducing a new opinion, different to those expected from the assumed bias. Perhaps it is therefore a huge assumption that novelty would be sought after within a document set. Perhaps people more commonly prefer to avoid “novel” opinions?

Blog Anxiety
Reluctance to blog stems from the perception that you have to be a professional to publish and the associated assumption that any publication will reach a significant audience. The blogosphere’s power law distribution of audiences bridges the gap between broadcast and clique. The intermediate region allows for greater interaction within groups of bloggers or Communities of Practice, and provides the greatest potential for group expansion and action. Weak ties provide more bridging capital between ideas.
 
Social Attractors
A power law also describes the size distribution of meme cascades, or the extent to the influence of particular ideas within the social network. As we become more connected we might expect to see an increasing number of monumental changes as cultural instability increases. Higher connectivity and lower transaction costs also make the economy more unstable and more susceptible to manipulation. As in any complex dynamic system, a large enough disturbance in group perception can in theory result in a paradigm-shifting cultural change, whether “good” or “bad”, into the next social attractor. Many of these are generally perceived as beneficial, like the events which lead to the fall of the Berlin wall, but the achievements of mob-rule are sometimes only desirable for the mob, and as history shows a mob may be misguided by a determined few.
 
News as Global Counseling
An undesirable global cultural state attractor might conceivably be compared to mental illness. Although a surprisingly large number of individuals suffer at some point during their lives, most recover and many are never affected, I would suggest in large part due to positive interactions with others. What does counseling do for people and what would be the equivalent in the global network? Perhaps we are able to move a brain from one attractor to another by exposing prevailing cyclic thought processes to alternative viewpoints. Perhaps encouraging people to be informed by a broad spectrum of the opinion-base is also a healthy thing to do, for a more stable society. Isn’t this what multicultural diversity is about after all? The liberal media would have us believe this is what they are for. So instead of a “read this similar article” you might receive a “read this similar article from a different point of view”. Although one can think of some potentially detrimental examples, giving everyone absolute control over their own blinkers might not be a good idea either. Proximity does not preclude segregation…

Advertisement

Web Science and the Novelty Engine

Web Science
There is a call from Sir Tim Berners-Lee and Wendy Hall for an academic discipline which studies the internet.

Although this may focus on the internet and its success stories by incorporating lessons learnt in computer and social sciences, I would hope it also draws heavily on work done in the past few decades in the natural sciences, given that pretty much every interesting system studied these days produces knowledge on complex networks. Web Science could be a real destination for generalists!

Social Semantic Web
Glad to see people thinking about the crossover between social networking and the semantic web. It will be interesting to see the spread of papers at the Stanford University symposium.

Novelty Engine
According to Nicholas Carr internet technology is bombarding us with so much information we are all rapidly losing our attention span. This not only forces us to speed-read everything but it prevents us from contemplating the deeper issues. One reason social network tools like Digg, Twitter etc help is because we trust others to read and filter on our behalf. However there is still so much to sift through for the knowledge nuggets.

What if we were also able to trust technology to do the reading and filtering? We are already seeing micro-blogging bots at the broadcast end. Social web browsers would also benefit from some intelligence, enabling messages to be organised and filtered on receipt. Again, perhaps the focus would be on identifying posts which are “related but novel”. Or at least routing them into personalised semantic buckets. Are there tools already out there, eg how much of this can you do with Flock? What would it take for us to build trust in personalised blog-reading bots?

Weak Ties and Emergent Contexts in Semantic Networks

Addressing information overload by taking a complexity approach to semantics.

Complexity Theory
The theory of complexity is based on the idea that complex behaviour emerges out of simple rules. Usually, we are talking about systems containing large numbers of interacting agents. In this sense ‘complex’ means that behaviour cannot be extrapolated from the rules of the system, only derived from models.

Living systems can be defined in terms of dissipative structures. These structures exist at the edge of chaos, or on the line between order and randomness. This means there is enough order for a distinct structure to be maintained over time, but enough randomness to allow for flexibility and evolution of that structure. This is usually achieved in living systems by gradually replacing the components of the system, by ingestion and excretion. For example, the human body is a continuous structure whose chemistry and components change over time. So is a human cell, and at a more basic level within the cell there are autocatalytic sets of enzymes and reactions which in the right circumstances are self-perpetuating.
 
Complex Networks
Complex networks can be found all over the natural and human world. Examples include the neocortex, the economy, ecosystems, control gene expression, metabolic pathways, contagion of disease or ideas, structure of the internet or languages etc. This is because they all are formed in a similar way following a few principles like growth by addition and preference for existing highly connected nodes. For example think of how the internet started, where people linked to a few popular sites, making them more popular, which made more people likely to link to them etc.

Small World Networks
This kind of growth often results in a scale-free network, whereby the distribution of node degree (how many links a node has) is described by a power law, or in other words follows a logarithmic graph. Another example of a power law is the ‘long tail’ curve utilised by iTunes, Amazon etc, who make a large proportion of their money from small volume sales of a large number of relatively unpopular items. For the same reason the size of an avalanche or earthquake is never predictable but fits statistically into a power law distribution.

Perhaps the best known example is the social network, which is also a small world network. The power law means that although most people only have a few contacts, there are a few social hubs with hundreds. Most people know about the ‘6 degrees’ idea in human society, where we are all connected to everyone else by around six steps in the network. A small world network achieves this by having hubs and weak ties, both of which emerge from the rules governing network growth.
 
Weak Ties
The point is that small world networks are very stable. If you take out a randomly chosen node then there are always plenty of other routes through the network. On the other hand, a strictly hierarchical network is vulnerable since removal of a single node will prevent information flow to all the nodes beneath it. This is also why some infectious diseases or computer viruses are never wiped out. There is a tipping point which occurs during formation of a network when the small world state is reached. After this, overall connectivity is not really improved by adding new connections. This state is achieved by weak ties, which are connections between nodes which don’t have overlapping neighbourhoods. For instance, I know people in Vancouver so friends in New Zealand are only two steps away from them in the social network.
 
Complex Network Dynamics
Why is this relevant to organisations? This is about how a network of humans responds to the outside world, which is a function of information flow and network growth. Or in other words, the dynamics IN the network and the dynamics OF the network. Open Space Technology is an example of how self-organisation can be encouraged within groups of people by flattening hierarchy, which helps create new weak ties between different levels of an organisation. This increases information flow through the network which strengthens it and enables it to be more adaptable.

Spider organisations, typically large hierarchical corporations, are slow, ordered and brittle. Starfish organisations, typically loose associations of individuals or smaller organisations, are more adaptable because their ability to move quickly allows them to change with the environment. Look at the impact of guerrilla tactics on warfare. Traditional armies have a hierarchical command structure, where information has to pass up the chain of command before decisions can be made centrally. They are slow to mobilise and suffer if key components are lost. The guerrilla approach is to have a distributed organisation of cells without a central command center, which can collaborate or act autonomously when required. They usually share general principles but are free to make decisions. Consider the account of Apache resistance in The Starfish and The Spider for an example of a successful leaderless organisation.
 
Semantic Profiling
Semantic profiling software reads text in order to represent context by extracting themes and the relationships between them. Context representations can be used for discovery, mapping, exploration etc. The topology of a semantic network is similar to a social network, so many of the same tools can be used. A semantic approach is complementary to manual tagging, partly due to the emphasis on implicit relationships rather than pre-defined categories, and partly because automation makes it consistent, which supports the pattern matching functions.

Whether a semantic or a manual approach is used, I believe that focusing on emergent relationships can help knowledge systems avoid the categorisation and domain evolution problems that top-down taxonomies suffer from. An example would be for expertise location or ad hoc team building in crisis/disaster management, or other applications where the emphasis is on immediate action.
 
Hierarchical Temporal Memory
It is currently maintained that the neocortex is a machine for matching patterns in time. Tracking the emergence of semantic relationships over time may be the only way we can realistically make sense of the huge amount of information most of us are bombarded with these days. Psychologists also theorise that a new concept may often come about by forming a relationship between two previously unrelated ideas.

I like to think that having a new idea is equivalent to forming a weak tie in the semantic network. The feeling of insight corresponds to linking previously unconnected ideas. As context then develops around the new relationship, nodes with no overlapping context gradually become more and more embedded. For instance, the words ‘mobile’ and ‘phone’ were once unrelated, but now have almost transitioned to a single word state due to their association. The key is to identify these new weak ties as they appear and start to develop surrounding context. For instance, ‘this article is related to those you have read but also contains some interesting new ideas’…