I plan to live forever. So far, so good.

I recently found a very interesting presentation held by Gary William Flake, on which I’d like to comment a bit. Note that this will be a content-heavy (but hopefully very interesting) lecture. I’ll try and give links to all the concepts for a better understanding of the text. Without further ado, let’s begin: first, the back story.

We’ll start by defining the term “technological singularity“: the idea behind the concept of singularity is that the exponential speed of technological improvements will produce super-human capabilities, making the future completely unknowable. It was introduced by Vernor Vinge, in 1993, and evangelized by Ray Kurzweil in multiple books. The name purposely suggests a link to the black hole event horizon, also known as the “point-of-no-return” (a boundary defined with respect to an observer, beyond which events cannot affect the observer).

The primary causes for this singularity effect appearing are cited as the realization of true AI and nanotechnology, yielding asymptotic upper limits of intelligence, power, and ability to manipulate reality. The corollary for this human society as we know it now will end because of this effect. So singularity is talked about in both optimistic and pessimistic ways.

Finally, here is a small diagram showing the concept and the paths leading to the singularity effect:

Gary Flake then comes and introduces a new concept, called the “Internet singularity“, which is loosely defined as the idea that a deeper and tighter coupling between the online and offline worlds will accelerate science, business, society and self-actualization. Its primary cause is claimed to be ubiquitous computing, democratization of computing resources, iterative processes of creation and discovery becoming continuous. As a corollary, the increased pace of discovery will fuel amazing innovations in the near future. It is also predicted to completely reshape society as we know it. All in all, it’s a kinder and gentler singularity.

But how does the Internet singularity manifest itself as we speak? Four different aspects are identified:

1. Democratization and “macro-ization”
2. Power law distributions and “long tails”
3. Internet ecosystems and network effects
4. The Innovators’ Dilemma

Ok, now for some details on each of them… First, democratization and “macro-ization”. The idea behind this is that there are great changes happening in the areas of personal and business computing, content, commerce and community.

• As far as personal and business computing goes, there’s a greater availability of raw CPU power (today’s desktop > yesterday’s super computer, and tomorrow’s cell phone > today’s desktop), a greater availability of powerful software tools and a greater integration among toolsets.
• When it comes to content, we see everything coming together: documents, images, movies, audio, publishing, software, research and metadata.
• In commerce, we have hosting (Y! Stores, Office Live, Amazon), P2P sales (eBay), transactions (PayPal), marketing (Overture, Adwords, AdCenter), syndication (Adsense, RSS), development (Web APIs) and business inteligence (Web analytics).
• While on the community side, things are shaping up as well, since we have: communication (IM, email, VOIP), distribution (RSS, BitTorrent, P2P), social (Friendster, blogs, Y/G/M groups, Meetup), romance (Match.com, eHarmony), virtual economies (Everquest, WoW, Project Entropia, Second Life), reputation (eBay, PageRank), preference (collaborative filtering) and affinity (implicit relationships via similarity).

All this leads to some powerful shifts, since today’s “amateur” has more resources than yesterday’s “professional”, and the difference between “amateurs” and “professionals” is visibly diminishing over time. Also, the number of producers/creators is increasing in both absolute and relative terms, which means that the distribution of “creators” is dramatically growing and radically changing in structure.

Next, power law distribution and “long tails”. Both of these concepts need some explaining.

Power law distribution refers to a state where a given environment has exponentially more small things than large things. As a canonical example, we can take the distribution of bio-mass: there’s a small number of whales and elephants, and an enormous number of bacteria. This state is commonly found on the Internet, where there are few large contestants in the field, and a lot of small, unknown ones.

The Long tails concept, on the other hand, refers to a state where the “weight” of the tail is greater than the head. This concept was first coined by Chris Anderson in the Wired article of the same title. A canonical example is the recording industry: there are few “Britney Spears” (i.e., performers with enormous record sales), and a vast number of unsigned performers with little or no record sales. This requires that “tail” producers be able to survive. Here’s a visualisation of the concept:

Back to our Internet singularity now, and the impact of long tails and power law distribution. Long tails are currently found in content, commerce, and communities. They exist because the “physics” of the online world differs from the offline world: the size of warehouse and shelf space is irrelevant, the distance and medium are irrelevant, the aggregation, remixing, and tagging all contribute, and scale can increase independently of human action.

As an implication, consumers are now slowly becoming producers and, in combination, the small producers may turn out to outweigh the large producers. Even with sites like eBay and Amazon, there are a lot of online resellers out there that manage to gain a profit, with minimal costs.

All right, next come the Ecosystems & Network Effects. Any ecosystem requires participants on all scales. The Internet can easily be viewed as such an ecosystem: we have head participants (old school companies), tail participants, aggregations and playlists, remixes and mashups, annotations, reviews, tags and meta data, and mere activity.

Now, what is the network effect? It’s a phenomenon whereby the value of a network increases as a function of the number of participants. For instance:

• A direct Network Effect – telephones: the more people that have one, the more valuable each phone is to each user.
• An indirect Network Effect - OS Development: users use an OS because it has the most applications, and in turn, developers write to an OS because it has the most users.

Both the Internet ecosystem and the network effects are key elements of the Internet singularity phenomenon. The main idea emerging behind this is that the whole is larger than the sum of the parts. Content is based on the language of preference, commerce promotes information flow through endorsements, reputation and market signals, and the community acts as a collective filtering mechanism. The implications of all this are that each component is the “chicken” to the other’s “egg”, in that lacking any makes the other two less valuable. In combination, networks mutually reinforce one another.

Finally, we get to the Innovators’ Dilemma. What’s this all about? Here’s the pattern one can easily identify when it comes to the Internet: the first people/companies in any industry focus on small number of large and high-margin customers. Late arrivals then have to focus on lower-margin customers, so they learn efficiencies because they compensate lack of margin with scale. Meanwhile, competition increases, and margins shrink. The established companies rarely learn the efficiencies that the younger companies grew up with, so the late arrivals win because they can take optimizations and apply them to the head.

Thus, the Innovator’s Dilemma defined goes somewhat like this: the first companies in an industry (the innovators) must eventually destroy their own business before someone else does. Why? Because disruptions happen from the bottom (e.g. Cray killed by SGI; SGI killed by Sun; Sun killed by PC; will the PC be killed by cell phones?).

Besides that, there are quite a few differences between the offline and online worlds. For instance, offline worlds require a huge startup costs for new businesses, they have an aggregate size of the tail is limited by physics, more business usually implies more employees (work harder), quality products usually implies high touch, and innovation iterations follow product and business cycles. When it comes to online worlds, we notice diminishing startup costs for new businesses, an aggregate size of the tail that is potentially unlimited, more business may not require more employees (work smarter), quality products can just be a better algorithm, and innovation iterations follow data flow cycles.

Taking all this into consideration, we can deduce that a bigger tail makes for more potential disruptions, and that the non-physical aspects of the Internet speed up the natural “clock cycle” of progress. This leads to a simple and obvious implication, which is that the process of societal evolution is itself changing.

To sum up, we have the Internet singularity phenomenon, characterized by four main elements:

• Democratization and “macro-ization” – which yields massive parallelization
• Power law distributions and “long tails” – which tells us that decentralization is bigger than centralization
• Internet ecosystems and network effects – which points out that the whole is larger than the sum of the parts
• The Innovators’ Dilemma – which brings higher bandwidth and lower latencies on information flow, and more frequent improvements

Starting from the paths leading to the Singularity phenomenon, here’s a small diagram illustrating the same concepts, now applied to the Internet Singularity, and the links between them:

As time goes on, the Internet’s content, composition, and participants more accurately reflects the physical world. In the limit, the physical world effectively becomes instrumented by the virtual world. Having this strong connection between the online and offline worlds allows for science to be carried out in a revolutionary new way: theories will be tested relative to Internet data, empiricists will have almost unlimited data, and simulations will truly allow us to experiment in a universe of theories.

Web search is now the greatest applied Computer Science research and development problem and it will drive the priorities of AI and other sub-fields for decades to come. The future of the search engine is to model the human mind in the aggregate. Also, virtual worlds are now used to study emerging economies, the online bestiary (viruses, Trojan horses, etc.) is now a legitimate way to study epidemiology, and empirical sociology is now carried out on Web data.

This about sums it up. I’d like to express my sincere appreciation if you were patient enough to read the entire post. Hopefully, you’ll find it as interesting as I did.