Researchers Scoured Billions of Links and Found the Internet Is Both Expanding and Shrinking

The online world is continuously expanding—always aggregating more services, more users, and more activity. Last year, the number of websites registered on the “.com” domain surpassed 150,000,000.

However, more than a quarter of a century since its first commercial use, the growth of the online world is now slowing down in some key categories.

We conducted a multi-year research project analyzing global trends in online diversity and dominance. Our research, published today in Public Library of Science, is the first to reveal some long-term trends in how businesses compete in the age of the web.

We saw a dramatic consolidation of attention towards a shrinking (but increasingly dominant) group of online organizations. So, while there is still growth in the functions, features, and applications offered on the web, the number of entities providing these functions is shrinking.

Web Diversity Nosedives

We analyzed more than six billion user comments from the social media website Reddit dating back to 2006, as well as 11.8 billion Twitter posts from as far back as 2011. In total, our research used a massive 5.6Tb trove of data from more than a decade of global activity.

This dataset was more than four times the size of the original data from the Hubble Space Telescope, which helped Brian Schmidt and colleagues do their Nobel-prize winning work in 1998 to prove the universe’s expansion is accelerating.

With the Reddit posts, we analyzed all the links to other sites and online services—more than one billion in total—to understand the dynamics of link growth, dominance, and diversity through the decade.

We used a measure of link “uniqueness.” On this scale, 1 represents maximum diversity (all links have their own domain) and 0 is minimum diversity (all links are on one domain, such as “youtube.com”).

A decade ago, there was a much greater variety of domains within links posted by users of Reddit, with more than 20 different domains for every 100 random links users posted. Now there are only about five different domains for every 100 links posted.

In fact, between 60—70 percent of all attention on key social media platforms is focused towards just ten popular domains.

Beyond social media platforms, we also studied linkage patterns across the web, looking at almost 20 billion links over three years. These results reinforced the “rich are getting richer” online.

The authority, influence, and visibility of the top 1,000 global websites (as measured by network centrality or PageRank) is growing every month, at the expense of all other sites.

App Diversity Is On the Rise

The web started as a source of innovation, new ideas, and inspiration—a technology that opened up the playing field. It’s now also becoming a medium that actually stifles competition and promotes monopolies and the dominance of a few players.

Our findings resolve a long-running paradox about the nature of the web: does it help grow businesses, jobs, and investment? Or does it make it harder to get ahead by letting anyone and everyone join the game? The answer, it turns out, is it does both.

While the diversity of sources is in decline, there is a countervailing force of continually increasing functionality with new services, products, and applications—such as music streaming services (Spotify), file sharing programs (Dropbox), and messaging platforms (Messenger, Whatsapp, and Snapchat).

Website ‘Infant Mortality’

Another major finding was the dramatic increase in the “infant mortality” rate of websites —with the big kids on the block guarding their turf more staunchly than ever.

We examined new domains that were continually referenced or linked-to in social media after their first appearance. We found that while almost 40 percent of the domains created 2006 were active 5 years on, only a little more than 3 percent of those created in 2015 remain active today.

The dynamics of online competition are becoming clearer and clearer. And the loss of diversity is concerning. Unlike the natural world, there are no sanctuaries; competition is part of both nature and business.

Our study has profound implications for business leaders, investors, and governments everywhere. It shows the network effects of the web don’t just apply to online businesses. They have permeated the entire economy and are rewriting many previously accepted rules of economics.

For example, the idea that businesses can maintain a competitive advantage based on where they are physically located is increasingly tenuous. Meanwhile, there’s new opportunities for companies to set up shop from anywhere in the world and serve a global customer base that’s both mainstream and niche.

The best way to encourage diversity is to have more global online businesses focused on providing diverse services, by addressing consumers’ increasingly niche needs.

In Australia, we’re starting to see this through homegrown companies such as Canva, SafetyCulture, and iWonder. Hopefully many more will appear in the decade ahead.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Image Credit: Markus Spiske from Pixabay

Paul X. McCarthy
Paul X. McCarthyhttps://www.onlinegravity.com/keynote-conference-speaker/
Professor Paul X. McCarthy is an author, researcher, and entrepreneur focused on technology and its role in accelerating global innovation and business growth. His book ''Online Gravity'' explains how technology has rebooted economics and is published by Simon & Schuster. A best selling Chinese edition (引力) is published by CITIC Press in Beijing and the leading Russian publisher AST in Moscow has recently published a Russian-edition (Бизнес в интернете). McCarthy is CEO and co-founder of League of Scholars a global research analytics service and CEO of Online Gravity Consulting a strategic technology incubator. He is also adjunct Professor at the University of New South Wales School of Computer Science and Engineering and Honorary Research Fellow at Western Sydney University. Previously McCarthy was the non-executive chairman of The Studio — a new Australian mediatech hub based in Sydney and an adviser to CSIRO's Data61—Australia’s National Science Agency and the inventors of Wi-Fi. McCarthy is also co-founder of several innovative enterprises for IBM, NSW Government and SIRCA the inventors of the worlds first financial markets tick archive — Thomson Reuters Tick History a service now used by most major banks, hedge funds and regulators worldwide. McCarthy received his Bachelor of Science in Computer Science and Graduate DipArts in Fine Arts both from the University of Sydney, where he won the Ian Langham Memorial Prize in History and Philosophy of Science, his Master of Design in Digital Media from the University of Western Sydney and his MBA from Macquarie University, where he won the MGSM Award for Advertising and Marketing.
RELATED
latest
Don't miss a trend
Get Hub delivered to your inbox

featured