And those investments have paid off. Since the 1980s, the top four companies in each industry have increased their market share by 4% to 5% in most sectors. My research shows that investments in proprietary software are driving most of this increase.
With this greater dominance of the industry by big business comes a corresponding decrease in the risk of it being disrupted, a prospect that has haunted business leaders since the arrival of Clayton Christensen. The innovator’s dilemma came out in 1997. By the time Christensen wrote his book, the disturbance was on the rise. But since around 2000, when large companies started investing in proprietary systems, this trend has declined sharply. In any given industry, the likelihood of a top-ranking company (measured by sales) dropping out of one of the top four spots in four years has fallen from over 20% to around 10%. Here too, the investments of the dominant firms in their internal systems largely explain the change. While some new technologies are disrupting entire industries – think what the Internet has done to newspapers or DVDs – others are now removing the disruption from mainstream companies.
How does this happen and why does it seem to affect so much of the economy? This is because these trading systems fill a major gap in modern capitalism. From the end of the 19th century, innovative companies found that they could often achieve considerable savings by producing on a large scale. This change dramatically reduced consumer prices, but there was a trade-off: For companies to reach these large volumes, products and services had to be standardized. Henry Ford said car buyers could have “any color as long as it’s black”. Retail chains have achieved their efficiency by supplying a limited set of products to their thousands of stores. Finance companies offered standard mortgages and loans. As a result, products had limited feature sets; stores had limited choice and were slow to respond to changing demand; and many consumers could not obtain credit or only obtained it on terms that were costly and unsuited to their needs.
The software modifies the equation, partly overcoming these limitations. Indeed, it reduces complexity management costs. With the right data and the right organization, software allows companies to tailor products and services to individual needs, offering greater variety or more functionality. And this allows them to be the best rivals, dominating their markets. Walmart stores offer far more choice than Sears or Kmart stores, and they respond more quickly to changing customer needs. Sears has long been the king of retail; now Walmart is, and Sears is bankrupt. Toyota quickly produces new models when it detects new consumer trends; small auto companies can’t afford the billions of dollars it takes to do that. Similarly, only Boeing and Airbus manage to build new, very complex jumbo jets. The four major credit card companies have the data and systems to effectively target offers to individual consumers, achieving maximum profits and market share; they dominate the market.
These software platforms have allowed large companies to consolidate their dominance. They have also slowed the growth of competitors, including innovative startups.
A variety of evidence supports the idea that startup growth has slowed significantly. A sign is the time it takes to startups backed by venture capital to receive funding: from 2006 to 2020, the median age of a startup in the seed phase increased from 0.9 years to 2.5 years. The median age of a late-stage startup rose from 6.8 years to 8.1 years over the same period. Among the companies that were acquired, the average time from first financing to acquisition tripled, from just over two years in 2000 to 6.1 years in 2021. The story was similar for companies that went public. But the clearest evidence of a slowdown is what happens when companies become more productive.
Large companies use large-scale technologies that make it difficult for startups to grow.
The main characteristic of dynamic economies, what the economist Joseph Schumpeter called “creative destruction”, is that more productive companies – those with better products or lower costs or better business models – grow faster. less productive incumbents, eventually displacing them. But after 2000, on average, firms with a given level of productivity grew only half as fast as firms with the same level of productivity in the 1980s and 1990s. In other words, productivity grew less effect on growth than before. And when productive companies grow more slowly, they are less likely to “overtake” industry leaders and displace them – the hallmark of disruption. Last year, research I conducted with my colleague Erich Denk a direct link between the decreasing impact of productivity improvement and the greater domination of the industry by large companies and their investments in software and other intangible assets.
Another one to see, expressed forcefully by congressional investigators in hearings and in a staff report released in 2020, attributes the decline in economic dynamism to another source: the weakening of government antitrust policy since the 1980s. Account, large companies were allowed to acquire their rivals, reducing competition. Acquisitions have made these companies more dominant, especially in big tech, driving down both the emergence of new technology companies and venture capital funding for start-ups. But in fact, the pace at which new tech companies are entering the market has fallen only slightly from the exceptional surge of the dot-com boom, and seed-stage venture capital funding is at record highs, with two times more financing today than in 2006. and four times the amount invested. The problem isn’t that big companies are preventing startups from entering markets or getting funding; the problem is that large companies use large-scale technologies that make it difficult for startups to grow. Moreover, big companies like Walmart and Amazon have grown primarily by adopting superior business models, not by buying up rivals. Indeed, the pace of acquisitions by dominant firms has declined since 2000.
Of course, such acquisitions sometimes affect the startup landscape. Some researchers have identified the so-called “kill zones,” where Big Tech makes acquisitions to shut down the competition, and where venture capital becomes hard to come by. But others researchers find that startups often react by moving their innovative activity to a different application. Additionally, the prospect of a large company acquisition often inspires people to found startups. Indeed, despite what happened to Nuance, the number of speech recognition and natural language processing startups entering the market has quadrupled since 2005, and 55% of those startups have received venture capital investments.
The slow growth of innovative startups is not just a problem for a few thousand tech companies; the headwinds blowing against companies like Nuance are responsible for issues that affect the health of the entire economy. Researchers at the U.S. Census Bureau have shown that the slowdown in the growth of productive firms largely explains the slowdown in overall productivity growth, a figure that measures the amount of output the economy produces per person and serves as a proxy economic well-being. My own work has also shown it to play a role in growing economic inequality, greater social division, and declining government effectiveness.
What will it take to reverse the trend? Stricter enforcement of antitrust laws could help, but shifts in economic dynamism are driven more by new technologies than by mergers and acquisitions. A more fundamental problem is that the most important new technologies are proprietary, accessible only to a small number of large companies. In the past, new technologies have spread widely, either through licensing or as independent companies developing alternatives; this allowed for greater competition and innovation. The government has sometimes facilitated this process. Bell Labs developed the transistor but was forced by antitrust authorities to license the technology broadly, creating the semiconductor industry. Similarly, IBM created the modern software industry when, in response to antitrust pressure, it began selling software separately from computer hardware.
Today, we are seeing similar developments even without government action. Amazon, for example, opened up its proprietary IT infrastructure to create the cloud industry, which greatly improved the prospects for many small start-ups. But antitrust policy can be used to encourage or compel more big companies to open up their proprietary platforms. Relaxing the restrictions that non-competition agreements and intellectual property rights place on employee mobility can also encourage greater diffusion of technology.
Finding the right balance between policies will be difficult and will take time – we don’t want to undermine the incentives for innovation. But the starting point is to recognize that in today’s economy, technology has taken on a new role. Once a disruptive and competitive force, it is now being used to suppress them.
James Bessen is a lecturer at Boston University School of Law and author of the forthcoming book The New Goliaths: How Companies Use Software to Dominate Industries, Kill Innovation and Undermine Regulationfrom which this essay is adapted.