14 August 2016
The early 2000s saw open source software and commodity hardware start to displace closed source software and proprietary hardware. In the future, public clouds similar to those operated by Amazon and Google will also displace open source software and commodity hardware. 

Ironically, the cloud will accomplish this using closed source software and proprietary hardware. Understanding the reasons behind the original ascendance of open source software and commodity hardware will help to explain this evolution.

Open source software emerged because of the new programming models and operating methodologies were used to build Internet-based and Internet-scale applications. Customers didn't want less innovation, vendor lock-in and high licensing costs of traditional enterprise IT vendors. Linux became a credible alternative to proprietary HP-UX, AIX, Solaris and Windows operating systems. MySQL and NoSQL databases started to erode Oracle's, IBM's and Microsoft's database hegemony. Hundreds of open source projects thrived because they provided open and free alternatives for almost every component of the software stack starting from hypervisors, such as Xen at the bottom, to application frameworks, such as Spring at the top, and everything in between. 

At the same time, hardware was undergoing commoditization from the dramatic compounding deflation in the cost of processors, memory and storage because of Moore's and Kryder's Laws. Intel became the dominant supplier of commodity processors. White box manufacturers emerged to deliver low-margin, undifferentiated servers and storage that displaced legacy vendors, such as HP's and IBM's offerings based on proprietary PA-RISC and RS6000 architectures. In the new Internet-centered IT world, proprietary hardware offered few durable benefits to customers. These were expensive compared to commodity hardware, and they were slow to innovate, which didn't make extending functionality to address emerging customer needs easy for an ecosystem. 

Legacy vendors of closed source software and proprietary hardware have certainly not faded away entirely. Instead, spending and growth shifted to open source and commodity alternatives, reducing the share of IT spend that incumbents previously enjoyed. This process continues, and the prior generation's incumbents are now forced to evolve or withdraw from the market.
There's a common misconception that hyper-scale clouds are being built using open source software and commodity hardware.
Share this

Using open source software and commodity hardware only makes sense when it offers better value than other alternatives. While this was true with the disruption of the prior generation of hardware and software, it is not longer true with the cloud. Cloud computing is driving a new cycle of disruption that achieves greater cost reductions at an accelerated pace of innovation beyond using open source software and commodity hardware alone. 

Reduced costs and speed of innovation have always been and will continue to be what matters to customers. Customers want scalable, reliable, performant and economical access to compute resources to meet today's IT requirements and to trust that their vendors will continue to address their needs in the future. They care about being able to run a broad variety of software for their applications, whether open source, closed or written themselves. They care that the underlying hardware and infrastructure meets their needs, not whether it's open and commodity.

There's a common misconception that hyper-scale clouds are being built using open source software and commodity hardware. While partly true in the early development of the cloud, this is no longer the case. Cloud infrastructure across the major providers relies increasingly on closed source software and proprietary hardware because there's a limit to the cost savings, performance and operational efficiencies gained from generic, off-the-shelf components. Incremental cost reductions are now being achieved by designing fully integrated environments that are comprised of customized hardware and software and tailored to reduce build and operating costs. 

Amazon's cloud exemplifies this trend. AWS controls all aspects of the design, deployment and operation of its data centers and has achieved enormous scale through automation. The result can drive down the cost of the hardware infrastructure comprising its offerings below what a commodity vendor that relies solely on Moore's Law can achieve. Amazon has publicly discussed this approach and that it designs its own servers and storage arrays. AWS has also started to design much of its own networking gear as it seeks areas to drive further hardware cost savings, performance improvements and reduced operating overhead.
New proprietary cloud platforms deliver value at a lower cost than the open alternatives.
Share this

Likewise, the software that manages the AWS IaaS and platform services is also purpose-built. While the offering incorporates some open source components, such as the Xen hypervisor and Amazon's Linux distribution, the software that manages EC2, RDS, S3, EBS, DynamoDB, Redshift and every other service is proprietary to Amazon -- in other words, it's closed source and gives AWS a tremendous defensible advantage. First, because the software is not available for external use, others cannot use it as the basis for their own competitive offerings. Since the software only runs in the AWS data centers, Amazon retains complete freedom to innovate and improve its offerings at its own pace, which has been significantly faster than its competitors.

Some believe that proprietary cloud platforms like AWS will create a new generation of vendor lock-in and that cost and speed of innovation are not a customer's primary criteria for choosing a cloud solution. Today, this argument isn't as strong. Any software or hardware infrastructure choice comes with some amount of lock-in, and there will always be switching costs if a customer wants to change from one to another. Given that some level of lock-in is unavoidable, the cost-benefit ratio is critical. In the past, vendor lock-in came with a high financial cost, something that open source software and commodity hardware significantly reduced because the cost outweighed the benefit. Today, the financial and innovation benefits of the new proprietary cloud platforms deliver value at a lower cost than the open alternatives -- the benefit outweighs the cost.

While open source software and commodity hardware may not be dead just yet, in their current form, they do not appear to be positioned to compete with the innovations delivered by the cloud providers. If this continues to be the case, growth will shift to the new cloud platforms. The result will be that open source software and commodity hardware will suffer the same negative impact and decline as the prior generation of infrastructure.
Tap to read full article