Part 2 - Navigating Disruption: Forces of Change for Engineering Excellence

With heartfelt gratitude, I extend my sincerest appreciation to our community of 310 newsletter subscribers. While my occasional tweets offer glimpses into my world, the art of writing demands deep research and dedication, from selecting the topic to crafting the content. My writing journey is twofold: to share authentic knowledge and to engage in a reciprocal learning process with this exceptional community. This endeavor humbles me beyond words.

Disruption is a fact of life for businesses today. New technologies, new business models, and changing customer expectations can all lead to disruption. But disruption doesn't have to be a bad thing. It can also be an opportunity for businesses to innovate and grow.

Sustaining innovations are the key to consistent performance, whereas disruptive innovations are the key to dramatic changes in power - Jeffrey Moore

This two-part article focuses on fostering a culture of innovation to address disruption head-on.

I strongly recommend starting with the first part, where I narrate the dynamics of business disruption in Hollywood-style. Here, we delve into how business leaders can transform disruption into an avenue for strategic growth and survival. This second part continues our exploration from an engineering perspective.

Raise your hand if the term "innovation" hasn't reverberated in your workplace for the past two decades. Every entity—be it a company, department, or individual—aspire to embrace innovation. Indeed, each entity has its own interpretation of innovation. However, what truly counts is the percentage and probability of success. Evidently, companies like Alphabet and Meta, investing a combined $75 billion in R&D in 2022, stand apart from those with a $5 million budget. While budget is not the sole determinant, its availability certainly expedites responses to external disruptions and fine-tunes M&A strategies when internal R&D falls short of delivering breakthroughs.

Many of us entrenched in the technology and engineering domain observe the ever-evolving nature of technologies. Some technologies, like COBOL, endure, while others like Visual Basic, Smalltalk, Sybase, and DBase quietly fade from prominence. These technologies, though instrumental, are often tactical solutions to larger distributed system challenges. Meanwhile, other technologies, such as Oracle or traditional RDBMS, persist despite the rise of NoSQL alternatives like MongoDB, Redis, Cassandra, and Kinesis. From my vantage point, five technological forces have reshaped the landscape:

1. Internet
2. Mobile
3. Social Media
4. Cloud
5. Artificial Intelligence

These forces are now integral to any tech-driven transformation. While their composition varies among firms, their impact is universal. Let's briefly explore each force within the context of transformation, followed by the tools and opportunities they present for supporting business disruption.

Ubiquitous Internet: The ubiquity of the Internet today is taken for granted, yet it wasn't always so. Venture Investment in tech often surges when opportunities arise. Though business ideas might falter initially, the foundations and infrastructure built endure. Reflecting on past tech bubbles—DotCom, Crypto, Web3—I'm convinced they will return with more refined, practical, and commercially viable use cases. An analogy is W3 SOAP/XML's fall and REST/JSON's rise as the de facto standard for web services.

Back in my college days, sending an email meant cycling four miles to an internet cafe with dial-up. Who recalls Perl, PHP, and CGI? Tech giants like Google, Amazon, and Apple, whose revenues hinge on internet performance and standards, actively shaped the landscape with Web2.0, Chrome, and HTTP/2.

Mobile and Social Symbiosis: Mobile and Social Media's symbiotic evolution is intriguing, transcending mere technical and economic implications. Their psychological and cultural impacts on society ripple through all aspects of business. While enterprise software used to be clunky, consumer software was user-friendly. The advent of Social Media and Mobile technologies, tackling distribution challenges, has set the bar for seamless and intuitive digital experiences. I recall a support engineer solving an issue from his child's scouts camp using an iPhone VPN to access server logs on a tiny SSH screen. Today, tools like NewRelic or Splunk Mobile apps makes me think of that experience as a bad dream. Social interaction wasn't always a software engineering forte; Social Media platforms like Yammer, Slack, and Microsoft Teams transformed every facet of business, even Sales, Customer Service, and Investor Relations.

Cloud offers the silver-lining Cloud technology has expedited innovation dramatically. In my early career, provisioning servers, switches, and data center space was time-intensive. Owning data centers—renting or collocating—proved inefficient. Cloud technology addressed these issues, offering elasticity, flexibility, and velocity. AWS, Google Cloud, and Microsoft Azure now provide critical application stacks and sub-stacks like below, to build and operate secure and scalable applications.

• Domain Name Service (DNS)
• Content Delivery Network (CDN)
• Load Balancer
• API gateway
• Various forms of Compute (bare metal, virtual instances, managed serverless compute, managed containers, serverless functions)
• Diverse Storage solutions balancing performance, compliance, and cost
• Traditional and NoSQL Databases
• Streaming and Data Pipeline technologies
• Networks (Interconnects/VPNs)
• Developer tools for automation, deployment, and operations
• Observability platforms for log management, alerts, self-healing, and auto-scaling
• Security services for data protection, compliance, and more

These elements seamlessly integrate with traditional data center-based apps, fostering hybrid-cloud architectures for interoperability and backward compatibility.

Artificial Intelligence's Trailblazing Power No discourse on disruption is complete without acknowledging Artificial Intelligence (AI). This force accelerates disruption differently. Unlike the other forces that demand deliberate and extensive resource allocation, AI amplifies technology and business disruption with minimal resources. Consider this: a substantial portion of application development involves integrating mature technologies. If a technology accelerates or automates this integration, it's revolutionary. Early AI iterations have already demonstrated potential in text/image generation, summarization, classification, and labeling. While challenges remain around security, data protection, compliance, and Intellectual Property, AI is at a point similar to the early Internet era. Just like the DotCom bubble, the AI bubble might burst, but winners will emerge. My bet is on those poised to become the next Amazon, Google, or Meta.

Returning to disruption, engineering's role is to support and partner with the business before, during, and after turbulent times. This article underscores that leveraging these five forces to experiment has never been easier. Ideas can be safely developed, tested, deployed, marketed, measured, and iterated. If there is not enough traction, ideas can be retired without significant capital drain.

It's also the engineering partner's responsibility to filter hype, offer realistic technology insights, and manage expectations. Imagine being part of a two-in-a-box model, where successful products are both commercially viable and technologically superior. Early analytics integration helps measure usage and adapt directions. Social features enable user feedback collection and organic referrals.

In essence, today's technology landscape empowers businesses to navigate disruption swiftly, safely, and cost-effectively.

• What other forces could be added to this list?
• Do you see any other trends or technology sub-stacks that will accelerate innovation or help disruption?
• Share your own experiences of how these forces influenced your thinking?

Don't miss out on the opportunity to refer your own network to be a part of this transformative conversation. Subscribe today and stay informed, inspired, and connected.