Importance of AI Infrastructure for Innovation

20 years ago Mary Meeker called the internet’s rise; 10 years ago, the mobile revolution. Last week, she made her biggest bet yet... And it has nothing to do with models. In her new 340-page report, Meeker reveals what’s actually driving AI’s future: Infrastructure. Not just chips. But power, land, CapEx, and velocity. Here are the 10 most important takeaways from her report ranked from most to least significant: 1. CapEx is now strategy. $455B in 2024 AI data center spend. A 63% YoY jump. Not a spike, this is a structural shift. Infrastructure is the product. 2. Power is the gating item. Data centers use ~1.5% of global electricity. Demand is growing 4x faster than grid supply. The bottleneck is the grid. 3. Inference eats the future. Training is episodic. Inference is forever. As AI agents scale, inference will drive long-term infra costs. 4. Speed is a strategic moat. xAI built a 750,000 sq. ft. facility in 122 days. Deployed 200K GPUs in 7 months. Fast build = competitive edge. 5. Custom chips = stack control. Amazon (#Trainium), Google (#TPU), Microsoft (#Phi). Silicon is no longer optional, it’s leverage. 6. Overbuild is intentional. Hyperscalers are doing what Amazon Web Services (AWS) did in 2006: build ahead of demand. Surplus compute becomes a platform. 7. Emerging markets are the next frontier. 50% of internet users. <10% of AI infra. With the right energy and capital stack, emerging markets could leapfrog legacy hubs. 8. AI data centers are AI factories. "Apply energy, get intelligence." - Jensen Huang, NVIDIA CEO. Not metaphor. New economics. 9. Cooling and grid tie-ins are the edge. Latency, liquid cooling, substation access, infra is no longer just real estate. It’s engineering. 10. Sovereignty is back. Governments are co-investing in “Sovereign AI.” Infra is no longer neutral, it’s strategic. The next wave of AI winners won’t be those with the smartest models. They’ll be the ones who control the stack those models run on. #datacenters

𝗦𝘁𝗼𝗽 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗼𝗳 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝗮𝘀 𝗮𝗻 𝗮𝗽𝗽–𝗶𝘁’𝘀 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲. Banks and credit unions exploring AI can move past pilots and prototypes—and start building real value. Many FIs are in a reactive posture toward generative AI—running pilot programs, deploying isolated tools, or setting general usage policies. While this exploratory stage is important, GenAI must ultimately be treated as foundational infrastructure, similar to broadband connectivity or cloud architecture. Institutions that approach generative AI this way are seeing measurable improvements in productivity and staff enablement in three key areas: 1) knowledge management; 2) process and workflow optimization; and 3) personal productivity. AI isn’t a project—it’s infrastructure because it: 1️⃣ 𝗣𝗼𝘄𝗲𝗿𝘀 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗹𝗮𝘆𝗲𝗿𝘀 𝗼𝗳 𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀. From fraud detection to customer service to credit underwriting, the same AI models (e.g., NLP, predictive analytics, anomaly detection) are being trained and reused across departments. Think of it as a digital power grid: once built, it feeds every part of the organization. 2️⃣ 𝗘𝗻𝗮𝗯𝗹𝗲𝘀 𝗮𝗱𝗮𝗽𝘁𝗶𝘃𝗲 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗲𝘀. Infrastructure isn’t just hard-coded logic—it’s built to learn and adapt. AI infrastructure allows banks to respond to customer behavior changes in real time, rather than through manual policy updates and IT rebuilds. 3️⃣ 𝗦𝘂𝗽𝗽𝗼𝗿𝘁𝘀 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻. Once AI infrastructure (data pipelines, model management, governance frameworks) is in place, product and experience teams can innovate faster without reinventing the tech every time. 4️⃣ 𝗕𝗲𝗰𝗼𝗺𝗲𝘀 𝗽𝗹𝘂𝗺𝗯𝗶𝗻𝗴. AI becomes most powerful when it disappears into the background—powering everything, not just one shiny thing. Generative AI becomes a query layer over data infrastructure, machine learning models become embedded in real-time decisioning engines, and chatbots evolve into AI-powered customer operating systems. See the link in the comments for the new #Fintech Snark Tank post 𝙂𝙚𝙣𝙚𝙧𝙖𝙩𝙞𝙫𝙚 𝘼𝙄 𝘼𝙨 𝙄𝙣𝙛𝙧𝙖𝙨𝙩𝙧𝙪𝙘𝙩𝙪𝙧𝙚: 𝘼 𝙋𝙧𝙤𝙙𝙪𝙘𝙩𝙞𝙫𝙞𝙩𝙮 𝙋𝙡𝙖𝙮𝙗𝙤𝙤𝙠 𝙁𝙤𝙧 𝘽𝙖𝙣𝙠𝙨. #BankingInnovation #AIinBanking #GenerativeAI #CreditUnions #FinancialServices #DigitalTransformation #ProductivityTools #AIInfrastructure #FutureOfWork Hapax Cornerstone Advisors Kevin Green Keara McGlynn Emily Osburn Hank Seale Jonny Rosen Aaron Kwan Connor Huddleston

Over the past few months, AI has moved from experimentation to expectation. And cloud infrastructure is feeling the pressure. Google Cloud’s 2025 State of AI Infrastructure Report — led by my friend Nirav Mehta, Google Cloud — captures this shift with precision. A few numbers that really made me stop and think: -- 98% of companies are no longer asking “if” they’ll adopt GenAI — they are building, experimenting, or already using it in production. -- 64% are balancing GenAI between improving internal operations and external customer experiences — it's no longer enough to just automate internally. -- 70% of organizations struggle with data governance and readiness — a silent bottleneck that will make or break AI projects. -- 74% prefer a hybrid cloud model for GenAI workloads — flexibility, compliance, and sovereignty are non-negotiable now. -- 83% put cost efficiency at the center of their GenAI infrastructure decisions — budgets are getting tighter even as ambitions grow. Reading through the report, one thing became very clear to me: Building AI isn't just about better models. It's about better infrastructure choices. -- In a way, infrastructure has become the new strategy. -- If your systems aren't scalable, your AI can't evolve. -- If your data isn't governed properly, your AI can't be trusted. -- If your cloud isn't flexible, your AI can’t reach the edge where users actually are. Another piece that stood out: Edge computing is no longer “coming soon.” It's already happening. Industries like healthcare, manufacturing, and retail are deploying GenAI models on IoT devices, mobile platforms, and distributed systems — because customers expect real-time intelligence, not just cloud-driven analysis. As GenAI becomes the backbone of how organizations operate, the gap between infrastructure leaders and laggards will only widen. If you’re thinking about your AI strategy today, this report isn’t just a good read — it’s a reality check. Kudos again to Nirav and the Google Cloud team for putting real numbers, real challenges, and real strategies in front of us.

Your SAP AI is only as good as your Data infrastructure. No clean data → No business impact. SAP is making headlines with AI innovations like Joule, its generative AI assistant. Yet, beneath the surface, a critical issue persists: Data Infrastructure. The Real Challenge: Data Silos and Quality Many enterprises rely on SAP systems - S/4HANA, SuccessFactors, Ariba, and more. However, these systems often operate in silos, leading to: Inconsistent Data: Disparate systems result in fragmented data. Poor Data Quality: Inaccurate or incomplete data hampers AI effectiveness. Integration Issues: Difficulty in unifying data across platforms. These challenges contribute to the failure of AI initiatives, with studies indicating that up to 85% of AI projects falter due to data-related issues. Historical Parallel: The Importance of Infrastructure Just as railroads were essential for the Industrial Revolution, robust data pipelines are crucial for the AI era. Without solid infrastructure, even the most advanced AI tools can't deliver value. Two Approaches to SAP Data Strategy 1. Integrated Stack Approach:   * Utilizing SAP's Business Technology Platform (BTP) for seamless integration.   * Leveraging native tools like SAP Data Intelligence for data management. 2. Open Ecosystem Approach:   * Incorporating third-party solutions like Snowflake or Databricks.   * Ensuring interoperability between SAP and other platforms. Recommendations for Enterprises * Audit Data Systems: Identify and map all data sources within the organization. * Enhance Data Quality: Implement data cleansing and validation processes. * Invest in Integration: Adopt tools that facilitate seamless data flow across systems. * Train Teams: Ensure staff are equipped to manage and utilize integrated data effectively. While SAP's AI capabilities are impressive, their success hinges on the underlying data infrastructure. Prioritizing data integration and quality is not just a technical necessity → It's a strategic imperative.

🌐 The $80B Inflection point - 2025's AI Data Center Revolution As an IDCA - International Data Center Authority Board member we observe Microsoft’s $80B FY2025 data center announcement signals a fundamental transformation in digital infrastructure. This isn't just expansion—it's a complete reimagining of our digital foundation. 📊 The Unprecedented Scale: • MSFT FY2025: $80B capex (84B with leases) • 2x YoY growth from FY2024's $44B • Industry projection: $500B+ total data center spend by 2025 • McKinsey: 33% CAGR in AI-ready demand through 2030 • Trajectory: 70% AI workload share by decade end 🔍 Recent Market Signals: • KKR's $50B AI infrastructure commitment • NVIDIA's H200/B200 2x performance gains • TSM's $40B Arizona expansion • Intel's $100B Ohio mega-site • Samsung's $230B chip investment plan • ASML's High-NA EUV deployment timeline • Micron's $100B NY investment ⚡ Three Critical Challenges: 1. Physical Reality: • GPU clusters spanning >1 mile • 100kW+ per rack cooling demands • 50 MW+ per facility power needs • AI training runs: 500,000 kWh each • 15-20% annual power density increase • Water usage: millions of gallons daily 2. Resource Constraints: • 2-3% global electricity consumption • 95% GPU market concentration • 54% foundry capacity in one region • 3nm production limited to 2 players • Critical mineral supply bottlenecks • 18+ month equipment backlog 3. Infrastructure Innovation: • CXL 3.0 adoption acceleration • Liquid cooling standardization • AI-driven optimization • Sustainable heat recapture • Distributed power systems • Quantum-ready infrastructure planning 💭 Market Analysis: • 65% capacity shift to secondary markets • 40% edge deployment surge • 3x sustainable cooling innovation • 85% new builds AI-optimized • 25% premium for AI-ready space • 40% increase in specialized talent demand 🔮 2025 Critical Watchpoints: • TSMC 2nm/Intel 18A ramp • High-NA EUV deployment • HBM3e production scale • Grid infrastructure readiness • Silicon photonics adoption • Chiplet architecture evolution • Sustainable power solutions ⚡ The Energy Equation: • Current AI centers: 2-3x traditional power density • Latest GPU clusters: 350-400W per square foot • Single chips pushing 800W+ • Cooling efficiency becoming critical • Grid modernization urgency The decisions made in the next 12 months will echo for decades. Through IDCA's global lens, we see both unprecedented opportunity and sobering challenges. The question isn't just about scaling—it's about scaling intelligently. Key Consideration: Are we building what we need, or just what we know? How do we balance immediate AI infrastructure demands with sustainable, long-term growth? What critical factors do you see missing from the current industry dialogue? #DataCenter #AIInfrastructure #Innovation #IDCA #DigitalTransformation #Sustainability #TechLeadership

🔬 Building AI Excellence in Academic Medical Centers: A Strategic Framework I strongly believe academic medical centers must lead in two critical AI frontiers: breakthrough scientific discovery and clinical care transformation. 🧬 The scientific potential is extraordinary. Just as AlphaFold revolutionized protein folding, we need ambitious AI projects tackling fundamental biomedical challenges - brain foundation models to decode neural function and disease, AI-powered drug design for personalized therapeutics, and virtual cell models to advance cell therapy. I think academic medical centers are uniquely positioned to drive these transformative discoveries. 🏥 But scientific breakthroughs aren't enough. We need to transform clinical care through AI innovation. This means building AI-enabled patient experiences (imagine AI-powered triage systems that combine imaging, symptoms, and medical history), pioneering new frontiers like AI-assisted robotic surgery, and reimagining how hospitals function with AI at their core. ⚡ The foundation for this dual mission starts with structure. A dedicated Department of AI serves as the beating heart of innovation - not as a siloed entity, but as a central hub radiating expertise throughout the institution. This department needs critical mass: dedicated faculty, technical experts, and hybrid leaders who understand both AI and medicine. Critical to success is transforming the culture from "publish and forget" to "implement and improve." This means building robust pathways for clinical validation of AI tools. Academic medical centers must excel at conducting efficient clinical trials of AI solutions, measuring real-world outcomes, and scaling what works. The goal isn't another paper - it's bringing validated AI innovation directly to patient care. The key pillars supporting this framework: 📊 First, data infrastructure must be modernized and standardized. Our clinical data is our greatest asset, but it needs to be AI-ready. This means investing in data quality, governance, and accessibility while maintaining rigorous privacy standards. ✓ Second, we need streamlined processes for clinical validation. This means building expertise in AI-specific trial design, creating efficient IRB pathways, and developing frameworks for measuring AI impact on clinical outcomes. The ability to rapidly test and validate AI tools should be part of our DNA. 🎓 Third, education and training must be reimagined. We're creating a new hybrid workforce that bridges AI and medicine, focused on both breakthrough discovery and clinical implementation. Success requires a fundamental cultural shift: viewing AI as a core capability for both scientific discovery and patient care improvement. I believe we're just scratching the surface of what's possible. The institutions that build both the technical infrastructure and implementation culture for AI will define the next era of medicine and scientific discovery.

🔥 Why DeepSeek's AI Breakthrough May Be the Most Crucial One Yet. I finally had a chance to dive into DeepSeek's recent r1 model innovations, and it’s hard to overstate the implications. This isn't just a technical achievement - it's democratization of AI technology. Let me explain why this matters for everyone in tech, not just AI teams. 🎯 The Big Picture: Traditional model development has been like building a skyscraper - you need massive resources, billions in funding, and years of work. DeepSeek just showed you can build the same thing for 5% of the cost, in a fraction of the time. Here's what they achieved: • Matched GPT-4 level performance • Cut training costs from $100M+ to $5M • Reduced GPU requirements by 98% • Made models run on consumer hardware • Released everything as open source 🤔 Why This Matters: 1. For Business Leaders: - model development & AI implementation costs could drop dramatically - Smaller companies can now compete with tech giants - ROI calculations for AI projects need complete revision - Infrastructure planning can possibly be drastically simplified 2. For Developers & Technical Teams: - Advanced AI becomes accessible without massive compute - Development cycles can be dramatically shortened - Testing and iteration become much more feasible - Open source access to state-of-the-art techniques 3. For Product Managers: - Features previously considered "too expensive" become viable - Faster prototyping and development cycles - More realistic budgets for AI implementation - Better performance metrics for existing solutions 💡 The Innovation Breakdown: What makes this special isn't just one breakthrough - it's five clever innovations working together: • Smart number storage (reducing memory needs by 75%) • Parallel processing improvements (2x speed increase) • Efficient memory management (massive scale improvements) • Better resource utilization (near 100% GPU efficiency) • Specialist AI system (only using what's needed, when needed) 🌟 Real-World Impact: Imagine running ChatGPT-level AI on your gaming computer instead of a data center. That's not science fiction anymore - that's what DeepSeek achieved. 🔄 Industry Implications: This could reshape the entire AI industry: - Hardware manufacturers (looking at you, Nvidia) may need to rethink business models - Cloud providers might need to revise their pricing - Startups can now compete with tech giants - Enterprise AI becomes much more accessible 📈 What's Next: I expect we'll see: 1. Rapid adoption of these techniques by major players 2. New startups leveraging this more efficient approach 3. Dropping costs for AI implementation 4. More innovative applications as barriers lower 🎯 Key Takeaway: The AI playing field is being leveled. What required billions and massive data centers might now be possible with a fraction of the resources. This isn't just a technical achievement - it's a democratization of AI technology.

Nations succeed when they harness their resources for technological advantage. For AI, this means advanced chips, data, and energy – and expanding America’s AI infrastructure is crucial to driving economic growth and maintaining our edge over China. Today, we’re proposing a set of ambitious ideas for how to build more of it. If we get this right – we can reindustrialize the country, revitalize the American Dream, and ensure democratic, free AI prevails over autocratic, authoritarian AI.🚀 1️⃣ AI Economic Zones: States can incentivize faster permitting and approvals for AI infrastructure, making it easier to bring dormant nuclear reactors back online and build new wind farms and solar arrays while incentivizing states and local communities to participate by allocating a portion of the AI compute generated to support the standing up of AI developer hubs in the local community. ☀️ 2️⃣ National Transmission Highway Act: We need an ambitious program akin to the 1956 Interstate Highway Act to expand transmission lines, fiber connectivity, and natural gas pipeline construction. Providing authority and funding to address the “Three P’s”—planning, permitting, and payment—will be essential to unlocking the vast amounts of renewable energy currently stuck in backlogs. ⚡ 3️⃣ Government Backstops for AI Public Works: The government can encourage private investors to fund high-cost energy infrastructure projects by committing to purchase energy and other means that lessen credit risk. The investment should include investment in training a new generation of workers to support and run this infrastructure. This collaboration plays to each sector’s strengths: government sets the goals, industry builds to meet them. 🏗️ 4️⃣ North American Compact for AI: The US and its neighbors can team up to streamline access to capital, supply chains, and talent in a way that supports AI infrastructure construction. Over time, it could expand to a global network of our allies, creating an economic bloc capable of competing with any in the world. 🌎 5️⃣ Tapping the Expertise of the Nuclear Navy: Nuclear power is America’s largest source of clean energy, yet our infrastructure is aging. By tapping into the expertise of our nuclear Navy—which operates about 100 small modular reactors on its submarines—we can help to revive America’s nuclear ecosystem. ⚛️ OpenAI is committed to working with forward-thinking leaders from both parties to turn these ideas into reality. It’s time to do what America does best: think big, act big, and build big. 💪

If there's one song that resonates with today's CIOs, it would be "All Eyes on Us." The AI transformation has placed a significant spotlight on them, compelling them to not only adopt Generative AI but also ensure that their data and infrastructure are AI-ready. Nearly half (47%) of CIOs are prioritizing the overhaul of their data platforms to fuel business growth (quoting from a recent PWC survey).   CIOs face a million-dollar dilemma: should they wait to deploy Generative AI until data transformation is complete, which could take years, or deploy GenAI without full access to sufficient data to provide accurate insights?    Here are four tips that may help ease some of this pressure: --> Streamline Data, Breaking Down Silos: CIOs must prioritize and Modernize, standardize, and consolidate all data systems before tackling specific functional needs. AI requires vast amounts of data, and with much of that data trapped in inaccessible silos, its effectiveness will be significantly reduced! --> Focus on Business Value: Prioritize data-driven projects that directly give business value, as these tend to have the greatest business impact and will help you explain the ROI. --> Think big with GenAI: Look beyond individual use cases and concentrate on repeatable patterns to scale AI solutions quickly across your businesses! --> Invest in Modern AI+BI solutions:  With the latest innovations in AI-powered business intelligence (BI), CIOs no longer have to choose between a massive, years-long data transformation project and fast, efficient deployments of Generative AI. A thoroughly modern BI platform can seamlessly integrate data—regardless of where it is stored—with AI to provide convenient, relevant insights to users within their existing apps and workflows.   As innovators, CIOs must always focus on the most important business priorities. Remember, It's not just about chasing the next shiny object. Innovation, for innovation's sake, is a waste of time!  

Imagine billions of specialised AI agents collaborating across a decentralised architecture. This is impossible with our current internet infrastructure. That’s precisely why a group of stellar folks at MIT are working on the new future of the internet - Project Nanda: Architecting the "Internet of AI Agents" I had the opportunity to sit down with these brilliant minds building this future. Here are my takeaways for the future of the internet 👇🏻 To understand why we need a new infrastructure, you need to know how AI development today parallels the internet evolution of the early 90s. - Phase 1: Mainframe AI (DOS-style interaction) - Phase 2: Intranet of AI (company-siloed AI agents) - Phase 3: Internet of AI (agents moving globally in a trustless environment) - Phase 4: World Wide Web of AI (decentralised architecture) The problem is the same as when the internet evolved to what we know today. We’re missing the foundation, trust, discovery & authentication layers for this agentic web to evolve. The current internet infrastructure is both inadequate for the World Wide Web of AI and also stifles innovation👇🏽 - We currently have only ~300 million IP addresses, but AI Agents will need a minimum of a trillion IP addresses to realise this dream. - Our current DNS propagation time (12-24 hours) is too slow for a dynamic agent ecosystem with millions or trillions of agents. - There is also the risk of walled gardens like the AOL/CompuServe era - We also face the risk of companies like Microsoft and Salesforce creating siloed agent ecosystems, as their user data is already locked within their own ecosystems. - Finally, we could also see the Potential "App Store" model with 30% revenue cuts limiting innovation. I understand this is all a bit too complex, but NANDA’s goal is simple: MCP provides standardised interaction between AI agents, tools, and other resources. This is great for centralised intelligence. But NANDA’s goal is to add the critical infrastructure needed for distributed & decentralised agent intelligence at scale👇🏽 - Discovery mechanisms for agents to locate one another within the network. - Search functionality for querying distributed knowledge across agent networks. - Authentication & secure verification protocols for trustworthy agent interactions. - Verifiable agent-to-agent data & monetary exchange & accountability systems You can learn more about this here: https://lnkd.in/gJQAW_NX