Scaling Secure AI Infrastructure: Key Lessons from Cisco and NVIDIA at GTC

At NVIDIA GTC 2026, Cisco's session Transforming Bottlenecks into Breakthroughs: Scaling Secure AI Networks for AI Factories pointed to a broader shift in AI infrastructure. Compute still matters, but it is no longer enough on its own. Networking, security, orchestration, deployment consistency and workload utilisation now have a direct effect on how AI environments perform in production.

That shift sits at the centre of how we are building at Sharon AI. Early demand was centred on GPU access. The next phase is about delivering secure, repeatable infrastructure that can support both training and inference at scale, without creating unnecessary complexity.

A few key takeaways from the session were:

• Networking now has a direct effect on AI performance
• Platform quality is becoming as important as raw capacity
• Repeatable deployment models are gaining importance
• Security is moving closer to the workload
• Inference and asset life are becoming part of infrastructure strategy

The Network Is Now Part Of The AI Stack

One of the strongest themes from the session was that networking can no longer be treated as a background layer. In AI environments, it directly affects throughput, latency, cluster efficiency, resilience and the protection of data in motion.

Cisco supported that point with several notable figures. AI data centre investment is forecast to reach US$5.2 trillion by 2030. Front end network markets are expected to grow at a 40% CAGR through 2029. Back end network traffic is expected to increase tenfold every two years. 800G is expected to become the majority of front end ports by 2029, while 1.6T is expected to become the majority of back end ports by 2027.

Those figures change how infrastructure decisions need to be made. When port speeds, bandwidth demand and power density are all increasing this quickly, network design cannot sit at the end of the process. It shapes how fast environments move into production, how much useful output they deliver and how well they scale across sites.

The session also moved beyond broad positioning and into benchmark methodology, network KPIs, RDMA KPIs and application KPIs. That is an important point in itself. AI infrastructure performance needs to be measured across layers, not only at box or port level. The cited test cases pointed to high bandwidth efficiency, zero packet drops, no PFC or ECN events and strong fairness across RDMA scenarios. For operators and enterprise buyers alike, networking is now part of the workload outcome.

Capacity Is No Longer The Only Question

The market is moving from capacity-first to platform-first. The question is no longer how many GPUs are available, but how effectively they are turned into a platform that drives utilisation, ROI and TCO.

A poorly integrated GPU environment can still be underused. A secure, repeatable and well-orchestrated platform can produce far more value from the same hardware base. That shifts the discussion away from raw capacity alone and towards consistency, usability and commercial output.

Our own direction reflects that change. With 54MW of data centre capacity and more than 20,000 GPUs planned across Australia, distributed AI infrastructure stops being the exception and starts becoming the model. Once infrastructure is spread across sites, the architecture needs to support:

• central control
• consistent deployment
• predictable security
• reliable performance across environments

Repeatability Is Becoming a Real Advantage

As release cycles tighten, the priority is repeatable, secure deployment without reworking the operating model every time the stack changes.

The presentation covered Cisco Nexus Hyperfabric, Cisco Secure AI Factory with NVIDIA, Cisco Silicon One, 102.4T systems, the N9100 scale-out fabric based on NVIDIA Spectrum 6 Ethernet and reference architectures aimed at hyperscale, neocloud, sovereign cloud, service provider and enterprise use cases. The market is moving towards more consistent, repeatable deployment models that scale more cleanly.

Repeatability improves more than deployment speed. It also supports cleaner operations, stronger consistency across sites, less wasted engineering effort and shorter time to value. In distributed environments, that becomes even more important.

Security Is Moving Closer To The Workload

Security also came through as a core part of AI infrastructure design. The focus was on building protection into the environment itself, with controls placed closer to workloads and aligned more tightly with how AI environments operate in production.

One example was AI workload protection for front end fabric servers. Cisco outlined an approach where policies can sit close to workloads, be automated based on VPC attachment and be offloaded through DPUs so CPU resources remain available for AI workloads. That supports stronger segmentation and policy enforcement without adding unnecessary complexity or reducing usable compute.

As AI deployments move further into enterprise production, this model carries more weight. Organisations working with regulated data, intellectual property and business-critical workloads need security controls embedded within the environment, with strong alignment between infrastructure, policy and workload behaviour.

AI Factories Will Not Be Built On Old Assumptions

The session underlined how quickly the infrastructure baseline is changing. AI environments are moving towards higher-throughput networks, denser systems, more advanced optics, greater power efficiency and architectures built specifically for large-scale workloads.

This shift affects how AI infrastructure is designed, deployed and operated. These environments are becoming denser, faster, more power-aware and more thermally demanding than most traditional data centre models were built to support.

Older planning assumptions do not carry across neatly into AI factories. The pressure now runs across the full environment, including:

• network design
• cooling requirements
• power planning
• operational consistency
• deployment models

The Future Is Inference, Utilisation And Asset Life

AI architectures are moving quickly and infrastructure strategy now needs to account for more than the latest deployment cycle. It also needs to consider how platforms continue delivering value over time across a wider mix of workloads.

That is part of how we think about long-term platform value at Sharon AI. As newer systems come online, older-generation GPUs can still play an important role by shifting into inference workloads as contracts roll off. With CUDA compatibility across generations, that creates more flexibility in how infrastructure is deployed, used and extended over time.

This supports a more efficient operating model, where hardware can continue contributing to commercially useful workloads across multiple phases of its lifecycle.

A strong infrastructure model includes:

• reserving the newest hardware for the most demanding workloads
• moving prior generations into inference and other commercially useful roles
• linking infrastructure planning to utilisation and cost per token
• treating asset life as part of the commercial model

This is becoming more important as token consumption grows and cost per token remains a key consideration across the market.

The Next Phase Of AI Infrastructure

AI infrastructure is advancing beyond raw GPU access alone. The real differentiator now lies in how effectively the entire stack works together. Networking, security, orchestration, repeatability and utilisation are shaping consistent, secure and commercially viable production performance.

The next phase elevates these elements from supporting layers to core performance drivers that directly influence throughput, resilience, ROI and time to value.

Operators and enterprises are moving toward dense high utilisation AI environments built with security by design, repeatability across sites and full lifecycle planning for both training and inference.

At Sharon AI this evolution matches our strategic direction. With 54MW of capacity and over 20,000 GPUs planned across distributed sites in Australia we are focused on infrastructure that scales not only in size but also in quality, security and long term commercial output.

The next phase of AI infrastructure will reward platforms that turn GPUs into high performing, secure and repeatable AI systems capable of delivering sustained value across years and workloads. This is the standard we are building toward.