Artificial Intelligence (AI), Cloud Native

AI Made easy - AI and cloud native innovation

How AI is Driving Innovation in the Cloud-Native Space

I have been playing with AI for some time, In fact, the featured image of this very blog post was generated by AI Art. With the rise of cloud-native ecosystems like Kubernetes and serverless, the world of software engineering has been completely revolutionized. As AI continues to influence every aspect of our lives, as a cloud-native enthusiast, I’m curious about what it will bring to the cloud-native world as well.

In this blog post, we’ll take a closer look at how AI is impacting the cloud-native system and what it means for the future of modern application development.

Cloud-Native and AI Synergy

If you’ve been using ChatGPT and the new Bing from Microsoft, you may know they’re both powered by generative AI! Generative AI is a type of artificial intelligence that uses neural networks and deep learning algorithms to create unique content such as text, images, videos, music, or even coding on your behalf. It doesn’t just recognize patterns in existing data but goes one step further to create something entirely new when presented with natural language prompts.

Generative AI is one of the most exciting and rapidly evolving areas of the tech industry today, and OpenAI is at the forefront of this innovation. With a total of $11.3B in funding across 7 rounds, according to data from Crunchbase, OpenAI has made significant strides in advancing the field of AI, particularly through their flagship project ChatGPT. 

In 2018,  at the 2-year mark after OpenAI started to use Kubernetes for deep learning model training, at a time OpenAI pushed the cluster to scale to over 2,500 nodes on Azure on D15v2 and NC24 VMs. 3-years later, this quickly got quickly pushed over again to scaling to over 7500 nodes reported in Jan 2021.

Scaling a single Kubernetes cluster to this size is actually not recommended based on CNCF’s official Kubernetes documentation where even the latest Kubernetes 1.27 supports only up to 5000 nodes (this limit hasn’t been changed and has been part of the release process for quite some time). Sidenote that the official recommendation was only 1000 nodes back in 2016 and bumped up to 5,000 from Kubernetes 1.6  in 2017 (Check out this AWS office hours Youtube video to know the whole story). Notably, AI has always been revolutionizing Kubernetes, as demonstrated by Alibaba’s need for a 10,000-node Kubernetes cluster for a major shopping festival in 2019.

The Rise of New AI Chips

Running AI is incredibly demanding, especially for open AI, it runs large machine learning jobs spanning so many Kubernetes nodes which need full GPU power. To achieve this, it relies on GPUDirect for direct communication with the NIC or NVLink for cross-communication with the GPU.  

On March 21st, 2023, NVIDIA announced the availability of the NVIDIA H100 Tensor Core GPU which is introduced as the world’s most powerful option ( and pricey ) for generative AI training and machine learning inference, which is crucial for advancements like GPT4 (you may get this if you have ChatGPT plus membership and OpenAI limits GPT-4 to 25 messages every 3 hours). This official white paper gives an overview of NVIDIA H100 Tensor Core GPU architecture

This aligns with Microsoft Azure introduced the ND H100 v5 VM on March 13th which is the most powerful and massively scalable AI virtual machine series in Azure by far. Amazon Web Services announced  EC2 UltraClusters of Amazon EC2 P5 instances is coming soon. And Oracle Cloud Infrastructure (OCI) announced the new OCI Compute bare-metal GPU instances featuring H100 GPUs in limited availability. I am intrigued to see what these announcements will bring to the table in terms of advancements in the cloud-native and AI synergy. 

Challenges and Opportunities in Cloud-Native with AI

Although computational power is not the only factor, there also comes the challenges with networking, service reliability (particularly from high-demanding incoming requests for API servers), observability (using tools like Prometheus and Grafana ), and of course, security also play critical roles in all scenarios. 

Over the past years, Cloud-native ecosystems such as Kubernetes, and serverless have revolutionized software design, development, and deployment. As the importance of AI capabilities continues to grow, cloud-native apps are increasingly being infused with AI to enable new use cases and improve business efficiencies.

At the core of your next software innovation lies a modular and extensible Kubernetes-based service architecture that can accelerate the construction of AI platforms while improving resource utilization and delivery efficiency. This approach allows for greater flexibility to integrate AI components while capitalizing on the benefits of cloud-native and AI technologies.


As we have seen, cloud-native and AI technologies can be leveraged to create powerful software solutions that are both efficient and reliable. By combining the strengths of these two cutting-edge technologies, businesses can capitalize on their benefits while driving innovation and business success. With more advancements in this space being made every day, it’s an exciting time for those interested in leveraging the power of cloud-native technology with AI capabilities.

If you enjoy reading blog posts like this one, don’t forget to subscribe to my private distribution list from so you don’t miss them out!

Related Resources

Kubernetes Up & Running

Kubernetes radically changes the way applications are built and deployed in the cloud. The updated edition of the ‘Kubernetes up and running’ ebook shows developers and ops personnel how to use Kubernetes and container technology to achieve new levels of velocity, agility, reliability, and efficiency. It’s available to download for free from here or purchase it from Amazon here if you wish to get a physical copy.

Certified Kubernetes Administrator (CKA) Exam Guide

This book helps individuals learn Kubernetes and obtain certification, opening doors to new career paths as Kubernetes administrators and adding value to their organizations. You can purchase it in either paperback or Kindle format using the code or direct link: with a 20% discount code by May 13th 2023.

Generative Deep Learning

Generative AI is the hottest topic in tech. This practical book teaches machine learning engineers and data scientists how to create impressive generative deep learning models from scratch. purchase it from Amazon here if you wish to get a physical copy from here.

follow us

Stay informed of Cloud Native tech today !

One response to “How AI is Driving Innovation in the Cloud-Native Space”

  1. […] Moreover, training generative AI models is a complex process as they do not just analyze and interpret data but create new content from it. This process involves the model learning patterns within the data and then generating similar but distinct data. The computational intensity of this process necessitates a supercomputer equipe with NVIDIA H100 Tensor Core GPU, which is introduced as the world’s most powerful option ( and pricey ) for generative AI training and machine learning inference, wheich is crucial for advancements like GPT4 (you may get this if you have ChatGPT plus membership and OpenAI limits GPT-4 to 25 messages every 3 hours). You can learn more about how AI and supercomputers from this blog post.  […]

Leave a Reply

Looking to take the next step in your career and unlock a world of possibilities? Look no further than Coursera Plus—the perfect plus one on your journey to success. Get your first month for just $1 with this limited-time offer. Don’t miss it out !

Join the 77% of learners on Coursera who have reported career benefits including new jobs, promotions, and expanded skill sets.


Check our latest news

Subscribe to our newsletter, so you don't miss anything !

New to Artificial Intelligence?Learn the basics of Artificial Intelligence (AI) with this complete beginner's guide. We'll…


Highlight the key takeaways from KubeCon Europe 2023 and how to effective strategies for attending sessions,…


Are you looking to make better decisions and have greater impact? Discover how Donella Meadow's book,…

logo with text CMV e1692108497137
CloudMelon Vision

Copyright @2018-2023 CloudMelon Vision is a trademark of iMelonArt LLC. All rights Reserved. 1100 Bellevue Way NE Bellevue WA 98004 United States. Privacy Policy | Terms of Services

error: Protect the unique creation from the owner !

Discover more from CloudMelon Vis

Subscribe now to keep reading and get access to the full archive.

Continue Reading