I had the opportunity to keynote the 2023 DevOps Experience hosted by TechStrong last month. My talk was on the New AI Stack, which is a term I don’t think applies in the new world of loosely coupled cloud native architectures. I have been working on creating a framework that makes sense to discuss AI Infrastructure.  If you’d rather watch the video, it’s at the bottom of this post. 

Concept of Stacks vs. Webs

Traditionally, we’ve leaned on the concept of “stacks” to describe the linear arrangement of technology layers. However, as AI permeates our work and daily lives, a shift towards a more interconnected, “web-like” structure is emerging. This blog post delves into this evolution, focusing on the shift from stacks to webs in AI infrastructure, and outlines the key areas and tools for AI infrastructure management, particularly for DevOps professionals now venturing into AI application development and management.

The term “stack” has been a staple in technology, originating from the early days of computing. It implies a linear, orderly arrangement of technologies, each layer building upon the one below. Classic examples include the LAMP stack (Linux, Apache, MySQL, PHP/Python/Perl) of the early 2000s. This model, while effective in its time, is increasingly seen as limited in the face of the dynamic, interconnected nature of modern AI technologies.

Webs: A More Apt Representation of AI Infrastructure

The conceptual shift from stacks to webs offers a more accurate and effective way to visualize and understand AI infrastructure. This change reflects the complex, dynamic nature of modern AI systems and their integration into broader technological and organizational contexts. This framework needs to be expanded to dive and better describe the depth of AI infrastructure but it’s a start that covers a lot of bases.

Loosely Coupled Infrastructure

Unlike the traditional stack conceptual model, which depicts infrastructure in a linear, layered format, the web model emphasizes multidimensionality and connectivity. In a web-like structure, various AI components—such as algorithms, data storage, processing units, and user interfaces—are interconnected in a flexible, non-linear fashion. This visualization aligns better with how AI elements interact, influence, and reinforce each other in practical settings.

Fluidity and Adaptability for AIOps

Webs symbolize fluidity and adaptability, essential in today’s rapidly evolving AI landscape. This framework accommodates the continuous flow of data and the seamless integration of new technologies, methodologies, and user demands. It supports the idea that AI infrastructure isn’t static but is an evolving network that adapts to new challenges and opportunities.

The Four Pillars of AI Infrastructure

For DevOps professionals, understanding the components of AI infrastructure is crucial. This infrastructure can be broadly categorized into four areas: Data, Fine-Tuning, Narrow AI/Autonomous Agents, and Model Integration Frameworks.

1. Data Management and Integration

Data is the lifeblood of AI. Effective AI infrastructure begins with robust data management, encompassing data warehouses, pipelines, and lakes. Key considerations include:

• • Data Provenance and Tracking: Ensuring the traceability and integrity of data as it moves through AI systems.

• • Vector Databases: Transforming data into vectors (mathematical representations) to facilitate faster, more efficient processing by AI algorithms.

2. Model Training and Fine-Tuning

AI models need training and fine-tuning to adapt to specific tasks or datasets. This process involves:

• • Resource-Intensive Training: Utilizing GPUs and TPUs for processing large datasets and complex algorithms.

• • Fine-Tuning: Adjusting pre-trained models to specific contexts with less resource commitment than fully training an LLM.

3. Narrow AI and Autonomous Agents

Narrow AI focuses on specific tasks, offering more specialized, task-oriented applications:

• • Task-Based AI: Ranging from virtual assistants to diagnostic tools, these models are trained for specific functions.

• • Autonomous AI Agents: Goal-oriented AIs that perform tasks autonomously based on set objectives and parameters.

4. Middleware and Frameworks

Middleware in AI infrastructure, like Langchain and LLamaIndex, provides the necessary tools and frameworks for building and integrating AI applications. They handle complex tasks like chaining conversations across models and supporting AI application development.

AI for Ops: Management Tools

 

Transitioning to AI-centric operations, DevOps professionals need to adapt their toolset. The management of AI infrastructure can be broken down into four areas: Continuous Integration and Deployment, Monitoring and Observability, Configuration Management, and Security.

1. Continuous Integration and Deployment (CI/CD)

AI integration requires adapting CI/CD processes for AI-specific infrastructure, including:

• • Model Version Control: Managing different versions of AI models, similar to software versioning.

• • Automated Testing and Deployment: Ensuring models are updated correctly and scaling AI infrastructure as needed.

2. Monitoring and Observability

Beyond traditional performance monitoring, AI systems demand:

• • AI-Specific Health Checks: Monitoring resource utilization, model performance, and output relevance.

• • Observability: Gaining insights into model behavior and output, facilitating root cause analysis in case of erratic model performance.

3. Configuration Management

Configuration management tools must evolve to handle AI-specific requirements, managing:

• • Model Configurations: Adjusting settings for training, deployment, and operation of AI models.

• • Infrastructure Adjustments: Adapting to the unique demands of AI workloads and data processing.

4. Security

Security in AI includes:

• • Data Security: Ensuring the integrity and confidentiality of data used by AI models.

• • Model Resilience: Protecting AI models from adversarial attacks and ensuring robustness against input manipulation.

The New AI Stack: Infrastructure and Management

The transition from stack-thinking to web-thinking in AI infrastructure represents a shift in how we think about how technology is structured and managed. For DevOps professionals, this shift necessitates a deeper understanding of AI-specific tools and processes. Embracing this web-like, interconnected approach not only aligns with the inherent nature of AI but also paves the way for more innovative, efficient, and scalable AI solutions. As we continue to integrate AI into various facets of business and technology, the ability to adapt and evolve with these changes will be crucial for success in the AI-driven world.