If AI is going to move the needle on OpEx or even top line growth—driving greater sales, better patient insights, faster decision-making—we need to have fewer toys and more tools that enable scale and deliver.

To get there, tech leaders must pave the way: investing in modular architecture, interoperable data layers and workflow-embedded solutions so seamless they feel invisible yet power the business with greater efficiency and productivity.

But here’s the risk: If you don’t recognize the gap between how your systems operate today and what transformation requires, you’ll miss this chance to shape the future.

Instead, you could be left with solutions bolted onto workflows that don’t talk to each other, on heterogenous infrastructure that’s difficult to scale and ignored by the people they’re supposed to empower.

5 common challenges to scale

While scaling AI is a central ambition for any business, it doesn’t follow the same playbook as traditional software development processes.

It’s made of a web of different decisions, both from the business and the technical side, which presents several challenges for data and technology leaders.

Here are five common challenges you’re facing as you plan for scale, both what’s possible—and what’s responsible.

1. As AI grows more powerful, so do the costs and complexity. Tech leaders face a critical question: Where does AI deliver real value, and what change will that demand? It’s no longer just about technical know-how. It’s about aligning AI investments to business goals, anticipating tech debt and navigating shifts in SaaS and PaaS models. The latest hype cycle has caused a flood of AI tools and features, making it harder for teams to focus on what to use and why. To lead through this, you must become a value translator, connecting AI decisions to an end-to-end vision for scale.
2. Islands of data, islands of understanding. Data is the great unifier, if we allow it to be. But now, it often reinforces separation—different systems, different standards, different truths. Your success with AI hinges not just on access to data, but on the ability to integrate and interpret it across these divides. Otherwise, you’ll train powerful systems on partial perspectives. Technology and data leaders must lead with a purpose-built data strategy, one tailored for how the organization will differentiate itself with data and AI.
3. The autonomous agent in a world of ensemble performance. There’s a certain allure to autonomous AI agents—self-directed, tireless and efficient. But in practice, meaningful work is messy. It spans various systems, teams, roles and unexpected dependencies that developers must account for. The real challenge is rethinking how to support processes with context-aware, connected systems that are scalable through reusable components—not just optimized for isolated wins.
4. Governing the chaos of federated development. As generative AI use cases grow, so do the number of teams exploring them, which means tech leaders need to rethink centralized services that effectively guide federated development. The goal is not to control teams, but to help them build smarter. Shared guardrails, reusable components and simple paths to adoption make governed development the default.
5. Privacy, purpose and what employees value. As AI systems become more intimate, shaping everything from personal health to future employability, we face a deeper question: Not just can we scale these tools, but should we? Meaningful deployment must go beyond technical feasibility. It demands that compliance, security and privacy are treated as first principles alongside a renewed commitment to equity and shared outcomes, especially as AI reshapes the future of work. Trusted AI is essential for adoption, and tech leaders must be at the forefront of this challenge.

What scalable AI looks like in practice: A low-risk, high-value starting point

To spark new thinking, it helps to start with a high-impact example that’s easy to test and scale, starting with one role, the field rep and his user journey.

By approaching the solution from a user experience journey, we can begin with clear KPIs and value metrics in mind before taking any other steps.

In our example, we’ll build an agentic ecosystem that supports the rep’s journey, supporting six key dimensions of a rep’s success: target, practice and coaching—to identify the right providers and ensure effective engagement; research and plan—to uncover the best opportunities aligned with relationship goals; and finally, execute—to carry out the plan, collect critical feedback and manage follow-ups (see Figure 1).

In this case, generative AI works like a teammate. It helps reps sharpen skills, offers real-time coaching and lightens the load on administrative tasks. From targeting the right HCPs and recommending tailored messages to planning calls, summarizing notes and suggesting smart follow-ups, the system is set up to automate tasks for specific results.

FIGURE 1: AI agents in action: Personalized support across the rep user journey

Achieving scale with AI in real-world business environments requires different thinking

From a development standpoint, the goal isn’t simply to build software to enable the rep’s co-pilot, it’s to design a system that works in sync with a rep’s daily rhythm, not against it.

To get there, the architecture needs to be rethought from the ground up. That means embracing a few non-negotiable principles, built into every layer.

Below are five essential layers for scaled AI, along with a few “must-haves” that should shape each one.

FIGURE 2: Designing for scaled AI: 5 essential layers

These layers—infrastructure, data, applications, workflow integration and operations—each has a distinct role in enabling AI to scale seamlessly within real-world business environments, and each must be intentionally designed with specific capabilities in mind. Here’s a closer look at what defines each one and the must-haves that bring them to life.

Layer 1: Infrastructure

Today’s technology landscape demands an adaptable infrastructure that evolves in days or weeks—not months or years. Three principles are foundational here:

• Self-service environments: Ensure development environments can be spun-up quickly and reliably to support parallel workstreams across teams.
• Reusable component design: Infrastructure for generative AI must be built with reusability in mind—building services, tools and agents that can deliver value across different tech stacks.
• Balanced platform strategy: Standardize on one or two core platforms for depth and efficiency, while maintaining an interoperable ecosystem that supports two to three additional technologies for targeted experimentation—without creating new silos.

Layer 2: Data

While most organizations have a strong grip on structured data, unstructured, multimodal data—data that comes from multiple sources such as text, audio, video, imagery or sensors—presents the next major opportunity for differentiation. The must-haves here are:

• Multimodal data management: Establish a technical framework to manage and govern multimodal data. This framework will support strong metadata and quality management at its core.
• Integrated data products: Invest in capabilities that connect structured and unstructured data into integrated, reusable data products that can be consumed by multiple applications and support diverse business needs.

Layer 3: Applications

Business applications have historically been built for a single purpose. Now they need to do more—adapt, solve and scale in real time. Here, the priorities are:

• Reusable agents: Build modular, decision-enabling agents that can operate independently and be chained and reused across use cases to solve complex, multistep problems.
• Context-aware design: Shift application development away from static UX or tightly coupled APIs and toward polymorphic agents that are context aware and can dynamically shift based on intent. This move will reduce redundancy and increase maintainability.
• Unified co-pilot experiences: Consolidate function-specific co-pilots (such as sales, HR and marketing) into a unified, intelligent user experience that routes each request to the right place based on intent. This streamlines the user experience, improves answer quality, boosts adoption and helps avoid the sprawl of redundant co-pilots, which can ultimately lower the total cost of ownership.

Layer 4: Workflow integration

Today’s enterprise and SaaS ecosystems often rely on people to bridge fragmented processes. AI has the potential to ease that burden, but only if it’s implemented with empathy and intention.

To do this, you must center activities around:

• AI readiness: Prepare systems—including legacy data, tools and services—to be agent-ready. This may involve exploring open-source protocols like the Model Context Protocol (MCP) to enable seamless agent collaboration.
• Designing for your workforce: A design-thinking approach helps ensure gen AI and AI agents are embedded into business processes in ways that feel intuitive, supportive and meaningful—leading to higher adoption, better outcomes and a more empowered workforce.

Layer 5: Operations

Maintaining consistent quality at scale for AI systems requires significant investment in IT operations spanning people, data, systems, applications, risk, compliance and privacy. You should support investment in these areas by:

• Design-to-scale behaviors: Designing agents and apps with scaled operations in mind from day one means fewer surprises later. When you plan for scale up front, you’re not just coding, you’re mapping across use cases, markets and whatever comes next.
• Enterprise-level ops: To future-proof how the workloads run at scale, consider your ops org. Consolidating teams at the enterprise level can promote standardized execution, reduce noise and build systems that can evolve with enterprise goals.
• Life cycle cost transparency: Make the full cost of ownership—especially operational costs—visible and manageable from start to finish.
• Leverage out-of-the-box vendor capabilities: Build the capability to efficiently assess and use vendor-provided agents, like those from Salesforce, ServiceNow, Veeva, SAP and Oracle. This step will help you scale development without adding technical debt and overhead while keeping the total cost of ownership contained.

Scaled AI: Clearing the path starts now

Executives don’t need to design AI systems—but they do need to make scaling possible.

That means clearing the path so teams can focus, while reinforcing essential principles across every layer: infrastructure, data, applications, workflow integration and operations.

Lead by setting clear guardrails, aligning teams on shared goals and removing blockers that get in the way. When you own the standards and remove the friction, you create the conditions for AI to scale—by design, not by chance.

If you’d like a deep-dive discussion into how this works, contact your ZS team or contact us.