Companies that educate, explore, experiment, and expand, perpetually, with the right pace and sequencing, are most likely to win with AI
This article was originally published on CIO.com
AI never sleeps. With every new claim that AI will be the biggest technological breakthrough since the internet, CIOs feel the pressure mount. For every new headline, they face a dozen new questions. Some are basic: What is generative AI? Others are more consequential: How do we diffuse AI through every dimension of our business?
Tactically, you can answer these questions in any number of ways. You can build an AI Center of Excellence (COE), launch a strategic task force, or designate a deputy to lead the charge. But whatever you do—if our advisory work and discussions with leading CIOs suggest anything—you’ll have to drive excellence in four related, though not necessarily sequential, streams of work: Educate, Explore, Experiment, Expand. It’s around these four work streams that leading organizations are positioning themselves to mature their data strategies and, in doing so, answer not only today’s AI questions but tomorrow’s.
Educate. You can’t wrangle AI by yourself. Your journey will be fruitful only to the extent that you can instill in those with whom you go to market a digital fluency and a confidence in your ecosystem.
Accordingly, many CIOs have fashioned themselves into the de facto AI professor within their organizations—developing 101 materials and conducting roadshows to build awareness, explain how generative AI differs from other types, and discuss its risks.
To ease collaboration on the topic where it’s likely to surface, Digi-key Electronics, a leading electronic component distributor in North America, has even built networks of influencers. As the company’s CIO, Ramesh Babu, explains, “We identify ambassadors in the organization and position them in the right meetings to drive a common understanding of the many terms floating around.”
Babu also warns against discussing only the benefits of AI. He and his peers make a point of emphasizing the risks. “We’re trying to have balanced conversations,” he says, a practice that underscores the duty CIOs have to develop appropriate policies and usage guidelines in order to mitigate the downsides of AI.
To help educate your own workforce about AI, provide them materials on the topic. Include common definitions, reimagined future states, risks, and policies and guidelines for usage. Have them ready for impromptu meetings, town hall presentations, and other settings. And direct your colleagues to self-service channels so that they may access materials and learn at their own pace.
Explore. To explore is to pose the question: How can I make AI work for my organization?Since the AI landscape is both large and complex, take a two-pronged approach: analyze internally and marry that analysis to marketplace activity.
Internally, start by looking at your value chain or the capabilities that deliver your value proposition. Brainstorm how generative AI could make your processes (and the people supporting those processes) more intelligent and productive. If you’re already using AI for some of the use-cases you brainstorm, no matter – record those too. And pay special attention to use-cases that concern customer service: Of the executives surveyed at the latest Metis Strategy Digital Symposium, 43% said their organizations are prioritizing customer service use-cases for generative AI in 2023.
From all of these sources, compile your use-cases into a backlog and rank them by impact and feasibility. You’ll learn where you can create new ways to win in both the short and long terms while weeding out those cases that are too difficult for their value.
Next, examine the market. At first, you might struggle to wrap your head around the size of it—a $150B addressable market, as estimated by Goldman Sachs—but by doing so you set in motion what should be a continuous evaluation. Search first for vertical-specific and enterprise-wide AI solutions. Categorize them by the capabilities they support. And if your organization permits it, maybe even ask ChatGPT.
Compare and contrast what’s available in the market to your top-ranked use cases and the capabilities you already have. Where an internal capability does not already exist, and the case relies on a large language model (LLM), you will need to determine how you want to proceed: by training and fine-tuning an off-the-shelf model, like Morgan Stanley did with OpenAI; or by building your own, like Bloomberg did.
Experiment. To experiment well is to work through your backlog with urgency and agility and—especially in the case of AI—with a bias for incremental progress. As Baker Tilly CIO Allen Smith explained at a recent panel, “There’s a difference between home runs and singles.” The singles are your friends, says Smith, and a great way to show something tangible, build momentum, and create a vehicle to fuel other interesting ideas.
At the tech juggernaut Lenovo, CIO Art Hu is taking a similar approach. Hu says they are running dozens of proofs of concept. One consequence of being in the early innings of Generative AI, according to Hu, is the rapid pace of development. “Because it’s fast, you can run proofs of concept for not massive investments.” This demonstrates how his team stays in lockstep with the business on investment priorities in a period where economic uncertainty has narrowed the scope of technology investment. “That’s the way you want it. You want small steps for the business without spending or committing a lot of money. They can see the result and decide ‘OK, double down, or shift the investment elsewhere.’”
Many attribute generative AI’s promise to its ascent to the very top of the tech stack, a promise that makes it more approachable than other disruptive technologies that, while undeniably promising, still require technical expertise to be exploited. Acknowledging this nuance, many companies have built experimentation sandboxes in which users from across the organization can try their hand at AI in a controlled environment.
Expand. Research reports have dangled that generative AI could add trillions to the global economy. But generally, these reports assume that AI can be implemented at scale. Here, AI leaps from the Batcave to the streets of Gotham, confronting a new set of challenges.
With regard to creating that scale, Chris Davis, a Partner at the digital advisory Metis Strategy and a leader of his firm’s AI practice, worries less about scaling the technology than he does about people’s role in that scale. “Someone has to develop, train, and supervise the models,” he explains. “…the irony is that people could actually be the limiting factor.”
As a means of overcoming this limitation, he stresses how necessary it is that organizations revisit—and where appropriate, revise—their operating models. “You need to re-envision business strategies with the exponential scale of AI in mind,” he says. “And train product managers on how they might weave AI into anything—core digital products, customer experiences, employee experiences, and so on.” He goes on to explain, that means also ironing out the roles and responsibilities among various players in your organization: “AI laboratories, data scientists, product teams—they all have to know how to work together efficiently every step of the way, from identifying use-cases to building algorithms and models, from following AI operating procedures to monitoring any models that are already in use.”
And there’s plenty of evidence to support Davis’s point. For example, after recently redefining the roles, responsibilities, and delivery methods of its IT product teams to suit its specific AI ambitions, a global financial services provider discovered many gaps in its capacity: some that it could address through upskilling, but also some that would require it to hire new people.
Looking forward. Meanwhile, hyperbolic headlines will continue to outpace adoption; yet, they won’t outpace the exponential rate at which the volume of data is growing, especially as technologies such as 5G and IoT hit their stride. So, if you, too, want to leverage AI to its fullest extent, you must first look in the mirror: Can I manage this growing volume of data? If you can’t convert the data into something meaningful, then, as Lenovo’s tech chief, Art Hu, suggests, you might lose ground: “If you don’t figure out as a company how to (manage a growing volume of data) effectively and efficiently, the competitor that does is potentially going to have a significant advantage.”
As you mature your data strategy, remember that you have many data-driven tools at your disposal, only one of which is AI. It’s wedged between an ocean of use-cases to the North and your core data foundation to the South, and progress in each of these layers is linked to the other two inextricably. There’s no use in thinking of your data strategy as something binary, as if it were a building under construction that will one day be complete. Those that educate, explore, experiment, and expand, perpetually, with the right pace and sequencing, are those most likely to win with AI.
Mike Bertha is a Partner at Metis Strategy
Chris Boyd co-wrote this article.
Leading digital transformations is the CEO’s top priority for CIOs, according to the 2020 IDG State of the CIO study. Doing so effectively requires an IT operating model that allows business and IT to work together to navigate a dynamic competitive landscape, a seemingly infinite set of digital tools and shifting stakeholder demands.
In our work with Fortune 500 companies, we have found IT organizations that use the traditional “plan, build, run” operating model struggle to conceptualize, launch and maintain momentum on digital transformations. To bolster their transformation capability, IT organizations across industries and geographies are shifting toward product-oriented operating models, or “product-based IT”. When done right, organizations experience increased agility, happier customers and more successful transformations.
A product is a capability brought to life through technology, business process and customer experience that creates a continuous value stream. Examples of products are eCommerce, supply chain, or HR. An operating model defines how an organization positions its people, process and technology to deliver value to both internal and external customers.
A product-oriented operating model, then, is one in which IT resources are organized around business capabilities or “products” instead of specific IT systems (e.g. SAP, CRM) or functions (QA, Engineering, Infrastructure). In this model, each product team works as if they are managing a market-facing product such as a consumer electronics device. They develop a product strategy and roadmap in lockstep with the business that clearly articulates how they will mature the product to better meet customer needs and optimize competitive positioning. Every feature on the roadmap is aligned with a measurable business outcome and goes through a rapid discovery phase to validate value, usability and feasibility before it is slotted in a sprint to achieve a minimum viable product.
Most organizations have honed their ability to deliver when the scope and desired outcome are static, but struggle when next steps aren’t defined or are painted with a broad brush. Several leading IT organizations have turned to product-based IT to cut through this ambiguity and elevate their role from service provider to business partner.
Art Hu, the global CIO of Lenovo, is one of the pioneers in the shift from project-to-product. He noted in a recent conversation that his organization grappled with the question of what to work on next after completing a series of legacy ERP integrations resulting from acquisitions. “The fundamental paradigm shift for us was that the level of uncertainty had changed when there was no longer one single imperative,” he said, referring to the ERP project. “When we took that away, it was a totally different world and traditional waterfall didn’t make sense anymore. Until we as an organization realized that, the business teams and my teams struggled.” Product-based organizations rely on continuous customer engagement to remove guesswork from the prioritization process, which often leads to better business outcomes and increased agility.
CIOs have targeted key behavioral changes to jumpstart the shift to a product-based operating model:
Project plans developed with fixed deliverables and timelines encourage predictability but rarely equate to business outcomes. This plan-driven work is increasingly yielding to continuous discovery and delivery, which seeks to answer two questions on a recurring basis: what should we build, and how should we build it? A discovery track intakes opportunities, ideas and problems to solve. Teams then engage with customers to validate that those ideas create value (desirability), will be used once released (usability) and are feasible in the current business model (feasibility/viability).
“Great companies that have built a product orientation start with desirability and leverage design thinking to have empathy-based conversations to get to the core of problems,” Srini Koushik, the CIO/CTO of Magellan Health, said during a recent product management panel. Ideas that make it through discovery are added to a product backlog and are slotted into sprints for delivery based on relative business priority. Discovery and delivery tracks operate concurrently to ensure that a steady stream of validated ideas and a working product that drives business outcomes is delivered at the end of each sprint.
In a recent strategic planning session, one CIO stated that “transformation is not a part time job,” noting that dedicated teams are critical for both digital transformation and building a product orientation in IT. Teams that are formed on a project-by-project basis spend valuable time ramping up subject matter expertise and building chemistry, but then are disbanded just when they start to hit their stride. Product-based IT organizations, on the other hand, favor dedicated teams that own a product from introduction until sunset, including the execution of discovery, delivery, testing and maintenance/support. In this model, the dedicated teams become true experts on the domain and avoid pitfalls resulting from intraorganizational handoffs and revolving resources.
The increased frequency and quality of customer interactions is a hallmark of product-based IT. Ideally, customers are engaged during the discovery phase to validate ideas and prototypes, and then provide feedback at regular intervals after the product is released. If your end customer is a business unit, you should strive to have even more interactions. Some organizations have business stakeholders participate in daily stand ups, and some may even have their product owners sourced directly from the business instead of IT.
Atticus Tysen, the Chief Information Security, Anti-Fraud and Information Officer at Intuit, is another pioneer in the shift from project to product. At the 2019 Metis Strategy Summit, he emphasized that true product organizations reflect on key questions that demonstrate their strong relationships with customers. For example, do you really know who your customers are, and are you organized around serving them? Do you have metrics to measure customer happiness and show you are working with them in the correct way? “You have to have customers if you’re going to have a product organization,” Tysen said. “Product managers in a lot of ways are relationship managers.”
To achieve the benefits of a product-centric operating model, the funding model must shift as well. Rather than funding a project for a specific amount of time based on estimated requirements, teams instead are funded on an annual basis. Also known as perpetual funding, this setup provides IT product teams with stable funding that can be reallocated as the needs of the business change. It also allows teams to spend time reducing technical debt or improving internal processes as they see fit, which can improve productivity and quality in the long run.
Here are a few key steps to begin the journey…
Organizations should first and foremost target business impact when shifting to product-based IT. For example, one Fortune 500 client chose to measure Net Promotor Score to assess business satisfaction, product team velocity to assess speed to market and the number of critical defects per product to assess quality. It is also prudent to create metrics that track the adoption of key aspects of the working model. For example, you may track the percentage of product teams that have developed strategic roadmaps, or survey product teams on a regular basis to see how many feel like they have the skills needed to succeed in the product-based operating model.
Start by identifying the highest-level customer-facing and internal capabilities in the organization, such as Product Development, Sales, Marketing, Supply Chain, HR and Finance. At the highest level, these are your Product Groups, or “Level 1.” If your organization is smaller and has a relatively simple technical estate, you may not need to break this down any further.
However, we have found most enterprises with multiple business units and geographies need to do so. Inside the Sales group at a SaaS company, business processes would likely include steps such as Discovery, Lead to Cash and Customer Success (which includes activation, adoption, expansion and renewals). These may become your product groups since each of these steps involves different business stakeholders, targeted KPIs and technology components. However, the way you design your product teams will ultimately depend on the intricacies of your organization.
Absent a one-size-fits-all approach, we suggest the following guiding principles:
A key to product-based IT is building cross-functional teams that have the business and technical skills needed to accomplish most tasks inside their teams. The most important role in your product team will be the Product Owner (or Product Manager). Referred to as unicorns by some, these individuals possess the unique blend of business (strategy, competitive analysis), technical (architectural vision, technical project management) and leadership (decision-making, stakeholder management) skills and are responsible for driving the product vision and strategy and leading execution.
To fill this role, many organizations will conduct a skill assessment with their organizations to determine the skills needed to be successful, gather an inventory of available skill sets and shape a training program to fill gaps. As you structure the rest of the product team, think about how the skills of other team members can complement the product owner skill set so you are creating a strong blend of business, technical and leadership skills in the team. Beyond the Product Owner, you may have a Business Analyst that serves as a Junior Product Owner and supports detailed data and process analysis. A Scrum Master would drive Agile ceremonies, a Technical Lead would create a solution architecture and orchestrate technical activities, and an Engineering/QA team would ensure delivery of a high-quality product.
Think about IT services that are BU agnostic, needed across all product teams and in demand only on a part-time basis by the product group. These are your Shared Services. Shared Services cut horizontally across the product groups and teams. Just like products, these specialized groups endeavor to mature and develop new capabilities and empower their customers (in this case the product teams themselves).
Typical Shared Service groups include Enterprise Architecture, Infrastructure & Cloud, Security, DevOps, Customer/User Experience, Data & Analytics, Integration, Program/Vendor Management and IT Operations/Support. The Office of the CIO is an increasingly prevalent Shared Service that is responsible for defining the enterprise IT strategy, setting metrics and measuring success. Each Shared Service should publish a service catalog detailing their offerings and processes for engagement with a bias towards self-service (where possible). Shared service resources can be “loaned” to product teams if there is demand for an extended period.
IT often starts with feasibility and viability, approaching desirability only if the former two boxes are checked. Product managers need to start with desirability and build the ability to adapt their storyline based on the audience. Avoiding technical speak and endless strings of three letter acronyms will also go far in building this rapport.
Shifting to product-based IT is a major cultural and operational change. When done well, it can result in better relationships with customers and business partners, increased agility and improved business outcomes.
This article originally appeared on CIO.com. Chris Boyd co-authored the piece.
As technology departments shift from traditional project management frameworks to treating IT as a product, it is triggering a broader re-think about how technology initiatives are funded.
Under the existing “plan, build, run” model, a business unit starts by sending project requirements to IT. The IT team then estimates the project costs, works with the business to agree on a budget, and gets to work.
This setup has several flaws that hamper agility and cause headaches for all involved. Cost estimates often occur before the scope of the project is truly evaluated and understood, and any variations in the plan are subject to an arduous change control process. What’s more, funding for these projects usually is locked in for the fiscal year, regardless of shifting enterprise priorities or changing market dynamics.
To achieve the benefits of a product-centric operating model, the funding model must shift as well. Rather than funding a project for a specific amount of time based on estimated requirements, teams instead are funded on an annual basis (also known as “perpetual funding”). This provides IT product teams with stable funding that can be reallocated as the needs of the business change. It also allows teams to spend time reducing technical debt or improving internal processes as they see fit, improving productivity and quality in the long run.
“We have to adapt with governance, with spending models, with prioritization,” Intuit CIO Atticus Tysen said during a 2019 panel discussion. “The days of fixing the budget at the beginning of the year and then diligently forging ahead and delivering it with business cases are over. That’s very out of date.”
Business unit leaders may be skeptical at first glance: why pay upfront for more services than we know we need right now? A closer look reveals that this model often delivers more value to the business per dollar spent. For example:
Shifting away from old ways and adapting a new funding model can seem like a daunting task, but you can get started by taking the following first steps:
First, establish the baseline to which you will measure the funding shift’s effectiveness. A technology leader must consider all the dimensions of service that will improve when making the shift. Two areas of improvement that have high business impact are service quality and price. To establish the baseline for service quality, it is important to measure things like cycle time, defects, net promotor score, and critical business metrics that are heavily influenced by IT solutions.
The price baseline is a little more difficult to establish. The most straightforward way we have found to do this is to look at the projects completed in the last fiscal year and tally the resources it took to complete them. Start with a breakdown of team members’ total compensation (salary plus benefits), add overhead (cost of hardware/software per employee, licenses, etc.) and then communicate that in terms of business value delivered. For example, “project A cost $1.2M using 6 FTE and improved sales associates productivity by 10%”. When phrased this way, your audience will have a clear picture of what was delivered and how much it cost. This clear baseline of cost per business outcome delivered will serve as a helpful comparison when you shift to perpetual funding and need to demonstrate the impact.
The shift to a new funding model will be highly visible to all business leaders. To create the greatest chance of success, focus on selecting the right teams to trial the shift. The best candidates for early adoption are high-performing teams that know their roles in the product operating model, have strong credibility with business unit stakeholders, and experience continuous demand.
In our work with large organizations piloting this shift, e-commerce teams often fit the mold because they have a clear business stakeholder and have developed the skills and relationships needed to succeed in a product-based model. Customer success teams with direct influence on the growth and longevity of recurring revenue streams are also strong candidates as their solutions (such as customer portals and knowledge bases) directly influence the degree to which a customer adopts, expands, and renews a subscription product.
Estimation in the product-based funding model is different than in the project model. Under the new model, teams are funded annually (or another agreed-upon funding cycle) by business units. As funding shifts to an annual basis, so should cost estimation. Rather than scoping the price of a project and then building a temporary team to execute it (and then disbanding after execution), leaders should determine the size and price of the team that will be needed to support anticipated demand for the year, and then direct that team to initiate an ongoing dialogue with the business to continuously prioritize targeted business outcomes.
When completing a team-based cost estimation, it is important to include the same cost elements ( salary, benefits, hardware, licenses, etc.) that were used to establish your baseline so that you are comparing apples to apples when demonstrating the ROI of product-based funding. Where you will see a difference in the team-based model is resource capacity needed to deliver demand. In a product model, a cross-functional team is perpetually dedicated to a business domain, and there is often zero ramp-up time to acquire needed business and technical knowledge.
Since the teams have been perpetually dedicated to the domain, they are encouraged to take a longitudinal view of the technology estate and are able to quickly identify and make use of reusable components such as APIs and microservices, significantly improving time to market. For these reasons, among others, teams in the product-based operating model with perpetual funding can achieve more business value for less cost.
Pilot teams should work closely with the BU leadership providing the funding. Stakeholders should work together to generate a list of quantitative and qualitative business outcomes for the year (or other funding cycle) that also satisfy any requirements for existing funding processes operating on “project by project” basis.
If you don’t already have a great relationship with finance, start working on it now. Your partnership with finance at the corporate and BU level will be critical to executing your pilot and paving the way to wider enterprise adoption of team-based funding models. Ideally. Leaders should engage with finance before, during, and after the team-based funding model so that everyone is in lockstep with you throughout the pilot. This alignment can help bolster adoption with other areas of the enterprise.
Each finance department has unique processes, cultures, and relationships with IT, so while you will need to tailor your approach, you should broach the following topics:
You will need to achieve success in the pilot to bolster adoption in other areas of the business. Your success needs to be communicated in terms that resonate with the business. As your pilot comes to an end, gather your baseline data and match it up with the results of your pilot. Put together a “roadshow deck” to show a side-by-side comparison of costs, resources, and business outcomes (Business KPIs, quality metrics, cycle times, NPS, etc.) before and after the shift to team-based funding.
Depending on your organization, it may be prudent to include other observations such as the number of change control meetings required under each funding model, indicators of team morale, and other qualitative benefits such as flexibility. Have conversations with other areas of the business that may benefit from team-based funding (start off with 1-on-1 meetings) and offer to bring in your partners from finance and the product teams as the discussion evolves. The most important part of your story is that the team-based funding model delivers more business impact at a lower cost than the old model.
Establish light and flexible governance mechanisms to monitor performance of the teams operating in the teams-based model. The purpose of these mechanisms is to validate that the increased level of autonomy is leading to high-priority business outcomes, not to review progress on design specs or other paper-based milestones. A $40B global manufacturing client adopting the team-based funding model established quarterly portfolio reviews with BU leadership and the CIO to review results. BU leadership reviews results of the teams and the planned roadmap for the subsequent quarter. BU leadership is then given the opportunity to reallocate investment based on changing business needs or can recommend the team proceed as planned.
It is important to communicate that this process requires constant buy-in from business units. While funds will be allocated annually, demand will need to be analyzed and projected on at least a quarterly basis, and funds should be reallocated accordingly. In cases where investments need to be altered in the middle of a fiscal year, it is important to note that the unit of growth in this model is a new cross-functional team focused on a targeted set of business outcomes. The idea is to create several high-performing, longstanding, cross-functional teams that have the resources needed to achieve targeted business outcomes, rather than throw additional contracted developers at teams as new scope is introduced.
Making the shift from project-based funding to product team-based funding is a major cultural and operational change that requires patience and a willingness to iterate over time. When executed successfully, CIOs often have closer relationships with their business partners, as well as less expensive, more efficient ways to deliver higher-quality products.
