Maximizing Returns: The Consulting Playbook for AI ROI and the Enterprise

How can enterprises maximize their return on AI? 💰

Discover IBM’s strategic playbook at the Six Five Summit! Jason Kelley, GM & Managing Partner at IBM, joins host Patrick Moorhead as an Enterprise AI track speaker. They discuss navigating the complexities of AI and effective enterprise implementation for long-term success.

Key takeaways include:

🔹The Foundational Pillars of AI Success: Unpack the non-negotiable importance of clean, high-quality data and robust, scalable architecture as the bedrock for any successful AI endeavor.

🔹Conquering Legacy to Scale AI: Learn actionable strategies for overcoming the inherent challenges posed by legacy systems, enabling the seamless transition of pilot AI models to enterprise-wide scale.

🔹Responsible AI in Practice: The imperative of implementing comprehensive and responsible AI practices across the entire enterprise, ensuring ethical deployment and fostering trust.

🔹Strategic Risk Management for AI: Explore proactive strategies for effective risk management in AI deployment, safeguarding investments, and ensuring the reliability and integrity of AI-driven outcomes.

Learn more at IBM.

Watch the full video at Six Five Media, and be sure to subscribe to our YouTube channel, so you never miss an episode.

Or listen to the audio here:

Patrick Moorhead: The Six Five Summit is back, and unsurprisingly, the headline here is AI Unleashed and actually getting value for enterprises with AI. The buildup is fun, the tech is fun, but getting ROI out of it is everything, and I can't think of a better person to have this discussion here than Jason Kelley who is IBM's General Manager, Managing Partner of IBM Consulting. Welcome to the show, Jason.

Jason Kelley: Well, thanks very much. Good to be here, Pat, it's always good to have a conversation with you and I know the depth in which you look at and analyze the industry. There's always some subtle pressure to make sure that I'm bringing the best discussion I can to and with you.

Patrick Moorhead: Yeah. And Jason, my first slide, I'm probably going to do 10 CIO conferences, round tables presentations, and the number one bullet on my slide is ROI. And the ROI, you lose track of that ROI and things just start to fall apart. I love the tech. Okay. Infrastructure's cool. APIs are cool. MCP, A2A, it's wonderful. But it's like what are we getting out of this? You and your team are front and center with clients trying to figure this out. The first thing I noticed ... I was at the Sam Altman event two years ago out in Seattle. I was thinking, man, this data governance, it's going to be really, really hard. And I'm curious, how do consultants help companies prepare data for AI? It's hard enough to get the data right in the stovepipes of let's say HCM, it's a whole other thing to connect HCM to PLM, to all the different manufacturing systems. How are you helping them?

Jason Kelley: I'll start with your first bullet that you always have, which is the ROI. And I often hear the first ... We won't even go to the O and the I. R typically stands for regret. Once someone has started it and not done what you've just said, which is thought about first, the objective they want to go to. So I've got to throw this out there because I'm going to come back to the data. But it's the business outcome around AI. Not just AI for AI's sake, not just another science experiment. But instead, what is the outcome that the business wants to drive? And I will say that some people also confuse the phrase use case because a use case could be just one off and not a true outcome, which is usually the result of multiple use cases that come together to give a business outcome such as closing the books faster. And I'm saying it in layman's terms. Making sure that everything that shows up on the doc is exactly what we ordered and is at the right price. So I want to say that the outcomes first.

Then as you look back, everything should be by design. And so when we think of looking back, if this is the outcome, then let's do this outcome by design, starting with the most important part of this. Some would argue it's models, but I don't want to get there yet because before we were saying the word models, we were talking about data and data models. And so it is the data. And you called out some of the things that do first come up with regards to data. How are you integrating that data? And these are old challenges to you. We could hit all the latest technology. So you're on the right point. Data's at the root of it. The root of all good or evil. And in this case it's starting with data quality. You can never get around that. Make sure you have data quality that's there. The data integration. And data integration could be in those stovepipes. It could be in the form and format in which that data is. It could be structured, unstructured. All these things that people as we start saying, they go, "Oh yeah, that's right. We've said this before with other capabilities."

So I could go down that list and I could also start talking about making sure that the data and privacy, because a lot of people forget about the data privacy, but also security. We start talking about the security kicks in and that becomes the challenge and all the ethics that would flow around using that data for the AI. Now I've covered all that list. So if you just chuck- Then what I would also have to lean on, and I'm sure we're going to talk about this, is that that data is generated and stored in multiple places on-prem, off-prem, one cloud, multiple clouds. So now you have multiple clouds that could be out there, whether it's AWS, whether it's Microsoft, Azure, GCP, Red Hat, and you could have that across ... Then just as you say HCM. HCM. Is it both Oracle HCM and SuccessFactors and Workday? Many cases-

Patrick Moorhead: Probably in a big company the answer is yes.

Jason Kelley: Yes. You just saved me naming a whole of my other partners in this because they have these, they are the cause, creator and the good part of it, I don't want to call them ... But this is why pulling that data is hard because now you have to say, how can I work with my legacy investment? One of my good friends, Jimet says PTSD. And my quick shout out to all my veterans, I'm a veteran, and when I say PTSD, people think it's military. So shout out to all the vets and current serving members. PTSD is that debt that we get from process, from the technology, from the systems and the data. So that's what I and people remember that term, PTSD. But that's that debt that comes with all of those different applications. So short question, long answer, but it's where we're kicking this off here to say, look, there are multiple players in this, and the way to make it happen is first looking at the data in context of the outcome and then figuring out how you're going to orchestrate this by design to get to that outcome.

Patrick Moorhead: Yeah. The fractalization of applications and infrastructure meant the data went, the data got fractalized too so it's a huge challenge. Hey, I want to hit on something that ... It's funny, I've been in and around IBM going on 35 years. And architectures were always ... Because IBM was there to build things that were resilient.

Jason Kelley: I should jump in and say, you've been around IBM for 35 years and you suffer from Benjamin Button syndrome as I'm looking at you. It must be good for you so keep-

Patrick Moorhead: I don't know Jason, our birthdays are within a few months from each other. I've had dinner with you, you look pretty good my friend. But there's a lot of talk that says, hey, in this new wave that we're looking at, we need an architecture that doesn't just satisfy the next year to crank out some POCs. We need to scale. And then you hit on it. There's a lot of legacy systems that are involved. I always like to say technology never dies. It's just additive, right?

Jason Kelley: That's right. Yes.

Patrick Moorhead: Talk to me about do you help clients do plan, strategize, architect and AI architecture that has legs for five to X years?

Jason Kelley: The quick answer is yes. Though it's a bit of a loaded question in the way that you've asked me that because to say an AI architecture, if that's what one of our clients wants to call it, they can start there. And we always love to say, look, it's client first with a point of view. So if that's what they're going to call it, we can call it that. However, then my point of view with them would be you're looking at a business capability architecture because you just said it. Had I said it was going to be a services oriented architecture and then a blockchain architecture, and then it's a AI architecture, is it a different architecture? Well, quite frankly, it is. It has evolved. And so I would start with an open architecture and that would be one that would allow them to have that flexibility, elasticity in what they do. And that's why I did purposely use the words by design earlier.

I always caution because the thoughts come to mind, the words that come to mind are not just flexibility and elasticity, but also resilience and also sustainable. And I mean sustainable in both senses. When you say sustainable, now they think it's green. It's not just that. When the current CIO and next CIO are gone, are you still going to be celebrating the decision that you made a decade ago? Not saying that that's how often they turn over. But I will ... Shameless, by the way, if I had a bell, I'd ring it and say, shameless self-promotion. We have a CEO here at IBM, Arvind Krishna who said, "Hey, we are going to be a hybrid cloud AI company." And when he said it, to his credit, a lot of people it's like the dog hearing the siren. They're going to start turning their heads. "What does that mean, hybrid? That's like a car. Hybrid. What are you talking-".

So I do think this thought of building purposely open as well as flexible and elastic. And when I say elastic, I mean also elastic that's for energy consumption because we know that there's more compute power that's always going to be needed. So can we always turn it on and ratchet it down? And then also finally, there's two things that are often forgotten when we say architecture, because we say architecture, you and I Pat, our propeller heads start going and we go, "Oh, yeah. Okay. Yeah."

Patrick Moorhead: It's true.

Jason Kelley: But it's also skills. It's also are we planning where our skills are going to be and making sure that we're skilling towards that architecture so there is a business architecture and capability architecture? In then in addition to skills, it fits the sustainable thing. But this change management that always gets kicked in and forgotten about when we start talking about an architecture because it's like, "Oh, they're developers. Just throw pizza at them and some caffeine and caffeinated drinks and they'll crank this out. Little AI to help them change the code. It'll be good." But no, I do think that those are all the things that are by design based on your question.

Patrick Moorhead: Yeah. Hey, I do want to get a little nerdy here if you don't mind. Models are fundamental to the technology. They're not the most important thing. I think in IBM, I give credit to IBM. They were the first company to come out and talk about multi model, multi-vendor models. It just made sense. You don't want one or two companies controlling every model. And how on earth can one company have control over a model that does best on CRM or SAP or connecting all these together? And oh, by the way, last time I checked, enterprises aren't too thrilled about rolling in a thousand kilowatt rack to do everything. Okay. You don't have to throw the kitchen sink at every single type of workflow here. So what does differentiate a successful AI model from a production ready AI model? One that can scale, one that can lead to ROI as opposed to, Hey, we had a great pilot, great science project, what are the characteristics to scale at enterprise level or I like to say planet scale?

Jason Kelley: I think that you've helped me answer the question in the way that you've asked it because it is going beyond a single model. If it's an open model and we of course open up our granite model is an open model, and you look at that and you say, "Well, why?" We go, "Well, you want to make sure that you have the flexibility. There's also some sense of the ethics around being able to have transparency and understand that there's not the bias and the things that you would think about inference, but where's the data coming from and are we using the data that we need?" And as we started to learn smaller models for more focused outcomes. There's less latency, there's less energy consumption. You get what you want, when you need it quickly. All of those things that you could say. So I'm saying back to you this thought of multiple models based on the outcome. And if you're planning that outcome correctly, then that will help guide which model.

I do and will continue to repeat myself on this thought of what are you trying to get, and you said it yourself. Are you trying to solve the latency of product shipping in your supply chain? Are you trying to make sure that you have rich images for your marketing capability? Are you trying to make sure that you have the right and very timely ... Right with air quotes around it. And very timely data for customer loyalty? So then you start saying, "Am I going to use another Copilot? Am I going to use Firefly? Will I tie into Joule? Will I use that with WatsonX? What am I doing?" So I think as I said, you asked the question in a way that it really points at it. It starts with understanding that outcome and then knowing that, okay, I can use multiple models based on what I'm trying to get to.

Patrick Moorhead: Yeah. The other top 10 list on keeping enterprises from scaling AI as much as they would like to, it's really where it started. It's responsible AI, and this word can mean multiple things to many people. For some people it's the model that gave me accurate data. Another one says that it doesn't have biases. There's regulatory scrutiny that based on certain regions and countries are different. And I'm curious, how do you help clients implement these responsible AI practices through which you're doing in consulting?

Jason Kelley: So I have the joy of saying that we've been doing this for more than a decade. And sometimes you can get a small spark when you say, yeah, you started beating Watson or Watson started beating Jeopardy back in the day. Okay, well great. You won in a game show, but it's the game of business. What'd you do after that? And I tell you that those learnings that we had way back helped us understand how to use AI responsibly. I will tell you that back then we started with AI code of ethics and we said first that augmented intelligence, artificial intelligence. And we would say augmented often because it's there to augment human intelligence not to replace it. And that was our first tenant. And our second was that the data and insights belong to the people or those things that created it. So those companies, those entities that created it.

And then finally to make sure that the AI that we were using along with the data was transparent and explainable, that it could be questioned, so it wasn't black box. So we definitely start with those basics that we have been doing for more than a decade now and say that, yes, we want to make sure that clients understand what responsible AI means, and it does dial back to where we started with the data and then progresses to the models. Because now as we are thinking and talking models, what are those guardrails that you want to put in to make sure that you continue to implement and enforce those standards and those values that you started with more than a decade ago? That includes as the inferences that you're getting out of those models and where that's coming from based on the data and the data that you're using to gather the inference from and how you're training the models. We keep going. It's really taking what we've done before and advancing that into the current, to your point earlier, the current technology and staying consistent because it worked then and we do believe as we tell our client, it works now, but you have to start there.

Patrick Moorhead: By the way, good reminder to everybody, AI, in fact, first AI algorithms were done in the 1960s. We were doing AI with analytics and then we did AI with machine learning, and then now we're doing it with LLM and through machine learning, but throughout that requires it to do it in a responsible manner. Last question here. What is the role that consultants should play to help clients anticipate and manage the risk of something going wrong? Because you could put the right ROI architecture, you can have the right data, you can have the most responsible AI practice, but there's going to be corner cases, things are going to happen. These models, quite frankly, are learning how to make sure they don't get shut down or shut off. There's vibe coding that is injecting. Is about the riskiest of code from a security point of view. What role do consultants play in this game?

Jason Kelley: So I think that the best way to answer is to get rid of the word consultant. And I would say, what role does a partner play? How can you be that partner for your client? Because, and even we could use the other role. You can say consultant, you can say developer, you could say salesperson. And I think that we're predisposed to think of what those roles are, and I want to pull it apart. And I just say, look, what we talk to our teams about is if you're going to partner with that client, you're going to tell and not sell. You're going to tell them what they should be considering. You're going to tell them, just as I've spoken to you about the design and the considerations. You're going to tell them about their legacy and where they want to go, and you're going to tell them that it should be the outcome.

And so I think it's less in the moment. I know as you asked the question, we could say, how does that apply to AI? But I have to go back. I think it's simpler than that. It really is. Easy to say, but becomes even more complicated to do with all of the moving parts. So I think that driving this thought of disciplined partnership and looking across an agentic ecosystem .... And so now I will get to the answer to the question is that I think that that agentic ecosystem, when I say it, people think multi-agent interaction. And I don't mean just that. I mean what's behind that, what's the ecosystem behind that, which does include the data sets, that does include the architecture, that does include everything that we have just described. So I'm glad you said this, the last question. It really pulls in that fact that it is an agentic ecosystem of multiple players, of architectures, of models of data and outcomes that we have to drive toward. And that's what we ask our team members to do, is to partner with our clients to make sure that they cover all of that. And at that point, you can build through very trusted communication the outcome for that client.

Patrick Moorhead: Yeah, Jason, I hear a lot of companies throwing around the partner moniker. It's where people want to go. I've been in and around IBM long enough to know that it is a reality for you. Some of your customers have been doing business with you for 50 years. And there's a lot of trust that goes into this. A lot of the core applications are some of the most sensitive and risk tolerant that are worked on. So I don't view it as a throwaway line, even though everybody uses it. I know you and I know your company.

Jason Kelley: I'm glad you're saying it so that anyone listening to this going, okay, Jason, you just pull it like, "Oh, I want be a partner." And I almost have to say, yeah, it's kind of cliche, but it really isn't. Now if you want to get really cliche since you've been around, you know that we had three simple values and it was part of a session back when it was a big deal for everybody to get online and communicate. We called them Jams. That was 2003. Someone's going to fact check me. We said, "What are our values?" And the one that first one is innovation that matters for our company and for the world, but on this point that you're talking about it it's dedication to every client's success. So that's the first thing. And then you said it best when you said, what do we do? Well, we have trust and responsibility in all of our relationships. And that's what this partner means. This means that I'm going to partner with my partners that are in there so when it's a Salesforce SAP on Azure, with all of the AI that would come with that, and in Adobe for Firefly, you'd say, "Yeah. Okay. I'm going to partner with them with trust and communication." And then we, we as IBM, would orchestrate that relationship for our client, we would be the trusted partner with our client.

Patrick Moorhead: No, I appreciate you saying that. I love that final piece there. Jason, man, thank you so much. I really appreciate it. It's a great session. I'm sure people got a lot out of it, so thank you for your time.

Jason Kelley: Hey, anytime Pat. Time flies when I'm with you. So you just give me a call and we'll do it again whenever you're ready. Take care.

Patrick Moorhead: Thank you so much. So thanks for joining us here at the Enterprise AI Spotlight for the Six Five Summit. We're here in our sixth year. Pretty excited about that. Stay connected with us on our website, and we're going to have more conversations, more insights coming up. Stick around.

Disclaimer: The Six Five Summit is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

Speaker

Jason Kelley
GM & Managing Partner, IBM Consulting
IBM

Jason Kelley is the Global Strategic Partnerships and GSI Executive is responsible for the development and execution of our relationships with our network of Global Systems Integrators and Strategic Partnerships including Kyndryl, Microsoft, AWS, Adobe, Salesforce and SAP.

Jason Kelley
GM & Managing Partner, IBM Consulting