Art of Procurement: Managing the New Spend Frontier – AI Tokens in Procurement

How can procurement stay ahead of skyrocketing enterprise AI spend? Philip Ideson, host of the Art of Procurement podcast, spoke with NPI CEO Jon Winsett about the challenges of managing AI spend, how AI consumption is reshaping enterprise IT budgets, and what procurement teams must do to take control. 

Transcript

[Philip Ideson] (0:05 – 1:00)

Hi there, my name is Philip Ideson. I want to welcome you to the Art of Procurement podcast, the podcast that helps you, a forward-thinking procurement professional, position yourself and your team to proactively take advantage of the revolution that’s taking place in procurement today. By interviewing industry trailblazers and sharing insights from our own experiences, my team and I pull back the curtain and shine a light on the strategies, tactics, and tools that procurement teams are using to elevate their impact.

And today on the show, I’m delighted to be joined by Jon Windsor. Jon is the CEO of NPI, an advisory firm focused primarily on IT spend, which now includes all the costs associated with using AI. Now, I invited Jon on the show to talk all about the management of what’s really a new category of spend for procurement, and that’s AI tokens.

So can’t wait to get into the conversation. First of all, Jon, thanks for joining me on the show today.

[Jon Winsett] (1:00 – 1:03)

Yeah, good morning, Phil. Thank you for having me.

[Philip Ideson] (1:03 – 1:09)

Yeah. And the first question I have to pose, as I always do, which is, did you find procurement or did procurement find you?

[Jon Winsett] (1:10 – 1:53)

You know, I asked that question as well, because I’m genuinely curious. You know, I don’t know who found who, but we found each other. You know, my first job out of college was selling IBM hardware, mainframes and such.

So I was in tech, and this goes back many years before tech was cool. And my last job before we started NPI, I was SVP of sales for a publicly traded software company. So that was purely on the sales side.

Then when we started NPI 23 years ago, it landed us squarely in the IT buying space, which leads us to procurement.

[Philip Ideson] (1:53 – 1:58)

Yeah. And 23 years, it’s been, I guess that that time has flown by.

[Jon Winsett] (1:59 – 2:09)

It really has. I mean, it blows me away that it’s been that long. You know, it was 2003.

I mean, the world was different, right? It was June 30th, 2003.

[Philip Ideson] (2:12 – 2:38)

So 23 years ago, we weren’t talking about AI tokens, which, you know, I’m very in fact, we were not really talking about AI tokens probably 12 months ago or even six months ago. So I’m really interested to kind of dig down into this topic. And can you kind of just for context, just help start us off by explaining a little bit about what an AI token is and kind of pricing models around AI tokens.

[Jon Winsett] (2:38 – 3:08)

Yeah. You know, a token is really text. It’s the amount of text that goes into the input where AI reads it, the model reads it, and then the model generates an output.

And the amount of text that comes out of that is a measure of a token. The clearest, you know, correlation to how much text for each token is about three quarters of a word. Random, I know, but that is the measure.

[Philip Ideson] (3:09 – 3:20)

Yeah. And so how, how, when you are consuming these tokens, what are the pricing models? Like, how are we actually buying them?

[Jon Winsett] (3:21 – 3:52)

The most common is consumption based. So, you know, how many tokens are you using? You know, the frontier models like OpenAI and Anthropic did start out with seek based and then they moved to hybrid and now they’re going full consumption.

And so they’re measuring how many tokens are being consumed in each task request. And so you put in a prompt, you get back a response. That’s so many tokens.

[Philip Ideson] (3:52 – 4:15)

Right. And how quickly is this spend category growing and changing? I said, we haven’t, you know, we barely talked about this six months ago.

And it just seems like it’s becoming exponentially more important, both to organizations and how they manage their budgets, but then therefore to procurement as well.

[Jon Winsett] (4:16 – 5:30)

Well, you know, if you look at a graph starting in January of 2025, it has grown 13 fold in that less than a year and a half. And so and it’s a hockey stick. And, you know, you rarely see that steep of a hockey stick in business, but you do here.

And the reason is that the token consumption is being driven from agentic AI. So it’s different consumption than if you’re just chatting with a chatbot like we do as consumers, you know, as we ask questions and get a response, call that, you know, some measure of tokens. But with agentic AI, it’s prompted, it processes, it returns a result and then it runs it again.

So and it compounds on each other. So picture an agentic process running 50 different turns and each one bigger than the last. And that’s where you get this, you know, 30 X more tokens consumed in that type of process than a standard chatbot.

And that’s really what kicked off the spike.

[Philip Ideson] (5:31 – 5:38)

And why the frontier models went to a consumption based model rather than more of a seat based model?

[Jon Winsett] (5:38 – 6:28)

Well, they had to. And, you know, it should be said that no one is making money in AI, not the frontier models, not the hyperscalers, not the software companies that embed, you know, open AI or the anthropic within their offerings. You know, you think of Microsoft Copilot.

Yeah, they’re not making any money off that. The only sector making money in AI is the chips and MIPS. Yeah.

So think of semiconductors and hardware. They’re providing the capacity to compute for tokens. So you could call them token sellers.

And the buyers of tokens are the frontier models and the hyperscalers. And buyers of tokens aren’t making money. It’s the sellers.

[Philip Ideson] (6:29 – 6:36)

It’s interesting. So it’s like the hardware providers, the token manufacturers of the AI world.

[Jon Winsett] (6:37 – 6:37)

Yeah.

[Philip Ideson] (6:38 – 6:46)

And I know it’s a little off topic, but what is that spike in demand for hardware done to the hardware markets that you work in?

[Jon Winsett] (6:47 – 7:55)

So, you know, I just did a note on memory inflation. So memories through the roof, you know, 300 percent increase. And that could be a reset, not a temporary spike.

We know chips are in great demand. Both GPUs and CPUs are in great demand. You know, the oil crisis caused by the Strait of Hormuz is not helping anything because petroleum is a big element in hardware and chip manufacturers.

So that’s a complicating dynamic in it all. And so the chip demand is, you know, through the roof. And so data center demand is through the roof.

And you can’t bring on any more data centers fast enough. You know, Anthropic just inked a deal to consume all of the data center capacity that XAI had built out. I mean, just in one deal.

So you can feel where the demand is coming from.

[Philip Ideson] (8:00 – 8:33)

When it comes to, because one of the things when you talked about kind of the how tokens are used, it feels like there’s not a lot of granularity in that, you know, is it because you ask, you either prompt, you know, a model to do that. Or you have a workflow that’s set up where something happens agentically, you don’t necessarily know how many things are being consumed and are other ways for you to know and understand proactively, rather than when you get another email with your latest Anthropic bill.

[Jon Winsett] (8:34 – 9:30)

Yeah. Yeah. It’s a problem, you know, because, you know, a task may consume four tokens or may consume 40.

You just don’t know. And then just, you know, the software providers that embed their AI within their products, you know, they add credit, you know, a credit system on top of the token consumption, which further obscures what’s going on. But so, you know, we say that’s job number one.

If you are in any type of governing of token consumption within an enterprise, you need to get visibility into the tokens, what workloads are being used for, what users are using them, what department it’s coming out of. And most organizations don’t have that visibility yet. They just get that bill, as you said.

And they’re like, you know, how do I even dispute this? I don’t even you know, it’s not broken out. Yeah.

[Philip Ideson] (9:31 – 10:23)

I want to go down a little bit later to talk more in more details about how we think about spend management when it comes to the AI tokens. One of the things that comes to mind is there was a, we hosted a Catalyst event a month ago as we’re recording this and HFS Research President Saurabh Gupta spoke at that event, talking a little bit about tokens and called tokens the FTE rate card of the future. You know, because his perspective was coming from more of a services based, you know, HFS covers a lot of outsourcing, obviously seeing a lot of those outsourcing deals being replaced by agentic.

And is that something that that you see? Like, I’d love your perspective, something that stood out to me in terms of how I think about how what we buy is changing and how ultimately is everything just going to be coming some denomination of an AI token.

[Jon Winsett] (10:24 – 12:09)

That’s a fascinating way to look at it. You know, I mean, so far this year, there’s been 50,000 layoffs, allegedly, because AI is taking up that productivity capacity. But now the chatter with all this, the backdrop of AI expense, the new chatter is, is it more expensive to use AI or to hire employees?

And when you drill into it a little bit, you know, coding, software development, it’s, there’s no dispute, I mean, AI is cheaper than humans. But when you move into call centers, or data entry, AI begins to be as expensive as a human or more. But let’s just assume, you know, in context of your question, let’s just assume that there is a scenario where AI is cheaper than a human counterpart.

So, so AI agents are on the rise. So it’s a little different than agentic. Agents are autonomous, standalone, you could equate them more with an employee.

And, you know, if you look across the ecosystem, and in procurement, there are companies that are building AI employees. And so there’s a cost to that. And there’s an output and an outcome and a, you know, an ROI from that employee.

So then you just got to now you can compare cost wise, what is that AI employee is to the other? And you can, you should benchmark it to, you know, your colleagues point, which is, you know, will there be a rate card for these AI employees? And I would, you know, based on what I said, I think that’s viable.

[Philip Ideson] (12:09 – 12:49)

It’s gonna be interesting to see kind of how the models go, because it also makes me wonder, you know, who bears the risk of token use? You know, is it, you know, you as an enterprise, because you’re, you’re, you’re working with the frontier models, and you’re buying in tokens? Or is it going to revert back to the software companies who are buying their tokens, you know, and delivering an outcome.

And so the risk is kind of on them and how they manage the token spend versus on how you and you manage the token spend. And I don’t know, you know, that’s probably got a ways to play itself out. But, you know, it’d be very interesting to kind of see where that goes as the market gets more mature.

[Jon Winsett] (12:50 – 14:16)

Yeah, it’s, um, well, everyone’s getting smarter on this, and they’re learning, and they’re understanding where the costs are coming from. You know, 70% of enterprises have some AI center of excellence. So that kind of dictates the direction they want to go.

Below that is an AI governance body. And I would say 80 plus percent have some type of existing governance body that, that has expanded into AI. And so that’s, that’s a functioning effort there, where the gap is, is where the rubber meets the road.

And I would guesstimate looking at our client base, which are, you know, we work with the world’s largest companies, you know, half of the Fortune 100. Yeah, we see it all. And where they’re really running to play catch up is the FinOps function that may have existed for cloud previously, are now they’re, they’re trying to expand into understanding the AI environment.

And where, you know, how do you dispute invoices? How do you understand the usage? How can they control it?

Those kind of things. And so they’re getting better and smarter. And, and they’re figuring it out.

[Philip Ideson] (14:17 – 14:24)

What, what roles or functions within organizations do you typically see have responsibility for managing token spend?

[Jon Winsett] (14:25 – 15:57)

Yeah, it’s typically a FinOps function within IT. Okay. It just naturally flows because it’s similar activities is what they used on cloud.

It’s different. AI is not cloud 2.0. You know, AI introduces several complexities that did not exist in cloud. So they’re, they’re trying to figure that out.

So either the existing FinOps is getting smart on it, or members of the FinOps team are exiting into an AI specific function, where they’re getting smart. Now, procurement still plays a big role here, where FinOps, you know, manages, governs the, the, the framework to control cost. Sourcing is left to negotiate with the AI suppliers.

And so they still serve a very important function because negotiation is key because they’re not only as you buy new AI, but as you renew existing AI. Negotiation is very important, but there’s repricing that’s happening within a renewal period. And sourcing has got to be smart at that moment.

Because whenever a contract is opened up for repricing, that is a negotiation event and sourcing has got to dig their heels in, understand what is on the table and push for the best outcome.

[Philip Ideson] (15:57 – 16:21)

Do you find that, because we hear stories, and I think, you know, it’s been cited many, many, many times by now, of examples like Uber, you know, running, you’re using up their full year budget in a quarter. Are companies struggling to budget their usage, you know, as as growth, or should I say not growth, as usage is increasing exponentially?

[Jon Winsett] (16:22 – 16:53)

Yeah, it’s, it’s a real problem. I would say most enterprises will run out of their original budget over the summer. Yeah, that’s my best guess.

And those budgets are coming from net new, they set out as AI budgets, or reallocated IT funds, or some combo of both. And so those are, you know, you know, those enterprises are running into those issues right now.

[Philip Ideson] (16:53 – 17:29)

What happens when they run out of that budget? You know, do you, do you see that, that, that organizations are just saying, okay, we, we, we miss budgeted, and therefore, we’re going to, you know, keep throwing funds at it and find other places, you know, to deploy money, you know, into the tokens, or there’s a, the kind of the party is over, so to speak. And there’s a little bit of a, okay, now we got to be smarter about how we do this, and make those make that money go further, even if we might need to invest a little bit more.

[Jon Winsett] (17:30 – 18:31)

Yeah, well, so we see a couple of approaches that enterprises are using. One is they are putting guardrails in, they’ll put a spend cap on the provider itself, where they say, you do not charge us, once we hit $10 million, do not charge us and cut us off, whatever you have to do, but we want that cap put in place. And so the providers, some will do it more easily than others.

The other part is at the app level, where, you know, it will, when you hit 50% of the threshold, or 75% of the threshold, it sends, it starts sending you alerts that, hey, you’re getting close to your limit. But the gold standard is at the session level, where users are limited to a certain amount, and you cannot initiate any more calls to the LLM if you exceed it. And so that’s one great approach just to put those guardrails in.

[Philip Ideson] (18:32 – 18:42)

Yeah, so it kind of teaches the, the user how to be more efficient in their use of tokens, because they want to get as much as they possibly can out of a session.

[Jon Winsett] (18:43 – 19:16)

Yeah, you know, there’s, it’s an old economic, it’s called the tragedy of the commons. And it’s where if individual users of a community call it, you know, employees of a company, if there’s no consequence for them, exhausting the resources in the community, they, you know, it’s human nature, they’ll do it. And so where governance begins to really take traction is where, you know, you, you take the individuals and give them a certain allocation, and then that’s it.

[Philip Ideson] (19:16 – 19:16)

Yeah.

[Jon Winsett] (19:18 – 20:04)

The, you know, the other thing that you, we may not think of as consumers of AI, is you don’t need to use the most expensive models for the majority of the workloads. And that’s, that concept is getting around to the enterprise. So for example, you know, Anthropix Opus 4.8, their premier model just came out, very expensive, choose through those tokens. You don’t need that to run most, you know, corporate functions. You can use the mid model, which is, I think it’s Sonnet 4.6. Yeah. Or even a base open source model.

You don’t have to pay the PhD prices off 4.8. Yeah. So that’s another effective way.

[Philip Ideson] (20:05 – 21:12)

Yeah, I think about that in the same way as, you know, how traditionally you would think about buying consultancy services, as an example. You know, where you may want to spend the top dollar on the strategy consultants, but you don’t necessarily need to spend those dollars on some of the more tactical activities that they do. And so organizations may look at their consulting spend and say, you know, we’re going to have a two tier kind of system.

And it’s really just the same, but as applied to the models that you use, because you’re right, you know, the default is I want to use the strongest, most powerful model every single time that I possibly can, because I want to maximize the chances that, you know, that what I get is the gold standard. Without learning, well, I could use Haiku, I think it is the base model for Anthropic, which will actually do this very, this subset of tasks perfectly well, and the cost is so, so much cheaper than using, than using Opus, but you kind of default to Opus every single time. I know I do that.

I’m guilty of doing that.

[Jon Winsett] (21:13 – 21:30)

You want it, you know, you want the latest and greatest. But I love the consulting services analogy, because, you know, you, when you think of it, you know, you wouldn’t bring a McKinsey consultant to do, you know, basic tasks, you know, it does, I think, help frame the mind to make the right choices.

[Philip Ideson] (21:31 – 22:30)

Yeah. I read something, as we’re recording this, we’re recording this at the beginning of June, Sam Altman, who’s the CEO of OpenAI, he was quoted as saying that, because there was a question to him about management of costs, and he said, I think we’ll have a lot of ways we can help people get more value for less spend, but we’ve gone from the beginning of this year, an issue that never came up before, and that was people were totally happy with the amount they were spending, to all of a sudden, it’s a huge issue in what they’re spending. It sounds like that’s something that you’re seeing, you know, when you talk about people being out of their budgets by mid year, and I wonder if that opens up a door for procurement as well, where the first six months of the year has been, I just need to get as much of this stuff as I can, and there’s absolutely zero from, well, I may be wrong in this, my assumption is there’s not a lot of leverage that procurement has in negotiating the cost of those tokens, but as the frontier models may start to see a slowdown, that may open an opportunity for procurement.

[Jon Winsett] (22:32 – 23:48)

Well, you know, Sam Altman seems to frequently be tone deaf, and that quote strikes me as such, I mean, he knows well, why the token consumption has gone out of control, because the nature of AI, how it’s being used has changed in the last five, six months, as agentic really takes hold, so he knows that. And why it’s an issue is because the consumption is costing too much to the enterprise. I mean, when you see OpenAI and Anthropic’s revenue, breaking records, you know, the combined ARR between OpenAI and Anthropic now is twice the size of Salesforce, you know, which took them 27 years for Salesforce to reach that scale, and now they’re bigger.

Who’s footing that bill, you got to look at the other side of the equation, who’s driving those revenues, the enterprise is. And, you know, as much as consumers are using these models, it is dwarfed by how much the enterprise is using, it’s the stats like it’s a enterprise spend on AI is 11 times greater than the consumer spend.

[Philip Ideson] (23:48 – 23:49)

Yeah.

[Jon Winsett] (23:51 – 25:49)

So when, you know, an Altman says, I don’t know why it’s an issue. Well, it’s because the costs have gone up so greatly. But it’s something’s got to give, because as OpenAI prepares for their IPO queue, you know, we’re getting more and more visibility into their financials, they’re projected to lose, I think it’s 14 billion this year, which equates to they’re losing, they’re burning $2 for every dollar they bring in the door.

So that’s not sustainable. And public markets, they’re going to want, you know, some clear timeline to profitability. But how is that going to happen?

You know, how the, you know, the token sellers are going to have to figure out a way to, you know, how to provide more capacity for a lower price. Or, I don’t know if you caught this headline, this was last week, where OpenAI had leaked or announced that they’ve are working on a software layer that sits in between the model and the GPU chips that will require less compute power from that part of the stack. So you can see they’re busy trying to figure it out.

And funny enough, and then I’ll shut up about it, the, you know, the Chinese models, you know, they’re out there, they’re being used by some US companies. They had to figure out a way to consume less compute when they run their models, because they don’t have the Nvidia type chips that we’ve got, because we banned it. And so they figured it out.

That’s, you know, the mother of invention is necessity. Yeah. So I think that’s one area where it could break is where we figure the best way to run these models.

[Philip Ideson] (25:50 – 26:14)

Yeah. Yeah, those, the model companies, it’s interesting, you know, if the model companies will become anything more than a commodity over time, and how that affects their pricing power. And so obviously, the rush is on for them to reduce their cost to serve as much as they possibly can as quickly as they possibly can.

[Jon Winsett] (26:14 – 27:03)

I’ve heard that comparison before, where these frontier models actually become utilities, like your electric company. Because, you know, there have been fantastic innovations in the world of business. You think of aviation, airlines.

Airlines have really, if you look over the course of time, they’ve never really been a big moneymaker. You know, it’s hard to make money in that business because of all the variable costs, fuel and such. And you’re all flying the same equipment, it’s just how do you make it differentiated.

Utility, you know, if when you think of LLMs in that utility context, it does make sense, because maybe they will never make money. You know, maybe they’re just part of the infrastructure. Yeah.

[Philip Ideson] (27:05 – 27:48)

And as they’re all readying to IPO at the moment, I think that’s one of the big unknowns at the moment. And it’s going to be really interesting for us, obviously, to keep watch on, because no doubt this is where the innovation is coming from. But will they have the ability to be the ones that can monetize, you know, that innovation?

As you said before, the hardware, the chips providers, you know, they figured out a way. The data center builders, you know, they’ll be making some good money out of this. But will the structural model of the industry have to change so that profitability flows to the right areas that incents everybody to continue to innovate?

[Jon Winsett] (27:49 – 28:55)

I think it’s going to be a fascinating summer, because think of the IPO queue. You’ve got OpenAI, Anthropic just announced, SpaceX. Now, if you think of SpaceX, you know, it’s a wonderful company.

I don’t know if it’s worth 1.75 trillion, but it’s a wonderful company with all the Starlink and the space hauling. But if you look at their XAI, you know, very little revenue has come from their AI. XAI has all that data center capacity, again, the chips and MIPS people.

So it has that element in there, and they’ve sold it all to Anthropic. So that’s really the only revenue that’s coming out of that, minus the former Twitter component that’s in there. So, you know, Grok’s not making any money, XAI as a whole as an AI company is not making any money.

So we’re going to get visibility in all of that as these things come through the queue.

[Philip Ideson] (28:56 – 29:30)

So I don’t want to ask a question about best practice, because this is so early that I think that best practice is probably still being defined. But what are some of the most interesting practices that you see amongst your client base and how the role of procurement in managing this spend? You know, what are those teams doing?

How are they getting involved? Are they more looking at demand management? You know, I’d love to just understand a little bit about what you see from those who you really think are kind of at the top of the curve in terms of how the procurement teams are thinking about managing token spend.

[Jon Winsett] (29:30 – 31:40)

Well, demand management is certainly a core component to it. You know, if I just kind of riff on this for a minute, you know, you’ve got to get visibility into the tokens. And so a lot of the FinOps platforms and a lot, you know, actually, I talked to a couple of companies this week where they are building their own token visibility dashboards using AI, you know, ironically.

So getting visibility on tokens is one. Number two is they are working on getting key clauses added to every agreement. Things like token visibility.

A lot of providers try to obscure that. Being able to audit the billing. There are certain rights you should push for.

So there are key clauses that we share with our clients that they should be pushing for. The other is model routing flexibility. You know, being able to route from 4.8 to 4.6 to open source, for example. Those guardrails we spoke of about limiting spin caps, you know, hitting thresholds and capping it at that. And then whenever the vendor comes to reprice, you just treat that as a negotiation event. I think that’s a key one.

Yeah. And one bonus one that is really just emerging. And in fact, we just published a blog on this, is that using open source models, orchestration layer, all the different layers that open source has as a negotiating lever for the monolithic AI tech stack that’s being pushed upon the enterprise today.

So instead of buying everything from Anthropic or everything from open AI, you fragment that tech stack, swap it out with open source, or at least leverage the fact that you can swap it out with open source as a negotiating lever. Right.

[Philip Ideson] (31:44 – 32:20)

Where do you think we’re going to be in six months? I know it’s like a, you know, I’m throwing my hand, for people listening, I’m throwing my arms in the air, like nobody really knows where we’re going to be in six months. But as you kind of look ahead and kind of you obviously have access to a lot of information and data and you can observe how people are using, you know, our organization is going to slow down the use of tokens.

Are we going to find ourselves in a market where procurement can play a greater role from a cost perspective beyond some of the things you talked about? I’d love to know kind of what you’re looking for, as you look ahead for some of the signals, perhaps on how the market is changing.

[Jon Winsett] (32:21 – 34:50)

You know, I’m going to go out on a ledge here. And because I didn’t know that question, I’d be asked to do some prediction, but let’s not waste this opportunity. Let’s see if I can, I will make some predictions and then we can come back and do this three or six months from now.

Yeah. I think, you know, currently OpenAI is, we see more OpenAI deals in the enterprise than we do Anthropic, but Anthropic is coming up fast. I think Anthropic is going to eclipse OpenAI in the enterprise and it’s going to remain the leader.

So that’s one prediction. Currently in the AI security market, CrowdStrike is the number one, the lead vendor in that. I think Anthropic Mythos is going to come in and take over that spot.

So that’s prediction number two. Prediction number three, there’s going to have to be some equilibrium achieved with these cost of tokens and the way that they are consumed by the enterprise. And that could be either cheaper pricing from the frontier models, because I don’t know, Chinese models come in at 30 times cheaper price, or it could be some other dynamic, but it’s got to break.

Fourth prediction, this is a good one. I believe that any AI that is embedded within a hyperscaler or within a SaaS platform will be consumption based within six months. So the reason why I say that is, for example, Microsoft Copilot, you pay 30 bucks a month per user, that’s list price.

And currently, you can go and run as many jobs, many workloads to chew up as many tokens you want for that 30 bucks. That subsidy is going to end. And so there’ll be a reckoning coming.

And that reckoning will be in the form of consumption based or credit based system. Credit based, I think, is more risky than a token because token, at least there’s some connection to what you’re doing. Credits just make up their own map.

[Philip Ideson] (34:50 – 34:55)

It’s just to make more smoke and mirrors around how much you’re paying for what you get.

[Jon Winsett] (34:55 – 34:58)

Totally. To obscure it even further.

[Philip Ideson] (34:58 – 34:59)

That’s what I was looking for.

[Jon Winsett] (34:59 – 35:01)

So those are four predictions.

[Philip Ideson] (35:01 – 35:02)

Okay.

[Jon Winsett] (35:02 – 35:03)

Let’s see how that…

[Philip Ideson] (35:03 – 36:10)

Well, I know I put you on the spot there. Yeah, but it’s fine. It’s one of the things I think that enterprise needs to figure out is how do we ascertain an ROI of token spend?

And I think that’s the big unknown, isn’t it? Because if you with confidence can determine this is the ROI, then there’ll be many organizations that say, I don’t care what I spend. I want to use as many tokens as I possibly can because I’ve got a lot of confidence in the ROI of that investment.

But until then, we’re in this kind of middle ground of we’re all going gung ho into using tokens because there’s a FOMO factor of that. I don’t want to be left behind relative to my competitors. Versus I can sense that what I’m doing is going to pay off in the future.

But those investments haven’t yet paid off. And it feels like some of them probably need to start paying off fairly quickly to give confidence in the continued investment in more tokens.

[Jon Winsett] (36:12 – 37:20)

Yeah. You know, we’re doing a research study. We’re midway through it.

And it’s about AI spend peer benchmarking. So it’s only for large enterprise. In fact, if any listeners are wanting to participate in this, it’s not too late.

You can reach out. But we’ve got all these large enterprises. They want to know, are they spending too much, too little?

Are they moving too fast, too slow? They want to know what’s going on with their cohort. And a key component of that is they want to know what ROI, what benefits.

Is there real outcomes that are changing, making an impact? And that’s part of what this research study measures, because it’s really murky right now. I mean, I’ve seen some chatter in the press that CEOs are like, we think we’re finding productivity from it, but we don’t know.

That’s got to change. We’ve got to be able to map cost to outcomes, cost to benefit, and measure ROI. But this research study is asking those questions.

[Philip Ideson] (37:22 – 37:43)

It’s going to be interesting to see kind of what comes out of that. What other things are you looking to? Because, one, I’m happy to put a link in our show notes.

I’ll talk about that just as we wrap up to the study for people to go and share their experiences as you’re kind of gathering the information. But what are the other kind of things you’re trying to ascertain by doing this study?

[Jon Winsett] (37:45 – 39:32)

Things that we’re asking is, how much are you spending on AI versus your total IT budget? That’s a key one. To really fill out the study, it takes a little under 30 minutes, but it’s good to have maybe someone that’s got IT perspective and sourcing perspective.

It’s best with two people. We have customers who are just opening up their laptops at lunch hour and working it through. So that’s one key one.

Also, what vendors are they using in the six core categories of AI, so like data foundation? Is it Snowflake, Databricks, someone else, Informatica, security? What system integrators are you using?

So that’s some good questions and insight that comes from that. Also, what use cases are you using it for? Is it more knowledge worker workflow like co-pilot?

Or is it more reasoning for analysis and research? It’s asking those level of questions. I like this one because I find it fascinating.

This one area is, what SaaS providers in your portfolio are most at risk for being disrupted by AI? And it’s great to get that perspective. So for example, the whole BI sector, so think Power BI, Tableau, Anaplan, Alteryx.

Those are right now in the midway through the study, that’s leading the sector that’s most disruptible. But other ones are things like project management. And the list goes on.

There’s like eight that are disrupted. So I think that’s fascinating.

[Philip Ideson] (39:33 – 40:28)

Well, again, it’s gonna be really interesting. We should chat about it once the results are in. We should chat about that.

Because I’m sure that’s really interesting information for folks to think about. Like, where should we be? Where’s the biggest opportunities for us to look at either alternative sources, to look at make versus buy, you know, and which of those areas that, you know, that’s still going to be our system of record.

That’s not changing. In fact, those are probably going to become more important, but in different ways, because it might not be that I use the UX, but I’m using everything kind of under the hood. And I’m plugging into that using kind of, you know, my agents going into their data lakes and all the information they hold on me to then use it in other ways versus me going in there and typing in Salesforce that, you know, this is what I’m doing, I’m working on a deal.

So there’ll be some of those that come out of it that are, you know, probably even stronger than they were before.

[Jon Winsett] (40:28 – 40:55)

I mean, that’s the exciting part of the vision of AI is where it just changes the nature of work. Right. You know, if you’re inputting something into SaaS, you’re already demoralized to some degree.

Yeah. But to picture having an agent really doing that kind of lifting and really you’re using your highest value, your best and highest fit is always alluring.

[Philip Ideson] (40:56 – 41:17)

All right. Well, I know it’s about time to wrap up. I’m just going to finish.

You’ve talked through some of the things that procurement can do, but I just wonder if you have any advice or best practices that you can share for somebody who may be listening to this and the procurement really hasn’t played a role yet in helping their organizations thinking about token spend. Where’s the best place, most effective place for them to start?

[Jon Winsett] (41:19 – 41:23)

Well, you know, from a procurement standpoint, they’re there to optimize the deals.

[Philip Ideson] (41:24 – 41:24)

Yeah.

[Jon Winsett] (41:25 – 42:24)

And so, you know, this is self-serving because we do some of this, which is you’ve got to benchmark aggressively. You’ve got to understand how your pricing stacks up to similar size deals and similar size environments. That’s important.

You’ve got to push for portability. We’ve said this for years related to cloud, but whenever you’re negotiating with an AI provider, you can’t get locked in. So you’ve got to be able to move models, move orchestration layers, any layer.

That gives you enormous leverage when the time comes. I would, you know, I’d be remiss not to mention energy because energy is quickly becoming an issue. You know, 50% of a data center’s expense is energy.

[Philip Ideson] (42:25 – 42:26)

Yeah.

[Jon Winsett] (42:26 – 43:18)

So as that cost goes up, that means the data center is going to start pushing those costs downstream. Yeah. And so the first level that’s hit is hyperscalers because hyperscalers at the end of the day are data centers.

So if you are working with an AWS, Azure or Google Cloud, you’ve got to make sure you’ve got to eliminate any energy pass-through clauses because that can be bad news where they can adjust on a monthly basis up or down based off the energy and it never comes down. It only goes up. So, you know, it’s multidimensional.

You know, we’ve got a white paper that talks to a lot of this. Perhaps we could put a link of that.

[Philip Ideson] (43:18 – 44:11)

Yeah, we can put it in the notes. If you want to give that to me when we’re finished, we’ll put that in the show notes too. Yeah.

All right. Well, perfect. Well, talking about show notes, I’ll put a link to your LinkedIn profile so people can reach out directly to you.

I’ll put a link to NPI in there. We’ll link up to the survey and also the paper that you talked to. And for anyone who’s listening to this conversation, then you can just go to outofprocurement.com slash podcast. That’s outofprocurement.com slash podcast. You’ll find this conversation with John and I or you can always just go into the search bar and type for John or NPI and this will come up and you’ll find it all there. So, John, I want to thank you so much for joining me, sharing, you know, just some of your experience, expertise and some of the data that you’ve been gathering around the token spend.

It’s a topic that we’re going to be talking a lot about, I’m sure, in the coming months and years. And this is really just scratching the surface, I know.

[Jon Winsett] (44:12 – 44:16)

Well, Phil, thank you. I appreciate the forum. This was fun.

[Philip Ideson] (44:16 – 45:14)

Yeah. All right. Well, we’ll do it again sometime soon.

Let’s do it. Thanks everybody for listening and we’ll talk to you all again soon. Take care, everyone.

If this episode struck a chord with you, please do send it to somebody. We grow here at Out of Procurement through word of mouth, and that would be really appreciated. You can also support us by giving us a thumbs up, a star rating, or a review wherever you listen to your podcasts.

Since 2015, we’ve built the world’s largest free resource for procurement professionals looking to elevate their impact. Our resources span podcasts like this, videos, blog posts, papers, and events. To join us on the inside and to ensure you never miss an episode, a webinar, an event, or a post, please do subscribe to our weekly newsletter, This Week in Procurement.

You can do that at outofprocurement.com slash subscribe. That’s outofprocurement.com slash subscribe. Thank you for joining us today and I look forward to seeing you again soon.