Nick Homan

03:19 - 03:39

Hi, everybody. Welcome to, this fireside chat with Martin from SAP.

Welcome, Martin. Great to to have you here today in SAP's office in the city of London, the backdrop of the Gherkin, which is fabulous, and also on the day you celebrate your tenth anniversary with SAP.

Yes.

Martin Norato Auer

03:39 - 05:20

Which is great. Very exciting to be in this office.

Ten years with SAP. It's a long time.

Good history. Fantastic memories.

So let's start. If you could tell the audience a bit about yourself, how you got into observability, your role at SAP.

Of course. No problem.

So, yeah, I'm Martin. I'm based in beautiful Germany, Munich.

So it's a good place to be here in London. Also, very nice city.

Yeah. Ten years with SAP, it's a long time.

Actually, I started my career in security. So I did a lot of security consulting all around the world, traveled.

And then in 2015, I thought, okay. Now it's time to go to SAP, work there in security in the beginning, did projects like typical consulting jobs.

Mhmm. And then I joined this small acquisition of SAP, which is Hybris, commerce shops.

It's quite part of the history because Hybris is not used anymore. So it's commerce now.

And I joined their security, built up some security practices. And then one day, one guy, during COVID, who was the expert in observability, left the company, and my manager said, hey, Martin.

You're in Munich. The guy's in Munich.

Can you do the handover? I don't have anyone else. No one can travel.

It's COVID. So just do the handover, and we figure it out later who will take it from you.

And that later never happened. So that was my start in observability.

Fantastic.

Nick Homan

05:20 - 05:30

So so great to hear about how you joined SAP. If we talk about you today, your role, and the team you have, if you could explain to the audience a bit about that.

Martin Norato Auer

05:30 - 07:38

Yeah. That's a very good question, and makes me proud to look also back.

When I started in observability, I had a small team of developers that were building services around our observability stack. And so that team is still there.

I think only one person of the team left. So it's a really proud Good achievement.

People for so long interested. And then, yeah, over time, I I saw, oh, there is actually stuff going on on the automation side, operational automation that is very close connection to that.

We should add that to the team. So this team joined at some point also BI, so the whole data lakes and how we interpret data and how we work with data Mhmm.

Joined my team. And then the whole process stuff, so how do we do incident management on a process? How do we do change management, service request management? All that, like, was added over time.

And, yeah, the scope just got bigger and bigger. And I was thinking, how can you structure that properly? Because it's it's kinda all connected, but still, like, very specific domains.

And then I came across, a garden article last year that talked about AI ops. And I was like, okay.

AI, we do a lot with that. We do operations.

What what is it? And then I saw it tested three pillars, observe, which is the observability monitoring piece, has engaged with all the process implementations, and has act as a definition, which is the automation. And I have that in the team.

So it was, like, straightforward. Okay.

Let's structure my team really in that way, build and observe, engage, and act pillar around an AI framework as a foundation for the whole CX area to consume. And that's how the team is set up now.

And it's it makes a lot of sense that these areas go together because there's a lot of touch points within these areas.

Nick Homan

07:38 - 07:49

Yep. And you've got team members all across the the globe and SAP collaborating together within that team.

It's you touch on kind of the the different members.

Martin Norato Auer

07:49 - 08:00

Yeah. So, my team is actually really widespread.

So it's from Vancouver to Sydney and a lot of locations in between. So it's an awesome team.

Nick Homan

08:00 - 08:01

Truly global.

Martin Norato Auer

08:01 - 12:00

Truly global. I think there's no common time where everyone is, like, working time at the same time.

And so from a team structure, what I'm trying to do, I'm trying to follow this model of a, product manager and the development manager. So, like, the product manager for Observe defines the strategy, defines the direction, make sure that we have the right things in place, and then we have a development manager that actually speaks the language of the developers trying to get this what the product management thinks into a product.

Yep. That's for all three pillars I have to set up and works also very well.

Works well. So we think now about how you've managed to bring some external vendor tools together and amalgamate that into your processes and some of the in house skills.

If you can maybe talk about Marvin a little bit and and what that's done to help your operations and deliver the the results that you're seeing in SAP. Before starting with Martin, I have to go back.

My first year, Black Friday, I mean, Black Friday in commerce is big event of the year, and it expanded now. There's a whole Cyber Monday, there's the whole Christmas, Boxing Day, Singles Day, Elbow and Finn.

You name it. There's a lot in the November, December time, but Black Friday is still the the number one.

And when I joined the first, we had war rooms, three war rooms across the globe. And there were so many screens.

And okay. I'm looking at this tool, and the other one looked at the other tool.

And I think there were seven, eight tools. I don't remember exactly, but there were, like, dedicated people looking at screens and then some are shouting, I see something in this tool.

And then, no. Can't be.

I don't see anything here. And then, no.

But I still see something. And then, yeah, but I see for this customer.

No. I mean the other customer.

And it it it's kind of like chaos to reflect. But we did okay.

And seeing that, it was clear we have to consolidate. We have to get rid of all these different fragmented tools and see what's really relevant.

Yep. And, I mean, we already had a good footprint in Dynatrace.

We had a very good footprint also in Catchpoint. So it was kinda obvious to go with these tools first and focus on these two tools.

So we tried to really get, like, decommission one tool after the other and bring all the logic that was in these tools into either Dynatrace or Catchpoint. Yep.

So that was also hard fights, of course, with the development teams because, yeah, teams have think, I have this awesome idea. I can monitor it with whatever tool, and there is open source, and they can do so much.

But then when it comes to operations, it's it's not just about availability or performance. It's a lot about total cost of ownership.

And you can only run a business on scale if you consolidate. If I need three experts for each tool because I need to do, like, a twenty four seven support, it doesn't scale.

If I have three tools out, suddenly, there are nine people. Yeah.

And it's it's simple examples, so it made a lot of sense to consolidate. And, also, there's a lot of convincing and talking why is this also possible in this tool and just looks different.

Anyway, so the consolidation to these two tools, I think, was was key. And then we had two alert sources, and that's when Marvin came in.

Actually, it was not Marvin in the beginning. We called it table dashboard.

‍

Nick Homan

12:00 - 12:04

And yeah. And and and the name Marvin is quite an interesting story as well.

Right?

Martin Norato Auer

12:04 - 13:07

I will come to that. So, so I I had the vision.

We need something, a visualization where someone who can someone that has no clue, no deep knowledge about the product, who is not an expert, can see right away, okay. This is something where I need help, or this is something that I can just watch and see how it develops.

So we started several attempts, like, with if there's more alerts, there's a bigger problem. It didn't work like that.

Then we started to give it weights. Also, it didn't work so well.

In the end, we developed an algorithm that takes all the alerts, takes the logic that we know from the business what's impacting, and from that, we can do it. But then now comes Marvin.

I had table dashboard. That sounds because it looks look like Excel in the end.

Nick Homan

13:07 - 13:10

It's it's not really a product. It's not exciting.

Martin Norato Auer

13:10 - 14:03

It's not exciting. We need we need a brand.

Yeah. And and it's two stories.

So my team says actually, I wanted to call it Martin, like, myself, but I didn't wanna make it that obvious. So just change one letter so it became Marvin.

But that's not the real story. The real story is that I thought I want to have something that is wanna be lazy.

That's that's the goal. I wanna be as lazy as possible.

And the first that came to my mind was the Hitchhiker's Guide to the Galaxy where this Marvin robot is super lazy, does his job somehow, but in the laziest way possible, and that's how it came to Marvin. Got it.

Great story. And so you you talked about tool consolidation and bringing it together with just Catchpoint Dynatrace.

Perhaps if.

Nick Homan

14:03 - 14:07

you could expand on how you use those those two technologies? The audience?

Martin Norato Auer

14:07 - 17:37

Yeah. Of course.

So, let's start with today. So today, we have a very clear setup.

We have Dynatrace as an application performance monitoring solution. We're looking into what's going on inside our systems, within our control, which is a really good solution.

And catch point, we use as Internet performance monitoring solution. So everything that is outside of our premises, that is not in our control but can affect our customers business.

So the CDNs, the DNS, the ISPs, and I think that's also important in the context of our customers to understand their whole business. You can say, there's no problem.

Our site is working. Yeah.

Your CDN might have a problem, but it's not my problem. I I think it's good to tell the customer, the customer we identified an issue with your CDN.

We can't really do anything about it, but please be aware we see it, and you should do something. And that also shows, like, the interest in the customer also establish a better collaboration.

Yep. But I wanna take a step back.

It's because I said tool consolidation. And, of course, everything some years ago was questioned.

What makes sense? What does not make sense? And I didn't know catchpoint back then. And I had a look and say, pinging, like, websites from outside.

What's what's the real benefit? What can I see from the Internet that I cannot see internally? Right? If my my web server has a problem, I already see that in Dynatrace. I have the alert, so what's the point of even doing that? And this is kind of how the conversation with Catchpoint started.

And this also may be a good advice to work closely with your vendors. Be like, understand the road map they're going, but also have your own road map ready where you wanna go and discuss that together.

And vendors will not always do what you want, but, yeah, it starts a discussion and people start to think. And, yeah.

So with Catchpoint, I think the really awesome turnaround was when Catchpoint released the term of Internet performance monitoring to see okay. I'm not just testing your SSL certificates or your CDN.

I'm really looking at Internet scale. Yep.

And, yeah, Catchpoint has the node network to it, so make use of it. So that really helps us to understand on we have multiple solutions in different hyperscalers in SAP data centers across the world, and some customers have many, some customers have only one product.

So you have to figure out what is actually relevant because it doesn't help me if my marketing solution runs fine and my commerce solution runs fine, but the interface between those two is not working. So then, again, you have an issue.

That's why looking also at the Internet is so important, and this development that CachePoint did with Internet performance monitoring is, for us, the perfect addition to what we already see on the APN side within our, data centers. Yep.

And obviously,.

Nick Homan

17:37 - 18:04

you now leverage not only the synthetic monitoring of the node network, your own testing, you're looking at aggregated views of Internet data through SONAR and and use the the professional services capability, as you've mentioned, for some of the, you know, Black Friday events, throughout the year. So really leveraging lots of the the the toolsets and and capabilities, which is great.

Martin Norato Auer

18:04 - 18:05

Yep.

Nick Homan

18:05 - 18:38

And, I think then if we think about impact. So the tool, the processes you put in place, you know, that's that's critical in terms of being efficient as you talked about with scale.

But then what this has meant for the operations within CX and the results you've delivered, it'd be great to understand what you're seeing as an impact. The war rooms you talked about or right at the beginning to now how you handle the operations and the the actual metrics behind that.

Martin Norato Auer

18:38 - 22:20

Yeah. So metrics is a really good point.

So you need data points to run a business. So when I look at one of the key indicators we had in the beginning, we looked at the amount of alerts we have.

But when I look at this now, we we thought it makes sense, but it really does not. So we have, I don't know, factor 10 the alerts today that we had five years ago.

But what matters is something like how fast am I to detect something. And not just to detect, but also to see, is it relevant? Because in the end, the the amount of experts is limited.

And if I have 10 situations at the same time, I need 10 experts. So if I can limit that to maybe three, I only need three experts.

So it's easy math. So that was one of the, like, key things.

How can we identify the relevant things? And we collect a lot of data, where it also becomes quite imminent. How long does it take us to inform a customer? And again, in history, some years ago, we were relieved.

That here took us hundred eighty minutes in average to inform a customer in a commerce scenario. It's Yeah.

Looking back, it was really ridiculous. So customers were calling us, like, why don't you see that? You should see that.

You're running our platform. You have the data.

Why don't you see it? So we took the effort and analyzed, like, all these situations where customers called, where we had a situation, where we missed a situation. But, okay, here, this was the alert pattern.

Why did we not see it? And then we tweak the algorithm. Or here, the customer called.

Why didn't we see? Oh, this is a scenario we didn't have on our radar. So let's include this scenario in our detection.

So slowly with the data we had and the analysis we did, became became better. And then we looked at the different points of that are needed to inform a customer.

So there has to be an alert first. If there's an alert, someone has to decide is that alert relevant or not, then you might need an expert that says, yes.

This is really a situation because that first line of defense stuff has keep knowledge. Then you need to create some kind of wording to inform the customer because just sending out, like, your customer, there is something.

Yes. Let you know is not good enough in the end.

So we took all this, like, how long does it take in average to even create this customer message? Go to the tool, find the customer, write the message, send it out. How long does this take? We clocked all that.

And then we found out, okay, we can do an automation here. We can use an API here because it takes too long to do it manually.

We still wanna keep a certain verification step because we don't trust the automation in the first place. Once we felt confident, we also automated certain steps to verify.

And, yeah, with all this and piece by piece, we're now in an average of two minutes to inform customers and not just inform, but really in a qualified way that we say, the customer, there's an issue on your storefront. It's most likely has to do with your CDN.

We're working on it, and we keep you posted. So it's within two minutes for that level.

And that's I'm very proud of what my team achieved here. Yeah.

And and you should be. It's, it's truly customer centric and, you know,.

Nick Homan

22:20 - 23:00

undoubtedly, SAP is about the market leading brand and maintaining that customer loyalty and that perception amongst your customers. It's clearly critical to have that kind of engagement with them, I guess.

One of the other things I love is is how you and the team have taken, Marvin and used it for internal, purposes in terms of, like, the leadership team and how they can get, you know, accurate information about incidents and what's going on because we've got our external customers. And, obviously, we've got our internal customers in any business.

So it'd be cool to hear how you've you've done that as well and what the direction you're headed.

Martin Norato Auer

23:00 - 25:35

I think here we're still at the beginning of the exploration phase. But as you say, it's obvious.

Right? If there is a situation going on, customers call the people they know when they have a situation. And, also, when I inform a customer, it doesn't mean that I reach everyone at the customer's level.

Normally, you reach the technical teams. Yeah.

You don't reach the c level executives. And it also makes sense because it I think it's not the right level to send out emails.

There is an issue because in the end, it's with the technical teams. But if that happens, of course, you get calls from customers and say, hey.

What's the situation? And I think everyone who is in this customer facing role knows, okay, then how can I get relevant information quickly? Right? It's, there is maybe a bridge call somewhere, but how can I find out where this bridge call is? SAP is not just one product. We talk about lots of products.

Right? I mean, just in the CX space, it's already more than two handful of products. And the customer calls and you might not even know which product he's talking about.

He's just saying, I have a problem with my CX product. So you have to be very quick to get the information, but also you have to know where to look.

There's a lots of places. There's the ticketing system.

There's the monitoring tools. There's bridges.

There's communication channels, there's emails. Where's the right information at the latest? So what we did here is we built a mobile app, that consolidates the most important information for a leadership level.

So if I get a call now, I can take up my phone, just open this app, see, yes, there is a situation. Someone's working on it.

I have the contact person to do a call to if I want to get more information. I already have a high level information.

And there, I think we come to AI, but also here as first AI use case, we use a a summary of the discussion that is going on either in bridge calls from the transcripts or from the communications that are going on. When, like, the AI does a summary of it and also displaces India.

So within, I would say, thirty seconds, I can give a qualified answer to every customer about a certain situation that is going on.

‍

Nick Homan

25:35 - 26:30

So that's a fantastic position to be in. So, I mean, if if we never just summarize that, we've we've created an operations team that's delivering, you know, within short periods of time, identification of issues.

We are able to, you know, contact our customers, and we're actually giving internal stakeholders the information at their fingertips to be able to answer questions as well. So, you know, fantastic set of outcomes, obviously.

You you sort of mentioned AI, and I guess we as as we we think about the audience and what they might be talking about in their organizations, Interested to get your personal view as a, you know, an industry leader as to the value of AI. I mean, positive and negative elements, obviously, and where it should be used and what you think maybe into the future as well.

Martin Norato Auer

26:30 - 31:20

Yeah. Of course, it's in our day to day AI is in everyone's mind at the moment.

And the way I like to think about it, especially about generative AI piece in a leadership position. So now I finally can ask questions I never asked because I know the effort is too high.

And to give you an example, like, look through all the tickets that were opened over the last year and correlate that with customer search events. Like, I don't know.

I mean, Valentine's Day is an easy one, but customers also have flash sales, have sales campaigns, whatever, throughout the year. So is there a correlation between the amount of tickets happening with these search events? And, yeah, you can do that for one customer or three or five, but you can do it on really on on scale because, also, you don't know when these events are in the first place.

So you have multiple places to look at, and you have to bring it together. And most likely, the outcome of that analysis is just like there's no path.

So I never asked the question. But now I have the data.

I can go to our AI models, our chatbots, and say, do this analysis for me? And I get this data out, and then I can say, actually, there is a pattern. I was curious.

So there's a lot of these questions you can now ask on data analysis, and I find that is super exciting because you actually find these edge cases that are hard to find. On the other side, of course, everything that is around, like, language, so summaries of things, how can you condense information together, for the right audience because, yeah, a technical person wants way more details, like, as in a leadership role or when you have to talk to to customer quickly, you don't wanna read through 100 pages.

You want to have, like, two sentences, and that must have the relevant information. If I need more, I can go further.

Mhmm. So, yeah, generative AI, we use a lot, and there is a lot of potential there where when it comes to agentic AI, and we have not found the the, like, the the killer use case here, the really strong use case.

And, this may be something to think about. When when you're in operations, you want to have a consistent way of doing it because every deviation from the standard creates one of these edge cases where you you can really measure it properly.

So, in the agentic AI, it is maybe like a human. So if I ask you, build a pattern for yourself, the first three days you do in the morning after you wake up.

They don't know. You wake up.

You take your phone. You check your emails, and then you drink a glass of water.

Try to do that for one month. You will not be able to do exactly the same order every day.

And it's just normal, and there's no problem. And a bit the same is with when you handle situations in operations, you want to have, like, a reliable way, always the same way of doing it.

And then you detect actually patterns. Yeah.

Here's something we're not doing right. You can tweak your process.

But you want to have it the same way always. At least I would like to have that.

And with an agentic AI, it's the same with a human. Most of the time, it's the same, but sometimes it's a bit different because there's some, like, additional factors that is seen by the AI and the factors that in and then just acts differently.

So I don't have a reliable outcome, and that's not good. So often, a simple automation is the way better way of doing things.

Yep. And, yeah, of course, there is agentic use cases, that we use, especially with browser integration that I find really, like, interesting to add an AI, like, control your browser, go to that site, get the following information, summarize it for me, send an email out of that in the workflow kind of way.

It's interesting use cases, but in the incident major incident scenario, the agent we don't have any agentic use cases Yeah. Today.

But a lot of automation use case.

Nick Homan

31:20 - 31:53

A lot of automation. And, yeah, I guess, there's more to discover in the whole Yes.

AI roadmap. I guess, just as a final question for the audience and someone maybe, who's either starting now or looking at a new strategy, would you have any advice for for anybody in the audience about, you know, what what pitfalls to avoid or or good practices to maybe follow that you've you've learned over your career? Oh, that's a tough one.

No pressure. Yeah.

No pressure. Thank you.

Martin Norato Auer

31:53 - 34:22

I think there's a lot of data out, so make use of the data. Try to understand the data and make your conclusions, but also discuss that with people and discuss that with people who did it in the past or have a clue about things or where you say, yeah, this is someone who does the same.

And then you figure out, hey, they have similar challenges or they maybe even solve one of the challenges. And even if you cannot one by one copy what that person did, but at least it helps you.

It broadens your mind. You get more understanding.

You learn more and also, yeah, follow, like, what's going on in the industry. Like, what are the trends? And don't jump on any trend right away.

But, again, think how does this benefit me? And maybe also one thing how I'm trying to run my team, so give out this direction of, so this core business, it has to work. There's no question about it.

That's just day to day. It's caught on, mission critical.

There's no discussion, and this should be around 60% of the workload of the whole team, not of an individual, but of the whole team. And then I have an emerging business section, where I say it's around 30%.

So this is the kind of things. I know they're coming.

I cannot avoid it as as a new product release. I know I have to avoid it.

There's no discussion. There is an initiative that is across the company.

I will have to do it. So this is, like, 30% of the time.

And then 10% of the time, the remaining 10, I call New Horizons. And this is whenever someone has a idea what we could do, we don't know today is that something that is beneficial or not.

Will it ever become part of the core business or not? So just exploratory. So stay curious.

Like, explore new ideas and don't it's like, I'm so busy. I'm I can't do anything.

You have to find the time somehow, and you have to structure things in a way that this time is not lost. Because if you lose innovation, you will lose business, and in the end, you will.

Nick Homan

34:22 - 34:38

be just Innovate innovate or die. Yes.

Yeah. No.

Wise words and and really good advice. Well, thank you very much for your time.

It's been great to have a discussion. I hope really, really useful for the audience today.

Martin Norato Auer

34:38 - 34:46

Thank you. Thank you very much, Nick.

It was a pleasure talking to you, and, yeah, it's amazing day in London, so time to explore.

Nick Homan

34:46 - 34:56

Time to explore.

Nick Homan

34:56 - 35:46

Okay. Great.

So, I hope you all enjoyed, the interview. I know Martin and I did, not least the the the day in London afterwards.

But, we moved to sort of the second section, of the webinar today where the opportunity for you guys to ask questions about the topics we've covered today. I've also got a few questions in follow-up, which I hope everyone enjoy listening to the answers for.

But, firstly, we've got Petros, who's, asked us a question, two in fact. So we'll start with the first one.

So given the number of products and the need for quick incident identification, what were the lessons learned during recruitment and onboarding of the operations team that you mentioned in the discussion, Martin?

Martin Norato Auer

35:46 - 37:39

That's a really good question. I like it, and I had to think a bit about it.

And then so I I would say, first of all, I personally hire attitude over knowledge. So if someone has a good attitude, like, is willing to learn, is is open to to explore.

I would prefer a person, over someone who already has all the knowledge, but that's just personally me. Because a curious person is always willing to learn, to explore, to go new ways, And that is something I I personally like.

And there are some people in my team have a different opinion, and they can understand because the senior person, you don't have to train. You don't have to spend so much time in it.

So that's always a bit of a discussion. Then once hired, I would say a very important, thing is is a buddy system.

So really bringing people together. The new trainer has to have someone who can ask any question.

But also that means you need to have an open culture. Right? Because if people are afraid to ask questions, then it it slows down the whole thing.

And one thing that is often forgotten is, yeah, just listen to these newcomers. You you do things for years for such a long time.

You're so focused on, yeah, this is how we do it. And then a newcomer says, why are you doing this? It makes no sense.

And then you say, yeah, actually, you're right. Like, why are we doing this? And then you start to improve just because you get new insights in.

So I don't know if that's really what you're looking for, Petros, as an answer, but I would say, yeah. So body system is super important, and then also use the newcomers to improve what you're doing.

Martin Norato Auer

37:39 - 37:41

Great. But,.

Nick Homan

37:41 - 38:09

I mean, if currently, if we've got anybody who's looking for a job in Martin's team over the next few years, they know exactly what to to say in the interview, so there's some other hints there as well. Petrush has also followed up, just asking if there's any examples, grassroots examples of innovation through the new horizons kind of process you mentioned, that 10%, sort of bad time you like to give the team to think about new things.

Martin Norato Auer

38:09 - 40:09

Yeah. So I mentioned in the video the the mobile version of of Marvin where we have this, overview, and that started, I would say, three years ago already where we had some students in the team.

And when I have students in the team, I always try to give them stuff where they can really are open to explore and use, like, all the creativity possible. And we came to that idea that, yeah, let's do something on on the mobile somehow.

And that first version actually failed miserably, so we we just stopped development after the students left. And then a new set of students came in last year, and they said, hey.

I I saw you had this in the past. We have an idea.

Let's do it. And it really became something that is is is super useful.

But, yeah, we we didn't know if that really will be something that, like, also people will accept. Right? And so, yeah, this is something this is an example.

We also tried, like, I don't know, to build certain dashboards on can we get more insights on business values? And so, by that also died in the process, I would say, because there's just better tools within within SAP to use so you don't have to work around what's already there. Yeah.

So whenever someone has an idea, I think Sonar was also one of these, ideas when the beginning were like, can we really make use of it? It was was in a very early stage when Catchpoint released it. We explored it.

I mean, today, it's part of our day to day. It's part of core business.

‍

‍