Podcast: Why your business should experiment more

The best way to generate new ideas and develop new offerings as a company is to experiment. The most successful companies always have one thing in common - they're keen to experiment with new initiatives and ways of working, even if they result in failure. Lots of companies would like to adopt this mindset - but when the daily pressures of business pile up, it's hard to do.

In this episode, I spoke to Anders and Stellan about their views. They're both big fans of experiments, and we conduct plenty on ourselves here at Zooma. Our discussion covered plenty of topics, but we spent most of our time talking about how Zooma uses experimentation to establish best practice, some of the success stories we've experienced, and how your company can start creating a culture of experimentation.

If this sounds interesting, you can listen and subscribe to The Onlinification Pod on the podcast platform of your choice. Below, you can find links to the most popular platforms, as well as the RSS feed. If you're really in a rush, you can also read the transcription of this episode further down. Enjoy!

Listening and subscription options

Apple Podcasts
Spotify
Soundcloud
Google Podcasts
Get the RSS feed
Subscribe to the blog
Watch the video episodes on YouTube

Transcription

AE: [00:00:00] So welcome to the show, Anders and Stellan.

SB: [00:00:04] Thank you.

AB: [00:00:06] Thank you, Alexander. Always a pleasure.

AE: [00:00:08] That's good. And how are you, Stellan?

SB: [00:00:12] I am very well. Thank you. And I'm looking forward to this episode and what you have in store for us.

AE: [00:00:18] Yeah. So today we're going to talk about the importance of experiment experimentation in business. And first of all, I would like to ask you, Stellan, how do you approach business experiments? What do we do?

SB: [00:00:37] Yeah. So a while ago, we decided that we have to, you can say in a way, lead by example, but it basically means establishing what we consider best practice in all areas where we advise customers. So based on our broad experience working across industries, across company sizes and so on, where we're building upon that and utilizing it and using Zooma as a sort of a playground in order to establish best practice. So that's what we call business experiments. I don't know if you want to add something to that, Anders?

AB: [00:01:21] No, but... That's a yes. We realized after a while that when we decide things, we need to prioritize them, and to prioritize them, we need to be more effective and more creative than we used to be. And there was an important distinction where we needed to say, sometimes this is an experiment to get to know if it's a best practice. So if we would have been a global B2B company, we could have done an experiment in Poland, or we could have done an experiment on the UK market or something else to know if we should roll it out as a best practice on comparable markets and that we do in a small scale with a lot of things. I'll give you an example, Alexander in a meeting a while ago, someone said, Should, shouldn't we do a promo video of this? And we did the promo video immediately, and then we used it for launch. And then someone sent an email saying, Hey, it works fine on mobile, but not on the browsers. And then we took it away on browsers and then someone said, I still think it looks a bit skanky on mobile. Let's take it away. The driver for this, the directly responsible individual, said, "Oh shit, I failed." "No, no, no. I was not clear on that. This was an experiment." "Ah, perfect, perfect. I should have known or good that you tell me now." We know how to do this quickly, to have high quality on it, and where not to put it when we do it at that speed. This was a recorded Zoom thing with a lot of people involved. And now we know. But important when you do business experiments or experiments, that everyone is aware that it's an experiment so that you dare to do it. I know that Stellan worked in a company, and in that company, they had an expression similar to 'test or die', which is a bit male-ish to formulate it like that, but very interesting, and we actually try to apply it on ourselves now.

AE: [00:03:48] Yeah. So Stellan, you mentioned that we do it in order to find best practice so we can guide our customers in it. But your previous company, the way you worked with that was a company that sold products instead of services. So how do you, what are the benefits for a product business if the motivation is not to test things that apply to customers?

SB: [00:04:18] I think as the world is progressively, let's say, moving ahead more and more rapidly, testing is going to be an essential part of operating a business. I don't know if we would have had Martin in the meeting right now, he would have said the exact statistic, but I think it's roughly half of the Fortune 500 companies that were on that list 10 years ago are still on the list. It might be less even, and that's I think it's a sort of a testimony that you really need to test and experiment continuously in order to be successful going forward. And I know Jeff Bezos used to say that if you have to bet your company, you're too late. And so essentially what he meant by that is that Amazon is currently or constantly running many, many, many small experiments, and many, many fail. But they don't cost a lot because they're small. But if you wait too long, you have to make a few big bets and then the risk is that you end up on the wrong side of that bet and you have no company. So I think all of that sort of goes in the direction of how important it is.

AE: [00:05:39] Yeah.

AB: [00:05:39] And one thing our listeners can never bet on is how long does it take before Anders or Stellan starts talking about Amazon and Jeff Bezos? And we had a couple of questions from our listeners, and I can say it officially now, no, they are not the sponsors of this pod. It's just that we like them.

AE: [00:06:06] That's good of you to clarify.

AB: [00:06:08] Very, very important to clarify.

AE: [00:06:14] So, you talk with a lot of different companies, and is everyone working to create a test and learn culture? Or how are companies approaching this, Anders?

AB: [00:06:30] I mean, we mainly work with the type of companies where this most likely is either a good way to start their ambitions and what they need to do. So that means, in phase one or what they need to begin with, or it's in step two, phase two, when they have put some foundation in place to be able to do things. So they do that after a while.

AE: [00:07:00] So Stellan, if our listeners want to get their leadership excited about business experimentation, where do you start?

SB: [00:07:13] So I think the one thing that you should do is to make sure that you explain, like we said before, why it is a necessity to experiment and that there are a lot of benefits of doing it. So for one, the cost of failure is very small, and that is something that business leaders often want to hear. So if you talk about sunk costs and so on, and no one likes a large sunk cost, and you can also move more quickly if you have a smaller project versus a larger project. Less people involved often means a quicker time to market. So that's also something I think businesspeople like to hear. And those are probably the main, the top two that I can spontaneously mention.

AE: [00:08:13] Yeah. And cost failure, can you define that?

SB: [00:08:18] By nature, experimentation means that you are sometimes right and you're sometimes wrong. So the smaller the experiment is, if it's a wrong bet, the less it costs. And so therefore, it's a sort of huge advantage to run limited tests. So, for example, if you're experimenting with your own e-commerce player and you're experimenting with a pricing strategy, for example, then doing that on one market or doing that on a certain product segment or a subset of products or anything is better than to change your whole pricing model and make a bet that this is better than the one we had before.

AE: [00:08:59] Yeah.

AB: [00:08:59] And I can mention that I spoke to a company a couple of days ago that asked for our view on how they should approach a very huge challenge for them, how they should approach that, how they should solve that. And when we have been discussing a while, he said to me, "But I have talked to so many people," and if I generalize this a bit, "they have all been advising us to do a pre-study, a strategy, steering documents. This and that. They all talk about periods of six to 12 months to build something to be able to solve the situation." So we said it's sort of two-year suggestions, but I know this decision-maker quite well. And he talked with me and asked for a second opinion, and I told him, "I would suggest that you spend four hours together with other people that have knowledge about how it is to be your customer, and decide what is true. And then you do this and then you do that. That's month number one." So he asked, "Why do you say this?" And I said, "Because I know the culture in your company. I know, as in all companies, the amount of workload on everyone. I am certain that you will fail with the sort of Big Bang rollout approach to solve things, running a massive project instead of doing these things in the beginning." And then, he said, "Can it actually be that pragmatic and easy?" And then I said, "Let's call it the next experiment. Let's do this for one month. And I am certain that it will be a natural part of how you do business. It would be a complement to that. And if you fail, nobody knows. It's just a few people involved and you can start a couple of things." And he said, "Oh, that was a good name. Is that commonly used, to do an experiment?" "I don't know," I said, "but let's do this experiment, and I'm certain that we will move what you do closer to solving the challenge in one month than what you would do with a sort of, sad to say, 'traditional' approach for two years, that will increase the workload and some massive investments, but most likely, no, you don't solve the challenge."

AE: [00:11:38] Yeah. And Stellan, like what is it important that you have in place when you start an experiment? How do you go back and evaluate where it has failed or whether it's successful?

SB: [00:11:54] Well, you have to have a person that is clearly in charge of the experiment. And there also has to be a very sort of defined set of expectations for the experiment. So what is the what is it that we're trying to solve and what is the predicted outcome of the experiment? And then you measure your success against that. So if it is to, you know, as I said before, run a pricing strategy experiment and the predicted outcome is that you increase your average order value, for example, or you increase your conversion rate, then that's what you measure against. You should never run an experiment if you don't have a clear theory of why you're running it and what the predicted outcome is, then it's just shooting from the hip. And that is not an experiment.

AE: [00:12:45] And Anders?

AB: [00:12:47] In a small scale, you could say that, for example, doing A-B testing is an experiment. It's very easy to do, and usually if you do it right, gives you again on a low level or small scale, much better conversion if you do it right. So, so even to have that within your sort of way of working instead of mumbling about, do we do waterfall, or shall we be agile or all the other mumbo jumbo expressions? I think that's a good way of looking at, for example, something like A-B testing. That's an experiment in my world. I don't know what you say, Stellan.

SB: [00:13:32] Yeah, of course it is. But you could say that A-B testing is a strategy that you can use while you're experimenting.

AB: [00:13:42] Or a tactic?

SB: [00:13:43] Yeah, it's a bit of semantics. Yeah, but yes, for sure. A-B testing is a really important component.

AB: [00:13:52] And we can be open about that, Stellan, that we actually love semantics. We do.

AE: [00:14:00] And do you have examples of other components if A-B testing is one?

SB: [00:14:07] Well, another one is using comparable periods. So if you're running an experiment with two weeks, then you can compare with the two weeks prior. For example, if you compare with a longer period, the risk is that you're comparing with something where something external happened that would affect the results in your test. So you want to limit sort of that time period. You can also use it to calculate, or sort of a classic way to do it is to calculate delta changes so you can calculate, you know, what was the change from the previous period to the testing period and then you can compare it to the previous year. So if there was a change in the previous year as well, then you can see if your change now was bigger or smaller than the change last year. So for example, if you're running a test over the Christmas period and you have to, you happen to have like a general peak in demand towards the end of the year. And then sort of your experiment could show a false positive. So if you then compare it to the year before and look at the deltas, then you can see that well, the uplift was actually smaller this year than last year. So that could then give you a better sense of whether it is a positive or a negative experience experiment. That's just an alternative when A-B testing isn't a suitable way to do it and you have to revert to that type of tactic.

AB: [00:15:31] And I can say that what you as a listener are listening to now is an experiment. This is not a best practice. This is actually, a while ago, Alexander telling someone, "Oh, I hear so many nice conversations internally. Should we do that in a pod?" So, so we started a pod. We do not claim that this is best practice. We do this through Zoom. We know that perhaps you have a nice voice in the ears of people's AirPods, Alexander, based on that, you use equipment now that is nice. And in a while, this will not be an experiment anymore. Then this will be a best practice so that when we advise customers to do things, another angle of experiments connected to the pod is that we don't know what you will ask us. So today we met one minute before we started recording and you said, "I'm thinking that we should talk about business experiments today", and we said, "OK, let's talk about business experiments." That's always, also a way in how we experiment because we don't want to be perceived as people who sit and read from the screen. We want to be perceived as who we are.

AE: [00:16:54] Yeah, yeah, and I mean, at Zooma we run a lot of experiments and I know that you both experiment yourselves privately. I see Anders, for example, trying new things on LinkedIn or I see a notification on the email that we have a new integration in HubSpot and so on. And before we end I just want to ask you, both of you, we can start with you, Anders, do you have any major learnings to find recently? Could be whatever?

AB: [00:17:28] I'll have to think a couple of seconds, if Stellan has something popping up immediately, give it to him.

SB: [00:17:35] I think what I've learned over the last week is that, sort of general, more fundamental things in how to do things online is sort of quite solid. It's not moving a lot. The perfect, the purchase journey that someone goes through, for example, you want transparency about capabilities, you want proof, you want to get guidance, you want relevance. So that sort of devil is in the details in how you do that. And that varies between, you know, companies and industries and so on.

SB: [00:18:22] But it's very clear when you're running experiments that if you go sort of outside of that, then you fail in a way.

AB: [00:18:34] And I would say after thinking about it, because why I use the technique to pass it over to Stellan is because I had to think through if I should tell what the main learning from experiments was connected to LinkedIn. But I'm not going to tell that, but I'm going to tell one thing, that Doug, Martin and I had a meeting about, where we were about to find a printer for our book. Then in the meeting, when we looked through all the contacts, I at the same time published on LinkedIn and asked people for a printer of the book, and had a lot of very, very good answers that Doug is following up. Everything from links and connections to people that knew people, to actually experiences of using a specific supplier. And I don't know if all of you in the meeting knows it, but because I don't think I have time to tell you. But we actually had an offering from someone who seemed to like Zooma, that five years ago heard a keynote that I had, and he offered the whole printing for free as long as we could find a bookbinder, because he said he was inspired by the keynote one year after he heard the keynote. So he offered us that, so we'll see how we solve that. But thank you. You know who you are, who offered it. So that's nice thing. So that was an experiment that took some different directions, and at least made the four of us now through the video smile.

AE: [00:20:17] Great. Well, thank you very much for this episode.

AB: [00:20:22] Thank you, Alexander.

SB: [00:20:24] And see you.

AE: [00:20:25] See you. Bye bye.