Siddhartha Sharan is working as a Senior Data Scientist at Microsoft. Before Microsoft he worked at Genentech, a subsidiary of Roche Pharmaceuticals and one of the largest Pharmaceutical/ Biotech companies in the world.
When he started at Genentech he was new to Marketing mix modeling (MMM). Today he is one of the most experienced MMM-profiles there is. Here he shares his story on how he got started, how he used MMM and his key learnings to succeed with MMM in a large enterprise.
How did you get into Marketing Mixed Modeling (MMM)?
I have an analytics and data science background. When I started working at Genentech there were very few brands actively using the results of Marketing-mix-modeling (MMM).
Genentech and other pharma companies I worked at preferred using test control (A/B testing) for individual tactics at a small scale. At Genentech, MMM was still a new analytical tool. There were very few ongoing initiatives at that time to better understand the impact of sales and marketing spend.
When the analytics team at Genentech proposed to use MMM, I was one of the first people to say yes. I raised my hand and said, “Okay, this is a great idea and I want to help with the modeling. So, I want to help explain this to the business partners.”.
I had very little previous experience with modeling. One of my strengths though is explaining technical concepts to decision makers. I think this is due to my background in both marketing and product analytics. After working with several business leaders in sales and marketing I could tell when something resonated with them or not. This increased their trust in the analytics team.
How do you work with MMM today?
Today, Genentech has been doing marketing mix modeling for almost three and a half years. About three years ago Genentech launched a pilot project to do Marketing Mix for all brands in their portfolio.
By looking at the model output and talking to key stakeholders (marketing team, finance team, and ultimately the leadership team) we could help the leadership team make decisions. They could more clearly see which brands the company should spend more money on and which not to invest more in. Based on their performance in the model.
After interacting with all types of stakeholders I realized that unless you have a buy-in from a very senior level you will face a lot of challenges in helping the organizations make data driven decisions, people will always question the results that you present. Including the numbers from your statistically significant models.
This is a natural reaction as sometimes the model outputs does not match their expectations, when being compared to their sales- and marketing performance reports.
How can you get a buy-in from senior stakeholders?
So, buy-in on the utility and impact of the model and suggested changes from senior level stakeholders is key. And for that, you need to be able to prove the value of those changes.
This is where I think the biggest challenge lies in Marketing Mix and other statistical models. Analyzing data and explaining the results are two different challenges. All the improvements that MMM can bring to a company are predicated on the willingness of the sales and marketing team to be open to understanding their performance and making changes when necessary.
For example, when working on marketing mix model of our brand at Genentech, we had approximately six months of sales and campaign data.
We built a marketing mix model which used AdStock for some high costs and high impact tactics (E.g. Sales team, industry conferences, etc.) and limited impact for some low cost and low impact tactics (E.g. Emails and website visits) in a multilevel regression model.
Using this model, I was able to show how the sales and marketing teams performed on their individual tactics. Using the model allowed us to benchmark our performance internally against similar brands as well as externally within the industry.
Are you also using multi-touch attribution (MTA) and how?
We did use MTA for measuring the impact of emails and website visits for our digital marketing team, but MTA is implemented in a very limited way in this industry.
As our products are prescription-only, most visits to our patient and physician section are informational in nature. We also don’t have the ability to track individual visits to be compliant with data privacy regulations.
What MTA does is measuring the number of users impacted at that time with digital media tactics and there is no statistically sound way to know if the users have been influenced in another way.
Relying on MTA might be dangerous
For instance, let’s say you look at a Consumer Packaged Goods (CPG) company like McDonald’s, you can have coupons, which are being redeemed at the point of sale, but how do you know that the customer who redeems that coupon was not also exposed to a television ad or an email campaign at that time?
There are many ways a customer can be influenced by either their surroundings or your marketing activities, and you can not tell what influenced the customer’s decision.
Attributing 100% of the sales to a single tactic, like the coupons, is giving it more credit than what it was actually driving sales. It also results in you neglecting the idea to spend resources on other campaigns that are also driving sales. All you know is that the coupon campaign brought customers into your store to buy stuff, right?
The problem with MTA is that it does not help you differentiate what is happening with other tactics at the same location and at the same level. That’s one of the problems analysts face with MTA since it’s not possible to have all the data which is touching/interacting with the user.
What generally happens is that you either allocate equal investments to all the tactics or you take the last tactic or the first tactic to be the most important one. You are not really getting a complete understanding of what’s happening at the overall level.
What is your best example of that MMM works?
The best decision made by the help of MMM is when we presented the results of our model when we had just launched the product. At that time there was a lot of concern around the overall sales team incentive compensation. We had a great launch and exceeded our targets. Both the sales and marketing teams took credit for achieving those results.
With the help of marketing mix modeling, I was able to show what percentage of sales could be attributed to the sales team, marketing team and individual tactics respectively. I was able to prove the utility of our sales team to our brand team.
The data showed that the fastest way to become front runners in the market was to increase the size of our sales team. Usually, analytics teams are not included in budget decisions, but thankfully because of the model which we created, we were included, and could really contribute. I am very happy about that.
What’s your best tip to succeed with MMM?
It’s really important to build a productive relationship with your sales and marketing team, where you can help them understand their tactic performance and correctly communicate the success metrics within their teams.
This is where most data science teams are inefficient, since modeling only accounts for a small part of the marketing mix modeling. Explaining the results is a much bigger part. And a part that requires communication and influencing as a key skillset.