Our latest episode of the Question Authority podcast features Sam McNerney talking to us about strategic market survey research.
Listen below, or grab it wherever you get your pods. Want to hear more? Each new episode brings in the best & brightest marketers to teach DTC brands about the art & science of questions. Subscribe for more, and let us know who you'd like to hear on the show.
Transcript
Mitch (00:00): Welcome to Question Authority, where the best and brightest marketers, teach brands about the art and science of questions. Today we're asking about survey research with Sam McNerney.
Mitch (00:28): Hey Sam, how's it going, man?
Sam (00:30): I'm okay. Thanks, guys.
Mitch (00:32): So Sam, you're going to be talking to us about surveys, and I guess, in the more specific sense, a kind of audience research and buyer personas and kind of approaches in that sense, is that an accurate way to put it. The problems that you're solving for brands right now?
Sam (00:48): Yeah, I would, in a way, we're in the same business, helping them brands get feedback on their customers, but you guys are like building a scalable product and I'm doing ad hoc insight work. So right now, I kind of have like an auto body shop model, where I'm like coming to me with your problems. I'll fix you. I'll get you a good rate. I'll do it quickly. Like the auto body shop of survey-based research. I come from the creative agency world. So I was at publicist for four or five years, and this is like one of those big standard Manhattan based ad firms. So the clients are various P&G brands, Walmart, Citibank, some really, really huge accounts.
Mitch (01:35): You were kind of alluding to this epiphany you had about working with smaller or medium sized brands versus global brands. That's pretty much what you're talking about on that front.
Sam (01:47): Yeah. I'd be really interested to hear what your general perspective is on how smaller DTC brands, what they do well, and what they don't do well.
Matt (01:58): I think our goal is to make you in higher demand. Just get people to respect this kind of methodology or a bit more. I think you said it in your thing a bit too, and I'm not sure if you were alluding to this, but just how addicted people are to like paid media to Instagram and Facebook as an ad platform. And a question Mitch and I say is how well do you actually know your customers and does Facebook and Instagram know your customers better than you do? And I'd say like 99% of direct to consumer brands would say, "Oh yeah, they know our customer way better than we did." We just spend money and people buy and we calculate a row, like that's the whole game. So I think that's just getting harder and harder and harder.
Sam (02:39): Yeah. When I'm trying to do my homework and networking on people, I would potentially do business with. So in this small to midsize DTC world, like one of the first questions I ask is what data do you currently have? And there's always this like VP of Marketing, "Oh." I'm wanting to understand that a little bit better. But I think what they're trying to say is they just don't have as much as they want. Another big difference between my, the creative agencies in the world, there is at the end of the day, it's like, okay, what are the insights? What are the survey results? What do they actually mean in terms of like something I can change right now? When you work with like Charmin and P&G is just like huge, like lofty comms planning and you wouldn't get anywhere near their website or their products. So the hands-on approach is... I've really loved that.
Mitch (03:34): I think you asked earlier on about like, what have we seen in the space working with DTC brands and that's one of the obvious advantages, right? If you're a small to medium-sized brand and especially if you're DTC and you have that connection to the customer, the path from insight to action is just so much shorter and more direct. Right? That's definitely one of the distinct advantages, and I think that's even ironic, Matt, you probably back me up.
Mitch (04:00): I think that someone starts at DTC brand and you think like, okay, well clearly they recognize that a big part of the value here is that it's DTC. But a lot of times they don't. Maybe because they haven't been in the space, like you said, they haven't worked with global brands to understand just how much effort and expense goes into being a P&G brand trying to actually reach and interact with a customer and have an exchange. Like that never happens. It's just so difficult to do that despite the millions and millions of dollars in the ad budget, just because you're not DTC. There's that disconnect between customer and the brand.
Sam (04:37): It's first party data. Isn't really a phrase that's uttered in DTC offices all the time yet. It's probably uttered every other word in a large CPG company.
Mitch (04:45): Any particular studies come to mind that are just like, these are the kinds of insights and a movement that you get from proper audience research and applied behavioral science that you can't get from just third-party data and click and cookie data and stuff like that.
Sam (05:03): So I'll quickly tell the Charmin example, because it really kind of set off in motion, like me trying my hand foolishly probably at becoming an independent Insights person that specializes in survey based research. So this was for Charmin. This is really simple. We just asked people, "How excited would you be if your office work bathroom suddenly started stocking all the toilet paper with Charmin."
Mitch (05:34): That's interesting.
Sam (05:35): Yes. So deliberately removing two things normally measured controlling price. Do you want this? Do you want to own this? And how much would you pay for this? So it's like, "All right, let's take that away from them." By the way, this is all based on a true story, and it was my wife's idea to ask this question. Our office, twitched away from Charmin, cause our, like the pipes were old and they couldn't handle the two ply. And it was like... Yeah, like the office broke down. Like we couped.
Matt (06:05): Oh my God.
**Sam (06:06): So anyway, so that's the survey and you can replicate with other brands we put in Quilted Northern and Cottonelle, and you could see that people were most excited about Charmin. So it was basically like a cool kind of like brand planning exercise.
Mitch (06:23): Yeah.
Sam (06:25): So yeah, that led me "Man, that's just a survey." Like it's like an 80 year old research technology that doesn't have anything in the limelight, but that's probably just because a lot of the people that do surveys are like really analytical and I have background in stats. It's like, what if you throw like a Humanities person at that?
Mitch (06:48): Exactly.
**Sam (06:48): Or I always wanted to get the creative department to do the surveys. It's like, they would come up with sweet questions. Cause they wouldn't have to think about like biasing respondents and all those mechanics, which are at times really important. And then you kind of need to forget about them after a while.
Mitch (07:05): You know, it's interesting when you talk about why do we lead these questions or why do we reserve the question asking to the analytical scientists and researchers and not open it up to the creative team and people like that. People with more of a Humanities background, I feel like that could be a potential solution when you're trying to build a small brand and you don't have the budget for like a huge research team or even a huge marketing team, right?
Mitch (07:33): This just becomes part of the culture of understanding and working with the customer, right? Like whether you're in HR and you're thinking about diversity, whether you're in customer service and you're thinking about whatever metric that you've got to achieve. If you've got everybody kind of thinking about it from the standpoint of like, "Hey man, we're, we're DTC. We can talk to the customer. That's like a benefit we have." So, anyone who's working here, no matter what you're doing, like you can talk to the customer or at least you can channel yourself through the website or whatever property to the customer. And so that could be like kind of an interesting shift there, it's like not so much building out the R&D team or the analytics team or the market research team, so much as just ingraining that sense of it into culturally into the whole org.
Sam (08:22): So obviously I couldn't agree more. I'll give you one example here, to punch this out a little. It's something I was just doing for my own blog, like a side project. And it was a survey asking about people who are interested in peacock NBC's new streaming platform, because I noticed something in myself, which is like, man, I actually like peacock's content. I like The Office and EPL. I think it's a good price. I actually think it's a really good price. It's like $5 a month. And yeah, I don't want to sign up for this. Like what is preventing me? And it wasn't really subscription fatigue in the sense that it would be too much content. What it was is like at some point the internet is going to go down and I'm going to get logged out of this account.
Sam (09:14): And I'm going to have to remember my username and password and that's going to be really annoying. Like that's, that was it. So anyways, I designed this survey around that experience, cause I was like, this can't be me. Just be me. And so it was something, it was like two questions again. It was short. It was as people, "If you got peacock for free, would you be excited?" And then the other question was, "Do you think the $5 a month is a good deal?" Then I basically took everyone that said yes to each question, and was like, okay, why don't you get it? You just said that you think the content is good and it's a good deal. Like why don't you get it?
Mitch (09:53): What's the blocker.
Sam (09:54): This is not how you're supposed to do surveys, by the way. You're supposed to be like neutral and robotic.
Mitch (10:02): You don't follow up your question with, "Well, what the hell man?"
Sam (10:05): But it's like, why not? Anyways, I mentioned this, just to riff on your point about that was kind of just out there for the taking like a marketing person or an analytics person or the founder. They're all in a good position to ask those kinds of questions. And then when it comes to actually like running a survey, you need someone to actually do it, but that's the easy part.
Mitch (10:33): Now, in product development and everything, there's something kind of generally understood, and you pointed out that you need different sample sizes of audiences to achieve different things. When you're doing like quality assurance, you basically can't have enough people hitting the thing. Right. And testing it, but we're doing usability.
Mitch (10:51): Usually it's like 10 people will basically tell you what you need to know. And they will give you the direction that you need. And to your point, arguably, the same thing could be said, and this is what I was, you know, focus groups were always about. Right. We don't need a huge sample size to give us direction on how we should be pursuing this, the stories and the value props and things of that nature.
Sam (11:12): Sure. So it could be claims testing. What language is actually put on this package. It could be stuff that we riffed on earlier about positioning. Do people know what we sell it? As I said, is it clear that our product is a unique solution to a problem? Is that coming through?
Mitch (11:36): And Matt actually, you have a customer even using Fairing who, actually, totally changed what their core audience was based on just asking a question, right?
Matt (11:46): Yeah, exactly.
Sam (11:47): We keep hearing that story time and time again, where it's like, oh, we just implemented a simple survey and just ask one question and decided to then update all of our landing pages, all of our ad copy and everything. Just because we got this little piece of feedback that we're talking to the wrong customer. We keep hearing that week after week. So, it's definitely not going away. And I think the more that people start to value this, the more they'll going to put their kind of time, effort and resources behind it.
Mitch (12:13): And again, that's like the local maxima thing, right. Where you always got to remember, like you can be talking to the wrong customer and still do okay. If you keep banging on optimizing for that wrong customer, like you'll do okay. But you know, it could be in a totally different place if you would acknowledge that alternative reality exists, but you need to be able to talk to people to understand that.
Sam (12:34): So it's not so much tunnel vision, it's just like not having access to different perspectives.
Matt (12:41): Yeah.
Sam (12:41): So, yeah, Matt, that's cool. That one question stuff. I mean, obviously I love that stuff.
Mitch (12:45): Sam, I think of that in the same sense with your Question of the Week, Newsletters , and the book that you were putting out here, I always enjoy the power of the way that the question guests ask and when you think about the underlying rationale behind why you asked a question that way.
Sam (13:06): Well, book is a very strong word. First of all, as I told you, this is a quarantine project that turned into a PDF, that, then, met the marketer inside of me to call it an ebook.
Mitch (13:20): All right. So what should people be looking out for as far as best practices that you think they could... I mean, even if it's just one specific example or just any kind of guidance you'd want to give a brand that is working with you or just a general brand that's of a smaller, medium size an e-com brand, a DTC brand best.
Sam (13:41): Best practices for survey-based research, which is kind of the tool and the, my methodology is to just really like trying to ignore the standard research questionnaire, guidelines as much as possible. I'm talking about don't lead the participants or no double-barrel questions or...
Mitch (14:07): What does that just explain that to a lot of the folks who buy...
Sam (14:11): It's like, "Do you like McDonald's and its coffee." It's like, "whoa, you asked two things. Those need to be separate questions." It's like, "Okay." But it's not about that. I'd actually don't care the answer to that because I'm trying to get them to talk about their coffee routine.
Matt (14:33): Yeah.
Sam (14:34): So I'm getting them in the zone a little. It's like, "Hey, you know how you like McDonald's and the coffee there? Yes. Tell me more about why you like it." It's like, okay, now, now we're we're off and running. So it's like, the double-barrel question is like technically incorrect or something, but that incorrectness doesn't matter if you use it in the right way.
Matt (15:00): Yeah.
Sam (15:01): I do not know why this is not obvious, I guess. And maybe I should just pass along where I really have been in meetings with Walmart and P&G where there's like passionate conversations about whether that counts as a double-barrel question or not.
Matt (15:17): Yeah.
Sam (15:17): Really nerdy, like God.
Mitch (15:21): Understand why the rules or guide rails are there. And there's some legitimacy behind that. It was like, "Oh, well you don't ask a double-barrel question because X, Y, Z." Right. And it's like, okay, that makes sense. I'm going to go ask it now. I was like, as long as you're doing that under that model, you're okay. Like if you're just sitting there asking it because you don't know any better and you don't know why there's a potential flaw in doing so or why it might screw with your answer, then you're in trouble. But once you know what you're doing and why you're doing it, the value that you can get out of it is as huge. And actually, the McDonald's thing kind of reminds me of the whole "jobs to be done" thing right.
Mitch (15:58): And the milkshake, right. There's a lot of questions you're going to ask to get to that understanding of, someone gets a milkshake in the morning because it takes the entire trip to work to drink it, and it makes you feel good the whole way. It's like, you're not going to get that from asking like your standardized questions. Like that's a real dig into the insight of the user there that you have to be more exploratory about.
Sam (16:25): Yeah. It's a blank canvas and it's like kind of guarded by a bunch of Excel purists.
Mitch (16:45): All right. We're out of here. Thank you for listening, subscribing and rating the show. If you're a brand interested in strategic survey guidance, check out sammcmerney.com or catch him on Twitter @sammcnerney. You want to chat with Fairing? Head on over to Fairing.co .
Mitch (17:03): See you next time.
In our last episode we talked about Offline Attribution with Krystina Rubino , join us next time for Behavioral Science with Lilly Kofler.
