Our latest episode of the Question Authority
podcast features Sam
McNerney talking to us about strategic
market survey research.
Listen below, or grab it wherever you get your pods. Want to hear more?
Each new episode brings in the best & brightest marketers to teach DTC
brands about the art & science of questions. Subscribe for more, and let
us know who you'd like to hear on the show.
Transcript
Mitch (00:00): Welcome to Question Authority, where the best and
brightest marketers, teach brands about the art and science of
questions. Today we're asking about survey research with Sam McNerney.
Mitch (00:28): Hey Sam, how's it going, man?
Sam (00:30): I'm okay. Thanks, guys.
Mitch (00:32): So Sam, you're going to be talking to us about
surveys, and I guess, in the more specific sense, a kind of audience
research and buyer personas and kind of approaches in that sense, is
that an accurate way to put it. The problems that you're solving for
brands right now?
Sam (00:48): Yeah, I would, in a way, we're in the same business,
helping them brands get feedback on their customers, but you guys are
like building a scalable product and I'm doing ad hoc insight work. So
right now, I kind of have like an auto body shop model, where I'm like
coming to me with your problems. I'll fix you. I'll get you a good rate.
I'll do it quickly. Like the auto body shop of survey-based research. I
come from the creative agency world. So I was at publicist for four or
five years, and this is like one of those big standard Manhattan based
ad firms. So the clients are various P&G brands, Walmart, Citibank, some
really, really huge accounts.
Mitch (01:35): You were kind of alluding to this epiphany you had
about working with smaller or medium sized brands versus global brands.
That's pretty much what you're talking about on that front.
Sam (01:47): Yeah. I'd be really interested to hear what your
general perspective is on how smaller DTC brands, what they do well, and
what they don't do well.
Matt (01:58): I think our goal is to make you in higher demand. Just
get people to respect this kind of methodology or a bit more. I think
you said it in your thing a bit too, and I'm not sure if you were
alluding to this, but just how addicted people are to like paid media to
Instagram and Facebook as an ad platform. And a question Mitch and I say
is how well do you actually know your customers and does Facebook and
Instagram know your customers better than you do? And I'd say like 99%
of direct to consumer brands would say, "Oh yeah, they know our customer
way better than we did." We just spend money and people buy and we
calculate a row, like that's the whole game. So I think that's just
getting harder and harder and harder.
Sam (02:39): Yeah. When I'm trying to do my homework and networking
on people, I would potentially do business with. So in this small to
midsize DTC world, like one of the first questions I ask is what data do
you currently have? And there's always this like VP of Marketing, "Oh."
I'm wanting to understand that a little bit better. But I think what
they're trying to say is they just don't have as much as they want.
Another big difference between my, the creative agencies in the world,
there is at the end of the day, it's like, okay, what are the insights?
What are the survey results? What do they actually mean in terms of like
something I can change right now? When you work with like Charmin and
P&G is just like huge, like lofty comms planning and you wouldn't get
anywhere near their website or their products. So the hands-on approach
is... I've really loved that.
Mitch (03:34): I think you asked earlier on about like, what have we
seen in the space working with DTC brands and that's one of the obvious
advantages, right? If you're a small to medium-sized brand and
especially if you're DTC and you have that connection to the customer,
the path from insight to action is just so much shorter and more direct.
Right? That's definitely one of the distinct advantages, and I think
that's even ironic, Matt, you probably back me up.
Mitch (04:00): I think that someone starts at DTC brand and you
think like, okay, well clearly they recognize that a big part of the
value here is that it's DTC. But a lot of times they don't. Maybe
because they haven't been in the space, like you said, they haven't
worked with global brands to understand just how much effort and expense
goes into being a P&G brand trying to actually reach and interact with a
customer and have an exchange. Like that never happens. It's just so
difficult to do that despite the millions and millions of dollars in the
ad budget, just because you're not DTC. There's that disconnect between
customer and the brand.
Sam (04:37): It's first party data. Isn't really a phrase that's
uttered in DTC offices all the time yet. It's probably uttered every
other word in a large CPG company.
Mitch (04:45): Any particular studies come to mind that are just
like, these are the kinds of insights and a movement that you get from
proper audience research and applied behavioral science that you can't
get from just third-party data and click and cookie data and stuff like
that.
Sam (05:03): So I'll quickly tell the Charmin example, because it
really kind of set off in motion, like me trying my hand foolishly
probably at becoming an independent Insights person that specializes in
survey based research. So this was for Charmin. This is really simple.
We just asked people, "How excited would you be if your office work
bathroom suddenly started stocking all the toilet paper with Charmin."
Mitch (05:34): That's interesting.
Sam (05:35): Yes. So deliberately removing two things normally
measured controlling price. Do you want this? Do you want to own this?
And how much would you pay for this? So it's like, "All right, let's
take that away from them." By the way, this is all based on a true
story, and it was my wife's idea to ask this question. Our office,
twitched away from Charmin, cause our, like the pipes were old and they
couldn't handle the two ply. And it was like... Yeah, like the office
broke down. Like we couped.
Matt (06:05): Oh my God.
**Sam (06:06): So anyway, so that's the survey and you can replicate
with other brands we put in Quilted Northern and Cottonelle, and you
could see that people were most excited about Charmin. So it was
basically like a cool kind of like brand planning exercise.
Mitch (06:23): Yeah.
Sam (06:25): So yeah, that led me "Man, that's just a survey." Like
it's like an 80 year old research technology that doesn't have anything
in the limelight, but that's probably just because a lot of the people
that do surveys are like really analytical and I have background in
stats. It's like, what if you throw like a Humanities person at that?
Mitch (06:48): Exactly.
**Sam (06:48): Or I always wanted to get the creative department to do
the surveys. It's like, they would come up with sweet questions. Cause
they wouldn't have to think about like biasing respondents and all those
mechanics, which are at times really important. And then you kind of
need to forget about them after a while.
Mitch (07:05): You know, it's interesting when you talk about why do
we lead these questions or why do we reserve the question asking to the
analytical scientists and researchers and not open it up to the creative
team and people like that. People with more of a Humanities background,
I feel like that could be a potential solution when you're trying to
build a small brand and you don't have the budget for like a huge
research team or even a huge marketing team, right?
Mitch (07:33): This just becomes part of the culture of
understanding and working with the customer, right? Like whether you're
in HR and you're thinking about diversity, whether you're in customer
service and you're thinking about whatever metric that you've got to
achieve. If you've got everybody kind of thinking about it from the
standpoint of like, "Hey man, we're, we're DTC. We can talk to the
customer. That's like a benefit we have." So, anyone who's working here,
no matter what you're doing, like you can talk to the customer or at
least you can channel yourself through the website or whatever property
to the customer. And so that could be like kind of an interesting shift
there, it's like not so much building out the R&D team or the analytics
team or the market research team, so much as just ingraining that sense
of it into culturally into the whole org.
Sam (08:22): So obviously I couldn't agree more. I'll give you one
example here, to punch this out a little. It's something I was just
doing for my own blog, like a side project. And it was a survey asking
about people who are interested in peacock NBC's new streaming platform,
because I noticed something in myself, which is like, man, I actually
like peacock's content. I like The Office and EPL. I think it's a good
price. I actually think it's a really good price. It's like $5 a month.
And yeah, I don't want to sign up for this. Like what is preventing me?
And it wasn't really subscription fatigue in the sense that it would be
too much content. What it was is like at some point the internet is
going to go down and I'm going to get logged out of this account.
Sam (09:14): And I'm going to have to remember my username and
password and that's going to be really annoying. Like that's, that was
it. So anyways, I designed this survey around that experience, cause I
was like, this can't be me. Just be me. And so it was something, it was
like two questions again. It was short. It was as people, "If you got
peacock for free, would you be excited?" And then the other question
was, "Do you think the $5 a month is a good deal?" Then I basically took
everyone that said yes to each question, and was like, okay, why don't
you get it? You just said that you think the content is good and it's a
good deal. Like why don't you get it?
Mitch (09:53): What's the blocker.
Sam (09:54): This is not how you're supposed to do surveys, by the
way. You're supposed to be like neutral and robotic.
Mitch (10:02): You don't follow up your question with, "Well, what
the hell man?"
Sam (10:05): But it's like, why not? Anyways, I mentioned this, just
to riff on your point about that was kind of just out there for the
taking like a marketing person or an analytics person or the founder.
They're all in a good position to ask those kinds of questions. And then
when it comes to actually like running a survey, you need someone to
actually do it, but that's the easy part.
Mitch (10:33): Now, in product development and everything, there's
something kind of generally understood, and you pointed out that you
need different sample sizes of audiences to achieve different things.
When you're doing like quality assurance, you basically can't have
enough people hitting the thing. Right. And testing it, but we're doing
usability.
Mitch (10:51): Usually it's like 10 people will basically tell you
what you need to know. And they will give you the direction that you
need. And to your point, arguably, the same thing could be said, and
this is what I was, you know, focus groups were always about. Right. We
don't need a huge sample size to give us direction on how we should be
pursuing this, the stories and the value props and things of that
nature.
Sam (11:12): Sure. So it could be claims testing. What language is
actually put on this package. It could be stuff that we riffed on
earlier about positioning. Do people know what we sell it? As I said, is
it clear that our product is a unique solution to a problem? Is that
coming through?
Mitch (11:36): And Matt actually, you have a customer even using
Fairing who, actually, totally changed what their core audience was
based on just asking a question, right?
Matt (11:46): Yeah, exactly.
Sam (11:47): We keep hearing that story time and time again, where
it's like, oh, we just implemented a simple survey and just ask one
question and decided to then update all of our landing pages, all of our
ad copy and everything. Just because we got this little piece of
feedback that we're talking to the wrong customer. We keep hearing that
week after week. So, it's definitely not going away. And I think the
more that people start to value this, the more they'll going to put
their kind of time, effort and resources behind it.
Mitch (12:13): And again, that's like the local maxima thing, right.
Where you always got to remember, like you can be talking to the wrong
customer and still do okay. If you keep banging on optimizing for that
wrong customer, like you'll do okay. But you know, it could be in a
totally different place if you would acknowledge that alternative
reality exists, but you need to be able to talk to people to understand
that.
Sam (12:34): So it's not so much tunnel vision, it's just like not
having access to different perspectives.
Matt (12:41): Yeah.
Sam (12:41): So, yeah, Matt, that's cool. That one question stuff. I
mean, obviously I love that stuff.
Mitch (12:45): Sam, I think of that in the same sense with your
Question of the Week,
Newsletters , and the book
that you were putting out here, I always enjoy the power of the way that
the question guests ask and when you think about the underlying
rationale behind why you asked a question that way.
Sam (13:06): Well, book is a very strong word. First of all, as I
told you, this is a quarantine project that turned into a PDF, that,
then, met the marketer inside of me to call it an ebook.
Mitch (13:20): All right. So what should people be looking out for
as far as best practices that you think they could... I mean, even if
it's just one specific example or just any kind of guidance you'd want
to give a brand that is working with you or just a general brand that's
of a smaller, medium size an e-com brand, a DTC brand best.
Sam (13:41): Best practices for survey-based research, which is kind
of the tool and the, my methodology is to just really like trying to
ignore the standard research questionnaire, guidelines as much as
possible. I'm talking about don't lead the participants or no
double-barrel questions or...
Mitch (14:07): What does that just explain that to a lot of the
folks who buy...
Sam (14:11): It's like, "Do you like McDonald's and its coffee."
It's like, "whoa, you asked two things. Those need to be separate
questions." It's like, "Okay." But it's not about that. I'd actually
don't care the answer to that because I'm trying to get them to talk
about their coffee routine.
Matt (14:33): Yeah.
Sam (14:34): So I'm getting them in the zone a little. It's like,
"Hey, you know how you like McDonald's and the coffee there? Yes. Tell
me more about why you like it." It's like, okay, now, now we're we're
off and running. So it's like, the double-barrel question is like
technically incorrect or something, but that incorrectness doesn't
matter if you use it in the right way.
Matt (15:00): Yeah.
Sam (15:01): I do not know why this is not obvious, I guess. And
maybe I should just pass along where I really have been in meetings with
Walmart and P&G where there's like passionate conversations about
whether that counts as a double-barrel question or not.
Matt (15:17): Yeah.
Sam (15:17): Really nerdy, like God.
Mitch (15:21): Understand why the rules or guide rails are there.
And there's some legitimacy behind that. It was like, "Oh, well you
don't ask a double-barrel question because X, Y, Z." Right. And it's
like, okay, that makes sense. I'm going to go ask it now. I was like, as
long as you're doing that under that model, you're okay. Like if you're
just sitting there asking it because you don't know any better and you
don't know why there's a potential flaw in doing so or why it might
screw with your answer, then you're in trouble. But once you know what
you're doing and why you're doing it, the value that you can get out of
it is as huge. And actually, the McDonald's thing kind of reminds me of
the whole "jobs to be done" thing right.
Mitch (15:58): And the milkshake, right. There's a lot of questions
you're going to ask to get to that understanding of, someone gets a
milkshake in the morning because it takes the entire trip to work to
drink it, and it makes you feel good the whole way. It's like, you're
not going to get that from asking like your standardized questions. Like
that's a real dig into the insight of the user there that you have to be
more exploratory about.
Sam (16:25): Yeah. It's a blank canvas and it's like kind of guarded
by a bunch of Excel purists.
Mitch (16:45): All right. We're out of here. Thank you for
listening, subscribing and rating the show. If you're a brand interested
in strategic survey guidance, check out sammcmerney.com or catch him on
Twitter @sammcnerney. You want to chat with Fairing? Head on over to
Fairing.co .
Mitch (17:03): See you next time.
In our last episode we talked about Offline Attribution with Krystina
Rubino
, join us next time for Behavioral Science with Lilly
Kofler.