Learning from ‘Frequent Flyers’: The Valuable Contribution of Repeat Respondents in MD Research
Debbie Kossman, Ph.D.
The large ‘silent majority’ – and the vocal few who speak for them
Anyone working in marketing research today is painfully aware that participation rates are lower than ever before. The problem is even more pronounced with physicians, whose response rates now barely reach 1%. Declining participation not only increases the cost and logistical complexity of research; it raises the even more troubling specter of non-response bias at a time when clients are increasingly concerned about representing only, or primarily, their target customers. A relative handful of physicians are providing critical feedback on commercialization strategy, giving us insight into market adoption patterns, and helping us map clinical pathways.
This begs a critical question: Can a tiny minority of physicians speak for all the rest? The answer is: They must. Even regulatory authorities (who require survey data for fulfillment of the REMS post-market surveillance mandate) have been forced to rely on a willing few to attest to the knowledge levels and practice protocols of the population. There is simply no going back to the halcyon days of broad research participation.
Among oncologists, a critical target for new product research, principal reasons for non-participation have been documented as lack of interest in the topic, lack of conviction about its importance, doubt about the value of their own contribution, and inadequate compensation.* These same considerations are mentioned anecdotally by physicians in all specialties – along with limited time and the desire to devote whatever scarce time they have to activities they find psychologically rewarding – not just financially rewarding. (It’s worth noting that “fair market value” guidelines used to dictate market research honoraria are typically keyed to reimbursement levels for clinical activities, but the calculus for “adequate” compensation may be quite different for physicians when they consider participation in market research.) In other words, there are many physicians whom we cannot pay enough. So what about the others?
Who are these people anyway?
We already know about why physicians won’t participate – though, of course, intractable non-responders tell us nothing at all – but what can we say about why others do? Of course, all expect and value compensation, but some are also rewarded by the opportunity to learn about new therapies in development (or look behind the scenes at the commercialization process). Some also like the challenge of reflecting on how they practice and articulating a point of view about it.
NAXION’s own analyses of tracking survey data suggest that respondents who participated in only one wave of research are not systematically different from those who participated in multiple waves over a period of years. They are no less clinically active, no less likely to adopt new therapies, no more likely to give high ratings.
And anecdotally, our experience with “regulars” in qualitative research suggests that they are no less sophisticated as a group (some are even more so) and no less occupied with matters of patient care. This news is heartening.
Up close and personal: a new sightline on the dynamics of change
Given the challenges of securing physician participation, researchers have developed best practices for achieving representative samples. But ultimately, our industry must rely on a relatively small group of proxies to speak for all their colleagues. So how can we wring even greater value from those respondents? Focusing especially on qualitative research (where the conversations are rich and fluid), might it be possible to develop an even more “nourishing” broth from insights contributed by those who make themselves more consistently available?
There are, in fact, certain very real advantages to repeated contact with “frequent flyers.” One virtue: familiarity breeds not just comfort, but added candor. When a physician and an interviewer get better-acquainted over time, social trust reduces the impulse to posture. An adept interviewer should be able to overcome whatever facades people erect – even in one-time encounters – but it’s also true that more frequent interactions embolden interviewers and relax respondents. The result can be a different, more authentic conversation.
Under the best of circumstances, interviews with repeat responders can provide a longitudinal perspective – not just a cross-sectional one. That allows us to follow changes over time at the individual level and, through that lens, observe the dynamics of market-level change. This type of “collaboration” between researchers and regulars has features in common with ethnography, which treats individuals as richly elaborated case studies exemplifying broader cultural trends. Repeat respondents often become more thoughtful and self-aware, better able to anticipate what may be relevant to us and contextualize their observations. As an added bonus, they are often more willing to participate in formal ethnographic studies – and even allow us to connect with their patients. In all these ways, an experienced, engaged respondent can become a valuable resource.
Of course, most conversations with physicians take place in the context of stand-alone, proprietary assignments, placing practical limitations on the extent of longitudinal analysis. Over the past decade, however, NAXION has conducted a series of self-funded interviews with a group of “frequent flyer” oncologists, which leaves us free to explore and share topics of general interest to our industry. Indeed, these repeat participants have become a de facto panel whose changing attitudes and practice trajectories we can follow on a prospective basis.
Making the best of a good thing
It’s getting harder and harder to persuade physicians to talk to us, but with the mounting challenges of commercialization, we need their input more than ever to improve odds and maximize ROI. Since we have little hope of increasing response rates, we must make effective use of the scant resources we have. That means relying more heavily on qualitative research for close-up perspectives on the customers we care most about.
In market research, representativeness is not just about sampling error, it’s about quality of respondent contributions. Thoughtful, articulate “regulars” are the best resource we have. We need to cherish, not devalue, them. Stone soup really can be a nourishing broth if properly prepared.
Three Tips to Maximize the Value of Research with 'Frequent Flyers'
- Ask ‘frequent flyers’ what they see as the range of views and practices among their colleagues and invite them to hypothesize about reasons for differences they observe among peers.
- Turn into the skid: Consider empaneling a group of ‘frequent flyers’ for the explicit purpose of following trends over time in an in-depth, recursive way.
- Create a ‘do not recruit’ list of respondents – frequent or rare – who can’t or won’t engage productively with thoughtful answers in the research setting. That’s not a good use of a research dollar.
Author: Debbie Kossman, Ph.D.