Ernesto Kahan, MD, MPH

Physician input when designing interventions for the promulgation of clinical practice guidelines (CPGs) can be an important component of the formulation and implementation process. Focus groups, a form of group interview that capitalizes on communication and interaction between research participants to generate data and gather information, can be a useful method for acquiring this potentially vital feedback. During focus groups, participants are encouraged to converse with one another: asking questions, exchanging anecdotes, and commenting on each others’ experiences and points of view.

 Consequently, of primary importance when conducting a focus group is creation of a permissive, comfortable, and nonjudgmental environment that fosters self-disclosure and encourages participants to share perceptions and points of view.

 Development of techniques the moderator can use to stimulate conversation among the participants is essential for conducting a successful focus group. 

In this study we investigated the feasibility of using interactive electronic-polling technology and nominal group technique (NGT) to present nationwide prescribing patterns for the purpose of initiating group discussion among primary care physicians. (For descriptions of these techniques, see the Methods section.) Brezis and Cohen have described their experience with these systems in teaching both students and physicians and have concluded that “this modality improves the quality of clinical learning and deserves further evaluation.”

 Because this modality facilitates anonymity and self-disclosure, we hypothesized that it could be used by researchers to attain the qualitative research objectives of focus groups.

As drug utilization studies conducted in the Leumit Health Fund of Israel (Leumit) have consistently shown low rates of physician adherence to Health Maintenance Organization (HMO) guidelines for use of antibiotics in the treatment of acute cystitis,

 we chose implementation of guidelines for the choice of drug to treat uncomplicated cystitis in adult community-dwelling women as the test case for this novel modality. We hypothesized that presenting prescribing data to small groups of physicians would elicit group discussion that would be beneficial in furthering our understanding of the underlying factors behind the prescribing behavior observed.

The primary research objective of the focus groups was to gain insights from practicing physicians concerning the observed nonadherence to the HMO’s guidelines, to learn why past efforts to promulgate HMO policies have failed, and to learn what tools and methods might be more successful in the future. The major themes ascertained from the focus groups would then be used to formulate a questionnaire to survey a large sample of HMO primary care physicians.

All primary care physicians practicing in clinics operated by the Leumit Health Fund in 3 different regions of Israel were invited to attend focus groups to discuss the implementation of guidelines for the treatment of infectious diseases. The physicians were representative and all saw a substantial number of cystitis cases. Similarly, because all invited physicians were full-time, salaried employees of the HMO, their knowledge levels and attitudes were relevant to our research objective. The sessions were conducted in a quiet room with the participants seated in a semicircle facing the screen with the presentation. The sessions were partly conducted during regular working hours or during the afternoon break immediately after the clinics were closed. A buffet lunch was served. 

The nominal group technique is a semiquantitative evaluation method implemented in a group setting that combines both qualitative and quantitative elements of data collection. It encourages members of the group to contribute their individual thoughts about an issue of common interest, and during an NGT session all participants have equal say in generating and rank-ordering of ideas and suggestions.

 Furthermore, because NGT sessions are driven by the participants and not by the perceptions of the researcher, the influence of the researcher and the effect of group dynamics are minimized.

Interactive-voting systems are a technology that is implemented in a group setting to induce audience participation. As the lecturer poses multiple-choice questions to the audience via a PowerPoint presentation, each participant responds by pressing the number key on a remote control electronic keypad corresponding to their chosen answer. A computer-based device retrieves and processes the responses. When cued by the moderator, the system displays the distribution of answers as a histogram. This immediate feedback to the lecturer and audience can then be channeled to initiate group discussion concerning possible explanations for the results observed.

The questions for the sessions appear in the 

). First, the moderator (NRK) presented a clinical vignette describing a case of a young woman diagnosed with simple uncomplicated cystitis. The participants were then asked via a multiplechoice question which antibiotic they would prescribe for the empiric treatment of this patient and the distribution of their answers was presented as a histogram, as described above. Although a trained moderator is generally a good idea, the structure of these focus groups with the NGT worked well. Also, the moderator had studied qualitative research methods with one of the authors (DPC), who teaches the course in qualitative methods at the Hebrew University Hadassah School of Public Health. The moderator’s familiarity with the organization and the data used in the focus group discussions contributed to the success of the process.

Second, the moderator asked participants via multiple-choice questions what they believed to be the current prescribing and antibiotic resistance patterns nationally in the HMO; again the distribution of answers was presented as a histogram. Afterward, the moderator presented the results of drug utilization studies and uropathogen resistance data that report the current patterns for this HMO’s patient and physician population. This immediate feedback then generated group discussion conducted with an NGT process previously described by Glasper et al,

 modified by the research team to be applicable to our research objectives as follows. 

Steps in the Nominal Group Technique Process

Step 1: Silent Generation of Ideas in Writing. 

The first question opened for group discussion was why do you think Leumit physicians did not generally comply with the guidelines for the empiric treatment of uncomplicated urinary tract infection in adult women? The participants were requested to write on index cards what they believed to be the 5 major reasons that HMO physicians did not generally adhere to the guidelines. 

The participants were asked in turn to present the first point recorded on their index card. Participants were instructed to say “pass” if at their turn they no longer had any original ideas to present. The moderator recorded each original contribution on a paper flip chart. This was done until the participants had exhausted their ideas. During this stage conferring and discussion was not permitted.

After all ideas had been recorded on the flip chart, the statements presented were discussed and clarified until the moderator perceived that the participants adequately understood the ideas. Similar ideas were collapsed into a single item to avoid duplication and dilution of impact in the subsequent voting stage.

The participants were asked to choose and rank what they thought were the top 5 ideas recorded on the flip chart. On the reverse side of the index card used in step 1, the physicians anonymously recorded in descending order of importance the 5 ideas (out of the 5-8 remaining after collapse) appearing on the flip chart that they believed best addressed the issue under discussion. Upon completion of this step, the moderator collected the cards.

The ideas listed on the cards were assigned a numerical rank order of 1 to 5 according to descending value of importance. The rank allotted by each participant for each item listed on the flip chart was tabulated. Items not scored by the participant were given a value of 6. Sums of the ranks allotted for each item were calculated. The rank order of the items was evaluated from this aggregate data. 

To prevent sessions from exceeding 90 minutes in length, vote tabulation was not conducted with the participants. Because exhaustive discussions already had been conducted at a previous stage to evaluate and clarify the merits and disadvantages of each idea, we saw no need to implement an additional time-consuming discussion session to evaluate the group ranking of the ideas. The purpose of this study was to identify the major themes to be later evaluated quantitatively via a survey of a large sample of physicians, not to calculate their relative ordinal value at the time of the sessions.

This process was repeated during a second discussion stage in which the participants were asked the following question: What techniques do you think would be most successful in the future for implementing clinical practice guidelines in the Leumit Health Fund? During this NGT process, the flip chart page with the ideas generated in the previous round was left on display. Participants were instructed to generate ideas for implementation that in their opinion would be effective in overcoming the barriers identified in the previous round of discussion. 

The sessions were taped and listened to in full. Supplementary information derived from the discussions that might enhance our understanding of the data captured on the flip chart (eg, anecdotes from routine practice, discussion during the electronic voting phase) was documented and analyzed.

The major findings of this study have been reported previously.

 Three focus groups comprised of 6, 10, and 15 physicians were conducted, each lasting approximately 90 minutes.

The results from the first round of discussion concerning barriers to guideline implementation are shown in 

. The 3 sessions generated 8 different ideas explaining the lack of adherence to the guidelines as follows: 

Drug of Choice Periodically Out of Stock. 

Over the past 4 years the supply of nitrofurantoin in Israel often was sporadic, with supplies often being depleted for extended periods of time.5 This point was raised during all 3 focus groups as contributing to the lack of adherence even during periods when the drug was in stock.

Similar to the findings of previous research,

 the physicians in all 3 groups attributed the prescribing behavior to lack of knowledge of the content of the guidelines. Additionally, the participants noted that they themselves were unaware of the current uropathogen resistance patterns as presented in the first part of the session by the moderator. 

Disagreement with the recommendations of the guideline was suggested by 2 of the groups to explain the findings. The major points of contention raised during the open discussion were that unlike ofloxacin, nitrofurantoin often causes gastric side effects and that patient adherence with ofloxacin is superior because it is taken twice daily as opposed to nitrofurantoin, which is taken 3 times daily.

One group attributed the results to what they described as an organizational flaw whereby physicians do not receive feedback concerning their prescribing habits. This theme was later developed and expanded in the second round of discussion.

One group raised the issue of patients who were treated in the emergency rooms of hospitals during hours when HMO clinics are closed, received a 1-day supply of the drug prescribed, and then later visited a Leumit physician to get an HMO prescription for the rest of the course of therapy with HMO coverage. The physicians claimed that because of this scenario, in many cases their prescriptions do not reflect their own clinical judgment. One physician expressed his aggravation with this situation by proclaiming that he was “fed up with being reduced to being a rubber stamp.”

Similar to the findings of previous studies,

 physicians in 2 groups attributed the patterns observed to a general proclivity not to change prescribing habits. The periodic shortage of nitrofurantoin exacerbated this situation because it conditioned physicians to prescribe other dugs, particularly ofloxacin. 

Physicians in 1 group articulated a general disenchantment with guidelines and with the pretense that one can describe a “usual” case to be treated in the clinical setting.

The results from the second round of discussion concerning barriers to guideline implementation appear in 

. The 3 sessions generated 7 different ideas for promulgating and implementing guidelines in the future as follows:

An idea suggested during all 3 sessions was that during a patient visit, the electronic patient record program could display the drug of choice when the diagnostic code was entered. 

Periodic, retrospective auditing of individual adherence to guidelines was suggested during all 3 sessions. Suggestions for the content of this feedback included current prescribing patterns and uropathogen resistance patterns. The participants did not believe that information regarding the costs of the different drugs would contribute to changing prescribing habits.

The technique of individual educational visits similar to those of drug representatives was suggested by the moderator and was considered to be a viable option in all 3 groups.

It was consistently suggested that small-group discussions similar to the NGT sessions themselves would be effective in conveying to physicians the rationale behind the guideline and would therefore be successful in changing prescribing practices.

One group surmised that because considerable portions of the nonadherence were due to shortages of the drug, ensuring a steady supply of nitrofurantoin would be sufficient to improve prescribing without the need for remedial interventions.

During 1 session the idea of inviting physicians to an academic lecture presented by an expert in the relevant field was suggested.

Leumit conducts a multisite continuing medical education program for primary care physicians. One group suggested that a lecture delivered in this format devoted to the subject of treating urinary tract infections would be beneficial in improving prescribing precision.

Sessions that used NGT reinforced with electronic voting technology were successful in actively involving small groups of target population physicians in the managerial decision-making process about how best to implement CPGs. This methodology enabled achievement of the objective: to facilitate presentation of population-based drug utilization data to practicing physicians and to elicit their interpretations of our findings. In view of the active and productive physician participation achieved during these sessions, the HMO studied may want to consider incorporating this methodology into its routine policy formulation processes. The combination of focus group discussion and NGT promoted less inhibited discussion than might otherwise be the case in a bureaucratic organization characterized by explicit hierarchy and chain of command.

Additionally, a major advantage suggested by our study is that although NGT was used by a group experienced in conducting focus groups, it appears that NGT also can be implemented without enlisting specially trained moderators or extensively training in-house personnel. The technique is likely to be applicable to other settings because it can be implemented by an in-house facilitator who is familiar with the data and the study objective. We therefore recommend that when feasible (as in this study), the primary investigator who evaluated the relevant prescribing data should function as the moderator.

The electronic voting technology proved highly effective for initiating group discussion. Being anonymous, it reduced reticence associated with exposing one’s opinions to peers, thereby enabling access to physician attitudes and knowledge levels. The method also allows for identification of knowledge gaps and promotion of awareness regarding misunderstandings concerning current clinical practice. 

The method described here at length has been proven to be useful for generating a quantitative study, one of the accepted roles for qualitative research. The utility of the survey designed on the basis of the focus group results has been reported elsewhere.

 Focus groups conducted with this method have been effective in generating information relevant to further study and management.

The ideas generated during the sessions about the reasons for suboptimal adherence to the guidelines as well as the suggestions for future efforts provide important information concerning these physicians’ perceptions of the organizational environment in this HMO. Although most of the ideas generated during the sessions have been well documented in the literature, these findings are important because they were obtained in a closed in-house managed care model. In attributing the nonadherence to lack of knowledge of the guidelines, these physicians are essentially identifying a flaw in the medical-managerial process, which failed to fulfill its obligation to transfer to them clinically relevant knowledge. This point is further emphasized by the persistent suggestion of feedback, indicating that these physicians perceive that dissemination of knowledge and monitoring of practice are legitimate and necessary practices in this organization. These findings are surprising because they indicate that these physicians accept managerial auditing of prescribing as a quality improvement measure and not as an invasive, paternalistic intrusion that suppresses the volitional nature of clinical practice.

Nominal group technique reinforced with electronic voting technology to stimulate conversation is a feasible technique for conducting focus groups with primary care physicians. Drug utilization data were successfully incorporated into this modality, which facilitated physician interpretation of the current prescribing patterns of the HMO studied. These findings will serve as the basis for quantitative surveys and subsequent design and evaluation of intervention strategies. This technique shows promise for becoming an integral part of the guideline formulation and dissemination process in the future and may be beneficial in other clinical settings.

Articles

Nominal group technique reinforced with electronic voting technology to stimulate conversation is a feasible technique for conducting focus groups with primary care physicians.