HomePartners & Projects Evaluators' ReportsReport #4

Evaluators' Report on the National Demonstration Project (NDP)
to the Board of Directors of TransforMED

October 19th, 2007

by The Center for Research in Family Medicine and Primary Care
Elizabeth E. Stewart, PhD (NDP Qualitative Analyst)
Carlos R. Jaén, MD, PhD
Benjamin F. Crabtreee, PhD
Paul A. Nutting, MD, MSPH
William F. Miller, MD, MA
Kurt C. Stange, MD, PhD

Overview:

With just eight months to go in the National Demonstration Project (NDP), there are few signs of slowdown from the practices, facilitators, or the TransforMED administrative office. The Evaluation Team has expanded the scope of its research data collection by sending the Qualitative Analyst to the self-directed practices in an effort to gather qualitative data (field notes, observations, interviews) that will be used in conjunction with the many quantitative instruments already being collected (chart audits, patient surveys, clinician and staff surveys). Even in the early stages, this additional data collection is yielding information that is not only useful to the overall NDP analysis, but could be of potentially great value to TransforMED as it moves forward to formulate its future organizational structure and market its facilitation services.

I. When planning from data generated from the NDP, it is important to remember that the NDP practices are a highly selective group. Most practices selected to participate in the NDP were familiar with the Future of Family Medicine report, and many had already implemented several principles into their existing practice or built a new practice based upon these principles. Although many practices showed characteristic levels of operational inefficiencies and dysfunction, most had a vision of what they hoped their practice could be. We (the Evaluation Team) have decades of experience working with primary care practices and can say with some certainty that the majority of practices in the United States are not like the NDP practices. For the NDP to ultimately be considered successful, we and TransforMED will need to identify the specific facilitation needs of more typical practices that are often cash-strapped and despondent. Practices might be willing to pay some money to achieve some progress, but most cannot afford intensive facilitation such as that seen in the NDP. Taking TransforMED to scale will require a range of differing strategies.

In addition to being a highly selected sample, the practices of the NDP are responding to tremendous external motivation. Even some of the self-directed practices noted that simply being in the NDP with the national spotlight pointed their way gives them an extra boost to push forward with change efforts. It is hard to know what will happen when that national spotlight disappears. Some practices may backslide, others may have enough momentum to keep going, fueled simply by the desire to improve and potential results. Follow-up of the current NDP practices is critical for determining the extent practices will still embrace change when the major external motivators are removed.

II. The self-directed practices are not a control group; they represent a comparison group with tremendous potential for TransforMED learning as they independently pursue their own paths towards creating practices of the future. We were fortunate to meet with most of the self-directed practices at their self-organized retreat in June 2007 and we are in the process of visiting each practice to gather qualitative data to complement the existing quantitative data. These additional data will help us to better understand the differences between the self-directed practices and facilitated practices, which in turn will increase understanding of the role of facilitation in practice change. The points below outline the learning derived from the self-directed retreat and the seven practices visited thus far. These points will likely change as more practices are visited.

A. The self-directed practices were not familiar with the TransforMED Patient-Centered Home Model at the beginning of the study. However, they were familiar with the FFM report, the concept of patient-centered medical care, and the notion that they are contributing to change at a national level. At the self-directed retreat, it was revealed that none of the self-directed practices had seen the "components" of the TransforMED Model until it appeared in the May 2007 issue of Family Practice Management. However, based on their interpretation of the FFM report, as well as an intuitive sense of patient-centered care, some practices were well on their way to implementing many or most components of the model. In these practices, it was up to the physician champion to get buy-in from staff —not an external facilitator.

B. The self-directed practices, originally frustrated at not being in the intervention group, re-energized and regrouped at their June 2007 retreat. The retreat anchored their self-organizing process and gave the practices an identity. In the words of one physician, they are "a loose federation of change leaders." A few characteristics of the retreat stood out:

1. Organized and facilitated by the practices themselves, this retreat was primarily about sharing and reflection. Although each practice volunteered to "own" a topic and share insights and experiences from its own practice, the format was discussion-based rather than didactic. In their opinion, the voice of experience spoke the , so no "experts" or consultants were asked to speak to the group.
   
2. The retreat was designed on the core elements of successful change: mission integrity, relationships, facilitative leadership, reflection, and a learning culture. Their mission was to learn from each other and renew optimism in the NDP and the change process itself. The FFM elements are self-explanatory, enhanced by a retreat structure that allowed plenty of breathing room for longer discussions, personal stories and insights, requests for advice from the group, and the voices and concerns of the practice managers who attended the retreat as well. Particularly memorable was the Saturday night session where one physician read aloud parts of the FFM report and facilitated a spirited and rich discussion that invited personal interpretation.
   
3. The learning at the retreat did not stop there – it stimulated and informed better conversations back home. At the Evaluation Team site visits, many practices explained that they tried a new method or experimented with a new product following the retreat. For example, one practice began to develop some scheduling solutions on the plane ride home based on conversations they had with other practices. At least three practices have either purchased or are looking into the purchase of the "Instant Medical History" computer program based on a presentation at the retreat.
   
4. The learning and sharing at the retreat is not unusual – this also occurs at the facilitated practice learning sessions, although these are more tightly scheduled with a roster of well-known speakers and consultants. What was unusual was the retreat's organic and reflective approach, and how well it worked. What made this retreat work so well was the belief in the power of collaboration and connection among individuals with a shared greater goal but different ways of approaching it.

C. The self-directed practices are also experiencing change fatigue. Although the retreat was energizing, and many claimed they were glad to be able to change at their own pace (sans facilitator), site visits by the Evaluation Team revealed that some of these practices are struggling from the same change fatigue that we are seeing so clearly in the facilitated practices. In fact, every physician leader in the self-directed group so far has commented that a facilitator would be useful as someone to "ride herd" and keep the practice motivated and on track. Otherwise, they found it was too easy to get bogged down in the daily details. In many practices, especially the larger ones, the NDP champions are the physicians who are known within the practice to be 'champions' of new ideas and change. Therefore, they carry the weight of convincing the other physicians and staff to go along with the rapid pace of change required by TransforMED. Physician champions in both practice groups acknowledge that they are tired and often discouraged. The self-directed champions feel that an energetic facilitator would be helpful.