More Opinions on Practice Management

The decision to add laser procedures to your current practice can be daunting yet exciting. Realistically, this is unlike Field of Dreams, as patients do not materialize just because you have a new laser. Certain questions are important to answer in making this decision.

Read more

The largest asset on your balance sheet is almost certainly, believe it or not, accounts receivable. Collecting balances due has always been a problem for physicians.

Read more

The word "volunteer" is unusual in that it may be used as a noun, verb, or adjective. As a noun, it identifies a person who performs a service of his or her own free will, and, importantly, as a verb, it means to do charitable or helpful work without pay.

Read more

Film is dead, it has gone the way of the daguerreotype and calotype. As DVDs have replaced VHS, digital cameras have replaced film cameras and for pretty much the same reason; digital cameras are easier to use.

Read more

Opinions on Practice Management

Steven R. Feldman, MD, PhD

Exploring Selection Bias

Steven R. Feldman, MD, PhD

Wednesday, January 13, 2010

A new doctor comes to town, announcing that they are a family doctor with an 'interest' in skin disease. The new doctor's office employs an assistant who had previously worked with a dermatologist. Their office advertises that they provide a variety of cosmetic dermatology procedures. Within 6 months, you have seen 12 of their former patients. Every one of them was unhappy with the care that they received. Some patients didn't get well, a couple of patients had been given the wrong diagnosis, and some patients thought that the doctor wasn't caring or thorough. What should you do?

  1. Ignore it, because it's none of your business;
  2. Get to know the doctor better and try to get a better sense of them and their practice;
  3. Contact your dermatology society and its attorneys; or
  4. Contact the regional medical board to report the doctor.

Caring and committed dermatologists would probably do something to make sure that patients weren't receiving bad medical treatment. Perhaps very caring doctors would go out of their way to contact the medical board or to petition for some form of government intervention to put a stop to the perceived problem.

But consider how selection bias affects impressions of the new doctor. For example, assume the new doctor sees 50 patients each day. Over the course of 6 months, that's roughly 6,000 patients. Perhaps 5988 of the 6,000 received great care and were totally happy with the doctor and his or her practice. It is unlikely that you will get to see any of those 5,988 happy patients, because they will probably continue to visit the new doctor for their skin problems. The only patients you are likely to see in your office are the 0.2% who weren't happy with the other doctor. Although it might seem, based on experience, that every one of the other doctor's patients are extremely unhappy or poorly treated, you wouldn't have seen a representative sample.

Consider what the other doctor's experience must be with patients who saw you. Out of the 6,000 patients he or she saw during those 6 months, probably a few score (if not a few hundred) will have previously seen you or another dermatologist locally. Few, if any, of those patients are likely to have been fully satisfied, much less ecstatic, about the care they have received. There might even have been a couple of missed diagnoses or possibly even a missed melanoma diagnosis among them. Of course, those few patients aren't at all representative of the thousands and thousands of patients you've seen. You probably wouldn't want a doctor to report you to the medical board based on such a sample of your practice.

Selection bias can have a profound effect on our observations. Because we practice in a different compartment, the impressions that dermatologists have of the dermatologic skills of primary care doctors are based on seeing the primary care doctors' failures, and never their successes. Likewise, a surgeon's impressions of a dermatologist's surgical skills are affected by similar biased sampling. Impressions of the effectiveness of medications and other treatments used by patients before they visit the dermatologist are also affected by selection bias. Selection bias can profoundly influence studies of the prevalence, incidence and impact of diseases such as melasma or psoriasis.

Selection bias can have an enormous effect on the apparent frequency of a disease; much higher frequencies of skin disease and higher impacts are seen in dermatology clinic populations than in primary care or in the general population. Studies on the frequency of arthritis in psoriasis patients have found that values can range from 10% to nearly 100%, depending on whether the patients were seen in a dermatology or rheumatology practice, respectively.1 There might be 6 million or more people in the United States with psoriasis and, although one third of patients with psoriasis have severe disease, one third of the 6 million don't have severe psoriasis.2,3 The people who see a dermatologist for psoriasis tend to be people with more severe diseases, and while the greater majority of people with psoriasis in the population (85-90%) have very limited involvement.2

When clinical studies are carried out, the potential for selection bias must be addressed. When information is sought for a particular population, an enrollment strategy that obtains a representative sample of that population should be used. The population we see in our clinics is rarely, if ever, a representative sample of anything other than the people who choose to see a dermatologist. Thus, any assumptions we choose to make about the patients who we don't see are made at our peril. Consequently, we are likely to be led astray if we use our experience to draw conclusions about the effectiveness of primary care doctors in managing skin diseases or about the effectiveness of over-the-counter treatments that patients might use before they visit us.

Selection bias is a very common phenomenon in dermatology and in our daily lives. We shouldn't think that Columbine High School is representative of secondary education in America or that the terrorists we read about in the newspapers are in any way representative of people of a particular religious faith. In fact, what we see in the news is almost by definition not representative of the norm, because the news reports what's new and different, not what is ordinary.

There is a strong tendency among people to trust their observations and experiences.4 We have to be very careful not to put too much trust in observations that we should know are strongly affected by selection bias. It is probably easier to be misled into thinking that another doctor practises poorly when in reality they take terrific care of their patients. Whenever we are tempted to think ill of others based on our observations, we really ought to consider the possibility that our observations aren't representative. We would certainly hope that others would extend us that same courtesy.

References

  1. Gladman DD, Shuckett R, Russell ML, Thorne JC, Schachter RK. Psoriatic arthritis (PSA) - an analysis of 220 patients. Q J Med. 1987;62:127-141.
  2. Stern RS, Nijsten T, Feldman SR, Margolis DJ, Rolstad T. Psoriasis is common, carries a substantial burden even when not extensive, and is associated with widespread treatment dissatisfaction.  J Investig Dermatol Symp Proc 2004:9;136-139.
  3. Fleischer AB Jr, Feldman SR, Rapp SR,et al. Disease severity measures in a population of psoriasis patients: The symptoms of psoriasis correlate with self-administered psoriasis area severity scores.  J Invest Dermatol 1996;107:26-29.
  4. Feldman SR. Compartments: How the Brightest, Best Trained, and Most Caring People Can Make Judgments That are Completely and Utterly Wrong. Xlibris: Philidelphia, PA, 2009.

For further reading on this topic, please see Feldman SR. Compartments: How the Brightest, Best Trained, and Most Caring People Can Make Judgments That are Completely and Utterly Wrong. Xlibris: Philidelphia, PA, 2009.

Back