Additional Validation Checks Help Identify Consequential Residual Confounding


The implementation of negative control outcomes and a complementary cohort demonstrate an estimated beneficial effect that associated with an individual’s overall health.

Observational study designs for drug-repurposing studies of metformin in those with type 2 diabetes (T2D) could lead to consequential residual confounding, according to the results of a study published in JAMA Network Open.

Two additional validation checks could help identifying consequential residual confounding, investigators said.

In the study, 404,458 individuals with T2D were included. Investigators used an administrative claims database for Medicare Advantage beneficiaries in the United States.

The investigators were categorized into 2 categories: individuals with T2D and the 81,791 with prediabetes. Clinical history for the individuals were observed in 2018, and end points were observed in 2019. The statistical analyses were conducted between May and December 2021.

Individuals with T2D also had a recent prescription and history of use on at least 90 of the preceding 365 days of metformin or insulin but not both, at the start of the observation period.

There was a strong metformin effect estimated associated with both reduced inpatient admissions and medical expenditures, according to the study results.

However, investigators found that the implementation of additional robust design features, which included negative control outcomes and a complementary cohort, demonstrated an estimated beneficial effect that was attributed to residual confounding associated with an individual’s overall health.

Investigators found inverse propensity weighting yields acceptable covariates balance in both cohorts. They concluded that covariate balance plots, extremely small P values, large E-values, and strong effect sizes could point to an association with metformin and these outcomes.

This suggests that though the observed covariates in the study were well balanced after adjustment, substantial residual confounding associated with overall health could have influenced the primary study results, according to investigators.

Furthermore, in the prediabetes cohort, the metformin users and nonusers were similar before adjustments, which suggests that the groups were comparable.

The results showed the inadequacy of the study design in its effectiveness at addressing residual confounding with overall health and disease severity, investigators said.

It also confirmed that the prediabetes cohort was biased in the opposite direction of the T2D cohort, which investigators said made it an ideal complementary cohort.

By using combined insight from negative control outcomes and complementary cohort designs, investigators were able to differentiate what a typical study design would conclude and what they found as part of their study.

Limitations of the study included only including individuals with qualifying health insurance, selective capture of variables associated with or incentivized by financial reimbursement, and missing key elements of disease severity, including duration.

The observational studies can help influence clinical practices, support randomized clinical trials, and get amplified to media outlets, which then share information with the public, investigators said.

However, investigators about the cost of overvaluing poorly designed observational studies, which can lead to a waste in research funds and could potentially put individuals’ health at risk.


Powell M, Clark C, Alyakin A, Vogelstein JT, Hart B. Exploration of residual confounding in analyses of associations of metformin use and outcomes in adults with type 2 diabetes. JAMA Netw Open. 2022;5(11):e2241505. doi:10.1001/jamanetworkopen.2022.41505

Related Videos
Image credit:  Gorodenkoff |
Pharmacist and Patient in Pharmacy | Image Credit: Gorodenkoff -
Sun Screen, Photosensitivity, Pharmacy | Image Credit: sosiukin -
Catalyst Trial, Diabetes, Hypertension | Image Credit: grinny -
Various healthy foods -- Image credit: New Africa |
LGBTQIA+ pride -- Image credit: lazyllama |
Modern pharmacy building facade with large window showcasing the interior, as seen from the street view, promoting a welcoming atmosphere for customers. Frontal view. Generative AI - Image credit: Karrrtinki |
Close up hands of helping hands elderly home care. Mother and daughter. Mental health and elderly care concept - Image credit:  ipopba |
Ozempic, obesity, drug shortage, media portrayal | Image Credit: zimmytws -
© 2024 MJH Life Sciences

All rights reserved.