A Case Study from Synthesa’s Research Team, conducting a Network Meta-Analysis using Artificial Intelligence and LLMs.
Comparative Effectiveness of GLP-1 Receptor Agonists: A Network Meta-Analysis Using ChatGPT
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5103748
Abstract
Background: Glucagon-like peptide-1 receptor agonists (GLP-1 RAs) are widely used for managing type 2 diabetes mellitus (T2DM), obesity, and cardiovascular disease. However, their comparative effectiveness in reducing major adverse cardiovascular events (MACE) and all-cause mortality (ACM) remains unclear.
Methods: This systematic review and network meta-analysis (NMA) followed PRISMA-NMA guidelines and was registered with PROSPERO. Randomized controlled trials (RCTs) comparing GLP-1 RAs with placebo or each other were included. MACE was the primary outcome, and ACM was the secondary outcome. ChatGPT was used for search strategy development, screening, data extraction, risk of bias assessment, and to generate the code for R to perform the NMA, with its performance evaluated through metrics like Cohen’s kappa, precision, recall, and F1. Frequentist NMA models analyzed outcomes, with sensitivity and subgroup analyses to assess robustness.
Results: Twelve RCTs involving 85,628 participants were included. Semaglutide consistently demonstrated high efficacy, reducing MACE with a hazard ratio (HR) of 0.80 (95% CI: 0.73–0.87) and ACM with an HR of 0.81 (95% CI: 0.73–0.89). Efpeglenatide also showed efficacy for MACE and ACM, somewhat less strongly than for semaglutide. Albiglutide demonstrated robust efficacy in reducing MACE but ranked among the lowest for ACM, indicating inconsistent performance across outcomes. ChatGPT performed well in generating search strategies, screening, and data extraction but flagged irrelevant studies and misinterpreted specific data points, requiring meticulous oversight by experienced reviewers.
Conclusions: Semaglutide consistently reduced MACE and ACM, with the most robust results for GLP-1 RA based on current data. Efpeglenatide and albiglutide showed promise, supporting a class effect, but further data are warranted to confirm their effectiveness. ChatGPT improved efficiency in evidence synthesis, though human validation remains necessary.
