2) You can find a large number at https://www.cochrane.org. You can also search Google Scholar. If it's a systematic review, it's almost always included in the title of the study. Whether or not you can access the article for free depends, though. Anything funded by the US gov't (in the past) had to be "open access," meaning anyone can read it without a paywall. However, the paywall sometimes remains in place during an embargo period. If you have a university login, you should be able to access a lot more, since universities pay subscriptions to various journals. (If you see something you really want to read and can't get it open access, DM me!)
1) Yes, you include ALL articles that meet the inclusion criteria in a systematic review. If you exclude them, you must declare why. Some reviews exclude studies because they are deemed "low quality" (as determined during their risk of bias assessment, which is also reported). This is especially true for the inclusion or exclusion of studies in the meta-analysis (MA) calculations. You could state that you are including only high- and medium-quality studies, but you would still show the low-quality ones in the tables, along with your risk of bias assessment of them to justify your decisions.
Authors may also run the MA both ways, with and without the low-quality studies, to see the difference, and report that process as a "sensitivity analysis." That may show, for example, that including the poor-quality studies did not change the overall conclusion, but did increase the uncertainty (error/confidence intervals) or heterogeneity (differences in results between the various studies). Both those things are undesirable.
One major potential source of bias in systematic reviews is called "publication bias." This is the idea that studies with positive results and big effects get published more often than studies that found null results (the data can't prove the intervention worked) or negative results (the drug actually made people worse!). There is a test called a funnel plot that can give a visual to assess if there might be publication bias.
Can bias happen? Yes. But there are lots of checks in place to minimize it.
That said, I have seen one systematic review on gender-affirming care that was quite biased in its inclusion and exclusion of studies. It excluded several large cohort studies (some of the best evidence we have) as "low quality," but included studies by known bad actors whose work has been heavily criticized and debunked. By examining the numbers reported, I could see that the studies included in the tables showed overall improvement, and yet the authors concluded that gender-affirming care was a negative thing!
That kind of bias and misinterpretation should have been caught and fixed in peer review, but, well, it was submitted to a journal with a transphobic editor, so he ensured other transphobic authors conducted the peer review. In that same journal, I have seen studies where the Discussion section is filled with content that is NOT supported by the Results section. Those authors count on people reading only the Discussion and Conclusion, so they can push their pet theories despite the evidence. It's like if the Results said 2+2=4, and the authors' interpretation was that 3 is a cursed number. Like, what?!?!
And that's why I wrote this post--so more people can effectively read these publications and catch mis- and disinformation.
I’m glad. Going through a queer identity crisis and faith crisis at the same time is a lot! And you’re not alone in that. My faith crisis came 15 years before my trans/queer identity development but I’m still working through effects from the religious harms. Learning to trust yourself and listen to your inner voice is key. It takes work, but it’s worth it.
Great questions about systematic reviews!
2) You can find a large number at https://www.cochrane.org. You can also search Google Scholar. If it's a systematic review, it's almost always included in the title of the study. Whether or not you can access the article for free depends, though. Anything funded by the US gov't (in the past) had to be "open access," meaning anyone can read it without a paywall. However, the paywall sometimes remains in place during an embargo period. If you have a university login, you should be able to access a lot more, since universities pay subscriptions to various journals. (If you see something you really want to read and can't get it open access, DM me!)
1) Yes, you include ALL articles that meet the inclusion criteria in a systematic review. If you exclude them, you must declare why. Some reviews exclude studies because they are deemed "low quality" (as determined during their risk of bias assessment, which is also reported). This is especially true for the inclusion or exclusion of studies in the meta-analysis (MA) calculations. You could state that you are including only high- and medium-quality studies, but you would still show the low-quality ones in the tables, along with your risk of bias assessment of them to justify your decisions.
Authors may also run the MA both ways, with and without the low-quality studies, to see the difference, and report that process as a "sensitivity analysis." That may show, for example, that including the poor-quality studies did not change the overall conclusion, but did increase the uncertainty (error/confidence intervals) or heterogeneity (differences in results between the various studies). Both those things are undesirable.
One major potential source of bias in systematic reviews is called "publication bias." This is the idea that studies with positive results and big effects get published more often than studies that found null results (the data can't prove the intervention worked) or negative results (the drug actually made people worse!). There is a test called a funnel plot that can give a visual to assess if there might be publication bias.
Can bias happen? Yes. But there are lots of checks in place to minimize it.
That said, I have seen one systematic review on gender-affirming care that was quite biased in its inclusion and exclusion of studies. It excluded several large cohort studies (some of the best evidence we have) as "low quality," but included studies by known bad actors whose work has been heavily criticized and debunked. By examining the numbers reported, I could see that the studies included in the tables showed overall improvement, and yet the authors concluded that gender-affirming care was a negative thing!
That kind of bias and misinterpretation should have been caught and fixed in peer review, but, well, it was submitted to a journal with a transphobic editor, so he ensured other transphobic authors conducted the peer review. In that same journal, I have seen studies where the Discussion section is filled with content that is NOT supported by the Results section. Those authors count on people reading only the Discussion and Conclusion, so they can push their pet theories despite the evidence. It's like if the Results said 2+2=4, and the authors' interpretation was that 3 is a cursed number. Like, what?!?!
And that's why I wrote this post--so more people can effectively read these publications and catch mis- and disinformation.
I’m glad. Going through a queer identity crisis and faith crisis at the same time is a lot! And you’re not alone in that. My faith crisis came 15 years before my trans/queer identity development but I’m still working through effects from the religious harms. Learning to trust yourself and listen to your inner voice is key. It takes work, but it’s worth it.