Code and data for the paper "Can Open-Source LLMs Compete with Commercial Models? Exploring the Few-Shot Performance of Current GPT Models in Biomedical Tasks".
If you have any questions about the code and data don't hesitate to contat me via email (Samy.Ateia [at] stud.uni-regensburg.de) or write an issue.
To rerun the code the original test set from BioASQ 12 is necessary. This can be retrieved by registering for the BioASQ challenge or contacting the organizers: http://www.bioasq.org/
Part of the training and test set for BioASQ 12 [1,2,3] was used to create trainings and few-shot examples and is contained in this repository and is accordingly published under the CC BY 2.5 license The license text can be found here: https://creativecommons.org/licenses/by/2.5/
- An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition: George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, Yannis Almirantis, John Pavlopoulos, Nicolas Baskiotis, Patrick Gallinari, Thierry Artiéres, Axel Ngonga, Norman Heino, Eric Gaussier, Liliana Barrio-Alvers, Michael Schroeder, Ion Androutsopoulos and Georgios Paliouras, in BMC bioinformatics, 2015
- BioASQ-QA: A manually curated corpus for Biomedical Question Answering: Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras in Sci Data 10, 2023
- Overview of BioASQ 2024: The twelfth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering: Nentidis, Anastasios and Katsimpras, Georgios and Krithara, Anastasia and Lima-López, Salvador and Farré-Maduell, Eulàlia and Krallinger, Martin and Loukachevitch, Natalia and Davydova, Vera and Tutubalina, Elena and Paliouras, Georgios in Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Fifteenth International Conference of the CLEF Association (CLEF 2024), 2024