note about retriever bug for customers taking early look

pyg-team · Jan 7, 2025 · 5775885 · 5775885
1 parent b3b1524
commit 5775885
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/examples/llm/hotpot_qa.py b/examples/llm/hotpot_qa.py
@@ -124,6 +124,10 @@
     We will use precision as a proxy for recall. This is because for recall,
     we must know how many relevant triples exist for each question,
     but this is not known.
+    
+    Note that the retrieval precision may be much lower than expected.
+    likely due to a bug in retriever causing similar issue here:
+    https://github.com/pyg-team/pytorch_geometric/pull/9806
     """
     precisions = []
     for QA_pair in relevant_triples.keys():