Hi, thank you for sharing the code and paper. I really enjoyed reading the paper — it was very interesting.
I wanted to clarify the DB lookup failure fallback used in the reported experiments.
The paper seems to describe an unknown fallback: if the highest similarity score is below the retrieval threshold, the model continues with the plain text string unknown. In that case, my understanding is that no additional retrieval is performed after the lookup failure.
However, the released code also includes a top1_anyway fallback policy, which retries retrieval with threshold=0.0 and returns the top-1 result if possible.
Could you confirm which fallback policy was used for the reported results, especially FactScore and T-REx? Was it strictly unknown, or was top1_anyway used in any experiment?
Hi, thank you for sharing the code and paper. I really enjoyed reading the paper — it was very interesting.
I wanted to clarify the DB lookup failure fallback used in the reported experiments.
The paper seems to describe an
unknownfallback: if the highest similarity score is below the retrieval threshold, the model continues with the plain text stringunknown. In that case, my understanding is that no additional retrieval is performed after the lookup failure.However, the released code also includes a
top1_anywayfallback policy, which retries retrieval withthreshold=0.0and returns the top-1 result if possible.Could you confirm which fallback policy was used for the reported results, especially FactScore and T-REx? Was it strictly
unknown, or wastop1_anywayused in any experiment?