-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue: Bug -> Querying Experiment by Dataset Version Error #1399
Comments
With some further digging I found this function which I think might be the root cause. langsmith-sdk/js/src/evaluation/_runner.ts Line 816 in 843d55d
Sometimes evaluation experiments are tagged with a dataset version using 3 digits of precision instead of 6. |
Issue fixed with the above PR by @tendev-liam |
PR #1406 does not completly solve this issue. It appears if the dataset version timestamp ends in a zero the problem still persists. e.g An expirment will gets its dataset version tagged as however the getDatasetVersions api will respond with the timestamp like: |
Issue you'd like to raise.
When working on #1390 I came across the following bug.
listDatasetVersions
returns versions with 6 digits to represent a fraction of a second.e.g.
2025-01-09T14:42:18.364000+00:00
When calling
listProjects
passing in a datasetVersion https://github.com/langchain-ai/langsmith-sdk/pull/1391 sometimes the datasetVersion returned fromlistDatasetVersions
will not match the one tagged to a value fromlistProjects
.This is because
listProjects
sometimes has the datasetVersion with only 3 digits to represent a fraction of a second.e.g.
2025-01-09T14:42:18.364+00:00
This issue also presents itself in the langsmith UI portal when filtering experiments by dataset version.
Suggestion:
standardise dataset versions to 6 digits to represent a fraction of a second across all references to dataset version.
The text was updated successfully, but these errors were encountered: