Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue: Bug -> Querying Experiment by Dataset Version Error #1399

Open
tendev-eoghan opened this issue Jan 9, 2025 · 3 comments
Open

Issue: Bug -> Querying Experiment by Dataset Version Error #1399

tendev-eoghan opened this issue Jan 9, 2025 · 3 comments

Comments

@tendev-eoghan
Copy link

tendev-eoghan commented Jan 9, 2025

Issue you'd like to raise.

When working on #1390 I came across the following bug.

listDatasetVersions returns versions with 6 digits to represent a fraction of a second.
e.g. 2025-01-09T14:42:18.364000+00:00

When calling listProjects passing in a datasetVersion https://github.com/langchain-ai/langsmith-sdk/pull/1391 sometimes the datasetVersion returned from listDatasetVersions will not match the one tagged to a value from listProjects.

This is because listProjects sometimes has the datasetVersion with only 3 digits to represent a fraction of a second.
e.g. 2025-01-09T14:42:18.364+00:00

This issue also presents itself in the langsmith UI portal when filtering experiments by dataset version.

Suggestion:

standardise dataset versions to 6 digits to represent a fraction of a second across all references to dataset version.

@tendev-eoghan
Copy link
Author

tendev-eoghan commented Jan 10, 2025

With some further digging I found this function which I think might be the root cause.

// Python might return microseconds, which we need

Sometimes evaluation experiments are tagged with a dataset version using 3 digits of precision instead of 6.

@tendev-eoghan
Copy link
Author

Issue fixed with the above PR by @tendev-liam

#1406

@tendev-liam
Copy link
Contributor

PR #1406 does not completly solve this issue.

It appears if the dataset version timestamp ends in a zero the problem still persists.

e.g An expirment will gets its dataset version tagged as
2025-01-13T13:48:12.64721+00:00
Missing one zero in fractional seconds

however the getDatasetVersions api will respond with the timestamp like:
2025-01-13T13:48:12.647210+00:00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants