Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Annotate the version of an item #1041

Open
1 task
tuscland opened this issue Jan 3, 2025 · 5 comments
Open
1 task

Annotate the version of an item #1041

tuscland opened this issue Jan 3, 2025 · 5 comments
Labels
enhancement New feature or request
Milestone

Comments

@tuscland
Copy link
Member

tuscland commented Jan 3, 2025

Is your feature request related to a problem? Please describe.

As a data scientist, I want to annotate the specific version of an item because it helps me keep track of what is happening.

As a data scientist, I want to update or delete the note I have previously created.

As a data scientist, the notes I create from the Python API must be visible in the UI.

Describe the solution you'd like

Python API

The value of the note kwarg is some Markdown text.

Add an note:

skore.put(
    "feat_imps",
    fig,
    note="added AggJoiner with max_observation=2"
)

More programmatic CRUD:

item_version = skore.get_item_versions("feat_imps")[0]

# If there is no note, it creates a note, otherwise update the note
item_version.set_note("set frunobulax level at 11")
note = item_version.get_note()

item_version.delete_note()

UI

Let the user interact with an item version:

  • Create / Update a note using Markdown text.
  • Delete the note.
  • View the note of an item (last version)
  • View the note of a specific item version

Describe alternatives you've considered, if relevant

No response

Additional context

Related to #889

@tuscland tuscland added enhancement New feature or request epic This issue represents major product increments labels Jan 3, 2025
@tuscland tuscland added this to the Skore 0.7 milestone Jan 7, 2025
@tuscland tuscland removed the epic This issue represents major product increments label Jan 8, 2025
@augustebaum
Copy link
Contributor

Without put_item it seems harder to achieve the set_note behaviour, as Items are just dataclasses with no link to a Project.

@rouk1
Copy link
Contributor

rouk1 commented Jan 8, 2025

To me, we need to introduce IDs for items. They could be a simple string interpolation {name}-{counter}.
Then note may be a new entity with a foreign key. Even without a proper db engine we can do this in repositories.

@augustebaum
Copy link
Contributor

Yes, then set_note could be a method of Project instead:

project.put("my_key", 3)

# Annotates the latest version by default
project.set_note("my_key", "this is 3 because 3 is good")

# Annotates version 10
project.set_note("my_key-10", "this is 3 because 3 is good")

@thomass-dev
Copy link
Collaborator

thomass-dev commented Jan 10, 2025

@augustebaum , we shouldn't add the version in the key, prefer a dedicated parameter.

In your example, imagine you have two items in your project with keys my_key (10 versions) and my_key-10 (1 version).
How to differentiate the version 10 of the my_key item from the my_key-10 item?

I would prefer something like:

 class Project:
     def set_note(self, key, note, *, version=-1):
         ...

Edit:

We can maybe add a new parameter to the put function too put(key, value, *, note=None).

@MarieS-WiMLDS
Copy link
Contributor

agree with Thomas's solution, I like it, it seems fluid. I also like to be able to put a note at first and also afterwards.

@MarieS-WiMLDS MarieS-WiMLDS removed their assignment Jan 10, 2025
augustebaum added a commit that referenced this issue Jan 14, 2025
This is for several reasons:
- it is not as explicit as several simple `put` calls: the mechanic
is ambiguous with regards to atomicity (if a key-value pair is
invalid, does it make the whole operation fail?)
- the mechanic makes it complicated to add options to `put`,
e.g. the `note` option proposed in
#1041
or the `display_as` option proposed in
#1045 (comment).
augustebaum added a commit that referenced this issue Jan 14, 2025
This is for several reasons:
- it is not as explicit as several simple `put` calls: the mechanic
is ambiguous with regards to atomicity (if a key-value pair is
invalid, does it make the whole operation fail?)
- the mechanic makes it complicated to add options to `put`,
e.g. the `note` option proposed in
#1041
or the `display_as` option proposed in
#1045 (comment).
augustebaum added a commit that referenced this issue Jan 14, 2025
This is for several reasons:
- it is not as explicit as several simple `put` calls: the mechanic
is ambiguous with regards to atomicity (if a key-value pair is
invalid, does it make the whole operation fail?)
- the mechanic makes it complicated to add options to `put`,
e.g. the `note` option proposed in
#1041
or the `display_as` option proposed in
#1045 (comment).
thomass-dev pushed a commit that referenced this issue Jan 14, 2025
This is for several reasons:
- it is not as explicit as several simple `put` calls: the mechanic
is ambiguous with regards to atomicity (if a key-value pair is
invalid, does it make the whole operation fail?)
- the mechanic makes it complicated to add options to `put`,
e.g. the `note` option proposed in
#1041
or the `display_as` option proposed in
#1045 (comment).
thomass-dev pushed a commit that referenced this issue Jan 14, 2025
New methods have been added to `Project`:

```python
# The default is to act on the latest version
project.set_note(key, message, version=-1)

project.get_note(key, message, version=-1)

project.delete_note(key, message, version=-1)
```

To this end,
- `Item` now has an additional attribute, `note`, which is a `str |
None`
- `ItemRepository` has similar methods

The note related methods fail with a KeyError when the key-version
combination does not exist, but:
- `get_note` returns `None` if the note is `None` (no error)
- `delete_note` never errors if the key-version combination exists (even
if the note is already `None`, it will be set to `None` again)

`set_note` fails with a TypeError if `key` or `message` is not a string.

Addresses part of #1041
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants