Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accuracy_multilabel function probably incorrect #23

Closed
53X opened this issue Jun 6, 2019 · 1 comment · May be fixed by #24
Closed

Accuracy_multilabel function probably incorrect #23

53X opened this issue Jun 6, 2019 · 1 comment · May be fixed by #24

Comments

@53X
Copy link

53X commented Jun 6, 2019

Hi,

def accuracy_multilabel(y_pred:Tensor, y_true:Tensor, sigmoid:bool=True):
    if sigmoid: y_pred = y_pred.sigmoid()
    outputs = np.argmax(y_pred, axis=1)
    real_vals = np.argmax(y_true, axis=1)
    return np.mean(outputs.numpy() == real_vals.numpy())

in this block.

This piece of code seems incorrect as the shape of y_pred and y_true is (Batch_size, class_space).
Doing a np.argmax with axis=1 returns a single class index value for each sample.
This is what we do for multi-class classification.

However in multi-class classification we don't normally use sigmoid on y_pred, although it is not wrong.

This function seems much like accuracy_multiclass rather than accuracy_multilabel

@kaushaltrivedi
Copy link
Collaborator

The idea behind this was to pick the accuracy of top predicted label. You are right, accuracy_thresh is meant to be used for multi-label models. This was more as an experiment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants