Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Correctness Issue: The Formula Used in OneDNN and PyTorch for MaxPool2d is Inconsistent #2355

Closed
knowledgaction opened this issue Jan 8, 2025 · 4 comments · May be fixed by #2359
Closed

Correctness Issue: The Formula Used in OneDNN and PyTorch for MaxPool2d is Inconsistent #2355

knowledgaction opened this issue Jan 8, 2025 · 4 comments · May be fixed by #2359
Assignees
@TaoLv
Labels
question

Comments

@knowledgaction
Copy link

The example provided in the OneDNN examples (https://github.com/oneapi-src/oneDNN/blob/v3.6.2/examples/primitives/pooling.cpp) uses a formula that is inconsistent with the formula in PyTorch (https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html).

The formula in OneDNN:

const memory::dim OH = (IH - ((KH - 1) * DH + KH) + PH_L + PH_R) / SH + 1;  
const memory::dim OW = (IW - ((KW - 1) * DW + KW) + PW_L + PW_R) / SW + 1;

The formula in PyTorch:

const memory::dim OH = std::floor((IH - DH * (KH - 1) - 1 + PH_L + PH_R) / SH + 1);  
const memory::dim OW = std::floor((IW - DW * (KW - 1) - 1 + PH_L + PH_R) / SW + 1);

微信截图_20250108150729
In my test case, I used the following parameters:

const memory::dim N = 3; // batch size  
const memory::dim IC = 3; // input channels  
const memory::dim IH = 112; // input tensor height  
const memory::dim IW = 112; // input tensor width  
const memory::dim KH = 3; // kernel height  
const memory::dim KW = 3; // kernel width  
const memory::dim PH_L = 1; // height padding: left  
const memory::dim PH_R = 1; // height padding: right  
const memory::dim PW_L = 1; // width padding: left  
const memory::dim PW_R = 1; // width padding: right  
const memory::dim SH = 2; // height-wise stride  
const memory::dim SW = 2; // width-wise stride  
const memory::dim DH = 1; // height-wise dilation  
const memory::dim DW = 1; // width-wise dilation

Using the OneDNN formula, I get OW = 55 and OH = 55.
Using the PyTorch formula, I get OW = 56 and OH = 56.

The resulting shapes are inconsistent!!!!

If I use the PyTorch formula to calculate OW and OH, and then generate the descriptor based on the OW and OH calculated from PyTorch, the program (https://github.com/oneapi-src/oneDNN/blob/v3.6.2/examples/primitives/pooling.cpp) will result in a compilation error!!!

I am even confused.
@TaoLv
Copy link
Contributor

TaoLv commented Jan 8, 2025

Could you please try DH = DW = 0 for this case with oneDNN? It's the default dilation in oneDNN.

@knowledgaction
Copy link
Author

Could you please try DH = DW = 0 for this case with oneDNN? It's the default dilation in oneDNN.

@TaoLv
Copy link
Contributor

TaoLv commented Jan 8, 2025

The default value of dilation (aka. non-dilated) is 0 in oneDNN while it's 1 in PyTorch. So DH = DW = 1 is used to create PyTorch max pooling operation, you may need to use DH = DW = 0 to create oneDNN pooling primitive.

@shu1chen shu1chen mentioned this issue Jan 9, 2025
Open
@knowledgaction
Copy link
Author

The default value of dilation (aka. non-dilated) is 0 in oneDNN while it's 1 in PyTorch. So DH = DW = 1 is used to create PyTorch max pooling operation, you may need to use DH = DW = 0 to create oneDNN pooling primitive.

The problem is solved, and this method is correct. Thank you very much. If you hadn't told me, I might have been stuck researching this for who knows how long.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TaoLv TaoLv

Labels
question
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

doc: conv: add note for the dilation computation formula shu1chen/oneDNN
2 participants
@TaoLv @knowledgaction