cpu: x64: enable matmul-based IP for forward inference

[densamoilov]

This PR introduces a matmul-based inner product implementation for inference that will be used instead of the current brgemm-based one. The latter will be disabled for inference.

This PR is targeting oneDNN v3.8 and will be merged after the oneDNN v3.7 code freeze date.

The implementation allows using blocked weights layouts directly or via the special tag any.
The Inner Product weights must meet ONE of the following requirements to enable using the blocked layouts:

• Weights don't have spatial.
• Weights have unit spatial.
• Weights have non-unit spatial but the number of input channels is a multiple of K block.

If none of the above requirements are met then a plain layout will be used.

The new matmul-based implementation is able to leverage blocked weights layouts for all cases used in the performance testing and therefore provides the best possible performance.
Below is a comparison of the matmul-based vs brgemm-based inner product. > 100% means matmul-based is better.

_________________

[densamoilov]

make test
enable device_cpu
disable device_gpu

[mgouicem]

Could you clarify why jit_brgemm_ip gets restricted to forward training?

IIUC, jit_brgemm_ip is after the new matmul_ip in the dispatch list, so there is a chance that if a forward inference case is not handled by matmul_ip (per the documented restrictions), it will also be skipped by jit_brgemm_ip and will go to a lower performance impl.

[densamoilov]

so there is a chance that if a forward inference case is not handled by matmul_ip (per the documented restrictions)

If a blocked layout cannot be used for weights then it'll fall back to a plain layout so all cases that can be handled by brgemm-ip can also be handled by matmul-based ip.

The goal is to remove brgemm-ip completely so we can't use it as a complement.

[densamoilov]

make test
enable device_cpu
disable device_gpu