fix gpt bigcode model loading with fp16 weights precision

[eaidova]

What does this PR do?

fix issue found during enabling https://huggingface.co/WizardLMTeam/WizardCoder-15B-V1.0

RuntimeError: Expected attn_mask dtype to be bool or float or to match query dtype, but got attn_mask.dtype: c10::Half and  query.dtype: float instead.

gpt-bigcode model cast attention mask to transformer.wte.weight.dtype that is float16 in case if we load model with preserving weights in original dtype (https://github.com/huggingface/transformers/blob/v4.46.3/src/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py#L916) and provide query as float32, as the result dtype mismatch may take place on scaled_dot_product operation inputs

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[IlyasMoutawwakil]

Does this happen in transformers as well ? if so can you please open an issue/PR there as well.

[eaidova]

Does this happen in transformers as well ? if so can you please open an issue/PR there as well.

this is related to how openvino model conversion works - to reduce memory consumption, we load model with auto as torch_dtype to avoid additional memory overhead for conversion fp16 to fp32. as usually, model exports happen on cpu that does not always support bf16/fp16 on pytorch level, for overcome that we patch some operations to be executed in fp32 and skip memory consumed ops like linear or embeddings during tracing replacing them directly with ov op, so as the result on our side model.transformer.wte is skipped and passed as float16 and and query calculated as float32, I do not think that this is matched with torch behavior, but I'll check

[echarlaix]

@eaidova would you mind taking a look at the conflicts, will merge once resolved

[eaidova]

@eaidova would you mind taking a look at the conflicts, will merge once resolved

@echarlaix, I resolved merge conflicts