Fix Julia template for binary string

[qiaojunfeng]

It seems there is a small issue with generated julia function signature, when handling

application/octet-stream:
  schema:
    type: string
    format: binary

To better reproduce this, I set up a MWE here
https://github.com/qiaojunfeng/test-julia-OpenAPI/tree/main

Since this is a julia-specific issue, I will ping technical committee member @tanmaykm. Thanks!

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package || exit
  ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
  ./bin/utils/export_docs_generators.sh || exit
  

  (For Windows users, please run the script in Git BASH)
  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
  IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
• File the PR against the correct branch: master (upcoming 7.x.0 minor release - breaking changes with fallbacks), 8.0.x (breaking changes without fallbacks)
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

[wing328]

triggered CI tests via

• https://github.com/OpenAPITools/openapi-generator/actions/runs/12443478387/job/34742694987?pr=20364
• https://github.com/OpenAPITools/openapi-generator/actions/runs/12443477103/job/34742692216?pr=20364

and both tests passed

[tanmaykm]

Thanks @qiaojunfeng, I shall check this out this weekend.

[tanmaykm]

The effect of this change would be that if something is specified as binary format, it would now be generated as a String instread of Vector{UInt8} bytearray. While strings in Julia can contain invalid UTF8 bytecodes (https://docs.julialang.org/en/v1/manual/strings/), canonically Vector{UInt8} is used for such purposes.

I would like to understand more the reason why you feel we should have this changed in the openapi generated code?

[qiaojunfeng]

The effect of this change would be that if something is specified as binary format, it would now be generated as a String instread of Vector{UInt8} bytearray. While strings in Julia can contain invalid UTF8 bytecodes (https://docs.julialang.org/en/v1/manual/strings/), canonically Vector{UInt8} is used for such purposes.

I would like to understand more the reason why you feel we should have this changed in the openapi generated code?

Thanks @tanmaykm!
A concise answer to this is that with the current generator, the generated julia code does not work with binary string for file upload. Probably it is easier to understand this by running this example
https://github.com/qiaojunfeng/test-julia-OpenAPI/tree/main

[tanmaykm]

Thanks @qiaojunfeng, I think the issue here is missing check for the isBinary flag in some places of the template. Should those be added instead of removing the check altogether? Then the code would be generated as Vector{UInt8} uniformly for binary strings and that should resolve the issue.

[tanmaykm]

@qiaojunfeng Here's my recommended changes: https://github.com/OpenAPITools/openapi-generator/compare/master...tanmaykm:tan/fixfile?expand=1
If it works please feel free to update this PR with that.

[qiaojunfeng]

@tanmaykm Thanks for the changes!

Indeed using Vector{UInt8} uniformly as the arg type feels cleaner, but unfortunately, it does not work for such cases.

The reason comes from these:

1. In the client part, the arg file::Vector{UInt8} is used to set the OpenAPI.Clients.Ctx.file, which has type file::Dict{String,String}, so it does not work

function _oacinternal_upload_post(_api::DefaultApi, file::Vector{UInt8}; _mediaType=nothing)
    _ctx = OpenAPI.Clients.Ctx(_api.client, "POST", _returntypes_upload_post_DefaultApi, "/upload", [])
    OpenAPI.Clients.set_param(_ctx.file, "file", file)  # type Vector{UInt8}
    ...

https://github.com/qiaojunfeng/test-julia-OpenAPI/blob/7bed81b4a84fbdb93f2fcaf951e31903b053bb08/client/src/apis/api_DefaultApi.jl#L47-L49
(Your tan/fixfile branch generates the same code as in this link (same as released version of openapi-generator))

2. When OpenAPI.jl prepares for uploading file, it internally uses curl_mime_filedata, and libcurl c interface expect it to be a filename not file content (https://curl.se/libcurl/c/curl_mime_filedata.html)

            for (_k,_v) in ctx.file
                part = LibCURL.curl_mime_addpart(mime)
                LibCURL.curl_mime_name(part, _k)
                LibCURL.curl_mime_filedata(part, _v)

https://github.com/JuliaComputing/OpenAPI.jl/blob/f18935ed12e40e56d007471e0690216e728e48b0/src/client.jl#L382-L385

Therefore, in my branch, I directly set file as String pointing to the filename, instead of the file content of type Vector{UInt8}

function _oacinternal_upload_post(_api::DefaultApi, file::String; _mediaType=nothing)
    _ctx = OpenAPI.Clients.Ctx(_api.client, "POST", _returntypes_upload_post_DefaultApi, "/upload", [])
    OpenAPI.Clients.set_param(_ctx.file, "file", file)  # type String

https://github.com/qiaojunfeng/test-julia-OpenAPI/blob/7bed81b4a84fbdb93f2fcaf951e31903b053bb08/client_new/src/apis/api_DefaultApi.jl#L47-L49

3. If we still want to use Vector{UInt8} as the type, probably we need to create a temp file, write the content to it, and pass the temp file name as _v to LibCURL.curl_mime_filedata(part, _v). Since in most cases the file content should be already stored some where as a file, maybe directly passing filename can avoid reading and writing again the content to a temp file, especially useful if the file size is large?

[imaami]

The best solution would not be to start writing tempfiles as a matter of routine. Just use curl_mime_data_cb() or curl_mime_data() instead of curl_mime_filedata().

The path of least resistance is curl_mime_data() as it takes a buffer pointer and size argument. The downside is having to keep the data in memory in full during transfer, but as the generated client code already assumes that to be the case, I'd go for curl_mime_data().

[imaami]

Although changing the current interface to something that uses tempfiles sounds to me like the wrong way to fix the bug, I also think it makes sense to use curl_mime_filedata() if the source is already known to be an existing file. So how about this: use curl_mime_data() for Vector{UInt8} input, but also add a new method to the client API that takes a file path and uses curl_mime_filedata().

[tanmaykm]

Thanks @qiaojunfeng for checking it out.
And thanks @imaami for the suggestions. That seems like a good approach and should be doable. I also agree that we should not be using temp files.

I have adjusted my branch with some changes to generate the client methods that take file parameters as strings pointing to existing files. That should then work for cases where there is a real file. @qiaojunfeng would you be able to try that out? Here's the diff: https://github.com/OpenAPITools/openapi-generator/compare/master...tanmaykm:tan/fixfile?expand=1

Next (in seprate PRs?), we could make the necessary changes in OpenAPI.jl to handle both file names (as String) and file data (as Vector{UInt8}). And follow that up with openapi-generator changes to generate methods that take file parameters as either existing files or file data as byte array.

[qiaojunfeng]

I have adjusted my branch with some changes to generate the client methods that take file parameters as strings pointing to existing files. That should then work for cases where there is a real file. @qiaojunfeng would you be able to try that out? Here's the diff: https://github.com/OpenAPITools/openapi-generator/compare/master...tanmaykm:tan/fixfile?expand=1

Thanks @tanmaykm for the commit! I cherry-picked your commit and made one additional commit ac95bb3 to fix some small issues. After these, I can reproduce correct code generations and everything works well.

Next (in seprate PRs?), we could make the necessary changes in OpenAPI.jl to handle both file names (as String) and file data (as Vector{UInt8}). And follow that up with openapi-generator changes to generate methods that take file parameters as either existing files or file data as byte array.

This sounds like a good idea. Let's fix file names in this PR and leave the file content to a new one.

[tanmaykm]

Great! Looks good to me.
Requesting @wing328 to approve the workflows please!