Build wheels using the conda package

[ocefpaf]

Comment:

We don't have wheels for esmpy on PyPI but we can do something similar to https://github.com/ocefpaf/netcdf4-win-wheels. What do you think @conda-forge/esmpy team?

[xylar]

I don't have much expertise in this kind of thing (Windows or wheels) but I certainly think it could be nice!

[zklaus]

No objection. I always feel a bit queasy about wheels for these packages that are backed by a whole stack (or web?) of binary dependencies. This is particularly true for these library style packages that must be used by some downstream program. Doesn't that lead to too much duplication and static linking?

But of course, if you think it's useful for someone and you are willing to support it, go ahead.

[ocefpaf]

Doesn't that lead to too much duplication and static linking?

Yes but that is the definition of wheels :-/

But of course, if you think it's useful for someone and you are willing to support it, go ahead.

I won't use it but it is useful to folks that are lock in places that can only use pip. Sadly that's quite common.

[valeriupredoi]

gents @ocefpaf @zklaus call me Tail End Charlie, but it only now dawned to me that ESMPY is not on PyPI - is there any push to deploy it there after all? Am asking since I've just taken it out from our setup.py (soon to be pyrproject.toml), and frankly, am not too sure why I did that only now 😁

[billsacks]

From the ESMF side, publishing ESMPy on PyPI has not been on our priority list but could potentially be if there was enough need from the community. What is your sense of the level of need, for yourself and others?

[bouweandela]

For some of my colleagues, ESMF not being on PyPI was one of the reasons for starting their own regridding package (xarray-regrid) instead of just using xESMF, so I would say there is demand.

[billsacks]

Thanks for your comment, @bouweandela .

I can propose this to the ESMF team. I don't have experience providing packages on PyPI – especially those like ESMF/ESMPy that depend on a large compiled binary. Can anyone with experience with this give me a sense of how much effort it might take to set up and maintain a package like this for PyPI? e.g., would we likely be looking at more like a few days or a few weeks of effort?

[ocefpaf]

I can give it a go and, Windows and macos should be relatively easy. Although it would be nice to get conda-forge/esmf-feedstock#119 in first.

Linux on the other hand is a bit more complex. We cannot use conda-forge packages easily for it yet. We would need a manylinux docker image with a ESMF build on it. (There are alternatives but this is usually the easiest path IMO.)

[billsacks]

Thank you, @ocefpaf ! We on the ESMF team haven't made any progress here yet because of competing priorities and not being able to find anyone in-house who has experience doing something similar. If you make progress on this, we'll be very interested to see it!

In case it helps, we do maintain ESMF docker images for two Linux architectures (amd64 and arm64): https://hub.docker.com/r/esmf/esmf-build-release/tags . We could look more into a manylinux docker image at some point if that would be helpful.