Bringing new features to OCaml is not a trivial procedure, and any new contribution is subject to rigorous testing and inspection. The introduction of Multicore OCaml added a whole new dimension of complexity to the process, and this post takes you behind-the-scenes of a project that sprang from troubleshooting the restoration of MSVC to OCaml 5.
+OCaml 5.3 is released and comes with a restored MSVC (Microsoft Visual C/C++ compiler) port. It had been removed with the introduction of the multicore runtime in OCaml 5.0. The Restore the MSVC port of OCaml #12954 PR lists all the prerequisite changes, and the final streak of changes, required to restore support.
+The OCaml 5 runtime requires C11 atomic support from the C compiler, which for MSVC was introduced in Visual Studio 2022 version 17.5 Preview 2. This early version had a few bugs during code generation and we were wondering if they would impact the OCaml runtime (fortunately, they did not). While the Microsoft team worked hard to fix the bugs, we learned about clang-cl, a driver program for Clang that attempts to be compatible with MSVC's cl.exe. It is ABI-compatible with MSVC, implements all MSVC compiler extensions, and is also a drop-in command-line replacement for cl.exe.
I wanted to try building OCaml with clang-cl, mostly because it would get us a second compiler's opinion on our code, as Clang has a different set of warnings and hints than MSVC. Seeing that clang-cl was already used to build Chrome and Firefox (1, 2), I was hoping the needs of the OCaml runtime would have already been covered and bugs fixed, and that we could adopt it seamlessly.
The OCaml 5 runtime uses the POSIX threads (pthreads) library on Unix-like systems for all its concurrency primitives. For the OCaml 5.3 branch we've chosen to use the winpthreads library, part of the MinGW-w64 project, which implements pthreads on Windows. The OCaml 5 MinGW-w64 port uses it, and we found out we could use it with MSVC too. I then submitted two patch series to winpthreads (Patches and cleanups towards MSVC support, MSVC support without GCC extensions), checking my work with MinGW-w64+GCC, MinGW-w64+clang, MSVC, and clang-cl, foreshadowing their use within the OCaml runtime. What an adventure! And thanks to the MinGW-w64 team for reviewing this work.
clang-clClang on Unix-like systems masquerades as GCC, supports all GNU C extensions and defines the __GNUC__ macro. On Windows, it masquerades as MSVC and defines the _MSC_VER macro instead, but still supports the GNU C extensions that we can take advantage of!
The build system only required a few changes. For instance, MSVC currently defaults to C99 and needs two flags to switch to C11 and enable experimental C11 atomic support, whereas clang-cl defaults to C17. We also discussed how to improve the support of MSVC in Autoconf, which led to a few patches. We could then use clang-cl to discover new problems reported by the warnings it raised and fix them. In conjunction with this work, I raised the warning level of MSVC on the OCaml runtime C code from none to -W2.
Fortunately, most of the warnings were quite minor (see #13081 and #13243), mainly consisting of warnings for deprecated functions or implicit truncations when converting integers or floating points values of different sizes. Switching to newer compilers also allowed us to remove dead code and workarounds for older versions of the compilers.
+I found out that most of the uses of compiler attributes or builtins were guarded by the __GNUC__ macro, and as such were only enabled by GCC or Clang on Unix, even though clang-cl on Windows supports them too. Compiler attributes may enable more checks and warnings from the compiler. For instance, the format attribute tags a function to be printf-like and checks the types of the list of values passed to it against the specifiers inside a format string. Compiler builtins may enable more optimisations, such as __builtin_expect. So, instead of guarding their use, we could discover whether the compiler provides them using newer macros such as __has_attribute or __has_builtin. This improved feature parity between the clang-cl port and an OCaml build using GCC or Clang.
In particular, we could detect the labels as values (also known as computed gotos) compiler extension to enable threaded code interpretation, which dramatically improves the speed of ocamlc, the OCaml bytecode interpreter. This optimisation isn't supported by MSVC and would require us to use inline assembly on x86 or write part of the bytecode interpreter in assembly on other architectures. If you're often using the bytecode interpreter, there's now a clear advantage of using clang-cl over MSVC. Another interesting optimisation uses software prefetching to speed up the GC when traversing the graph of values. We had worked on restoring it in OCaml 5 from OCaml 4 but forgot to port it to Windows!
The overall work on restoring the MSVC port of OCaml and also building it with clang-cl led to a few bug reports to Microsoft and to the LLVM project, which I hope will benefit the community. The OCaml project now has an extra set of compiler eyes that scrutinise each and every change on Windows. Windows users may now take advantage of a (compliant) C11 and C23 FOSS compiler, with a wide range of optimisations and checks available.
I'm grateful to my colleagues at Tarides for helping me with this work, the OCaml core team for reviewing it, and Jane Street for sponsoring this effort.
+With the release of opam 2.2 (now 2.3) supporting Windows, the restoration of the Cygwin port in OCaml 5.1, the MSVC port in OCaml 5.3, and the option to build OCaml with clang-cl, and the swarm of bug fixes that accompanied them, using OCaml on Windows has never been easier! Give it a whirl!
Connect with Tarides online on Bluesky, X, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.
+