[srimanachanta]

Closes #3256. Built on #3258; the first two commits are #3258 rebased onto main.

Summary

A pure-Python package cannot give host_launch a GIL-free host function: a ctypes callback built from Python takes the GIL, and libc memcpy does not match CUhostFn. This PR adds one in cuda.core, as agreed in #3256.

• cuda.core.HOST_COPY_FN: a ctypes CUhostFn pointer to a noexcept nogil function that copies nbytes from src to dst. It works as fn in host_launch, the graph callback methods, and update().
• cuda.core.pack_host_copy_args(dst, src, nbytes) -> bytes: packs the record for user_data. It accepts ints, Buffer, and buffer-protocol objects, and checks bounds, overlap, null addresses, and the host accessibility of a Buffer's memory resource. It makes no driver calls, so a Python host callback can use it. dst and src are not retained.

The record layout is public: uintptr_t dst, uintptr_t src, size_t nbytes, the same as struct.pack("@PPN", ...). A caller can pre-pack records and pass an address as an int user_data. The host callback resolver rejects HOST_COPY_FN with no user_data, a zero address, or bytes of the wrong length, so a bad call raises instead of crashing the driver thread.

The HOST_COPY_FN docs note that the driver may run all host functions of a context on one thread, so a long copy delays other host functions, including Python callbacks. They recommend chunks of a few MiB.

Changes

• cuda_core/cuda/core/_host_callback.pyx: the copy function, HOST_COPY_FN, pack_host_copy_args, and the resolver check. Exported from cuda.core.
• cuda_core/docs/source/api.rst: HOST_COPY_FN and pack_host_copy_args. The host_launch and graph callback docstrings point to them. A release-note entry for 1.3.0.
• cuda_core/examples/host_copy.py: a chunked device-to-NumPy copy through two pinned slots, one stream per slot.
• cuda_core/tests/test_host_copy.py: stream ordering, int user_data, graph replay, zero-byte copies, the packed layout, and each validation error.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/cuda-python
• Branch: main
• File: .coderabbit.yaml
• Commit: a69a782
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: ca53bf7c-0763-4fc1-8cdb-fc601c9f8281

Commits

Reviewing files that changed from the base of the PR and between a69a782 and b11ec0f.

Files selected for processing: 27

• cuda_core/cuda/core/__init__.py
• cuda_core/cuda/core/_cpp/rt/DESIGN.md
• cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md
• cuda_core/cuda/core/_cpp/rt/api.hpp
• cuda_core/cuda/core/_cpp/rt/driver_api.hpp
• cuda_core/cuda/core/_cpp/rt/host_launch.cpp
• cuda_core/cuda/core/_host_callback.pxd
• cuda_core/cuda/core/_host_callback.pyi
• cuda_core/cuda/core/_host_callback.pyx
• cuda_core/cuda/core/_launcher.pyi
• cuda_core/cuda/core/_launcher.pyx
• cuda_core/cuda/core/_rt.pxd
• cuda_core/cuda/core/_rt.pyx
• cuda_core/cuda/core/graph/_graph_builder.pyi
• cuda_core/cuda/core/graph/_graph_builder.pyx
• cuda_core/cuda/core/graph/_graph_node.pyi
• cuda_core/cuda/core/graph/_graph_node.pyx
• cuda_core/cuda/core/graph/_host_callback.pyi
• cuda_core/cuda/core/graph/_host_callback.pyx
• cuda_core/cuda/core/graph/_subclasses.pyx
• cuda_core/docs/source/api.rst
• cuda_core/docs/source/examples.rst
• cuda_core/docs/source/release/1.3.0-notes.rst
• cuda_core/examples/host_copy.py
• cuda_core/examples/host_launch.py
• cuda_core/tests/test_host_copy.py
• cuda_core/tests/test_host_launch.py

Files with no reviewable changes: 2

• cuda_core/cuda/core/graph/_host_callback.pyi
• cuda_core/cuda/core/graph/_host_callback.pyx

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Summary

Summary by CodeRabbit

• New Features
  • Added stream-ordered host callbacks, supporting Python callables and compatible CUDA callback pointers.
  • Added a built-in host-memory copy callback and a helper for preparing its arguments. These can be used with streams and graph callbacks.
• Documentation
  • Added API guidance and examples for host callbacks and chunked host-memory copies.

The PR adds stream-ordered host function launch support and a built-in host-memory copy callback. It adds argument packing and validation, integrates callback resolution with stream and graph APIs, and includes tests, documentation, and runnable examples.

Changes: Host callback and copy support

Layer / File(s)	Summary
Runtime host-function launch path cuda_core/cuda/core/_cpp/rt/api.hpp, cuda_core/cuda/core/_cpp/rt/driver_api.hpp, cuda_core/cuda/core/_cpp/rt/host_launch.cpp, cuda_core/cuda/core/_rt.pxd, cuda_core/cuda/core/_rt.pyx, cuda_core/cuda/core/_cpp/rt/DESIGN.md, cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md	Adds the driver entry and runtime launch function. The launch record retains callback and data owners, invokes the callback, and enters deferred cleanup after the callback returns.
Callback resolution and copy arguments cuda_core/cuda/core/_host_callback.pyx, cuda_core/cuda/core/_host_callback.pyi	Adds HOST_COPY_FN, pack_host_copy_args, callback validation, and callback/data ownership handling. The packer validates addresses and buffers, then returns a native-width argument record.
Python launch and graph callback integration cuda_core/cuda/core/__init__.py, cuda_core/cuda/core/_launcher.pyx, cuda_core/cuda/core/_launcher.pyi, cuda_core/cuda/core/graph/*	Exports the host callback symbols and adds host_launch for streams and GraphBuilder inputs. Graph callback modules import the shared resolver from cuda.core._host_callback, and their documentation describes the packed-record requirement for HOST_COPY_FN.
Examples, documentation, and tests cuda_core/tests/test_host_copy.py, cuda_core/tests/test_host_launch.py, cuda_core/examples/host_copy.py, cuda_core/examples/host_launch.py, cuda_core/docs/source/api.rst, cuda_core/docs/source/examples.rst, cuda_core/docs/source/release/1.3.0-notes.rst	Adds tests for stream ordering, graph replay, callback arguments, and validation. Adds examples for stream callbacks and chunked host copies, plus API, example, and release documentation.

Assessment against linked issues

Objective	Addressed	Explanation
Provide a GIL-free CUhostFn host copy callback and a helper to pack (dst, src, nbytes) for use with host launch and graph callbacks [#3256]	✅

Possibly related issues

• Issue 2058: The PR adds host_launch, which enqueues a host function in stream order.

Possibly related PRs

• NVIDIA/cuda-python#3258: Adds the stream-side host launch path that this PR extends with the copy callback and packed user data.
• NVIDIA/cuda-python#2925: Covers callback contracts that this PR tests and shares through the relocated graph callback resolver.

Suggested labels: feature, cuda.core

Suggested reviewers: andy-jost

Priority: Normal

Change: Feature

Merge Risk: Minimal · up to b11ec

No concrete merge-blocking issue is established. The change is mergeable after normal build and test checks.

Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[Andy-Jost]

/ok to test b11ec0f

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-3269/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[Andy-Jost]

Thanks, this implements #3256 as agreed. I reviewed only the three commits on top of #3258. _host_copy is noexcept nogil, reads the documented 24-byte @PPN record, and skips memcpy for zero bytes; _resolve_host_callback rejects None, 0, empty, and wrong-length records before it allocates, and the packed bytes ride the existing user_data ownership on the stream, builder, definition, and update() paths. pack_host_copy_args makes no driver call on any built-in path: it reads _memory_resource.is_host_accessible directly and so avoids the cuPointerGetAttributes that Buffer.is_host_accessible falls back to. The example orders its pinned slots correctly. Since the branch carries #3258, it merges after #3258.

Three changes before merge:

1. The Notes of pack_host_copy_args say a Python host callback can use it "to enqueue the next copy". Enqueueing is a CUDA call, which host_launch forbids inside a host function. Say instead that a host callback can call it, for example to fill a record that a later HOST_COPY_FN reads through an int user_data.
2. A negative or oversized nbytes or int address raises OverflowError (and a test pins it), but Raises lists only TypeError, ValueError, and RuntimeError. Document it, in the .pyx and the .pyi, or convert it to ValueError.
3. dst and src have no annotations; the public-API rule wants int | Buffer | BufferProtocol, as Buffer.fill uses.

Smaller points:

• Only a Python int is an address. True packs as address 1, and a NumPy integer scalar (np.uintp(addr)) or a ctypes.c_void_p exports the buffer protocol, so it packs the address of its own 8 bytes rather than the address it holds. Reject bool and these scalars with a TypeError that says to pass int(addr), or say in the docstring that only an int is an address.
• "The caller must keep both alive" is not quite enough for a bytearray: it must also not be resized until the copy has run.
• HostCallbackNode.update and ExecutableHostCallbackNode.update raise the same HOST_COPY_FN record error but their docstrings do not say so.
• Tests: a Buffer.from_handle(ptr, n) with no memory resource would pin the no-query path (GPU-free), and the overlap test covers only dst > src; add the src > dst case and its adjacent boundary.