Add ubs-virt-enpu slicing and optional mem-swap integration

[maverick-woo]

https://project.feishu.cn/dynamia-rd/story/detail/7103930628

What would you like to be added:

Add ubs-virt-enpu (vCANN-RT) as an opt-in Ascend sharing backend alongside hami-vnpu-core and template-based hardware vNPU, with optional integration for the upstream mem-swap feature and metrics in HAMi's existing Ascend monitoring endpoint.

The proposed scope is:

• Select the backend with huawei.com/vnpu-mode: enpu, reuse existing Ascend memory/core resources, and support fixed-share, elastic, and best-effort policies.
• Generate per-container runtime configuration for the physical DIE selected by HAMi and inject libvruntime.so, enpu-monitor, and ld.so.preload following the upstream container configuration. Keep runtimeClassName: ascend and expose only the selected device by default.
• Build the ordinary-slicing runtime from official release 1.0.0 using the official build image, record provenance/checksums, and package the assets in the device-plugin image alongside the existing hami-vnpu-core assets.
• Add optional node-local enpu-manager integration for mem-swap. Use the HAMi allocation as the memory reservation, allow a separate huawei.com/enpu-memory-limit, and validate the returned physical device, Pod/container identity, quotas, and policy before mounting the manager-generated configuration. Workloads should not have to construct these mappings manually.
• Export ENPU metrics through the existing device-plugin :9395/metrics endpoint. Attribute DCMI process HBM usage to the current Pod/container through host PID and cgroup mappings, cross-check the selected physical DIE and UUID, and expose configured memory request/limit, compute share, scheduling policy, allocation identity, and collection status.
• Provide Helm/node configuration, ordinary-slicing and mem-swap Pod examples, bilingual monitoring instructions, an optional PodMonitor example, and links to upstream hardware/runtime prerequisites.

Why is this needed:

Users should be able to deploy ENPU workloads through normal HAMi scheduling and allocation. Centralizing runtime injection and manager configuration avoids mismatches between the device HAMi selects and the device/configuration used inside the workload. Optional memory oversubscription should preserve the same scheduler reservation contract. ENPU resource usage should also be observable through the same Ascend Prometheus endpoint, without requiring operators to run a command inside every workload.

Anything else we need to know?:

• Coordinated implementation PRs: ascend-device-plugin #141 and HAMi #3110. Deploy both changes together for admission, node capability discovery, scheduling, runtime allocation, and chart configuration.
• ENPU must be disabled by default. Existing hami-vnpu-core and hardware-template behavior must remain unchanged when it is disabled. ENPU and hami-core must not share a physical DIE, and ENPU workloads sharing a DIE must use compatible scheduling policies.
• Initial ENPU scope is one physical DIE per container. Target configurations are A2/910B and A3/910C; A3 requires independent-DIE mode. This does not imply validation on every device model.
• Official release 1.0.0 provides ordinary slicing, not mem-swap. Mem-swap integration must remain optional and require compatible upstream runtime/manager components; it must not silently replace the default release build with preview sources. Without oversubscription settings, memory request and limit remain equal. fixed-share must reject unequal request/limit values.
• No changes to ubs-virt's internal swap implementation are proposed here. Automatic cleanup of manager allocation records after Pod exit is not implemented in the current plugin PR; manager allocation lifecycle and failure recovery remain explicit follow-up work.

Monitoring scope and status:

The monitor integration has been pushed to device-plugin PR #141 in commit 850eda1 for review. Deployment of this update remains pending.

• Reuse hami_vgpu_memory_used_bytes and hami_vgpu_memory_limit_bytes for container HBM usage/limit, and add ENPU memory reservation, configured compute-share, allocation-info, and collection-success metrics. Preserve the existing Pod/container/device labels.
• Follow the official monitor's DCMI memory-query semantics. The collector does not invoke or parse the enpu-monitor CLI on each scrape. It uses the plugin's existing privileged context and a read-only host /proc mount at /host/proc; no new workload environment variables, hostPID, or pods/exec permission are required.
• Export AI Core utilization at physical-DIE scope only. Official release 1.0.0 has no HTTP endpoint or per-Pod compute-utilization output. Configured compute share is not measured utilization and is not a strict ceiling under every scheduling policy; a best-effort CLI quota of 0 must not be interpreted as a zero HAMi allocation.
• Report resident HBM in bytes. Mem-swap configurations may have different memory request and limit values, but CPU swap bytes and sleep state are not exported. Collection failures omit affected measurements and set the corresponding success metric to 0, instead of fabricating zero usage.
• Preserve hami-core-only and hardware-template collection behavior. When hami-core and ENPU are both enabled, shared metric descriptors are registered together and whole-device series are exported once.
• The PodMonitor is an optional manual example, with no default Prometheus Operator CRD dependency. Its namespaces, Helm release selectors, and Prometheus selectors must match the deployment.

Usage at this commit: English guide, 中文说明, PodMonitor example.

Upstream references: ubs-virt-enpu source, official vCANN-RT configuration guide.

[maverick-woo]

English

ENPU integration test report: basic slicing, memory oversubscription, and performance comparison with hami-core

Related PRs: ascend-device-plugin #141 and HAMi #3110. This report consolidates earlier basic diagnostics, single-instance validation, and archived measurements from September 22–23, 2026. It separates functional validation from backend performance comparisons. Different stages used different builds, identified below; these results do not constitute a full hardware regression test of the latest PR HEAD.

1. ENPU basic slicing and memory oversubscription

Test environment and build provenance. The formally archived tests used the following environment. Earlier diagnostic runs do not have equally complete environment and runtime provenance records.

Item	Configuration
NPU	Ascend 910C / A3; 8 cards, 2 DIEs per card; 65,536 MiB HBM per DIE
CPU / RAM	Kunpeng 920 7280Z; 640 logical CPUs and 8 NUMA nodes; approximately 2013 GiB host RAM
Host OS	openEuler 22.03 LTS-SP4, aarch64; kernel 5.10.0-216.0.0.115.oe2203sp4.aarch64
Kubernetes / runtime	K3s v1.36.4+k3s1; containerd 2.3.4-k3s1.36; both the RuntimeClass name and handler are ascend
Driver / CANN	Driver 26.1.1, ascendhal 7.35.23; CANN 9.1.0 inside the container
Workload image	quay.io/ascend/vllm-ascend:v0.23.0-a3; Ubuntu 22.04.5 LTS inside the container
Frameworks	torch 2.10.0+cpu + torch-npu 2.10.0.post4; vLLM 0.23.0+empty / vLLM-Ascend 0.23.0
Model for ordinary inference	Qwen3-8B
Model for memory oversubscription	Qwen3.6-27B-w8a8

The formal release build for basic slicing followed this process: official release 1.0.0 source → runtime built in the official build image → packaged into the open-source device-plugin image → deployed and tested through HAMi. The source commit was 46b6d55a892429008b1e02f96b0a0f81d1176db8, and the build image was swr.cn-north-4.myhuaweicloud.com/ubscore/ubs-virt:oe2403-v1. SHA256 of the actual libvruntime.so used at runtime:

1d65bbf247aa524866ee7d77dcf4ebc9b216ca862c6f1d98c2fb5996722edda1

The mem-swap tests used an isolated build of preview source commit 1072945b066ee354a99aa1400ee6d43a7b3cfd91, with compatibility and diagnostic patches for runtime logical-device numbering and the manager's A3 DCMI handling, state format, REST request reception, and UID parsing. The tests used the open-source plugin image with mounted build artifacts overriding the runtime components. This did not change the default release build and packaging process for basic slicing. SHA256 of the runtime used for memory oversubscription:

637ec7d649d342380e6fd24f79e44092f71f2485ee933191272b6d7488760fd7

Official release 1.0.0 does not include the mem-swap functionality validated here; the oversubscription results apply to the preview runtime/manager combination with the patches described above. The later single-DIE Kubernetes tests used A3 independent-DIE mode and retained runtimeClassName: ascend. HAMi selected the device and generated/mounted its configuration; the containers exposed the selected compute device and the required management devices. The required systemd-detect-virt executable and its shared-library dependencies were also installed in the test containers.

Single-instance basic computation and device mapping. Open-source HAMi was used to create 6 Pods: three policies across two independent DIEs. Each Pod requested one device, 16,384 MiB of memory, and core=20. Each Pod performed four 512×512 FP16 matrix multiplications with all-one inputs, synchronized after each operation, and asserted that the result elements were 512.0.

Policy	First independent DIE on the same card	Second independent DIE on the same card
fixed-share	4/4 passed	4/4 passed
elastic	4/4 passed	4/4 passed
best-effort	4/4 passed	4/4 passed

All 6 Pods exited normally with zero restarts. Device nodes, ASCEND_VISIBLE_DEVICES, ENPU physical IDs, the mapping from logical device 0 to the actual physical DIE, and DCMI Card/Chip mappings were checked individually. torch.npu.device_count()=1; no additional ASCEND_RT_VISIBLE_DEVICES setting was used. The earlier DCMI -8008 error and out-of-range logical-device numbering did not recur in these six cases. An initial attempt reported no healthy devices before device-plugin registration had completed; the rerun passed after registration. These six Pods used the preview runtime with numbering fixes and diagnostic patches, so they do not count as validation of unmodified release 1.0.0.

Single-instance vLLM: three compute policies and three execution configurations. Qwen3-8B, TP1, 32,768 MiB memory, core=100, gpu-memory-utilization=0.80, max-model-len=2048, and max-num-seqs=4. Each table cell represents 8 requests at concurrency 4, with the benchmark configured for 128 input tokens and 32 output tokens.

Policy	eager (enforce-eager)	graph (FULL_DECODE_ONLY)	FULL graph configuration (called force in the historical scripts)
fixed-share	8/8, 0 failures	8/8, 0 failures	8/8, 0 failures
elastic	8/8, 0 failures	8/8, 0 failures	8/8, 0 failures
best-effort	8/8, 0 failures	8/8, 0 failures	8/8, 0 failures

Across all nine cells, 72/72 requests completed with zero failures. Here, force refers to the FULL graph execution configuration, not forced swap-in or swap-out. This stage used a historical open-source integration build, with the runtime compiled from source in the official build image. The exact source commit and the runtime SHA256 for each cell could not be fully recovered. The conclusion is therefore limited to successful short-batch inference under these startup configurations, not a complete release 1.0.0 graph-mode regression test. A preceding graph trial recorded capture 3/3, but graph replay was not traced individually for all nine cells. These tests also did not use the same steady-state warmup and sampling settings as Part 2, so their latency results are not included in the backend performance tables.

Memory quotas and early basic diagnostics. In historical tests, a 4 MiB allocation succeeded under a 256 MiB quota, while a subsequent 512 MiB allocation returned ENPU Out of memory / RuntimeError, error code 300000. The script caught the expected exception and exited; the container was not observed to be directly OOM-killed. A 64 MiB quota was smaller than the approximately 113.3 MiB context overhead at the time. The ENPU module initialized, but memory usage already exceeded the quota after initialization; memory queries and subsequent 4/128 MiB allocations failed. This stage involved an older scheduler deployment, mixed interception libraries, and exposure of a DIE pair. It is historical diagnostic evidence only, not acceptance evidence for a hard memory boundary in the current release or an ENPU-only environment. The formal release tests verified the 20 GiB quota view and an 18 GiB vLLM budget; memory-overlimit boundaries still require a separate test in a clean environment.

Early quota20/100 tests each performed four 128×128 matrix multiplications, taking 99.167/46.360 ms respectively. The two four-sample runs used different physical DIEs and logging levels, had no explicit matrix-multiplication warmup, and included first-operator preparation and per-operation printing in their timing. Other timeout records also exist. These measurements were not used to establish proportional accuracy or stability. The sustained-load tests under matching conditions below provide the subsequent compute-quota evidence.

Official release 1.0.0: single-instance fixed-share compute quotas. Three Docker containers were run sequentially on the same DIE, using the same image, runtime library, CPU configuration, and 20 GiB memory quota. The workload was 8192×8192 FP16 matrix multiplication with preallocated inputs and outputs, synchronizing after every 8 operations. Each quota level warmed up for at least 20 seconds and completed at least 64 groups before the measured run of at least 60 seconds. There was no per-operation printing during measurement.

aicore-quota	Completed matrix multiplications	Measured duration (s)	Throughput (operations/s)	Relative to quota 100
100	12,384	60.038	206.270	100.000%
50	6,224	60.079	103.597	50.224%
20	2,488	60.168	41.351	20.047%

Numerical checks passed after warmup and after measurement. All three containers exited with code 0 and were not OOM-killed. For this workload, sustained fixed-share throughput was close to the configured proportions. The quota-100 case also loaded ENPU and is the baseline for that runtime. There was only one run per quota level. This result does not imply that elastic/best-effort or vLLM tokens/s scale by the same strict proportions: elastic can borrow idle resources, and best-effort does not limit compute according to quota.

The script's final check for a pre-existing Kubernetes Pod returned NotFound, so the overall script exited with code 1. Measurements at all three quota levels, result validation, and Docker cleanup had already completed. The reason the original Pod disappeared was not established, and background load on other DIEs changed during the run. This must not be described as an environment check with zero anomalies or a precision qualification under fully controlled background conditions.

Official release 1.0.0: three policies with three concurrent inference instances. In each group, 3 Pods shared the same DIE and were allocated through HAMi. Each Pod had a 20 GiB memory quota, core=30, and an 18 GiB vLLM budget, using Qwen3-8B and TP1/eager. Each policy was tested for 3 rounds. Each instance performed 4 warmup requests per round, followed by 64 measured requests at concurrency 4, with 128 input and 128 output tokens. The three clients started synchronously.

ENPU policy	Valid rounds	Successful / total measured requests	Failures	Output tokens
fixed-share	3	576/576	0	73,728
elastic	3	576/576	0	73,728
best-effort	3	576/576	0	73,728

In total, 1,728 measured requests completed with zero failures. Pods, imageIDs, allocated devices, runtime libraries, and policy configurations were checked before and after the runs; the workloads did not restart. In this group, request=limit=quota, and memory oversubscription was not enabled. The release emitted warnings that it did not recognize the memory-request / memory-limit configuration keys, while reading memory-quota=20480. The conclusion is therefore that requests and configuration checks passed, not that the complete logs were warning-free. Throughput and latency are reported in Part 2.

mem-swap: three large-model instances with ten rounds of on-demand swapping. Three Pods shared one 64 GiB DIE. Each Pod ran Qwen3.6-27B-w8a8 with TP1/eager, max-model-len=32768, max-num-seqs=1, and gpu-memory-utilization=0.95. Each instance had a 60.8 GiB vLLM budget and stable resident HBM usage of approximately 55–56 GiB.

Each Pod had a HAMi memory reservation/request of 256 MiB, an ENPU memory limit of 65,536 MiB, core=30, and the best-effort policy. The manager ran as a host service following the upstream deployment model. The test DIE used oversub-ratio=2, a total CPU swap pool of 192 GiB, and swap-pre-watermark=95. HAMi called the manager to allocate vNPUs for the Pods and mounted the resulting configurations. Physical devices, vNPUs, shm-ids, quotas, and device nodes were checked; the vNPU mappings for these three Pods were not assigned manually.

After initialization, requests rotated in A→B→C order for 10 rounds, giving 30 scored requests. Each request started with the target instance swapped out. At each stable state, one instance was resident on the NPU and the other two were swapped out.

Instance	Active HBM (MiB)	HBM after swap-out (MiB)	Memory released per swap-out (MiB)	Inference after swap-in
A	56,731	197	56,534	10/10 successful
B	56,931	197	56,734	10/10 successful
C	57,211	197	57,014	10/10 successful

All 30 responses generated 32 tokens and exactly matched the corresponding instance's initialization baseline output. Pod UIDs, container IDs, and NPU PIDs remained unchanged, with zero restarts and one model load per instance. A and B each underwent one force-out during initialization so that the models could be loaded in sequence. During the 30 scored requests, there was no force-out/force-in, vLLM sleep, container pause, or model reconstruction; requests triggered swapping on demand.

All final states were verified using npu-smi and PID-to-Pod ownership mappings. The CPU swap pool reached a transient peak of 166.29 GiB. For each active instance, the corresponding stable pool usage remained consistent across all ten rounds; no sustained growth was observed within this finite rotation test. The ten rounds took 569.25 s in total. Mean request duration was 16.82 s: the first request took 31.79 s, and the remaining 29 averaged 16.30 s. These durations include swapping and 32-token inference, so they are not treated as pure-inference TTFT/TPOT or included in the throughput comparison in Part 2.

Oversubscription pass conditions and outstanding failures. With the same binaries and the default 80% watermark, all three requests in the first round succeeded, but the second-round C→A transition timed out after 300 s. Memory had been released, but completion acknowledgment did not return. The 95% setting avoided the pre-swap-out race for this workload; it does not establish that the protocol issue has been fixed. During startup, each Pod also logged 17 Python multiprocessing/forkserver signal-handler TypeErrors before model initialization and rotation completed. Hook/CANN compatibility warnings were also present. No swap/DCMI/memory errors or model reloads were found during the scored rotation phase, but the check for anomaly-free complete startup logs still did not pass.

The oversubscription conclusion is therefore limited to the following: the preview compatibility/diagnostic build passed ten rounds of three-instance swap-in/swap-out under the 95% watermark, best-effort policy, single-DIE, TP1/eager, sequential-request configuration. The default watermark and the complete environment have not passed production-readiness acceptance. Coverage does not include fixed-share/elastic with oversubscription, graph execution, multi-card TP, concurrent requests, long-term stability, allocation lifecycle management, or complete failure recovery. There are also no hardware test conclusions for 910A/910B from this host.

Supplement: ENPU monitoring collection. Subsequent monitoring integration tests held 128/640 MiB tensors in two Docker containers and exercised the real collector and HTTP metrics collection; all three rounds passed. Actual DCMI/HAMi HBM readings were 259,330,048 / 796,200,960 bytes, including context overhead, consistent with the official monitor's integer-MiB readings of 247/759. Metrics attributed to the two containers remained separate, and their collection-success metrics were both 1. This hardware harness validates DCMI, cgroup attribution, and HTTP exposure using known allocation identities. It is not yet an end-to-end acceptance test of the deployed Kubernetes informer/Prometheus path. Monitoring reports resident HBM only, not CPU swap bytes or sleep state, and does not provide actual per-Pod compute utilization.

Deployment prerequisites and parameters should follow the official vCANN-RT configuration guide and upstream source. Results for the formal release and the preview oversubscription components must be interpreted separately.

2. ENPU and hami-core performance comparison

This section uses the earlier three-instance comparison records from deployments through HAMi. It compares hami-core's default relative-weight mode with the three ENPU release 1.0.0 policies, retaining native as a full-DIE reference baseline. The model, image, device, request workload, and statistical method were checked.

Common conditions. The same physical DIE, Qwen3-8B model directory, and workload imageID were used. All groups used BF16 without quantization, three independent vLLM services, TP1/eager, max-model-len=2048, max-num-seqs=4, and generation-config=vllm, with prefix caching disabled. The total vLLM budget was 18 GiB per instance in every group.

Mode	Compute configuration / deployment	Memory budget and actual KV cache
hami-core default mode	3 Pods × 1 service; NPU_PRIORITY=30; NPU_FIXED_SHARE_RATIO unset; token scale / avg alpha not overridden	quota 20 GiB × util 0.9; 18 GiB budget; approximately 2.39 GiB KV
Three ENPU policies	3 Pods × 1 service per group; aicore-quota=30; policy values 1/2/3 respectively	quota=request=limit=20 GiB; 18 GiB budget; approximately 2.19–2.35 GiB KV
native	1 full-DIE Pod containing 3 independent services; no interception library	The matched-budget group sets util from the actual physical total so each service has exactly an 18 GiB budget; approximately 2.45 GiB KV

The value 30 in hami-core is a relative scheduling weight. ENPU fixed-share, elastic, and best-effort also have different quota semantics. This table compares these actual configurations; it does not imply equivalent hard compute ceilings. The total vLLM budget was matched, but actual KV sizes and native's Pod layout still differed.

Each instance performed 4 warmup requests per round, followed by 64 measured requests at concurrency 4, with 128 input and 128 output tokens. Settings were random dataset, seed=42, temperature=0, ignore-eos=true, request-rate=inf, random-range-ratio=0, and random-prefix-len=0. The three clients started through a shared barrier; warmup and service startup were excluded from timing. Each mode ran for 3 rounds with 192 requests per round, totaling 576 requests and 73,728 output tokens per mode. Across all five groups, there were 2,880 measured requests and 0 failures.

Total output throughput is calculated as total output tokens from all three instances / (latest finish − earliest start). The table reports the median throughput across the 3 rounds. TTFT is time to first token. TPOT is calculated per request as (latency − TTFT) / (output_tokens − 1). Both latency columns below are means across that mode's 576 successful requests, not medians or P99 values.

Mode	Median total output throughput (tokens/s)	Mean TTFT (ms)	Mean TPOT (ms)	Successful / failed
hami-core default relative-weight mode	83.210	870.277	115.643	576 / 0
ENPU fixed-share	231.591	220.167	50.468	576 / 0
ENPU elastic	234.858	194.048	49.893	576 / 0
ENPU best-effort	258.515	175.557	45.170	576 / 0
native (full-DIE baseline)	258.426	167.218	45.220	576 / 0

Total throughput by round, showing variation across the three rounds:

Mode	Round 1 (tokens/s)	Round 2 (tokens/s)	Round 3 (tokens/s)
hami-core default relative-weight mode	79.473	84.073	83.210
ENPU fixed-share	231.534	231.591	231.607
ENPU elastic	234.858	234.574	235.288
ENPU best-effort	244.376	262.449	258.515
native (full-DIE baseline)	253.525	258.426	259.766

For these specific builds and this workload, median throughput for ENPU fixed-share / elastic / best-effort was approximately 2.783 / 2.822 / 3.107 times that of hami-core's default mode, with lower mean TTFT and TPOT. Best-effort's median throughput was close to native. These are observations from this comparison only, not a general performance ranking across all models, workloads, or versions.

Runtime identity and comparability evidence. The SHA256 of the hami-core library actually loaded was 7f6e4de8e89e4578efed06e0fd0367c5788ae987058b2368fd6a4933bf4bc6ed; ENPU used the release 1.0.0 runtime listed in Part 1. Before/after maps checks confirmed that native loaded neither interception library. All five groups used the same workload-image OCI index digest, sha256:3b8fcd8dfc5f1244d9c3eac2d9dffbe46d08619fc9a8266521dca3b8e447ee84, and the same benchmark source and synchronization hook. This review reread 45 client result files and 2,880 request samples; recomputed throughput, TTFT, and TPOT matched the archived summaries.

Interpretation limits. Request counts were fixed, and no additional traffic was supplied after clients finished. The three hami-core instances finished at different times, with all three request streams simultaneously active for approximately 51%–53% of the common timing window. The corresponding ranges were approximately 92.51%–99.97% for ENPU and 96.68%–99.32% for native. These values therefore describe total throughput for completing the same fixed batch. They must not be described as throughput under continuous full load from all three clients throughout the entire window, nor do they establish the root cause of the performance differences on their own.

Mode execution was not interleaved or randomized. The comparison uses completed reference groups under matching conditions, with only 3 rounds; it is not a full rerun against the latest PR HEAD. Runtime-library, device, and process-residency checks were discrete observations. The hami-core archive did not record the engine's own start_ticks, so PID reuse cannot be strictly excluded. Zero failed requests also does not imply anomaly-free complete logs: hami-core logged warnings such as manager takeover before timing; ENPU emitted configuration/graph-mode messages; and native logged an EngineDeadError and resource-cleanup warnings during shutdown after the final timed round.

Additional attempt with hami-core fixed-share mode. With the same library and workload, NPU_FIXED_SHARE_RATIO=true was set for one planned round. All three services became Ready with zero restarts, but warmup timed out after 600 s: two instances each completed 4 warmup requests, while the third did not. The shared measurement start was never reached, and no valid benchmark results were produced. Throughput, TTFT, and TPOT for this mode must therefore be recorded as N/A (no valid timed samples), not 0; receiving streaming HTTP 200 response headers must not be treated as completed generation.

The functional and performance records, configurations, logs, before/after snapshots, raw requests, and checksums have been archived locally. Functional validation and performance comparison retain their respective versions, pass conditions, and outstanding failures. Oversubscription swap latency is not included in the ordinary-inference performance tables.

---

中文

ENPU 适配测试报告：基础切分、超分功能及 hami-core 性能对比

关联：ascend-device-plugin #141、HAMi #3110。本报告整理此前基础诊断、单实例验证及 2026-09-22—23 的归档实测，分为功能验证和后端性能对比两部分。各阶段使用的构建不同，以下分别注明；这些结果不等于最新 PR HEAD 已完成全量硬件回归。

一、ENPU 基础切分与超分功能测试

测试环境与构建来源。 正式归档测试使用以下环境；早期历史诊断没有同等完整的环境与运行库追溯记录。

项目	配置
NPU	Ascend 910C / A3；8 卡、每卡 2 个 DIE；每 DIE 65,536 MiB HBM
CPU / RAM	Kunpeng 920 7280Z，640 个逻辑 CPU、8 个 NUMA 节点；主机 RAM 约 2013 GiB
主机系统	openEuler 22.03 LTS-SP4，aarch64；内核 5.10.0-216.0.0.115.oe2203sp4.aarch64
Kubernetes / runtime	K3s v1.36.4+k3s1；containerd 2.3.4-k3s1.36；RuntimeClass 名称与 handler 均为 ascend
驱动 / CANN	驱动 26.1.1、ascendhal 7.35.23；容器 CANN 9.1.0
业务镜像	quay.io/ascend/vllm-ascend:v0.23.0-a3；容器 Ubuntu 22.04.5 LTS
框架	torch 2.10.0+cpu + torch-npu 2.10.0.post4；vLLM 0.23.0+empty / vLLM-Ascend 0.23.0
普通推理模型	Qwen3-8B
超分模型	Qwen3.6-27B-w8a8

基础切分的正式 release 构建流程为：官方 release 1.0.0 源码 → 官方编译镜像构建运行库 → 打包进开源 device-plugin 镜像 → 通过 HAMi 部署测试。源码 commit 为 46b6d55a892429008b1e02f96b0a0f81d1176db8，编译镜像为 swr.cn-north-4.myhuaweicloud.com/ubscore/ubs-virt:oe2403-v1。实际运行的 libvruntime.so SHA256：

1d65bbf247aa524866ee7d77dcf4ebc9b216ca862c6f1d98c2fb5996722edda1

mem-swap 使用预发布源码 1072945b066ee354a99aa1400ee6d43a7b3cfd91 的隔离构建，包含运行时逻辑设备编号以及 manager 的 A3 DCMI、状态格式、REST 收包/UID 解析兼容与诊断补丁。测试采用开源插件镜像加编译产物挂载覆盖，未改变基础切分默认的 release 构建打包流程。超分运行库 SHA256：

637ec7d649d342380e6fd24f79e44092f71f2485ee933191272b6d7488760fd7

官方 release 1.0.0 不包含本次验证的 mem-swap 功能；超分结果属于上述预发布、带补丁的 runtime/manager 组合。 后期 Kubernetes 单 DIE 测试使用 A3 独立模式，保留 runtimeClassName: ascend；由 HAMi 选择设备并生成/挂载配置，容器暴露所选计算设备及必要管理设备。所需 systemd-detect-virt 及其动态库依赖也已安装到测试容器。

单实例基础计算与设备映射。 使用开源 HAMi 创建三策略 × 两个独立 DIE，共 6 个 Pod；每 Pod 申请 1 个设备、16,384 MiB 显存、core=20。每 Pod 完成 4 次 512×512 FP16 全一矩阵乘法，逐次同步并断言结果元素为 512.0。

策略	同一卡上的第一个独立 DIE	同一卡上的第二个独立 DIE
fixed-share	4/4 通过	4/4 通过
elastic	4/4 通过	4/4 通过
best-effort	4/4 通过	4/4 通过

6 个 Pod 均正常退出、零重启。逐个核对了设备节点、ASCEND_VISIBLE_DEVICES、ENPU physical ID、逻辑设备 0→实际物理 DIE、DCMI Card/Chip 映射；torch.npu.device_count()=1，没有额外设置 ASCEND_RT_VISIBLE_DEVICES。此前的 DCMI -8008 和逻辑编号越界未在这六组中复现。首批 device-plugin 尚未完成注册时曾出现 no healthy devices，等待注册后重跑通过。这组六 Pod 使用带编号修复/诊断补丁的预发布运行库，不计作原版 release 1.0.0 的验证。

单实例 vLLM：三种算力策略与三种执行配置。 Qwen3-8B，TP1，显存 32,768 MiB、core=100，gpu-memory-utilization=0.80、max-model-len=2048、max-num-seqs=4。每格 8 个请求、并发 4，benchmark 指定输入 128、输出 32 tokens。

策略	eager（enforce-eager）	graph（FULL_DECODE_ONLY）	FULL 图配置（历史脚本名 force）
fixed-share	8/8，0 失败	8/8，0 失败	8/8，0 失败
elastic	8/8，0 失败	8/8，0 失败	8/8，0 失败
best-effort	8/8，0 失败	8/8，0 失败	8/8，0 失败

九格合计 72/72 请求完成，零失败。这里的 force 是 FULL 图执行配置，与强制换入/换出无关。此阶段为历史开源集成构建，运行库由官方编译镜像中的源码构建；精确源码 commit、每格实际库 SHA256 未完整恢复，因此只认定这些启动配置下短批次推理通过，不认定为 release 1.0.0 图模式完整回归。前置 graph 试跑有 capture 3/3 日志，九格没有逐项图回放追踪；这组也没有与第二部分相同的稳定态预热和采样参数，不将其延迟混入后端性能表。

显存额度与早期基础诊断。 历史测试中，256 MiB 配额下申请 4 MiB 成功，再申请 512 MiB 时返回 ENPU Out of memory / RuntimeError，错误码 300000；脚本捕获预期异常后退出，未观察到容器被直接 OOMKill。64 MiB 配额小于当时约 113.3 MiB 的上下文开销；ENPU 模块初始化成功，但初始化后已用显存超过配额，显存查询及后续 4/128 MiB 申请失败。该阶段存在旧调度部署、混合拦截库与成对 DIE 暴露，只能作为历史现象记录，不能作为当前 release / ENPU-only 的显存硬边界验收。正式 release 已验证 20 GiB 配额视图及 18 GiB vLLM 预算；干净环境中的显存超限边界仍需单独补测。

早期 quota20/100 各 4 次 128×128 矩阵乘法，分别为 99.167/46.360 ms；两份四次样本的物理 DIE、日志级别不同，且均无显式矩阵乘预热，计时包含首次算子准备和逐次打印；另有超时记录，未将它们用于比例精度或稳定性结论。后续以以下同条件持续负载作为算力配额证据。

官方 release 1.0.0：单实例 fixed-share 的算力配额。 在同一 DIE 上依次运行三个 Docker 容器，使用相同镜像、运行库、CPU 配置和 20 GiB 显存配额。负载为 8192×8192 FP16 矩阵乘法，输入/输出预分配，每 8 次运算同步一次；每档至少预热 20 秒且完成 64 组，然后正式计时至少 60 秒，计时内无逐次打印。

aicore-quota	完成矩阵乘法次数	实际计时（s）	吞吐（次/s）	相对 100 档
100	12,384	60.038	206.270	100.000%
50	6,224	60.079	103.597	50.224%
20	2,488	60.168	41.351	20.047%

预热后与测量后的数值检查均通过，三个容器均退出 0、无 OOMKill。本负载下 fixed-share 的持续吞吐接近配置比例。100 档也加载 ENPU，属于该运行库下的基线。每档仅一轮，此结果不推定 elastic/best-effort 或 vLLM tokens/s 也严格按相同比例变化：elastic 可借用空闲资源，best-effort 不按 quota 限制算力。

该脚本末尾检查原有 Kubernetes Pod 时返回 NotFound，总脚本 exit=1；三档测量、结果校验与 Docker 清理已完成。原 Pod 消失原因未确定，其他 DIE 上的背景负载中途发生变化；不能把本次写成整个环境检查零异常或完全控制背景条件的精度认证。

官方 release 1.0.0：三策略、三实例并发推理。 每组 3 个 Pod 共享同一 DIE，均由 HAMi 分配；每 Pod 20 GiB 显存配额、core=30、vLLM 预算 18 GiB，Qwen3-8B、TP1/eager。每策略测 3 轮；每实例每轮预热 4 次、正式 64 请求、并发 4、输入/输出各 128 tokens，三客户端同步开始。

ENPU 策略	有效轮次	成功 / 总计时请求	失败	输出 tokens
fixed-share	3	576/576	0	73,728
elastic	3	576/576	0	73,728
best-effort	3	576/576	0	73,728

合计 1,728 个计时请求，零失败。前后核验了 Pod、imageID、分配设备、运行库和策略配置，工作负载无重启。本组 request=limit=quota，未启用超分。release 对配置中的 memory-request / memory-limit 存在不识别键的告警，同时读取 memory-quota=20480；因此结论为请求与配置核验通过，不表述为完整日志无告警。吞吐和延迟见第二部分。

mem-swap：三个大模型实例，十轮按需换入换出。 三 Pod 共享一个 64 GiB DIE；每 Pod 运行 Qwen3.6-27B-w8a8，TP1/eager，max-model-len=32768、max-num-seqs=1、gpu-memory-utilization=0.95。每实例 vLLM 预算为 60.8 GiB，实际稳定 HBM 约 55–56 GiB。

每 Pod 的 HAMi 显存 reservation/request=256 MiB、ENPU memory limit=65,536 MiB、core=30，策略 best-effort。manager 按官方宿主服务方式运行；测试 DIE 设置 oversub-ratio=2、CPU swap 池总量 192 GiB、swap-pre-watermark=95。HAMi 调用 manager 为 Pod 分配 vNPU 并挂载配置，核对物理设备、vNPU、shm-id、配额和设备节点；这三个 Pod 的 vNPU 映射没有手工指定。

初始化后按 A→B→C 顺序轮换 10 轮，共 30 次计分请求；每次从目标实例已换出的状态开始，稳定后一个实例驻留 NPU、另两个已换出。

实例	活动时 HBM（MiB）	换出后 HBM（MiB）	每次释放（MiB）	换入后推理
A	56,731	197	56,534	10/10 成功
B	56,931	197	56,734	10/10 成功
C	57,211	197	57,014	10/10 成功

30 次响应均生成 32 tokens，并与各实例初始化基线输出完全一致。Pod UID、容器 ID、NPU PID 保持不变、零重启，每实例只加载模型一次。初始化 A/B 时各执行过一次 force-out，以便依次加载模型；30 次计分阶段没有 force-out/force-in、vLLM sleep、暂停容器或重建模型，交换由请求按需触发。

所有结束状态均通过 npu-smi 与 PID→Pod 归属核验。CPU swap 池过渡峰值 166.29 GiB，同一活动实例对应的稳定池用量在十轮中保持一致，有限轮换内未观察到持续增长。十轮共耗时 569.25 s；请求平均 16.82 s，首个请求 31.79 s，其余 29 次平均 16.30 s。这些时间包含交换和 32-token 推理，不作为纯推理 TTFT/TPOT 或吞吐参与第二部分对比。

超分通过条件与未通过项。 同一组二进制在默认 80% 水位下，第一轮三次成功，第二轮 C→A 等待 300 s 后超时；虽然显存已释放，完成确认没有返回。95% 配置使本次负载避开了预换出竞争，不能视为协议问题已修复。每 Pod 启动阶段还存在 17 次 Python multiprocessing/forkserver signal-handler TypeError，之后模型初始化和轮换完成；另有 hook/CANN 兼容 WARN。计分轮换阶段未发现 swap/DCMI/内存错误或模型重载，但完整启动日志无异常检查仍未通过。

因此，超分结论限定为：该预发布兼容/诊断构建，在 95% 水位、best-effort、单 DIE、TP1/eager、顺序请求下，三实例十轮换入换出功能通过；默认水位和完整环境尚未通过上线验收。 尚未覆盖超分的 fixed-share/elastic、graph、多卡 TP、并发请求、长期稳定性，以及 allocation 生命周期和完整故障恢复。910A/910B 也没有本机硬件实测结论。

补充：ENPU 监控采集。 后续监控适配在两个 Docker 容器中分别保持 128/640 MiB 张量，运行真实 collector 与 HTTP metrics 采集，三轮均通过。实际 DCMI/HAMi HBM 为 259,330,048 / 796,200,960 bytes（含上下文开销），与官方 monitor 的 247/759 整数 MiB 输出一致；归属到两个容器的序列独立、采集成功指标均为 1。此硬件 harness 验证 DCMI、cgroup 归属和 HTTP 暴露，使用已知 allocation 身份，尚不是部署后的 Kubernetes informer/Prometheus 全链路验收。监控仅报告驻留 HBM，不报告 CPU swap 字节或睡眠状态，也不提供每 Pod 的实际算力利用率。

部署前提和参数以官方 vCANN-RT 配置文档及上游源码为准；正式 release 与预发布超分组件的结果需分别解读。

二、ENPU 与 hami-core 的性能对比

本部分使用此前 通过 HAMi 部署的三实例对照记录，比较 hami-core 默认相对权重模式、ENPU release 1.0.0 三策略，并保留 native 作为完整 DIE 参考基线。模型、镜像、设备、请求负载和统计方法已核对。

共同条件。 同一物理 DIE、同一 Qwen3-8B 模型目录、同一业务 imageID；BF16、无量化，3 个独立 vLLM 服务，TP1/eager，max-model-len=2048、max-num-seqs=4、generation-config=vllm、关闭 prefix caching。每实例 vLLM 总预算均为 18 GiB。

模式	算力配置 / 部署	显存预算与实际 KV
hami-core 默认模式	3 Pod × 1 服务；NPU_PRIORITY=30；未设置 NPU_FIXED_SHARE_RATIO，未覆盖 token scale / avg alpha	quota 20 GiB × util 0.9；预算 18 GiB；KV 约 2.39 GiB
ENPU 三策略	每组 3 Pod × 1 服务；aicore-quota=30；policy 分别为 1/2/3	quota=request=limit=20 GiB；预算 18 GiB；KV 约 2.19–2.35 GiB
native	1 个完整 DIE Pod 内 3 个独立服务；无拦截库	使用 matched-budget 组，按实际物理 total 配置 util，使每服务预算恰为 18 GiB；KV 约 2.45 GiB

hami-core 的 30 是相对调度权重；ENPU 的 fixed-share、elastic、best-effort 也有不同的配额语义。本表比较这些实际配置，不代表它们具有等价的硬算力上限。匹配的是 vLLM 总预算，实际 KV 大小和 native 的 Pod 布局仍有差异。

每实例每轮预热 4 次，正式发 64 个请求、并发 4、输入/输出各 128 tokens；random dataset，seed=42、temperature=0、ignore-eos=true、request-rate=inf、random-range-ratio=0、random-prefix-len=0。三客户端通过共享屏障同步开始；预热和服务启动不计时。每模式 3 轮、每轮 192 个请求，合计每模式 576 个请求、73,728 输出 tokens。五组累计 2,880 个计时请求，0 失败。

总输出吞吐按 三实例输出 tokens 总和 / (最晚结束 − 最早开始) 计算，表中报告 3 轮吞吐的中位数。TTFT 是首 token 等待时间，TPOT 按每请求 (latency − TTFT) / (output_tokens − 1) 计算；下表两种延迟均为该模式 576 个成功请求的均值，不是中位数或 P99。

模式	总输出吞吐中位数（token/s）	平均 TTFT（ms）	平均 TPOT（ms）	成功 / 失败
hami-core 默认相对权重模式	83.210	870.277	115.643	576 / 0
ENPU fixed-share	231.591	220.167	50.468	576 / 0
ENPU elastic	234.858	194.048	49.893	576 / 0
ENPU best-effort	258.515	175.557	45.170	576 / 0
native（完整 DIE 基线）	258.426	167.218	45.220	576 / 0

逐轮总吞吐如下，用于展示三轮波动：

模式	第 1 轮（token/s）	第 2 轮（token/s）	第 3 轮（token/s）
hami-core 默认相对权重模式	79.473	84.073	83.210
ENPU fixed-share	231.534	231.591	231.607
ENPU elastic	234.858	234.574	235.288
ENPU best-effort	244.376	262.449	258.515
native（完整 DIE 基线）	253.525	258.426	259.766

在上述具体构建与负载下，ENPU fixed-share / elastic / best-effort 的中位吞吐分别约为 hami-core 默认模式的 2.783 / 2.822 / 3.107 倍，平均 TTFT、TPOT 也更低；best-effort 的中位吞吐接近 native。此处只报告本组观测，不推定所有模型、负载、版本的性能排序。

运行库与可比性证据。 hami-core 实际加载的库 SHA256 为 7f6e4de8e89e4578efed06e0fd0367c5788ae987058b2368fd6a4933bf4bc6ed；ENPU 为第一部分列出的 release 1.0.0 运行库。native 前后 maps 核对未加载两种拦截库。五组使用相同业务镜像 OCI index digest sha256:3b8fcd8dfc5f1244d9c3eac2d9dffbe46d08619fc9a8266521dca3b8e447ee84，benchmark 源码与同步 hook 一致。此次复核重新读取了 45 份客户端结果与 2,880 条请求样本，重新计算的吞吐、TTFT、TPOT 均与归档汇总一致。

解释边界。 请求数固定，客户端完成后不补充流量。hami-core 三个实例完成速度存在差异，三路请求活动同时重叠约占共同计时窗口的 51%–53%；ENPU 约 92.51%–99.97%，native 约 96.68%–99.32%。因此这些数值是同一固定批次完成的总吞吐，不能写成整个窗口持续三路满载的吞吐，也不能仅据此确定差异根因。

模式执行顺序没有交错随机化，使用已完成的同条件参考组，只有 3 轮；结果不是最新 PR HEAD 的全量重测。运行库、设备和进程驻留核验为离散观测；hami-core 归档未记录 engine 自身 start_ticks，不能严格排除 PID 重用。请求零失败也不等于完整日志无异常：hami-core 有计时前 manager 接管等 WARN，ENPU 有配置/图模式提示；native 在最后一轮计时结束后的关闭阶段出现过 EngineDeadError 和资源清理告警。

hami-core 固定比例模式的额外尝试。 同一库与工作负载下设置 NPU_FIXED_SHARE_RATIO=true，计划一轮。三个服务 Ready、零重启，但预热等待 600 s 后超时：两个实例各完成 4 次预热，另一个未完成；没有进入共同计时点，也没有有效 benchmark 结果。该模式的吞吐、TTFT、TPOT 均应记为 N/A（无有效计时样本），不能填成 0 或把流式 HTTP 200 响应头当作生成完成。

以上功能与性能记录、配置、日志、前后快照、原始请求和校验和已本地归档。功能验证和性能比较分别保留版本、成功条件及未通过项；超分交换延迟未混入普通推理性能表。