New Wasm workloads: Transformers.js ML sentiment analysis and Speech-to-Text by danleh · Pull Request #148 · WebKit/JetStream

danleh · 2025-08-18T16:32:24Z

These could be replacements for the tfjs workloads (even though the model file size issue remains). Run via transformersjs-bert-wasm and transformersjs-whisper-wasm.

TODOs: Evaluate startup/model loading performance, take CPU profile, decide whether Whisper task is too long-running, compress model files on-disk for repo size, modify NPM server to handle those?

netlify · 2025-08-18T16:32:30Z

✅ Deploy Preview for webkit-jetstream-preview ready!

Name	Link
🔨 Latest commit	`6b52dad`
🔍 Latest deploy log	https://app.netlify.com/projects/webkit-jetstream-preview/deploys/68bfec3b0ebfd90008dcdcb2
😎 Deploy Preview	https://deploy-preview-148--webkit-jetstream-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

eqrion · 2025-08-19T16:01:14Z

I am getting the following error when running the deploy preview in Firefox:

Uncaught (in promise) Error: Unexpected fetch: blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/bb376a7e-5209-4208-ac76-defa885759a2
    fetch blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:23
    getBinary https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm line 688 > injectedScript:38
    init blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:45
    doRun https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:57
    <anonymous> https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:80
[ac76acf2-8e08-42c9-a0b5-087158373d53:23:11](blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53)
    start https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:257
    AsyncFunctionThrow self-hosted:804
    (Async: async)
    onclick https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:453
    (Async: EventHandlerNonNull)
    prefetchResources https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:451
    InterpretGeneratorResume self-hosted:1332
    AsyncFunctionNext self-hosted:800
    (Async: async)
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:405
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:53
    onload https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:1

danleh · 2025-08-20T14:56:32Z

I am getting the following error when running the deploy preview in Firefox:

Uncaught (in promise) Error: Unexpected fetch: blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/bb376a7e-5209-4208-ac76-defa885759a2
    fetch blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:23
    getBinary https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm line 688 > injectedScript:38
    init blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:45
    doRun https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:57
    <anonymous> https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:80
[ac76acf2-8e08-42c9-a0b5-087158373d53:23:11](blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53)
    start https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:257
    AsyncFunctionThrow self-hosted:804
    (Async: async)
    onclick https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:453
    (Async: EventHandlerNonNull)
    prefetchResources https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:451
    InterpretGeneratorResume self-hosted:1332
    AsyncFunctionNext self-hosted:800
    (Async: async)
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:405
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:53
    onload https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:1

My bad, the dynamic import of ONNX runtime didn't work with the blob/preloading in the browser, fixed.

kmiller68 · 2025-08-26T15:35:34Z

Seems like both Bert and Whisper spend a majority of the time in whatever function index 7460 is (I assume some matrix multiplication function). Although, it does look like about 70-75% vs 50-55% in 7460, respectively, so Whisper is doing other things. They also look pretty similar in where they spend their time within that function.

The fact that Whisper does other things (and IIUC, is more popular) makes it somewhat more interesting. On the other hand, it seems like the dominant function in either case is the same, and running faster is a significant benefit.

danleh · 2025-09-03T16:29:54Z

I didn't look at the Wasm in detail yet, but I think your assumption about function 7460 is correct, at least it contains a bunch of SIMD shuffles, dot products, and adds. WAT excerpt:

...
           local.get $var6
           v128.load offset=16 align=1
           local.get $var8
           v128.load align=1
           local.tee $var73
           v128.const i32x4 0x00000000 0x00000000 0x00000000 0x00000000
           i8x16.shuffle 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3
           local.tee $var78
           i32x4.dot_i16x8_s
           local.get $var71
           i32x4.add
           local.get $var6
           v128.load offset=48 align=1
           local.get $var73
           v128.const i32x4 0x00000000 0x00000000 0x00000000 0x00000000
           i8x16.shuffle 4 5 6 7 4 5 6 7 4 5 6 7 4 5 6 7
           local.tee $var74
           i32x4.dot_i16x8_s
           i32x4.add
           local.get $var6
           v128.load offset=80 align=1
           local.get $var73
           v128.const i32x4 0x00000000 0x00000000 0x00000000 0x00000000
           i8x16.shuffle 8 9 10 11 8 9 10 11 8 9 10 11 8 9 10 11
           local.tee $var75
           i32x4.dot_i16x8_s
           i32x4.add
...

I'll take a more in-depth look tomorrow, i.e., overlap / difference between the Bert and Whisper tasks, and what Whisper does besides the computational kernel.

danleh · 2025-09-03T17:10:50Z

Regarding the profiles of the Whisper vs. Bert task, it does seem to me that Whisper is substantially "flatter" or more diverse, and subsumes Bert. See:

Whisper bottom-up profile:

Bert bottom-up profile:

So in that light, I'd rather search for ways to make Whisper quicker (although the audio snippet is already pretty short and low number of iterations) and keep that.

In either case, how about merging this since we are generally happy with having this style of workload, and we can still disable and tune in follow-up PRs?

kmiller68

LGTM

kmiller68 · 2025-09-16T02:53:29Z

Interestingly, on my M2 Pro Mac I see way more time in 7460 than you're reporting (at least in V8 cli, FF browser, Safari/jsc cli).

Whisper:

Bert:

In either case, how about merging this since we are generally happy with having this style of workload, and we can still disable and tune in follow-up PRs?

Sounds good to me

danleh · 2025-09-16T09:52:20Z

Thanks, merging! Also interesting difference in the profiles, maybe our x64 backend got more love than arm64, so we should probably look into this in the next weeks/months.

danleh added 9 commits August 14, 2025 17:24

Transformers.js models working in d8; WIP other shells, another model

7b2bdb9

fix paths

6d48daa

both distilbert and Whisper running in d8

8bab51e

cleanup dependencies, TODOs

e1fc06b

make it compatible with all shells

25bb946

reuse same TextEncoder/Decoder polyfill as other wasm line item

9ae0fb4

WIP adapt to runner changes, new preloading code

18b6918

Merge branch 'main' into transformersjs-rebase

0e86613

fix both tasks, use preloading from runner/JetStream global

06610f5

danleh mentioned this pull request Aug 18, 2025

Stop running tfjs benchmarks by default #145

Merged

fix accidentally committed dir

abec374

danleh added 9 commits August 19, 2025 19:31

fix browser fetch with blob preloading, fix print calls

1c425af

print is no longer used

8f14f8f

shorten audio clip to speed up whisper task

7aa469a

shorten iterations for bert task

f8147cd

add output validation

1d122da

polyfill performance.timeOrigin for transformersjs

c9c67c0

update transformersjs dependency to latest

71e1ff0

cleanup, rebuild

f3bcfb4

fix module resolution with blob URL/preload

fb0ab2b

danleh mentioned this pull request Sep 1, 2025

Zlib compress all wasm files and decompress them during prefetch #170

Merged

add tag to transformersjs workloads

65b7e97

danleh requested a review from kmiller68 September 8, 2025 14:26

Merge branch 'main' into transformersjs-rebase

28b203c

danleh added 2 commits September 9, 2025 10:55

fix duplicate TextDecoder polyfill

5b5e6c5

fix new allowUtf16 test

6b52dad

kmiller68 approved these changes Sep 16, 2025

View reviewed changes

Comment thread transformersjs/benchmark.js Resolved

danleh merged commit 2848b14 into WebKit:main Sep 16, 2025

danleh deleted the transformersjs-rebase branch October 30, 2025 13:53

Navigation Menu

Repository navigation

New Wasm workloads: Transformers.js ML sentiment analysis and Speech-to-Text - #148

New Wasm workloads: Transformers.js ML sentiment analysis and Speech-to-Text#148
danleh merged 23 commits into
WebKit:mainfrom
danleh:transformersjs-rebase

danleh commented Aug 18, 2025

Uh oh!

netlify Bot commented Aug 18, 2025 •

edited

Loading

Uh oh!

eqrion commented Aug 19, 2025

Uh oh!

danleh commented Aug 20, 2025

Uh oh!

kmiller68 commented Aug 26, 2025

Uh oh!

danleh commented Sep 3, 2025

Uh oh!

danleh commented Sep 3, 2025

Uh oh!

kmiller68 left a comment

Uh oh!

Uh oh!

kmiller68 commented Sep 16, 2025 •

edited

Loading

Uh oh!

danleh commented Sep 16, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

danleh commented Aug 18, 2025

Uh oh!

netlify Bot commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for webkit-jetstream-preview ready!

Uh oh!

eqrion commented Aug 19, 2025

Uh oh!

danleh commented Aug 20, 2025

Uh oh!

kmiller68 commented Aug 26, 2025

Uh oh!

danleh commented Sep 3, 2025

Uh oh!

danleh commented Sep 3, 2025

Uh oh!

kmiller68 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kmiller68 commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danleh commented Sep 16, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify Bot commented Aug 18, 2025 •

edited

Loading

kmiller68 commented Sep 16, 2025 •

edited

Loading