feat!: localize emoji pickers and reduce package sizes

PACKAGE_SIZE_REPORT.md

@@ -0,0 +1,81 @@
+# Package size relative to main
+
+Compared actual `pnpm pack` tarballs from local `main` / `origin/main` at **3a37d23fc1060fb8e3222e8fb670787ddac6b071** (`fix(core): remove unused source preview anchor (#3099)`) with the current branch including its uncommitted optimizations. No remote refresh was performed. KB/MB are decimal. These are package download and unpacked sizes, excluding external dependency downloads and application bundle sizes.
+
+| Package           | Main tarball (KB) | Current tarball (KB) | Reduction | Main unpacked (MB) | Current unpacked (MB) |
+| ----------------- | ----------------: | -------------------: | --------: | -----------------: | --------------------: |
+| ariakit           |              67.0 |                 37.5 |     44.1% |              0.345 |                 0.192 |
+| code-block        |              15.0 |                  9.8 |     34.5% |              0.078 |                 0.042 |
+| core              |           2,097.4 |              1,634.7 |     22.1% |              9.316 |                 4.695 |
+| diagram-block     |           4,696.1 |                 38.1 |     99.2% |             18.608 |                 0.136 |
+| mantine           |             140.9 |                 37.0 |     73.7% |              0.826 |                 0.176 |
+| math-block        |             817.0 |                 41.0 |     95.0% |              8.284 |                 0.160 |
+| react             |           4,208.7 |                219.5 |     94.8% |             23.604 |                 0.944 |
+| server-util       |              16.9 |                 10.3 |     39.2% |              0.085 |                 0.040 |
+| shadcn            |             110.5 |                 51.8 |     53.2% |              0.580 |                 0.255 |
+| xl-ai             |           1,256.8 |                113.7 |     91.0% |              6.928 |                 0.463 |
+| xl-ai-server      |              26.4 |                 19.3 |     26.9% |              0.099 |                 0.058 |
+| xl-docx-exporter  |           1,495.6 |                605.0 |     59.5% |              3.378 |                 1.138 |
+| xl-email-exporter |              67.2 |                 30.1 |     55.2% |              0.378 |                 0.115 |
+| xl-multi-column   |             784.7 |                 33.4 |     95.7% |              8.275 |                 0.130 |
+| xl-odt-exporter   |           1,293.0 |                397.8 |     69.2% |              3.270 |                 1.003 |
+| xl-pdf-exporter   |          25,039.1 |              6,277.3 |     74.9% |             47.985 |                12.026 |
+| xl-typst-compiler |           9,004.2 |              8,999.5 |      0.1% |             26.026 |                25.999 |
+| xl-typst-exporter |              79.0 |                 35.5 |     55.1% |              0.270 |                 0.117 |
+
+Across all 18 packages, including the private AI server:
+
+- Compressed: **51.22 MB → 18.59 MB**, saving **32.62 MB (63.7%)**.
+- Unpacked: **158.33 MB → 47.69 MB**, saving **110.64 MB (69.9%)**.
+- Packed files: **2,557 → 1,674**.
+
+For the 17 public packages only: **51.19 MB → 18.57 MB compressed**, and **158.23 MB → 47.63 MB unpacked**.
+
+Core now includes this branch's localized emoji data and is still **22.1% smaller compressed** than main. Main uses external `@emoji-mart/data` and `emoji-mart` dependencies; their download sizes are outside these tarball totals. This is a net branch comparison, including other source differences from main, rather than an isolated measurement of individual optimizations.
+
+The earlier **56.52 MB → 18.59 MB** comparison reconstructed the old packaging settings against this branch's source, including its added emoji data. It was useful for isolating packaging cleanup but was not a main comparison. The table above supersedes it when comparing this branch with main.
+
+## Measurement method
+
+- Built main in an isolated detached worktree with its own frozen lockfile and original build configuration. All 18 library builds passed with task caches disabled after building the shared declaration prerequisite.
+- Used this branch's successfully built library artifacts and prepared README files on both sides as the publish workflow does. Temporary README preparation in the working branch was restored after packing.
+- The unchanged Typst Rust output was reused after verifying all Rust source/configuration inputs and the wasm-pack version matched; its generated JS and WASM assets are identical between sides. Both package-level JS/declaration builds ran normally.
+- Packed all 18 matching packages and measured the `.tgz` file bytes plus the sum of regular file sizes inside each archive. Raw results and tarballs are in `/tmp/blocknote-vs-main/`; main build logs are `/tmp/blocknote-main-build-retry.log`.
+- Main ships 93.69 MB of sourcemaps and 14.28 MB of CJS files unpacked across these packages; both categories are absent from the current tarballs. Those raw byte counts are not additive compressed savings.
+
+## Changes
+
+- Removed CJS builds/exports and sourcemap emission from library packages.
+- Excluded tests, snapshots, fixtures, test helpers, build metadata, and build-stat reports. Core packages runtime files, declarations, and fonts without its source tree.
+- Emoji encoding reduced the shared payload from 11,364 to 7,367 gzip bytes (35.2%). This improvement was already present on both sides of the earlier reconstructed packaging baseline; main uses external emoji data. Locale loads also reuse promises.
+- Confirmed that used `react-icons` icons are bundled. It is a development dependency; published runtime modules/declarations do not import it.
+- Removed AI's direct `lodash.isequal` and `lodash.merge` dependencies and their type dependencies. Schema equality handles JSON values, including a property named `constructor`. Request composition preserves custom metadata, headers, and body fields while replacing generated document state and tool definitions. This deliberately avoids stale blocks/selections from recursive array merging and does not mutate caller-owned options.
+- Applied valid e18e syntax suggestions: `.at()`, `toSorted()`, `toReversed()`, `Object.hasOwn()`, nullish assignment, exponentiation, and array/object composition. Missing values are handled explicitly where necessary.
+- Aligned Floating UI and Vitest versions, then ran `pnpm dedupe`. This final dedupe reduced distinct lockfile package-version entries from 2,070 to 2,036 without introducing new package versions. Incompatible ranges, exact pins, peer environments, and workspace-link paths still require separate entries.
+- Set the private AI server's `sideEffects` to `true`: its entry point starts the server and configures the global dispatcher.
+
+The final e18e pass primarily reduces dependency requirements and workspace installation duplication. Its syntax changes have no meaningful tarball saving: combined tarball size remains approximately 18.59 MB. The large download reductions come from packaging/build cleanup. External dependencies are not included in tarball totals, so their removal must not be counted again as tarball savings. Base64 fonts remain unchanged for bundler and offline compatibility.
+
+## Remaining e18e findings
+
+Audited with `@e18e/cli` 0.7.0 against isolated package copies plus source. Each copy used its workspace importer projected into the lockfile root. Duplicate reports include the shared workspace lockfile and link aliases; zero install size is not an installation measurement because audit copies omit node_modules. Raw reports and validation logs are in `/tmp/blocknote-e18e-apply/` for this session.
+
+All direct dependency replacement findings were addressed. The remaining 23 syntax file findings are false positives or non-equivalent suggestions:
+
+- Eleven `arrayFill` suggestions match empty-array declarations but have no applicable rewrite: loops create varying values, decorations, operations, or UI elements. The CLI's own transform leaves these unchanged.
+- The multi-column factory must create independent objects; `.fill(object)` would share state.
+- Four sorting suggestions operate on Map iterators, which do not have `toSorted()`. Spreading and sorting the new array already makes a single copy.
+- Three `.at()` suggestions target assignment destinations. Indexed assignment is required; corresponding reads were modernized.
+- Three exporters combine a dynamic list of style objects with `Object.assign({}, ...stylesArray)`. The proposed spread rewrite is invalid syntax; repeated spread reduction would add allocations.
+- Table-cell normalization uses `concat` to accept both arrays and scalar content. Blind spreading changes strings into characters and fails for non-iterables.
+
+The server must retain side effects. Incompatible dependency majors were not forced together to suppress duplicate warnings.
+
+## Validation
+
+- All 18 library builds passed with task caches disabled.
+- Core: 777 tests passed, 9 skipped. React: 13 passed.
+- Targeted AI stream-tool, ProseMirror, and request tests: 72 passed, 27 skipped. Request composition and schema comparison regression tests also passed after final edits.
+- Type-aware lint and frozen-lockfile installation passed.
+- All 18 tarballs passed checks for excluded files, export targets, and relative runtime/declaration imports. No missing module references were found.
+- Browser E2E was not run.

README.md

@@ -29,6 +29,9 @@ See our homepage @ [https://www.blocknotejs.org](https://www.blocknotejs.org/) o
 
 [![npm version](https://badge.fury.io/js/%40blocknote%2Freact.svg)](https://badge.fury.io/js/%40blocknote%2Freact)
 
+BlockNote packages are ESM-only. Use `import` (or dynamic `import()`) to load them;
+CommonJS `require()` entry points are no longer provided.
+
 ```typescript
 import { useCreateBlockNote } from "@blocknote/react";
 import { BlockNoteView } from "@blocknote/mantine";

docs/content/docs/features/localization.mdx

@@ -25,11 +25,13 @@ BlockNote supports the following languages out of the box:
 - **Japanese** (`ja`) - 日本語
 - **Korean** (`ko`) - 한국어
 - **Norwegian** (`no`) - Norsk
+- **Persian** (`fa`) - فارسی
 - **Polish** (`pl`) - Polski
 - **Portuguese** (`pt`) - Português
 - **Russian** (`ru`) - Русский
 - **Slovak** (`sk`) - Slovenčina
 - **Spanish** (`es`) - Español
+- **Turkish** (`tr`) - Türkçe
 - **Ukrainian** (`uk`) - Українська
 - **Uzbek** (`uz`) - O'zbekcha
 - **Vietnamese** (`vi`) - Tiếng Việt
@@ -110,6 +112,32 @@ function CustomEditor() {
 
 The dictionary object contains translations for various parts of the editor:
 
+### Locale
+
+The optional `locale` field selects the localized emoji names, categories,
+skin-tone names, and search terms used by the emoji picker. BlockNote's bundled
+dictionaries already set this field. When creating a custom dictionary, set it
+to the matching locale code; if it is omitted or unsupported, the emoji data
+falls back to English.
+
+```tsx
+const customFrenchDictionary = {
+  ...fr,
+  locale: "fr",
+  // Your translated or customized dictionary fields.
+};
+```
+
+The picker interface strings use the same dictionary as the rest of BlockNote:
+
+```tsx
+emoji_picker: {
+  search: "Search",
+  search_no_results: "No emoji found",
+  loading: "Loading…",
+}
+```
+
 ### Placeholders
 
 Text that appears when blocks are empty:
@@ -203,6 +231,8 @@ To add support for a new language, you can:
 
 When creating translations, make sure to:
 
+- Set `locale` to the matching locale code so the emoji picker uses the
+  appropriate localized data
 - Translate all text strings in the dictionary
 - Maintain the same structure as the English dictionary
 - Test the translations with different content types

examples/01-basic/10-localization/README.md

@@ -1,6 +1,6 @@
 # Localization (i18n)
 
-In this example, we pass in a custom dictionary to change the interface of the editor to use Dutch (NL) strings.
+In this example, we use a dropdown to switch the editor between all of the languages that BlockNote ships with, by passing the matching dictionary from `@blocknote/core/locales` to the editor. Selecting a language re-mounts the editor so the interface strings update immediately.
 
 You can also provide your own dictionary to customize the strings used in the editor, or submit a Pull Request to add support for your language of your choice.
 

examples/01-basic/10-localization/src/App.tsx

@@ -1,21 +1,79 @@
-import { nl } from "@blocknote/core/locales";
+import type { Dictionary } from "@blocknote/core";
+import * as locales from "@blocknote/core/locales";
 import "@blocknote/core/fonts/inter.css";
 import { BlockNoteView } from "@blocknote/mantine";
 import "@blocknote/mantine/style.css";
 import { useCreateBlockNote } from "@blocknote/react";
-// import { useTranslation } from "some-i18n-library"; // You can use any i18n library you like
+import { useState } from "react";
+
+// Every dictionary exported by `@blocknote/core/locales`, keyed by its
+// language code (the same key used in the export, e.g. `zhTW`).
+const dictionaries: Record<string, Dictionary> = locales;
+
+// Human-readable names shown in the language picker.
+const languageNames: Record<string, string> = {
+  ar: "العربية",
+  de: "Deutsch",
+  en: "English",
+  es: "Español",
+  fa: "فارسی",
+  fr: "Français",
+  he: "עברית",
+  hr: "Hrvatski",
+  is: "Íslenska",
+  it: "Italiano",
+  ja: "日本語",
+  ko: "한국어",
+  nl: "Nederlands",
+  no: "Norsk",
+  pl: "Polski",
+  pt: "Português",
+  ru: "Русский",
+  sk: "Slovenčina",
+  tr: "Türkçe",
+  uk: "Українська",
+  uz: "Oʻzbekcha",
+  vi: "Tiếng Việt",
+  zh: "简体中文",
+  zhTW: "繁體中文",
+};
+
+const languageKeys = Object.keys(dictionaries).sort((a, b) =>
+  (languageNames[a] ?? a).localeCompare(languageNames[b] ?? b),
+);
+
+// Creates the editor with the given dictionary. This is a separate component so
+// that changing its `key` (see below) re-mounts it, re-creating the editor
+// instance with the newly selected language.
+function LocalizedEditor(props: { dictionary: Dictionary }) {
+  const editor = useCreateBlockNote({ dictionary: props.dictionary });
 
-export default function App() {
-  // const { lang } = useTranslation('common'); // You can get the current language from the i18n library or alternatively from a router
-  // Creates a new editor instance.
-  const editor = useCreateBlockNote({
-    // Passes the Dutch (NL) dictionary to the editor instance.
-    // You can also provide your own dictionary here to customize the strings used in the editor,
-    // or submit a Pull Request to add support for your language of your choice
-    dictionary: nl,
-    // dictionary: locales[lang as keyof typeof locales], // Use the language from the i18n library dynamically
-  });
-
-  // Renders the editor instance using a React component.
   return <BlockNoteView editor={editor} />;
 }
+
+export default function App() {
+  // The currently selected language.
+  const [language, setLanguage] = useState<keyof typeof dictionaries>("en");
+
+  return (
+    <div>
+      <label>
+        Language:{" "}
+        <select
+          value={language}
+          onChange={(event) => setLanguage(event.target.value)}
+        >
+          {languageKeys.map((key) => (
+            <option key={key} value={key}>
+              {languageNames[key] ?? key}
+            </option>
+          ))}
+        </select>
+      </label>
+
+      {/* Keying the editor by the language re-mounts it whenever the selection
+          changes, so the editor picks up the newly selected dictionary. */}
+      <LocalizedEditor key={language} dictionary={dictionaries[language]} />
+    </div>
+  );
+}

packages/ariakit/package.json

@@ -15,7 +15,23 @@
   "files": [
     "dist",
     "types",
-    "src"
+    "src",
+    "!**/*.cjs",
+    "!**/*.map",
+    "!**/*.test.*",
+    "!**/*.spec.*",
+    "!**/*.snap",
+    "!**/__test__/**",
+    "!**/__tests__/**",
+    "!**/tests/**",
+    "!**/__snapshots__/**",
+    "!**/__msw_snapshots__/**",
+    "!**/fixtures/**",
+    "!**/testUtil/**",
+    "!**/testUtils/**",
+    "!**/test-utils/**",
+    "!**/*.tsbuildinfo",
+    "!dist/webpack-stats.json"
   ],
   "keywords": [
     "react",
@@ -34,17 +50,15 @@
   "type": "module",
   "source": "src/index.tsx",
   "types": "./types/src/index.d.ts",
-  "main": "./dist/blocknote-ariakit.cjs",
+  "main": "./dist/blocknote-ariakit.js",
   "module": "./dist/blocknote-ariakit.js",
   "exports": {
     ".": {
       "types": "./types/src/index.d.ts",
-      "import": "./dist/blocknote-ariakit.js",
-      "require": "./dist/blocknote-ariakit.cjs"
+      "import": "./dist/blocknote-ariakit.js"
     },
     "./style.css": {
       "import": "./dist/style.css",
-      "require": "./dist/style.css",
       "style": "./dist/style.css"
     }
   },

packages/ariakit/tsconfig.json

@@ -7,7 +7,7 @@
     "moduleResolution": "Bundler",
     "jsx": "react-jsx",
     "strict": true,
-    "sourceMap": true,
+    "sourceMap": false,
     "resolveJsonModule": true,
     "esModuleInterop": true,
     "noEmit": false,

packages/ariakit/vite.config.ts

@@ -46,16 +46,15 @@ export default defineConfig(
               } as Record<string, string>),
       },
       build: {
-        sourcemap: true,
+        sourcemap: false,
         lib: {
           entry: {
             "blocknote-ariakit": path.resolve(__dirname, "src/index.tsx"),
           },
           name: "blocknote-ariakit",
           cssFileName: "style",
-          formats: ["es", "cjs"],
-          fileName: (format, entryName) =>
-            format === "es" ? `${entryName}.js` : `${entryName}.cjs`,
+          formats: ["es"],
+          fileName: (_format, entryName) => `${entryName}.js`,
         },
         rollupOptions: {
           // make sure to externalize deps that shouldn't be bundled
@@ -80,14 +79,6 @@ export default defineConfig(
               source.startsWith("node:")
             );
           },
-          output: {
-            // Provide global variables to use in the UMD build
-            // for externalized deps
-            globals: {
-              react: "React",
-              "react-dom": "ReactDOM",
-            },
-          },
         },
       },
     }) as UserConfig,

packages/code-block/package.json

@@ -13,7 +13,23 @@
   "files": [
     "dist",
     "types",
-    "src"
+    "src",
+    "!**/*.cjs",
+    "!**/*.map",
+    "!**/*.test.*",
+    "!**/*.spec.*",
+    "!**/*.snap",
+    "!**/__test__/**",
+    "!**/__tests__/**",
+    "!**/tests/**",
+    "!**/__snapshots__/**",
+    "!**/__msw_snapshots__/**",
+    "!**/fixtures/**",
+    "!**/testUtil/**",
+    "!**/testUtils/**",
+    "!**/test-utils/**",
+    "!**/*.tsbuildinfo",
+    "!dist/webpack-stats.json"
   ],
   "keywords": [
     "react",
@@ -32,13 +48,12 @@
   "type": "module",
   "source": "src/index.ts",
   "types": "./types/src/index.d.ts",
-  "main": "./dist/blocknote-code-block.cjs",
+  "main": "./dist/blocknote-code-block.js",
   "module": "./dist/blocknote-code-block.js",
   "exports": {
     ".": {
       "types": "./types/src/index.d.ts",
-      "import": "./dist/blocknote-code-block.js",
-      "require": "./dist/blocknote-code-block.cjs"
+      "import": "./dist/blocknote-code-block.js"
     }
   },
   "scripts": {

packages/code-block/tsconfig.json

@@ -7,7 +7,7 @@
     "moduleResolution": "bundler",
     "jsx": "react-jsx",
     "strict": true,
-    "sourceMap": true,
+    "sourceMap": false,
     "resolveJsonModule": true,
     "esModuleInterop": true,
     "noEmit": false,