'from docling.chunking import HybridChunker' fails: No module named 'tree_sitter'

[yurivict]

The user has this problem with docling-chunker:

I'm having the following error trying tu use Docling:

 from docling.chunking import HybridChunker
   File "/usr/local/lib/python3.12/site-packages/docling/chunking/__init__.py", line 9, in <module>  
 from docling_core.transforms.chunker.base import BaseChunk, BaseChunker, BaseMeta
   File "/usr/local/lib/python3.12/site-packages/docling_core/transforms/chunker/__init__.py", line 12, in <module>
 from docling_core.transforms.chunker.code_chunking.standard_code_chunking_strategy import (
   File "/usr/local/lib/python3.12/site-packages/docling_core/transforms/chunker/code_chunking/standard_code_chunking_strategy.py", line 8, in <module>
 from docling_core.transforms.chunker.code_chunking._language_code_chunkers import (
   File "/usr/local/lib/python3.12/site-packages/docling_core/transforms/chunker/code_chunking/_language_code_chunkers.py", line 8, in <module>
 from tree_sitter import Node, Parser, Tree ModuleNotFoundError: No module named 'tree_sitter'

ChatGPT says:

 * HybridChunker cannot even be imported because tree_sitter is absent.
 * Upstream declares Tree-sitter, grammar packages, semchunk, and transformers in the chunking extra.
 * The FreeBSD port installs only the base docling-core dependencies.
 * The immediate fix depends on PR 289772 because current Docling requires a newer Tree-sitter API (tree-sitter >= 0.25)
 * Additional ports may be needed for tree-sitter-c and tree-sitter-typescript.

The FreeBSD port has all required dependencies, but optional dependencies are used and they are missing.
If they are used in the course of normal operation, why are they optional?

Please all imports used in the code required dependencies.

[maxmilian]

Some context in case it helps, both for the port and for triage.

The chunker is intentionally behind an extra. tree-sitter (plus the grammar packages), semchunk and transformers are declared in docling-core[chunking], and transformers in particular is a large dependency most docling-core users never need. On the docling side, the matching extra is docling[feat-chunking], which pulls in docling-core[chunking]. So from docling.chunking import HybridChunker is expected to fail on a base install. To use it, install:

pip install 'docling[feat-chunking]'
# or, docling-core only:
pip install 'docling-core[chunking]'

Adding tree_sitter alone won't be enough. I tried a base-only venv: once tree_sitter is installed, the import fails next on huggingface_hub, which comes in through transformers. For the FreeBSD port, the chunker needs everything listed under chunking in docling-core's pyproject.toml: semchunk, tree-sitter>=0.25,<0.27, tree-sitter-python, tree-sitter-c, tree-sitter-javascript, tree-sitter-typescript and transformers. One option is to package it as a separate chunking flavor rather than adding all of that to the base port.

What could be improved upstream is the error message. docling_core.transforms.chunker imports the code-chunking module eagerly, so a missing extra shows up as a bare ModuleNotFoundError: No module named 'tree_sitter' with no hint about which extra to install.