Skip to content

Add model training, export, and read-only inference - #2

Open
Kaushik-Kumar-CEG wants to merge 5 commits into
gsoc/package-foundationfrom
gsoc/training-pipeline
Open

Add model training, export, and read-only inference#2
Kaushik-Kumar-CEG wants to merge 5 commits into
gsoc/package-foundationfrom
gsoc/training-pipeline

Conversation

@Kaushik-Kumar-CEG

@Kaushik-Kumar-CEG Kaushik-Kumar-CEG commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

adds DeBERTa BIOES+CRF training, export, and read only inference.

the training command validates the generated dataset, selects the best checkpoint using validation F1, verifies the final model by reloading it offline, and writes a reproducibility manifest. Export supports CRF matrices and optional ONNX emissions. The inference API returns phrase text, offsets, model scores and truncation status without changing rules

Usage:

python -m pip install ".[training]"
train-required-phrase-model --data-dir dataset-output --output-dir model-output --model-revision COMMIT
export-required-phrase-model --model-dir model-output/final-model --output-dir model-export

Test it with:

./configure --dev
python -m pip install -e ".[training]"
venv/bin/pytest -q tests/test_training.py tests/test_model.py tests/test_export.py tests/test_inference.py
venv/bin/ruff check

The final model run reached 0.9008 validation F1 and 0.8766 test F1 with no invalid BIOES paths. This PR is stacked on PR #1

Package the DeBERTa BIOES and CRF training pipeline with strict dataset and checkpoint validation. Add ONNX export, focused tests, documentation, and installed command entry points.

Signed-off-by: Kaushik Kumar <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
@Kaushik-Kumar-CEG Kaushik-Kumar-CEG changed the title Add model training and export commands Add model training, export, and read-only inference Sep 7, 2026
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant