Repository navigation
Training session 02: slide deck, notebooks, and Slides page - #64
Merged
rafmacalaba merged 31 commits intoNov 12, 2026
Merged
Conversation
Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Adds training/session-02-small-ai-synthetic-data/slides/: - deck.html: 40 slides in one file, with speaker notes, a contents panel, and demonstrations on the slides. - outline.md: the headline and the speaker notes of every slide, generated for review. - src/: the slide sources, the styles, and the demonstration script. - build_deck.py, fit_slides.py, check_deck.py: the build, the text scale of each slide, and the browser checks. The deck reads its recorded data from assets/overview.json. The training index lists the slides of session 02 as drafted. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The slides, the speaker notes, and the demonstration labels no longer mention experiments. The notes on steps 8 and 9 now state what the worked example did: people did not compare the cases, and the example ends at the test on real reports. Slide 3 fits at a slightly larger text scale after the shorter wording. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The deck referred to "the worked example" on slides that come before the task is presented. Those references now name the report-tagging model, and part 5 is called "Report tagging". - The step slides (16 to 24) no longer carry a note on the use of the step for that model. Slide 26 holds this information for all nine steps. - The task slide is now slide 25 and the steps-used slide is slide 26, so the task is presented first. - The captions of the model sizes on slide 8 are set in build_deck.py. The data file of the overview page is unchanged. - src/slides-26-40.html is renamed to src/slides-25-40.html. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 3 said the small model for occupation coding was tested on job titles from a survey. The test titles come from a job board. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The deck followed a model that sorts complaint messages. It now follows a model that suggests ISCO-08 codes for the occupations written in a household survey, and a coder chooses among the suggestions. - Slide 2 shows a survey response, the code given by a coder, and the five codes suggested by the small model, for eight responses. - Slides 10 to 13 and the demonstrations of the nine steps use survey responses and occupation groups. - Slide 21 shows the share of 1,000 job-board titles with the right code among the first 1, 3, 5 and 10 suggestions (35.1, 50.6, 56.5 and 63.6 percent, means of three training runs). - Slide 22 shows the occupation model next to the two report-tagging repeats. - src/occupation.json holds the recorded scores and suggestions. The survey responses and the coders' codes are illustrations written for the session, and the slides state this. Group names follow ISCO-08; the group descriptions are shortened paraphrases. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 5 listed the five questions of the session. It now shows the process in five boxes: the task, the practice cases written by a large model, the training of the small model, the test on real cases, and the small model in use. A loop runs from the test back to the practice cases. The parts of the session appear under the boxes that they cover, numbered in their order, and each one links to its first slide. The slide stays at the end of the Tasks part, before Small model, so the number of slides is unchanged. The notes of slide 15 refer back to the diagram. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 21, the sources slide, the step 6 notes and the slides README now name the test data of the occupation-coding model: the dataset tabiya/occupation_titles_esco (CC BY 4.0), with job titles from the job board hahu.jobs. They state that each code is the ESCO occupation assigned by the job board, mapped to its ISCO-08 unit group. The sources slide also cites the ILO publication of ISCO-08. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The label over the parts on slide 5 now reads "Parts of the session". Seven other labels and captions that joined a second phrase to a statement with a comma are rewritten as one phrase. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 15 marks steps 1 to 4 as specific to synthetic data and steps 5 to 9 as standard machine learning practice, with a definition of machine learning and a check of the marks. The speaker notes and the overview page state the same after the list of nine steps. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 3 lists eleven tasks in two rows. The first row holds the applications named in the session description: occupation codes (ISCO), industry codes (ISIC), document classification, metadata quality, semantic search, and entity extraction. The second row keeps the five service tasks. Each card states the synthetic examples that a large model would write for the task. The semantic-search card presents GISTEmbed and shows the first result recorded in the repository README. Slide 36 states the two measured comparisons with large models, and slide 40 cites ISIC Rev. 4 and the GISTEmbed paper. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 15 has an arrow from the group Improve and maintain back to the group Write the cases. Its caption says that the large model writes more cases where the small model fails. The notes describe the loop, and a check tests the position of the arrow. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Slide 15 shows labelled job titles rewritten by a large model as survey answers, with the scores of the model before and after. Slide 36 gives the requests, tokens, kept examples, training runs, and minutes of the three builds from the run logs, with an example price for the tokens. Slide 41 lists the people, the real cases, and the computer for a first attempt. The sources slide links the two notebooks. The deck has 43 slides. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
assets/occupation holds the job titles with ISCO-08 codes from the Tabiya dataset (CC BY 4.0), the survey-style answers written from them with the result of the second model's check, the titles written from the ISCO-08 definitions, the two test sets, the illustrative answers, and the recorded scores. assets/topics holds the topic list, the sentences written by the large model, and the report abstracts for the threshold and the test. assets/README.md describes each file and its source. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Notebook 01 trains a small model for ISCO-08 unit groups on 44,058 texts: job titles from a job board, survey-style answers written from those titles by a large model, and titles written from the ISCO-08 definitions. Notebook 02 trains a small model for 86 report topics on 20,296 sentences written by a large model and tests it on 300 report abstracts. Both notebooks are generated by build_notebooks.py and committed with their recorded outputs. The training texts come from the assets folder, so the notebooks run without a key for a large model. visuals.py and explainers.py draw the animated SVG pictures. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The training index and the session README list the two session 02 notebooks with their Colab links. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
…dels Slide 10 defines a labelled example and states that the model needs examples of every group. Slide 12 gives the count of ISCO-08 groups absent from 1,000 real job titles as a fact about the titles. Both slides no longer refer to the occupation-coding model, which the deck presents later. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Each of the six parts now opens with a divider slide that names the part, describes its content in one line, and lists the six parts with the current one marked. The dividers are generated from the DIVIDERS table in build_deck.py and have the ids p1 to p6. The other slides keep the ids s01 to s43, so an id is no longer the slide number. The deck has 49 slides. Navigation: the right and left arrows step through the reveals as before. The down and up arrows and the two new buttons of the bar move by one slide. A number followed by Enter opens the slide with that number. The bar has a strip with one mark for each slide; a mark shows the number and headline of its slide under the pointer and opens it on a click. R hides the reveals of the current slide again. A slide reached by any move other than a step forward is shown with all its reveals. check_deck.py reads the slide order from the page and covers the dividers and the new controls. The README describes the keys, the bar, and the ids. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
A session with slides/deck.html gets a Slides page. The site build copies the deck to generated/static, where it is served at /training/session-NN/deck, and the page shows that address in a frame of 1280 by 760 pixels scaled to the page width. Below 560 pixels the frame is unscaled and the deck uses its scrolling layout. The frame passes the colour mode of the site to the deck, and two buttons open the deck in a new tab and in full screen. The secondary action links on training pages now use a dark text colour in both colour modes. In dark mode the text was white on a light grey button, which also affected the Canva link of session 01. Two routes are added to published-routes.txt: /training/session-02/deck and /training/session-02/slides. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Part 5 follows the build of the second occupation model: the three sets of training texts, the request that rewrites a job title as survey answers, the check by a second model, a word-counting model trained in the page on four occupation groups, distillation, five suggested codes for eight responses, the scores by training set, the comparison with three large models, speed, and the cost of the builds. Report tagging is part 6 and has two slides. The deck has seven parts and 50 slides. The slide on the steps used for the report model and the slide on the limits of its result are removed. Source lines and speaker notes state the limits of each result. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Part 4 presents the nine steps as the framework. Its slides use illustrative examples from occupation coding and show no scores of the models: step 6 defines the two scores of the test, and step 7 goes through one round of the loop in four stages. Part 5 applies the steps to the occupation model. The line above each headline names the step. Slide s29 shows the training of the occupation model. Three runs were repeated with the settings of the model trained on the three sets, and each was scored on the 1,000 survey-style test answers before training, at twelve points up to six passes, and after training. The slide plots the mean scores and lists the first code of one run for six test answers. The data are in src/learn.json. The word-counting model that trained in the page is removed. fit_slides.py steps each slider when it sizes a slide. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The distillation slide (s30) follows the office comparison in part 3. The teacher and the student of the slide are the colleague and the new staff member of that comparison. Part 5 holds the slides of the occupation build. The slide keeps its id. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The main part has 45 slides for a session of 60 minutes. Five slides move to an annex after the sources slide: where a model can run (s08), the real cases of a rare group (s12), the request and the check of the rewritten answers (s27, s28), and the cost of building (s35). The slides keep their ids. The source line of s07, s11, s26, and s34 links to the related annex slides, and each annex slide has a button that returns to its slide in the main part. The notes of those four slides carry the main fact of the moved slide. The counter shows the position in the main part, and on an annex slide the position in the annex. check_deck.py covers the links, the counter, and the divider of the annex. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The two report-tagging slides (s36, s37) join the annex, and the main part has six parts and 42 slides. Slide s38 keeps the result of the report-tagging model and links to both annex slides. The process diagram on s05, the part dividers, the speaker notes, the checker, and the README follow the new layout. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
In part 6 the order is now criteria, checks, first attempt, exercise, summary. The notes of the exercise slide name the criteria it uses and allow three of the six tasks when time is short. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
The welcome notes describe one model followed in detail and a second in the annex. The notes of the training texts and of the test responses say that the survey response at the start was coded by the earlier occupation model. The closing criteria slide states that the report-tagging test used 300 real reports. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
…uff ISC004) The lint step added on main flags an implicit string concatenation inside a list. The generated Slides page is unchanged. Signed-off-by: Aivin Solatorio <avsolatorio@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds the material of training session 02, "Building specialized Small AI using Synthetic Data". It contains a slide deck, two notebooks with their recorded data, and support on the training site for a deck kept as one HTML file.
Slide deck
training/session-02-small-ai-synthetic-data/slides/deck.htmlis the whole deck in one HTML file. It has 42 slides in six parts and an annex of 8 slides. The parts cover the tasks, small models, synthetic data, a nine-step process, the build of an occupation-coding model, and the criteria for choosing a small or a large model.build_deck.pybuildsdeck.htmlandoutline.mdfrom the sources inslides/src/.outline.mdlists the headline and the speaker notes of each slide.check_deck.pyopens the built deck in headless Chromium (Playwright) and runs 92 checks: the fit of every slide at two stage sizes, the reading layout at phone width, both themes, the controls, and the interactive figures.slides/README.mddocuments the slide plan, the build, and the checks.Notebooks
01-occupation-coding.ipynbtrains a small model that assigns ISCO-08 unit group codes to survey answers on occupation.02-report-tagging.ipynbtrains a small model that assigns topics from a fixed list to reports.notebooks/build_notebooks.py. They read recorded data fromassets/occupation/andassets/topics/, so they run without a key for a large model.assets/README.mdlists each data file with its content and source. The job titles come fromtabiya/occupation_titles_esco(CC BY 4.0). The ISCO-08 structure is not stored in the repository; notebook 01 downloads it from the ILO website when it runs.Training site
scripts/docs/build_training_site.pyacceptsslides/deck.htmlfor a session. The Slides page shows the deck in a frame (website/src/components/training/DeckFrame), and the deck is served on its own at/training/session-02/deck.website/docusaurus.config.jsaddsgenerated/staticas a static directory when the folder exists.training/AUTHORING.mddescribes the two kinds of deck (Canva link or HTML file).website/published-routes.txtgains five routes: the deck, the Slides page, the two notebook pages, and the worked-example data page.Checks
python3 check_deck.py: 92 checks pass.cd website && npm run build: passes, including the published-routes check.python3 scripts/docs/build_training_site.py --strict: one warning, "the session has no preface.md" for session 02. The site build runs the script without--strict.main, which applied without conflicts.ruff check website/static/cookbook-files scripts/docs: passes.testandbuild-siteworkflows pass on the head commit.Known limits
preface.mdyet.assets/occupation/are synthetic, and no trained occupation coder has reviewed their codes.assets/README.mdstates this.