Initial/beta version of speech recognition plugin. #117
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a PR to begin work on a speech-recognition plugin for jsPsych.
An initial version of the speech recognition plugin has been created. It uses Huggingface's Transfomer.js module to import a speech recognition model based on OpenAI's whisper-tiny. Due to build issues, the import of the transformer.js module and the subsequent inference is done through a direct eval() method. The plugin outputs as trial data the transcript of the audio, and the timestamp of the first utterance of the words in the choices parameter (if recognized). The following is a list of things to do next:
An example file with functionality has been added to the plugin.