feat: move parsing scripts for support matrix to top-level dir (#188)

This PR moves the scripts that parse the CSV output of pytest to derive the support matrix to the new top-level directory `scripts`. Signed-off-by: Ingo Müller <[email protected]>
substrait-io · Dec 19, 2024 · c3c50f4 · c3c50f4
1 parent ac4367f
commit c3c50f4
Show file tree

Hide file tree

Showing 5 changed files with 15 additions and 10 deletions.
diff --git a/.github/workflows/support_matrix.yml b/.github/workflows/support_matrix.yml
@@ -63,7 +63,7 @@ jobs:
 
     - name: Parse results
       run: |-
-        python substrait_consumer/tests/functional/extension_functions/parse_${{ matrix.target }}r_pytest_output.py \
+        python scripts/parse_${{ matrix.target }}r_pytest_output.py \
             --input ${{ matrix.target }}r_pytest_output.csv \
             --output app/${{ matrix.target }}r_results.csv
 

diff --git a/SUPPORT_MATRIX.md b/SUPPORT_MATRIX.md
@@ -9,14 +9,16 @@ Substrait Producer and Consumer Function Support
 
 ## How to regenerate producer test results
 ```commandline
-cd substrait_consumer/tests/functional/extension_functions
-
 # Run the producer tests to create the pytest result output.
-pytest --tb=no --csv producer_pytest_output.csv --csv-delimiter ';' --csv-columns 'id,status' -m produce_substrait_snapshot
-
+pytest -v --tb=no \
+    -m produce_substrait_snapshot \
+    -k "function/ or ibis_expressions" \
+    substrait_consumer/ \
+    --csv producer_pytest_output.csv --csv-delimiter ';' \
+    --csv-columns 'id,status,properties_as_columns'
 # Parse pytest output.  This creates a new csv file with the parsed results, which
 # the streamlit app uses to generate the table.
-python parse_producer_pytest_output.py producer_results.csv
+python scripts/parse_producer_pytest_output.py producer_results.csv
 ```
 
 
@@ -28,13 +30,16 @@ python parse_producer_pytest_output.py producer_results.csv
 
 ## How to regenerate consumer test results
 ```commandline
-cd substrait_consumer/tests/functional/extension_functions
-
 # Run the consumer tests to create the pytest result output.
-pytest --tb=no --csv consumer_pytest_output.csv --csv-delimiter ';' --csv-columns 'id,status' -m consume_substrait_snapshot
+pytest -v --tb=no \
+    -m consume_substrait_snapshot \
+    -k "function/ or ibis_expressions" \
+    substrait_consumer/ \
+    --csv consumer_pytest_output.csv --csv-delimiter ';' \
+    --csv-columns 'id,status,properties_as_columns'
 
 # Parse pytest output.  This creates a new csv file with the parsed results, which
 # the streamlit app uses to generate the table.
-python parse_consumer_pytest_output.py consumer_results.csv
+python scripts/parse_consumer_pytest_output.py consumer_results.csv
 ```
 
diff --git a/...functions/parse_consumer_pytest_output.py → scripts/parse_consumer_pytest_output.py b/...functions/parse_consumer_pytest_output.py → scripts/parse_consumer_pytest_output.py
diff --git a/...functions/parse_producer_pytest_output.py → scripts/parse_producer_pytest_output.py b/...functions/parse_producer_pytest_output.py → scripts/parse_producer_pytest_output.py
diff --git a/substrait_consumer/tests/functional/extension_functions/__init__.py b/substrait_consumer/tests/functional/extension_functions/__init__.py