Fix sparse matrix interactions in `sq.gr.utils`. #891

alam-shahul · 2024-09-23T07:57:24Z

Made to be compatible with new anndata release.

IMPORTANT: Please search among the Pull requests before creating one.

Description

In sq.gr.utils, we change mentions of SparseCSRView and SparseCSCView to SparseCSRMatrixView and SparseCSCMatrixView, respectively. We also update pyproject.toml to use anndata>=0.11.0rc1. I suppose that the latter is still under development, but I feel it will soon be part of the main release.

How has this been tested?

Unfortunately I ran into an error when trying to test this locally:

(squidpy_development) [shahula@lanec1 squidpy]$ tox -e py310-linux -- tests/graph/test_utils.py
.pkg: _optional_hooks> python /work/magroup/shahula/bin/miniconda3/envs/squidpy_development/lib/python3.10/site-packages/pyproject_api/_backend.py True hatchling.build
.pkg: get_requires_for_build_editable> python /work/magroup/shahula/bin/miniconda3/envs/squidpy_development/lib/python3.10/site-packages/pyproject_api/_backend.py True hatchling.build
.pkg: build_editable> python /work/magroup/shahula/bin/miniconda3/envs/squidpy_development/lib/python3.10/site-packages/pyproject_api/_backend.py True hatchling.build
py310-linux: install_package_deps> python -I -m pip install 'Pillow>=8.0.0' 'aiohttp>=3.8.1' 'anndata>=0.11.0rc1' 'cycler>=0.11.0' 'dask-image>=0.5.0' 'dask[array]>=2021.02.0' 'docrep>=0.3.1' 'fsspec>=2021.11.0' 'leidenalg>=0.8.2' 'matplotlib-sc
alebar>=0.8.0' 'matplotlib>=3.3' 'networkx>=2.6.0' 'numba>=0.56.4' 'numpy>=1.23.0' 'omnipath>=1.0.7' 'pandas>=2.1.0' 'scanpy>=1.9.3' 'scikit-image>=0.20' 'scikit-learn>=0.24.0' 'spatialdata>=0.2.0' 'statsmodels>=0.12.0' 'tifffile!=2022.4.22' 'tq
dm>=4.50.2' 'validators>=0.18.2' 'xarray>=0.16.1' 'zarr>=2.6.1'
py310-linux: install_package> python -I -m pip install --force-reinstall --no-deps /work/magroup/shahula/spatiotemporal_transcriptomics_integration/squidpy/.tox/.tmp/package/2/squidpy-1.6.2.dev1+ge6b5b0c.d20240923-py3-none-any.whl
py310-linux: commands[0]> python -m pytest --cov --cov-append --cov-report=xml --cov-config=/work/magroup/shahula/spatiotemporal_transcriptomics_integration/squidpy/tox.ini --ignore docs/ tests/graph/test_utils.py --test-napari
ImportError while loading conftest '/work/magroup/shahula/spatiotemporal_transcriptomics_integration/squidpy/tests/conftest.py'.
tests/conftest.py:20: in <module>
    import spatialdata as sd
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/__init__.py:54: in <module>
    from spatialdata import dataloader, datasets, models, transformations
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/datasets.py:21: in <module>
    from spatialdata._core.operations.aggregate import aggregate
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/_core/operations/aggregate.py:20: in <module>
    from spatialdata._core.operations._utils import _parse_element
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/_core/operations/_utils.py:8: in <module>
    from spatialdata.models import SpatialElement
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/models/__init__.py:18: in <module>
    from spatialdata.models.models import (
.tox/py310-linux/lib/python3.10/site-packages/spatialdata/models/models.py:20: in <module>
    from multiscale_spatial_image import to_multiscale
.tox/py310-linux/lib/python3.10/site-packages/multiscale_spatial_image/__init__.py:14: in <module>
    from .multiscale_spatial_image import MultiscaleSpatialImage
.tox/py310-linux/lib/python3.10/site-packages/multiscale_spatial_image/multiscale_spatial_image.py:7: in <module>
    from zarr.storage import BaseStore
.tox/py310-linux/lib/python3.10/site-packages/zarr/__init__.py:2: in <module>
    from zarr.codecs import *
.tox/py310-linux/lib/python3.10/site-packages/zarr/codecs.py:2: in <module>
    from numcodecs import *
.tox/py310-linux/lib/python3.10/site-packages/numcodecs/__init__.py:25: in <module>
    from numcodecs.version import version as __version__
E   ModuleNotFoundError: No module named 'numcodecs.version'
py310-linux: exit 4 (10.90 seconds) /work/magroup/shahula/spatiotemporal_transcriptomics_integration/squidpy> python -m pytest --cov --cov-append --cov-report=xml --cov-config=/work/magroup/shahula/spatiotemporal_transcriptomics_integration/squidpy/tox.ini --ignore docs/ tests/graph/test_utils.py --test-napari pid=9214
.pkg: _exit> python /work/magroup/shahula/bin/miniconda3/envs/squidpy_development/lib/python3.10/site-packages/pyproject_api/_backend.py True hatchling.build
  py310-linux: FAIL code 4 (79.67=setup[68.77]+cmd[10.90] seconds)
  evaluation failed :( (80.53 seconds)

but this should be a drop-in fix...

Closes

This closes #890 .

Made to be compatible with new `anndata` release.

codecov-commenter · 2024-09-23T08:04:55Z

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 68.68%. Comparing base (4a632d6) to head (70e9bb2).
Report is 50 commits behind head on main.

Files with missing lines	Patch %	Lines
src/squidpy/gr/_utils.py	50.00%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #891      +/-   ##
==========================================
- Coverage   69.99%   68.68%   -1.32%     
==========================================
  Files          39       38       -1     
  Lines        5532     5572      +40     
  Branches     1037     1186     +149     
==========================================
- Hits         3872     3827      -45     
- Misses       1367     1448      +81     
- Partials      293      297       +4

Files with missing lines	Coverage Δ
src/squidpy/gr/_utils.py	`59.65% <50.00%> (-0.70%)`	⬇️

... and 10 files with indirect coverage changes

Also update dependencies to use compatible versions of newest `squidpy` and `anndata`. See `https://github.com/scverse/squidpy/pull/891`.

* Added simulation framework tests. * Add initialization test. * Update `README` to better reflect current dev workflow. Also make some formatting improvements. * Added multigroup test for spatial affinity groups. * Remove deprecated arguments in `load_anndata`. Also reformat `io.py`. * Add linter configurations to `pyproject.toml`. * Fix import name for `test_util.py`. * Add configuration for pre-commit. * Update test data. * Fix analysis tests. Fix analysis functions to match new tests. Fix code style. * Add tests for and refactor `simulation_framework.py`. * Finish `MultiReplicateSyntheticDataset` tests. Refactor accordingly. Add test data/expected outputs. * Fixes to `_canvas.py` and `simulation_framework.py`. * Add documentation for simulations. Fix intiialization tests. * Fix up binning test. Also update dependencies to use compatible versions of newest `squidpy` and `anndata`. See `https://github.com/scverse/squidpy/pull/891`.

src/squidpy/gr/_utils.py

ilan-gold

Without testing this locally, seems reasonable

Please update the pin @alam-shahul

pyproject.toml

ilan-gold · 2024-10-10T07:32:39Z

src/squidpy/gr/_utils.py

+CAN_USE_SPARSE_ARRAY = Version(anndata.__version__) >= Version("0.11.0")
+if not CAN_USE_SPARSE_ARRAY:
+    from anndata._core.views import ArrayView
+    from anndata._core.views import SparseCSCMatrixView as SparseCSCView
+    from anndata._core.views import SparseCSRMatrixView as SparseCSRView
+else:
+    from anndata._core.views import ArrayView, SparseCSCView, SparseCSRView


This is a ping for @timtreis and @giovp that sparse array support is coming: https://docs.scipy.org/doc/scipy/reference/sparse.html

See the note

Remove high `anndata` pin.

…arse-matrix-fix

Sparse matrix fix

ilan-gold · 2024-10-14T08:53:21Z

@timtreis @giovp Can we merge and make a release?

alam-shahul · 2024-10-19T23:04:58Z

Hi, just wondering if the release/PyPI update will be made soon?

Fix sparse matrix interactions in sq.gr.utils.

387795b

Made to be compatible with new `anndata` release.

alam-shahul referenced this pull request in alam-shahul/popari Sep 23, 2024

Fix up binning test.

f2ec316

Also update dependencies to use compatible versions of newest `squidpy` and `anndata`. See `https://github.com/scverse/squidpy/pull/891`.

ilan-gold reviewed Oct 9, 2024

View reviewed changes

src/squidpy/gr/_utils.py Show resolved Hide resolved

Make import depend on anndata version.

dff6eb1

ilan-gold approved these changes Oct 10, 2024

View reviewed changes

alam-shahul added 5 commits October 10, 2024 12:08

Fix logic for anndata import.

24940cc

Remove high `anndata` pin.

Merge branch 'main' into main

72d601b

Make comparison version correct.

0e72ea5

Merge branch 'main' of https://github.com/alam-shahul/squidpy into sp…

b012df3

…arse-matrix-fix

Merge pull request #1 from alam-shahul/sparse-matrix-fix

70e9bb2

Sparse matrix fix

timtreis merged commit 80621fc into scverse:main Oct 14, 2024
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix sparse matrix interactions in `sq.gr.utils`. #891

Fix sparse matrix interactions in `sq.gr.utils`. #891

alam-shahul commented Sep 23, 2024

codecov-commenter commented Sep 23, 2024 •

edited

Loading

ilan-gold left a comment •

edited

Loading

ilan-gold Oct 10, 2024

ilan-gold Oct 10, 2024

ilan-gold commented Oct 14, 2024

alam-shahul commented Oct 19, 2024

Fix sparse matrix interactions in sq.gr.utils. #891

Fix sparse matrix interactions in sq.gr.utils. #891

Conversation

alam-shahul commented Sep 23, 2024

Description

How has this been tested?

Closes

codecov-commenter commented Sep 23, 2024 • edited Loading

Codecov Report

ilan-gold left a comment • edited Loading

Choose a reason for hiding this comment

ilan-gold Oct 10, 2024

Choose a reason for hiding this comment

ilan-gold Oct 10, 2024

Choose a reason for hiding this comment

ilan-gold commented Oct 14, 2024

alam-shahul commented Oct 19, 2024

Fix sparse matrix interactions in `sq.gr.utils`. #891

Fix sparse matrix interactions in `sq.gr.utils`. #891

codecov-commenter commented Sep 23, 2024 •

edited

Loading

ilan-gold left a comment •

edited

Loading