Merge pull request #43 from martinvonk/dev

Update main to v0.5.0

.github/workflows/tests.yml

@@ -12,11 +12,6 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - name: Test suite with py39-ubuntu
-            python: "3.9"
-            os: ubuntu-latest
-            toxenv: py39
-            experimental: false
           - name: Test suite with py310-ubuntu
             python: "3.10"
             os: ubuntu-latest
@@ -32,23 +27,22 @@ jobs:
             os: ubuntu-latest
             toxenv: py312
             experimental: false
+          - name: Test suite with py313-ubuntu
+            python: "3.13-dev"
+            toxenv: py313
+            experimental: true
           - name: Type check with mypy
-            python: "3.9"
+            python: "3.10"
             os: ubuntu-latest
             toxenv: type
             experimental: false
-          - name: Formatting with black + isort
-            python: "3.9"
-            os: ubuntu-latest
-            toxenv: format
-            experimental: false
-          - name: Linting with flake8 + ruff
-            python: "3.9"
+          - name: Formatting and linting with ruff
+            python: "3.10"
             os: ubuntu-latest
             toxenv: lint
             experimental: false
           - name: Codacy Coverage Report
-            python: "3.9"
+            python: "3.10"
             os: ubuntu-latest
             toxenv: coverage
             experimental: false

README.md

@@ -10,17 +10,14 @@
 [![Tests](https://img.shields.io/github/actions/workflow/status/martinvonk/spei/tests.yml?style=flat-square)](https://github.com/martinvonk/SPEI/actions/workflows/tests.yml)
 [![CodacyCoverage](https://img.shields.io/codacy/coverage/908b566912314666b84e1add22ea7d66?style=flat-square)](https://app.codacy.com/gh/martinvonk/SPEI/)
 [![CodacyGrade](https://img.shields.io/codacy/grade/908b566912314666b84e1add22ea7d66?style=flat-square)](https://app.codacy.com/gh/martinvonk/SPEI/)
-[![MyPy](https://img.shields.io/badge/type_checker-mypy-2A6DB2?style=flat-square)](https://mypy-lang.org/)
-[![Format: isort](https://img.shields.io/badge/imports-isort-ef8336?style=flat-square)](https://pycqa.github.io/isort/index.html)
-[![Format: Black](https://img.shields.io/badge/code_style-black-black?style=flat-square)](https://github.com/psf/black)
-[![Linter: flake8](https://img.shields.io/badge/linter-flake8-yellowgreen?style=flat-square)](https://flake8.pycqa.org/)
-[![Linter: ruff](https://img.shields.io/badge/linter-ruff-red?style=flat-square)](https://github.com/charliermarsh/ruff)
+[![Typed: MyPy](https://img.shields.io/badge/type_checker-mypy-2A6DB2?style=flat-square)](https://mypy-lang.org/)
+[![Formatter and Linter: ruff](https://img.shields.io/badge/linter-ruff-red?style=flat-square)](https://github.com/charliermarsh/ruff)
 
 SPEI is a simple Python package to calculate drought indices for time series such as the SPI (Standardized Precipitation Index), SPEI (Standardized Precipitation Evaporation Index), and SGI (Standardized Groundwater Index). This package uses popular Python packages such as Pandas and Scipy to make it easy and versatile for the user to calculate the drought indices. Pandas Series are great for dealing with time series; providing interpolation, rolling average, and other manipulation options. SciPy enables us to use all different kinds of [distributions](https://docs.scipy.org/doc/scipy/reference/stats.html#probability-distributions) to fit the data.
 
 For the calculation of potential evaporation, take a look at [pyet](https://github.com/phydrus/pyet). This is another great package that uses pandas Series to calculate different kinds of potential evaporation time series.
 
-Please feel free to contribute or ask questions! 
+Please feel free to contribute or ask questions!
 
 If you happen to use this package, please cite: Vonk, M. A. (2024). SPEI: A simple Python package to calculate and visualize drought indices (vX.X.X). Zenodo. https://doi.org/10.5281/zenodo.10816741.
 
@@ -47,14 +44,14 @@ To get the development version download or clone the GitHub repository to your l
 
 ## Literature
 
-1.  B. Lloyd-Hughes and M.A. Saunders (2002) - A Drought Climatology for Europe. DOI: 10.1002/joc.846
-2.  S.M. Vicente-Serrano, S. Beguería and J.I. López-Moreno (2010) - A Multi-scalar drought index sensitive to global warming: The Standardized Precipitation Evapotranspiration Index. DOI: 10.1175/2009JCLI2909.1
-3.  J.P. Bloomfield and B.P. Marchant, B. P. (2013) - Analysis of groundwater drought building on the standardised precipitation index approach. DOI: 10.5194/hess-17-4769-2013
-4.  A. Babre, A. Kalvāns, Z. Avotniece, I. Retiķe, J. Bikše, K.P.M. Jemeljanova, A. Zelenkevičs and A. Dēliņa (2022) - The use of predefined drought indices for the assessment of groundwater drought episodes in the Baltic States over the period 1989–2018. DOI: 10.1016/j.ejrh.2022.101049
-5.  E. Tijdeman, K. Stahl and L.M. Tallaksen (2020) - Drought characteristics derived based on the Standardized Streamflow Index: A large sample comparison for parametric and nonparametric methods. DOI: 10.1029/2019WR026315
+1. B. Lloyd-Hughes and M.A. Saunders (2002) - A Drought Climatology for Europe. DOI: 10.1002/joc.846
+2. S.M. Vicente-Serrano, S. Beguería and J.I. López-Moreno (2010) - A Multi-scalar drought index sensitive to global warming: The Standardized Precipitation Evapotranspiration Index. DOI: 10.1175/2009JCLI2909.1
+3. J.P. Bloomfield and B.P. Marchant, B. P. (2013) - Analysis of groundwater drought building on the standardised precipitation index approach. DOI: 10.5194/hess-17-4769-2013
+4. A. Babre, A. Kalvāns, Z. Avotniece, I. Retiķe, J. Bikše, K.P.M. Jemeljanova, A. Zelenkevičs and A. Dēliņa (2022) - The use of predefined drought indices for the assessment of groundwater drought episodes in the Baltic States over the period 1989–2018. DOI: 10.1016/j.ejrh.2022.101049
+5. E. Tijdeman, K. Stahl and L.M. Tallaksen (2020) - Drought characteristics derived based on the Standardized Streamflow Index: A large sample comparison for parametric and nonparametric methods. DOI: 10.1029/2019WR026315
 
 Note that the method for calculating the drought indices does not come from these articles and SciPy is used for deriving the distribution. However the literature is helpful as a reference to understand the context and application of drought indices.
 
 ## Alternatives
 
-There are other great packages available to calculate these indices. However, they are either written in R such as [SPEI](https://github.com/sbegueria/SPEI) or don't have the Standardized Groundwater Index such as [climate_indices](https://github.com/monocongo/climate_indices). Additionaly, these packages provide ways to analyse spatial data and calculate potential evaporation. This makes these packages complex, because it is easier to only deal with time series. However, support for spatial data is something on the to-do list so help is appreciated.
+There are other great packages available to calculate these indices. However, they are either written in R such as [SPEI](https://github.com/sbegueria/SPEI) or don't have the Standardized Groundwater Index such as [climate\_indices](https://github.com/monocongo/climate_indices). Additionaly, these packages provide ways to analyse spatial data and calculate potential evaporation. This makes these packages complex, because it is easier to only deal with time series. However, support for spatial data is something on the to-do list so help is appreciated.

doc/examples/example01_indices.ipynb

@@ -37,10 +37,10 @@
     }
    ],
    "source": [
-    "import spei as si  # si for standardized index\n",
+    "import matplotlib.pyplot as plt\n",
     "import pandas as pd\n",
     "import scipy.stats as scs\n",
-    "import matplotlib.pyplot as plt\n",
+    "import spei as si  # si for standardized index\n",
     "\n",
     "print(si.show_versions())"
    ]

doc/examples/example02_distributions.ipynb

@@ -35,10 +35,11 @@
    ],
    "source": [
     "from calendar import month_name\n",
-    "import spei as si  # si for standardized index\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
     "import pandas as pd\n",
     "import scipy.stats as scs\n",
-    "import matplotlib.pyplot as plt\n",
+    "import spei as si  # si for standardized index\n",
     "\n",
     "print(si.show_versions())"
    ]

doc/examples/example03_drought_NL.ipynb

@@ -43,13 +43,13 @@
    ],
    "source": [
     "import datetime\n",
-    "import spei as si  # si for standardized index\n",
-    "import pandas as pd\n",
-    "import pastas as ps\n",
     "\n",
     "import hydropandas as hpd\n",
-    "import scipy.stats as scs\n",
     "import matplotlib.pyplot as plt\n",
+    "import pandas as pd\n",
+    "import pastas as ps\n",
+    "import scipy.stats as scs\n",
+    "import spei as si  # si for standardized index\n",
     "\n",
     "print(si.show_versions())"
    ]

pyproject.toml

@@ -13,10 +13,10 @@ requires-python = ">=3.9"
 dependencies = ["numpy", "scipy", "matplotlib", "pandas"]
 classifiers = [
         'Programming Language :: Python :: 3 :: Only',
-        'Programming Language :: Python :: 3.9',
         'Programming Language :: Python :: 3.10',
         'Programming Language :: Python :: 3.11',
         'Programming Language :: Python :: 3.12',
+        'Programming Language :: Python :: 3.13',
         'Topic :: Scientific/Engineering :: Hydrology',
         'Intended Audience :: Science/Research',
         "License :: OSI Approved :: MIT License",
@@ -29,22 +29,15 @@ homepage = "https://github.com/martinvonk/spei"
 repository = "https://github.com/martinvonk/spei"
 
 [project.optional-dependencies]
-linting = ["flake8", "ruff"]
-formatting = ["black[jupyter]", "isort"]
+ruffing = ["ruff"]
 typing = ["mypy", "pandas-stubs"]
 pytesting = ["pytest>=7", "pytest-cov", "pytest-sugar"]
 coveraging = ["coverage"]
-dev = ["spei[linting,formatting,typing,pytesting,coveraging]"]
+dev = ["spei[ruffing,typing,pytesting,coveraging]"]
 
 [tool.setuptools.dynamic]
 version = { attr = "spei._version.__version__" }
 
-[tool.black]
-line-length = 88
-
-[tool.isort]
-profile = "black"
-
 [tool.mypy]
 mypy_path = "src"
 
@@ -56,42 +49,44 @@ ignore_missing_imports = true
 pythonpath = ["src"]
 
 [tool.ruff]
-line-length = 88
+extend-include = ["*.ipynb"]
+lint.extend-select = ["I"]
+show-fixes = true
+fix = true
 
 [tool.tox]
 legacy_tox_ini = """
     [tox]
     requires = tox>=4
-    env_list = format, type, lint, py39, py310, py311, py312
+    env_list = format, type, lint, py310, py311, py312, py313
 
     [testenv]
     description = run unit tests
     extras = pytesting
     commands =
             pytest tests
 
-    [testenv:format]
-    description = run formatters
-    basepython = python3.9
-    extras = formatting
-    commands =
-            black src --check --verbose
-            isort src --check
-
     [testenv:type]
     description = run type checks
     basepython = python3.9
     extras = typing
     commands =
             mypy src
 
-    [testenv:lint]
-    description = run linters
-    basepython = python3.9
-    extras = linting
+    [testenv:ruff]
+    description = run ruff checks
+    basepython = python3.10
+    extras = ruffing
+    commands =
+            ruff check --extend-select I --preview
+            ruff format --check
+
+    [testenv:ruff_fix]
+    description = run ruff locally and fix issues
+    extras = ruffing
     commands =
-            flake8 src --max-line-length=88 --ignore=E203,W503,W504
-            ruff check src
+            ruff check --extend-select I --fix
+            ruff format
 
     [testenv:coverage]
     description = get coverage report xml

src/spei/_version.py

@@ -1,7 +1,7 @@
 from importlib import metadata
 from platform import python_version
 
-__version__ = "0.4.2"
+__version__ = "0.5.0"
 
 
 def show_versions() -> str:

src/spei/plot.py

@@ -55,7 +55,7 @@ def si(
         colormap = plt.get_cmap(cmap)
 
     if background:
-        ax.plot(si.index, si.values, linewidth=0.8, color="k")
+        ax.plot(si.index, si.values.astype(float), linewidth=0.8, color="k")
         ax.axhline(0, linestyle="--", linewidth=0.5, color="k")
 
         droughts = si.to_numpy(dtype=float, copy=True)

src/spei/utils.py

@@ -3,9 +3,18 @@
 from typing import Union
 
 from numpy import array, nan
-from pandas import DataFrame, DatetimeIndex, Grouper, Index, Series, Timedelta
+from pandas import (
+    DataFrame,
+    DatetimeIndex,
+    Grouper,
+    Index,
+    Series,
+    Timedelta,
+    concat,
+    infer_freq,
+    to_datetime,
+)
 from pandas import __version__ as pd_version
-from pandas import concat, infer_freq, to_datetime
 
 
 def validate_series(series: Series) -> Series:

tests/conftest.py

@@ -1,6 +1,5 @@
 import pytest
 from pandas import Series, read_csv
-
 from spei.si import spi
 
 

tests/test_climdex.py

@@ -1,5 +1,4 @@
 from pandas import Series
-
 from spei import climdex
 
 

tests/test_plots.py

@@ -1,6 +1,5 @@
 import matplotlib as mpl
 from pandas import Series
-
 from spei.plot import monthly_density
 from spei.plot import si as plot_si
 

tests/test_si.py

@@ -1,7 +1,7 @@
-from pandas import Series, Timestamp
+from pandas import DataFrame, Series, Timestamp
 from scipy.stats import norm
-
 from spei import SI, sgi, spei, spi, ssfi
+from spei.dist import Dist
 
 
 def test_spi(prec: Series) -> None:
@@ -33,3 +33,75 @@ def test_SI(prec: Series) -> None:
     si.pdf()
     dist = si.get_dist(Timestamp("2010-01-01"))
     dist.ks_test()
+
+
+def test_SI_post_init_timescale(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=30, fit_freq="ME")
+    assert si.series.equals(
+        prec.rolling(30, min_periods=30).sum().dropna()
+    ), "Timescale rolling sum not applied correctly"
+
+
+def test_SI_post_init_fit_freq_infer(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=0)
+    assert si.fit_freq is not None, "Frequency inference failed"
+
+
+def test_SI_post_init_grouped_year(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=0, fit_freq="ME")
+    assert isinstance(si._grouped_year, DataFrame), "Grouped year DataFrame not created"
+
+
+def test_SI_post_init_fit_window_adjustment(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=0, fit_freq="D", fit_window=2)
+    assert si.fit_window == 3, "Fit window not adjusted to odd number"
+
+
+def test_SI_post_init_fit_window_minimum(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=0, fit_freq="D", fit_window=1)
+    assert si.fit_window == 3, "Fit window not adjusted to minimum value"
+
+
+def test_fit_distribution_normal_scores_transform(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=30, fit_freq="ME", normal_scores_transform=True)
+    si.fit_distribution()
+    assert (
+        not si._dist_dict
+    ), "Distribution dictionary should be empty when using normal scores transform"
+
+
+def test_fit_distribution_with_fit_window(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=30, fit_freq="D", fit_window=5)
+    si.fit_distribution()
+    assert (
+        si._dist_dict
+    ), "Distribution dictionary should not be empty when using fit window"
+    for dist in si._dist_dict.values():
+        assert isinstance(
+            dist, Dist
+        ), "Items in distribution dictionary should be of type Dist"
+
+
+def test_fit_distribution_with_fit_freq(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=30, fit_freq="M")
+    si.fit_distribution()
+    assert (
+        si._dist_dict
+    ), "Distribution dictionary should not be empty when using fit frequency"
+    for dist in si._dist_dict.values():
+        assert isinstance(
+            dist, Dist
+        ), "Items in distribution dictionary should be of type Dist"
+
+
+def test_fit_distribution_invalid_fit_freq_with_window(prec: Series) -> None:
+    si = SI(prec, dist=norm, timescale=30, fit_freq="M", fit_window=5)
+    try:
+        si.fit_distribution()
+    except ValueError as e:
+        assert (
+            str(e)
+            == "Frequency fit_freq must be 'D' or 'W', not 'M', if a fit_window is provided."
+        )
+    else:
+        assert False, "ValueError not raised for invalid fit frequency with fit window"