Repository navigation
Conversation
…ethod p-value combiner - ewstools/spatial.py: morans_i(), morans_i_permutation_test(), SpatialEWS class following the existing MultiTimeSeries conventions. - ewstools/pvalues.py: combine_pvalues_ebm(), a real implementation of Poole et al. (2016)'s Empirical Brown's Method. - 21 new tests, 0 regressions on the existing 32. - CONTRIBUTION_spatial_significance.md documents the identified gap. Local branch, not yet submitted as a pull request.
…liques - ewstools : pull request réellement soumise (ThomasMBury/ewstools#482), statut passé de "développé, pas encore soumis" à "soumis (pull request ouverte)". - hopfieldkit : dépôt GitHub public créé (https://github.com/C95234/hopfieldkit), lien ajouté à la page. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Hi @C95234 — thanks for this, and apologies that it sat quiet for a few days. That's on me. I've read it properly now. You did the things that make a contribution easy to take seriously: you checked whether the gap was real before writing anything, the tests are substantive rather than decorative, you matched the conventions already in the package, and the design follows Dakos et al. (2010) closely. Moran's coefficient at distance 1, plus Kendall tau against the control parameter, is exactly that protocol. I had a look at Hélios as well. Pushing early-warning methods out into domains they weren't developed on is a particular interest of mine, too, so the cross-domain framing is one I actively care about. There are a few things I'd want to work through with you — a couple of them scientific rather than cosmetic — but none of them change my interest in the direction you're taking. I'm going through it carefully and talking it over with Tom, and I'll come back early next week with something specific. Thanks for taking the trouble to contribute this. — Bruce |
|
Hi @energyscholar — happy to help however's useful here. |
|
Sorry this is later than "early next week" — I wanted to check the implementation properly rather What holds upI went through this carefully, and the core of it is right.
I also ran a mutation check on The one thing I'd want addressed before merge: there is no trend controlThis is a library gap rather than a mistake in your code, and it is worth stating carefully because Moran's I is computed on the raw field at each time point. Moran's I is spatially mean-centred, so a
Dakos et al. control this by construction — they know what their model is doing. A library The consequence for the PR is that The good news is that the package already has the operation.
I'll add that as a documented recipe beside your PR, together with a shipped null and positive Two honest qualifications, both of which I'll put in the docstring rather than leave implied:
And one interpretation point, which is why the docstring I'm proposing reads conditionally rather A p-value that can be exactly zeroIn One line fixes it: "p_value": float((1 + np.sum(null_values >= observed)) / (1 + n_permutations)),That floors it at One note, not a request
|
…ues.py Applying Bruce Stephenson's (energyscholar) review of this PR, in full: - Permutation p-value floored at 1/(n_permutations + 1) (Davison & Hinkley 1997; North, Curtis & Sham 2002) so it can never land on exactly 0. - Renamed test_spatialews_compute_ktau_detects_increasing_trend -> test_spatialews_ktau_rises_under_strengthening_gradient_without_trend_control and marked it explicitly as a NEGATIVE CONTROL (it demonstrates a confound, not a real detection) -- while at it, also moved its fixture from 3x3/T=30 (passes only 526/1000 seeds -- essentially a coin flip) to the 6x6/T=60/>0.30 configuration Bruce validated at 1000/1000 seeds, verified independently here on another 500 seeds before committing. - Added a documented trend-control recipe and a weights-parameter caveat to the SpatialEWS docstring. - ewstools/spatial.py now exported from ewstools/__init__.py, and added to docs/source/ewstools.rst so it appears in the built docs. - Added the NaN-warning behaviour (and its test) to both compute_moran and compute_moran_significance. - Dropped ewstools/pvalues.py, tests/test_pvalues.py and CONTRIBUTION_spatial_significance.md per Bruce's scope call: combining significance across indicators belongs outside this package (CONTRIBUTING draws the line at indicators). It also had a real bug (anti-conservative under negative correlation) and a naming error (it's Kost & McDermott's method, not Brown's) -- both fixed in a corrected copy kept outside this repository. Full suite: 46 passed, 1 skipped (0 regressions on the 32 that existed before this PR). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Thanks for this — genuinely one of the most careful reviews I've had on anything, and the mutation testing especially is not something I'd have thought to do myself. Applied the checklist in the latest commit:
Full suite: 46 passed, 1 skipped, 0 regressions on the 32 that existed before this PR. Everything above I did myself rather than leaving it to you — happy to take a pass at any of the "follow-ups, not merge conditions" list too if useful, just say which. 🤖 Generated with Claude Code |
…eference-period note Not merge conditions per Bruce's review, but small enough to include now rather than leave for a separate pass: - lattice_weights(n_rows, n_cols=None, connectivity="rook"|"queen"): a convenience for the common regular-grid case, with a pointer to libpysal.weights for irregular real-world networks -- the split Bruce suggested (an arbitrary weights matrix IS the irregular-network generalisation; what was missing was just a grid builder). - esda.moran.Moran interop pointer in the SpatialEWS docstring, for users who want analytic single-snapshot inference rather than a trend across time. - Reference-period note: Moran's I depends on the unit set and weights as much as on the field, so a single value needs a reference period to be meaningful. 4 new tests for lattice_weights. Full suite: 50 passed, 1 skipped. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Also went ahead and added the three follow-ups from your "not merge conditions" list, since they were small enough to fold in now rather than leave for later:
Full suite: 50 passed, 1 skipped. 🤖 Generated with Claude Code |
energyscholar
left a comment
There was a problem hiding this comment.
Checked the new head properly rather than on trust: 50 passed / 1 skipped locally, the
trend-control recipe runs verbatim (tau +0.68 -> -0.07 on the strengthening gradient), and the
new fixture holds on 1000/1000 seeds. Thank you for all of it, the fixture change and the
follow-ups included. Approving.
Three small things I found, none blocking. I'll carry them in my follow-up PR, so there is
nothing more for you to do here:
- The p-value floor has no test (my checklist's omission, not yours).
- esda's Moran row-standardises by default, so it gives a different I from morans_i on the
same W; the pointer will say transformation='b'. - lattice_weights(-2) quietly returns a 4x4 zero matrix; the follow-up rejects it.
Next from me: the shipped null/positive control and the reference-period baseline, built on
your lattice_weights.
ThomasMBury
left a comment
There was a problem hiding this comment.
Thanks for this contribution! Spatial early warning signals are a natural addition to ewstools. SpatialEWS follows the MultiTimeSeries conventions nicely, so it will feel familiar to existing users. I also appreciated the trend-control notes and recipe in the class docstring. That's practical guidance people will need. The tests are thorough too: hand-computed examples, limiting cases and the negative control are exactly what I'd want for a new indicator.
I ran the suite locally (50 passed, 1 skipped) and then tried some edge cases. A few of them produce incorrect results, so I've marked this "request changes". Each fix is small, and I've left suggestions inline. A regression test for each would be great.
Fixes needed
-
A constant field returns I = 1.0 instead of NaN. The check
denominator == 0is exact. When all values are equal, the computed mean is often off by one rounding step, so every deviation is the same tiny nonzero number and I comes out as exactly 1. A series that settles to a constant value (0.23) gives[-0.10, -0.12, -0.18, 1.0, 1.0, 1.0], and Kendall tau is +0.45, which looks like an early warning. Across constants from 0.01 to 1.99, this happens for 29 of them on a 3×3 grid and 141 on a 10×10 grid. The existing test uses 3.0, which is exactly representable, so it doesn't catch this. -
A transition at time 0 is ignored. The checks
if transitionandif self.transitiontreat 0 as no transition. On an index from −5 to 5 withtransition=0, Moran's I is computed all the way to 5. Separately,float(transition)raisesTypeErrorfor apd.Timestamp, so datetime-indexed data can't use a transition. -
Every time point gets the same permutations.
compute_moran_significancepasses the same integerseedto each row, and each row creates its owndefault_rng(seed). I confirmed that every row's generator starts in the identical state. As a result, random errors in the p-values are shared across time and can move together. -
Tied permutations are undercounted in the p-value.
null_values >= observedcompares floats exactly. Permutations that give exactly the same I can come out one rounding step lower and are then not counted. This is common with discrete fields, such as presence/absence data. On 20 random binary fields on a 10×10 grid with 500 permutations each, 76 tied permutations were missed. I checked all 76 with exact rational arithmetic: every one is a true tie. The p-value was understated by up to 0.02. -
Weights are only checked for shape. The docstring says weights need a zero diagonal and finite values, but this isn't enforced. Self-weights (
W + np.eye(n)) move I from −0.11 to +0.19 with no warning. A NaN weight makes every time point NaN, and the warning then blames missing units.
Minor
- The NaN warning in
compute_moran_significanceis copied fromcompute_moran, so it talks about Moran's I instead of the p-values. - The module docstring explains why the module belongs in ewstools. That's helpful context for this PR, but it will read oddly once the module is merged. I've suggested a shorter version inline.
- Could you add the MIT license and author header that the other modules (e.g.
core.py) start with?
Tutorial (optional)
This isn't required for the PR, but a tutorial notebook for the new functionality would be a great addition, like the others in tutorials/. That's where most users learn the package, and the trend-control recipe in your docstring would make a nice worked example. It could come in this PR or a follow-up, whichever is easier for you. Following the existing notebooks, tutorials/tutorial_spatial.ipynb could:
- open with "Tutorial 5: Spatial early warning signals" and a "By the end of this tutorial you should know how to:" list
- simulate a spatial system approaching a transition. A grid of diffusively coupled units with a slowly changing parameter would work, along the lines of the vegetation model in Dakos et al. (2010).
- build weights with
lattice_weights, compute Moran's I and its p-values withSpatialEWS, and measure the trend with Kendall tau - show the trend-control step (detrending with
MultiTimeSeriesbeforeSpatialEWS) and plot the field and the indicator over time - set a random seed and run in a reasonable time, as the other notebooks do
If you add one, please also list it in tutorials/readme.md and the main README.md.
Thanks again! This will be a valuable addition, and I'm happy to help if anything here is unclear.
| """ | ||
| Spatial early warning signals. | ||
|
|
||
| Most of `ewstools` treats early warning signals (EWS) as a purely | ||
| temporal phenomenon: a rolling window slides along a single time series | ||
| (`TimeSeries`) or a handful of co-measured series (`MultiTimeSeries`), | ||
| and indicators like variance or lag-1 autocorrelation are tracked over | ||
| time. But a large and separate branch of the critical-slowing-down | ||
| literature -- starting with Dakos et al. (2010), "Spatial correlation as | ||
| leading indicator of catastrophic shifts", and still an active area (see | ||
| e.g. MacLaren, Aihara & Masuda, 2025, on generalising spatial EWS from | ||
| regular lattices to irregular real-world networks) -- looks instead | ||
| across SPACE at a single instant: do neighbouring units of a system | ||
| become more correlated with each other as a critical transition | ||
| approaches? Moran's I is the standard statistic for this, but it is not | ||
| implemented anywhere in `ewstools` today. | ||
|
|
||
| This module adds that missing piece, following the existing package's | ||
| conventions (`ewstools.core.MultiTimeSeries`): a spatial analogue that | ||
| takes one column per spatial unit and computes a spatial indicator at | ||
| EVERY time point (cross-sectional at each row), rather than a temporal | ||
| indicator over a rolling window. | ||
| """ |
There was a problem hiding this comment.
Suggest trimming this to what the module does. The motivation fits well in the PR description.
| """ | |
| Spatial early warning signals. | |
| Most of `ewstools` treats early warning signals (EWS) as a purely | |
| temporal phenomenon: a rolling window slides along a single time series | |
| (`TimeSeries`) or a handful of co-measured series (`MultiTimeSeries`), | |
| and indicators like variance or lag-1 autocorrelation are tracked over | |
| time. But a large and separate branch of the critical-slowing-down | |
| literature -- starting with Dakos et al. (2010), "Spatial correlation as | |
| leading indicator of catastrophic shifts", and still an active area (see | |
| e.g. MacLaren, Aihara & Masuda, 2025, on generalising spatial EWS from | |
| regular lattices to irregular real-world networks) -- looks instead | |
| across SPACE at a single instant: do neighbouring units of a system | |
| become more correlated with each other as a critical transition | |
| approaches? Moran's I is the standard statistic for this, but it is not | |
| implemented anywhere in `ewstools` today. | |
| This module adds that missing piece, following the existing package's | |
| conventions (`ewstools.core.MultiTimeSeries`): a spatial analogue that | |
| takes one column per spatial unit and computes a spatial indicator at | |
| EVERY time point (cross-sectional at each row), rather than a temporal | |
| indicator over a rolling window. | |
| """ | |
| """ | |
| Spatial early warning signals. | |
| Computes spatial indicators of critical slowing down (Dakos et al., 2010) | |
| from a field observed over time. Input is a DataFrame with one column per | |
| spatial unit and one row per time point. Indicators are computed | |
| cross-sectionally at each time point, then tracked for a trend in the | |
| same way as the temporal indicators in `ewstools.core`. | |
| References | |
| ---------- | |
| Dakos, V., van Nes, E. H., Donangelo, R., Fort, H., & Scheffer, M. (2010). | |
| Spatial correlation as leading indicator of catastrophic shifts. | |
| Theoretical Ecology, 3(3), 163–174. | |
| """ |
| x = np.asarray(values, dtype=float) | ||
| w = np.asarray(weights, dtype=float) | ||
| n = len(x) | ||
| if w.shape != (n, n): |
There was a problem hiding this comment.
Could this also enforce the documented requirements of a zero diagonal and finite values? Self-weights currently inflate I silently: adding np.eye(9) to a 3×3 rook matrix moved I from −0.11 to +0.19. The same check should run in SpatialEWS.__init__.
| return float("nan") | ||
|
|
||
| denominator = (deviations**2).sum() | ||
| if denominator == 0: |
There was a problem hiding this comment.
An exact zero check misses constant fields. np.full(9, 0.23) returns I = 1.0 because the mean is off by one rounding step and every deviation is the same tiny number. A relative tolerance handles it. I checked this version against all constants from 0.01 to 1.99 on 3×3 and 10×10 grids:
if denominator <= np.finfo(float).eps * n * np.max(x**2):
return float("nan")It would be good to extend test_morans_i_constant_values_is_nan with a value that isn't exactly representable, such as 0.1 or 0.23.
|
|
||
| return { | ||
| "observed_i": observed, | ||
| "p_value": float((1 + np.sum(null_values >= observed)) / (1 + n_permutations)), |
There was a problem hiding this comment.
Exact >= drops permutations whose I equals the observed value but comes out one rounding step lower. These are common with binary or integer fields. A small tolerance fixes it, e.g. null_values >= observed - 1e-12 * max(1.0, abs(observed)).
|
|
||
| self.state = data | ||
| self.weights = weights | ||
| self.transition = float(transition) if transition else transition |
There was a problem hiding this comment.
if transition treats 0 as "no transition", and float() rejects timestamps. Suggest:
| self.transition = float(transition) if transition else transition | |
| self.transition = transition |
and then if self.transition is not None: in _pre_transition. A test with transition=0 on an index that spans negative and positive values would cover it.
| """ | ||
| df_pre = self._pre_transition() | ||
| values = df_pre.apply( | ||
| lambda row: morans_i_permutation_test(row.to_numpy(), self.weights, n_permutations, seed)["p_value"], |
There was a problem hiding this comment.
Passing the integer seed into every row gives every time point the same permutations. Creating one generator up front, rng = np.random.default_rng(seed), and passing rng through fixes this. default_rng returns a Generator unchanged when given one, so morans_i_permutation_test needs no changes.
| n_nan = int(values.isna().sum()) | ||
| if n_nan: | ||
| warnings.warn( | ||
| f"Moran's I is nan for {n_nan} of {len(df_pre)} time points. A time " |
There was a problem hiding this comment.
Small one: this warning says "Moran's I is nan", but here it's the p-values that are missing. Factoring the warning into a helper that takes the indicator name would keep both versions in sync.
…row permutations, ties, weight validation - morans_i: a constant field now returns nan within floating-point resolution (ptp <= 8*eps*max|x|). The previous exact `denominator == 0` returned I = 1.0 for e.g. np.full(9, 0.23). Tolerance is relative to the field's magnitude, so genuinely varying fields (even at 1e-12 relative variation) are still computed. - SpatialEWS: `transition` is stored as given and tested with `is not None`, so transition=0 and pandas Timestamps work. - compute_moran_significance: one generator shared across time points (previously every row re-created default_rng(seed) and got identical permutations). - morans_i_permutation_test: ties within 1e-12 (relative) of the observed I count as "at least as large"; verified against exact rational arithmetic. - Weights validated (shape, finite, zero diagonal) in morans_i, the permutation test and SpatialEWS.__init__. - NaN warning factored into one helper that names the indicator (Moran's I vs its p-value); module docstring trimmed; MIT/author header added. 43 tests in tests/test_spatial.py (19 failed on the previous code), full suite 74 passed / 1 skipped. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Thanks for this — a thorough review, and I appreciate you running the edge cases rather than stopping at the test suite. All eight points are addressed in the latest commit. Each of the five fixes has a regression test that fails on the previous code and passes now (19 of the new tests failed before). Full suite: 74 passed, 1 skipped (Python 3.12, numpy 1.26, pandas 3.0 — I couldn't see CI results on the branch).
Minor points: the NaN warning is one helper that names the indicator (Moran's I vs its p-value), the module docstring is your suggestion, and the MIT/author header is copied from I'd be glad to add |
|
One more thing, outside the scope of this PR, in case it's useful. The
So on a series indexed from, say, −5 to 5 with If you'd like, I can open a separate small PR for this with a failing-first test for each class. Otherwise I'm happy to leave it with you, and I didn't want to widen the scope of #482 without asking. |
|
Good catch on |
|
@ThomasMBury, I checked the latest commit against your review. All 8 points are fixed, and the tests pass on Python 3.9–3.14. One change differs from your suggestion: the constant-field check uses I also ran it end to end on a simulated 6×6 lattice approaching a transition. Moran's I trends up on all 20 runs, stays flat on a no-transition control, and the docstring's trend-control recipe works as written. Three quick decisions, each with my suggestion:
If you agree, an approval saying "agree" is all that's needed. |
Summary
A spatial early-warning-signal module, developed while building Hélios, a research/outreach project that replicates early-warning-signal methods on new domains and found itself needing this while doing so:
ewstools/spatial.py:morans_i(),morans_i_permutation_test(), and aSpatialEWSclass following the existingMultiTimeSeriesconventions (data->state->ews, transition-aware).ewstoolscovers the temporal branch of the critical-slowing-down literature thoroughly but had no equivalent for the spatial branch (Dakos et al., 2010) — confirmed absent by direct inspection ofcore.py/helpers.pybefore writing any code.(An earlier version of this PR also included
ewstools/pvalues.py, a correlated-p-value combiner. Removed per review: out of scope for this package, and it had its own bugs — see the review thread below for the full story.)Testing
tests/test_spatial.py: hand-computed Moran's I examples (verified independently againstesda.moran.Moran), known limiting cases (checkerboard, smooth gradient, constant field), the classical permutation-test propertyE[I] -> -1/(N-1), a p-value floor test, a NaN-handling warning test, and a negative control confirming that a strengthening spatial gradient raises Moran's I with no actual critical slowing down (see theSpatialEWSdocstring's trend-control recipe).CHANGELOG.mdupdated to match.Happy to adjust API shape, naming, or scope based on further maintainer feedback.
🤖 Generated with Claude Code