Allow standardizing a DenseMatrix by shifting the data - #539
Closed
abelianbee wants to merge 1 commit into
Closed
abelianbee wants to merge 1 commit into
abelianbee wants to merge 1 commit into
Conversation
Expanding a centered second moment from uncentered sums loses the significant digits when a column's mean is large relative to its standard deviation. Add an opt-in materialize_shift flag that folds the shift and multiplier into a copy of the data instead, so sandwich never forms outer(shift, shift) * sum(d). Sparse and categorical matrices ignore the flag and are never densified. unstandardize() stays exact via a reference to the pre-standardization matrix.
abelianbee
requested review from
Marc-Antoine Schmidt (MarcAntoineSchmidtQC),
Jan Tilly (jtilly) and
Martin Stancsics (stanmart)
as code owners
September 11, 2026 08:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #414.
standardizekeepsshift = -mean/stdandmult = 1/stdon theStandardizedMatrix, sosandwichexpands the product into four terms, one of themouter(shift, shift) * sum(d). That is a centered second moment computed from uncentered sums, so when a column's mean is large relative to its standard deviation those terms are orders of magnitude larger than their sum and the significant digits cancel away. No regrouping of the algebra avoids it. Centering the data does.This adds an opt-in
materialize_shiftflag tostandardize. When set, the shift and multiplier are folded into a copy of the data and the returned matrix carries a zero shift, so the expansion never happens. The default is unchanged._materialize_standardizationreturnsNoneby default, soSparseMatrixandCategoricalMatrixignore the flag and are never densified.StandardizedMatrixholds a reference to the pre-standardization matrix, sounstandardize()stays exact and free, whichglumrelies on in_glm.py.Measured against a
longdoubleoracle, same matrix both ways, on an M1 Max:Better on both axes, which is worth saying explicitly because the obvious objection is that you pay for a copy. You don't: the expansion does three p x p outer products and an extra
transpose_matvec, where the materialized path is one BLAS-3 call on contiguous data.Tests cover representation equivalence across all seven matrix fixtures with both flag values, operation equivalence for
sandwich/matvec/transpose_matvec, and an accuracy regression against alongdoublereference. 5666 passed, 45 skipped, 4 xpassed.I moved the unreleased changelog section from 4.2.2 to 4.3.0, since this adds functionality. Happy to put it back if you would rather keep the number.
Checklist
CHANGELOG.rstentry