Repository navigation
Speed up Satterthwaite df with empirical (sandwich) vcov - #584
Conversation
Keep the per-subject contrast_matrix sparse through h_df_1d_sat_empirical() instead of densifying it with as.matrix() before every tcrossprod()/crossprod() call. Only affects method = "Satterthwaite" with a sandwich vcov.
Bump DESCRIPTION to the development version so it matches the 0.3.19.9000 NEWS heading, following the same pattern as openpharma#578. Reword the NEWS entry to use only vocabulary already present in the package: "G-computation" as in the 0.3.18 entry for emmeans_gcomp_vars, and plain wording instead of "densifying". This keeps inst/WORDLIST unchanged. Co-Authored-By: Lilly Code <lillycode@lilly.com>
|
Fixed the two failing checks that came from this branch:
The version bump also cleared the
|
danielinteractive
left a comment
There was a problem hiding this comment.
Thanks @tomonorioura for the PR, please address below comments
- Reassert numeric mode on contrast_matrix for both base matrix (via assert_numeric) and sparse Matrix (via dMatrix class check), restoring the guarantee lost when assert_matrix(mode = "numeric") was replaced by assert_multi_class(). - Split the as.matrix(Matrix::crossprod(...)) line into two for readability. - Update @PARAM contrast_matrix roxygen to reflect that a sparse Matrix is also accepted.
|
Thanks for the review, @danielinteractive. Addressed all three in 6bcb8d9:
Verified |
Closes #583.
contrast_matrixis built withMatrix::.bdiag()as a sparse block-diagonalmatrix, but was densified with
as.matrix()before every use inh_df_1d_sat_empirical(), growing quadratically with the number of subjects.This only affects
method = "Satterthwaite"with a sandwichvcov(
"Empirical","Empirical-Jackknife","Empirical-Bias-Reduced"), which isrequired for g-computation/IMMRM workflows (see #567). For models with many
subjects and coefficients, this dense allocation and multiply dominated the
per-contrast df calculation (25.4s -> 1.0s on a 240-subject, 201-coefficient
model, see #583 for details).
h_df_1d_sat_empirical(): accept sparseMatrixinput, useMatrix::tcrossprod()/crossprod()to keepcontrast_matrixsparse throughthe multiplication; only the final small result is densified
h_df_1d_sat(),h_df_md_sat(): drop theas.matrix()conversion beforecalling the helper, now that it accepts sparse input
empirical_df_matpath (fits from mmrm < 0.3.15) isunaffected;
as.matrix()is kept there sinceh_quad_form_mat()stillexpects a dense matrix
Verified df/SE output is unchanged on
fev_datawith 2-way/3-way interactionmodels combined with
emmeans_gcomp_vars. Full test suite passes (2862/2862).