You might be surprised to learn that the term "least squares mean" has no intrinsic or logical meaning in any traditional sense in statistics. Unlike the concepts of “least squares” and “mean”, the phrase “least squares mean” originated from statistical computing (i.e., SAS GLM procedure) rather than statistics. What? Why is that? Let me explain the reason behind this using a two-way (rows-by-columns) analysis of variance (ANOVA) model without interaction.
Let 𝑦ᵢⱼₖ be the 𝑘th observation in the 𝑖th (𝑖 = 1, 2, ..., 𝑎) row and 𝑗th (𝑗 = 1, 2, ..., 𝑏) column, with an expected value
where 𝜇 is the general mean, 𝛼ᵢ is the effect of 𝑖th row, and 𝛽ⱼ is the effect of 𝑗th column.
It’s easy to see that the design matrix of model (1) does not have full column rank, and none of the individual model parameters 𝜇, 𝛼ᵢ or 𝛽ⱼ is estimable (no unique solution to the normal equations for estimating these parameters). As a result, the least squares estimator for these parameters cannot be defined in the usual way. This creates an awkward situation for us, as we need something descriptive and meaningful to represent the effects associated with these parameters.
The good news is that the parameter 𝜇ᵢⱼ = 𝜇 + 𝛼ᵢ + 𝛽ⱼ is always estimable for any 𝑖 and 𝑗, and so are the following two parameters:
and
Parameters (2) and (3) are known as "expected marginal means" for the 𝑖th row effect and 𝑗th column effect, respectively. The LS-means for 𝛼ᵢ and 𝛽ⱼ are then defined as the least squares estimates of the parameters (2) and (3), respectively, in the same manner as described in the SAS online documentation. Simply speaking, LS-means are the least squares estimates of expected marginal means. In other words, LS-means are not the least squares estimates of the individual model parameters, which may or may not be estimable for a given model. Clearly, the term "least squares mean" by itself does not convey its "intended" meaning in the traditional sense.
I have observed that many blog posts discussing LS-means are somewhat misleading. Those authors typically present data (both balanced and unbalanced) in two-way tables (rows-by-columns) without an assumed model and claim that the LS-mean for each row/column effect is simply the row/column (arithmetic) average of the respective cell means. This interpretation is correct only if the model includes the interaction term. In general, however, there is no simple algebraic expression for the LS-means under ANOVA model (1). In this case, calculating LS-means requires solving the normal equations, just as the SAS GLM procedure does.
LS-means are now featured in many SAS procedures beyond GLM, such as MIXED (mixed models) and GENMOD (generalized linear models). However, these extensions are no longer associated with least squares methods.
Commentaires