T-Tests and ANOVA for Higher-Order Construct Scores: Points Requiring Caution

Exporting latent variable scores and running t-tests or ANOVA turns a latent-variable question into a score-based question. This is convenient but does not automatically propagate score-estimation error into group tests.

HIGHER-ORDER SEM AND GROUP COMPARISONSPrepared by Lê Minh4 minUpdated 16/10/2026

KEY ARGUMENT

Before comparing HOC scores across groups, validate the higher-order structure and establish measurement invariance. If using scores, describe how they were generated and treat the analysis as a two-stage approximation; latent mean comparisons in SEM are preferable when rigorous inference is required.

Start reading

Comparing means: design first, test second

Design
Independent groups or repeated measures?
Measurement
Ensure scores are comparable
Test
Choose for the groups and assumptions
Reporting
Difference, effect size and CI

Conceptual diagram. Statistical significance does not establish practical importance.

Source: Lê Minh · Conceptual illustration, not software output.

1. Two approaches to group comparison

Table 1
Approach	Advantages	Limitations
T-test/ANOVA on scores	Straightforward and intuitive	Treats scores as error-free observations
Latent mean comparison/MGA	Models measurement error	Requires invariance and a more complex specification

2. Establish invariance before comparison

If the same construct has different meanings across groups, score differences may reflect measurement rather than latent levels. CB-SEM usually evaluates configural, metric and scalar invariance; PLS-SEM uses MICOM for appropriate comparison objectives (Putnick & Bornstein, 2016).

Scalar invariance or its relevant counterpart is particularly important for mean comparisons. Checking each group’s alpha is insufficient.

A factor score is not the true latent variable

Scores are estimates that depend on the algorithm, standardization and sample. Call them factor/construct scores and retain the export procedure.

3. Two-stage workflow

Validate the HOC

Assess LOCs, the HOC and their measurement relationships first.

Test invariance

Establish whether the structure and measurements are comparable across groups.

Export scores

Use the same algorithm and score metric for all observations.

Choose the group test

Use Welch t/ANOVA for unequal variances, and paired/repeated approaches for linked data.

Conduct sensitivity analysis

Compare score-based conclusions with MGA or a latent mean model when possible.

*Application screenshot 1. Export latent variable scores from the original SmartPLS example for subsequent analysis. Scores are model-dependent estimates, not error-free measurements.* Source: amosleminh.com · View full size

4. Adjust for multiple testing

Testing multiple LOCs, HOCs and groups rapidly increases the number of tests. Define the primary outcome, address multiplicity or clearly distinguish confirmatory from exploratory analyses.

Report effect sizes and CIs for priority differences; mean plots should display uncertainty rather than bare bars.

Table 2
Layer	Content
Measurement	Valid HOC and invariance
Scores	Generation, standardization and range
Testing	Welch/ANOVA/post hoc tests and multiplicity
Magnitude	Mean difference, CI, effect size
Sensitivity	MGA/latent means where available

*Application screenshot 2. Levene, ANOVA and Welch tables for construct scores in the source example. Group-mean comparisons also require attention to measurement invariance and score uncertainty.* Source: amosleminh.com · View full size

5. Reporting template

Reporting example in English

The higher-order construct [ ] was estimated using [method]; invariance was established at [level]. Construct scores were exported using [ ] and compared between [groups] with [Welch t/ANOVA]. The mean difference was [ ], 95% CI [ ; ], effect size [ ]. Results were treated as [a two-stage approximation/confirmed by a latent mean model].

Assess the HOC before comparing groups.
Test invariance.
Describe score generation.
Use robust tests when required.
Adjust for multiple comparisons.
Do not describe a score as a directly observed latent variable.

Conclusion

Higher-order construct comparisons are trustworthy only when groups share a comparable measure. T-tests and ANOVA on scores can be useful, but should be identified as two-stage analyses.

References

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting. Developmental Review, 41, 71–90.https://doi.org/10.1016/j.dr.2016.06.004

Methodological note: If the primary objective is latent mean comparison, prioritize a multigroup model with invariance rather than exporting scores and ignoring measurement error.