Does a Model That Explains Well Necessarily Predict Well?

1. Two Different Questions: Explanation and Prediction

R² asks how much variation in the dependent variable the model explains within the collected dataset. It is an in-sample measure, calculated on the same data used to estimate path coefficients. Shmueli and Koppius (2011) distinguish this explanatory objective from the predictive objective of accurately predicting observations the model has not previously seen.

The issue is not that one objective matters more. They are assessed through different procedures on different data, so no theoretical basis guarantees that a model scoring highly on one will automatically score highly on the other. A model can fit observed data very well, partly because its coefficients were estimated for that purpose, without predicting better than a simple benchmark on new data.

2. Hypothetical Illustration: High R² May Not Pass CVPAT

Consider the following hypothetical situation. These figures are not drawn from a specific empirical study.

A hypothetical illustration, not empirical study data. The R² values and CVPAT outcomes are shown together to demonstrate that they can vary independently.

Judged by R² alone, Model A appears better because it explains more variation. But if the question is which model predicts better than a specific benchmark, the relevant evidence comes from CVPAT. In this hypothetical situation, only Model B has that evidence. Reporting R² alone would miss this important distinction.

For an empirical comparison, Jumani's (2026) study of Islamic brand consumers in Thailand reported R² = 0,706 for consumption behavior in an extended TPB model, alongside statistically significant CVPAT loss differences (p < 0,001) against both benchmarks. Both axes are high in this case. That is one possible combination rather than a general rule, as the hypothetical example illustrates.

3. Where CVPAT Originated

CVPAT, the Cross-Validated Predictive Ability Test, was not first introduced in the technical guide by Sharma et al. (2022), as many citations suggest. Liengaard et al. (2021) first introduced the test in Decision Sciences, in an article titled "Prediction: Coveted, Yet Forsaken?" The title captures the issue: predictive ability is widely desired, but for many years PLS-SEM practice largely neglected it because a suitable statistical testing tool was unavailable.

Sharma et al. (2022), published in European Journal of Marketing, extended CVPAT rather than introducing it. That article provides more detailed practical guidance, including integration with the existing PLSpredict procedure. Distinguishing the articles helps authors cite the theoretical origin of the test as well as its practical guidance.

4. Why Two Benchmarks Are Needed: IA and LM

CVPAT compares the PLS-SEM model with two benchmarks serving different roles. Indicator Average (IA) is the simplest imaginable benchmark, predicting each target indicator using that same indicator’s mean in the training sample without a statistical model. Linear Model (LM) is a more competitive benchmark, using linear regression on the same antecedent variables.

A model that outperforms IA but not LM can still have positive Q²predict in PLSpredict because Q²predict, by its original definition, compares only against IA. However, if CVPAT finds no statistically significant difference from LM, claims of predictive superiority need greater care: outperforming a weak benchmark does not establish superiority over a stronger one.

5. The Gap between Recommendations and Published Practice

Evidence suggests a substantial gap remains between methodological recommendations and actual reporting practice. Zhang et al. (2026), in European Journal of Education, systematically reviewed 62 articles applying PLS-SEM in educational leadership and management, following PRISMA. They examined reasons for selecting PLS-SEM, data and model characteristics, advanced analyses, and reporting. They identified several areas for improvement in advanced assessment tools, including predictive ability assessment.

This finding aligns with a general observation in PLS-SEM research: although PLSpredict has been available since 2019 and CVPAT since 2021, most applied articles still focus on R², f², and traditional explanatory criteria. This does not mean the articles are wrong. It means they have not answered the predictive ability question, which increasingly interests journals and reviewers.

6. Cross-validation Limitations to Understand

CVPAT and PLSpredict both rely on k-fold cross-validation, a powerful technique with limitations that introductory guidance rarely discusses.

• K-fold cross-validation is not equivalent to testing on a completely independent sample collected separately after the model is finalized. The folds still come from the same data collection exercise, context, and time period.
• If data cleaning, variable selection, or model refinement uses information from the dataset that will also be used for testing, data leakage may make predictive results appear overly optimistic.
• For panel data, grouped data, or interdependent observations, conventional random fold splitting may be inappropriate because it disrupts the data's dependence structure.
• Repeatedly running cross-validation to select the best-looking model version, then reporting only that version, also biases the final performance assessment even if each individual run is conducted correctly.

7. Avoid Turning CVPAT into a Mechanical Rule

CVPAT answers a specific question: does the model have statistically significantly lower average prediction error than the benchmark? It does not test causality, and accurate prediction does not automatically validate the theoretical structure underlying the model. A model may predict well by capturing a stable correlation without accurately representing the causal mechanism proposed by theory.

Conversely, a non-significant CVPAT result does not mean the theory is wrong. It means that, for the particular data and benchmark used, there is insufficient statistical evidence of predictive superiority. Requiring significant CVPAT in every PLS-SEM article as a quality threshold is an overly mechanical reading that disregards studies' different objectives.

8. Example Wording and Checklist

• Present R² and CVPAT as separate evidence rather than combining them into a single conclusion.
• Cite Liengaard et al. (2021) for CVPAT's theoretical origin and Sharma et al. (2022) for practical guidance.
• State which benchmark CVPAT uses, IA or LM, because conclusions may differ between them.
• For time-structured, grouped, or dependent data, consider whether the default random k-fold procedure remains appropriate.
• Do not treat CVPAT as evidence of causality or theoretical correctness.