IA, LM, RMSE, and MAE in PLSpredict: Four Symbols, Two Different Questions
Anyone opening a PLSpredict table in SmartPLS 4 encounters these four symbols, but they do not answer the same question. IA and LM are comparison benchmarks; RMSE and MAE are measures of prediction error. Confusing these roles commonly produces interpretations that are wrong even when the reported figures are correct.
Start readingPrediction needs observations held out from estimation
- Split observations
Set folds and repetitions
- Fit
Use the training observations
- Evaluate
RMSE, MAE and Q²predict
- Benchmark
Read IA, LM and CVPAT in their roles
Conceptual diagram. Use consistent data splits when comparing prediction benchmarks.
Source: Lê Minh · Conceptual illustration, not software output.1. Four Symbols, Two Different Questions
A standard SmartPLS 4 PLSpredict table usually contains at least four columns per indicator: Q²predict, PLS-SEM RMSE, LM RMSE, and sometimes MAE for both models. They may all look like accuracy measures, but they address two distinct questions. First, does PLS-SEM predict better than a naive guess? Q²predict, calculated against IA, addresses this. Second, among models with predictive value, how well does PLS-SEM perform against ordinary linear regression? Comparing RMSE or MAE against LM addresses this.
Conflating these questions is the most common PLSpredict interpretation error. A model may greatly outperform IA, giving high Q²predict, yet improve only slightly on LM, or vice versa. Report both pieces of information because they show different aspects of predictive ability.
IA (Indicator Average): the most naive benchmark, predicting an indicator's value using its own mean in the training sample. LM (Linear Model): a stronger benchmark using OLS regression of an endogenous indicator on its direct antecedent indicators, ignoring PLS-SEM's measurement and path structures. RMSE and MAE: two measures of average prediction error used to compare PLS-SEM with LM, rather than to calculate Q²predict.
2. IA: The Benchmark behind Q²predict
In the PLSpredict procedure of Shmueli et al. (2019), Q²predict compares the PLS-SEM model's sum of squared prediction errors (SSE) with the IA benchmark's sum of squared errors (SSO), using Q²predict = 1 − SSE / SSO. IA is not a statistical model: it uses the indicator's training-sample mean in each fold to predict every holdout observation, regardless of the actual antecedent-variable values.
Because IA uses no predictive information beyond the mean, a PLS-SEM model needs only to outperform an average-for-everyone guess to obtain positive Q²predict. Positive Q²predict is therefore a necessary rather than sufficient condition for good predictive ability. Negative Q²predict is a serious warning: the model predicts worse than using the sample mean without any antecedent variables.
IA deliberately sets an easy benchmark, resembling the role of the original Stone-Geisser Q² in blindfolding. Positive Q²predict establishes little about predictive strength; it rules out performance worse than a guess requiring no predictor input. Assessing strength requires the subsequent LM comparison.
3. LM: The Benchmark for Classification
LM is a separate OLS regression automatically estimated by PLSpredict for each endogenous indicator. It uses indicators of the direct antecedent constructs as independent variables without passing through latent variables. In essence, it bypasses PLS-SEM's measurement model: no outer weights or paths between constructs, only direct indicator-to-indicator linear regression.
As a statistical model using antecedent information, LM is much harder to outperform than IA. Lower PLS-SEM RMSE than LM for most indicators suggests that the measurement and path structures absent from LM contribute additional predictive value rather than simply representing the same linear regression more elaborately.
IA asks whether the model improves on prediction without predictor data. LM asks whether it improves on a simple linear regression that already uses those data. An ideal PLS-SEM model outperforms both, but its margin over LM is the practically meaningful comparison with other analytical techniques.
4. RMSE and MAE: Different Sensitivity to the Same Errors
RMSE (root mean square error) and MAE (mean absolute error) both measure average differences between predicted and actual holdout values, but calculate them differently. RMSE takes the square root of mean squared error, magnifying a few very large errors. MAE averages absolute errors, treating their sizes proportionally.
Penalizes large errors heavily through squaring. Appropriate when prediction errors are expected to be approximately normally distributed and large deviations are particularly concerning. Sensitive to holdout outliers.
Weights errors proportionally to their magnitude. More stable when the error distribution is strongly skewed or the holdout sample contains unusual observations.
Because of these calculations, RMSE is always at least as large as MAE for the same predictions. A wider gap indicates less uniform prediction errors: some observations are predicted very poorly while the rest are relatively accurate. Shmueli et al. (2019) recommend RMSE as the primary measure, switching to MAE when prediction errors are strongly skewed. This draws on Chai and Draxler's (2014) argument for RMSE when approximately normal errors are expected. Willmott and Matsuura (2005) argued instead that MAE represents average error more intuitively and should be preferred in climate model assessment. This statistical debate remains unsettled, so reporting both is the safest PLSpredict approach.
If PLS-SEM predicts most observations well but makes a few severe errors, its RMSE may exceed LM's while its MAE remains lower, because MAE does not magnify those extreme errors. Where the measures disagree for an indicator, report both and explain the error distribution or holdout outliers rather than selecting the more favorable measure.
5. The Predictive Ability Classification Rule
Shmueli et al. (2019) propose counting indicator-level RMSE or MAE comparisons between PLS-SEM and LM to assign a single predictive ability level for an endogenous variable, provided its Q²predict is positive. If Q²predict is negative, the following steps are not pursued.
| Indicators where PLS-SEM beats LM (lower RMSE/MAE) | Predictive ability level |
|---|---|
| All indicators | High |
| Most, but not all, indicators | Medium |
| A minority of indicators | Low |
| No indicators (LM wins throughout) | No practical predictive ability, even though Q²predict may remain positive |
Adapted from Shmueli, Ray, Estrada, and Chatla's (2019) practical PLSpredict guidance. This rule counts indicator-level wins and losses as a heuristic rather than a statistical test. Obtaining a p-value for the difference requires CVPAT (Liengaard et al., 2021; Sharma et al., 2022).
6. Reading an Actual PLSpredict Table
Jumani's (2026) extended TPB model for consumer behavior toward Islamic brands (CBIB) provides a practical example of reading these symbols in the correct order. CBIB achieved Q²predict = 0.52, clearly positive and meeting the necessary condition against IA. The next step compares PLS-SEM and LM RMSE for each CBIB indicator. Table 12 reports lower PLS-SEM RMSE for most indicators, classifying CBIB's predictive ability as medium to high, beyond the minimal predictive value established by positive Q²predict alone.
These steps cannot be reversed. Comparing RMSE against LM before confirming positive Q²predict is uninformative: once a variable underperforms even IA, the model has failed the most basic condition, making LM wins or losses secondary.
7. Common Mistakes
| Common wording | Why it is wrong | Better wording |
|---|---|---|
| "Q²predict is positive, so the model predicts well." | Positive Q²predict establishes that the model beats IA, a low benchmark, without establishing the strength of predictive ability. | State that positive Q²predict is necessary, then report RMSE/MAE comparisons with LM to determine predictive ability. |
| "RMSE is lower than LM, so RMSE alone is sufficient." | This ignores possible disagreement between RMSE and MAE under skewed errors. | Report both RMSE and MAE; explain indicator-level disagreement through the error distribution. |
| "LM and IA are the same: both are linear benchmarks." | IA is not a statistical model and does not use antecedent variables; LM is an actual OLS regression using those variables. | Distinguish IA for Q²predict from LM for RMSE/MAE classification, keeping the benchmarks separate. |
- Check positive Q²predict for every endogenous variable before examining its RMSE/MAE.
- Report both RMSE and MAE at the indicator level rather than selecting the more favorable result.
- Use the counting rule of Shmueli et al. (2019) to label predictive ability high, medium, or low rather than inventing thresholds.
- Do not describe IA and LM as the same benchmark in Methods or Results.
Example Results Wording
[Endogenous variable] achieved Q²predict = [value], outperforming the IA benchmark (Shmueli et al., 2019). Compared with the linear model (LM), PLS-SEM RMSE was lower for [X/Y] indicators (MAE: [X/Y] indicators), indicating [low/medium/high] predictive ability under the classification rule of Shmueli et al. (2019).
Conclusion
IA, LM, RMSE, and MAE form two pairs of tools addressing different questions. IA establishes minimal predictive value, LM establishes performance against an actual statistical benchmark, and RMSE and MAE provide different views of the same errors. Reading them in sequence, from Q²predict against IA to RMSE/MAE against LM, prevents premature claims of good prediction based on a positive value alone and strengthens the defensibility of PLSpredict reporting in theses and articles.
References
Primary Sources on PLSpredict and Error Measures
- Shmueli, G., Ray, S., Estrada, J.M.V. and Chatla, S.B. (2019). Predictive model assessment in PLS-SEM: Guidelines for using PLSpredict. European Journal of Marketing, 53(11), 2322-2347. https://doi.org/10.1108/EJM-02-2019-0189
- Chai, T. and Draxler, R.R. (2014). Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geoscientific Model Development, 7(3), 1247-1250. https://doi.org/10.5194/gmd-7-1247-2014
- Willmott, C.J. and Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30(1), 79-82. https://doi.org/10.3354/cr030079
- Liengaard, B.D., Sharma, P.N., Hult, G.T.M., Jensen, M.B., Sarstedt, M., Hair, J.F. and Ringle, C.M. (2021). Prediction: Coveted, yet forsaken? Introducing a cross-validated predictive ability test in partial least squares path modeling. Decision Sciences, 52(2), 362-392. https://doi.org/10.1111/deci.12445
- Sharma, P.N., Liengaard, B.D., Hair, J.F., Sarstedt, M. and Ringle, C.M. (2022). Predictive model assessment and selection in composite-based modeling using PLS-SEM: Extensions and guidelines for using CVPAT. European Journal of Marketing, 57(6), 1662-1677. https://doi.org/10.1108/EJM-08-2020-0636
Illustrative Source
- Jumani, Z.A. (2026). Predictive validation of an extended TPB model in Islamic branding: PLSpredict and CVPAT evidence from Muslim and non-Muslim Thai consumers. Journal of Islamic Marketing, ahead-of-print. https://doi.org/10.1108/JIMA-09-2025-0574
Academic note: Illustrative figures come from Jumani (2026), published in Journal of Islamic Marketing. RMSE and MAE formulas are expressed verbally to suit the article format. SmartPLS 4 automatically calculates both for each indicator in its PLSpredict output.