Revista de Psicología del Trabajo y de las Organizaciones Revista de Psicología del Trabajo y de las Organizaciones
Journal of Work and Organizational Psychology 33 (2017) 175-82 - Vol. 33 Num.3 DOI: 10.1016/j.rpto.2017.07.004
The Five-Factor model and job performance in low complexity jobs: A quantitative synthesis
El modelo de Cinco Factores y el desempeño en el trabajo en puestos de poca complejidad: una síntesis cuantitativa
Mario Lado, , Pamela Alonso
University of Santiago de Compostela, Spain
Received 20 June 2017, Accepted 26 July 2017
Abstract

This article presents the results of four primary studies that investigated the degree to which the Big Five personality dimensions predict job performance in occupations with a low level of job complexity. Job performance was assessed as overall job performance (OJP), task performance (TP), and contextual performance (CP). The results showed that conscientiousness and emotional stability proved to be predictors of the three performance measures. In addition, extroversion was a relevant predictor of OJP and TP, and agreeableness was a predictor of CP. Implications for the theory and practice of job performance and personnel selection are discussed.

Resumen

Este artículo presenta los resultados de cuatro estudios primarios que investigaron el grado en que los Cinco Grandes factores de personalidad predecían el desempeño en el trabajo en ocupaciones de bajo nivel de complejidad. El desempeño en el trabajo fue evaluado como desempeño global (DG), desempeño de tarea (DT) y desempeño contextual (DC). Los resultados mostraron que los factores de conciencia y estabilidad emocional fueron predictores de las tres medidas de desempeño. Además, extroversión fue un predictor relevante de DG y DT y amabilidad fue predictor de DC. Finalmente se discuten las implicaciones de los resultados para la teoría y la práctica del desempeño en el trabajo y la selección del personal.

Keywords
Five Factor model, Personality, Job performance, Job complexity, Meta-analysis
Palabras clave
Modelo de los Cinco Factores, Personalidad, Desempeño en el trabajo, Complejidad del puesto, Metaanálisis

Personality inventories are frequently used by small and medium-size companies in their personnel selection processes (Alonso, Moscoso, & Cuadrado, 2015) and several meta-analytical reviews on the relationship between personality and job performance have been carried out during the last two decades (e.g., Barrick & Mount, 1991, 1993; Barrick, Mount, & Judge, 2001; Hogan & Holland, 2003; Hough, 1992; Judge, Rodell, Klinger, Simon, & Crawford, 2013; Salgado, 1997, 2002, 2003; Salgado, Anderson, & Tauriz, 2015; Salgado & Tauriz, 2014; Tett, Jackson, & Rothstein, 1991). These meta-analyses have shown that the Big Five personality factors are valid predictors of important work behaviors. For example, Barrick and Mount (1991) found that conscientiousness was a valid predictor of job performance, and that it generalized the validity across jobs and criteria. These researchers also showed that the other factors are predictors of some criteria for some jobs. Hough (1992) also found that personality measures are predictors of several organizational and educational criteria. In Europe, Salgado (1997) found that conscientiousness and emotional stability were valid predictors of job performance across jobs and that extroversion, openness, and agreeableness were valid predictors for specific occupations and criteria.

The most recent meta-analyses (e.g., Judge et al., 2013; Salgado, 2003; Salgado, Anderson et al., 2015; Salgado, Moscoso et al., 2015; Salgado & Tauriz, 2014) have confirmed the previous findings and shown that the validity of the Five-Factor Model (FFM) can be larger depending on the way in which the personality factors are measured (e.g., with personality inventories developed using the FFM framework and if the personality inventories are quasi-ipsative forced-choice personality inventories).

Recent research has also shown that the FFM predicts occupational attainment, expatriate cross-cultural adjustment and outcomes, creativity and innovation, and counterproductive behaviors at work (AlDosiry, Alkhadher, AlAqraa, & Anderson, 2016; Costa, Páez, Sánchez, Garaigordobil, & Gondim, 2015; Gilar, De Haro, & Castejón, 2015; Raman, Sambasiva, & Kumar, 2016; Salgado & Bastida, 2017). Moreover, research on applicant reactions has shown that personality inventories are well rated across the world (Aguado, Rico, Rubio, & Fernandez, 2016; Anderson, Ahmed, & Costa, 2012; Anderson, Salgado, & Hülsheger, 2010; Liu, Potocnik, & Anderson, 2016; Snyder & Shahani-Denning, 2012).

Despite the empirical evidence of the validity of the FFM, several researchers have criticized personality variables on the basis that they can be affected by faking when the individual is motivated to do it, for instance, in personnel selection processes (Grieve & Hayes, 2016; Morgeson et al., 2007; Salgado, 2016).

Taken together, the results of these reviews make two conclusions appear reasonable. First, the FFM can reasonably predict job performance and its sub-dimensions. Second, the Five Factor Model is a good taxonomy for integrating the personality measures developed from different theoretical perspectives.

However, there are a number of issues that require additional research. For instance, the potential moderator effects of job components on the validity of the FFM have received less attention. For example, few studies have examined whether the validity of the Big Five can be affected by job complexity, defined as the degree of information processing required by the tasks (Hunter & Hunter, 1984; Schmidt, Shaffer, & Oh, 2008). Hunter and Hunter (1984) found that the data dimension of the occupational code of the Dictionary of Occupational Titles (DOT; US Department of Labor, 1991) mainly represents a job complexity dimension. Family II (feed) and V (compare/copy) would include the low complexity jobs. According to Hunter and Hunter (1984), low complexity jobs represent about 20.1% of occupations and they are more characterized by individual duties. Therefore, these characteristics may require a different set of personality variables than more complex occupations.

There is some empirical evidence that job complexity can moderate the relationship between personality and job performance. For example, Spector (1982) found that high anxiety (low emotional stability) was negatively related to job performance in complex tasks but was not related to performance in simple, less complex tasks. Judge, Bono, and Locke (2000) found that job complexity correlated negatively with neuroticism and positively with self-esteem, generalized self-efficacy, and locus of control.

Moreover, performance is defined as any behavior or activity, under the individual's control, adjustable in terms of ability and relevant for the organizational goals (Campbell, 1990). Nowadays, there is a consensus that the job performance domain includes at least two dimensions: task and contextual performance (Aguinis, 2007; Moscoso, Salgado, & Anderson, 2017; Salgado & Cabal, 2011). Task performance is defined as the proficiency with which employees perform the core technical activities that the job description includes (Borman, Bryant, & Dorio, 2010). Contextual performance refers to the contributions of the employee, that go beyond the technical obligations of the work, and that impact on the organizational, social, and psychological environment, helping to accomplish organizational goals (Borman, Penner, Allen, & Motowidlo, 2001; Dorsey, Cortina, & Luchman, 2010; Hoffman & Dilchert, 2012). Furthermore, Van Scotter and Motowidlo (1996) subdivided contextual performance into two narrower dimensions: interpersonal performance and job dedication.

The main objective of this article is to analyze the validity of the Big Five personality dimensions for predicting performance in jobs that involve a low level of complexity. From a theoretical perspective, job complexity can be a relevant moderator of validity. For example, Judge et al. (2000) showed that job complexity moderated the relationship between personality and job satisfaction. Therefore, understanding the personality correlates of occupations of a low complexity level can be relevant in developing models and theories of work behaviors and performance.

In order to provide some insight into the validity of the Big Five, we first present the results of four primary studies in which a FFM-based inventory was used to assess personality and we examine the relationship of the Big Five with overall job performance, task performance, and contextual performance. The four studies were conducted in jobs of a low complexity level. Next, we report the results of a meta-analytic integration of the findings of the four studies.

Previous research has demonstrated that the facets included in the FFM did not show incremental validity over the Big Five factors (Salgado, Moscoso, & Berges, 2013; Salgado, Moscoso et al., 2015). Therefore, this study has been conducted at the Big Five level. This approach has the advantage of not requiring the correction of the observed validities for imperfect construct measurement, as some researchers have done in previous meta-analyses in which scales of facets were meta-analyzed with factor scales (e.g., Mount, Barrick, & Stewart, 1998; Salgado, 1997, 2003).

Based on the findings of previous meta-analyses and primary studies, we posit three hypotheses:Hypothesis 1

Emotional stability and conscientiousness are valid predictors for overall, task, and contextual performance ratings.

Hypothesis 2

Extraversion is a valid predictor for overall job performance and task performance.

Hypothesis 3

Agreeableness is a valid predictor of contextual performance.

MethodSamples

As four independent studies were carried out, four independent samples were collected for doing the primary studies. The characteristics of these samples are described below:

Study 1: The participants of study 1 were 32 individuals who worked as private security agents in a Spanish based company at the time of testing. All the subjects were male and their age ranged from 25 to 32 years. All of them had been employed by the company for at least four years. The subjects and the supervisors of a local office of the company were invited to participate in this study. They were informed that a validation study of a personality questionnaire was being conducted. Around 50% of the employees agreed to participate in the study. The supervisors provided performance ratings of the employees they supervised. The study design was concurrent.

Study 2: The participants of this study were 46 male employees of a Spanish cold storage company who participated voluntarily in an organizational assessment program at the company. All subjects were male and their age ranged from 31 to 56 years. All of them had been employed by the company for several years. They were informed that an organizational assessment program was being conducted in order to provide information, suggestions, and future improvements. Around 96% of the employees agreed to serve as subjects for the study. The study design was concurrent.

Study 3: The sample of the third study consisted of 72 weavers in a medium-size Spanish textile factory. In this case, the sample was selected on the basis of their scores on a FFM-based personality inventory and supervisory ratings of job performance were collected six months later. Therefore, the study design was predictive.

Study 4: The participants consisted of 44 male operators of a medium-size Spanish ceramics company. They participated voluntarily in the study and their direct supervisor provided job performance ratings. Their age and tenure was similar to the previous samples. The study design was concurrent.

Measures

In all the studies personality was assessed at the factor level, that is, we assessed emotional stability, extroversion, openness, agreeableness, and conscientiousness. Two personality inventories, developed within the FFM framework, were used for assessing the factors. Their psychometric characteristics are described below.

IP/5F. This 200-item inventory was developed through rational and factor analytical methods to measure the five personality factors of the Big Five model (Salgado, 1996, 1998). Items were grouped into 29 homogeneous item clusters based on item content. Items are answered by one of 3 alternative options: agreement, indecision, and disagreement. The Emotional Stability (ES) factor includes the clusters of hostility, stress, sadness, anxiety, not-worrying, insecurity, and control/relaxation. Extraversion (EX) includes the clusters of assertiveness, group sense, reserved, sense of humor, leadership, and self-centered. Openness to Experience (O) includes the clusters of imagination and intuition, creativeness, adventurousness, amplitude of interests, and non-conventionality. Agreeableness (A) includes the clusters of honesty, modesty, cooperativeness, tolerance, and trust. Conscientiousness (C) includes the clusters of work effort, order and organization, carefulness, prudence, high performance, and dutifulness. The internal consistency coefficients for ES, EX, O, A, and C are .90, .86, .80, .74, and .87, respectively. One year test-retest reliabilities for ES, EX, O, A, and C were .91, .90, .79, .65, and .72, respectively. The reliability of the homogeneous item clusters ranged from .57 for tolerance (agreeableness) to .84 for anxiety (emotional stability), and the mean was .71, .69, .68, .67, and .72 for ES, EX, O, A, and C, respectively. Convergent and discriminant validity evidence was found using the HPI (Hogan & Hogan, 1995; Salgado et al., 2013), the NEO-FFI (Costa & McCrae, 1992), the EPI (Eysenck & Eysenck, 1975), the CEP (Moscoso & Salgado, 2004), and the 16PF (Cattell, Eber, & Tatsuoka, 1970). For example, the correlations between IP/5F and the NEO-FFI factors measuring corresponding constructs were .70, .88, .55, .55, and .58 for ES, EX, O, A, and C, respectively. In another study (N = 200), the correlations between the HPI and the IP/5F were .75 for ES, .69 and .74 for EX (the HPI divides Extraversion into two subfactors), .85 for O, .51 for A, and .67 for C (Salgado, Moscoso, & Lado, 2003). A further study (N = 410) shows that the correlations between IP/5F and the NEO-PI-R were .84, .80, .64, .72, and .78 for ES, EX, O, A, and C, respectively (Salgado, Moscoso et al., 2015). Therefore, as a whole, the IP/5F has shown excellent convergence with other personality inventories assessing the same personality dimensions. Exploratory and confirmatory factor analyses confirmed the five-factor structure of the IP/5F (Salgado, 1996). This inventory was used in studies 1, 3, and 4.

Description en Cinq Dimensions (D5D). The D5D questionnaire measures the five domains of the FFM (Rolland & Mogenet, 2001). The questionnaire includes 55 trait adjectives as markers for the five domains; each domain is represented by 11 adjectives that are scored on a 6-point Likert scale, ranging from absolutely not characteristic of me (1) to fully characteristic of me (6). The D5D has accumulated evidence of adequate reliability for the big five dimensions: emotional stability (.84), introversion (.79), openness (.74), agreeableness (.84), and conscientiousness (.77); moreover, the D5D has also demonstrated adequate convergent correlations with the domains assessed by the NEO-PI-R (Costa & McCrae, 1992), ranging from .54 (Agreeableness) to .73 (Conscientiousness). This personality inventory was used in study 2.

Job Performance

Two measures were used to evaluate job performance. The first one was specifically designed to evaluate job performance for the study of security agents. The second instrument was used in the other three studies. Next, we describe the two scales used in this research to appraise job performance and its sub-dimensions.

Security Agent Performance Scale: Twenty competencies for the security agent job were identified in this research. Job analysis using interviews with the agents and their supervisors were carried out. Also, 44 subjects were asked to communicate critical incidents in the job and an extensive description of an “ideal” security agent. 26 critical incidents and 38 descriptions of an “ideal” agent were collected. Then, three researchers using this information developed the following competencies: (1) compliance with labor standards and rules, (2) personal initiative, (3) stable and controlled mood (calm), (4) relationships with co-workers, (5) relationships with supervisors (6) order and thoroughness at work, (7) personal presence, (8) punctuality, (9) resistance to monotony, (10) ability to solve problems/difficulties, (11) positive vision of the service, (12) ability to persuade and influence effectively, (13) analytical and reflexive behavior, (14) self-confidence, (15) professional knowledge, (16) relationships with customers/users, (17) availability for service, (18) work motivation, (19) decision-making ability and (20) global assessment of the employee.

Two immediate supervisors rated the participants on each competence using a 5-point scale (1= unsatisfactory, 5 = excellent). After, the mean score was calculated for each competence. The inter-rated reliability ranges from .504 to .80 with a mean of .643 (SD=.096).

Job Performance Scale: In the other three studies, performance was rated by the employee's immediate supervisor using 10 behaviorally anchored rating scales (BARS), which were developed through the critical incident technique (Flanagan, 1954). Each BARS used a 5-point scale ranging from poor performance (1) to excellent performance (5). The job dimensions rated in studies 2, 3, and 4 were: (1) technical knowledge, (2) ability to persuade and influence effectively, (3) relationships with supervisors, (4) order and organization in the workplace, (5) meticulousness in the tasks, (6) stable and calm mood, (7) availability for service, (8) productivity, (9) job motivation, (10) relationships with co-workers, and (11) overall job performance.

Procedure

The participants in the first study completed the IP/5F while attending a training session at the company. Later, each subject received two independent appraisals from two supervisors, who used the scale described above. The supervisors were blind to the subjects’ scores on the IP/5F. The ratings were used for research purposes only.

In the case of sample two, the subjects participated in an organizational climate study. During this study, they completed the D5D (Rolland & Mogenet, 2001), which was not relevant for the climate study purposes. The employee performance was rated by the direct supervisor. The ratings were collected for research purposes only, which means that the supervisors did not have to justify their evaluations. The supervisor was blind to the scores of the participants in the personality inventory.

In the case of sample three, the participants were applicants and they answered the IP/5F together with several additional tests with no relevance for the present study. The scores were used to make hiring decisions. Job performance was rated by the immediate supervisor, who was unaware of the scores of the employee on the IP/5F. The performance ratings were used for research purposes exclusively. The study design was predictive.

Finally, in the case of the sample four, the participants completed the IP/5F voluntarily and their performance ratings were provided by the direct supervisor, who agreed to participate in the study, and who were blind to the employee personality scores. In this case, the supervisor only provided the overall evaluations of the workers, so it was not possible to know the scores obtained for task and contextual performance. The study design was concurrent and the performance ratings and the personality scores were used for research purposes only.

Statistical Analyses

Two types of statistical analyses were done. First, the correlations between the FFM and job performance were calculated for each individual sample. Next, the results of the four samples were used to conduct a psychometric meta-analysis.

In order to carry out the meta-analysis, four artefactual errors were considered: sampling error, measurement error in predictor and criterion, and range restriction in the predictor. The four errors produce error variance and the measurement error and range restriction also produce an underestimation of the true correlation between predictor and criterion. Consequently, three artifact distributions were developed.

Predictor reliability. In Table 1 the reliability for each personality factor for the two personality inventories and the average of the reliability and the standard deviation for each personality factor appears. In the meta-analysis, we used these distributions to correct for measurement error in the predictors.

Table 1.

Reliability Coefficients for the Big Five.

Personality
Inventory  N    ES  EX 
D5D  501    .86  .73  .74  .75  .86 
IP/5F  760    .90  .86  .80  .74  .87 
    X¯:  .88  .88  .77  .75  .87 
    SD.03  .09  .04  .01  .01 

Note. ES=Emotional Stability; EX=Extroversion; O=Openness to Experience; A=Agreeableness; C=Conscientiousness; X¯=mean; SD=standard deviation.

Criterion reliability. When a measure of employee performance is the criterion used in validation studies, there are several reliability coefficients that can be used to correct for criterion reliability. However, the inter-rater coefficient is the most accurate, since it controls for the majority of the measurement error sources (Rodriguez, 2016; Salgado, 2015; Salgado & Moscoso, 1996; Salgado, Moscoso, & Anderson, 2016; Salgado, Anderson et al., 2003; Schmidt & Hunter, 1996; Schmidt, Le & Ilies, 2003; Viswesvaran, Ones, & Schmidt, 1996). Due to the fact that the interrater reliability was available only for the first study, we used an empirical distribution based on the coefficient reported by Salgado et al. (2016). The results of their second-order meta-analysis reported that the interrater reliability for a single rate was .52, and the combined standard deviation was .105.

Range restriction of predictor. In order to check if the personality measures were affected by range restriction, we compared the standard deviation of the samples with the standard deviation reported by the authors of the instruments used (Rolland & Mogenet, 2001; Salgado, 1998). The u values are reported in Table 2.

Table 2.

Range Restriction (u) Values for the Big Five Personality Factors.

Job  N  EE  EX 
Security Guard  32  0.95  1.04  0.99  1.07  0.92 
Operator  46  0.81  0.85  0.87  0.92  0.78 
Weaver  72  0.93  0.83  1.05  1.15  0.80 

Note. ES=Emotional Stability; EX=Extroversion; O: Openness to Experience; A=Agreeableness; C=Conscientiousness.

We conducted a psychometric meta-analysis, aggregating the results found with the primary samples, using the software package developed by Schmidt and Le (2014), based on the meta-analysis methods of Schmidt and Hunter (2015).

Results

Firstly, the validity coefficients for the Big Five found in the four samples that make up this study are reported. The results for overall job performance are explained in the first place and results for task and contextual performance are presented below. Finally, a more detailed explanation of the results is offered when the results of the meta-analysis (carried out from the aggregate of the primary samples) are reported.

Table 3 shows the observed validity coefficients of the Big Five for predicting overall job performance in the four samples included in this study. The first column presents the sample sizes, which oscillate between 32 in the first sample and 72 in the third one. The next column reports the validity coefficients for each personality factor. The observed validity coefficients found for emotional stability varied from .10 in the sample of unskilled workers to .28 in the sample of security agents. With regard to the extroversion factor, the lowest coefficient was .13 for the sample of operators and the highest value was .39 (security agents). On the other hand, the values obtained for openness to experience ranged from -.18 (operators) to .22 (security agents), and the values for agreeableness ranged from -.07 (security agents) to .31 (unskilled workers). The validity coefficients for conscientiousness ranged from .09 (operators) to .36 (unskilled workers).

Table 3.

Observed Validity Coefficients of the Big Five for Predicting Overall Job Performance.

Job  N  EE  EX 
Security guard  32  .28  .39  .22  -.07  .22 
Operator  46  .12  .13  -.18  -.05  .09 
Weaver  72  .26  .19  .03  .01  .11 
Unskilled workers  44  .10  .15  .17  .31  .36 

Note. ES=Emotional Stability; EX=Extroversion; O: Openness to Experience; A=Agreeableness; C=Conscientiousness.

The validity coefficients of the Big Five for predicting task performance are reported in Table 4. Task performance was evaluated in only two of the samples included in the current study (operators and weavers). The results for emotional stability ranged from .03 for the sample of operators to .28 for the weavers. With regard to extroversion, the lowest value was .06 (operators) and the highest was .16 (weavers). For their part, the validity coefficients for openness to experience ranged from -.08 to .03. In the case of agreeableness, the values ranged from -.06 to -.03. The values found for conscientiousness were both positive and very similar (.17 and .14).

Table 4.

Validity Coefficients of the Big Five for Predicting Task Performance.

Job  N  EE  EX 
Operator  46  .03  .06  -.08  -.06  .17 
Weaver  72  .28  .16  .03  -.03  .14 

Note. ES=Emotional Stability; EX=Extroversion; O: Openness to Experience; A=Agreeableness; C=Conscientiousness.

The results related to contextual performance are presented in Table 5. The samples that included a measure of contextual performance were the same as in the case of task performance. The validity coefficients for contextual performance were .11 and .21 for the factor of emotional stability, .17 and .18 for extroversion, -.13 and -.02 for openness to experience, .02 and .14 to agreeableness, and .03 and .11 for conscientiousness.

Table 5.

Validity Coefficients of the Big Five for Predicting Contextual Performance.

Job  N  EE  EX 
Operatives  46  .11  .17  -.13  .02  .03 
Weavers  72  .21  .18  -.02  .14  .11 

Note. ES=Emotional Stability; EX=Extroversion; O: Openness to Experience; A=Agreeableness; C=Conscientiousness.

Table 6 presents the results of the meta-analysis of the predictive validity of the Big Five in low complexity jobs that we calculated using the results of the primary samples collected in this research. From left to right, the first column shows the total sample size (N), the second column indicates the number of effect size coefficients (K), the third column is the average effect size weighted by sample size (rw), and the next three columns show the observed variance (S2Obs), the observed standard deviation (SDObs) and the variance due to sample error (S2SE). The operational validity (rop) and the true effect size (ρ) are shown in the seventh and eighth columns, respectively. The variance of the true effect (S2ρ) and the percentage of the observed variance accounted for by artefactual errors (% VE) are reported in the next columns. Finally, the 90% credibility value (90% CV) and the 95% confidence interval of ρ (95% ICρ) appear in the last two columns.

Table 6.

Results of the Meta-analysis of the Predictive Validity of the Big Five in Low Complexity Jobs.

  N  K  rw  S2Obs  SDObs  S2SE  r0p  ρ  S2ρ  % VE  90% CVρ  95% ICρ 
Overall Job Performance
Emotional Stability  194  .19  .006  .08  .0195  .31  .32  100  .32  .29 / .34 
Extroversion  194  .20  .008  .09  .0194  .32  .35  100  .35  .32 / .38 
Openness to Experience  194  .04  .021  .14  .0210  .06  .07  100  .07  .06 / .08 
Agreeableness  194  .05  .021  .14  .0298  .06  .08  100  .08  .07 / .09 
Conscientiousness  194  .18  .011  .11  .0197  .31  .32  100  .32  .30 / .34 
Task Performance
Emotional Stability  118  .18  .015  .12  .0161  .30  .31  100  .31  .28 / .34 
Extroversion  118  .12  .002  .05  .0167  .21  .23  100  .23  .21 / .25 
Openness to Experience  118  -.01  .003  .05  .0172  -.02  -.02  100  -.02  -.02 / -.02 
Agreeableness  118  -.04  .000  .01  .0172  -.05  -.06  100  -.06  -.05 / -.07 
Conscientiousness  118  .15  .000  .01  .0165  .28  .29  100  .29  .26 / .32 
Contextual Performance
Emotional Stability  118  .17  .002  .05  .0163  .28  .29  100  .29  .26 / .32 
Extroversion  118  .18  .000  .00  .0162  .31  .33  100  .33  .30 / .36 
Openness to Experience  118  -.06  .003  .05  .0171  -.09  -.10  100  -.10  -.09 / -.11 
Agreeableness  118  .09  .003  .06  .0169  .12  .14  100  .14  .13 / .15 
Conscientiousness  118  .08  .001  .04  .0170  .15  .15  100  .15  .14 / .16 

Note. N=accumulated sample size; K=number of studies; rw=average effect size weighted by sample size; S2Obs=observed variance of the observed effect sizes; SDObs=standard deviation of the observed effect sizes; S2SE=variance due to sample error; rOp=operational validity; ρ=true validity; S2ρ=variance of ρ; % VE=percentage of observed variance accounted for by artifactual errors; 90% CVρ=90% credibility value of ρ. 95% ICρ=95% confidence interval of ρ.

The findings for overall job performance are shown at the top of the Table. The number of samples used in these meta-analyses was 4, with a total accumulated sample of 194 individuals. The first row shows the results for emotional stability. The observed validity was .19, which increased to .31 when it was corrected for criterion reliability and range restriction in the predictor (operational validity). The true validity (fully-corrected correlation) was .32, which coincides with the 90% credibility value, as the percentage of the variance accounted for by artefactual errors was 100%. The lower and upper limits of the 95% confidence interval for ρ ranged from .29 to .34, which confirms that, besides generalizing the validity, the result is statistically significant. These results supported Hypothesis 1.

The results for extraversion are shown in the second row. The observed validity was .20, the operational validity was .32, and the true validity .35 (validity generalization is confirmed). In this case, the limits of the 95% confidence interval were .32 and .38. These results showed that extroversion was the personality factor with the highest validity coefficient for the low complexity jobs included in these meta-analyses, even higher than emotional stability and conscientiousness. The results supported Hypothesis 2.

In the next row appear the results for openness to experience, whose estimates of validity were .04, .06, and .07 for observed, operational and true validity, respectively. For openness to experience, the lower and upper limits of the 95% CI were .06 and .08, respectively. The 90% CV was positive, which means that the validity is generalized, but near to 0. For agreeableness, the observed validity was .05 for agreeableness. The operational validity was .06 and the true validity was .08. The 95% CI ranged from .07 to .09, but as in the previous factor, near to 0.

In the case of conscientiousness, the observed validity was .18, the operational validity was .31, and the true validity was .32 (predictive validity is generalized). The 95% CI for this factor ranged from .30 to .34, and the 90% CV was positive. Therefore, these results supported Hypothesis 1.

The next section of Table 6 shows the meta-analysis results for task performance. In this case, the meta-analysis included the two samples which measured this performance facet. The results show that emotional stability obtained the higher values, for observed validity (.18), operational validity (.30) and true validity (.31), which generalized its validity. With regard to extroversion, the observed validity was .12, the operational validity was .21, and the true validity was .23. These results confirm Hypothesis 2, but although this factor is a good predictor of task performance and its validity is generalized, its predictor capacity is lower than for overall job performance.

For its part, openness to experience obtained an observed validity of -.01, which increased to -.02 in the case of true validity and operational validity. In relation to agreeableness, the results obtained were similar to the previous factor. In fact, the observed validity was -.04 and the operational validity and the true validity were -.05 and -.06, respectively. With regard to these two factors, we can conclude that neither is a predictor of task performance.

Finally, the results related to conscientiousness showed an observed validity of .15 that, when corrected for predictor reliability and range restriction, increased to .28 (operational validity) and to .29 when in addition it was corrected for criterion reliability (true validity). Predictive validity is generalized for this factor. These results are in accordance with the previous literature and, therefore, support Hypothesis 1.

The final part of the table shows the results in relation to contextual performance. Firstly, the observed validity reported for emotional stability was .17, which increased to .28 for operational validity and to .29 for true validity, and that generalizes its validity as a predictor of contextual performance. Secondly, the values found for the extroversion factor were the highest for contextual performance: .18 (observed validity), .31 (operational validity), and .33 (true validity), which also generalizes its validity. The next values reported on the table were about openness to experience, which ranges from an observed validity of -.06 to a true validity of -.10 (in this case the operational validity was -.09), in accordance with the previous results for overall job performance and task performance. Regarding agreeableness, the observed value was .09, the operational value .12, and the true validity .14. This means that agreeableness proves to be a good predictor of contextual performance in low complexity jobs and that it generalizes its validity, which confirms Hypothesis 3. Finally, the observed validity for conscientiousness was .08, which increased to .15 both for operational and true validity, which means that this factor predicted contextual performance and generalizes its validity too.

Discussion

The capacity of the FFM to predict various organizational criteria has been extensively studied. Research has also suggested that job complexity can moderate the validity of the Big Five personality dimensions. However, there are hardly any studies on the relationships between personality and job performance for low complexity jobs. This research aimed to provide some evidence of the validity of the Big Five in low complexity occupations for predicting OJP, TP, and CP. For this purpose, four independent samples were used and, subsequently, a meta-analysis was conducted with the results of the four primary studies. The results supported the three hypotheses we had posited.

This article makes several relevant contributions. First, it shows that emotional stability and conscientiousness were consistently predictors of overall job performance, task performance, and contextual performance and that they generalized validity across the four low complexity jobs included in this research. The findings also show that conscientiousness predicts overall job performance and task performance better than it does contextual performance in low complexity jobs. Emotional stability predicted the three criteria similarly.

The second contribution of this research has been to show that extroversion was a relevant predictor of the three performance criteria. Extroversion showed a particularly large validity for predicting overall performance and contextual performance. The validity size in both cases was over .30. The validity was also significant for task performance although slightly smaller. We had hypothesized that extroversion would be a predictor of overall job performance and task performance but we did not hypothesize its predictive validity for contextual performance. Therefore, this is an unexpected but important finding. In fact, the results for extroversion reveal that this personality factor is the most important factor for predicting overall performance in the four low complexity occupations examined in the current research.

With regard to agreeableness, the results supported Hypothesis 3, as this personality factor was shown to be a predictor of contextual performance, although the magnitude of the validity is small.

A third contribution of this research has been to show that the variability observed in the validities was fully explained by artefactual errors, mainly sampling error. Consequently, the predictive validity for the FFM as a whole generalized across the four occupations.

These findings have some implications for the theory and practice of personnel selection. From the theoretical point of view, the findings suggest that job complexity can moderate the validity of the Big Five. This is particularly relevant in the case of extroversion as previous meta-analytic research had shown that it was a relevant predictor for occupations characterized by interpersonal situations. None of the jobs included in this research had significant interpersonal requirements. However, extroversion was a predictor of the three performance criteria and not only of contextual performance. Another theoretical implication is that conscientiousness may be a better predictor of task performance and overall performance than of contextual performance. In regard to emotional stability, the results disagree with Spector's (1982) findings that emotional stability is not related to performance in low complexity tasks. On the contrary, emotional stability is a crucial predictor of the three performance criteria in these four occupations. Therefore, as a whole, the findings suggest two points. First, a theory of job performance should include job complexity as a potential moderator of the validity of personality factors. Second, the theory of job performance has to be comprehensive in the sense that a different set of personality factors may be necessary to predict the various performance criteria.

From a practical point of view, the findings have implications for the practice of personnel selection. The findings suggest that a combination of emotional stability, extroversion, and conscientiousness can predict job performance and its sub-dimensions very efficiently in low complexity occupations. Taking into account that the applicant reactions to personality inventories are generally good (Anderson et al., 2010), the combination of these three factors can be very efficient alone or in combination with other predictors, such as cognitive ability tests and behavioral interviews.

This research has some limitations that should be mentioned. The first limitation is that we tested the hypotheses in only four jobs and they are not necessarily a representative sample of low complexity occupations. Therefore, future research should be conducted with additional occupations characterized by a low complexity level. The second limitation is that the samples were small in all cases and the cumulative sample was also small. Thus, the validity estimates can change as the sample increases. Therefore, additional studies for these occupations are also needed.

In summary, the findings suggest that emotional stability, conscientiousness, and extroversion are three robust predictors of supervisor ratings of performance in four low complexity jobs. A combination of these three factors can result in a substantial amount of predictive efficiency in personnel selection processes.

Conflict of Interest

The authors of this article declare no conflict of interest.

Financial Support

The two authors contributed equally. The research reported in this manuscript was supported by Grant PSI2014-56615-P from the Spanish Ministry of Economy and Competitiveness and Grant 2016 GPC GI-1458 from the Consellería de Cultura, Educación e Orden, Xunta de Galicia.

References
Aguado et al., 2016
D. Aguado,R. Rico,V.J. Rubio,L. Fernández
Applicant reactions to social network web use in personnel selection
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 183-190 http://dx.doi.org/10.1016/j.rpto.2016.09.001
Aguinis, 2007
H. Aguinis
Performance management
Edinburgh Business School, Heriot-Watt University, UK, (2007)
AlDosiry et al., 2016
K.S. AlDosiry,O.H. Alkhadher,E.M. AlAqraa,N. Anderson
Relationships between emotional intelligence and sales performance in Kuwait
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 39-45 http://dx.doi.org/10.1016/j.rpto.2015.09.002
Alonso et al., 2015
P. Alonso,S. Moscoso,D. Cuadrado
Procedimientos de selección de personal en pequeñas y medianas empresas españolas
Revista de Psicología del Trabajo y de las Organizaciones, 31 (2015), pp. 79-89 http://dx.doi.org/10.1016/j.rpto.2015.04.002
Anderson et al., 2012
N. Anderson,S. Ahmed,A.C. Costa
Applicant reactions in Saudi Arabia: Organizational attractiveness and core-self evaluation
International Journal of Selection and Assessment, 20 (2012), pp. 197-208 http://dx.doi.org/10.1111/j.1468-2389.2012.00592.x
Anderson et al., 2010
N. Anderson,J.F. Salgado,U.R. Hülsheger
Applicant reactions in selection: Comprehensive meta-analysis into reaction generalization versus situational specificity
International Journal of Selection and Assessment, 18 (2010), pp. 291-304 http://dx.doi.org/10.1111/j.1468-2389.2010.00512.x
Barrick and Mount, 1991
M.R. Barrick,M.K. Mount
The big five personality dimensions and job performance: a meta-analysis
Personnel Psychology, 44 (1991), pp. 1-26 http://dx.doi.org/10.1111/j.1744-6570.1991.tb00688.x
Barrick and Mount, 1993
M.R. Barrick,M.K. Mount
Autonomy as a moderator of the relationships between the Big Five personality dimensions and job performance
Journal of Applied Psychology, 78 (1993), pp. 111-118 http://dx.doi.org/10.1037/0021-9010.78.1.111
Barrick et al., 2001
M.R. Barrick,M.K. Mount,T. Judge
Personality and performance at the beginning of the new millennium: What do we know and where do we go next?
International Journal of Selection and Assessment, 9 (2001), pp. 9-30 http://dx.doi.org/10.1111/1468-2389.00160
Borman et al., 2010
W.C. Borman,R.H. Bryant,J. Dorio
The measurement of task performance as criteria in selection research
Handbook of employee selection, pp. 439-462
Borman et al., 2001
W.C. Borman,L.A. Penner,T.D. Allen,S.J. Motowidlo
Personality predictors of citizenship performance
International Journal of Selection and Assessment, 9 (2001), pp. 52-69 http://dx.doi.org/10.1111/1468-2389.00163
Campbell, 1990
J.P. Campbell
Modeling the performance prediction problem in industrial and organizational psychology
Handbook of industrial and organizational psychology, 2nd ed, pp. 687-732
Cattell et al., 1970
R.B. Cattell,H.W. Eber,M.M. Tatsuoka
Handbook for the sixteen personality factor questionnaire, (16 PF)
Institute for Personality and Ability Testing, (1970)
Costa and McCrae, 1992
P.T. Costa,R.R. McCrae
Revised NEO personality inventory, (NEO-PI-R), and NEO five factor inventory (NEO-FFI) professional manual
Psychological Assessment Resources, (1992)
Da Costa et al., 2015
S. Da Costa,D. Páez,F. Sánchez,M. Garaigordobil,S. Gondim
Personal factors of creativity: A second order meta-analysis
Revista de Psicología del Trabajo y de las Organizaciones, 31 (2015), pp. 165-173 http://dx.doi.org/10.1016/j.rpto.2015.06.002
Dorsey et al., 2010
D. Dorsey,J. Cortina,J. Luchman
Adaptive and citizenship-related behaviors at work
Handbook of employee selection, pp. 463-487
Eysenck and Eysenck, 1975
H.J. Eysenck,S.B.G. Eysenck
Manual for the Eysenck personality questionnaire
Hodder & Stoughton, (1975)
Flanagan, 1954
J.C. Flanagan
The critical incident technique
Psychological Bulletin, 51 (1954), pp. 327-358
Gilar et al., 2015
R. Gilar,J.M. De Haro,J.L. Castejón
Individual differences in predicting occupational success: the effect of population heterogeneity
Revista de Psicología del Trabajo y de las Organizaciones, 31 (2015), pp. 101-107 http://dx.doi.org/10.1016/j.rpto.2015.03.002
Grieve and Hayes, 2016
R. Grieve,J. Hayes
Employment testing online, offline, and over the phone: Implications for e-assessment
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 95-101 http://dx.doi.org/10.1016/j.rpto.2016.04.001
Hoffman and Dilchert, 2012
B.J. Hoffman,S. Dilchert
A review of citizenship and counterproductive behaviors in organizational decision-making
The Oxford handbook of personnel assessment and selection, pp. 543-569
Hogan and Hogan, 1995
R. Hogan,J. Hogan
Manual for the Hogan personality inventory
Hogan Assessment Systems, (1995)
Hogan and Holland, 2003
J. Hogan,B. Holland
Using theory to evaluate personality and job performance relations: A socioanalytic perspective
Journal of Applied Psychology, 88 (2003), pp. 100-112 http://dx.doi.org/10.1037/0021-9010.88.1.100
Hough, 1992
L.M. Hough
The ‘Big Five’ personality variables--construct confusion: Description versus prediction
Human Performance, 5 (1992), pp. 139-155 http://dx.doi.org/10.1080/08959285.1992.9667929
Hunter and Hunter, 1984
J.E. Hunter,R.F. Hunter
Validity and utility of alternative predictors of job performance
Psychological Bulletin, 96 (1984), pp. 72-98 http://dx.doi.org/10.1037/0033-2909.96.1.72
Judge et al., 2000
T.A. Judge,J.E. Bono,E.A. Locke
Personality and job satisfaction: The mediating role of job characteristics
Journal of Applied Psychology, 85 (2000), pp. 237-249 http://dx.doi.org/10.1037/0021-9010.85.2.37
Judge et al., 2013
T.A. Judge,J.B. Rodell,R.L. Klinger,L.S. Simon,E.R. Crawford
Hierarchical representations of the Five-Factor Model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives
Journal of Applied Psychology, 98 (2013), pp. 875-925 http://dx.doi.org/10.1037/a0033901
Liu et al., 2016
X. Liu,K. Potocnik,N. Anderson
Applicant reactions to selection methods in China
International Journal of Selection and Assessment, 24 (2016), pp. 296-303 http://dx.doi.org/10.1111/ijsa.12148
Morgeson et al., 2007
F.P. Morgeson,M.A. Campion,R.L. Dipboye,J.R. Hollenbeck,K. Murphy,N. Schmitt
Reconsidering the use of personality tests in personnel contexts
Personnel Psychology, 60 (2007), pp. 683-729 http://dx.doi.org/10.1111/j.1744-6570.2007.00089.x
Moscoso and Salgado, 2004
S. Moscoso,J.F. Salgado
“Dark side” personality styles as predictors of task, contextual, and job performance
International Journal of Selection and Assessment, 12 (2004), pp. 356-362 http://dx.doi.org/10.1111/j.0965-075X.2004.00290.x
Moscoso et al., 2017
S. Moscoso,J.F. Salgado,N. Anderson
How do I get a job, What are they looking for? Personnel Selection Assessment
An Introduction to Work and Organizational Psychology, pp. 25-47
Mount et al., 1998
M.K. Mount,M.R. Barrick,G.L. Stewart
Five-Factor Model of personality and Performance in Jobs Involving Interpersonal Interactions
Human Performance, 11 (1998), pp. 145-165 http://dx.doi.org/10.1080/08959285.1998.9668029
Raman et al., 2016
P. Raman,M. Sambasiva,N. Kumar
Counterproductive work behavior among frontline government employees: Role of personality, emotional intelligence, affectivity, emotional labor, and emotional exhaustion
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 25-37 http://dx.doi.org/10.1016/j.rpto.2015.11.002
Rodríguez, 2016
A. Rodríguez
Validez predictiva e impacto adverso de la entrevista conductual estructurada en el sector público
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 75-85 http://dx.doi.org/10.1016/j.rpto.2016.04.003
Rolland and Mogenet, 2001
J.P. Rolland,J.L. Mogenet
Manuel d’application Description en Cinq Dimensions (D5D) [Technical manual Description in Five Dimensions - D5D]
Les Editions du Centre de Psychologie Appliquée, (2001)
Salgado, 1996
J.F. Salgado
Análisis exploratorio y confirmatorio del Inventario de Personalidad de Cinco Factores (IP/5F)
Psicológica, 17 (1996), pp. 353-366
Salgado, 1997
J.F. Salgado
The five factor model of personality and job performance in the European Community
Journal of Applied Psychology, 82 (1997), pp. 30-43 http://dx.doi.org/10.1037/0021-9010.82.1.30
Salgado, 1998
J.F. Salgado
Manual Técnico para el Inventario de Personalidad de Cinco Factores IP/5F
Tórculo, (1998)
Salgado, 2002
J.F. Salgado
The Big Five personality dimensions and counterproductive behaviors
International Journal of Selection and Assessment, 10 (2002), pp. 117-125 http://dx.doi.org/10.1111/1468-2389.00198
Salgado, 2003
J.F. Salgado
Predicting job performance using FFM and non-FFM personality measures
Journal of Occupational and Organizational Psychology, 76 (2003), pp. 323-346 http://dx.doi.org/10.1348/096317903769647201
Salgado, 2015
J.F. Salgado
Estimating Coefficients of equivalence and stability for job performance ratings: The importance of controlling for transient error on criterion measurement
International Journal of Selection and Assessment, 23 (2015), pp. 37-44 http://dx.doi.org/10.1111/ijsa.12093
Salgado, 2016
J.F. Salgado
A Theoretical Model of Psychometric Effects of Faking on Assessment Procedures: Empirical findings and implications for personality at work
International Journal of Selection and Assessment, 24 (2016), pp. 209-228 http://dx.doi.org/10.1111/ijsa.12142
Salgado et al., 2003a
J.F. Salgado,N. Anderson,S. Moscoso,C. Bertua,F. De Fruyt,J.P. Rolland
A meta-analytic study of GMA validity for different occupations in the European Community
Journal of Applied Psychology, 88 (2003), pp. 1068-1081 http://dx.doi.org/10.1037/0021-9010.88.6.1068
Salgado et al., 2015a
J.F. Salgado,N. Anderson,G. Tauriz
The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis
Journal of Occupational and Organizational Psychology, 88 (2015), pp. 797-834 http://dx.doi.org/10.1111/joop.12098
Salgado and Bastida, 2017
J.F. Salgado,M. Bastida
Predicting Expatriate Effectiveness: The role of personality, cross-cultural adjustment, and organizational support
International Journal of Selection and Assessment, 25 (2017), pp. 267-275 http://dx.doi.org/10.1111/ijsa.12178
Salgado and Cabal, 2011
J.F. Salgado,A.L. Cabal
Evaluación del Desempeño en la Administración Pública del Principado de Asturias: Análisis de las Propiedades Psicométricas
Revista de Psicología del Trabajo y de las Organizaciones, 27 (2011), pp. 75-91
Salgado and Moscoso, 1996
J.F. Salgado,S. Moscoso
Meta-analysis of interrater reliability of job performance ratings in validity studies of personnel selection
Perceptual and Motor skills, 83 (1996), pp. 1195-1201 http://dx.doi.org/10.2466/pms.83.3f.1195
Salgado et al., 2016
J.F. Salgado,S. Moscoso,N. Anderson
Correction for criterion reliability in validity generalization: The consistency of Hermes, the utility of Midas
Revista de Psicología del Trabajo y de las Organizaciones, 32 (2016), pp. 17-23 http://dx.doi.org/10.1016/j.rpto.2015.12.001
Salgado et al., 2013
J.F. Salgado,S. Moscoso,A. Berges
Conscientiousness, Its Facets, and the Prediction of Job Performance Ratings: Evidence against the narrow measures
International Journal of Selection and Assessment, 21 (2013), pp. 74-84 http://dx.doi.org/10.1111/ijsa12018
Salgado et al., 2003b
J.F. Salgado,S. Moscoso,M. Lado
Evidence of cross-cultural invariance of the big five personality dimensions in work settings
European Journal of Personality, 17 (2003), pp. S67-S76 http://dx.doi.org/10.1002/per.482
Salgado et al., 2015b
J.F. Salgado,S. Moscoso,J.I. Sanchez,P. Alonso,B. Choragwicka,A. Berges
Validity of the five-factor model and their facets: The impact of performance measure and facet residualization on the bandwidth-fidelity dilemma
European Journal of Work and Organizational Psychology, 24 (2015), pp. 325-349 http://dx.doi.org/10.1080/1359432x.2014.903241
Salgado and Tauriz, 2014
J.F. Salgado,G. Tauriz
The Five-Factor Model, forced-choice personality inventories, and job performance: A comprehensive meta-analysis of academic and occupational studies
European Journal of Work and Organizational Psychology, 23 (2014), pp. 3-30 http://dx.doi.org/10.1080/1359432X.2012.716198
Schmidt and Hunter, 1996
F.L. Schmidt,J.E. Hunter
Measurement error in psychological research: Lessons from 26 research scenarios
Psychological Methods, 1 (1996), pp. 199-223 http://dx.doi.org/10.1037/1082-989X.1.2.199
Schmidt and Hunter, 2015
F.L. Schmidt,J.E. Hunter
Methods of meta-analysis
Third edition, Sage, (2015)
Schmidt and Le, 2014
F.L. Schmidt,H. Le
Software for the Hunter-Schmidt meta-analysis methods. Department of Management and Organizations
Iowa City, IA, (2014)
Schmidt et al., 2003
F.L. Schmidt,H. Le,R. Ilies
Beyond Alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs
Psychological Methods, 8 (2003), pp. 206-224 http://dx.doi.org/10.1037/1082-989X.8.2.206
Schmidt et al., 2008
F.L. Schmidt,J.A. Shaffer,I.-S. Oh
Increased accuracy for range restriction corrections: implications for the role of personality and general mental ability in job and training performance
Personnel Psychology, 61 (2008), pp. 827-868 http://dx.doi.org/10.1111/j.1744-6570.2008.00132.x
Snyder and Shahani-Denning, 2012
J. Snyder,C. Shahani-Denning
Fairness reactions to personnel selection methods: A look at professionals in Mumbai, India
International Journal of Selection and Assessment, 20 (2012), pp. 297-307 http://dx.doi.org/10.1111/j.1468-2389.2012.00601.x
Spector, 1982
P.E. Spector
Behavior in organizations as a function of employee's locus of control
Psychological Bulletin, 91 (1982), pp. 482-497 http://dx.doi.org/10.1037/0033-2909.91.3.482
Tett et al., 1991
R.P. Tett,D.N. Jackson,M. Rothstein
Personality measures as predictors of job performance: a meta-analytic review
Personnel psychology, 44 (1991), pp. 703-742 http://dx.doi.org/10.1111/j.1744-6570.1991.tb00696.x
US Department of Labor, 1991
US Department of Labor (1991). Dictionary of Occupational Titles (DOT). Forth Edition, Revised 1991. Lincolnwood, IL: VGM Career Books.
Van Scotter and Motowidlo, 1996
J.R. Van Scotter,S.J. Motowidlo
Interpersonal facilitation and job dedication as separate facets of contextual performance
Journal of Applied Psychology, 81 (1996), pp. 525-531 http://dx.doi.org/10.1037/0021-9010.81.5.525
Viswesvaran et al., 1996
C. Viswesvaran,D.S. Ones,F.L. Schmidt
Comparative analysis of the reliability of job performance ratings
Journal of Applied Psychology, 81 (1996), pp. 557-560 http://dx.doi.org/10.1037/0021-9010.78.4.679
Corresponding author. University of Santiago de Compostela. 15782 Santiago de Compostela, Spain. (Mario Lado mario.lado@usc.es)
Copyright © 2017. Colegio Oficial de Psicólogos de Madrid
Journal of Work and Organizational Psychology 33 (2017) 175-82 - Vol. 33 Num.3 DOI: 10.1016/j.rpto.2017.07.004