Social skills inventory for adolescents : evidence of construct validity and reliability

Social skills are considered as an important indicator of social and emotional development. In the Brazilian context, the Social Skills Inventory for Adolescents (IHSA-Del-Prette) is available for the assessment of this repertoire, and is widely used in research and intervention. Favorable psychometric properties have been established through Principle Components Analysis (PCA). The present study aimed to compare the structure established through PCA with that established through Exploratory Factor Analysis (EFA) and to verify the indicators of validity and reliability produced by Confirmatory Factor Analysis (CFA). A total of 2291 students of 6th, 7th and 8th years of Elementary Education and 1st and 2nd years of High School Education participated. The results showed a factorial structure very similar to that originally produced with PCA, with six oblique factors confirmed through CFA and with suitable indicators of reliability. The IHSA-Del-Prette presented psychometric qualities that recommend indicate its use in the Brazilian context.


Introduction
The literature extensively demonstrates positive correlations between a good social skills repertoire in adolescence and positive outcomes such as quality of life, health, social-emotional development, and prospects for professional success and psychosocial adjustment.Negative correlations with behavior problems, psychological disorders, aggression, and other adjustment difficulties are also highlighted (Barkin, Smith, & Durant, 2002;Del Prette, Falcone, & Murta, 2013;Fraser & Williams, 2004;Goldberg-Lillehoj, Spoth, & Trudeau, 2005).According to Del Prette and Del Prette (2005), deficits in social skills may be part and/or effect of the symptomology of psychological disorders and can constitute warning signs of possible problems in later stages of development.Psychological resilience research results point out the importance of both, social skills and social competence, in particular during the developmental phases that immediately precede and succeed the transition to adulthood (Obradovic, Burt, & Masten, 2010;Masten & Tellegen, 2012).
As explained by Del Prette and Del Prette (2001) and Del Prette and Del Prette (2005;2010/2012), social skills and social competence are distinct constructs, although they are linked in the analysis of social performance: the term social skills is descriptive of the classes of learned social behaviors available in the repertoire of the individual; social competence is an evaluative attribute of performance, based on its functions (instrumental and ethical) and on the coherence with the thoughts and feelings of the individual (Del Prette & Del Prette, 2001;2010/2012).According to these authors, an elaborate social skills repertoire can be considered a necessary although insufficient condition for social competence.In fact, other competing factors, such as beliefs, values, plans, and goals, may affect the social performance.These authors emphasize the role of culture on the social skills, since it defines the patterns expected and valued on different contexts or roles, taking into account demographic characteristics, such as age, gender or social roles.
To monitor the social skills repertoire in adolescence is important in order to prevent psychopathology and to promote the promotion of mental health (Del Prette & Del Prette, 2006;Del Prette et al., 2013), in particular within those groups who are exposed to significant risk (Del Prette & Del Prette, 2009), such as: negative parenting practices of physical punishment, neglect and psychological threats (Salvo, 2010); mothers and fathers with psychopathologies such as depression and consequent difficulty in educating their children (Goodman et al., 2010); friendships with deviant peers (Salvo, 2010); use and abuse of substances and increased conflicts with parents and feelings of loneliness (Barbosa Filho, Campos, & Lopes, 2012); violent partners (Murta, Santos, Martins, & Oliveira, 2013); low academic performance (D'Abreu & Marturano, 2010) or even difficulty in initiating and maintaining friendships (Gresham, Cook, Crews, & Kern, 2004).
Exposure to these conditions increases the likelihood of the emergence of internalizing (depression, anxiety and withdrawal) and externalizing (breaking rules, aggression, agitation) behavior problems for the adolescent, having both a high prevalence within adolescent population (Achenbach & Rescorla, 2001;Rocha, 2012).Therefore, as highlighted in the metaanalysis by Durlak, Weissberg and Pachan (2010), acquiring a socially skilled and socially competent repertoire can prevent, alleviate or reduce existing behavioral problems.For example, self-control and feelings expression make it more likely to effective problem solving and decision making.Chen (2006), in his literature review on the acquisition of social skills by young people, highlighted that social skills help to develop behavioral patterns that contribute to the development of positive relationships, better academic performance, more satisfactory family relationships and with peers, leading therefore to more social acceptance and less exclusion.
Taking into consideration the above, it is essential to have precise and valid instruments, which can be used to develop effective interventions in this area.The Social Skills Inventory for Adolescents was recently developed in Brazil (IHSA-Del-Prette, Del Prette & Del Prette, 2009).This instrument has already approved by the Federal Psychology Council and is provided with reference standards.In addition to the favorable psychometric properties, described in the IHSA-Del-Prette Manual, some studies have already evidenced other indicators of its criterion validity (Del Prette, Rocha, Silvares, & Del Prette, 2012), as well as additional indicators of reliability and convergent validity (Del Prette, Teodoro, & Del Prette, 2014).
The quality of psychological assessment instruments represents a process that must be continually reevaluated based on new indicators and for each sample (Anastasi & Urbina, 1997).The initial construct validation study of the IHSA-Del-Prette (Del Prette & Del Prette, 2009) produced satisfactory results regarding the percentage of explained variance of the scales (from 40% to 62%) and items with adequate factor loadings (greater than 0.40).However, the initial structure was strictly based on Principal Component Analysis (PCA) which has been considered as a preliminary but not sufficient method for the construct validation of psychological instruments (Tabachnick & Fidell, 2007), specially for further confirmatory analysis of its factorial structure.Alternatively, the Exploratory Factor Analysis (EFA) is more appropriate, not only due to its emphasis on the covariance or commonality of the variables, excluding the specific variance and error variance (Tabachnick & Fidell, 2007), but also because it produces factors, assumed to be the hypothetical "causes" of the correlations between the variables.Principal Component Analysis groups the items but does not associate them with the factors, a procedure which is necessary to perform the Confirmatory Factor Analysis (CFA) for new samples.Obtaining indicators of fit for a factorial structure to new samples, through CFA, is important in the validation of psychological tests due to ratifying the theorization inherent to the factors and the empirical validity of this structure (Pilati & Laros, 2007).
Considering that, in Brazil, the IHSA-Del-Prette is the only available scale to evaluate the adolescent needs and deficits for social skills training and to evaluate preventive and therapeutic programs aimed to improve this repertoire.Therefore, this instrument is widely used in research and intervention, which makes it crucial to monitor its psychometric properties.Based on the above considerations, this study aimed: (a) to compare the original subscale structure of the IHSA-Del-Prette, obtained through PCA in the original study (IHSA-Del-Prette-Manual), with that obtained through Exploratory Factor Analysis (EFA), using the same sample; (b) to confirm, through CFA, the structure previously produced through EFA; (c) to present internal consistency indicators of the factors obtained through EFA and CFA.
The second and third samples (referred from now on as IHSA-Del-Prette-Conf) were taken from two studies conducted with adolescents.The first study (Campos, 2014) collected data with 642 adolescents, and the second (Leme, 2013) involved 477 adolescents.Thus the IHSA-Del-Prette-Conf total sample was composed of 1119 students, being 51.9% (581) female and with ages between 12 and 17 years (M = 14.11,SD = 1.31).The adolescents were attending the 6 th (n = 138, 12.3%), 7 th (n = 318, 28.4%) and 8 th years (n = 186, 16.6%) of Elementary Education and the 1 st (n = 309, 27.6%) and 2 nd (n = 168, 15.0%) years of High School Education of public schools of two medium sized cities of the states of São Paulo and Minas Gerais.Regarding the socioeconomic level, the highest proportion of students were concentrated in the C (24.3%), B2 (38.4%) and B1 (27.1%) classes.

Data Collection
Data collection followed the same procedure in the three studies.This was performed collectively in the classrooms of the schools of the adolescents, at a time arranged by the teachers, when the terms of consent forms, signed by the person legally responsible for the student, were returned.At the beginning of each application, the aim of the study was presented, and guarantees regarding data confidentiality were given, reiterating that participation in the research was voluntary.The researchers remained in the classroom throughout the application of the instruments answering to possible questions, which lasted for an average of 30 minutes.

Data Analysis
Data were entered into the SPSS 18.0 program.First, the analysis of the missing data was performed through the Expectation-maximization method (EM).Next, the diagnosis of the multivariate outliers was made through the Mahalanobis distance (D 2 ) (p1 and p2 <0.001) according to Marôco (2010), and the normality of the variables verified through the coefficients of skewness (sk) and kurtosis (ku), with values of | sk |>3 indicators of severe asymmetry, and values of | ku |>10 indicators of severe kurtosis (Kline, 2011).Finally, the existence of multicollinearity was analyzed through the verification of the correlation tables of items with values greater than 0.90, with their suppression being indicated (Marôco, 2010).
For the Exploratory Factor Analysis (EFA), with the Principal Axis Factoring method, the original validation sample of the IHSA-Del-Prette-Manual were used.Factorial validity was determined based on Confirmatory Factor Analysis (CFA) using the Amos 18.0 program.The parameters were estimated based on the correlation matrix, using the robust maximum likelihood extraction method.Following the recommendations of Kline (2011) and Schweizer (2010), the overall fit of the hypothesized model was evaluated based on the following set of indices and their reference values for an acceptable fit: ratio χ 2 /g.l. < 5 (Arbuckle, 2008); Bentler Comparative Fit Index -CFI > 0.90 (Hu & Bentler, 1999), Root Mean Square Error of Approximation -RMSEA <0.08 (Arbuckle, 2008) and Standardized Root Mean Square Residual -SRMR <0.10 (Hu & Bentler, 1999).The quality of the local fit was evaluated based on standardized factor loadings and the individual coefficients of reliability of the items, considering values higher than 0.50 and 0.25, respectively, to be indicators of a good fit and, consequently, confirmation of the factorial validity of the model (Marôco, 2010).
The reliability of the IHSA-Del-Prette was assessed using Cronbach's alpha, considering a value over 0.70 to be a good level of consistency (Anastasi & Urbina, 1997), and Composite Reliability (CR) that is obtained for each factor by dividing the square of the sum of the standardized factor loadings by this value plus the sum of the errors or residues of each item.Values above 0.70 were considered to be indicative of good construct reliability (Marôco, 2010).

Exploratory Factor Analysis of the IHSA-Del-Prette: Structure and Reliability
The adequacy of the sample to the PAF (Principal Axis Factoring) was verified using the Kaiser-Meyer-Olkin index (KMO = 0.992) and Bartlett's sphericity test (χ² = 9717.621;df = 630; p = 0.001), which indicated that factor analysis was strongly recommended (Tabachnick & Fidell, 2007).The 38 items of the original instrument were submitted to PROMAX rotation (oblique), considering that factors are expected to be related to each other (Sass & Schmitt, 2010) and taking an eigenvalue value higher than 1 as the factor extraction criterion.For the selection of the items, commonalities and saturations higher than 0.30 were considered (Worthington & Whittaker, 2006).
The analysis revealed a factor structure very similar to that originally produced using PCA, with six factors, however, showing two differences: some changes in the order of some items within the factors; exclusion of items 6 ("When an opportunity arises, I do small favors, such as offering my chair or opening the door for someone, without being asked" from Civility factor) and 36 ("In relation to affection, at any time, I frankly say what displeases me" from Affective Approach factor).The factor structure resulted in 36 items distributed within six factors that presented factor loadings above 0.30, with eigenvalues between 1.17 and 7.92, and with a total explained variance of 33.28%.Because items 7 and 35 scored in more than one factor (Empathy and Civility factors; Empathy and Affective Approach factors, respectively), following the suggestions of Sass and Schmitt (2010), the selection of these items was made using the value of greater commonality in the factor as the criterion.
The analysis of internal consistency, based on Cronbach's Alpha coefficient, produced reasonable values in the six factors of the IHSA-Del-Prette-Manual ranging between 0.61 and 0.80, and α = 0.89 for the Overall Score.

Confirmatory Factor Analysis of the IHSA-Del-Prette: Structure and Reliability
Prior to performing the CFA, examination of the multivariate outliers (Mahalanobis distance, D 2 ) was conducted.A total of 50 outlier observations were found, and a new CFA was performed without them.As the results did not differ from those obtained previously, the choice was made to maintain them in the sample, as suggested by Marôco (2010).The absence of multicollinearity and the psychometric sensitivity of the items of the IHSA-Del-Prette-Conf, analyzed from the distribution of the values of skewness and kurtosis, showed no marked difference from normal distribution (Kline, 2011).
Figure 1 presents the Confirmatory Factor Analysis of the IHSA-Del-Prette-Conf for the sample of adolescents.
In the identification of the model, it was specified that each item of the IHSA-Del-Prette belonged to a single factor without cross saturation (according to the CFA), and that the measurement errors were not correlated.The correlation between the six factors was freely estimated.
The testing of the model, with six oblique factors of the IHSA-Del-Prette-Conf, presented acceptable indices and a good overall fit: χ 2 /df = 2.84; CFI 0.90; RMSEA = 0.04; SRMR = 0.04), with no need for respecification of the model, with its theoretical fit being prioritized.All the estimated parameters remained statistically significant (p < 0.001).Regarding the local fit, with the exception of items 8 and 38 (Self-control), 27 (Assertiveness), 10 (Affective Approach), and 17 (Social Resourcefulness), items had factor loadings and individual reliability coefficients higher than 0.50 and 0.25 respectively. Suitable

Discussion
The self-report inventories are widely used by practitioners and researchers, not only due to their simplicity and ease, but because they allow a normative and intra-individual analysis of the repertoire of any individual.The process of validation of a specific psychological measure is always an ongoing process that should be accurately described in order to guarantee its quality and adequacy to the populations they were designed to.It is then important to constantly update information regarding the accuracy of the measurement and demonstration of reliability to relevant population in order to avoid decisions on selecting tests that rely solely on face value or on test-users testimonials (ITC, 2014).Considering the social skills inventories, in the first case, the aim is usually to compare the results from one individual with a standard reference sample, to identify his/her resources and deficits; in the second, the aim is to analyze relationships between subsets of social skills and their possible relationships with processes, such as self-awareness, self-esteem, cognitive schemas etc. (Caballo, Del Prette, Monjas, & Carrillo, 2006).In case of preventive and therapeutic interventions, a valid and reliable measure is indispensable to check the effectiveness of social skills programs as well as to identify and to improve their critical ingredients.
Over the last few years, Brazilian researchers and practitioners had only the IHSA -Del-Prette scale to evaluate the repertoire of social skills of adolescents.Although recommended by the Federal Council of Psychology, this scale was validated only by the method of Principal Component Analysis.Thus, the items were grouped into components, and not on factors such as derived from factor analysis.This prevented from checking its validity for new samples, which is usually done by the method of Factor Confirmatory Analysis.Considering this situation, the present study aimed to re-evaluate the construct validity and reliability of the Social Skills Inventory for Adolescents (IHSA-Del-Prette), both by Exploratory Factor Analysis and Confirmatory Factor Analysis.
At the first moment, the comparison between the initial structure, based on PCA, and the structure obtained in this study, based on EFA suggest the stability of the factorial structure it was verified that considering the substantial overlapping between the factors.This suggests that the theoretical model proposed for the social skills with adolescents was confirmed: interrelated items that satisfactorily explain the latent constructs investigated in each factor (Marôco, 2010;Pilati & Laros, 2007).However, it can be seen that the explained variance was smaller (33.28%) than the desirable (>50%), although similar to some other psychological scales.The internal consistency of the Empathy, Assertiveness and Affective Approach factors were slightly smaller than those originally obtained by the PCA, what is common when comparing PCA and EFA.Anyway, these results suggest that the theoretical model proposed for the social skills with adolescents was confirmed, with inter-related items that satisfactorily explain the latent constructs investigated in each factor (Marôco, 2010;Pilati & Laros, 2007).
Further evidence for these assumptions was found by the CFA: the theoretical model of the IHSA-Del-Prette, with six oblique factors, presented a satisfactory local and global fit, thus suggesting a fit of the inventory with the data of the sample (Hu & Bentler, 1999;Kline 2011;Schweizer, 2010).It was considered that the factor loadings of two items (0.47 for item 8 and 0.46 for item 10) did not invalidate the factorial validity of the instrument, especially considering that the other overall indicators of the scale were acceptable (Figure 1).
Some possible explanations may be raised for the lower loading values observed in the case of three of the items (38, 27 and 17).In relation to items 27 -I show my annoyance at my/my brother (sister) when he (she) aggravates me, messes with my stuff, etc. -and 38 -I manage to control my anger when my brother (sister) irritates me in some way -it is possible that the adolescents who had no siblings had difficulty imagining this particular interpersonal situation.With regard to item 17 -"I talk about sex with my parents" -the descriptive analyzes indicated that the majority of the adolescents (57.8%) presented a low frequency of occurrence (between 0 and 4 times) in this item, indicating difficulty in performing this social skill.This value was discrepant in relation to the other social skills of the Social Resourcefulness factor, most of them with a frequency of occurrence between 3 and 4 times, thus influencing the validity and reliability of this item.For this reason, and considering the importance of these social skills in the adolescence (Barbosa Filho et al., 2012;D'Abreu & Marturano, 2010;Goodman et al., 2010;Gresham et al., 2004;Murta et al., 2013;Salvo, 2010), it was decided to retain these items in the inventory.
In summary, this study confirmed the construct validity of the IHSA-Del-Prette, and indicated, through CFA, better indicators of internal consistency than those obtained, through PCA, in the original study.The instrument showed good psychometric qualities for adolescents in the Brazilian context, demonstrating a stable structure, which allows the Social Skills construct to be assessed in line with what was theoretically proposed.Considering the potential of the instrument evaluated, future studies could further investigate the cross-cultural invariance of the IHSA-Del-Prette, as well as refine the comprehension regarding gender, producing more evidence for the validity of the instrument.
Considering the problem guiding this study, it is reasonable to state that the IHS-Del-Prette is a valid and reliable resource for use in research and practice of social skills evaluation.In addition, the similarity of its structure of items, as obtained by CPA EFA and then confirmed by CFA, allows to sustain that the previous studies with IHSA-Del-Prette are not compromised by being based on its principal component structure.However, considering the cumulative nature of the validity evidence for any psychometric instrument, it is understood that further studies and methodological care with the quality of evaluations in this area are desirable and cannot be disregarded.

Conclusion
Despite the good psychometric indicators obtained for the IHSA-Del-Prette, some limitations of this study should be considered.Participants were assessed by convenience, providing a non-representative sample of the Brazilian population.The psychometric property results relate to a determined sample, not to the instrument per se.Given the cultural diversity of Brazil, it would be interesting if the IHSA-Del-Prette was applied with samples from different Brazilian regions, in order to expand its normative basis (or to define the different normative standards for each region).The cross-sectional and correlational design should also be considered, as this prevents cause and effect relationships from being established.Furthermore, the fact that the sample was composed of volunteers should also be considered.Finally, the similarities and differences between the demographic indicators verified in this sample and the sample of the IHSA-Del-Prette Manual should be considered in the range of the results.Despite these limitations, the results indicate that the IHSA-Del-Prette is a useful psychological instrument for assessment and intervention studies that focus on the interpersonal repertoire of adolescents in the Brazilian context.

Figure 1 .
Figure 1.Confirmatory Factor Analysis of the Social Skills Inventory for Adolescents (IHSA-Del-Prette-Conf, N=1119) majority of the items of the IHSA-Del-Prette presented adequate factor loadings in their respective factors, which allows the comprehensive use of the separate factors.The results support the factorial validity and the reliability of the IHSA-Del-Prette (IHSA-Del-Prette-Manual sample between α = 0.68 and α = 0.82 and IHSA-Del-Prette-Conf sample between Psico (Porto Alegre), 2016; 47(3), 171-178 α = 0.70 and α = 0.82; CR between 0.70 and 0.82) This values are close to what is considered as an indicator of adequate reliability and Composite Reliability of the construct

TABLE 1
Factor loadings of the items of the IHSA-Del-Prette-Manual (N=1172) obtained through the Principal Axis Factoring method, descriptive statistics, and reliability indicators IHSA = Social Skills Inventory for Adolescents, EMP = Empathy, SCT = Self-Control, CIV = Civility, ASS = Assertiveness, AFT = Affective Approach, SOC = Social Resourcefulness, SD = Standard Deviation.