Reliability of scores from the National Exam of Upper Secondary Education (Enem)
DOI:
https://doi.org/10.15448/1980-8623.2020.2.31145Keywords:
national exam of upper secondary education (Enem), reliability, composite reliability, composite reliabilitybifactor modelAbstract
The National Exam of Upper Secondary Education (ENEM) generates a score for each domains it assess: mathematics, languages, natural sciences, and humanities. Considering the relevance of the Exam in the access to higher education and in other practical aspects of Brazilian students’ life, the present study investigates the reliability of the scores from the four domains. We used as a sample the scores of the students who participated in the 2011 edition of the Exam. The analyzes involved the estimation of the parameters of an oblique four-factor model and a bifactor model using confirmatory factor analysis, as well as the estimation of composite and omega reliability of the four domains and the general performance factor, in the case of the bifactor model. We used the 30 competences of each domain as observable variables. The results indicated high reliability only for the scores from the general factor.
Downloads
References
Beaujean, A. A. (2014). Latent variable modeling using R: A step-by-step guide. Routledge.
Cangur, S., & Ercan, I. (2015). Comparison of model fit indices used in structural equation modeling under multivariate normality. Journal of Modern Applied Statistical Methods, 14(1), 152-167. 10.22237/jmasm/1288584240
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. London, UK: Cambridge University Press. 10.1017/CBO9780511571312
Carroll, J. B. (2003). The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In H. Nyborg (Ed.), The scientific study of general intelligence: tribute to Arthur R. Jensen (pp. 1-20). Oxford, UK: Pergamon Press. 10.1016/B978-008043793-4/50036-2
Golino, H. F. (2014). ENEM: an implementation of functions to help automatic downloading, importing, cleaning and scoring of the Brazilian’s National High School Exam (ENEM). Software não-publicado. (DOI INEXISTENTE)
Golino, H. F., Gomes, C. M. A., Amantes, A., & Coelho, G. (Eds.). (2015). Psicometria contemporânea. Compreendendo os modelos Rasch. Casa do Psicólogo.
Gomes, C. M. A. (2010). Avaliando a avaliação escolar: notas escolares e inteligência fluida. Psicologia em Estudo, 15(4), 841-849. Recuperado de http://www.redalyc.org/articulo.oa?id=287123084020
Gomes, C. M. A., & Borges, O. (2008). Limite da validade de um instrumento de avaliação docente. Avaliação Psicológica, 7(3), 391-401. Recuperado de http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712008000300011&lng=pt&tlng=pt.
Gomes, C. M. A. & Borges, O. N. (2009). O ENEM é uma avaliação educacional construtivista? Um estudo de validade de construto. Estudos em Avaliação Educacional, 20(42), 73-88. 10.18222/eae204220092060
Gomes, C. M. A., Golino, H. F., & Peres. (2016). Investigando a validade estrutural das competências do ENEM: quatro domínios correlacionados ou um modelo bifatorial. Boletim na Medida (INEP-Ministério da Educação), 5(10), 33-30. Recuperado de http://portal.inep.gov.br/documents/186968/494037/BOLETIM+NA+MEDIDA+-+N%C2%BA+10/4b8e3d73-d95d-4815-866c-ac2298dff0bd?version=1.1
Gomes, C. M. A., Golino, H. F., & Peres, A. J. S. (2018). Análise da fidedignidade composta dos escores do ENEM por meio da análise fatorial de itens. European Journal of Education Studies, 5(8), 331-344. 10.5281/zenodo.2527904
Gomes, C. M. A., & Jelihovschi, E. (2019). Presenting the regression tree method and its application in a large-scale educational dataset. International Journal of Research & Method in Education. 10.1080/1743727X.2019.1654992
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2013). Multivariate data analysis. Pearson new international edition (7th ed.). Pearson Education Limited.
Hogan, T. P. (2013). Psychological testing: A practical introduction (3rd ed.). John Wiley & Sons.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (2012a). Microdados do ENEM – 2011.
Exame Nacional do Ensino Médio: Manual do Usuário. Recuperado de http://portal.inep.gov.br/web/guest/microdados
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2012b). Matriz de Referência do ENEM. Recuperado de http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2018). Edital nº 16, de 20 de Março de
Exame Nacional Do Ensino Médio - Enem 2018. Recuperado de http://download.inep.gov.br/educacao_basica/enem/edital/2018/edital_enem_2018.pdf
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2018). semTools: Useful tools for structural equation modeling (version 0.5-1). [Computer software]. Recuperado de https://CRAN.R-project.org/package=semTools
Kline, R. B. (2016). Principles and practice of Structural Equation Modeling (4th ed.). New York, USA: The Guilford Press.
McDonald, R. P. (1999). Test theory: A unified treatment. L. Erlbaum Associates
Pires, A. A. M., & Gomes, C. M. A. (2017). Three mistaken procedures in the elaboration of school exams: explicitness and discussion. PONTE International Scientific Researches Journal, 73(3), 1-14. 10.21506/j.ponte.2017.3.1
Pires, A. A. M., & Gomes, C. M. A. (2018). Proposing a method to create metacognitive school exams. European Journal of Education Studies, 5(8), 119-142. 10.5281/zenodo.2313538
R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Recuperado de http://www.R-project.org/
Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. British Journal of Mathematical and Statistical Psychology, 54, 315-323. 10.1348/000711001159582
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47 (5), 667-696. 10.1080/00273171.2012.715555
Revelle, W., & Condon, D. M. (2018). Reliability. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). The Wiley Handbook of Psychometric Testing: A multidisciplinary reference on survey, scale and test development (Vols. 1-2) (pp. 709-749). Hoboken, NJ, EUA: John Wiley & Sons.
Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26 (1), 108-116. 10.7334/psicothema2013.260
Rosseel, Y., Jorgensen, T. D., Oberski, D., Vanbrabant, J. B. L., Savalei, V., Hallquist, E. M., M., Rhemtulla, M., Katsikatsou, M., Barendse, M., Scharf, F. (2019). lavaan: Latent Variable Analysis (version 0.6-5). [Computer software]. Recuperado de https://cran.r-project.org/web/packages/lavaan/index.html
Valentini, F., Gomes, C. M. A., Muniz, M., Mecca, T. P., Laros, J. A., & Andrade, J. M. (2015). Confiabilidade dos índices fatoriais da Wais-III adaptada para a população brasileira. Psicologia: Teoria e Prática, 17(2), 123-139. 10.15348/1980-6906/psicologia.v17n2p123-139
Downloads
Published
How to Cite
Issue
Section
License
Copyright
The submission of originals to Psico implies the transfer by the authors of the right for publication. Authors retain copyright and grant the journal right of first publication. If the authors wish to include the same data into another publication, they must cite Psico as the site of original publication.
Creative Commons License
Except where otherwise specified, material published in this journal is licensed under a Creative Commons Attribution 4.0 International license, which allows unrestricted use, distribution and reproduction in any medium, provided the original publication is correctly cited.
