Email this article 
1.jpg)
SCIENCE LITERACY PROFILE OF SCIENCE EDUCATION STUDENTS IN THE CONTEXT OF APPLIED SCIENCE: ANALYSIS OF QUESTION DIMENSION PROPORTIONS AND SCORE ACHIEVEMENTS
Abstract
Purpose- This research was carried out to explain the scientific literacy profile of undergraduate science education students in the context of Applied Science, by examining (1) the proportion of science literacy question dimensions in multiple-choice test instruments, and (2) student achievement scores on each of the question dimensions of scientific literacy.
Methodology- This is a descriptive, quantitative study with a one-shot, cross-sectional measurement design. The study group included 115 pre-service science teachers from four parallel (A-D) classes, sampled using total sampling. The test contained 20 multiple-choice questions, which were pre-aligned to four dimensions of scientific literacy, including knowledge, attitudes, context, and competencies, and each of these dimensions was explicitly defined. Procedures included administering the test, scoring dichotomously (correct/incorrect), summarizing achievement by dimension using percent correct, and conducting Classical Test Theory item analysis by calculating percent correct for each item and classifying difficulty.
Finding- This study reveals that dimension achievement showed the highest performance in attitudes and moderate performance in knowledge, whereas performance in context and competencies was lower. Item difficulty analysis indicated that most items were easy (p > 0.70), a smaller portion were moderate (0.30–0.70), and none were difficult (p < 0.30), suggesting the instrument is effective for screening foundational mastery but may be less effective at differentiating mid-to-high performance levels.
Contribution- This study provides an applied-science, explicit scientific literacy profile for science education students. It delivers practical diagnostic evidence to improve the MCQ instrument by balancing the blueprint, strengthening distractors, and increasing cognitive demand to capture contextual decision-making betterKeywords
Full Text:
PDFReferences
Abdellatif, H. (2023). Test results with and without blueprinting: Psychometric analysis using the Rasch model. Educación Médica, 24(3), 100802. https://doi.org/10.1016/j.edumed.2023.100802
Abell, S. K. (2013). Handbook of Research on Science Education. Routledge. https://doi.org/10.4324/9780203824696
Al Sultan, A., Henson, H., & Lickteig, D. (2021). Assessing preservice elementary teachers’ conceptual understanding of scientific literacy. Teaching and Teacher Education, 102, 103327. https://doi.org/10.1016/j.tate.2021.103327
Arfiani, Y., Yulianti, D., Masturi, M., & Nugroho, S. E. (2025). Feasibility analysis of instruments for scientific reasoning skills for prospective science teachers using the Rasch model. Unnes Science Education Journal, 14(2), 405–416. https://doi.org/10.15294/usej.v14i2.25493
Ayu, G. N., Putri, C. A., Riyanto, A. R., & Koto, I. (2025). The Scientific Literacy Competence of Students in Indonesia and Mexico Based on PISA 2022: An International Comparative Study. TOFEDU: The Future of Education Journal, 4(5), 1033–1038. https://doi.org/10.61445/tofedu.v4i5.525
Çalik, M., & Wiyarsi, A. (2025). The effect of socio-scientific issues-based intervention studies on scientific literacy: a meta-analysis study. International Journal of Science Education, 47(3), 399–421. https://doi.org/10.1080/09500693.2024.2325382
Ceyhan, G. D., Lombardi, D., & Saribas, D. (2021). Probing into Pre-Service Science Teachers’ Practices of Scientific Evaluation and Decision-Making on Socio-scientific Issues. Journal of Science Teacher Education, 32(8), 865–889. https://doi.org/10.1080/1046560X.2021.1894762
Chauhan, G. R., Chauhan, B. R., Vaza, J. V, & Chauhan, P. R. (2023). Relationships of the number of functioning distractors with the item difficulty index and the item discrimination power in the multiple-choice questions. Cureus. https://doi.org/10.7759/cureus.42492
Cian, H. (2020). The influence of context: comparing high school students’ socioscientific reasoning by socioscientific topic. International Journal of Science Education, 42(9), 1503–1521. https://doi.org/10.1080/09500693.2020.1767316
Dabran-Zivan, S., & Baram-Tsabari, A. (2025). The Importance of Science Education, Scientific Knowledge, and Evaluation Strategies for the Successful Detection of COVID-19 Misinformation. Science Education, 109(6), 1733–1759. https://doi.org/https://doi.org/10.1002/sce.70000
Dauer, J., Gaona, V. B., & Meulenbroeks, R. (2025). Vision II Scientific Literacy Assessment Instruments for Secondary Education: A Scoping Review. International Journal of Education in Mathematics, Science and Technology.
Dewi, H. H., Damio, S. M., & Sukarno, S. (2023). Item analysis of reading comprehension questions for the English proficiency test using the Rasch model. REID (Research and Evaluation in Education), 9(1), 24–36. https://doi.org/10.21831/reid.v9i1.53514
Fauzie, M., Pada, A. U. T., & Supriatno, S. (2021). Analysis of the difficulty index of the item bank according to cognitive aspects during the Covid-19 pandemic. Jurnal Penelitian Dan Evaluasi Pendidikan, 25(2). https://doi.org/10.21831/pep.v25i2.42603
García-Carmona, A. (2025). Scientific Thinking and Critical Thinking in Science Education. Science & Education, 34(1), 227–245. https://doi.org/10.1007/s11191-023-00460-5
Gebremichael, M. W., Baraki, B., Mehari, M.-A., & Assalfew, B. (2025). Item analysis of multiple choice questions from the assessment of health sciences students, Tigray, Ethiopia. BMC Medical Education, 25(1), 441. https://doi.org/10.1186/s12909-025-06904-6
Hartono, W., Hadi, S., Rosnawati, R., & Retnawati, H. (2023). Exploration of diagnostic testing instruments: Validity, reliability, and item characteristics. Pegem Journal of Education and Instruction, 13(3). https://doi.org/10.47750/pegegog.13.03.39
Himmah, R., Jatmiko, B., Prahani, B. K., & Rizki, I. A. (2024). Science Literacy Competency Profile of Science Education Students in Understanding the Concept of Thermodynamics. IJORER : International Journal of Recent Educational Research, 5(6), 1600–1615. https://doi.org/10.46245/ijorer.v5i6.724
Högström, P., Gericke, N., Wallin, J., & Bergman, E. (2025). Teaching Socioscientific Issues: A Systematic Review. Science & Education, 34(5), 3079–3122. https://doi.org/10.1007/s11191-024-00542-y
Hu, X., & Bi, H. (2025). Exploring and validating the componential model of students’ scientific critical thinking in science education. Thinking Skills and Creativity, 55, 101695. https://doi.org/10.1016/j.tsc.2024.101695
Hüfner, S., Weirauch, K., List, F., Menthe, J., & Abels, S. (2025). Context-based science education to promote diversity, equity, and inclusion – a systematic literature review on the understanding of context in science education. Studies in Science Education, 1–41. https://doi.org/10.1080/03057267.2025.2563946
Irawan, I. D. A., Indraloka, R. M., Basri, N. A., Salmah, U., & Parno, P. (2025). Analysis of concept-understanding test items on static fluid material using the Rasch model. Jurnal Pendidikan Fisika, 13(1), 1–13. https://doi.org/10.26618/jpf.v13i1.15687
Janoušková, S., Pyskatá Rathouská, L., Žák, V., & Urválková, E. S. (2023). The scientific thinking and reasoning framework and its applicability to manufacturing and services firms in the natural sciences. Research in Science & Technological Education, 41(2), 653–674. https://doi.org/10.1080/02635143.2021.1928048
Khery, Y., Sarjan, M., Ahzan, S., & Efendi, I. (2022). Konseptualisasi literasi sains mengacu pada kerangka Sains Pisa sejak tahun 2000. Educatoria : Jurnal Ilmiah Ilmu Pendidikan, 2(4), 200–231. https://doi.org/10.36312/ejiip.v2i4.117
Lee, H. Y., Yune, S. J., Lee, S. Y., Im, S., & Kam, B. S. (2024). The impact of repeated item development training on the prediction of medical faculty members’ item difficulty index. BMC Medical Education, 24(1), 599. https://doi.org/10.1186/s12909-024-05577-x
Lin, F., & Puntambekar, S. (2025). Epistemic scaffolding: understanding and designing the support for epistemic growth in science. International Journal of Science Education, 47(6), 769–793. https://doi.org/10.1080/09500693.2024.2349973
Mellberg, S., Danielsson Friberg, A., & Nygren, T. (2025). Science education against misinformation: an educational intervention in upper-secondary schools. International Journal of Science Education, 1–33. https://doi.org/10.1080/09500693.2025.2571131
National Academies of Sciences, Engineering, and Medicine. (2025). Understanding and Addressing Misinformation About Science (K. Viswanath, T. E. Taylor, & H. G. Rhodes (eds.)). The National Academies Press. https://doi.org/10.17226/27894
Nieminen, J. H., & Ketonen, L. (2024). Epistemic agency: a link between assessment, knowledge, and society. Higher Education, 88(2), 777–794. https://doi.org/10.1007/s10734-023-01142-5
Nur, F., Makmur, F., & Fajriah, N. A. (2025). Test instrument development with local-wisdom context, assisted by the iSpring application, in the mathematics course. Matematika Dan Pembelajaran, 13(1), 43–63. https://doi.org/10.33477/mp.v13i1.9225
OECD. (2023a). Education GPS: Indonesia – Student performance (PISA 2022).
OECD. (2023b). PISA 2022 results (Volume I): The state of learning and equity in education. https://doi.org/https://doi.org/10.1787/53f23881-en
OECD. (2023c). PISA 2025 science framework.
Ohiri, S. C., & Okoye, R. O. (2023). Application of Classical Test Theory as Linear Modeling to Test Item Development and Analysis. International Research Journal of Modernization in Engineering Technology and Science, 5(10), 2152–2159. https://doi.org/10.56726/IRJMETS45379
Pawade, Y. R., Chalak, A. S., & Pawade, D. Y. (2022). Blueprints in biochemistry: Systematic assessment in undergraduate medical education. Research and Development in Medical Education, 11, 22. https://doi.org/10.34172/rdme.2022.022
Rame, G., & Hoinbala, F. R. (2024). Item analysis of multiple-choice questions: An assessment of the English summative test. Journal on Language, Education, Literature, and Culture, 2(1), 58–65. https://doi.org/10.33323/l.v2i1.45
Rashwan, N. I., Aref, S. R., Nayel, O. A., & Rizk, M. H. (2024). Postexamination item analysis of undergraduate pediatric multiple-choice questions exam: implications for developing a validated question Bank. BMC Medical Education, 24(1), 168. https://doi.org/10.1186/s12909-024-05153-3
Rathnayake, B. R. M. A. S. M., & Senevirathne, W. (2024). The Relationship between Competency in Interpreting Data and Evidence Scientifically and The Academic Performance of Students in Biological and Physical Science Streams. International Journal of Research and Scientific Innovation, XI(II), 535–544. https://doi.org/10.51244/IJRSI.2024.1102043
Sarini, P., Widodo, W., Sutoyo, S., Suardana, I. N., & Info, S. (2024). Scientific Literacy Profile of Prospective Science Teacher Students. IJORER : International Journal of Recent Educational Research.
Schüttpelz-Brauns, K. (2022). Does assessment drive learning, or should learning for professional life drive assessment? GMS Journal for Medical Education, 39.
Seema, P. V. (2024). Developing scientific literacy to promote 21st-century skills. I-Manager’s Journal on School Educational Technology.
Srisomsak, V., Sitticharoon, C., Keadkraichaiwat, I., Meethes, S., & Inpaen, I. (2025). Detection of flawed multiple-choice questions in preclinical medical education using item difficulty and discrimination indices: a six-year analysis. BMC Medical Education, 26(1), 92. https://doi.org/10.1186/s12909-025-08204-5
Steinbach, M., Eitemüller, C., Rodemer, M., & Walpuski, M. (2025). The influence of representations on task difficulty in organic chemistry: an exploration using a novel paired-items test instrument. International Journal of Science Education, 47(14), 1941–1963. https://doi.org/10.1080/09500693.2024.2378218
Tani, E. W., & Nchia, L. N. (2025). An item difficulty analysis of the 2023 CEP and FSLC examinations in the Center Region of Cameroon: Implications for valid assessment practices in basic education. International Journal of Assessment and Evaluation in Education. https://doi.org/10.70382/mejaee.v9i8.051
Tirumalamba, D. K., Y.Sridevi, D. Rao, D. M. V., & Neelima, K. (2024). Teaching Socio-Scientific Issues Through Evidence-Based Thinking Practices. Nanotechnology Perceptions.
Wahyuni, S., Barid, S. S. A. Q., Gunawan, A. S., Rusdiana, D. U., & Sya’adah, H. (2026). Development of Integrated Case-Based Learning Science Modules on Coconut Shell Waste Management to Improve Science Literacy. Jurnal Pendidikan MIPA, 27(2), 935–959. https://doi.org/10.23960/jpmipa.v27i2.pp935-959
Yanto, N., Sari, N. I., Negeri Makassar, U., & Lutra. (2025). Exploring Scientific Literacy in Science Classrooms: A Literature Study. Venn: Journal of Sustainable Innovation on Education, Mathematics and Natural Sciences.
Yudhistian, & Wulandari, T. S. H. (2025). Profile Analysis of Scientific Literacy Skills in Terms of Competencies, Knowledge, and Attitudes. Jurnal Penelitian Pendidikan IPA, 11(6), 820–832. https://doi.org/10.29303/jppipa.v11i6.11580
Zetterqvist, A., & Bach, F. (2023). Epistemic knowledge – a vital part of scientific literacy? International Journal of Science Education, 45(6), 484–501. https://doi.org/10.1080/09500693.2023.2166372
Zhang, L., Liu, X., & Feng, H. (2023). Development and validation of an instrument for assessing scientific literacy from junior to senior high school. Disciplinary and Interdisciplinary Science Education Research, 5(1), 21. https://doi.org/10.1186/s43031-023-00093-2
Zuleika, P., & Legiran. (2022). Cross-Sectional Study as Research Design in Medicine. Archives of The Medicine and Case Reports, 3(2), 256–259. https://doi.org/10.37275/amcr.v3i2.193
DOI: https://doi.org/10.36987/jes.v13i4.8969
Refbacks
- There are currently no refbacks.
Copyright (c) 2026 Arvidhea Safira Gunawan, Firdha Yusmar, Eka Puspita Kartika Sari, Ika Dewi Sumiati, Sri Wahyuni
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
