Demonstration of a mixed-methods illuminative evaluation: the case of a sports and exercise academic program
DOI:
https://doi.org/10.47197/retos.v67.110124Keywords:
Illuminative Evaluation, Mixed-Methods Research, Importance-Performance Analysis, Sports, Exercise, EvaluationAbstract
Introduction: In sports and exercise science education, ensuring that curricular components translate into practical skills is particularly critical. Higher education programs often struggle to align their instructional design with the actual learning experiences of students.
Objective: This study aimed to evaluate the alignment between the instructional system (intended program design) and the learning milieu (implemented experiences) of a sports and exercise science academic program, demonstrating an innovative mixed-methods illuminative evaluation framework.
Methodology: A three-phase sequential mixed-methods design was employed. Qualitative interviews with stakeholders were first conducted to identify key program components and themes. A survey instrument was developed based on these qualitative insights and underwent content validation. Finally, the validated survey was administered to current students and alumni, and the data were analyzed using Importance–Performance Analysis.
Results: The evaluation found that in-class program elements – including teacher preparation, learning activities, and learning environment – were regarded as highly important and were delivered effectively, indicating strong alignment between the program’s design and student experiences. In contrast, the on-the-job training component showed lower perceived importance and weaker performance.
Discussion: These findings mirrored prior research in showing that well-prepared instructors and supportive learning environments enhance student engagement and learning outcomes. The results underscored the persistent challenge of bridging theoretical instruction with practical application in higher education.
Conclusions: The study provided actionable recommendations to strengthen the OJT segment of the curriculum and introduced a novel mixed-methods evaluation model. This integrated approach offered alternative assessment of the program and can serve as a blueprint for future evaluations.
References
Angelito, G. A., & Binasoy, J. P. (2023). Evaluating the Bachelor of Public Administration Program Ob-jectives Through Importance-Performance Analysis: A Business Model Approach. International Journal of Professional Business Review: Int. J. Prof. Bus. Rev., 8(3), 1.
Behle, H. (2020). Students’ and graduates’ employability. A framework to classify and measure em-ployability gain. Policy reviews in higher education, 4(1), 105-130.
Bergsteiner, H., Avery, G. C., & Neumann, R. (2010). Kolb's experiential learning model: critique from a modelling perspective. Studies in continuing education, 32(1), 29-46.
Black, N., Johnston, D. W., Propper, C., & Shields, M. A. (2019). The effect of school sports facilities on physical activity, health and socioeconomic status in adulthood. Social Science & Medicine, 220, 120-128.
Bores-García, D., Hortigüela-Alcalá, D., Fernandez-Rio, F. J., González-Calvo, G., & Barba-Martín, R. (2021). Research on cooperative learning in physical education: Systematic review of the last five years. Research quarterly for exercise and sport, 92(1), 146-155.
Chan, R. Y. (2016). Understanding the purpose of higher education: An analysis of the economic and social benefits for completing a college degree. Journal of Education Policy, Planning and Ad-ministration, 6(5), 1-40.
Chou, H. K., Lin, I. C., Woung, L. C., & Tsai, M. T. (2012). Engagement in e-learning opportunities: An em-pirical study on patient education using expectation confirmation theory. Journal of medical systems, 36, 1697-1706.
Da Wan, C., Chapman, D. W., Zain, A. N. M., Hutcheson, S., Lee, M., & Austin, A. E. (2015). Academic cul-ture in Malaysia: Sources of satisfaction and frustration. Asia Pacific Education Review, 16, 517-526.
Dublin, R. J. H., Lo, M. S. L., & Perez, A. B. (2023). Competency level of practicum/work integrated lean-ing among CBAE students of the University of Mindanao (Doctoral dissertation).
Finlay, M. J., Tinnion, D. J., & Simpson, T. (2022). A virtual versus blended learning approach to higher education during the COVID-19 pandemic: The experiences of a sport and exercise science stu-dent cohort. Journal of hospitality, leisure, sport & tourism education, 30, 100363.
Florentino, M. S. (2023). Stakeholders’ Awareness, Understanding, and Acceptability of the Goals and Objectives of a Sports Science and Exercise Program: Basis for Formative Curriculum Evalua-tion. Slobozhanskyi Herald of Science and Sport, 27(1), 26-33.
Gökhan, A. C. A. R. (2023). The evolution of sports management: through the lens of management sci-ence, trends, and future vision. management.
Gudmundsdottir, S., & Shulman, L. (1987). Pedagogical content knowledge in social stud-ies. Scandinavian Journal of Educationl Research, 31(2), 59-70.
Gunio, M. J. (2021). Determining the influences of a hidden curriculum on students’ character devel-opment using the Illuminative Evaluation Model. Journal of Curriculum Studies Research, 3(2), 194-206.
Gutiérrez-Conejo, M., González-Rievera, M. D., & Campos-Izquierdo, A. (2024). Planning and evaluation of sports programs for people with disabilities. Retos, 53, 472–480. https://doi.org/10.47197/retos.v53.102566
Hansford, B., & Ehrich, L. C. (2006). The principalship: how significant is mentoring?. Journal of educa-tional administration, 44(1), 36-52.
Hay, K. E., & Barab, S. A. (2001). Constructivism in practice: A comparison and contrast of apprentice-ship and constructionist learning environments. The journal of the learning sciences, 10(3), 281-322.
Helyer, R. (2011). Aligning higher education with the world of work. Higher Education, Skills and Work-Based Learning, 1(2), 95-105.
Herbert, I. P., Rothwell, A. T., Glover, J. L., & Lambert, S. A. (2020). Graduate employability, employment prospects and work-readiness in the changing field of professional work. The International Journal of Management Education, 18(2), 100378.
Kuehn, D., & Hecker, I. (2018). How Do Workers Think about the Relationship between Their Job and Their Degree? Evidence from the Science and Engineering Workforce. Research Report. Urban Institute.
Kyriakides, L., Christoforou, C., & Charalambous, C. Y. (2013). What matters for student learning out-comes: A meta-analysis of studies exploring factors of effective teaching. Teaching and teacher education, 36, 143-152.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
Lombardi, D., Shipley, T. F., & Astronomy Team, Biology Team, Chemistry Team, Engineering Team, Geography Team, Geoscience Team, and Physics Team. (2021). The curious construct of active learning. Psychological Science in the Public Interest, 22(1), 8-43.
Martilla, J. A., & James, J. C. (1977). Importance-performance analysis. Journal of marketing, 41(1), 77-79.
McCarthy, M. (2016). Experiential learning theory: From theory to practice. Journal of Business & Eco-nomics Research, 14(3).
Merxhani, H., & Ibraimi, F. (2024). Enhancing The Potential of Sports Facilities at Schools. JAS-SUT Journal of Applied Sciences-SUT, 10(19-20), 47-56
Nozaleda, B. M., & Mabborang, M. H. A. (2023). Congruency of the Goals and Objectives of a Graduate Program in Physical Education: An Assessment of Stakeholders Thru Importance-Performance Analysis Framework. Slobozhanskyi Herald of Science and Sport, 27(2), 70-80.
Parlett, M., & Hamilton, D. (1972). " Evaluation as Illumination: A New Approach to the Study of Inno-vatory Programs". Occasional Paper.
Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being report-ed? Critique and recommendations. Research in nursing & health, 29(5), 489-497.
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Ap-praisal and recommendations. Research in nursing & health, 30(4), 459-467.
Resch, K., & Schrittesser, I. (2023). Using the Service-Learning approach to bridge the gap between the-ory and practice in teacher education. International Journal of Inclusive Education, 27(10), 1118-1132.
Rogan, M., & Reynolds, J. (2016). Schooling inequality, higher education and the labour market: Evi-dence from a graduate tracer study in the Eastern Cape, South Africa. Development Southern Africa, 33(3), 343-360.
Salas-Velasco, M. (2021). Mapping the (mis) match of university degrees in the graduate labor mar-ket. Journal for Labour Market Research, 55(1), 14.
Shapiro, V. B., Hudson, K. D., & Downey, M. M. (2017). Institutional expectations, opportunities, and in-terest in the professoriate: A mixed-methods examination of satisfaction among doctoral stu-dents in social work. Journal of Social Work Education, 53(3), 520-534.
Stewart, J. (2021). Understanding the Impact of Faculty Mentorship on Student Motivation to Partici-pate in Experiential Learning Within the Sports Management Curriculum: A Phenomenological Study (Doctoral dissertation, Northcentral University).
Vergara Tapia, M., Flores Bastías, M., Illanes Aguilar, L., & Poblete-Valderrama, F. (2022). Implementa-tion of the formative evaluation by students of Pedagogy in Physical Education . Retos, 43, 916–922. https://doi.org/10.47197/retos.v43i0.86558
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Bryan Nozaleda, Hans Freyzer Arao, Maricel Dayag-Tungpalan, Jasmin Saquing, Chita Ramos, Daniel Casauay
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and ensure the magazine the right to be the first publication of the work as licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of authorship of the work and the initial publication in this magazine.
- Authors can establish separate additional agreements for non-exclusive distribution of the version of the work published in the journal (eg, to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Is allowed and authors are encouraged to disseminate their work electronically (eg, in institutional repositories or on their own website) prior to and during the submission process, as it can lead to productive exchanges, as well as to a subpoena more Early and more of published work (See The Effect of Open Access) (in English).
This journal provides immediate open access to its content (BOAI, http://legacy.earlham.edu/~peters/fos/boaifaq.htm#openaccess) on the principle that making research freely available to the public supports a greater global exchange of knowledge. The authors may download the papers from the journal website, or will be provided with the PDF version of the article via e-mail.
