Flipped classroom insights after nine-year experience applying the method Maria Luisa Sein-Echaluce Departament of Applied Mathematics. University of Zaragoza. Zaragoza, Spain, https://orcid.org/0000-0002-6873-0996, mlsein@unizar.es Ángel Fidalgo-Blanco Departament of Geological and Mining Engineering. Technical University of Madrid. Madrid, Spain, https://orcid.org/0000-0003-4034- 7757, angel.fidalgo@upm.es Ana María Balbín Facultad de Educación. Pontifica Universidad Católica del Perú. Lima, Perú, https://orcid.org/0000-0002-0543-0240, abalbin@pucp.edu.pe Francisco José García-Peñalvo GRIAL Research Group, Computer Science Department, Research Institute for Educational Sciences, University of Salamanca, Salamanca, Spain, https://orcid.org/0000-0001-9987-5584, fgarcia@usal.es ABSTRACT Any new trend in Educational Innovation creates many expectations about its impact on the teaching context. This happens with the Flipped Classroom method, it presents numerous advantages: personalized learning, increased active participation, meaningful learning, autonomous learning, motivation, among others. This work collects the experience of applying the Flipped Classroom Method from 2012 up to the present. An analysis of the empirically demonstrated impact of this method on student learning is carried out. The outcome of this work is the real impact of this method in three educational environments: theoretical classes, practical classes, and teamwork competence. CCS CONCEPTS •Applied computing~Education KEYWORDS Flipped Classroom, impact indicators, teamwork, theoretical classes, practical laboratories 1 Introduction The Flipped Classroom method can be summarized with the phrase “the lesson at home and the homework in class” [1]. This idea is what has originated the name since the usual place where the “lesson” and “homework” activities are carried out is inverted. Usually, the lesson is taught in class and the homework is done at home. Thus, the order and sequence of activities is maintained. but not the place. In the traditional model, the lesson takes place in the classroom, but the students are usually in a passive attitude [2], they pay attention to the teachers and take notes. The performance of homework (practical application of the concepts studied in theory) requires higher cognitive activities and usually they do it individually at home [3]. In the Flipped classroom model, the students take the lesson at home (or outside the classroom) and do so with similar attitude to that of the traditional model, for example, they watch a video in which the teacher teaches a lesson (in the same way that he would impart it in the classroom), attend the video and take notes. However, during class, when doing homework, he does it with the rest of the students together with the teachers, taking advantage of cognitive abilities to do them in a cooperative, participatory way and under the tutelage of the teachers. For this reason, the Flipped Classroom is considered an active methodology [4]. For decades, the advantages of taking the lesson prepared to class have been remarked. Therefore, the idea of preparing the lesson at home is not typical of the Flipped Classroom. What the flipped classroom does is take the lesson out of the classroom and replace it with participatory activities and practical applications. For this reason, it is considered an active methodology [5]. The first published works [6, 7], in which the name of the methodology is defined, are based on the idea of removing the lesson from the classroom and replacing it with various practical activities. The Flipped Classroom method is a trend in educational innovation [8, 9, 10, 11]. When a method is a trend, it is in a phase of over- expectations; In other words, more impact on learning is expected than it will actually have. Sein-Echaluce, M. L., Fidalgo-Blanco, Á., & García-Peñalvo, F. J. (2021). Flipped classroom insights after nine-year experience applying the method. In M. Alier & D. Fonseca (Eds.), Proceedings TEEM’21. Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (Barcelona, Spain, October 27th – 29th, 2021) (pp. 266-270). ACM. https://doi.org/10.1145/3486011.3486458 This article includes the research work on the application of the Flipped Classroom method that has been carried out in various university subjects by the authors since the 2012-2013 academic year. This continuous work allows to have a vision of the real impact of the model of a Flipped Classroom method called Micro Flip Teaching (MFT) [12]. 2 Functional model The common pattern of the first Flipped Classroom models was based on the students, prior to attending class, studying the lesson at home (usually through a video that the teachers had recorded). In class, the first thing that was done was to answer the doubts of the students regarding the lesson learned, encourage debates, and carry out practical activities derived from said lesson. Currently, this model is widely used, but, functionally, it has a series of features that can be improved, both in terms of technology and methodology. From a technological point of view, when these models were applied, the internet was not incorporated, therefore, it limited the actions of the model: • If a student had a question related to the “lesson at home”, he had to wait until he was in the classroom to do it or ask it in a previous tutorship. • The teachers could not obtain information, in real time, on the involvement of the students in learning the lesson. From the methodological point of view, in the first works, situations that were a real barrier were detected: • For the method to work, all students would have to bring the lesson learned, or at least have studied it. This aspect is difficult to achieve, especially if the students have to learn the lesson completely. In situations where students do not usually learn the lesson in person, they will not learn it online either. • Between the phases of the lesson learned and the homework in class there was a disconnect that endangered the continuity of the method. It could be seen as two independent methods, without methodological continuity. Despite having methodological and technological deficiencies of the first models, they are still being used today. In 2013, the research group developing the method called MFT [12]. This method tries to solve the deficiencies detected so far. New processes were incorporated to both activities “lesson at home” and to the “homework in class”. Figure 1 shows the characteristic processes of the MFT method in the home lesson and Figure 2 the characteristic processes of the MFT method for homework in class. Figure 1. Characteristic processes of the MFT method in the home lesson Figure 2. Characteristic processes of the MFT method for homework in class 2.1 Lesson at home Following the workflow presented in Figure 1, we can say that: • 1A. To make up for the difficulty of taking the lesson learned, the lesson is replaced by a small part of the lesson (micro-lesson). This model has currently evolved incorporating personalized and adaptive learning methodologies. • 2A. To make up for the disconnection between the phases and the passive attitude of the students during the lesson at home, the students carry out a micro-activity of application of the concepts exposed in the micro-lesson. Micro-activities can be carried out individually and in groups. The results (3A) of the micro-activity will be used as a learning link in the homework in class. • 1P. To ensure that higher number of students carry out the lesson at home, dynamization activities are included. • 1PA. A module is created so that students can ask or reflect on any activity carried out in the lesson at home. • 2P. A new module is incorporated so that the teachers collect data on all the activities carried out in the lesson at home, in this way they will have a “photo” of both the effort of the students to carry out the activities of the lesson at home and the learning achievements obtained. 2.2 Homework in class Related to the Figure 2 process, we can say that: • Micro-activity results are incorporated as a connection and learning method. The teachers (or students) provide feedback on the results of the micro-activities, both wrong and correct results. In this way, peer learning takes place in a similar way to that which happens in case studies (both positive and negative cases). To carry out this activity, it is enough for the teachers to identify one or two correct cases and as many incorrect ones. From them, work is done in the classroom. • The analysis of the results of the different activities that the student carries out is incorporated so that the teachers have a common vision of what happened. For example, by reading the results of all the micro-activities performed, teachers can identify common mistakes, the concepts acquired, the student's application strategy, among others. Currently, in this section of the Flipped Classroom method, Learning Analytics and Big data are incorporated to facilitate the monitoring effort that teachers must develop [13, 14]. • Master-type lessons do not disappear. Micro-lessons are incorporated that complement the activities carried out in the lesson at home and as a guide for carrying out practical activities. • Dynamization activities are incorporated: debates, promotion of questions, consultations, among others. • The practical activities are maintained individually or cooperatively. 3 Context This work includes results of the application of the Flipped Classroom method in the degrees of the Polytechnic University of Madrid: Mining Engineering, Energy Engineering and Biotechnology through the subjects "Computer Science and Programming" and "Fundamentals of Programming", taught in the first course of the aforementioned degrees, respectively. The research works were applied in three different learning contexts: • Theoretical lessons. • Teamwork competence. • Laboratory classes (laboratory practices), 4 Results All the results have been obtained through the “quasi-experimental” research method where the innovation is applied in an experimental group and, in another group, the control group, the innovation is not applied. It should be noted that for the contrast to be valid, in the first place, it has been verified that the experimental and control groups are homogeneous. In other words, that one group does not have higher learning expectations than others, for example, it has been verified that the university entrance grade is homogeneous, the number of repeaters, gender, and age. It has also been verified that the perception of learning is the same in the control and experimental groups (for example, the difficulty of the subjects, the attention received by the teachers) and the hours dedicated to learning. This last point is important since in many works on Flipped Classroom it is not specified whether the experimental group has had more hours of learning than the control group. For example, the control group has only the classroom hours, while the experimental group has the classroom hours plus the lessons learned. Thus, the experimental group would receive more hours of learning, which makes the comparison unfeasible. In this sense, in our investigations the experimental and control group have had the same hours of learning. Once the homogeneity in the contrast groups and results has been demonstrated, the results have been contrasted through measurable indicators. In the case of theoretical classes, 3 sessions of 2 hours each, of the Flipped Classroom method have been held, which represents 25% of the total theoretical classes of the subject. Regarding teamwork, 5 sessions of two hours have been applied out of a total of 7, which represents 71.5% of the learning destined to the acquisition of competence. Regarding the practical classes, an average of 10 sessions of 2 hours have been applied out of a total of 12, which represents 83% of the laboratory. The results have been published in different articles and book chapters. The table is obtained from their analysis: 12 book chapters and / or international congresses and 7 articles in JCR of Web of Science indexed journals. Table 1 shows the results of the investigation. The first column collects the measured impact / the tool with which it was measured, the second column (if it has the x mark) indicates that this impact was detected in the application of the method or Flipped Classroom to the theoretical classes. The second column, if it has the mark, indicates that the impact was detected in the application of the Flipped Classroom method to the competence of teamwork. The mark in the fourth column indicates that the impact was detected in the application of the Flipped Classroom method in the practical laboratories. Table 1. Results of the study Impact / measurement tool Theoretical classes Teamwork Practical classes Retention of learning by using more cognitive abilities / “surprise” exam [12, 15], teamwork result [16], laboratory tests [17] x x x Positive assessment of the methodology / survey [12, 16, 18] x x x Students create contents [16] that can be used as a didactic resource that is demanded by students [19, 20] / contents creation x x Peer learning [19, 20] / perception of the usefulness of the content created by the students. x x Students can organize content so that organizational learning happens [20, 21] / evidence x x Individual responsibilities of people who work in a team increase [18] / questionnaire, evidence x x The Flipped Classroom method can be considered as an active methodology [12, 18] and the students transform their role into a “producer-consumer” of contents [22] / questionnaire, accomplishment activities x x x Feedback as an improvement in individual and collective learning [18, 23]and increases demand [24] / questionnaire, evidence x x Leadership action incorporates ethical and moral values [25] / survey, evidence x Interactions between students increase [12] / evidence x Debates increase [12] / evidence x x It allows the transparency of the learning process [26] / evidence x The Flipped Classroom model can be used as organizational learning (individual and collective learning) [23, 27, 28] / evidence x x The perception of complexity decreases [28] / survey x There is shared leadership [29] / evidence x The Flipped Classroom method improves indicators applicable to any field and area of knowledge [17, 30, 31] / survey x x x 5 Conclusions The Flipped Classroom method was born to be applied to theoretical classes and, in this way, make them more active in terms of student participation. The MFT Flipped Classroom method has shown that it can be applied not only to theoretical classes but also to the formation of general competencies such as teamwork and fully practical classes, which by themselves are more active than theoretical classes. In the three application models, it is highlighted that the students have a positive perception of the method, that high cognitive abilities are used, which increase debates and the method is applicable in any area of knowledge. In addition to the above, the MFT Flipped Classroom method in theoretical classes has managed to improve learning in knowledge that requires deep learning. Likewise, students increase their active participation in the classroom, for example, in debates and transforming their passive role into a content “producer - consumer” role, since they create and use those created by other classmates. Both the teamwork method and the practical classes taught in laboratories are considered contexts of active student participation. However, the application of the Flipped Classroom method has considerably improved the impact. Those already mentioned in a common way are indicated below those that have only been detected in teamwork and then, those that have been detected jointly in teamwork and practical classes. Regarding the teamwork competence, it is the one that has had the most impact since it transforms the monitoring model of the same, allowing continuous monitoring of the individual involvement of the students and the results of the group. The evidence detected exclusively for this modality is detailed below. Leadership action incorporates ethical and moral values, reduces the perception of complexity of the tasks to be developed and there is shared leadership. Interactions increase, also increasing the exchange of messages between the group. Likewise, there are other evidence of impact that are shared with its application in the practical classes and that are explained in the next paragraph. Teamwork and practical classes share a set of evidence that focuses on content creation and management, peer learning, and individual responsibility. Regarding content, more content is created, organized, and used by students, having the same bases as organizational learning. Increases peer learning (since content is jointly created, shared, and used) and increases individual responsibility, mainly because there is a transparent process of development, both in teamwork and in the collaboration of classes practices. Acknowledgments This work has been partially funded by the Spanish Government Ministry of Science and Innovation throughout the AVisSA project grant number [PID2020-118345RB-I00], the Educational Innovation Services of the Technical University of Madrid (Project MOOC Flip Teaching: Una metodología activa) and the University of Zaragoza (Project PRAUZ_19_326). The authors would like to thank the research groups EtnoEdu (https://socioconstructivismo.unizar.es), GRIAL (http://grial.usal.es) and LITI (http://www.liti.es) for their support. References [1]M. F. Massut Bocklet, and N. Rosich Sala. 2018. Los videos tutoriales, en casa: la tarea, en clase: Matemáticas con “Flipped Classroom.” EDU REVIEW / Revista Internacional de Educación y Aprendizaje. 6, 1, 43–50. DOI:10.37467/GKA-REVEDU.V6.1389 [2]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2021. An overview of passive students’ characteristics. In Proceedings of the 9th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2021) (Barcelona, Spain, October 27-29, 2021), ACM, New York, NY, USA [3]M. Marqués. 2016. Qué hay detrás de la clase al revés (flipped classroom). Actas de las XXII JENUI, 77–84. [4]H. Galindo-Dominguez. 2021. Flipped Classroom in the Educational System: Trend or Effective Pedagogical Model Compared to Other Methodologies? Educational Technology & Society 24, 3, 44-60. [5]M. Smallhorn. 2017. The flipped classroom: A learning model to increase student engagement not academic achievement. Student Success 8, 2, 43-53. DOI:10.5204/ssj.v8i2.381. [6]J. W. Baker. 2000. The 'Classroom Flip': Using Web Course Management Tools to Become the Guide by the Side. In Selected Papers from the 11th International Conference on College Teaching and Learning, J.A. Chambers Ed. Community College at Jacksonville, Jacksonville, FL: Florida, 9-17. [7]M. J. Lage, G. J. Platt, and M. Treglia. 2000. Inverting the Classroom: A Gateway to Creating an Inclusive Learning Environment. The Journal of Economic Education 31, 1, 30-43. [8]Á. Fidalgo-Blanco, and M. L. Sein-Echaluce. 2019. TEEM 19. Track 12. Educational Innovation. Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality - TEEM’19 (New York, New York, USA, 2019), 687–688. [9]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2019. ¿Pueden las tendencias de innovación educativa predecir los cambios que transformarán el modelo educativo? https://doi.org/10.5281/ZENODO.2672967 [10]V. Fornons and R. Palau. 2021. Flipped classroom en la enseñanza de las matemáticas: una revisión sistemática. Education in the Knowledge Society 22. DOI:10.14201/eks.24409. [11]C. Lázaro-Carrascosa, I. Hernán-Losada, D. Palacios-Alonso, and J. Á. Velázquez-Iturbide. 2021. Flipped Classroom and Aronson´s Puzzle: a combined evaluation in the master´s degree in pre universitary teaching. Education in the Knowledge Society 22. DOI:10.14201/eks.23617. [12]Á. Fidalgo-Blanco, M. Martinez-Nuñez, O. Borrás-Gene, and J. J. Sánchez-Medina. 2017. Micro flip teaching ‒ An innovative model to promote the active involvement of students. Computers in Human Behavior 72, 713–723. DOI:10.1016/j.chb.2016.07.060. [13]A. Álvarez-Arana, M. Villamañe-Gironés, and M. Larrañaga-Olagaray. 2020. Mejora de los procesos de evaluación mediante analítica visual del aprendizaje. Education in the Knowledge Society 21. DOI:10.14201/eks.21554. [14]F. J. García-Peñalvo. 2020. Learning Analytics as a Breakthrough in Educational Improvement. In Radical Solutions and Learning Analytics: Personalised Learning and Teaching Through Big Data, D. Burgos Ed. Springer Singapore, Singapore, 1-15. DOI:10.1007/978-981-15-4526-9_1. [15]M. L. Sein-Echaluce, Á. Fidalgo-Blanco, and F. J. García-Peñalvo. 2015. Metodología de enseñanza inversa apoyada en b-learning y gestión del conocimiento. In La Sociedad del Aprendizaje. Actas del III Congreso Internacional sobre Aprendizaje, Innovación y Competitividad. CINAIC 2015 (14-16 de Octubre de 2015, Madrid, España), Á. Fidalgo Blanco, M.L. Sein-Echaluce and F.J. García-Peñalvo Eds. Fundación General de la Universidad Politécnica de Madrid, Madrid, Spain, 464- 468. [16]F. J. García-Peñalvo, Á. Fidalgo-Blanco, M Sein-Echaluce, and M. Á. Conde. 2016. Cooperative Micro Flip Teaching. In Learning and Collaboration Technologies. Third International Conference, LCT 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings, P. Zaphiris and I. Ioannou Eds. Springer International Publishing, Switzerland, 14-24. DOI:10.1007/978-3-319-39483-1_2. [17]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2019. Indicadores de participación de los estudiantes en una metodología activa. In Actas del V Congreso Internacional sobre Aprendizaje, Innovación y Competitividad. CINAIC 2019 (9-11 de Octubre de 2019, Madrid, España), M.L. Sein-Echaluce Lacleta, Á. Fidalgo-Blanco and F.J. García-Peñalvo Eds. Servicio de Publicaciones Universidad de Zaragoza, Zaragoza, Spain, 596-600. DOI:10.26754/CINAIC.2019.0121. [18]F. J. García-Peñalvo, Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and M. Sánchez-Canales. 2019. Active Peer-Based Flip Teaching: An Active Methodology Based on RT-CICLO. In Innovative Trends in Flipped Teaching and Adaptive Learning, M.L. Sein-Echaluce, Á. Fidalgo-Blanco and F.J. García-Peñalvo Eds. IGI Global, Hershey, PA, USA, 1-16. DOI:10.4018/978-1-5225-8142-0.ch001. [19]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2017. APFT: Active Peer-Based Flip Teaching. In Fifth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’17) (Cádiz, Spain, October 18-20, 2017) J.M. Dodero, M.S. Ibarra Sáiz and I. Ruiz Rube Eds. ACM, New York, NY, USA, Article 83. DOI:10.1145/3144826.3145433. [20]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2018. Micro flip teaching with collective intelligence. In Learning and Collaboration Technologies. Design, Development and Technological Innovation. 5th International Conference, LCT 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part I, P. Zaphiris and A. Ioannou Eds. Springer, Cham, Switzerland, 400-415. DOI:10.1007/978-3-319-91743-6_30. [21]Á. Fidalgo-Blanco, M. Sánchez-Canales, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2018. Ontological Search for Academic Resources. In Proceedings TEEM’18. Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality (Salamanca, Spain, October 24th-26th, 2018), F.J. García-Peñalvo Ed. ACM, New York, NY, USA, 788-793. DOI:10.1145/3284179.3284315. [22]M. L. Sein-Echaluce, Á. Fidalgo-Blanco, and J. Esteban-Escaño. 2019. Technological ecosystems and ontologies for an educational model based on Web 3.0. Universal Access in the Information Society 18, 3, 645-658. DOI:10.1007/s10209-019-00684-9. [23]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2019. The Neuro-Subject: A Living Entity with Learnability. In Learning and Collaboration Technologies. Design, Experiences. 6th International Conference, LCT 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019. Proceedings, Part I, P. Zaphiris and A. Ioannou Eds. Springer Nature, Cham, Switzerland, 127-141. DOI:10.1007/978-3-030-21814- 0_11. [24]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2019. Enhancing the main characteristics of active methodologies: A case with Micro Flip Teaching and Teamwork. International Journal of Engineering Education (IJEE) 35, 1B, 397-408. [25]M. L. Sein-Echaluce, Á. Fidalgo-Blanco, J. Esteban-Escaño, F. J. García-Peñalvo, and M. Á. Conde-González. 2018. Using learning analytics to detect authentic leadership characteristics at engineering degrees. International Journal of Engineering Education 34, 3, 851-864. [26]M. L. Sein-Echaluce, Á. Fidalgo-Blanco, F. J. García-Peñalvo, and D. Fonseca. 2021. Impact of Transparency in the Teamwork Development through Cloud Computing. Applied Sciences 11, 9. DOI:10.3390/app11093887. [27]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2017. Inteligencia Colectiva en el aula. Un paradigma cooperativo. In La innovación docente como misión del profesorado. Actas del IV Congreso Internacional sobre Aprendizaje, Innovación y Competitividad. CINAIC 2017 (4-6 de Octubre de 2017, Zaragoza, España), M.L. Sein-Echaluce Lacleta, Á. Fidalgo-Blanco and F.J. García-Peñalvo Eds. Servicio de Publicaciones Universidad de Zaragoza, Zaragoza, España, 599-603. DOI:10.26754/CINAIC.2017.000001_125. [28]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2018. Ontological flip teaching: A flip teaching model based on knowledge management. Universal Access in the Information Society 17, 3, 475–489. DOI:10.1007/s10209-017-0556-6. [29]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, F. J. García-Peñalvo, and M. Á. Conde. 2015. Using Learning Analytics to improve teamwork assessment. Computers in Human Behavior 47, 149-156. DOI:10.1016/j.chb.2014.11.050. [30]Á. Fidalgo-Blanco, M. L. Sein-Echaluce, and F. J. García-Peñalvo. 2019. Impact indicators of educational innovations based on active methodologies. In TEEM’19 Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality (Leon, Spain, October 16th-18th, 2019), M.Á. Conde-González, F.J. Rodríguez-Sedano, C. Fernández-Llamas and F.J. García-Peñalvo Eds. ACM, New York, NY, USA, 763-769. DOI:10.1145/3362789.3362894. [31]M. L. Sein-Echaluce, Á. Fidalgo-Blanco, and F. J. García-Peñalvo. 2019. Diseño de un proyecto de innovación educativa docente a partir de indicadores transferibles entre distintos contextos. In Actas del V Congreso Internacional sobre Aprendizaje, Innovación y Competitividad. CINAIC 2019 (9-11 de Octubre de 2019, Madrid, España), M.L. Sein-Echaluce Lacleta, Á. Fidalgo-Blanco and F.J. García-Peñalvo Eds. Servicio de Publicaciones Universidad de Zaragoza, Zaragoza, Spain, 617-622. DOI:10.26754/CINAIC.2019.0126