New Approach to Design and Assess Metaverse Environments for Improving Learning Processes in Higher Education: The Case of Architectural Construction and Rehabilitation

Abstract

This research paper has defined and applied a new approach to develop and optimize augmented reality-based metaverse environments for learning construction and rehabilitation in architectural studies. This research paper is part of a broader project that aims to improve the learning process of architectural students by approaching construction and rehabilitation realities to the classroom in a feasible and pedagogical way. The approach has been applied successfully to develop a new environment with augmented reality that allows teachers to carry out activities using computers, tablets, cellphones and smartphone headsets in classrooms. Students’ satisfaction regarding the new environment is high, though there is room for improvement. The assessment of this new environment has involved both questionnaires and a multicriteria decision-making method which have holistically evaluated the new proposal by achieving complementary results. The main advantages of this new environment are in terms of application and inclusion, while its weaknesses will be solved in future steps that will introduce both mixed reality, to enlarge students’ perception and comprehension, as well as neuroeducation, to diminish students’ potential annoyances and dissatisfaction.

1. Introduction

In numerous countries, teaching construction is one of the current teaching challenges in architecture and civil engineering. Students often find it difficult to understand how architectural elements (structure, foundations, façades, etc.) materialize constructively, since they are often hidden in current buildings. Most related learning materials, which show these elements using images and texts, have proven to be inadequate for architecture students [1], so site visits have become very important. However, site visits to building works have several drawbacks, such as security risks that must be covered with specific insurance or transport to places that are not always accessible by public transport [2].

This can be improved by the application of digitization in the world of architecture education, favored by a context where students are digital natives, with the incorporation of augmented reality visits to works in a metaverse environment that can complement traditional media. Metaverses, immersed and interactive environments [3] represent a new paradigm based on technologies such as augmented reality (AR), mirror worlds (MWs), virtual worlds (VWs) and lifelogging [3,4].

1.1. Literature Review

Metaverse is a term from the end of the 20th century that has been and is still being redefined at present. As Davy Tsz Kit (2022) states “educational researchers have used this term to describe how learners engage and socialize in the metaverse using digital technologies such as augmented reality (AR), virtual reality (VR) and mixed reality (MR)” [5,6,7] (the complete list of abbreviations used in the text can be found in Appendix A). This research paper considers metaverse from a general point of view, considering it as an environment for students to engage by means of digital technologies such as AR [7]. The application of AR can additionally provide students with the opportunity to engage in the real-world context, which supersedes teaching methods that are confined and restricted to traditional classroom learning [8].

The use of the metaverse in education is expanding rapidly, despite the fact that scientific articles that describe it are still few. In 2022, Inceoglu et al. [9] reviewed scholarly studies scanned from the Web of Science and Scopus databases, identifying 128 publications by searching the word metaverse, though only seven of these papers were related to the field of education. Most of these studies focused on VWs, and quite a few focused on ARs, but very few focused on MWs or life logs.

The field in which metaverse application in education has been studied the most is in higher education in general (62.9%) [10]. Regarding specific disciplines, 53% of studies focus on natural sciences, mathematics and engineering, while the least number of projects focused on the fields of architecture or civil Engineering.

Much has been written, specifically, about virtual reality (VR) focusing on teaching, due to its dynamic characteristics for the teaching and learning processes, its updated and organized contents as well as its easy interaction between the teacher and the student. Virtues in the use of VR are highlighted, such as its dynamism and the promotion of critical thinking, socializing participation, collaboration, interaction and communication when learning [11].

There have been interesting reviews on the application of AR in education in general, such as those by Chen et al. [12] or Hajirasouli et al. [8], and papers discussing the use of AR experiences to develop students’ higher-order thinking capabilities [13].

It has been proved that, generally, learning in the metaverse has multiple advantages, such as its potential to explain complex or abstract concepts, performing rich immersive educational experiences for learning by doing and interacting in a dynamic and amusing way [14] as well as providing social learning and long-distance exchanging of information [9,15], which are related to the characteristics required for transnational education [16]. Learning in the metaverse allows unrestricted time and space, personalization and prevention of academic misconduct through blockchain technology, and constitutes a teaching revolution as, with the help of tools for analyzing big data and educational behaviors, teachers can assign students complex, personalized homework that can help them become more independent [17]. Metaverse environments provide an integrated holistic learning experience and development of responsible use values to students [18,19].

Some negative aspects of teaching in the metaverse are the great consumption of time and a risk of excessive gamification. Risks may also exist on the knowledge level for the use of information technologies by students and teachers or issues related to computer security [9,19]. It should also be considered that instructional designers and instructors who want to use the metaverse for education should adequately understand the technical characteristics of each type of metaverse and design classes so that they can solve projects collaboratively and creatively [20]. Moreover, introducing the metaverse in teaching must also consider the loss of tactile contact between student and teacher, in addition to the great technological advances, not always available to everyone and not always within the reach of all educational centers. Another important limitation is that most people do not enjoy prolonged experiences within virtual worlds [21].

In the fields of architecture and civil engineering education, mobile learning, VR and AR have been used with some frequency in the last decade [22,23]. Of note is the research developed by Abdelhameed [24], who used VR Studio software to increase students’ awareness during designing processes, in terms of the structural properties and component assembly of the structural system. Bashabsheh et al. [25] developed and applied a software program that presented a 4D model (3D model and time dimension) for certain building construction phases, using VR technology to carry out immersive and non-immersive virtual reality experiences for users. Bustamante Escapa [26] studied the application of immersive VR in architecture studies, arguing that spatial perception is crucial in the field of architecture, giving a special consideration to the experience of space through multiple senses, so that the qualities that are not considered in drawing, such as color, texture, lighting and sounds, become fundamental, thus allowing the development of a spatial sensitivity. Diao et al. [27] reported that the main advantage of AR in architectural and civil engineering (ACE) education curricula was being an effective tool in traditional pedagogical settings to improve students’ learning interest, academic performance, motivation, satisfaction and participation in the course. They also stated that AR challenges in ACE education are the difficulty to generate AR model content and insufficient study time or too much course content.

Hajirasouli and Banihashemi analyzed the implications of AR on architectural and construction students’ skills as well as AR implementation effects on pedagogical, curricula, technological issues and contents of these higher education experiences. This study aimed to assist teachers and future generations on issues related to the rapid evolution of the digital industry [8].

Other application cases for architecture students include representing historical buildings in VR [28], facilitating the investigation of architectural details through an interactive and immersive medium such as VR [29], using VR and AR technologies to motivate students in graphic expression subjects [30], and employing VR and AR as methods of presenting architectural projects [31]. Several of these students pointed out smartphone and standalone headsets as the best tools, and noted how AR technology can help them to succeed in their postgraduate qualifications. Thus, it is expected that by combining an attractive technology, and by the user–machine interaction that involves AR, students feel more motivated, their graphic competences and space skills are increased in shorter learning periods and their academic performance is highly improved [32].

It is necessary to consider that three types of AR teaching methods exist in ACE education. The first type emphasizes the roles of students and their interaction. The second type emphasizes the locations; therefore, students leave the traditional classroom and go to a given location outdoors to use AR to complete their studies [2,33]. The third type emphasizes the task, and students use AR to complete learning tasks in the form of a game or direct communication [27].

Regarding the different technologies on which AR is mostly based, these include platforms, interfaces and tools. Platforms can be mobile-based, station-based, PC-based or smartphone-based, wired or wireless, and can employ a head-mounted device (HMD) or hand-held device (HHD).

Of the ACE educational research projects carried out to date, there exist very few cases using AR technologies to improve the comprehension of architecture and construction. Most of the projects deal with architectural design, making new proposals on existing environments [34] or using augmented reality to allow urban planning experts to move around a city’s streets and project virtual, three-dimensional buildings, allowing researchers to see the real city and virtual buildings at the same time [35], and even incorporating human sensations and feelings into architectural designs [36].

Very few research papers propose to improve our understanding of the construction aspects of architecture. These studies are based on the application of layers containing virtual objects overlaid on the existing reality that has been photographed in different phases of construction works. These are shown through edited videos of a limited duration [37].

1.2. Definition of the Problem

As explained at the beginning of this article, the main objective of the research presented here is to improve the learning process regarding construction and rehabilitation in architectural studies through the development of AR-based metaverse environments.

This research paper considers the metaverse from a general point of view, as an environment for students to engage in learning by means of digital technologies such as AR, VR and MR [5]. This research is supported by the application of AR technology and can provide students with the opportunity to engage in a real-world context, which supersedes teaching methods that are confined and restricted to traditional classroom learning [8].

AR technology can be interpreted in different ways and, therefore, delimiting this concept is a requirement. A common definition is that of Fazel and Izadi [38], who understand AR as a line between the virtual and real-world that overlays supplementary and additional virtual information over real objects and scenes and, therefore, enhances our perception of the real world. It is also explained as a way to allow a participant’s clear view of the real world [1], but it can also be understood as methods and ways to submerge users in educational simulations where they can practice acquired theoretical knowledge and concepts by interacting with the environment [8]; further, AR can allow one to view the real world with the addition of external information intended to provide a new understanding to what is being seen [39]. The latter is the definition that best suits the present research project.

This article proposes to carry out AR visits using a platform equally valid for computers and mobiles [8]. These are visits to a real environment that has been documented in all its phases and no longer exists once the building is finished. In addition, users can freely access and move along the construction site, in time and space, and can consult data about the objects observed. However, they cannot socialize or interact with other students, a feature that is common to several types of metaverses.

Another difference between previous experiences in the same field and the one shown here is that we attempt to improve the users’ degree of comprehension. While previous research projects assess the increase in the students’ degree of attention and motivation, as well as users’ improved learning process compared to theoretical classes, this research evaluates the increase in learning by comparing AR visits to onsite visits. These differences are the main novelty of this research, as they differ from the augmented reality research applied to learning architecture developed until present.

The research questions that derive from the objectives and articulate the research are the following: (a) Can metaverse environments reproduce the built reality and substitute for onsite visits? (b) What are the pros and cons of these environments?

This article is the main result of a competitive, funded research project developed at the Universitat Politècnica de Catalunya. Section 2 presents the developed methodology, and explains the case study, Section 3 shows the results, Section 4 discusses them and, finally, conclusions and future projects are drawn in Section 5.

2. Methodology

The methodology followed consists in two consecutive cyclic phases that include design, application and assessment steps. These phases rely on an initial analysis of the courses involved in the project. Figure 1 presents the main steps, actors, methods, tools, outcomes and outputs of this initial analysis and the subsequent two main phases.

During the initial analysis, the teaching team for the courses involved studied the strengths and weaknesses of these courses. This analysis relied on past theories in higher education such as Bloom’s taxonomy revised by Anderson [40], the learning principles of Chickering and Gamson [41] and neuroeducation [42]. In the analysis of the courses, we sought to determine their strengths, weaknesses, opportunities and threats (using the SWOT technique) [43].

The first-phase designs tested and evaluated a first version for the learning platform based on 360-degree panoramas. Both the design and the assessment were carried out by the teaching team and external experts, who brought knowledge about the courses and expertise on the software tools, respectively. In this first phase, the external experts were professionals, specialized in augmented reality, from outside the university, chosen by the teaching team relying on their initial analysis. Students from higher courses within the architecture program used this first design and provided feedback. This feedback was collected using questionnaires which were defined by the research team. The first questionnaire had 13 questions, was designed in order to detect the strengths and weaknesses of the platform and was divided into three blocks: (i) a first block related to ethics, asking for permission to use the answers for scientific researches; (ii) a second block to contextualize the students’ level of construction knowledge, meaning the level of the construction subject followed or the fact of having been on a construction site before this AR experience; (iii) the third and final block dealt with the easiness of the platform’s use and its settings, advantages and disadvantages and a comparison to onsite visits (Appendix B shows the full fist questionnaire). A list of suggestions for improving the mock-up of the learning environment was the outcome of this first phase.

In the second phase, we redesigned the first version of the 360-degree learning platform by replicating similar steps from the former phase, keeping the previous software tools and external experts group apart from another group of experts during the assessment phase, as explained in the following paragraphs. The teaching team and external experts collaborated again to design a more complete learning environment that took into account the previous trial and assessment. The software tools were changed if necessary, and the resulting learning environment was improved by increasing and adapting its functions to the courses’ needs. Moreover, smartphone headsets were added in order to improve students’ immersion and experience during the digital visit. This phase did not incorporate the environment optimization, as this was beyond the scope of this research project.

The second trial was carried out by students from the construction and rehabilitation courses who will use these learning environments. These students provided feedback as real users. This feedback was collected again via questionnaires. This second-phase questionnaire also had 13 questions and 3 similar blocks, but there were 3 substantial differences: (i) the suppression of questions related to the level of construction level, not because it was a minor issue, but because, to facilitate the reading of results, the questionnaires were separated by courses; (ii) the introduction of specific questions about the use of the smartphone headsets; and (iii) the fact that students had to carry out a specific activity, in addition to the possibility of free navigation, and answer 2 questions related to the effects of this activity on learning (Appendix C presents the full second questionnaire).

This final step combined the analysis of the questionnaire with a sustainability assessment. This sustainability assessment used a tool previously developed by the researchers to assess learning alternatives for practical sessions [44], and was successfully applied to one case study. This tool evaluates learning alternatives using a MIVES-Delphi tool. Delphi is a method for structured research that enables researchers to obtain highly reliable data using strategically prepared questionnaires sent to certified experts [45]. MIVES, an acronym for integrated value model for sustainability assessment, is a multicriteria decision-making method from the Spanish Modelo Integrado de Valor para una Evaluación Sostenible (MIVES) [46]. The resulting MIVES-Delphi tool is able to quantitatively assess learning alternatives and integrate the results in global and partial indexes. The agility of this assessment process and the ability to minimize the possible subjectivity related to some indicators were also considered when choosing this tool [47]. This specific tool has a tree of requirements that is composed of 4 requirements, 8 criteria and 20 indicators, which were defined and weighted by external experts following Delphi [44]. Therefore, the second phase had two groups of external experts: (i) the group involved in improving the software that was chosen following the same criteria as in the previous phase; and (ii) the external experts that defined the MIVES tool, who were chosen following Delphi [44]. The four MIVES requirements are applicability and the three broadly accepted sustainability branches (economic, environmental and social) [48]. This tool was already validated, in the aforementioned article, for the assessment of learning activities for practical sessions such as challenge-based learning (CBL), team-based learning (TBL), flipped classrooms, project-based learning (PBL), reflective learning, industry–community projects, videos of real cases, case studies, gamification activities, interdisciplinary activities, problem-solving activities, storytelling, real material practices, hands-on activities, role play and site visits. The main outcomes of this final phase were, obviously, the improvement of the 360-degree learning environment as well as a guideline for its application.

Research Setting

This project was applied to courses about construction and rehabilitation in architecture that are part of the architectural higher education program at Barcelona Architecture School at the Universitat Politècnica de Catalunya. Specifically, the courses included: Technical Bases (first year), Construction I (secnd year), Construction II (third year), Construction IV (fifth year), and Innovations in Architecture and Technology (IAT) (third–fourth year). It must be highlighted at this point that the teaching team has extensive experience in the field of teaching innovation and were already applying good practices that had become, in fact, the subject’s strengths, such as the use of cooperative learning activities [49,50,51], gamification techniques [52] or flipped classroom methodologies [53]. Table 1 summarizes the main information these courses include.

It must be taken into account that part of the relevance of this study relies in the fact that the field of architectural construction is of special importance in the training of Spanish architects since, unlike differences that exist in other countries in the world, Spanish architects have complete authority, competencies and legal responsibility over the work, its construction, its structure, its facilities and, of course, its design [54].

Some details regarding the studied subjects that should be stated:

• Technical Bases aims for students to learn the foundations for all the technical knowledge for their whole degree. The main objective this course has is that students understand architecture as the simultaneous fulfillment of all its principles: suitability to space, suitability to the environment, integrity, sustainable production and aesthetic convenience.

• During Construction I, students come to know and understand the main construction materials and techniques through identifying the typological characteristics of the usual construction systems incorporated into structures, façades and roofs. This course also presents solutions for the singular points of construction systems, especially for residential buildings, and proposes critical evaluations for those systems in order to guide and justify project decisions.

• Construction II focuses on the structural part of a building, from its foundations and retaining walls to its columns, load-bearing walls, beams, steel and reinforced concrete, wood and masonry slabs. This course aims to improve students’ knowledge about the materiality of these parts in a building project, as well as to provide students with resources to be able to face the transition from design to the real architectural object.

• Construction IV deals with construction for rehabilitation, so it provides the technical knowledge to diagnose, repair and improve existing buildings. Specifically, this course instructs student on how to apply the methods and resources required to diagnose the physical condition of buildings and evaluate their conditions of safety and habitability and to, subsequently, select the suitable techniques and resources to intervene in existing buildings.

• Innovations in Architecture and Technology focuses on novelties and innovations in the world of architecture and technology, from its design to its sustainability assessment process, including prototyping, materials, production, offsite technologies, onsite techniques and rehabilitation of buildings.

3. Results

This research relied on several former studies regarding the courses involved. For example, in the case of Construction I [55] and Construction II [56], professors collected data from questionnaires and informal encounters during several years [2,57]. From these, the following SWOT conclusions were obtained:

• Strengths: (i) the complete and detailed learning materials incorporated into these courses; (ii) the teaching experience and professional expertise of the courses’ professors and teachers.

• Weaknesses: (i) the students’ low level of engagement and participation; (ii) the low level of digitalization of the course material, and (iii) the low capacity of the courses to place students into contact with real construction and rehabilitation works. This contact usually only consisted of pictures, videos and few site visits, due to the aforementioned difficulties to organize site visits, because of their risks and resource availability.

• Opportunities: (i) new available digitization software tools such as 360-degree panorama platforms that are being introduced now to the construction sector to ease several phases such as preparing models for buildings, for the real estate market, etc.; (ii) newly available digitalization hardware such as smartphone headsets.

• Threats: (i) difficulties for university professors and teachers to learn and prepare materials with the aforementioned new software and hardware due to time and budget limitations; (ii) similar difficulties for applying them.

The teaching team of the aforementioned courses worked with external experts on software tool development for architecture learning [1,33,58].

3.1. First Phase

In this phase, the software tool PANO2VR [59] and the plugin “buildings 360°” [1] were used to develop the first version of metaverse environments for students to digitally visit construction and rehabilitation sites. Figure 2 shows a screenshot of one moment during a visit [60]. For the smartphone headsets, students used their mobile phones after verifying that their devices had the applications Google cardboard and VR Google services [61]. As previously explained, questionnaires were carried out to obtain data on the usefulness of augmented reality site visits and on the advantages and disadvantages in relation to onsite visits. The survey was answered by 110 students—none from the higher course levels—and it must be highlighted that there were no major differences in the students’ responses depending on the level of studies they were at.

The qualities of augmented reality visits that were voted most advantageous were being able to move through different phases of the building process, accessing these visits at any time of the day and any day of the week, avoiding traveling to the work sites and the improved accessibility for people with reduced mobility.

The disadvantages pointed out were the difficulty to really grasp the material implications of the building construction works, as well as to understand the scale and materiality of what was seen. Figure 3, Figure 4, Figure 5 and Figure 6 display the main advantages and disadvantages pointed out in both the first and second phases.

There was an absolute consensus about which format of visit should have priority: the onsite one. The augmented reality visit is much more useful after an onsite visit, to remember certain aspects or to zoom in on specific details without having to return to the site. In addition, more importantly, the augmented reality visit was useful in order to be able to access the different phases of the photographed works at any time during the same session.

3.2. Second Phase

This specific phase aimed to obtain data on the improvement gained in learning and understanding during virtual visits when using smartphone headsets, and, as in the previous phase, to detect the advantages and disadvantages in relation to onsite visits. Figure 7 presents the resulting metaverse environment, which can be accessed through an open website containing different construction and rehabilitation processes [62]. Figure 8 and Figure 9 depict different moments of this phase.

Up to 108 students answered. Most of them had been at least once to a construction site, although they had not been to the analyzed construction site before. Again, they highlighted the same three aspects: the fact of being able to see different phases of the construction works during the same session without traveling to the site, the improved accessibility for people with reduced mobility and the possibility to visit the platform at any time. Regarding the disadvantages, they pointed out the difficulties they had in understanding what a construction site is, what the materiality and dimensions of construction elements are, the difficulties in receiving information through senses other than sight and even the loss of interaction between the different construction work agents.

As in the first-phase questionnaire, the onsite visit was considered to be necessary before any virtual visit; the virtual visit would always be complementary.

The visualization through smartphone headsets was not greatly considered due to a lack of comfort using the device related to dizziness suffered after sustained use, technical connection problems, the specific adaptation of headsets to each student and difficulties managing and navigating within the online platform. These discomforts have already been reported in previous research papers [63].

In short, adding low-quality smartphone headsets does not provide an improvement over a visit using a computer. Table 2 shows the main similarities and differences between the two phases as well their statistical reliability [64,65,66,67].

For the MIVES assessment, the following alternatives were considered:

• A31—web-based 360-degree environment: students grouped in three carry out practical activities using the web-based solution.

• A32—web-based and smartphone headset environment: students grouped in three carry out practical activities using the web-based solution combined with the solution for smartphone headsets. Table 3 presents the results from analyzing the sustainability of these alternatives.

These results are useful to improve these alternatives in the future because they point out their strengths and weaknesses. The resulting global sustainability index (GSI) indicates that these learning alternatives have significant room for improvement, especially regarding applicability and environmental issues. In any case, considering other alternatives assessed using the same tool, the measurements of these GSIs achieved almost the average (0.65) or the median (0.66) [44]. Thus, in terms of performance, these are common performing learning solutions.

Table 4 presents the evaluation of the applicability indicators in detail. At present, the main problem of these new alternatives is the difficulty of transferring them, especially to other teachers, because they are innovative technologies, most of which involve teachers’ new learning and recycling. These alternatives require adaptation before being transferred to other disciplines, because they are specialized in construction and rehabilitation in architecture.

The new alternatives perform better than well in most economic indicators, as shown in Table 4. Both A31 and A32 are excellent alternatives regarding logistic issues or students’ dedication outside class and perform well in dedicated time in class. Costs can be borne by an institution’s sources or university funds, which would change if the latest models of smartphone headsets were used. On the other hand, at present, the implementation of these alternatives requires a lot of time outside class by the teachers, who must prepare and learn about them.

Regarding environmental issues, this assessment tool focuses on energy consumption and waste generation by these learning solutions. In this regard, these solutions require the use of hardware—both computers and cellphones—which consumes power. Table 4 shows the heterogenous performance of the ten social indicators. The novel learning alternatives perform excellently as innovation for the university learning processes. Similarly, solution A31 is an excellent learning alternative to promote students’ interest and participation, while A32 has a good role too. These alternatives are also good at enhancing feedback to students’ time and autonomous work. On the other hand, these alternatives should improve the encouragement of cooperative work, the incorporation of different roles and talents in learning as well as learning outcomes, among other factors. These social indicators show that from A31 to A32, there is an improvement in different indicators such as I11, I12, and I14-I16.

4. Discussion

The results of both phases directly answer the two previously introduced research questions:

• (a). Can metaverse environments reproduce the built reality and substitute for onsite visits?

• (b). Which are the pros and cons of these environments?

The vast majority of students surveyed said that AR visits and onsite visits are complementary and that at least one real visit is necessary. This real visit would improve the perception of the materiality and dimensions of the construction elements, issues detected in the second phase of the results.

In this second phase, comparing the results from the questionnaire and the MIVES tool, the researchers found that some issues coincided, while other aspects were complementary, when assessing the new learning alternatives. Both the questionnaire and MIVES detected that the new version has positive strengths in terms of application: the ability to show different moments in building and rehabilitation works for all students without requirements in terms of mobility, fewer safety risks and accessibility at any time for students. These detected advantages coincide with the scientific literature that highlights the advantages of learning in the metaverse for facilitating distance learning [9,15]. In this regard, MIVES has confirmed these alternatives’ strengths in terms of economic feasibility.

On the one hand, the questionnaires have been able to detect more specific advantages—such as the capacity of these learning alternatives to be accessible at any time—and problems—such as feelings and physical effects on students, detected previously in the scientific literature [21,63], and technical problems. Another important shortcoming detected by the questionnaires is that these AR alternatives, compared to a real visit to the construction site, are less able to transmit essential issues such as: (i) construction elements’ materiality, (ii) building components’ organoleptic properties or (iii) onsite building professionals’ advice.

On the other hand, MIVES found some important implementation weaknesses regarding the partial maturity of these learning alternatives. For example, these alternatives require high dedication outside class by the teaching teams at present, and they are still difficult to transfer among teachers, schools and universities. These teaching alternatives also require improvements in encouraging teamwork, diversity in ways of learning, learning cognition and affect outcomes.

5. Conclusions

The main novelty of this research project is the fact that it proposes and develops augmented reality visits to an environment that already exists, has been documented in all its phases and can never be seen again once the construction of the building is finished, for learning architectural construction and rehabilitation in the university. This fact is important because, to the best of the author’s knowledge, the research carried out so far on the field of metaverse use for architecture education is mainly based on the study of virtual worlds that do not exist yet (VR), in order to consider their feasibility, beauty, etc. Further, as documented in the introduction, AR has been used very infrequently for the improvement of architectural learning, and mainly to figure out the result of changing the existing buildings and cities.

This novel metaverse project experience implies moving virtually to an existing site to be able to examine it in detail throughout its evolution, without time or movement limits, on any day at any time, without using transportation to get to the real site, without risks, etc., as proved in the results and discussion sections.

The significance of this is crucial for architecture learning, since teachers and professors involved consider that a visit to construction works is probably the main method to improve understanding and learning, but that physical visits are impossible to carry out every day for each class.

The methods used in this research project to assess the application of this new augmented reality environment have proved useful. Both questionnaires and the MIVES-Delphi method have provided useful insights about strengths and weaknesses of the environment in a complementary manner. Both methods have detected that the new version has positive strengths in terms of application. The questionnaires were able to detect more specific advantages and problems. On the other hand, MIVES found some important implementation weaknesses regarding the partial maturity of these learning alternatives.

The aforementioned limitations of the present augmented reality platform will be solved in future research steps, which will also work on its optimization. The weaknesses related to this research experience are expected to be solved with the incorporation of other technologies, such as cloud point combined with 360-degree panoramas with new functions such as taking measurements of objects. Furthermore, the forthcoming incorporation of mixed reality will also enable students to better comprehend these environments. These future steps will involve neuroscience experts to precisely control any potential effects from these applications on students’ brains.

Author Contributions

All authors carried out this research project and the investigation. B.O., C.C., J.M. and O.P. led this research project, conceptualized the methodology, and executed the writing. B.O. and C.C. carried out the analysis. Supervision was carried out by O.P. All authors have read and agreed to the published version of the manuscript.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Most of this project data is available in the article and Appendix A, Appendix B and Appendix C More data can be provided by asking the main author.

Acknowledgments

We are grateful to the construction students, teaching team and external experts. We thank the project IE22.0307 of the School of Architecture of Madrid (ETSAM) entitled “Buildings 360 (Integration of 360 approaches in construction learning)” for collaboration in this project. We also acknowledge the specialized guidance of ColomaArmengol Company and VIMAC lab from UPC.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. General framework for this research project. Legend: G1 indicates group 1; G2 indicates group 2; SWOT indicates strengths, weaknesses, opportunities and threats technique; AR indicates augmented reality; MIVES indicates the integrated value model for sustainability assessment.

Figure 2. Screenshot of one moment during a visit using the first phase metaverse environment.

Figure 3. First-phase results of the platform advantages showing the differences between first-level students and medium-level students.

Figure 4. First-phase results of the platform disadvantages showing the differences between first-level students and medium-level students.

Figure 5. Second-phase results of the platform advantages showing the differences between first-level students and medium-level students.

Figure 6. Second-phase results of the platform disadvantages showing the differences between first-level students and medium-level students.

Figure 7. Screenshot of the website prepared during the second phase.

Figure 8. Learning activity during the second phase involving smartphone headsets.

Figure 9. Learning activity during the second phase involving the website and smartphone headsets.

Table 1

Course information.

	Technical Bases	Construction I	Construction II	Construction IV	IAT
Eligibility	Compulsory	Compulsory	Compulsory	Compulsory	Eligible
Curricular block	First	Second	Second	Third	N/A
Degree year	1	2	3	5	3–5
ECTS	6			7.5	3
Class hours/week	5			6	3
Lecture/workshop hours	2/3			3/3	0.5/2.5
Lecture groups	65				20
Workshop groups	33				20
Project groups	4	3			2–4
Contents area	Construction			Rehabilitation	Constr. and Rehab.
Topics	Construction, structures, environmental comfort, services	Structures, façades, roofs	Soil, foundations and structures	Building pathology and rehabilitation techniques	Innovations
Learning approach	FL and PBL	Lectures and PBL, ALA and FL		Lectures and PBL	PBL and ALA
Assessment	Formative and summative
Competences	General, transversal and specific

Legend: IAT, Innovations in Architecture and Technology; N/A, non-applicable; ECTS, European Credit Transfer System; Constr. and Rehab., construction and rehabilitation; PBL, project basic learning; ALA, active learning activities; FL, flipped classroom.

Table 2

Comparison between first- and second-phase characteristics and results.

	Questionnaire 1	Questionnaire 2
N. answersN. potential answers Reliability	11058592.5%	10899592%
Level of construction/subject/answers/potential answers	Initial Technical bases: 59/390MediumConstruction II: 51/195Upper-	InitialTechnical Bases: 38/390MediumConstruction I: 10/195Construction II: 28/195UpperConstruction IV: 13/195IAT: 19/20
Previous experience visiting construction sites	Limited for first-level students, but at least one experience for the rest	Limited for first-level students, but at least one experience for the rest
Platform characteristics	360° panoramaVideosStandard picturesTechnical informationMapPossibility to move from different phases	360° panoramaVideosStandard picturesTechnical informationMapPossibility to move from different phasesVisualization through smartphone headsets
Connection means	Visit using a computer or mobile	Visit using a mobile and a smartphone headset
Kind of construction works	New building construction	New building construction and rehabilitation works in an existing building
Main advantages	Access to the platform at any time Avoid traveling to the site See more than in a work day	See more than in a work day Accessibility for people with reduced mobility Avoid traveling to site Possibility to visit the platform at any time
Main disadvantages	Difficult to understand what a real building site is Difficult to interpret the real scale of elements Lack of information through senses other than sight	Lack of comfort: dizziness after a while Lack of information from senses other than sight Technical problems from mobiles Difficult to understand what a work site really is

Table 3

Global sustainability index (GSI) and requirement satisfaction.

Code	Alternative	R1	R2	R3	R4	GSI
A31	Web-based 360	0.60	0.80	0.56	0.66	0.66
A32	Web-based and AR 360	0.50	0.77	0.56	0.74	0.68

Legend: R1, applicability requirement; R2, economic requirement; R3, environmental requirement; R4, social requirement.

Table 4

Evaluation of the applicability indicators.

Requirement	Indicators		A31	A32
Applicability	I01	Ease of application	0.74	0.70
	I02	Flexibility for adaptation	0.85	0.85
	I03	Transferability to other teachers	0.05	0.06
	I04	Transferability to other disciplines	0.62	0.47
Economic	I05	Direct costs	0.81	0.61
	I06	Logistic and scheduling issues	1.00	1.00
	I07	Dedication in class	0.88	0.88
	I08	Teachers’ dedication outside	0.21	0.21
	I09	Students’ dedication outside	1.00	1.00
Environmental	I10	Extra environmental impact	0.56	0.56
Social	I11	Roles, talents and ways of learning	0.44	0.62
	I12	Encouraging cooperative work	0.27	0.45
	I13	Autonomous work	0.76	0.76
	I14	Students’ cognitive load	0.69	0.73
	I15	Students’ interest and participation	0.77	0.95
	I16	Students and faculty contact	0.35	0.68
	I17	Feedback to students’ time	0.87	0.87
	I18	Learning outcomes (cognition and affect)	0.61	0.61
	I19	University learning innovation	0.98	0.98
	I20	Teachers’ new functions	0.64	0.64

Appendix A

Table A1

Abbreviations used in the text.

Abbreviations	Relevant Values
ALA	Active learning activities
ACE	Architectural and civil engineering
AR	Augmented reality
CBL	Challenge-based learning
ECTS	European Credit Transfer System
FL	Flipped classroom
GSI	Global sustainability index
IAT	Innovations in architecture and technology
MIVES	Integrated value model for sustainability assessment
MW	Mirror worlds
PBL	Project-based learning
N/A	Non-applicable
SWOT	Strengths, weaknesses, opportunities and threats technique
TBL	Team-based learning
VR	Virtual reality
VW	Virtual worlds

Appendix B

Table A2

First questionnaire.

Block	Question	Possible Answers
1st blockETHICS	1—Do you agree that your answers to this questionnaire are used to improve the 360° Architecture learning platform being developed by ETSAB and ETSAM?	YesNo
2nd blockCONTEXTUALIZATION	2—During which course are you doing this questionnaire?	Technical BasesConstruction IConstruction IIConstruction IIIConstruction IVMaster MBarch, CP, Materiality and ProjectAnother course
	3—If it is another course, which one is it?	Free answer
	4—Have you ever visited a building under construction?	Yes, oneYes, from one to threeYes, more than threeNo
	5—Have you ever visited the building of the platform (Batlle i Roig student residence and offices at c/Cristòbal de Moura 196)?	YesNo
3rd blockPLATFORM USE	6—What aspects did you find most useful about the Architecture 360° platform?	360° panoramasVideosStandard picturesTechnical informationThe function to move from different construction phases The function to move between the three pointsThe mapThe bar to move thorough the panoaramasOthers
	7—If there are other aspects that you have found most useful, what are they?	Free answer
	8—What advantages do you think that the Arquitectura 360° platform has compared to going on a site visit?	You avoid having to travel to the siteYou don’t need the personal protective equipment (helmet, vest, boots)There are no risksAllows access for people with reduced mobilityIt can be done at any time and dayYou can see more than one day of workThere are no vision/audio issues that large groups may encounter during an onsite visit, when most of the group has problems seeing and listeningOthers
	9—If the platform has other strengths and advantages, which are they?	Free answer
	10—What disadvantages do you think the Arquitectura 360° platform has compared to going on a site visit?	It’s hard to get an idea of the scale of the things you see in the platformYou do not receive information from other senses other than sight (touch, smell, sound)You don’t get much of an idea of materialityYou have no idea what a work really isOthers
	11—If the platform has other weaknesses and disadvantages, which are they?	Free answer
	12—To sum up, the Arquitectura 360 platform, compared to go to the building site you think it is?	BetterWorseComplementaryOthers
	13—Why?	Free answer

Appendix C

Table A3

Second questionnaire.

Block	Question	Possible Answers
1st blockETHICS	1—Do you agree that your answers to this questionnaire are used to improve the 360° Architecture learning platform being developed by ETSAB and ETSAM?	YesNo
2nd blockCONTEXTUALIZATION	2—During which course are you doing this questionnaire?	Technical BasesConstruction IConstruction IIConstruction IIIConstruction IVMaster MBarch, CP, Materiality and ProjectAnother course
	3—If it is another course, which one is it?	Free answer
	4—Have you ever visited a building under construction?	Yes, oneYes, from one to threeYes, more than threeNo
	5—Have you ever visited the building of the platform (Batlle i Roig student residence and offices at c/Cristòbal de Moura 196)?	YesNo
3rd blockPLATFORM USE	6—What aspects did you find most useful about the Architecture 360° platform?	360° panoramasThe function to move from one point to anotherThe function to move from one day to anotherThe fact to be able to visualize it through smartphone headsetsOthers
	7—If there are other aspects that you have found most useful, what are they?	Free answer
	8—What advantages do you think that the Arquitectura 360° platform has compared to going on a site visit?	You avoid having to travel to the siteYou don’t need the personal protective equipment (helmet, vest, boots)There are no risksAllows access for people with reduced mobilityIt can be done at any time, time, dayYou can see more than one day of workThere are no vision/audio issues that large groups may encounter during an onsite visit, when most of the group has problems seeing and listeningOthers
	9—If the platform has other strengths and advantages, which are they?	Free answer
	10—What disadvantages do you think the Arquitectura 360° platform has compared to going on a site visit?	Technical: the mobile does not fit the smartphone headsets well, the mobile closesComfort: headsets hurt, botherConnection: the platform does not connect well, there are times with connection lossesComfort: dizziness after a whileUsage: it is difficult to orientate within the work, I get lost, I lose points to change point or dayIt’s hard to get an idea of the scale of the things you see in the platformYou do not receive information from other senses other than sight (touch, smell, sound)You don’t get much of an idea of materialityYou have no idea what a work really is
	11—If the platform has other weaknesses and disadvantages, which are they?	Free answer
	12—How did the activity you did go?	Positive, I have been able to learn more about the classes contentsNot very useful, I have been able to understand better some things, though some doubts have raised upUseless
	13—Do you have more comments about the activity?	Free answer
	14—To sum up, the Arquitectura 360 platform, compared to go to the building site you think it is?	BetterWorseComplementaryOthers
	15—Why?	Free answer

References

1. Sánchez-Aparicio, L.J.; Sánchez-Guevara Sánchez M del, C.; Gallego Sánchez-Torija, J.; Olivieri, F. Buildings 360°: Un Nuevo Enfoque Para La Enseñanza En Construcción. Proceedings of the VIII Jornadas sobre Innovación Docente en Arquitectura (JIDA’20), Escuela Técnica Superior de Arquitectura de Málaga; Barcelona, Spain, 12–13 November 2020; libro de actas Publicaciones y Divulgación Científica, Universidad de Málaga: Barcelona, Spain, 2020; pp. 162-174.

2. Pons Valladares, O. Actividades de Aprendizaje Para Sesiones Prácticas Sobre La Construcción En Arquitectura. Proceedings of the IX Jornadas Sobre Innovación Docente en Arquitectura (JIDA’21), Escuela Técnica Superior de Arquitectura de Valladolid; Barcelona, Spain, 11–12 November 2021; libro de actas Universitat Politècnica de Catalunya, Iniciativa Digital Politècnica: Barcelona, Spain, 2021; pp. 249-260.

3. de la Fuente Prieto, J.; Lacasa, P.; Martínez-Borda, R. Approaching Metaverses: Mixed Reality Interfaces in Youth Media Platforms. New Techno Humanit.; 2022; 2, pp. 136-145. [DOI: https://dx.doi.org/10.1016/j.techum.2022.04.004]

4. Phakamach, P.; Senarith, P.; Wachirawongpaisarn, S. The Metaverse in Education: The Future of Immersive Teaching & Learning. RICE J. Creat. Entrep. Manag. (RJMC); 2022; 3, pp. 75-88.

5. Ng, D.T.K. What Is the Metaverse? Definitions, Technologies and the Community of Inquiry. Australas. J. Educ. Technol.; 2022; 38, pp. 190-205. [DOI: https://dx.doi.org/10.14742/ajet.7945]

6. Mystakidis, S. Metaverse. Encyclopedia; 2022; 2, pp. 486-497. [DOI: https://dx.doi.org/10.3390/encyclopedia2010031]

7. di Natale, A.F.; Repetto, C.; Riva, G.; Villani, D. Immersive Virtual Reality in K-12 and Higher Education: A 10-year Systematic Review of Empirical Research. Br. J. Educ. Technol.; 2020; 51, pp. 2006-2033. [DOI: https://dx.doi.org/10.1111/bjet.13030]

8. Hajirasouli, A.; Banihashemi, S. Augmented Reality in Architecture and Construction Education: State of the Field and Opportunities. Int. J. Educ. Technol. High. Educ.; 2022; 19, 39. [DOI: https://dx.doi.org/10.1186/s41239-022-00343-9]

9. Inceoglu, M.M.; Ciloglugil, B. Use of Metaverse in Education. Proceedings of the Computational Science and Its Applications–ICCSA 2022 Workshops; Malaga, Spain, July 4–7 2022; pp. 171-184.

10. Tlili, A.; Huang, R.; Shehata, B.; Liu, D.; Zhao, J.; Metwally, A.H.S.; Wang, H.; Denden, M.; Bozkurt, A.; Lee, L.-H. et al. Is Metaverse in Education a Blessing or a Curse: A Combined Content and Bibliometric Analysis. Smart Learn. Environ.; 2022; 9, 24. [DOI: https://dx.doi.org/10.1186/s40561-022-00205-x]

11. Anacona Ortiz, J.D.; Millán Rojas, E.E.; Gómez Cano, C.A. Aplicación de Los Metaversos y La Realidad Virtual en La Enseñanza. Entre Cienc. E Ing.; 2019; 13, pp. 59-67. [DOI: https://dx.doi.org/10.31908/19098367.4015]

12. Chen, P.; Liu, X.; Cheng, W.; Huang, R. A Review of Using Augmented Reality in Education from 2011 to 2016. Innovations in Smart Learning; Springer: Singapore, 2017; pp. 13-18. [DOI: https://dx.doi.org/10.1007/978-981-10-2419-1]

13. Bower, M.; Howe, C.; McCredie, N.; Robinson, A.; Grover, D. Augmented Reality in Education—Cases, Places and Potentials. Educ. Media Int.; 2014; 51, pp. 278-294. [DOI: https://dx.doi.org/10.1080/09523987.2014.889400]

14. Lepez, C.O. Metaverso y Educación: Una Revisión Panorámica. Metaverse Basic Appl. Res.; 2022; 1, 2. [DOI: https://dx.doi.org/10.56294/mr20222]

15. Hwang, G.-J.; Chien, S.-Y. Definition, Roles, and Potential Research Issues of the Metaverse in Education: An Artificial Intelligence Perspective. Comput. Educ. Artif. Intell.; 2022; 3, 100082. [DOI: https://dx.doi.org/10.1016/j.caeai.2022.100082]

16. Sun, Y.; Li, N.; Hao, J.L.; Di Sarno, L.; Wang, L. Post-COVID-19 Development of Transnational Education in China: Challenges and Opportunities. Educ. Sci.; 2022; 12, 416. [DOI: https://dx.doi.org/10.3390/educsci12060416]

17. Lin, H.; Wan, S.; Gan, W.; Chen, J.; Chao, H.-C. Metaverse in Education: Vision, Opportunities, and Challenges. arXiv; 2022; [DOI: https://dx.doi.org/10.48550/arxiv.2211.14951] arXiv: 2211.14951

18. Hirsh-Pasek, K.; Zosh, J.; Hadani, H.S.; Golinkoff, R.; Clark, K.; Donohue, C.; Wartella, E. A Whole New World: Education Meets the Metaverse; Center for Universal Education at Brookings: Washington, DC, USA, 2022; pp. 1-13.

19. Márquez, I.V. Metaversos y Educación: Second Life Como Plataforma Educativa. Rev. ICONO14. Rev. Científica Comun. Tecnol. Emerg.; 2011; 9, 151. [DOI: https://dx.doi.org/10.7195/ri14.v9i2.30]

20. Kye, B.; Han, N.; Kim, E.; Park, Y.; Jo, S. Educational Applications of Metaverse: Possibilities and Limitations. J. Educ. Eval. Health Prof.; 2021; 18, 32. [DOI: https://dx.doi.org/10.3352/jeehp.2021.18.32]

21. Contreras, G.S.; González, A.H.; Fernández, M.I.S.; Cepa, C.B.M.; Escobar, J.C.Z. The Importance of the Application of the Metaverse in Education. Mod. Appl. Sci.; 2022; 16, 34. [DOI: https://dx.doi.org/10.5539/mas.v16n3p34]

22. Redondo, E.; Fonseca, D.; Sánchez, A.; Navarro, I. Mobile Learning En l’àmbit de l’arquitectura i l’edificació. Anàlisi de Casos d’estudi. RUSC. Rev. Univ. Soc. Conoc.; 2014; 11, 152. [DOI: https://dx.doi.org/10.7238/rusc.v11i1.1844]

23. Redondo Domínguez, E.; Sánchez Riera, A.; Moya Sala, J. La Ciudad Como Aula Digital: Enseñando Urbanismo y Arquitectura Mediante Mobile Learning y La Realidad Aumentada: Un Estudio de Viabilidad y de Caso. ACE Archit. City Environ.; 2012; 7, pp. 27-54. [DOI: https://dx.doi.org/10.5821/ace.v7i19.2560]

24. Abdelhameed, W.A. Virtual Reality Use in Architectural Design Studios: A Case of Studying Structure and Construction. Procedia Comput. Sci.; 2013; 25, pp. 220-230. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.027]

25. Bashabsheh, A.K.; Alzoubi, H.H.; Ali, M.Z. The Application of Virtual Reality Technology in Architectural Pedagogy for Building Constructions. Alex. Eng. J.; 2019; 58, pp. 713-723. [DOI: https://dx.doi.org/10.1016/j.aej.2019.06.002]

26. Escapa, B.; Bustamante Escapa, J.B.; Santy, P. Dibujo Tridimensional, Percepción Espacial y Habilidades Espaciales de Estudiantes Universitarios de Primer Año y de Último Año de La Carrera de Arquitectura Usando Realidad Virtual Inmersiva; Universidad Nacional de San Agustín de Arequipa: Arequipa, Peru, 2022.

27. Diao, P.H.; Shih, N.J. Trends and Research Issues of Augmented Reality Studies in Architectural and Civil Engineering Education—A Review of Academic Journal Publications. Appl. Sci.; 2019; 9, 1840. [DOI: https://dx.doi.org/10.3390/app9091840]

28. Gaafar, A.A. Use of Fully Immersive Virtual Reality Techniques to Generate Scale 1: 1 Interactive Models of Pharaonic Tombs. J. SSPub Sci. Stay. True Here; 2021; 6, pp. 66-86.

29. Maghool, S.A.H.; Moeini, S.H.; Arefazar, Y. An Educational Application Based on Virtual Reality Technology for Learning Architectural Details: Challenges and Benefits. Int. J. Archit. Res. ArchNet-IJAR; 2018; 12, 246. [DOI: https://dx.doi.org/10.26687/archnet-ijar.v12i3.1719]

30. De, L.I.; Torre, L.A.; Homenaje, D.; Castillo, D.; Montilla, D.E.; Mohedo Gatón, A. El Metaverso En El Proceso de Enseñanza-Aprendizaje: La Interpretación de La Torre Del Homenaje Del Castillo de Montilla. Doctoral Thesis; Universidad de Granada: Granada, Spain, 2022.

31. Rovira-Collado, J.; Llorens García, R.F.; Ruiz Bañuls, M.; Martín-Martín, A.; Soler-Quílez, G.; Pérez Beneyto, J.C.; Ríbes Lafoz, M.; Martínez Carratalá, F.A.; Lanseros Sánchez, R.; Santos Maestre, L. et al. Redes de Investigación e Innovación En Docencia Universitaria; Universidad de Granada: Granada, Spain, 2022; 2022.

32. Redondo, E.; Fonseca, D.; Sánchez, A.; Navarro, I. New Strategies Using Handheld Augmented Reality and Mobile Learning-Teaching Methodologies, in Architecture and Building Engineering Degrees. Procedia Comput. Sci.; 2013; 25, pp. 52-61. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.007]

33. Corso, J.; Garcia-Almirall, P.; Marco, A. High Resolution Model Mesh and 3D Printing of the Gaudí’s Porta Del Drac. IOP Conf. Ser. Mater. Sci. Eng.; 2017; 245, 052091. [DOI: https://dx.doi.org/10.1088/1757-899X/245/5/052091]

34. Birt, J.; Cowling, M. Toward Future “Mixed Reality” Learning Spaces for STEAM Education. Int. J. Innov. Sci. Math. Educ.; 2017; 25, pp. 1-16.

35. Cirulis, A.; Brigmanis, K.B. 3D Outdoor Augmented Reality for Architecture and Urban Planning. Procedia Comput. Sci.; 2013; 25, pp. 71-79. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.009]

36. Gonzalez, N.A.A.; Warden, F.S.; Milian, H.N.Q.; Hosseini, S. Interactive Design and Architecture by Using Virtual Reality, Augmented Reality and 3D Printing. Int. J. Simul. Process Model.; 2020; 15, 535. [DOI: https://dx.doi.org/10.1504/IJSPM.2020.112468]

37. Bademosi, F.; Blinn, N.; A Issa, R.R. Use of Augmented Reality Technology to Enhance Comprehension of Construction Assemblies. J. Inf. Technol. Constr.; 2019; 24, pp. 58-79.

38. Fazel, A.; Izadi, A. An Interactive Augmented Reality Tool for Constructing Free-Form Modular Surfaces. Autom. Constr.; 2018; 85, pp. 135-145. [DOI: https://dx.doi.org/10.1016/j.autcon.2017.10.015]

39. Kim, J.; Irizarry, J. Evaluating the Use of Augmented Reality Technology to Improve Construction Management Student’s Spatial Skills. Int. J. Constr. Educ. Res.; 2021; 17, pp. 99-116. [DOI: https://dx.doi.org/10.1080/15578771.2020.1717680]

40. Anderson, L.W.; Krathwohl Peter W Airasian, D.R.; Cruikshank, K.A.; Mayer, R.E.; Pintrich, P.R.; Raths, J.; Wittrock, M.C. Taxonomy for Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives; Longman: Harlow, UK, 2001.

41. Chickering, A.W.; Gamson, Z.F. Seven Principles for Good Practice in Undergraduate Education. AAHE Bull.; 1987; 3, 7.

42. Torrijos-Muelas, M.; González-Víllora, S.; Bodoque-Osma, A.R. The Persistence of Neuromyths in the Educational Settings: A Systematic Review. Front. Psychol.; 2021; 11, 3658. [DOI: https://dx.doi.org/10.3389/fpsyg.2020.591923] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33510675]

43. Benzaghta, M.A.; Elwalda, A.; Mousa, M.; Erkan, I.; Rahman, M. SWOT Analysis Applications: An Integrative Literature Review. J. Glob. Bus. Insights; 2021; 6, pp. 55-73. [DOI: https://dx.doi.org/10.5038/2640-6489.6.1.1148]

44. Pons-Valladares, O.; Hosseini, S.M.A.; Franquesa, J. Innovative Approach to Assist Architecture Teachers in Choosing Practical Sessions. Sustainability; 2022; 14, 7081. [DOI: https://dx.doi.org/10.3390/su14127081]

45. del Casanovas-Rubio, M.; Armengou, J. Decision-Making Tool for the Optimal Selection of a Domestic Water-Heating System Considering Economic, Environmental and Social Criteria: Application to Barcelona (Spain). Renew. Sustain. Energy Rev.; 2018; 91, pp. 741-753. [DOI: https://dx.doi.org/10.1016/j.rser.2018.04.040]

46. Boix-Cots, D.; Pardo-Bosch, F.; Blanco, A.; Aguado, A.; Pujadas, P. A Systematic Review on MIVES: A Sustainability-Oriented Multi-Criteria Decision-Making Method. Build Environ.; 2022; 223, 109515. [DOI: https://dx.doi.org/10.1016/j.buildenv.2022.109515]

47. Josa, I.; Pons, O.; de la Fuente, A.; Aguado, A. Multi-Criteria Decision-Making Model to Assess the Sustainability of Girders and Trusses: Case Study for Roofs of Sports Halls. J. Clean. Prod.; 2020; 249, 119312. [DOI: https://dx.doi.org/10.1016/j.jclepro.2019.119312]

48. ICLEI. Towards Sustainable Cities & Towns: Report of the First European Conference on Sustainable Cities & Towns; ICLEI Local Goverments for Sustainability: Freiburg, Germany, 1994.

49. Cornadó Bardón, C.; Ramos Sanz, A.A. Edulearn13 Proceedings CD; International Association of Technology, Education and Development (IATED): Valencia, Spain, 2013; ISBN 9788461638222

50. Cornadó Bardón, C.; Crespo Sánchez, E.; Martín García, E. Asignaturas Tecnológicas En Arquitectura En El Confinamiento: Hacia Una Enseñanza Aplicada. Proceedings of the VIII Jornadas sobre Innovación Docente en Arquitectura (JIDA’20), Escuela Técnica Superior de Arquitectura de Málaga; Barcelona, Spain, 12–13 November 2020; libro de actas Publicaciones y Divulgación Científica, Universidad de Málaga: Barcelona, Spain, 2020; pp. 175-185.

51. Onecha Pérez, A.B.; Caballero Marcos, I.A.; López Valdés, D.; Sanz Prat, J. El Aprendizaje Cooperativo Aplicado a Las Asignaturas Técnicas Del 1r Curso de Arquitectura. Proceedings of the V Jornadas sobre Innovación Docente en Arquitectura (JIDA’17), Escuela Técnica Superior de Arquitectura de Sevilla; Barcelona, Spain, 16–17 November 2017; libro de actas Grup per a la Innovació i la Logística Docent en l’Arquitectura (GILDA): Barcelona, Spain, 2017; pp. 176-189.

52. Onecha Pérez, B.; Sanz Prat, J.; López Valdés, D. Los Límites de La Ludificación En La Enseñanza de La Arquitectura. La Técnica Del Escape Room. Zarch; 2019; 12, pp. 122-133. [DOI: https://dx.doi.org/10.26754/ojs_zarch/zarch.2019123549]

53. Onecha Pérez, B.; Berbegal-Mirabent, J. La Versatilidad Del Método de La Clase Invertida: Estudio de Un Caso de Aplicación Durante La Crisis de La COVID-19. REDU. Rev. Docencia Univ.; 2020; 18, 49. [DOI: https://dx.doi.org/10.4995/redu.2020.14419]

54. Echarte Ramos, J.M. Dimensión Económica y Laboral En La Formación de Los Arquitectos Españoles. Zarch; 2019; 12, pp. 48-61. [DOI: https://dx.doi.org/10.26754/ojs_zarch/zarch.2019123542]

55. Pons-Valladares, O.; González-Barroso, J.M.; López-Olivares, R.; Arias, I. Educational Project to Improve Problem-Based Learning in Architectural Construction Courses Using Active and Co-Operative Techniques. Rev. Construcción. J. Constr.; 2015; 14, pp. 35-43. [DOI: https://dx.doi.org/10.4067/S0718-915X2015000200005]

56. Pons, O.; Franquesa, J.; Hosseini, S.M.A. Integrated Value Model to Assess the Sustainability of Active Learning Activities and Strategies in Architecture Lectures for Large Groups. Sustainability; 2019; 11, 2917. [DOI: https://dx.doi.org/10.3390/su11102917]

57. Pons Valladares, O.; Franquesa Sànchez, J. Actividades y Estrategias de Aprendizaje Activo Para Clases Teóricas En Grupos Numerosos. Proceedings of the VI Jornadas sobre Innovación Docente en Arquitectura (JIDA’18), Escuela de Ingeniería y Arquitectura de Zaragoza; Zaragoza, Spain, 22–23 November 2018; Universidad de Zaragoza Servicio de Publicaciones de la Universidad de Zaragoza: Zaragoza, Spain, 2018; pp. 10-23.

58. Coloma Picó, E. Tecnologia BIM per al Disseny Arquitectònic. Ph.D. Thesis; Universitat Politècnica de Catalunya-Barcelonatech: Barcelona, Spain, 2011.

59. Cao, K. Development and Design Case Function Comparison of Panoramic Roaming System of Virtual Museum Based on Pano2VR. Mob. Inf. Syst.; 2022; 2022, 7363221. [DOI: https://dx.doi.org/10.1155/2022/7363221]

60. Arquitectura360.Ta Crist. de Moura. Available online: https://arquitectura360.ta.upc.edu/GenCristMoura/ (accessed on 16 May 2023).

61. Google Cardboard—Google VR. Available online: https://arvr.google.com/cardboard/ (accessed on 10 February 2023).

62. Arquitectura360—ENGLISH. Available online: https://sites.google.com/upc.edu/arquitectura360/english?pli=1 (accessed on 10 February 2023).

63. Zhdanov, A.D.; Zhdanov, D.D.; Bogdanov, N.N.; Potemin, I.S.; Galaktionov, V.A.; Sorokin, M.I. Discomfort of Visual Perception in Virtual and Mixed Reality Systems. Program. Comput. Softw.; 2019; 45, pp. 147-155. [DOI: https://dx.doi.org/10.1134/S036176881904011X]

64. Ángel, M.; Villegas, G.; de Santos, D. Inferencia Estadística; Ediciones Díaz de Santos: Madrid, Spain, 2005.

65. Freedman, D.A. Statistical Models: Theory and Practice; Cambridge University Press: Cambridge, UK, 2009; pp. 1-442. [DOI: https://dx.doi.org/10.1017/CBO9780511815867]

66. Neyman, J. On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection. J. R. Stat. Soc.; 1934; 97, 558. [DOI: https://dx.doi.org/10.2307/2342192]

67. Statistical Inference—Encyclopedia of Mathematics. Available online: https://encyclopediaofmath.org/wiki/Statistical_inference (accessed on 27 February 2023).