Evaluation of Enterprise Learning Performance in the Process of Cooperation Innovation Using Heronian Mean Operator

Abstract

In order to clarify the achievement and efficiency of enterprise learning in the process of cooperative innovation, a comprehensive criteria framework for the evaluation of enterprise learning performance is constructed taking the learning process and learning results as the construction idea based on organizational learning theory. And this paper proposes a novel dynamic evaluation method considering the interaction between attributes of learning performance. In this study, the criterion framework for the evaluation of enterprise learning performance in the process of cooperative innovation includes learning process performance and learning outcomes performance. The original matrices are given by managers and experts using fuzzy set theory, and a dynamic time sequence weight vector is calculated based on information entropy and time degree. The weight of learning performance attributes under different time series is calculated based on entropy measure method. The interactive information of learning performance attributes is integrated through the weight of learning performance attributes and the three-parameter weighted Heronian mean operator considering the interaction between attributes. And then, the dynamic and comprehensive evaluation result of learning performance in the process of cooperative innovation could be computed by integrating the learning performance information under different time series with time sequence weight vector. Finally, a real case is studied to verify the scientificity and validity of the criteria framework for the evaluation of enterprise learning performance and the method proposed in this study. This study not only helps cooperative enterprises get feedback in time and adjust cooperative relationships and learning styles but also enriches the theory of interorganizational management and provides a theoretical basis for the process of enterprise cooperative innovation.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

With the rapid development of Internet industry and the deep integration of artificial intelligence and Internet of Things, cooperative innovation among enterprises has become an important way for enterprises to gain competitive advantage. Mining and learning new knowledge from cooperative innovators have gradually become a key factor in the long-term development of enterprises [1]. In the process of cooperative innovation, interenterprise learning plays undoubtedly an important way to enhance their competitive advantages [2]. Discernible learning effect has an important impact on adjusting learning content, perfecting learning management mechanism, improving learning management ability, and whether to end current cooperation innovation projects. The evaluation of enterprise learning performance plays a vital tool for enterprises to clarify learning effectiveness and is also an important mechanism to judge whether cooperative goals are achieved and to manage cooperative relationships among enterprises [3]. In the process of cooperative innovation, learning performance evaluation can also help cooperative enterprises get feedback in time and understand the degree of implementation of learning objectives among enterprises [4]. The evaluation results of learning performance determine whether enterprises adjust their interenterprise cooperative relationships and learning styles. The evaluation of enterprise learning performance can effectively realize the intention of cooperative innovation among enterprises and avoid the difficulty of cooperative innovation. With the deep integration of Internet and Internet of Things, resource sharing, complementary advantages, risk sharing, and achievement sharing among enterprises have become a new normal, which further strengthens the cooperative innovation relationship among enterprises [5]. Therefore, in the process of cooperative innovation, enterprise learning performance evaluated scientifically and reasonably has important theoretical value and practical significance.

In the process of cooperative innovation, enterprises can acquire knowledge and skills of other enterprises across organizational boundaries through inter-enterprise learning. Interenterprise learning not only enriches the knowledge source channels and knowledge structure of enterprises but also makes up for the lack of internal learning [6]. With the change of environment and technology, more and more knowledge is acquired through innovative cooperation of different types or fields outside enterprises. Moreover, knowledge could flow across organizational boundaries to innovative members through cooperative innovation activities [1, 2]. Thus, how to manage knowledge among enterprises has become an important connotation of interenterprise learning. Interenterprise learning is a means for enterprises to acquire and use knowledge resources from cooperative innovation partners. The essence of interenterprise learning is the ability to acquire, disseminate, and maintain new knowledge in the cooperative innovation network for improving future performance [7]. Interenterprise learning, as the most valuable learning resource, can enable enterprises to maintain long-term competitive advantage. This kind of learning resource acquires the knowledge and skills needed by enterprise innovation based on learning from other enterprises and generates the new knowledge needed in the process of internalization of external knowledge [4]. Trust is an important behavioral model to establish cooperative relationships among enterprises, so as to realize learning among enterprises [3–8]. In the process of cooperative innovation, the culture of mutual benefit is an important guarantee mechanism for the effective use of interenterprise learning [9]. Interenterprise learning is a bilateral or multilateral learning process, which enables different enterprises to achieve different goals in the process of cooperative innovation [10]. In addition, learning behavior and learning process are the core content of cooperative innovation. However, how to measure the effectiveness of interenterprise learning is the key to long-term cooperative innovation and learning among enterprises. From the function of performance evaluation itself, performance evaluation theory shows that performance evaluation is a management control tool to implement organizational strategy. By comparing the differences between learning outcomes and learning objectives in the process of cooperative innovation based on performance evaluation, enterprises can dynamically track the process of strategy implementation and find problems in management [4, 6–8, 11]. Moreover, learning performance evaluation can provide feedback for the next innovation planning of enterprises by analyzing the deviation of past performance. Discernible learning effect has an important impact on adjusting learning content, perfecting learning management mechanism, improving learning management ability, and whether to end current cooperation innovation projects [3].

In the process of cooperative innovation, the evaluation of enterprise learning performance is a comprehensive evaluation of complex multidimensional factors, which belongs to interorganizational performance. Many scholars have studied the learning mechanism, learning methods, and influencing factors of inter-organizational learning performance [12–20]. Kellogg [13] empirically tested the importance of learning in specific organizational relationships based on high-frequency data from oil and gas drilling. Şengün studied the effect of trust types on knowledge sharing between small and medium-sized manufacturers and retailers in furniture industry cluster, and this research shows that the trust based on competence, reliability, and predictability has no significant correlation with interenterprise learning, while the trust based on goodwill, benevolence, and nonopportunism has a significant positive correlation with interenterprise learning [8]. Subsequently, Şengün and Önder [14] further studied the interactive effect of ability and goodwill trust on interenterprise learning performance and thought that goodwill trust has a positive main effect on interenterprise learning performance, while ability trust has no positive main effect on interenterprise learning performance. Gupta and Polonsky [15] concluded that multinational enterprises could operate and interact more effectively by sharing knowledge reflecting customer structure and suitability with outsourcing enterprises, and learning and knowledge sharing are closely coupled in the product development stage. Saenz et al. [16] thought that interorganizational learning practice is one of the important ways to solve the problems of lack of trust among trading partners and difficulties in strategic collaboration. Gibb et al. [3] investigated the interorganizational learning mechanism in network and believed that interorganizational learning including learning how to compete and how to behave helps to coordinate network learning among horizontal enterprises in order to achieve performance goals. Lis and Sudolska [17] studied the relationship between learning process and learning content among enterprises through internal learning and thought that there was synergy between them. Akbar [10] highlighted that interaction and open communication enable enterprises not only to understand customer needs but also to acquire more knowledge based on twelve interviews with four Finnish SMEs. In terms of theoretical research on interorganizational performance, Ittner and Larcker [18] thought that interorganizational performance evaluation is of great significance and will be the focus of future research. However, the studies on interorganizational cooperation control are gradually increasing, and interorganizational performance evaluation still does not occupy the mainstream. Although some scholars have studied the concept of interorganizational cost and related evaluation methods [19], there are still some deficiencies in the theoretical and practical research of linking interorganizational performance evaluation with interorganizational learning. Then, Zhi and Dai [20] proposed a systematic balanced scorecard framework for strategic alliance performance evaluation and used this framework to study the relationship between interorganizational cooperation and intrafirm performance dimensions. Through qualitative research, Dai and Zhi [4] thought that interorganizational learning in strategic alliances could improve the development ability of enterprises, and the evaluation of learning performance could improve learning behavior. In order to further explore the significance of the theoretical framework of interorganizational learning performance evaluation, Petersen et al. [12] thought that interactive learning and trust evolution have a synergistic relationship, which could improve the efficiency of knowledge transfer in cooperative network.

In terms of evaluation methods, there are few studies on the methods of performance evaluation in the process of cooperative innovation. Kusi-Sarpong et al. [21] used the best-worst multicriteria decision-making model to evaluate sustainable innovation management standards with five Indian manufacturing companies as samples in order to prove the applicability and efficiency of the proposed framework. Song et al. [22] believed that it is very important to apply existing theories and methods to evaluate environmental performance successfully in practice, and environmental performance evaluation provides scientific basis and guidance for formulating environmental protection policies. Arbolino et al. [23] used principal component analysis model to measure ecoindustrial policy and industrial environmental sustainability index by identifying the application strategies of relevant participants through variables. Kazancoglu et al. [24] put forward a performance evaluation model of green supply chain management in cement enterprises and used the technology of fuzzy decision-making test and evaluation laboratory to analyze the causality. Cook et al. [25] extended data envelopment analysis to the performance evaluation of performance incentive plan, and the proposed method was applied to evaluate the performance of decision-making units after the implementation of incentive plan according to achievable goals and representative best practices. Luo et al. [26] used AHP and matter-element analysis to evaluate the performance of Public-Private Partnership. Malings et al. [27] assessed the temporal variability of spatial distribution and air quality in cities using various algorithms for correcting slope measurements, which include linear and quadratic regression, clustering, neural network, Gauss process, and mixed stochastic forest-linear regression model. Santos et al. [28] evaluated the performance of green suppliers using entropy weight method, technique for order preference by similarity to an ideal solution method, and decision-making trial and evaluation laboratory method. Moldavska and Welo [29] proposed a new method to assess the sustainability of manufacturing enterprises and thought that enterprise sustainability assessment in manufacturing industry is a tool framework to guide organizations towards sustainable practices. Lee and Choi [30] decomposed the sequential Malmquist-Luenburger index into two indicators, efficiency change and technology change, and evaluated the environmental performance of Korean manufacturing industry using Malmquist-Luenburger index and generalized directional distance function. In addition, the above traditional performance evaluation methods including AHP [11], BSC [31], and EVA [32] mostly assume that attributes are independent of each other. However, attributes are not always independent, and there are always some interaction relationships, such as complementarity, redundancy, and preference. Even if the C-POWA operator [33], DLWA operator [34], and other related integration operators are used for performance evaluation, the same problem exists. Moreover, most of these methods are based on static information of a single period. Even though these methods involve dynamic related elements, the comprehensiveness of evaluation is still neglected. Although some scholars, such as Su et al. [35], Park et al. [36], and Cao et al. [37], introduced time weight vectors, they established these time sequence weight vectors by direct assignment of subjective randomness, which may lead to unreasonable comprehensive evaluation results.

Although the above studies provide a support for the evaluation of enterprise learning performance in the process of cooperative innovation, there is still a lack of relevant research on the practical application value of enterprise learning performance evaluation. The above studies seldom deal with the content related to the evaluation system of learning performance in the process of enterprise cooperative innovation from a systematic perspective. The above methods of performance evaluation also have great limitations, without considering the interaction between attributes. The evaluation of learning performance in the process of cooperative innovation plays an important role in improving the management mechanism of intraorganizational learning and interorganizational learning [12]. In the process of enterprise cooperative innovation, the evaluation results of learning performance have an important impact on the adjustment of cooperation mode, cooperation content, and cooperation strategy. The application of reasonable method is one of the important guarantees for the accuracy and rationality of the performance evaluation results [22–37]. Therefore, it is necessary to further study the practical application of the evaluation of enterprise learning performance in the process of cooperative innovation by considering the interaction between attributes and determining a scientific and reasonable sequential weight vector.

To address these shortcomings, a comprehensive criteria framework for the evaluation of enterprise learning performance is constructed taking the learning process and learning results as the construction idea based on organizational learning theory. This study further proposes a novel dynamic evaluation method considering the interaction between attributes of learning performance. The practical contribution of this study is that the method and criteria framework proposed in this study not only help to improve the effectiveness and stability of the comprehensive evaluation results of enterprise learning performance in the process of cooperative innovation but also help cooperative enterprises get feedback in time and adjust cooperative relationships and learning styles. In theory, the comprehensive and representative framework proposed in this study not only provides a theoretical basis for the study of factors affecting interorganizational learning and performance evaluation but also enriches the theory of interorganizational management and provides a theoretical basis for the process of enterprise cooperative innovation. In addition, the novel method proposed in this study expands the application scope of fuzzy theory and time series in learning performance evaluation.

The rest of this paper is structured as follows: the criteria framework for the evaluation of enterprise learning performance in the process of cooperative innovation is presented in Section 2. In Section 3, a novel dynamic comprehensive evaluation method considering the interaction between attributes will be proposed to evaluate enterprise learning performance. A case study is furnished in Section 4 to illustrate how the approach could be applied to multicriteria evaluation problems. Conclusions including managerial and practical implications and future research are highlighted in Section 5.

2. Criteria Framework for the Evaluation of Enterprise Learning Performance

A large number of cooperative information data could be produced in the process of cooperative innovation. The selection of cooperative information data has an important impact on the scientificity and reliability for the evaluation results of learning performance. Some studies directly measure interorganizational learning effectiveness criteria, which include knowledge transfer, knowledge accumulation, strategic coherence, relational capital, and learning control [38, 39]. In addition, Lane and Lubatkin thought that the correlation between interorganizational learning performance and capacity accumulation is stronger than that between interorganizational learning performance and business performance, and the capacity accumulation is regarded as the evaluation criterion of interorganizational learning performance [7]. According to the content and evolution characteristics of interorganizational learning, the evolution content of interorganizational learning is the process of clarification, sharing, and internalization of relevant knowledge into another organization based on the perspective of system view.

From different perspectives, scholars put forward many factors that affect the effectiveness of inter-organizational learning [40–44]. Zhu et al. [40] proposed that there are four main factors affecting the effectiveness of interorganizational learning, namely, culture, structure, technology, and absorptive capacity. Among them, absorptive capacity is the basis of technical learning in an organization, and shared knowledge links at any level are conducive to the dissemination of absorptive capacity elements. Zhang et al. [41] thought that the factors affecting the success of alliance learning can be divided into three categories: the availability of alliance knowledge, the effectiveness of knowledge acquisition, and the learning connection based on the availability of alliance knowledge and the effectiveness of knowledge acquisition. Yayavaram et al. [42] thought that the key elements of organizational learning are interaction between partners, high learning goals, trust, and long-term goal orientation. The degrees of trust between organizations, learning intention, partner’s knowledge attribute, and organizational learning ability have great influence on knowledge transfer and interorganizational learning [43]. In order to realize the learning opportunities provided by the alliance, collaborators must attach importance to learning and consciously think about how to learn. In the process of learning, the learning ability of alliance enterprises is influenced by organizational knowledge transfer ability, acceptance ability, core competence, and past experience [44]. The transfer and accumulation of knowledge in interorganizational cooperation depend on the factors of learning process. These studies found that learning objectives, tasks, environments, and knowledge sharing about skills and processes are key process factors in interorganizational learning. However, knowledge exists in the specific organizational path of partnership enterprises, which has the characteristics of tacit and embedded [40–42]. Therefore, it is necessary to set reasonable learning objectives and arrange appropriate learning tasks and activities under appropriate circumstances, so as to complete the transfer and accumulation of knowledge. This paper constructs a criteria framework for the evaluation of enterprise learning performance in the process of cooperative innovation. Enterprise learning performance is divided into learning process performance and learning outcomes performance, so as to establish a practical criteria framework for the evaluation of enterprise learning performance, which is shown in Figure 1.

[figure omitted; refer to PDF]

As shown in Figure 1, the criterion framework for the evaluation of enterprise learning performance in the process of cooperative innovation includes learning process performance and learning outcomes performance, in which learning process performance is the learning outcomes of enterprises in cooperation, and learning outcomes performance is the core goal of enterprise core competence development [4, 13–15]. Learning process performance includes learning content, learning management, learning objectives, and learning processes, which affect the choice of learning methods and the formulation of learning mechanism in the process of cooperative innovation [6–8, 14–17]. Learning outcomes performance includes knowledge accumulation, ability accumulation, and performance creation [6–8]. Knowledge accumulation and performance creation mainly represent the outcomes of internal learning, while ability accumulation is the comprehensive outcome of internal and external learning. That is to say, technological progress could be achieved through internal knowledge acquisition and transformation, and internal management process could be optimized, which help to enhance external cooperative management capability. Learning process performance not only directly affects the internal learning outcomes of enterprises but also indirectly affects the external learning outcomes, thus affecting the core objectives of enterprises [10, 15–17, 20]. Learning outcomes performance could provide information feedback based on the rationality of learning content and learning management of enterprises [4, 6–8]. This also reflects the scientificity of the criterion framework established by the integrated system view and the rationality of the evaluation method proposed in this study for the evaluation of enterprise learning performance in the process of cooperative innovation.

The abovementioned criteria for the evaluation of enterprise learning performance in the process of cooperative innovation could be set as qualitative criteria, except for the quantitative criteria of technological progress. For quantitative criteria, there are many ways to calculate them, while quantification of qualitative criteria could generally be divided into two categories. The data acquisition of a class of criteria could be through questionnaires or the design of its numerical value, and this kind of criterion data could be obtained through the form of questionnaires, such as the appropriateness of learning methods, learning efficiency and so on. The data acquisition of the other criteria is difficult to determine by questionnaire or design. The data of these criteria could be obtained by the form of fuzzy linguistic information or fuzzy numbers of experts and enterprise managers, such as the improvement degree of organizational learning ability and the improvement degree of external cooperative management ability. In addition, with the rapid development of artificial intelligence and Internet, the cooperation among enterprises has been further deepened and developed, and the amount of information generated by cooperation has become more and more huge. Because of the complexity and fuzziness of learning performance evaluation, some evaluation criteria often fail to be given accurate evaluation values. With the introduction of fuzzy set theory, it has certain scientific and practical significance in dealing with fuzziness [45–52]. The method proposed in this study has paid attention to the practical application of the criterion evaluation process, so the above criteria could be quantified using fuzzy set theory.

3. Solution Methodology

3.1. Dimensionless Criteria and Distance Measurement of Fuzzy Numbers

In the process of cooperative innovation, the criteria for the evaluation of enterprise learning performance include qualitative and quantitative factors, and the dimensions of different attributes are different. In order to reduce the difference of evaluation results caused by different measurement units, it is necessary to deal with the criteria of learning performance evaluation in dimensionless way. According to [24, 28, 46–48, 53], different types of criteria have different dimensionless methods which are shown below.

If the evaluation values are clear numbers, many methods could be used to deal with it. For example, the range transformation method could be used to dimensionless quantitative attributes. Let $v_{ij}$ denote the clear number of criterion $j$ of the $i$ system. ${\alpha }_{ij}$ and ${\beta }_{ij}$ are the upper and lower limits of the order parameters of critical points of the learning evaluation system. Let $p_{ij}$ represent the contribution value of variable $v_{ij}$ to learning system, where $p_{ij}\in [\mathrm{0},\mathrm{1}]$. The dimensionless results of the criteria for the evaluation of enterprise learning performance could be expressed as follows:$$\begin{array}{}\text{(1)}& p_{ij}=\left\{\begin{array}{cc}\frac{\left(v_{ij}-{\beta }_{ij}\right)}{\left({\alpha }_{ij}-{\beta }_{ij}\right)}\hfill & \mathit{\text{Benefit\hspace{0.17em}\hspace{0.17em}criterion}}\hfill \\ \frac{\left({\alpha }_{ij}-v_{ij}\right)}{\left({\alpha }_{ij}-{\beta }_{ij}\right)}\hfill & \mathit{\text{Cost\hspace{0.17em}\hspace{0.17em}criterion}}.\hfill \end{array}\right.\end{array}$$

If the evaluation values are interval numbers $\left[x_{ijh}^L,x_{ijh}^M\right]$, the dimensionless results $y_{ijh}=\left[y_{ijh}^L,y_{ijh}^M\right]$ are assumed. Especially, the original data need to be reciprocated before standardization of cost-based criteria.

For benefit-oriented attributes, there is$$\begin{array}{}\text{(2)}& y_{ijh}=\left[y_{ijh}^L,y_{ijh}^M\right]=\left[\frac{x_{ijh}^L}{{\sum }_{i=\mathrm{1}}^n{x}_{ijh}^M},\frac{x_{ijh}^M}{{\sum }_{i=\mathrm{1}}^n{x}_{ijh}^L}\right].\end{array}$$

For cost attributes, there is$$\begin{array}{}\text{(3)}& y_{ijh}=\left[y_{ijh}^L,y_{ijh}^M\right]=\left[\frac{\left(\mathrm{1}/x_{ijh}^M\right)}{{\sum }_{i=\mathrm{1}}^n\left(\mathrm{1}/x_{ijh}^M\right)},\frac{\left(\mathrm{1}/x_{ijh}^L\right)}{{\sum }_{i=\mathrm{1}}^n\left(\mathrm{1}/x_{ijh}^L\right)}\right].\end{array}$$

If the evaluation values are linguistic values, the dimensionless processing of qualitative criteria requires quantitative conversion of qualitative evaluation values [53]. The basis of converting linguistic value into triangular fuzzy number is shown in Table 1, which is applied to the case study in this study. Language variables can be set to multiple levels. The level of variables can be set according to the demand of business managers who carry out learning performance evaluation. Five linguistic variables in Table 1 are only applied to the case study in this study.

Table 1

Fuzzy linguistic variable.

Linguistic variables	Abbreviations	TFNs
Very poor	VP	(0.0,0.0,0.2)
Poor	P	(0.0,0.2,0.4)
Medium	M	(0.2,0.4,0.6)
Medium good	MG	(0.4,0.6,0.8)
Good	G	(0.6,0.8,1.0)
Very good	VG	(0.8,1.0,1.0)

If the evaluation values are triangular fuzzy numbers $\left[x_{ijh}^L,x_{ijh}^M,x_{ijh}^N\right]$, the dimensionless results $y_{ijh}=\left[y_{ijh}^L,y_{ijh}^M,y_{ijh}^N\right]$ are assumed. Especially, the original data need to be reciprocated before standardization of cost-based criteria. For benefit-oriented attributes, there is$$\begin{array}{}\text{(4)}& y_{ijh}=\left[\frac{x_{ijh}^L}{{\mathrm{m}\mathrm{a}\mathrm{x}}_i{x}_{ijh}^N},\frac{x_{ijh}^M}{{\mathrm{m}\mathrm{a}\mathrm{x}}_i{x}_{ijh}^M},\frac{x_{ijh}^N}{{\mathrm{m}\mathrm{a}\mathrm{x}}_i{x}_{ijh}^L}\wedge \mathrm{1}\right].\end{array}$$

For cost attributes, there is$$\begin{array}{}\text{(5)}& y_{ijh}=\left[\frac{{\mathrm{m}\mathrm{i}\mathrm{n}}_i{x}_{ijh}^L}{x_{ijh}^N},\frac{{\mathrm{m}\mathrm{i}\mathrm{n}}_i{x}_{ijh}^M}{x_{ijh}^M},\frac{{\mathrm{m}\mathrm{i}\mathrm{n}}_i{x}_{ijh}^N}{x_{ijh}^L}\wedge \mathrm{1}\right].\end{array}$$

The distance and defuzzification formulas of the dimensionless fuzzy numbers, such as two interval numbers $\left[y_{ijh}^L,y_{ijh}^M\right]$ and $\left[z_{ijh}^L,z_{ijh}^M\right]$, are as follows:$$\begin{array}{}\text{(6)}& d\left(y_{ijh},z_{ijh}\right)=\frac{\sqrt{\mathrm{2}}}{\mathrm{2}}\sqrt{{\left(y_{ijh}^L-z_{ijh}^L\right)}^{\mathrm{2}}+{\left(y_{ijh}^M-z_{ijh}^M\right)}^{\mathrm{2}}},\text{(7)}& m\left(z_{ijh}\right)=\frac{\mathrm{1}}{\mathrm{2}}\left(z_{ijh}^L+z_{ijh}^M\right).\end{array}$$

For two triangular fuzzy numbers $\left[y_{ijh}^L,y_{ijh}^M,y_{ijh}^N\right]$ and $\left[z_{ijh}^L,z_{ijh}^M,z_{ijh}^N\right]$, the distance and the formula of defuzzification are as follows:$$\begin{array}{}\text{(8)}& d\left(y_{ijh},z_{ijh}\right)=\frac{\sqrt{\mathrm{3}}}{\mathrm{3}}\sqrt{{\left(y_{ijh}^L-z_{ijh}^L\right)}^{\mathrm{2}}+{\left(y_{ijh}^M-z_{ijh}^M\right)}^{\mathrm{2}}+{\left(y_{ijh}^N-z_{ijh}^N\right)}^{\mathrm{2}}},\text{(9)}& m\left(z_{ijh}\right)=\frac{\mathrm{1}}{\mathrm{4}}\left(z_{ijh}^L+\mathrm{2}{z}_{ijh}^M+z_{ijh}^N\right).\end{array}$$

3.2. Three-Parameter Weighted Heronian Mean Operator

Definition 1 (see [49]).

Let $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, and $p$, $q$ not take the value 0 simultaneously. $a_i\hspace{0.17em}\hspace{0.17em}\left(i=\mathrm{1,2},\dots ,n\right)$ is a collection of nonnegative numbers. If$$\begin{array}{}\text{(10)}& \mathrm{G}\mathrm{H}{\mathrm{M}}^{p,q}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)=\frac{\mathrm{1}}{p+q}\left({\displaystyle \prod }_{i=\mathrm{1},j=\mathrm{1}}^n{\left(p{a}_i+q{a}_j\right)}^{\mathrm{2}/n\left(n+\mathrm{1}\right)}\right),\end{array}$$then GHM is called the geometric Heronian mean operator.

Definition 2 (see [50]).

Let $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, and $p$, $q$ not take the value 0 simultaneously. $a_i\hspace{0.17em}\hspace{0.17em}\left(i=\mathrm{1,2},\dots ,n\right)$ is a collection of nonnegative numbers, and $w={\left(w_{\mathrm{1}},w_{\mathrm{2}},\dots ,w_n\right)}^{\mathrm{T}}$ is a weight vector, which satisfies $\mathrm{0}\le w_j\le \mathrm{1}$ and ${\sum }_{j=\mathrm{1}}^n{w}_j=\mathrm{1}$. If $$\begin{array}{}\text{(11)}& \mathrm{G}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{p,q}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)=\frac{\mathrm{1}}{{\sum }_{i=\mathrm{1}}^n{\sum }_{j=\mathrm{1}}^n{w}_i^p{w}_j^q}{\left(\sum _{i=\mathrm{1}}^n\sum _{j=\mathrm{1}}^n{\left(w_i{a}_i\right)}^p{\left(w_j{a}_j\right)}^q\right)}^{1/\left(p+q\right)},\end{array}$$then GWHM is called the generalized weighted Heronian mean operator.

Definition 3 (see [51]).

Let $a_i\hspace{0.17em}\hspace{0.17em}\left(i=\mathrm{1,2},\dots ,n\right)$ be a collection of nonnegative numbers, $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, and $r\ge \mathrm{0}$. If$$\begin{array}{}\text{(12)}& \mathrm{G}\mathrm{B}{\mathrm{M}}^{p,q,r}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)={\left(\frac{\mathrm{1}}{n\left(n-\mathrm{1}\right)\left(n-\mathrm{2}\right)}\sum _n{a}_i^p{a}_j^q{a}_k^r\right)}^{1/\left(p+q+r\right)},\end{array}$$then GBM is called the generalized Bonferroni mean operator.

Definition 4 (see [51]).

Let $a_i\hspace{0.17em}\hspace{0.17em}\left(i=\mathrm{1,2},\dots ,n\right)$ be a collection of nonnegative numbers, $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, $r\ge \mathrm{0}$, and $w={\left(w_{\mathrm{1}},w_{\mathrm{2}},\dots ,w_n\right)}^{\mathrm{T}}$ a weight vector, which satisfies $\mathrm{0}\le w_j\le \mathrm{1}$, ${\sum }_{j=\mathrm{1}}^n{w}_j=\mathrm{1}$. If$$\begin{array}{}\text{(13)}& \mathrm{G}\mathrm{W}\mathrm{B}{\mathrm{M}}^{p,q,r}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)={\left(\frac{\mathrm{1}}{n\left(n-\mathrm{1}\right)\left(n-\mathrm{2}\right)}\sum _{i,j,k=\mathrm{1}}^n{w}_i{w}_j{w}_k{a}_i^p{a}_j^q{a}_k^r\right)}^{1/\left(p+q+r\right)},\end{array}$$then GWBM is called the generalized weighted Bonferroni mean operator.

According to the above definition, a novel operator could be obtained; that is, if$$\begin{array}{}\text{(14)}& \mathrm{T}\mathrm{P}\mathrm{H}{\mathrm{M}}^{p,q,r}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)={\left(\frac{\mathrm{1}}{\lambda }\sum _{i=\mathrm{1}}^n\sum _{j=i}^n\hspace{0.17em}\sum _{k=j}^n{a}_i^p{a}_j^q{a}_k^r\right)}^{1/\left(p+q+r\right)}\end{array}$$where $\lambda ={\sum }_{l=\mathrm{1}}^n\left(l(l+\mathrm{1})/\mathrm{2}\right)$, $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, and $r\ge \mathrm{0}$, then TPHM is called the three-parameter Heronian mean operator [50].

Based on the above analysis, let $a_i\hspace{0.17em}\hspace{0.17em}\left(i=\mathrm{1,2},\dots ,n\right)$ be a collection of nonnegative numbers, $p\ge \mathrm{0}$, $q\ge \mathrm{0}$, $r\ge \mathrm{0}$, and $w={\left(w_{\mathrm{1}},w_{\mathrm{2}},\dots ,w_n\right)}^{\mathrm{T}}$ a weight vector, which satisfies $\mathrm{0}\le w_j\le \mathrm{1}$, ${\sum }_{j=\mathrm{1}}^n{w}_j=\mathrm{1}$. If$$\begin{array}{}\text{(15)}& \mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{p,q,r}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_n\right)={\left(\frac{\mathrm{1}}{\lambda }\sum _{i=\mathrm{1}}^n\sum _{j=i}^n\hspace{0.17em}\sum _{k=j}^n{\left(w_i{a}_i\right)}^p{\left(w_j{a}_j\right)}^q{\left(w_k{a}_k\right)}^r\right)}^{1/\left(p+q+r\right)},\end{array}$$where $\lambda ={\sum }_{i=\mathrm{1}}^n{\sum }_{j=i}^n{\sum }_{k=j}^n{w}_i^p{w}_j^q{w}_k^r$, then TPWHM is called the three-parameter weighted Heronian mean operator [50].

3.3. Time Sequence Weight Vector

Compared with evaluation methods of traditional performance, the influence of time factor should be considered in the process of performance evaluation in this study. In the process of general performance evaluation in reality, decision-makers do not have sufficient information, and the distribution of information is not clear. In order to fully reflect the timeliness of the attribute information for the evaluation of enterprise learning performance in the process of cooperative innovation, a time sequence weight vector is acquired based on the principle of “thick present, thin ancient” [45]. The connotation of this principle is that the closer the attribute information is, the stronger the timeliness is and the closer it is to the real value of the target. The larger the weight coefficient is, the more attention is paid to the new information and the more effective it is. Therefore, in this paper, time weight vectors are obtained by using the method of “thick, present and thin, ancient” based on time degree and information entropy.

Let $I=-{\sum }_{k=\mathrm{1}}^p{w}_k\ln w_k$ be an information entropy, which is used to objectively reflect the amount of information about learning performance evaluation attributes under different time series. The smaller the amount of information, the greater the information entropy [45].

Let $\lambda ={\sum }_{k=\mathrm{1}}^p\left(\left(p-k\right)/\left(p-\mathrm{1}\right)\right){\eta }_k$ be a time degree, where $\eta =\left\{{\eta }_{\mathrm{1}},{\eta }_{\mathrm{2}},\cdots ,{\eta }_p\right\}$. When $\lambda =\mathrm{0}$, it shows that the evaluators only pay attention to the information of the current moment, which is called the positive time weight vector. When $\lambda =\mathrm{1}$, it shows that the evaluators only pay attention to the oldest information, which is called negative time weight vector. When $\lambda =\mathrm{0.5}$, that is, $V=\left\{\mathrm{1}/p,\mathrm{1}/p,\cdots ,\mathrm{1}/p\right\}$, the evaluators pay equal attention to all time information. Based on the above analysis, the definition of time sequence weight vector is as follows.

Definition 5 (see [45]).

Under the condition of given time degree, the time weight ${\eta }_k\hspace{0.17em}\hspace{0.17em}\left(k=\mathrm{1,2},\cdots ,p\right)$ is determined by the criterion of time degree and information entropy maximization. The nonlinear programming model of time sequence weight vector is as follows:$$\begin{array}{c}\text{(16)}& \mathrm{m}\mathrm{a}\mathrm{x}\hspace{1em}I=-{\displaystyle \sum }_{k=\mathrm{1}}^p{w}_k\ln w_k\mathit{\text{s.t.}}\hspace{1em}\lambda ={\displaystyle \sum }_{k=\mathrm{1}}^p\frac{p-k}{p-\mathrm{1}}{\eta }_k,\hspace{1em}{\displaystyle \sum }_{k=\mathrm{1}}^p{\eta }_k=\mathrm{1},\hspace{0.17em}\hspace{0.17em}{\eta }_k\in \left[\mathrm{0,1}\right],\hspace{0.17em}\hspace{0.17em}k=\mathrm{1,2},\dots ,p.\end{array}$$

The time sequence weight vector could be obtained by solving the model based on the relevant software of MATLAB.

In order to fully reflect the advantages of Definition 5, a comparative analysis with Chen et al.’s [52] study is shown as follows.

The distance between two time weight vectors could be expressed as follows:$$\begin{array}{}\text{(17)}& d\left({\phi }^{\prime },{\phi }^{\prime \prime }\right)=\sqrt{{{\displaystyle \sum }}_{k=\mathrm{1}}^p{\left|{{\phi }_k}^{\prime }-{{\phi }_k}^{\prime \prime }\right|}^{\mathrm{2}}}.\end{array}$$

Then the distances between any time weight vector $\phi =\left\{{\phi }_{\mathrm{1}},{\phi }_{\mathrm{2}},\cdots ,{\phi }_p\right\}$ and positive and negative ideal time weight vectors are $d\left(\phi ,{\phi }^{+}\right)=\sqrt{{\sum }_{k=\mathrm{1}}^p{\phi }_k^{\mathrm{2}}+{\left(\mathrm{1}-{\phi }_p\right)}^{\mathrm{2}}}$ and $d\left(\phi ,{\phi }^{-}\right)=\sqrt{{\left(\mathrm{1}-{\phi }_{\mathrm{1}}\right)}^{\mathrm{2}}+{\sum }_{k=\mathrm{1}}^p{\phi }_k^{\mathrm{2}}}$. Thus, the approximation degree of the time weight vector to the ideal time weight vector can be expressed as$$\begin{array}{}\text{(18)}& C\left(\phi \right)=\frac{d\left(\phi ,{\phi }^{-}\right)}{d\left(\phi ,{\phi }^{+}\right)+d\left(\phi ,{\phi }^{-}\right)}.\end{array}$$

According to the idea of “thick past, thin present”, the time series weight vector of solving the nonlinear programming model can be obtained by maximizing $C\left(\phi \right)$ as far as possible under the given time scale. The nonlinear programming model can be expressed as$$\begin{array}{c}\text{(19)}& \mathrm{m}\mathrm{a}\mathrm{x}\hspace{1em}\begin{array}{c}\begin{array}{c}C\left(\phi \right)=\frac{\sqrt{{\left(\mathrm{1}-{\phi }_{\mathrm{1}}\right)}^{\mathrm{2}}+{\sum }_{k=\mathrm{2}}^p{\phi }_k^{\mathrm{2}}}}{\sqrt{{\sum }_{k=\mathrm{1}}^p{\phi }_k^{\mathrm{2}}+{\left(\mathrm{1}-{\phi }_p\right)}^{\mathrm{2}}}+\sqrt{{\left(\mathrm{1}-{\phi }_{\mathrm{1}}\right)}^{\mathrm{2}}+{\sum }_{k=\mathrm{2}}^p{\phi }_k^{\mathrm{2}}}}\end{array}\end{array}\mathit{\text{s.t.}}\hspace{1em}\lambda ={{\displaystyle \sum }}_{k=\mathrm{1}}^p\frac{p-k}{p-\mathrm{1}}{\phi }_k,\hspace{1em}{{\displaystyle \sum }}_{k=\mathrm{1}}^p{\phi }_k=\mathrm{1},\hspace{0.17em}\hspace{0.17em}{\phi }_k\in \left[\mathrm{0,1}\right].\end{array}$$

From the above analysis, we can see that when $p=\mathrm{4}$, the time weight vector calculated by the formula (19) method in [52] exists a case where the weight of a time point is zero, for example, when $p=\mathrm{4}$ and $\lambda =\mathrm{0.4}$; that is to say, the data comes from four time points and is used to evaluate enterprise learning performance according to four time points. The time weight vector calculated by the formula (16) method used in this study is ${\eta }_k\hspace{0.17em}\hspace{0.17em}\left(k=\mathrm{1,2},\mathrm{3,4}\right)=\left(\mathrm{0.1671,0.2133,0.2722,0.3474}\right)$. However, the time weight vector calculated by the formula (19) method in [52] is ${\phi }_k\hspace{0.17em}\hspace{0.17em}\left(k=\mathrm{1,2},\mathrm{3,4}\right)=\left(\mathrm{0.2549,0.2177,0.0000,0.5274}\right)$. Obviously, compared with the method in Chen et al.’s study, the method of obtaining time weight vector in this study has distinct advantages. The method in this study can not only reflect objectively the importance of different time points but also avoid the transition of subjective operation evaluation results. In addition, the dynamic comprehensive evaluation results of enterprise learning performance are more objective, authentic, and practical using this method.

3.4. Entropy Measure Method

Entropy weight method is a method to determine weight objectively. According to the basic principle of information theory, information is an ordered measure of the whole system, and entropy is a disordered measure of the whole system [28]. In the process of learning performance evaluation, the smaller the information entropy of evaluation criteria, the larger the amount of information represented by the criteria, the higher the weight. On the contrary, the opposite is true. Some basic steps of entropy measure method used in this paper are given as follows [28].

Step 1.

Standardize the original decision matrix according to the formulas (1)-(5).

Step 2.

Entropy of each criterion is calculated by$$\begin{array}{}\text{(20)}& e_j=-\frac{\mathrm{1}}{\ln m}{{\displaystyle \sum }}_{i=\mathrm{1}}^m{p}_{ij}\ln p_{ij},\hspace{1em}{p}_{ij}=\frac{r_{ij}}{{\sum }_{i=\mathrm{1}}^m{r}_{ij}},\hspace{0.17em}\hspace{0.17em}\mathrm{0}\ln \mathrm{0}=\mathrm{0}.\end{array}$$

Step 3.

The normalized criteria weight is computed by$$\begin{array}{}\text{(21)}& w_j=\frac{\mathrm{1}-e_j}{{\sum }_{j=\mathrm{1}}^n\left(\mathrm{1}-e_j\right)}.\end{array}$$

3.5. Solution Procedure

In this study, not only a three-parameter weighted Heronian mean operator considering the interaction between attributes is proposed, but also a time sequence weight vector based on time degree and information entropy is extended. Aiming at the problem of dynamic performance evaluation with completely unknown attribute weights and time weights, a dynamic method considering the interaction between attributes for the evaluation of enterprise learning performance in the process of cooperative innovation is proposed in this paper. The specific steps are as follows.

Step 1.

In view of the dynamic evaluation for enterprise learning performance, relevant experts on cooperative innovation and interenterprise cooperation are invited and then determine different time series. Each expert evaluates the values of learning performance attributes under different time series. After many rounds of comprehensive feedback and experts’ evaluation, the results of experts’ evaluation are consistent. The fuzzy evaluation matrix $G\left(t_k\right)={\left(g_{ij}\left(t_k\right)\right)}_{m\times n}$ of learning performance under different time series is obtained.

Step 2.

Determine the entropy weight $w_j$ of the $j$ attribute $c_j$ under the $k$ periods based on entropy weight method of measure information in (20) and (21).

Step 3.

Aggregate learning performance attribute information using the formula (15) considering the interaction between attributes, and the comprehensive values $a_i^k$ of learning performance $T_i$ under the $k$ periods are obtained.

Step 4.

According to the suggestions of enterprise cooperation managers and experts, a scientific and reasonable time parameter $\lambda$ is set to solve the nonlinear programming equation (16), and the time weight vector ${\eta }_k$ of the $k$ periods is obtained.

Step 5.

Aggregate the comprehensive information of learning performance under different periods based on the formula (15); the comprehensive evaluation value $Z$ of learning performance $T_i$ under $k$ periods is obtained.

Step 6.

Analyse the comprehensive value $a_i^k$ and $Z$ of enterprise learning performance in the process of cooperative innovation under different periods, so as to find out the learning deficiency in enterprise cooperation, and then enterprise cooperation managers adjust the learning content and learning management.

The solution procedure for the evaluation of enterprise learning performance in the process of cooperative innovation could be shown in Figure 2 including associated techniques. Major steps include (a) structuring a criteria framework to evaluate enterprise learning performance, (b) determining the weight vector $w_j$ of the $j$ attribute $c_j$, (c) aggregating learning performance attribute information, (d) calculating the time weight vector ${\eta }_k$ of the $k$ periods, (e) obtaining the comprehensive evaluation values, (f) analyzing the comprehensive values $a_i^k$ and $Z$ of enterprise learning performance, and (g) adjusting the learning content and learning management.

[figure omitted; refer to PDF]

4. Case Study

4.1. Case Background

ZX company is a medium-sized enterprise in the information and communication technology industry, founded in 2005, located in Beijing, China. ZX company is mainly engaged in research and development (R & D), design, production, and operation of communication equipment and other products. The main products include communication network wiring, information cabinet products, and medical information products. Communication network wiring and information cabinet products include ODN products, optical devices products, wireless access products, and information cabinet products. There are more than 20 kinds of products, which are widely used in communication network, cloud platform IDC room, railway communication network, and urban rail transit communication network. The main customers include China Telecom, China Mobile, China Unicom, Railway Communications Corporation, and Radio and Television Corporation. In the field of information equipment, the company is committed to providing complete medical information solutions and equipment to hospitals.

ZX company has certain advantages in innovation resources of communication equipment, but its ability of R & D, design, and operation status is general in the field of information and communication. At present, ZX company has been cooperating with HW, which is a large enterprise in the field of information and communication for five years. In the process of cooperative innovation, ZX company’s internal processes level, management level, and product R & D, design and operation have improved to a certain extent, but compared with enterprises in the same situation in the industry, there is still a certain gap. At the same time, with the personalized demand of customers, the trend of digital transformation is becoming more and more prominent, and ZX company is facing enormous pressure of customer-oriented communication equipment product innovation. In order to improve the customer service level, technological innovation ability, and market competitiveness of enterprises, ZX company decided to adjust the innovation cooperation with HW enterprise, which involves the adjustment of learning from HW enterprise. For this reason, ZX company needs to carry out a dynamic comprehensive evaluation of learning performance in the process of cooperative innovation with HW enterprise, so as to find out the deficiencies of enterprise learning in the process of cooperative innovation and provide practical guidance for adjusting the learning planning of the next innovation cooperation project.

4.2. Evaluation of Enterprise ZX Learning Performance

Based on the criterion framework for the evaluation of enterprise learning performance in the process of cooperative innovation, the TPWHM operator considering the interaction between attributes, time entropy, and entropy weight method are applied to evaluate the learning performance of ZX company. In order to pre-evaluate the learning performance, five subcriteria in the criteria framework as shown in Figure 1 are applied for fuzzy evaluation, and the practice of learning performance evaluation is carried out as follows.

Step 1.

Five subcriteria in the criteria framework are applied by ZX company. Ten evaluation experts in the fields of cooperative innovation and interenterprise cooperation are invited by ZX company. Five different time series are determined by experts according to the cooperation time; that is, each year is regarded as a time series. The attributes of learning performance are evaluated under different time series. Learning performance evaluation criteria are learning content (language variables), learning management (language variables), knowledge accumulation (interval number), ability accumulation (language variables), and performance creation (explicit number). After several rounds of comprehensive feedback and cooperative expert evaluation, the results of expert evaluation tend to be consistent. Five fuzzy evaluation matrices of learning performance attribute information under different time series are obtained as shown in Table 2, and the evaluation matrices after standardization and defuzzification are shown in Table 3.

Table 2

Evaluation matrices of learning performance under different time series.

	C1	C2	C3	C4	C5
T1	G	P	(0.35,0.45)	M	0.35
T2	MG	M	(0.45,0.58)	G	0.45
T3	VP	VG	(0.74,0.85)	G	0.50
T4	M	VG	(0.55,0.65)	VG	0.55
T5	VG	P	(0.60,0.77)	M	0.65

Table 3

Evaluation matrices after standardization and defuzzification.

	C1	C2	C3	C4	C5
T1	0.288	0.078	0.140	0.120	0.538
T2	0.213	0.153	0.180	0.240	0.692
T3	0.018	0.360	0.270	0.240	0.769
T4	0.140	0.360	0.205	0.285	0.846
T5	0.343	0.078	0.235	0.120	1.000

Step 2.

The entropy weight $w_j$ of the $j$ attribute $c_j$ under the $k$ periods is determined based on entropy weight method of measure information in (20) and (21) and is as follows:$$\begin{array}{}\text{(22)}& w_j^{\mathrm{1}-\mathrm{5}}=\left(\mathrm{0.1637},\mathrm{0.1653},\mathrm{0.1654},\mathrm{0.1638},\mathrm{0.3418}\right).\end{array}$$

Step 3.

Learning performance attribute information is aggregated using the formula (15) considering the interaction between attributes, and the comprehensive values $a_i^k$ of learning performance $T_i$ under the $k$ periods are calculated as follows:$$\begin{array}{c}\text{(23)}& \mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_{\mathrm{5}}\right)={\left(\frac{\mathrm{1}}{\lambda }\sum _{i=\mathrm{1}}^{\mathrm{5}}\sum _{j=i}^{\mathrm{5}}\hspace{0.17em}\sum _{k=j}^{\mathrm{5}}{\left(w_i{a}_i\right)}^{\mathrm{1}}{\left(w_j{a}_j\right)}^{\mathrm{1}}{\left(w_k{a}_k\right)}^{\mathrm{1}}\right)}^{1/3}\text{and\hspace{0.17em}\hspace{0.17em}}\lambda ={\displaystyle \sum }_{i=\mathrm{1}}^{\mathrm{5}}{\displaystyle \sum }_{j=i}^{\mathrm{5}}\hspace{0.17em}{\displaystyle \sum }_{k=j}^{\mathrm{5}}{w}_i^{\mathrm{1}}{w}_j^{\mathrm{1}}{w}_k^{\mathrm{1}}=\mathrm{0.2985}.\end{array}$$Then $$\begin{array}{c}\text{(24)}& \mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_{\mathrm{1}}\right)={\left(\frac{\begin{array}{c}\mathrm{0.0114}\end{array}}{\begin{array}{c}\mathrm{0.2985}\end{array}}\right)}^{1/3}=\mathrm{0.3364},\mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_{\mathrm{2}}\right)={\left(\frac{\begin{array}{c}\begin{array}{c}\begin{array}{c}\mathrm{0.0236}\end{array}\end{array}\end{array}}{\begin{array}{c}\mathrm{0.2985}\end{array}}\right)}^{1/3}=\mathrm{0.4294},\\ \mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_{\mathrm{3}}\right)={\left(\frac{\begin{array}{c}\begin{array}{c}\mathrm{0.0332}\end{array}\end{array}}{\begin{array}{c}\mathrm{0.2985}\end{array}}\right)}^{1/3}=\mathrm{0.4809},\\ \mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_{\mathrm{4}}\right)={\left(\frac{\begin{array}{c}\begin{array}{c}\mathrm{0.0442}\end{array}\end{array}}{\begin{array}{c}\mathrm{0.2985}\end{array}}\right)}^{1/3}=\mathrm{0.5289},\\ \text{and}\hspace{0.17em}\hspace{0.17em}\mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_{\mathrm{5}}\right)={\left(\frac{\begin{array}{c}\begin{array}{c}\mathrm{0.0593}\end{array}\end{array}}{\begin{array}{c}\mathrm{0.2985}\end{array}}\right)}^{1/3}=\mathrm{0.5834}.\end{array}$$

Step 4.

According to the suggestions of enterprise cooperation managers and experts, in order to reflect the importance of recent data, the time degree is set to 0.4 and the nonlinear programming equation (16) is solved. The time weight of the $k\hspace{0.17em}\hspace{0.17em}\left(k=\mathrm{1,2},\cdots ,\mathrm{5}\right)$ period is obtained as follows:$$\begin{array}{}\text{(25)}& {\eta }_k\left(k=\mathrm{1,2},\cdots ,\mathrm{5}\right)=\left(\mathrm{0.1277,0.1566,0.1920,0.2353,0.2884}\right).\end{array}$$

Step 5.

Aggregate the comprehensive information of learning performance under different periods based on the formula (15), and the comprehensive evaluation value $Z$ of learning performance $T_i$ under $k$ periods is computed as follows:$$\begin{array}{}\text{(26)}& Z={\displaystyle \sum }_{i=k=\mathrm{1}}^{\mathrm{5}}\mathrm{T}\mathrm{P}\mathrm{W}\mathrm{H}{\mathrm{M}}_w^{\mathrm{1,1},\mathrm{1}}\left(T_i\right){\eta }_k=0.4952.\end{array}$$

Step 6.

Analyse the comprehensive value $a_i^k$ and $Z$ of enterprise learning performance in the process of cooperative innovation under different periods. From Step 3, learning performance of ZX company shows an upward trend as shown in Figure 3.

[figure omitted; refer to PDF]

From Figure 3, although the learning performance of ZX company has increased from 0.3364 to 0.5834 in the five years of innovation cooperation with HW, it is still lower than the passing level. The reason is that ZX company has improved its learning content, knowledge accumulation, ability accumulation, and performance creation by cooperating with HW. The effect of cooperative innovation on the improvement of learning management level is not obvious. The comprehensive evaluation value of learning performance is only 0.4952, which is at a low level. Through interviews with the executive managers of the cooperative innovation projects of ZX company, the actual cooperative learning situation is basically consistent with the results of this evaluation. Therefore, ZX company should adjust learning styles and develop learning mechanism to further improve learning performance.

4.3. Comparative Analysis of Evaluation Results

In order to verify the validity and scientificity of the approach proposed in this study, it is compared with the integration operators which assume that the attributes are independent of each other. OWA operator, OWG operator, and OWH operator are used to evaluate the learning performance of ZX company, respectively. The evaluation results are compared with the result of the TPWHM operator proposed in this study, which considers the interaction between attributes. The evaluation results are shown in Table 4.

Table 4

The evaluation results of learning performance under different integration operators.

	OWA operator	OWG operator	OWH operator
T1	0.0262	0.0201	0.0164
T2	0.0518	0.0439	0.0396
T3	0.0795	0.0377	0.0138
T4	0.0511	0.0412	0.0350
T5	0.0719	0.0458	0.0332
Z	0.2805	0.1887	0.1379

From the results of Step 5 and Table 4, it can be seen that TPWHM operator considering the interaction between attributes, OWA operator, OWG operator, and OWH operator are, respectively, used to integrate learning performance attributes information. The results of dynamic comprehensive evaluation of learning performance obtained by these operators are different.

In the aspect of the changing trends, the changing trends of learning performance evaluation results using TPWHM operator, OWA operator, OWG operator, and OWH operator are shown in Figures 3 and 4. As shown in Figures 3 and 4, it can be seen that the changing trends of learning performance of ZX company evaluated by TPWHM operator, OWA operator, OWG operator, and OWH operator are different. The evaluation results based on TPWHM operator tend to increase gradually, while the results evaluated by the other operators show certain fluctuation, which changes greatly, and the evaluation results of learning performance are unstable. Through interviews with the executive managers of the cooperative projects of ZX company, the actual cooperative learning situation is basically consistent with the results using TPWHM operator. It is more stable and scientific to use TPWHM operator considering the interaction between attributes to evaluate learning performance.

[figure omitted; refer to PDF]

In the aspect of evaluation values, the results evaluated by TPWHM operator considering the interaction between attributes, OWA operator, OWG operator, and OWH operator are, respectively, 0.4592, 0.2805, 0.1887, and 0.1379. The results evaluated by TPWHM operator are better than those based on OWA operator, OWG operator, and OWH operator. The reason is that TPWHM operator considers the interaction between attributes, while OWA operator, OWG operator, and OWH operator assume that attributes are independent when attribute information is integrated. Therefore, the evaluation results in this study are closer to the actual evaluation values and in accord with the actual results of learning performance evaluation.

5. Conclusions

In order to judge whether cooperative goals are achieved and manage cooperative relationships among enterprises, the scientific and reasonable evaluation of learning performance in the process of cooperative innovation has become an important issue. Therefore, from the perspective of system view, this paper constructs a comprehensive and representative criterion framework for the evaluation of enterprise learning performance taking learning process performance and learning outcomes performance as the core ideas in the process of cooperative innovation. In order to consider the interaction between attributes of learning performance and determine a scientific and reasonable time series weight vector, a dynamic comprehensive evaluation approach for evaluating learning performance is proposed to improve the validity and stability of the results of group dynamic comprehensive evaluation.

In this study, the criterion framework for the evaluation of enterprise learning performance in the process of cooperative innovation includes learning process performance and learning outcomes performance. Learning process performance reflects the comprehensive level of learning content and learning management, and learning outcomes performance is the core goal of enterprise core competence development. Learning process performance affects the choice of learning methods and the formulation of learning mechanism in the process of cooperative innovation. Learning outcomes performance is the comprehensive outcome of internal and external learning. That is to say, technological progress could be achieved through internal knowledge acquisition and transformation and internal management process could be optimized, which help to enhance external cooperative management capability. Learning process performance not only directly affects the internal learning outcomes of enterprises but also indirectly affects the external learning outcomes, thus affecting the core objectives of enterprises. Learning outcomes performance could provide information feedback for the rationality of learning content and learning management.

The original matrices are obtained using fuzzy set theory, and time sequence weight vector is calculated based on information entropy and time degree. The weight of learning performance attributes under different time series is calculated based on entropy measure method. The interactive information of learning performance attributes is integrated through the weight of learning performance attributes and the TPWHM operator considering the interaction between attributes. And then, the dynamic and comprehensive evaluation result of learning performance in the process of cooperative innovation could be computed by integrating the learning performance information under different time series with time sequence weight vector. Finally, a real case is studied to verify the scientificity and validity of the criteria framework for enterprise learning performance and the method proposed in this study.

5.1. Management Implications

The criteria framework and dynamic evaluation method considering the interaction between attributes proposed in this study are of both theoretical and practical significance. On the one hand, the criterion framework for the evaluation of enterprise learning performance proposed in this study fully embodies the core objective of learning in the process of cooperative innovation. The criterion framework based on learning process performance and learning outcomes performance has theoretical and practical significance. On the other hand, the dynamic evaluation method of enterprise learning performance fully considers the interaction of complementarity, redundancy, and preference among learning performance attributes under different time series, which accords with the reality where attributes are often interrelated to each other in different degrees. The time sequence weight vector based on information entropy and time degree is more stable, which avoids the unreasonable situation of information integration, and makes the dynamic comprehensive evaluation results more realistic.

The practical contribution of this study is that the method and evaluation criteria framework proposed in this study not only help to improve the effectiveness and stability of the comprehensive evaluation results of enterprise learning performance in the process of cooperative innovation but also help cooperative enterprises get feedback in time and adjust cooperative relationships and learning styles. In theory, this paper proposes a comprehensive and representative criteria framework for the evaluation of enterprise learning performance in the process of cooperative innovation. This framework not only provides a theoretical basis for the study of factors affecting interorganizational learning and performance evaluation but also enriches the theory of interorganizational management and provides a theoretical basis for the process of enterprise cooperative innovation. In addition, the novel method proposed in this study expands the application scope of fuzzy theory and time series in learning performance evaluation.

5.2. Limitations and Future Work

The scope of this study is the evaluation of enterprise learning performance in the process of cooperative innovation. In this study, the criteria framework for enterprise learning performance is constructed taking the learning process and learning outcomes as the core idea, and a novel dynamic evaluation approach considering the interaction between attributes of learning performance is proposed. This paper not only helps cooperative enterprises get feedback in time and adjust cooperative relationships and learning styles but also enriches the theory of interorganizational management and provides a theoretical basis for the process of enterprise cooperative innovation. Although the research goal of this paper was achieved, there are still some limitations which deserve the attention of future research. First of all, the influential criteria should be extended with the change of different types and different forms of cooperative innovation. Secondly, the weight method should be developed based on subjective and objective combination weighting method. In addition, artificial intelligence technology is gradually applied to multicriteria evaluation problems and plays an important role in enlightenment of performance evaluation approach in the future.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research was supported by the Fundamental Research Funds for the Central Universities (HEUCFW170901).

References

[1] F. Wei, W. Chan, "The cooperative stability evolutionary game analysis of the military-civilian collaborative innovation for China’s satellite industry," Mathematical Problems in Engineering, vol. 2019,DOI: 10.1155/2019/3938716, 2019.

[2] S. Yin, B. Z. Li, "A stochastic differential game of low carbon technology sharing in collaborative innovation system of superior enterprises and inferior enterprises under uncertain environment," Open Mathematics, vol. 16 no. 1, pp. 607-622, DOI: 10.1515/math-2018-0056, 2018.

[3] J. Gibb, A. Sune, S. Albers, "Network learning: episodes of interorganizational learning towards a collective performance goal," European Management Journal, vol. 35 no. 1, pp. 15-25, DOI: 10.1016/j.emj.2016.09.001, 2017.

[4] L. Dai, X. Q. Zhi, "Could firms learning process and outcome based on inter-firm cooperation be effectively evaluated?: the approach of performance evaluation theory," Accounting Research, vol. 45 no. 12, pp. 45-52, 2015.

[5] G. Santoro, D. Vrontis, A. Thrassou, L. Dezi, "The Internet of Things: building a knowledge management system for open innovation and knowledge management capacity," Technological Forecasting & Social Change, vol. 136 no. 11, pp. 347-354, DOI: 10.1016/j.techfore.2017.02.034, 2018.

[6] T. Y. Jung, J. Kwak, L. Jiang, "The role of distance in inter-firm learning for global R and D alliances," International Journal of Applied Business and Economic Research, vol. 14 no. 1, pp. 191-208, 2016.

[7] P. J. Lane, M. Lubatkin, "Relative absorptive capacity and interorganizational learning," Strategic Management Journal, vol. 19 no. 5, pp. 461-477, DOI: 10.1002/(SICI)1097-0266(199805)19:5<461::AID-SMJ953>3.3.CO;2-C, 1998.

[8] A. E. Şengün, "Which type of trust for inter-firm learning?," Industry & Innovation, vol. 17 no. 2, pp. 193-213, DOI: 10.1080/13662711003633447, 2010.

[9] J. Kratzer, D. Meissner, V. Roud, "Open innovation and company culture: internal openness makes the difference," Technological Forecasting & Social Change, vol. 119 no. 6, pp. 128-138, DOI: 10.1016/j.techfore.2017.03.022, 2017.

[10] B. W. Akbar, "Relationship learning through inter-firm conduits in Finnish small and medium enterprises," Journal of Small Business and Entrepreneurship, vol. 31 no. 6, 2018.

[11] M. Modak, K. Pathak, K. K. Ghosh, "Performance evaluation of outsourcing decision using a BSC and fuzzy AHP approach: a case of the Indian coal mining organization," Resources Policy, vol. 52 no. 6, pp. 181-191, DOI: 10.1016/j.resourpol.2017.03.002, 2017.

[12] I.-H. Petersen, G. Kruss, M. Gastrow, P. C. Nalivata, "Innovation capacity-building and inclusive development in informal settings: a comparative analysis of two interactive learning spaces in South Africa and Malawi," Journal of International Development, vol. 30 no. 5, pp. 865-885, DOI: 10.1002/jid.3232, 2018.

[13] R. Kellogg, "Learning by drilling: interfirm learning and relationship persistence in the Texas oilpatch," The Quarterly Journal of Economics, vol. 126 no. 4, pp. 1961-2004, DOI: 10.1093/qje/qjr039, 2011.

[14] A. E. Şengün, Ç. Önder, "The conditional impact of competence trust on inter-firm learning in a collectivist SME context," Industry and Innovation, vol. 18 no. 8, pp. 791-812, DOI: 10.1080/13662716.2011.621746, 2011.

[15] S. Gupta, M. Polonsky, "Inter-firm learning and knowledge-sharing in multinational networks: an outsourced organization's perspective," Journal of Business Research, vol. 67 no. 4, pp. 615-622, DOI: 10.1016/j.jbusres.2013.02.043, 2014.

[16] M. J. Saenz, E. Ubaghs, A. I. Cuevas, Enabling Horizontal Collaboration through Continuous Relational Learning, 2015.

[17] A. Lis, A. Sudolska, "Inter- and intra-firm learning synergy through integrating absorptive capacity and employee suggestion processes: the case study of Frauenthal Automotive Toruń company," Journal of Entrepreneurship, Management and Innovation, vol. 13 no. 1, pp. 25-67, DOI: 10.7341/20171312, 2017.

[18] C. D. Ittner, D. F. Larcker, "Innovations in performance measurement: trends and research implications," Journal of Management Accounting Research, vol. 10 no. 10, pp. 205-238, 1998.

[19] M. Hemmert, "The relevance of inter-personal ties and inter-organizational tie strength for outcomes of research collaborations in South Korea," Asia Pacific Journal of Management,DOI: 10.1007/s10490-017-9556-6, 2018.

[20] X. Q. Zhi, L. Dai, "Theoretical development of inter-firm performance evaluation and improvement of balance scorecard: based on strategic alliance scenario," Accounting Research, vol. 42 no. 4, pp. 81-88, 2012.

[21] S. Kusi-Sarpong, H. Gupta, J. Sarkis, "A supply chain sustainability innovation framework and evaluation methodology," International Journal of Production Research, vol. 57 no. 7, pp. 1990-2008, DOI: 10.1080/00207543.2018.1518607, 2018.

[22] M.-L. Song, R. Fisher, J.-L. Wang, L.-B. Cui, "Environmental performance evaluation with big data: theories and methods," Annals of Operations Research, vol. 270 no. 1-2, pp. 459-472, DOI: 10.1007/s10479-016-2158-8, 2018.

[23] R. Arbolino, L. De Simone, F. Carlucci, T. Yigitcanlar, G. Ioppolo, "Towards a sustainable industrial ecology: implementation of a novel approach in the performance evaluation of Italian regions," Journal of Cleaner Production, vol. 178 no. 3, pp. 220-236, DOI: 10.1016/j.jclepro.2017.12.183, 2018.

[24] Y. Kazancoglu, I. Kazancoglu, M. Sagnak, "Fuzzy DEMATEL-based green supply chain management performance: application in cement industry," Industrial Management & Data Systems, vol. 118 no. 2, pp. 412-431, DOI: 10.1108/IMDS-03-2017-0121, 2018.

[25] W. D. Cook, N. Ramón, J. L. Ruiz, I. Sirvent, J. Zhu, "DEA-based benchmarking for performance evaluation in pay-for-performance incentive plans," Omega, vol. 84 no. 4, pp. 45-54, DOI: 10.1016/j.omega.2018.04.004, 2019.

[26] Z. H. Luo, Y. F. Yang, H. Z. Pan, M. F. Zhong, "Research on performance evaluation system of shale gas PPP project based on matter element analysis," Mathematical Problems in Engineering, vol. 2018,DOI: 10.1155/2018/4657383, 2018.

[27] C. Malings, R. Tanzer, A. Hauryliuk, S. P. Kumar, N. Zimmerman, L. B. Kara, A. A. Presto, R. Subramanian, "Development of a general calibration model and long-term performance evaluation of low-cost sensors for air pollutant gas monitoring," Atmospheric Measurement Techniques, vol. 12 no. 2, pp. 903-920, DOI: 10.5194/amt-12-903-2019, 2019.

[28] B. M. Dos Santos, L. P. Godoy, L. M. S. Campos, "Performance evaluation of green suppliers using entropy-TOPSIS-F," Journal of Cleaner Production, vol. 207 no. 1, pp. 498-509, DOI: 10.1016/j.jclepro.2018.09.235, 2019.

[29] A. Moldavska, T. Welo, "A Holistic approach to corporate sustainability assessment: incorporating sustainable development goals into sustainable manufacturing performance evaluation," Journal of Manufacturing Systems, vol. 50 no. 1, pp. 53-68, DOI: 10.1016/j.jmsy.2018.11.004, 2019.

[30] H. S. Lee, Y. Choi, "Environmental performance evaluation of the Korean manufacturing industry based on sequential DEA," Sustainability, vol. 11 no. 3,DOI: 10.3390/su11030874, 2019.

[31] A. Basso, F. Casarin, S. Funari, "How well is the museum performing? A joint use of DEA and BSC to measure the performance of museums," OMEGA - The International Journal of Management Science, vol. 81 no. 12, pp. 67-84, 2018.

[32] Z. Babaei, F. Shahveisi, "Studying the information content, economic value added, liquidity, and activity in market value added determination," International Journal of Economic Perspectives, vol. 11 no. 1, pp. 843-850, 2017.

[33] J. Lu, L. G. Zhou, H. Y. Chen, "Approach to the performance evaluation of bank employees based on the C-POWA operator," Operations Research and Management Science, vol. 25 no. 4, pp. 227-233, 2016.

[34] J. Huang, W. Liang, W. Cheng, L. Hui, "Study on the comprehensive evaluation of financial sustainable development with linguistic information," Advances in Information Sciences and Service Sciences, vol. 4 no. 4, pp. 237-243, 2012.

[35] Z.-X. Su, M.-Y. Chen, G.-P. Xia, L. Wang, "An interactive method for dynamic intuitionistic fuzzy multi-attribute group decision making," Expert Systems with Applications, vol. 38 no. 12, pp. 15286-15295, DOI: 10.1016/j.eswa.2011.06.022, 2011.

[36] J. H. Park, H. J. Cho, Y. C. Kwun, "Extension of the VIKOR method to dynamic intuitionistic fuzzy multiple attribute decision making," Computers & Mathematics with Applications, vol. 65 no. 4, pp. 731-744, DOI: 10.1016/j.camwa.2012.12.008, 2013.

[37] X. Cao, G. W. Liu, X. M. Fu, "Analysis on group decision making of industry-university-research partners choice based on preferences and dynamic intuition," Operations Research and Management Science, vol. 22 no. 4, pp. 33-41, 2013.

[38] J. Hartley, L. Rashman, "Innovation and inter-organizational learning in the context of public service reform," International Review of Administrative Sciences, vol. 84 no. 2, pp. 231-248, DOI: 10.1177/0020852318762309, 2018.

[39] M. Le Pennec, E. Raufflet, "Value creation in inter-organizational collaboration: an empirical study," Journal of Business Ethics, vol. 148 no. 4, pp. 817-834, DOI: 10.1007/s10551-015-3012-7, 2018.

[40] Q. Zhu, H. Krikke, M. C. J. Caniëls, "Supply chain integration: value creation through managing inter-organizational learning," International Journal of Operations and Production Management, vol. 38 no. 1, pp. 211-229, DOI: 10.1108/IJOPM-06-2015-0372, 2018.

[41] J. M. Zhang, H. Jiang, R. Wu, J. Z. Li, "Reconciling the dilemma of knowledge sharing: a network pluralism framework of firms' R and D alliance network and innovation performance," Journal of Management,DOI: 10.1177/0149206318761575, 2018.

[42] S. Yayavaram, M. K. Srivastava, M. B. Sarkar, "Role of search for domain knowledge and architectural knowledge in alliance partner selection," Strategic Management Journal, vol. 39 no. 8, pp. 2277-2302, DOI: 10.1002/smj.2791, 2018.

[43] J. B. Gu, Z. Chen, Q. Huang, H. F. Liu, S. L. Huang, "A multilevel analysis of the relationship between shared leadership and creativity in inter-organizational teams," Journal of Creative Behavior, vol. 52 no. 2, pp. 109-126, DOI: 10.1002/jocb.135, 2018.

[44] O. Serrat, "Learning in strategic alliances," Knowledge Solutions, 2017.

[45] S. Yin, B. Z. Li, H. M. Dong, "A novel dynamic multi-attribute decision-making method based on the improved weights function and score function, and its application," Journal of Intelligent & Fuzzy Systems, vol. 35 no. 6, pp. 6217-6227, DOI: 10.3233/JIFS-171505, 2018.

[46] N. T. K. Son, "A foundation on semigroups of operators defined on the set of triangular fuzzy numbers and its application to fuzzy fractional evolution equations," Fuzzy Sets and Systems, vol. 347 no. 9,DOI: 10.1016/j.fss.2018.02.003, 2018.

[47] S. Yin, B. Z. Li, "Matching management of supply and demand of green building technologies based on a novel matching method with intuitionistic fuzzy sets," Journal of Cleaner Production, vol. 201 no. 11, pp. 748-763, DOI: 10.1016/j.jclepro.2018.08.055, 2018.

[48] Y. Liu, G. T. Shen, Z. Y. Zhao, Z. W. Wu, "A new model for deriving the priority weights from hesitant triangular fuzzy preference relations," Mathematical Problems in Engineering, vol. 2019,DOI: 10.1155/2019/8586592, 2019.

[49] P. D. Liu, T. Mahmood, Q. Khan, "Group decision making based on power Heronian aggregation operators under linguistic neutrosophic environment," International Journal of Fuzzy Systems, vol. 20 no. 3, pp. 970-985, DOI: 10.1007/s40815-018-0450-2, 2018.

[50] W. F. Liu, J. Chang, Y. X. Du, "(Linguistic) Heronian mean operators and applications in decision making," Chinese Journal of Management Science, vol. 25 no. 4, pp. 174-183, 2017.

[51] P. D. Liu, J. Liu, S.-M. Chen, "Some intuitionistic fuzzy Dombi Bonferroni mean operators and their application to multi-attribute group decision making," Journal of the Operational Research Society, vol. 69 no. 1,DOI: 10.1057/s41274-017-0190-y, 2018.

[52] W. Chen, Z. L. Yang, W. Zhou, H. Chen, "Dynamic intuitionistic fuzzy compromise decision making method based on time degrees," Operations Research and Management Science, vol. 25 no. 4, pp. 83-89, 2016.

[53] M.-L. Tseng, K.-J. Wu, A. S. Chiu, M. K. Lim, K. Tan, "Service innovation in sustainable product service systems: improving performance under linguistic preferences," International Journal of Production Economics, vol. 203 no. 9, pp. 414-425, DOI: 10.1016/j.ijpe.2018.07.020, 2018.