The Cognition of Audience to Artistic Style Transfer

Abstract

Artificial Intelligence (AI) is becoming more popular in various fields, including the area of art creation. Advances in AI technology bring new opportunities and challenges in the creation, experience, and appreciation of art. The neural style transfer (NST) realizes the intelligent conversion of any artistic style using neural networks. However, the artistic style is the product of cognition that involving from visual to feel. The purpose of this paper is to study factors affecting audience cognitive difference and preference on artistic style transfer. Those factors are discussed to investigate the application of the AI generator model in art creation. Therefore, based on the artist’s encoding attributes (color, stroke, texture) and the audience’s decoding cognitive levels (technical, semantic, effectiveness), this study proposed a framework to evaluate artistic style transfer in the perspective of cognition. Thirty-one subjects with a background in art, aesthetics, and design were recruited to participate in the experiment. The experimental process consists of four style groups, including Fauvism, Expressionism, Cubism, and Renaissance. According to the finding in this study, participants can still recognize different artistic styles after transferred by neural networks. Besides, the features of texture and stroke are more impact on the perception of fitness than color. The audience may prefer the samples with high cognition in the semantic and effectiveness levels. The above indicates that through AI automated routine work, the cognition of the audience to artistic style still can be kept and transferred.

1. Introduction

In recent years, more significant progress has been made in Artificial Intelligence (AI) research, including the area of art creation. The continuous development of technology has promoted increasing interaction between art and AI. The neural network as a popular machine learning method (ML) can also be applied in artistic stylization, called neural style transfer (NST). There have been many methods proposed by computer science researchers to get a better conversion effect. However, when methods of NST are adopted to transfer artistic style, evaluating their results is so complex that only employing quantitative methods commonly used in the computer graphics community is not enough [1,2]. So far, there are few evaluations of machine learning artistic style from human cognitive factors. The artwork encodes from the artist’s inner expression to the outer form, while the audience decodes it from external recognition to inner feeling. Even at the intersection of AI and art, human cognition can still feedback to the optimization direction and application mechanism of the AI generator model in the field of art.

This study is intended to provide researchers who focus on AI application in art with a framework of how to obtain the art cognitive effects for machine-generated results. As shown in Figure 1, this study can be divided into three sections. In Section 1, a literature review was used to explore the communication matrix for evaluating artistic style transfer. Section 2 invited nine experts in artistic and/or aesthetic backgrounds to select painting samples according to cognitive attributes and choose the suitable method for transferring abstract art style. In Section 3, the data collected from the grouping experiments of four art schools were used for analysis and discussion. Finally, the conclusion of this study was given.

1.1. Application of NST into Art

Neural Style Transfer (NST) is a type of algorithm that uses deep neural networks to render a content image in different styles. In the early research stage of Non-Photorealistic Rendering (NPR), many stylization algorithms were designed to automatically turn photos into synthetic artworks by stimulating artists to create art [3,4]. However, these patch-based methods’ limitation is that only low-level image features are adopted while often failing to capture image structures effectively [5]. Gatys et al. [6] are the first to apply Convolutional Neural Networks (CNNs) to transfer famous painting styles into photography. Since the algorithm of Gatys et al. does not have any explicit restrictions on the type of style images and needs ground truth results for training, it breaks the constraints of previous approaches. In this case, it ushered in the new field called Neural Style Transfer (NST). The method put forward by Gatys et al. based on the Gram-matrix-matching-based style representations requires a slow iterative optimization process, which is computationally expensive. Still, it is usually regarded as the gold-standard method in the community of NST [5,7].

During the follow-up years, there are a series of optimization methods for NST that can fall into paired and non-paired methods. To be specific, the former is to transfer the style of one image to another, while the latter is to learn the style of multiple images in a dataset and then transfer it. As the experiment in this study tends to provide each painting and its corresponding conversion as a paired stimulus, it is more suitable to choose a pairing method. In addition to Gatys’ original algorithm, this type of methods consists of several representative algorithms, such as AdaIN [7] and WCT [8] for improving speedy, as well as MST [9] and SEMST [10] which focuses on the matching of semantic patterns between content images and style images. Inspired by the instance normalization, Huang and Belongie [7] proposed a new interpretation by normalizing feature statistics to adaptive instance normalization (AdaIN), which first enables arbitrary style transfer in real-time. Like Huang and Belongie’s method but replacing the AdaIN layer between the encoder and the decoder, Li et al. [8] embedded a pair of feature transforms, whitening, and coloring (WCT) into an image reconstruction network. However, these algorithms based on feed-forward networks failed to consider the feature details and local structure and suffered from wash-out artifacts [5,9]. To match the content structure, Zhang et al. [9] first introduced a multimodal style transfer method called MST. He considered the semantic content structure and the matching with style patterns by using K-means to split style patterns from style image and combine it with content images via graph-cut. Recently, Chen [10] analyzed the shortcomings of MST, such as the inability to consider the structural information of the content image or the loss of the style characteristics due to the feature’s high dimensional and low-resolution characteristics space and the characteristics of the graph cut method. In further research, he proposed a structure-emphasized multimodal style transfer (SEMST) that can flexibly match the content cluster and the style cluster even if the number of clusters differs. Although there are many other papers and studies in the NST field, the purpose of this study is to explore the differences in audience cognition to the results of artistic style transfer, rather than review all methods.

The way to compare different NST algorithms includes both qualitative and quantitative methods. The main quantitative evaluation metrics focus on transfer speed and loss comparison [5,9]. Regarding the qualitative evaluation, the observation of style features and user study by voting are usually used. In the experiment of Li et al. [8], they invited 80 subjects to vote for their favorite result for each method. Then, the study shows the WCT received more votes than other methods like AdaIN and Gatys et al. Yet, as the qualitative assessment of Zhang et al. [9], the result showed that the MST obtained the highest percentage of votes, while Gatys et al. ranked second above AdaIN and WCT. Due to the varied choices of samples and users, the results by using the qualitative methods are also different [8,9]. The evaluation of response to artistic imagery is a complex process that includes both perceptions of shape and evaluating aesthetics and other properties of images [1]. To obtain accurate and stable results from the qualitative assessment, a reasonable evaluation mechanism needs to be constructed.

The field of NST has received increasing attention from academics and industry in recent years, resulting in some related industrial applications, such as the websites Ostagram [11] and Deep Dream Generator [12], the mobile application Prisma [13], and even film production [14]. People can use those platforms to create some interesting images for pleasure or spawning creativity. Machine-based intelligence learning from human intelligence will also be inspired by human insight. Therefore, the evaluation results from audience cognition can provide the algorithm with some suggestions for further optimization and the application regarding the intersection with art.

1.2. Attributes of Artistic Style

In the visual arts, style is a distinctive manner that permits the grouping of works into related categories [15], and it refers to similar critical features for recognition, such as characteristic subject matter or materials, distinctive ways of drawing or applying paints, preferences for specific color combinations, brushstrokes, distortion, and exaggeration. These visual pieces of information can direct the viewer’s attention and affect the viewer’s perception [16,17]. The perception of visual art is a complex performed by the brain to perceive different elements’ features in the painting [18,19]. The various painting attributes, such as colors, shapes, and boundaries, are selectively redistributed to the brain for processing [20]. There are three critical features in the receptive field: position, shape, and specificity, similar to modern art, with its accent on simplification [21].

Artistic visual styles such as Fauvism, Expressionism, and Cubism in the early 20th century have the commonality of abstraction and expression. Meanwhile, they have distinctive attributes that support the grouping of artworks into related art movements [22,23,24,25]. Both Fauvism and Expressionism use pure colors and subject distortion. However, unlike the thick coating method or squeezing the paint directly from the tube in Fauvism, Expressionism is characterized by broad brush strokes to exaggerate artists’ inner emotions and feelings [23,26]. Cubism is known for its reduction of subjects into geometric shapes to produce a more three-dimensional perspective. It is a calm and reflective experimental art that balances representation and abstraction and abandons colors in favor of an almost monochrome palette [22]. Different from the modernist style, realism such as the Renaissance involves several techniques that make the subjects and backgrounds look like what they would be in real life. As Renaissance artists began using the scientific perspective and new ways of painting light, their paintings became more meaningful and realistic [27].

1.3. Communication between Audience and Artwork

Art is the media that provides an understanding of visual communication [28]. As a communication channel, artwork can connect the artist and audience through the creator’s encoding and the audience’s decoding [28,29,30]. The interplay between the internal (cognitive) representation and the external (physical) representation is a fascinating problem in cognitive psychology, art, science, and philosophy [18]. Each of us approaches an art object with a significantly different perspective because of our unique personal social experiences [18,31]. Whoever can feel the beauty of colors and forms has understood non-objective painting [32]. The aesthetic experience involves the processing of stylistic information closely related to the artist’s movements [31,33] and simultaneously performing the movements can enhance the preference for paintings of the corresponding style [34]. In abstract paintings, the ideas, emotions, and visual sensations are expressed solely through lines, shapes, color, and textures that have no symbolic significance [35]. According to people’s feelings, color may be warm or cold, and cheerful or somber [20]. Audiences can perceive the painter’s actions by merely observing the brushstroke of the painting [36]. Texture includes all painting areas enriched by combing lines, shapes, tones, and colors [16,35]. Therefore, from the perspective of cognition, we can learn more about the process of creation [37].

In the symbol communication mode, cognition can be divided into three levels: technical, semantic, and effectiveness levels [38]. To evaluate the artwork, it is necessary to find out the cognitive factors affecting them. Lin et al. [39] proposed a communication matrix that combined the cognition and communication theory, which integrated into three dimensions for evaluating artwork. There are three stages for artists to express their thought through artworks: inspiration, ideation, and implementation. For audiences, the three stages of experience include aesthetic, meaning, and emotional. Shusterman [40] and Bergeron and Lopes [41] suggest that an aesthetic experience is featured with an evaluative dimension involving semantic and affective aspects to confer the aesthetic quality to the experience. However, in the literature, there is little evidence of discussing the evaluation concerning the results of artistic style transfer by computer algorithms from the aspects of aesthetic cognition. In the creative process of AI-Art, the artists choose AI algorithms according to their intentions for creating the artwork, and audience acceptance is a critical defining step to decide whether it is “art” [42]. Studying the factors that affect artistic cognition from the perspective of the audience can help build a bridge between artists and the audience [43,44].

1.4. Research Purpose

The perception of artistic style is the complex processing of visual information. The audience is a vital participant of art, especially when AI technology is involved in the art field. The artistic style transfer methods by using neural networks have improved efficiency and effectiveness in Non-Photorealistic Rendering. If trying to further optimize industry applications, the audience’s perception of the style transfer results should be considered. Technology comes from humanity, so the integration of Hi-tech and Hi-touch is necessary for art creation and product design, especially in the 21st-century digital technology world [42].

In this study, the style transfer results of four painting schools were evaluated from the perspective of audience cognition. Through four cognitive experiments, including three abstract schools of modernism in the early 20th century (Fauvism, expressionism, cubism) and Realism (Renaissance), it tends to explore whether the audience can still distinguish the different art schools and keep the cognition from technical, semantic and effectiveness levels. The main factors affecting the audience’s cognition in artistic style will be discussed to understand the difference before and after style transfer. The result of this study can be used as an optimization suggestion to improve the performance of NST methods closer to human cognition.

2. Materials and Method

2.1. Pre-Experiment

To select the appropriate algorithm for this experiment, the method of Gatys et al. [6], AdaIN [7], WCT [8], MST [9], and SEMST [10] were used to transfer four samples’ style to the same content image. Gatys et al. [6]’s seminal work synthesized stylization through an iterative optimization scheme. AdaIN [7] and WCT [8] are methods using feed-forward networks to enhance efficiency to a great extent. MST [9] and SEMST [10] solved the above methods’ artifact problem by matching the semantic pattern in style and content images via K-means. It should be mentioned that the codes employed here come from GitHub provided by the author or written according to papers. The links are shown in Table 1.

The five methods were adopted with their default parameters to create stylized images by learning the features of four schools’ representative paintings. A photograph of Guangong in opera was used as the content image for its rich semantic involving character, decorative patterns, and background. Then, nine experts with artistic and/or aesthetic backgrounds were invited to vote for the best method through perception, and the best method in each art school was marked with the blue line in Figure 2. The feedback from experts is that the results of Fauvism and Expressionism can keep the balance between style and content by using the method of Gatys et al. method. Although the use of WCT in the former two schools produced serious distortions and lacked content structure, the result of Cubism still retained the feeling of geometric fragments. For the Renaissance style, SEMST is more semantic and more suitable for transferring realistic style. However, the sense of abstract style was lost in the other three schools of modernists. Based on the purpose of this research, the method of Gatys et al. was selected to produce all the stimuli of experiments.

2.2. Stimuli

Based on previous studies [29,30,39,43,44], the artwork can be evaluated from technical, semantic, and effectiveness levels. The technical level focus on the visual elements. The semantic level means letting the audience accurately understand the meaning of the artwork through his/her realization. The effectiveness level concerns the inner feeling of the audience on the emotional expression of the artist. Color, stroke, and texture as important factors of painting style can still be evaluated from the above three levels. Therefore, the matrix for evaluating the cognitive effect of artistic style transfer through machine learning was reconstructed, as shown in Figure 3.

The nine experts who are mentioned above continued to pick up paintings with typical color, stroke, and texture from wikiart.org and then chosen one sample matching each cognitive attribute in the evaluation matrix, and they are Color-technical, Color-semantic, Color-effectiveness, Stroke-technical, Stroke-semantic, Stroke -effectiveness, Texture-technical, Texture-semantic, and Texture-effectiveness. After obtaining nine paintings of each school, the experts further describe their characteristics according to the representative attribute of each work, which was used as the questionnaire items. For example, according to the feature of the F1 sample in the attribute it represents, the Color-technical item in the Fauvism group was described to be “complementary”, which is shown in Table 2. Theoretically, the F1 sample should get the highest score on this option. Besides, a Mona Lisa portrait was inserted into each group as the reference sample, and the score should be higher in the Renaissance group. In the Fauvism, Expressionism, and Cubism groups, the Mona Lisa should have significant cognitive differences from other samples due to different periods and styles.

The content image, a photograph of a character in opera called Guangong, was combined with the style of paintings using the method of Gatys et al., shown in Table 2. Before transfer, all of the style and content images were resized to 600 px high. Furthermore, the default parameters suggested by the authors and the pre-trained VGG19 network were also adopted. In layer options, style reconstructions used the conv1_1, conv2_1, conv3_1, conv4_1, and conv5_1 and content reconstructions used the conv4_2 layer. The process was initialized with the content image instead of random noise in the paper to exclude random results. For the style setting, the ratio content_weight/style_weight was 1 × 10⁻³. Experiments conducted by Gatys et al. [7] prove that this ratio can better balance content and style. Besides, 1.0 as the scale of the style was used to keep the proportional transfer of texture. The number of iterations was 1000 as default parameters because there is no obvious change after increasing more.

2.3. Participants

All of the study participants were artists, professors, or Ph.D. with art and/or design background. A total of 31 subjects (17 male and 14 female) participated in the experiment. The mean age of the male participants was 48.29 (SD = 9.45) years. The mean age of the female participants was 42.71 (SD = 10.91) years.

2.4. Experimental Procedure

This study consisted of four groups of experiments. The stimuli of each group contained nine samples representing nine cognitive attributes of one art school, as well as the corresponding results combining with their artistic style and content images by using the method of Gatys et al. The famous painting Mona Lisa was inserted into each group as the reference sample to find whether there is a difference in its cognition from others.

The researcher first introduced the process to the subjects. Then, starting from Fauvism, there were ten sets of stimuli, including nine pairs of paintings belonging to Fauvism and one pair of Mona Lisa. Each painting and its transfer result were projected onto the screen simultaneously. Subjects had 2–3 min to complete the evaluation of the samples. For every group, the experiment lasted for less than 1 h with at least one week apart to avoid learning effects, and the duration was four weeks.

The subjects were informed of the experimental task and then asked to rate each pair of samples according to their subjective opinion. The dependent variables were presented in the questionnaire on the participants’ perception. In the first part, the degree of fitness for style attributes involving the color, stroke, texture, and overall fitness between the original painting and transfer result, with a score range from 0 to 100%—the higher the score, the higher the fit. The second part explored the degree of fitness on the nine cognitive attributes in each pair of stimuli. The attribute category includes “Color-technical”, “Color-semantic”, “Color-effectiveness”, “Stroke-technical”, “Stroke-semantic”, “Stroke-effectiveness”, “Texture-technical”, “Texture-semantic” and “Texture-effectiveness”. The item description of each attribute in the questionnaire was provided by the nine experts mentioned above, based on each sample’s feeling. A 5-point Likert scale was used to score the fitness of nine cognitive attributes of each pair from 1 (“Very unfit”) to 5 (“Very fit”). Finally, the subjects were asked to choose a favorite sample and a least favorite sample respectively in each group.

2.5. Statistical Analysis

Firstly, one-way ANOVA was adopted to test whether the style attributes and cognitive attributes of ten pairs of painting factors per group were significant (significance level was set at 0.05). For programs reaching the significance level, Duncan was used to figure out whether there was a significant difference among ten averages. In addition, a Pearson correlation coefficient was also employed to investigate the potential correlation between overall fitness and three style factors, including color, stroke, and texture.

3. Results and Discussion

3.1. Results of Descriptive Statistics and ANOVA Analysis

According to the results of variation analysis on the first part of the questionnaire as shown in Table 3, after the same subjects viewed the four groups of paintings, they displayed a significant difference in their feelings for style attributes (i.e., “color fitness”, “stroke fitness”, “texture fitness”, and “overall fitness”). Different from the results in Fauvism, Expressionism, and Cubism that the sample of Mona Lisa independently exists in the lowest score group, it was classified into the same group as other samples in Renaissance. The result indicates that subjects could still distinguish the different schools of painting even after style transfer. Otherwise, similar to the literature discussion, the method based on iterative optimization and feed-forward networks lacking to consider the feature details and content structure [5,9], the algorithm of Gatys et al. is more suitable to transfer abstract style than realism style.

It is worth noting that the samples which belong to stroke and texture attributes in Fauvism, Expressionism, and Cubism got a higher score in their fitness evaluation. However, among the Renaissance stimuli, the samples with color characteristics scored higher than others. The relevant scores have been marked with red lines in Table 3.

As the results of the evaluation for cognitive levels in part two, some results of samples with certain cognitive features can still be perceived by subjects, as shown in Table 4. The score belongs to style attributes of color, strokes, and texture was marked by the red line, as well as the gray background color emphasizes the score of samples on their corresponding attributes. Furthermore, the highest score on the corresponding attribute is marked by the blue dashed line, which is mainly distributed in stroke and texture attributes in Fauvism, Expressionism, and Cubism. However, in the Renaissance group which lacked stroke and texture features, the participants had a stronger perception of color. The reference sample (Mona Lisa) obtain the lowest scores in all cognitive attributes of Fauvism, Expressionism, and Cubism. However, in the style that it belongs to, Mona Lisa had a higher ranking in various indicators of Renaissance, further explaining that the subjects can distinguish the different schools of painting even after style transfer.

3.2. Pearson Correlation Coefficient

A strong positive association was established using a Pearson correlation coefficient between overall fitness and three style attributes including color, stroke, and texture, shown in Table 5. On the one hand, it means that color, stroke, and texture can be used as factors to evaluate the algorithm of artistic style transfer. In addition, stroke and texture have a higher correlation with the overall fit than color. Hence, compared with color, the simulation stroke and texture in artistic style transfer play a more critical role and should be paid more attention.

3.3. The Favorite and Least Favorite Paintings

To further determine the critical point that affects participants’ preference, in this study, subjects were invited to choose their favorite and least favorite samples respectively from each group. Table 6 shows the proportion of each painting being selected as a favorite. The stimulus ranking top in the preference vote had higher scores of overall fitness. Besides, their score in corresponding cognitive attributes is the highest. Based on the previous discussion, users have a higher perception of abstract style strokes and textures. The samples that can accurately transfer the cognitive features of strokes and textures are more welcome. However, as the realistic style without evident stroke and texture features, the transition result of Renaissance received more attention in color. In Figure 4, the most favorite sample of each group was listed. Subjects can feel the corresponding cognitive attribute of P9, P8, P5, and P3: Cute Pattern, Wantonly, Bright & Fast, and Lively.

The proportion of each painting being selected as the least favorite was shown in Table 7. Subjects voted P10 as their least favorite sample in all groups except Renaissance. In the Renaissance group, there was no significant difference in cognition because the Mona Lisa portrait belonged to the same style as other samples. Therefore, it can be further explained that the subjects can distinguish different artistic styles even reproduced by machine learning. In the Fauvism, expressionism, and Cubism groups, the image with the lowest score was also the subjects’ least favorite samples, which showed a positive correlation between the preference and cognitive fitness. From Figure 5, it can be found that the two least favorite samples also lost the content’s details seriously. The more details lost, the fewer subjects like it. The wash-out artifact problem in the feed-forward networks-based method [5,9] causes the bad transfer result of realistic style.

4. Conclusions

The experimental results show that subjects could still distinguish the different art schools and keep the cognition from technical, semantic, and effectiveness levels. The cognitive attributes have a strong relationship with the overall feeling and audience’s preference. It supported that the matrix based on style attributes and cognitive levels can be used for subjective evaluation of the NST method. Among the three style attributes, the fitness of stroke and texture have a higher degree of correlation with overall fitness than color. However, for the Renaissance featured with the style without obvious stroke and texture, the audience turned their attention to the feeling of color and the painting’s content. Therefore, which style attribute should be strengthened is based on the style sample’s features during the NST encoding. Besides, the sample whose semantic or effectiveness level was accurately perceived usually has a higher preference, so the two cognitive levels are the key to improve the audience’s preference. The correlation between style feature and their corresponding cognitive effects should be further discussed.

This study presented a framework that can be used as the fundamental research for subject evaluating AI algorithms in the art field. However, this study still had several limitations. Firstly, as the classification of painting styles is complex and subjective, there are some uncertainties in the choice of painting samples and the corresponding evaluation criteria. Therefore, in the future study, the two samples with the highest scores in each group will be selected and evaluated by a large number of subjects without professional background for further discussion. Secondly, the methods of comparison in this study are difficult to fully cover the newest technologies with the rapid development in this field. The algorithms selected in this study only involve the typical paired methods, while failing to include the non-paired ways that represent the features of datasets.

Author Contributions

Y.L. performed the writing conceptualization, formal analysis, original draft, editing investigation, resources; C.-L.L. contributed to the methodology, review; P.-H.L. contributed review; R.L. contributed to review, editing. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data sharing not applicable.

Acknowledgments

The authors would like to appreciate the experts and participants that took part in the experiments. We also thank the researchers who shared the source code on GitHub.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. The procedures for this study.

Figure 2. The example results of Gatys et al. [6], AdaIN [7], WCT [8], MST [9], and SEMST [10]. The most suitable method in each school was selected by experts and marked by a blue line.

Figure 3. Communication matrix for evaluating artistic style transfer.

Figure 4. Participants’ most favorite paintings in each group.

Figure 5. Participants’ least favorite paintings in each group.

Table 1

The GitHub links of the five methods.

Methods	GitHub Links
Gatys et al. [6]	https://github.com/ProGamerGov/neural-style-pt (accessed on 1 June 2020)
AdaIN [7]	https://github.com/naoto0804/pytorch-AdaIN (accessed on 1 June 2020)
WCT [8]	https://github.com/sunshineatnoon/PytorchWCT (accessed on 1 June 2020)
MST [9]	https://github.com/irasin/Pytorch_MST (accessed on 1 June 2020)
AEMST [10]	https://github.com/irasin/Structure-emphasized-Multimodal-Style-Transfer (accessed on 10 July 2020)

Table 2

Thirty-seven pairs of stimulus paintings.

Fauvism	Color	Stroke	Texture
Technical level	[Image omitted. Please see PDF.]A1 Complementary	[Image omitted. Please see PDF.]A2 Wantonly stroke	[Image omitted. Please see PDF.]A3 Decorative curve
Semantic level	[Image omitted. Please see PDF.]B1 Strong	[Image omitted. Please see PDF.]B2 Chaos & power	[Image omitted. Please see PDF.]B3 Regular progress
Effectiveness level	[Image omitted. Please see PDF.]C1 Unrestrained	[Image omitted. Please see PDF.]C2 Rough & natural	[Image omitted. Please see PDF.]C3 Cute Pattern
Expressionism	Color	Stroke	Texture
Technical level	[Image omitted. Please see PDF.]A1 Contrasting	[Image omitted. Please see PDF.]A2 Solid and dashed	[Image omitted. Please see PDF.]A3 Regular
Semantic level	[Image omitted. Please see PDF.]B1 Mysterious	[Image omitted. Please see PDF.]B2 Frantic	[Image omitted. Please see PDF.]B3 Wantonly
Effectiveness level	[Image omitted. Please see PDF.]C1 Pessimist	[Image omitted. Please see PDF.]C2 Heartfelt	[Image omitted. Please see PDF.]C3 Vivid-generated
Cubism	Color	Stroke	Texture
Technical level	[Image omitted. Please see PDF.]A1 Plane division	[Image omitted. Please see PDF.]A2 Short & powerful	[Image omitted. Please see PDF.]A3 Geometric
Semantic level	[Image omitted. Please see PDF.]B1 Block structure	[Image omitted. Please see PDF.]B2 Bright & fast	[Image omitted. Please see PDF.]B3 Fragment space
Effectiveness level	[Image omitted. Please see PDF.]C1 Space misplace	[Image omitted. Please see PDF.]C2 Manifest personality	[Image omitted. Please see PDF.]C3 Abstract beauty
Renaissance	Color	Stroke	Texture
Technical level	[Image omitted. Please see PDF.]A1 Gradient	[Image omitted. Please see PDF.]A2 Delicate depiction	[Image omitted. Please see PDF.]A3 Stereoscopic
Semantic level	[Image omitted. Please see PDF.]B1 Obvious levels	[Image omitted. Please see PDF.]B2 Realistic	[Image omitted. Please see PDF.]B3 Progressively
Effectiveness level	[Image omitted. Please see PDF.]C1 Lively	[Image omitted. Please see PDF.]C2 Implicit	[Image omitted. Please see PDF.]C3 Natural
Reference sample	[Image omitted. Please see PDF.]Mona Lisa

Table 3

Results of descriptive statistics and ANOVA analysis of four groups.

Fauvism	F1	F2	F3	F4	F5	F6	F7	F8	F9	Reference Sample	Significance
Color	64% c	75% a	82% a	70% b	83% a	72% b	72% b	77% a	78% a	51% d	***
Stroke	53% b	66% a	76% a	69% a	74% a	61% b	74% a	74% a	75% a	40% c	***
Texture	53% b	61% b	70% a	70% a	73% a	62% b	73% a	74% a	78% a	38% c	***
Overall	54% c	67% b	76% a	72% a	79% a	61% b	73% a	75% a	77% a	39% d	***
Expressionism	E1	E2	E3	E4	E5	E6	E7	E8	E9	Reference Sample	Significance
Color	74% b	59% c	59% c	84% a	76% b	86% a	80% a	74% b	71% b	59% c	***
Stroke	71% b	54% c	60% c	84% a	72% b	84% a	80% a	73% b	71% b	50% d	***
Texture	70% b	52% c	57% c	83% a	73% b	83% a	79% a	74% b	69% b	43% d	***
Overall	72% b	52% c	58% c	84% a	73% b	84% a	76% a	74% b	69% b	46% d	***
Cubism	C1	C2	C3	C4	C5	C6	C7	C8	C9	Reference Sample	Significance
Color	76% a	67% b	69% b	75% a	81% a	81% a	83% a	66% b	75% a	54% c	***
Stroke	63% b	64% b	64% b	71% a	75% a	73% a	78% a	64% b	68% a	47% c	***
Texture	60% b	62% b	62% b	69% a	74% a	73% a	77% a	64% b	67% a	46% c	***
Overall	65% b	63% b	65% b	72% a	77% a	76% a	75% a	64% b	69% a	47% c	***
Renaissance	R1	R2	R3	R4	R5	R6	R7	R8	R9	Reference Sample	Significance
Color	80% a	72% a	76% a	61% b	68% b	57% c	64% b	47% c	47% c	54% c	***
Stroke	70% a	67% a	70% a	50% b	60% a	47% b	60% a	42% b	42% b	47% b	***
Texture	67% a	64% a	67% a	48% b	59% a	47% b	58% a	41% b	41% b	45% b	***
Overall	71% a	66% a	71% a	50% b	61% a	48% b	59% a	42% b	41% b	46% b	***

*** p < 0.001; ^a, ^b, ^c and ^d are Duncan ex-post-test grouping results; the attribute with the highest score is marked in red.

Table 4

Result of average score and standard deviation for cognitive attributes.

Fauvism	P01	P02	P03	P04	P05	P06	P07	P08	P09	P10
A1. Complementary	3.58	3.84	4.03	3.51	3.97	3.48	3.90	3.77	4.19	2.10
B1. Strong	3.55	3.74	3.97	3.61	3.94	3.26	3.74	3.94	4.29	2.10
C1. Unrestrained	3.71	3.65	3.45	3.45	4.00	3.13	3.87	3.65	3.58	2.16
A2. Wantonly stroke	3.71	3.45	3.42	3.58	3.45	3.13	3.94	3.36	3.42	2.32
B2. Chaos & power	3.45	3.65	3.26	3.45	3.74	2.81	3.71	3.55	3.52	2.13
C2. Rough & natural	3.55	3.58	3.52	3.65	3.81	3.03	3.45	3.29	3.42	2.26
A3. Decorative curve	3.03	3.55	3.77	3.00	3.42	3.13	3.87	3.71	4.00	2.07
B3. Regular progress	2.87	3.16	3.68	3.19	3.58	3.16	3.29	3.65	3.45	2.10
C3. Cute Pattern	2.94	3.39	3.42	3.32	3.52	3.13	3.48	3.39	3.71	2.00
Average Score	3.38	3.56	3.61	3.42	3.71	3.14	3.69	3.59	3.73	2.14
Expressionism	P01	P02	P03	P04	P05	P06	P07	P08	P09	P10
A1. Contrasting	3.47	2.72	3.00	3.69	3.41	3.81	3.28	3.69	3.09	2.63
B1. Mysterious	3.19	3.31	3.31	3.03	3.19	3.19	3.19	3.72	3.00	3.41
C1. Pessimist	2.81	3.25	2.72	2.78	2.75	2.47	2.88	3.09	2.84	2.72
A2. Solid and dashed	2.38	2.72	2.50	2.94	2.56	2.84	2.81	2.56	2.53	2.31
B2. Frantic	2.69	2.88	2.91	3.22	2.97	3.50	3.00	3.59	2.94	2.63
C2. Heartfelt	2.69	2.59	2.72	3.22	3.09	3.22	3.22	3.34	2.41	2.16
A3. Rich in regular	2.66	3.06	3.19	3.56	3.34	3.59	3.63	3.50	2.97	2.23
B3. Wantonly	2.91	3.34	3.03	3.47	3.25	3.69	3.56	3.75	2.84	2.56
C3. Vivid-generated	2.94	2.97	2.75	3.66	3.25	3.72	3.34	3.19	2.56	2.50
Average Score	3.47	2.72	3.00	3.69	3.41	3.81	3.28	3.69	3.09	2.63
Cubism	P01	P02	P03	P04	P05	P06	P07	P08	P09	P10
A1. Plane division	2.63	3.00	3.00	3.16	3.28	3.28	3.88	3.13	3.00	2.00
B1. Block structure	3.00	3.13	2.94	3.19	3.16	3.34	3.94	3.16	2.75	1.88
C1. Space misplace	2.81	2.97	2.72	3.03	3.34	3.16	3.25	2.84	2.88	2.28
A2. Short & powerful	3.19	3.25	2.84	3.13	3.59	3.59	3.59	3.28	3.06	2.06
B2. Bright & fast	2.84	3.41	3.13	3.31	3.66	3.44	3.56	3.06	2.94	2.13
C2. Manifest personality	2.88	3.38	3.06	3.59	3.81	3.78	3.81	3.19	3.09	2.25
A3. Geometric	2.81	3.03	2.69	3.13	3.09	2.97	3.84	3.19	2.69	2.00
B3. Fragment space	3.03	2.84	2.97	2.94	3.19	3.16	3.84	3.41	2.91	2.22
C3. Abstract beauty	3.16	3.13	3.16	3.44	3.38	3.13	3.20	2.94	2.94	2.63
Average Score	2.63	3.00	3.00	3.16	3.28	3.28	3.88	3.13	3.00	2.00
Renaissance	P01	P02	P03	P04	P05	P06	P07	P08	P09	P10
A1. Gradient	3.55	3.19	3.25	2.47	3.00	2.47	3.19	2.22	2.38	2.50
B1. Obvious levels	3.06	3.25	3.75	2.47	3.09	2.44	2.78	2.16	2.22	2.38
C1. Lively	3.09	3.22	3.69	2.19	2.97	2.25	2.72	2.06	2.00	2.28
A2. Delicate depiction	2.97	2.88	3.31	1.94	2.69	1.97	2.38	2.03	1.91	2.09
B2. Realistic	2.63	3.00	3.19	1.84	2.47	1.75	2.34	1.88	1.78	2.16
C2. Implicit	3.09	2.88	3.00	2.50	2.72	2.31	2.75	2.19	2.09	2.28
A3. Stereoscopic	2.66	2.72	3.28	2.03	2.72	2.09	2.28	1.91	1.84	1.94
B3. Progressively	3.06	2.78	3.22	2.31	2.88	2.13	2.66	2.00	1.84	2.22
C3. Natural	2.97	2.72	3.16	2.06	2.47	1.78	2.44	1.94	1.75	2.28
Average Score	3.55	3.19	3.25	2.47	3.00	2.47	3.19	2.22	2.38	2.50

The background color is used to emphasize the score of samples in their corresponding attributes; the attribute with the highest score is marked in red; the blue dotted line is used to mark the actual highest score that should obtain.

Table 5

Result of Pearson Correlation Analysis for the style attributes.

Fauvism	Color	Stroke	Texture	Overall	Expressionism	Color	Stroke	Texture	Overall
Color	1				Color	1
Stroke	0.773 ***	1			Stroke	0.815 ***	1
Texture	0.695 ***	0.894 ***	1		Texture	0.789 ***	0.928 ***	1
Overall	0.759 ***	0.909 ***	0.902 ***	1	Overall	0.821 ***	0.876 ***	0.899 ***	1
Cubism	Color	Stroke	Texture	Overall	Renaissance	Color	Stroke	Texture	Overall
Color	1				Color	1
Stroke	0.793 ***	1			Stroke	0.846 ***	1
Texture	0.784 ***	0.955 ***	1		Texture	0.822 ***	0.969 ***	1
Overall	0.797 ***	0.936 ***	0.946 ***	1	Overall	0.870 ***	0.971 ***	0.975 ***	1

*** p < 0.001.

Table 6

The proportion of each painting being selected as a favorite.

Fauvism
Order	1	2	3	4	4	4	4	8	8	8
No.	P9	P5	P7	P2	P3	P4	P8	P1	P6	P10
n (%)	9(29.0)	8(25.8)	6(19.4)	2(6.5)	2(6.5)	2(6.5)	2(6.5)	--	--	--
Expressionism
Order	1	2	3	4	5	5	5	8	8	8
No.	P8	P6	P7	P3	P1	P2	P5	P4	P9	P10
n (%)	16(50.0)	8(25.0)	3(9.4)	2(6.3)	1(3.1)	1(3.1)	--	--	--	--
Cubism
Order	1	2	2	4	4	6	7	7	9	9
No.	P5	P3	P6	P7	P8	P4	P2	P9	P1	P10
n (%)	6(19.4)	5(16.1)	5(16.1)	4(12.9)	4(12.9)	3(9.7)	2(6.5)	2(6.5)	--	--
Renaissance
Order	1	2	3	4	4	6	6	6	6	6
No.	P3	P1	P2	P5	P7	P4	P6	P8	P9	P10
n (%)	17(54.8)	6(19.4)	6(19.4)	1(3.2)	1(3.2)	--	--	--	--	--

Table 7

Proportion of each painting being selected as least favorite.

Fauvism
Order	1	2	3	4	4	6	6	6	6	6
No.	P10	P4	P5	P3	P6	P1	P2	P7	P8	P9
n (%)	24(77.4)	3(9.7)	2(6.5)	1(3.2)	1(3.2)	--	--	--	--	--
Expressionism
Order	1	2	3	4	4	4	7	7	7	7
No.	P10	P4	P5	P2	P7	P8	P1	P3	P6	P9
n (%)	20(64.5)	6(19.4)	2(6.5)	1(3.2)	1(3.2)	1(3.2)	--	--	--	--
Cubism
Order	1	2	3	4	4	6	6	6	6	6
No.	P10	P5	P6	P7	P9	P1	P2	P3	P4	P8
n (%)	22(71.0)	5(16.1)	2(6.5)	1(3.2)	1(3.2)	--	--	--	--	--
Renaissance
Order	1	2	3	4	4	6	6	8	8	8
No.	P9	P8	P6	P2	P3	P5	P10	P1	P4	P7
n (%)	11(35.5)	10(32.3)	4(12.9)	2(6.5)	2(6.5)	1(3.2)	1(3.2)	--	--	--