More Evidence for A Computational Memetics Approach to Music Information and New Interpretations of An Aesthetic Fitness Measure Tak-Shing T. Chan and Geraint A. Wiggins 1 Abstract. Previously [5], we proposed a music information model and a related aesthetic fitness measure in the computational memetics context. Here we attempt to extend this work by (a) explicating our multi-layer research strategy, (b) providing empirical evidence for our information model, and (c) presenting a more complete theoretical interpretation of our aesthetic measure. Potential applications to creative systems are also considered.

1

Introduction

Following the computational memetics approach to music [12, 3, 4], we have previously proposed a cognitive information model of music memory and a related musical fitness measure [5]. Using three types of random music as stimuli, our fitness measure was able to produce predictions [5] that agree with Jeong et al.’s neuroscientific data [13]. One criticism2 of this test is that random music lacks ecological validity, so a successful replication tells us very little. Secondly, in the same paper, we have put forward a “Wundt curve” interpretation of our fitness measure [5], but in hindsight, we now felt that this interpretation has unfairly downplayed the contribution of stimulus complexity to aesthetic fitness. In this paper, we will first expand on our theoretical model by explicating our multi-layer research strategy (heretofore unclear in [5]). Then we will touch upon the internal validity of our information model and provide empirical validation with ecologically valid stimuli. Third, we will provide (in our view) a more complete interpretation of memetic fitness that goes beyond the Wundt curve. Finally, we will discuss model fitting and generalisation, and possible applications of our theoretical model to creative systems.

2 2.1

Previous Work Music Information

Chan and Wiggins [5] presented a cognitive model of music information (Figure 1) based on the Atkinson-Shiffrin [1] memory model, with OPM (onset, pitch and metrical level) as input, NCF (neural cancellation filter [8]) as sensory register, and H (the square root of Tcomplexity [20]) as the amount of short-term store required to memorise the OPM input loselessly. The long-term store was not modelled.

This theory was then extended for two given musical objects [5] by defining the joint cognitive information as H(x, y) = min{H(x) + H(y), H(!x, y")}, the conditional cognitive information as

2

Centre for Cognition, Computation and Culture, Goldsmiths’ College, University of London, New Cross, London SE14 6NW, United Kingdom, email: {t.chan,g.wiggins}@gold.ac.uk According to an anonymous reviewer of Chan and Wiggins [5].

H(x|y) = min{H(x), H(!x, y") − H(y)},

(2)

I(x; y) = max{0, H(x) + H(y) − H(!x, y")},

(3)

and the mutual cognitive information as where !x, y" is the pairing complexity theory [2]. OPM

operator3

−→

NCF

borrowed from Kolmogorov

−→

H

Figure 1. Architecture of Chan and Wiggins’ [5] information model.

2.2 Aesthetic Fitness As an important application of this new information theory, Chan and Wiggins [5] defined musical fitness as: unity × diversity = I(x; y) × H(x|y),

(4)

unity × diversity = H(x)2 [1 − F(x, y)]F(x, y),

(6)

where x models the music and y models the listener. This formulation is based on Eysenck’s [9] formula, but an important difference is that Chan and Wiggins [5] have adopted the “to each his own” (subjectivity) principle in defining both unity and diversity, taking into account that musical fitness may vary from person to person. Chan and Wiggins [5] demonstrated that their unity in diversity measure can be interpreted as a Wundt curve, for if we define familarity as: H(x|y) F(x, y) = , (5) H(x) then we have which is a concave quadratic (inverted-U) function of F(x, y) ∈ [0, 1] when x is held fixed. We will come back to this equation later in this paper, when we examine the case where x is not held fixed. 3

1

(1)

Roughly speaking, the pairing operator !x, y" for two strings x and y returns the concatenation of x and y. More precisely [2], it returns the concatenation of x and y, prefixed by the length of x and further prefixed by the length of length of x. Theoretically, the prefixes are required to make the pairing operator invertible [2], but we will ignore it here for simplicity.

2.3

Psychological Experiments

This subsection summarises two experiments done by other researchers, which we will replicate computationally here in this paper. Both experiments used stimuli that sounded realistic and therefore should not suffer from the criticism of being ecologically invalid. The first experiment was done by Shmulevich and Povel [18]. In their experiment, they asked 25 subjects to listen to 35 rhythmic patterns and to rate the complexity of each. The patterns and mean ratings are reproduced in Table 1. Shmulevich and Povel [18] noted that a compression-based measure of complexity (the Lempel-Ziv measure) could only account for 2.25% of the variances in the human data (r = 0.15), whereas their Povel-Shmulevich measure could account for 56.25% (r = 0.75). They attributed this to two factors: first, that the Lempel-Ziv measure is unsuitable for short sequences; and second, that the Povel-Shmulevich measure is an empiricallytested perceptual model and therefore it is likely to do better than the Lempel-Ziv measure. Table 1. Complexity data from Shmulevich and Povel [18]. Here a bar represents a tone in middle C, a dot represent a rest, and the events are spaced 200ms apart. Pattern Comp. Pattern Comp. |||||..||.|.|... 1.56 ||..||.||||.|... 2.64 |||.|.|||..||... 2.12 ||..||.|||.||... 3.24 |.|||.|||..||... 2.08 |||||.||.|..|... 3.08 |.|.|||||..||... 1.88 ||||.|..|||.|... 3.04 |..||.|.|||||... 1.80 |||..||.|||.|... 3.04 |||.|||.||..|... 2.44 |.|||..|.||||... 2.56 |.||||.||..||... 2.20 |.|..||||.|||... 2.56 ||..|||||.|.|... 2.56 ||||.|.|..|||... 2.84 ||..|.|||.|||... 3.00 ||.|||.|..|||... 3.60 |.|||.||||..|... 2.04 ||.|..|||.|||... 2.68 |||.||..||.||... 2.76 |.||||.|..|||... 3.28 ||.||||.|..||... 2.72 |..|||||.||.|... 3.08 ||.||.||||..|... 3.00 ||||.|||..|.|... 3.52 ||..||.||.|||... 3.16 ||||..||.||.|... 3.60 |..|||.|||.||... 2.04 ||.||||..||.|... 3.04 ||.||||.||..|... 2.88 ||.|..|||||.|... 2.88 ||.|||.|||..|... 2.60 |.|..|||.||||... 3.08 ||.|||..||.||... 2.60

The second experiment was done by Conley [6]. In this seminal work on the perception of complexity in actual art music, Conley specified 10 predictors of musical complexity and set out to correlate them to human judgements of musical complexity. In her experiments, selected Beethoven Eroica Variations (played by Sviatoslav Richter, Angel S-40183) were used as her stimuli (see Table 2). These stimuli were chosen because of their ecological validity [6]. The effect of musical training is controlled for by dividing the subjects into Graduate, Sophomore and Non-major groups [6]. Her main results are reproduced in Table 3. For this particular experiment, musical training had an effect and that the best predictor of complexity for all three groups is the rate of rhythmic activity, accounting for more than 70% of the variances in each group [6]. At the end of her paper, Conley was cautious to warn us that her results might not necessarily generalise to any other settings.

3

Multi-Layer Research Strategy

Implicit in our previous paper [5] is the following research strategy (Table 4) which we will expand on below. Our strategy consists of four layers. The Data Layer corresponds to low-level perceptual inputs (e.g., the OPM format). Its role is to provide the cul-

Table 2. Complexity stimuli from Conley’s [6] experiment. Var. 1 2 3 4 5 6 7 8

Table 3.

Title Introduzione col Basso del Tema A due A tre A quattro Tema Variation I Variation II Variation III

Var. 9 10 11 12 13 14 15 16

Title Variation IV Variation V Variation VIII Variation IX Variation X Variation XI Variation XII Variation XIII

Mean standardised complexity data from Conley’s [6] experiment. Var. Graduate Sophomore Non-major 1 -1.65 -1.51 -1.17 2 -0.57 -0.99 -1.07 3 -0.90 -1.09 -0.95 4 0.12 0.14 0.05 5 -0.64 -0.48 -0.22 6 0.00 0.40 0.63 7 1.12 1.89 1.39 8 0.88 0.62 0.29 9 -0.24 0.11 0.04 10 -0.08 -0.78 -0.53 11 0.23 0.09 0.15 12 0.20 0.45 0.65 13 1.49 0.93 0.62 14 -0.29 -0.39 -0.07 15 0.69 0.64 0.60 16 -0.36 -0.02 -0.39

tural molecules from which the memetic codes are built. The Information Layer corresponds to our memetic information measure [5], where its output should ideally be validated by psychophysical experiments. The Psychology Layer would include aesthetic fitness, categorisation, familiarity, similarity, and so on, where each component must be validated psychologically. Finally, the Application Layer would include, inter alia, creative systems, evolutionary musicology and music information retrieval. The advantages of multilayer modelling is that it allows us to reuse the same Application and Psychology Layers to describe phenomena across different domains (e.g., music, paintings and mathematics) while requiring only changes in the Information and Data Layers. Table 4. Our multi-layer research strategy. Level 4 3 2 1

4

Layer Application Psychology Information Data

Examples Creative systems Aesthetic fitness, familiarity, etc. H(x), H(x, y), H(x|y), I(x; y) OPM input

Main Contentions of Our Information Model

In examining the internal validity of our information measure, we noted that there are two potential contentions: the short sequence objection and the polyphonic objection. We will address them in turn.

4.1 Short Sequence Objection As reviewed above, Shmulevich and Povel [18] found that a compression-based measure of complexity (Lempel-Ziv) could only

account for 2.25% of the variances in human judgements of rhythmic complexity. This raises the question whether our compression-based information measure would perform as poorly for short sequences. The following replication of [18] will provide further evidence for or against our model.

4.1.1

Experiment 1

Stimuli All thirty-five rhythmic patterns from Shmulevich and Povel’s [18] experiment (as shown in Table 1). Methods Stimuli were entered into a computer and then converted into the OPM format [5]. The cognitive information H [5] of all 35 OPM files were then calculated. The correlation between the values thus calculated and the human judgements in Table 1 are then reported, along with its statistical significance. Results There is a statistically significant correlation between our music information measure and the human data, r(33) = 0.44174, p < 0.01. Discussion Our results are much better than the Lempel-Ziv measure (r = 0.15) but not as good as the Povel-Shmulevich measure (r = 0.75). One way to look at it is that the Povel-Shmulevich measure deals only with beat music and does not generalise to polyphonic music; nor does it recognise any pitch-based features in the first place. Music like Gregorian chants are defined mainly by their pitches, so the Povel-Shmulevich measure would not be applicable. With more data, we can expect to claim that our model is a reasonable trade-off between model specificity and sensitivity.

4.2

The Polyphonic Objection

Our choice [5] of the OPM format was based on Narmour’s [14] theory which was originally proposed for monophonic music. The validity of our extrapolation (to the polyphonic domain) is not yet demonstrated. It is possible that our model might not be able to handle polyphonic music well. To test this hypothesis, we have duplicated Conley’s experiment [6] below.

4.2.1

Results Results are shown in Table 5. The correlations are statistically significant for the Sophomore and Non-major groups, but not for the Graduate group. Table 5. Correlation with Conley’s [6] data (Experiment 2) Group r Graduate 0.41527 Sophomore 0.66564** Non-major 0.59758** * p < 0.05, ** p < 0.01

df 14 14 14

Discussion The results showed that our information measure is able to model the Sophomore and Non-major judgements reasonably well, but has failed on the Graduate data. One possible explanation is that, as all the Beethoven Variations contained the same theme (Var. 1), there might be a priming effect where the main theme was memorised by the subjects at least partially (more so by the Graduate group, assuming that they have a better memory for music than the other two groups). If this is the case, then the conditional cognitive information (conditioned on Var. 1) might be a more appropriate measure of judged complexity. We explore this possibility in the next Experiment.

4.2.2 Experiment 3 Stimuli The same stimuli used in Experiment 2. Procedure The procedure was the same as that of Experiment 2, except that the conditional cognitive information is used (conditioned on Var. 1). Results Results are shown in Table 6. The correlations are statistically significant for all three groups. Table 6. Correlation with Conley’s [6] data (Experiment 3) Group r Graduate 0.47790* Sophomore 0.67563** Non-major 0.60899** * p < 0.05, ** p < 0.01

df 14 14 14

Experiment 2

Methods Stimuli All sixteen Beethoven Eroica Variations from Conley’s [6] experiment (as shown in Table 2). Conley [6] used a Sviatoslav Richter recording (Angel S-40183) which is out-of-print and, even if available, would be extremely difficult to convert into proper MIDI files. To establish an approximate correspondence with her experiment, we used a publicly available MIDI performance by Bunji Hisamori (several versions are published on the Internet; we used his “Revision 2”, dated July 1999). Procedure Stimuli were converted into the OPM format [5]. The cognitive information H of all 16 OPM files were then calculated. The correlation between the calculated values and the human judgements in Table 3 were then reported (along with statistical significances).

Discussion With the priming assumption, results added support to our cognitive information model. By comparing the r-values in the two tables, we can see that the priming assumption produced the highest improvement for the Graduate group, while producing only negligible improvements for the Sophomore and Non-major groups. We interpret this to mean that the Graduate group had a better memory for music and were able to memorise the main theme while listening. The percentage of variances accounted for are 23%, 46% and 37%, respectively, which are not as good as Conley’s best model. However, when we apply the best Conley model (rate of rhythmic activity) to Shmulevich and Povel’s [18] rhythmic patterns (Table 1), we obtained a constant for all 35 patterns because they all have the same number of sounding notes per minute. This certainly does not fit the data. Therefore, we claim that our model avoids overfitting and that it scores better in generalisability.

5

New Interpretations of Memetic Fitness

Now we move up to the Psychology Layer and consider interpretations of our fitness measure beyond the Wundt Curve. Positive Correlation Interpretation According to our previous definition of fitness [5], for a constant level of familiarity, the musical fitness of a piece of music x correlates positively with the squared complexity of x, specifically, unity × diversity ∝ H(x)2 .

(7)

Combined Interpretation See Figure 2 for a three-dimensional representation combining both the Wundt Curve interpretation [5] and the Positive Correlation interpretation (above). This half-saddle figure is a mathematical consequence of our fitness measure [5] and unites several seemingly contradicting experimental results: • Inverted-U relationship [7, 11, 21, 22, 15] • Positive correlation [22]; can also be interpreted as the left half of the inverted-U • Negative correlation [19]; can also be interpreted as the right half of the inverted-U • No correlations [16]

The most interesting case is that of no correlations. Orr and Ohlsson [16] suggests that “musical expertise dissolves the [inverted-U] relationship between liking and complexity” (p. 583). Here we offers a different explanation (restricted sampling). We claim that the majority of jazz consists of familiar elements followed by unfamiliar elements (themes and free improvisations), which corresponds to very low and very high familiarity. As half-familiar elements are rarer in jazz, a more careful sampling methodology is required in experiments. In other words, it is possible that Orr and Ohlsson has not acquired a representative sample of our proposed half saddle surface. Musical Fitness (bits^2)

25 25 20

7

Potential Applications to Creative Systems

The fitness surface detailed above could be a useful theoretical tool as well as a practical heurestic for evolutionary creative systems. For systems that use only our Psychology Layer for fitness evaluation, there is a possibility to “reverse-engineer” their underlying Information Layer (one that is different from our current formulation). For example, in an agent-based creative system, we can define custom complexity and familiarity measures based on other theories, with the requirement that familiarity ∈ [0, 1]. To “reverse-engineer” their underlying information measures, we could simply relabel the H(x|y) complexity measure as H(x) and the familiarity measure as H(x) . Then, one can use these “reverse-engineered” information measures to predict other psychological properties of the creative system, using other components of the Psychological Layer. However, note that it may or may not be possible to properly4 reconstruct the full range of information measures (such as mutual information) as they depend on the exact form of the custom measures.

ACKNOWLEDGEMENTS The first author is supported by a bursary from the Department of Computing at Goldsmiths’ College, University of London. We would like to thank Jia Yang and Ulrich Speidel for sharing their C implementation of their fast T-decomposition [23] algorithm.

20

15

15

10

10 5

5

0 0

100

0 80

2 60

4 40

6 Complexity (bits)

20

8

Familiarity (%)

0

Figure 2. Musical fitness as a function of stimulus complexity and familiarity.

6

cognition. While good models would require some goodness-of-fit, beyond a certain point the extra goodness-of-fit could mean overfitting, thus reducing the generalisability of the model [17]. Our memetic approach might potentially suffer from overgeneralisation, or the possibility of ignoring crucial cognitive constraints (imprecise psychology); however, this approach would avoid overfitting, as we have discussed at the end of Experiment 3, and this might actually be a good thing when we have to consider more general, multi-purposes hypotheses at the socio-cultural level (such as creative systems).

General Discussion

Pitt and Myung [17] stated that if experimental data are noisy then standard goodness-of-fit measures (such as the square of the correlation coefficient) may not be the best way to compare models of

REFERENCES [1] R. C. Atkinson and R. M. Shiffrin, ‘Human memory: a proposed system and its control processes’, in The Psychology of Learning and Motivation, Vol. 2, eds., K. W. Spence and J. T. Spence, 89–195, Academic Press, New York, (1968). [2] C. H. Bennett, P. Gacs, M. Li, P. M. B. Vit´anyi, and W. Zurek, ‘Information distance’, IEEE Transactions on Information Theory, 44(4), 1407–1423, (1998). [3] M. L. Best, ‘Towards computational memetics’, Journal of Memetics– Evolutionary Models of Information Transmission, 4, (2001). [http://jom-emit.cfpm.org/2001/vol4/editorial.html]. [4] T.-S. T. Chan and G. A. Wiggins, ‘Computational memetics of music: memetic network of musical agents’, in Proceedings of the ESCOM 10th Anniversary Conference on Musical Creativity, (2002). [5] T.-S. T. Chan and G. A. Wiggins, ‘A computational memetic approach to music information and aesthetic fitness’, in Proceedings of the IJCAI’05 Workshop on Computational Creativity, eds., P. Gerv´as, T. Veale, and A. Pease, pp. 105–108, Madrid, (2005). Departamento de Sistemas Inform´aticos y Programaci´on, Universid Complutense de Madrid. [6] J. K. Conley, ‘Physical correlates of the judged complexity of music by subjects differing in musical background’, British Journal of Psychology, 72, 451–464, (1981). 4

Purists would require the information measures to obey a number of inequalities for Shannon entropy and Kolmogorov complexity [10].

[7] J. B. Crozier, ‘Verbal and exploratory respnses to sound sequences varying in uncertainty level’, in Studies in the New Experimental Aesthetics: Steps Toward an Objective Psychology of Aesthetic Appreciation, ed., D. E. Berlyne, 27–90, Hemisphere, Washington, DC, (1974). [8] A. de Cheveign´e, ‘Separation of concurrent harmonic sounds: fundamental frequency estimation and a time-domain cancellation model of auditory processing’, Journal of the Acoustic Society of America, 93(6), 3271–3290, (1993). [9] H. Eysenck, ‘The experimental study of the “good Gestalt”: a new approach’, Psychological Review, 49, 344–364, (1942). [10] D. Hammer, A. Romashchenko, A. Shen, and N. Vereshchagin, ‘Inequalities for shannon entropy and kolmogorov complexity’, Journal of Computer and System Sciences, 60(2), 442–464, (2000). [11] R. G. Heyduk, ‘Rated preference for musical compositions as it relates to complexity and exposure frequency’, Perception and Psychophysics, 17(1), 84–91, (1975). [12] S. Jan, ‘The memetics of music and its implications for psychology’, in Proceedings of the 6th International Conference on Music Perception and Cognition, (2000). [13] J. Jeong, M. K. Joung, and S. Y. Kim, ‘Quantification of emotion by nonlinear analysis of the chaotic dynamics of electroencephalograms during perception of 1/f music’, Biological Cybernetics, 78(3), 217– 225, (1998). [14] E. Narmour, ‘Hierarchical expectation and musical style’, in The Psychology of Music, ed., D. Deutsch, 441–472, Academic Press, San Diego, (1999). [15] A. C. North and D. J. Hargreaves, ‘Subjective complexity, familiarity, and liking for popular music’, Psychomusicology, 14, 77–93, (1995). [16] M. G. Orr and S. Ohlsson, ‘Relationship between complexity and liking as a function of expertise’, Music Perception, 22(4), 583–611, (2005). [17] M. A. Pitt and I. J. Myung, ‘When a good fit can be bad’, Trends in Cognitive Sciences, 6(10), 421–425, (2002). [18] I. Shmulevich and D.-J. Povel, ‘Measures of temporal pattern complexity’, Journal of New Music Research, 29(1), 61–69, (2000). [19] J. D. Smith and R. J. Melara, ‘Aesthetic preference and syntactic prototypicality in music: ’Tis the gift to be simple’, Cognition, 34, 279–298, (1990). [20] M. R. Titchener, ‘A measure of information’, in Proceedings of the Data Compression Conference (DCC’00), eds., J. A. Storer and M. Cohn, pp. 353–362, Los Alamitos, CA, (2000). IEEE Computer Society. [21] P. C. Vitz, ‘Affect as a function of stimulus variation’, Journal of Experimental Psychology, 71(1), 74–79, (1966). [22] E. L. Walker, ‘Psychological complexity and preference: a hedgehog theory of behavior’, in Pleasure, Reward, Preference: Their Nature, Determinants, and Role in Behavior, eds., D. E. Berlyne and K. B. Madsen, 65–97, Academic Press, New York, (1973). [23] J. Yang and U. Speidel, ‘A T-decomposition algorithm with O(n log n) time and space complexity’, in Proceedings of the 2005 International Symposium on Information Theory, pp. 23–27, Piscataway, NJ, (2005). IEEE.