Grapheme and Letter Name Based Patterns in Synesthetic Colors
E298
Edward Hubbard, Pearl Ambrosio, Shai Azoulai, & V.S. Ramachandran Center for Brain and Cognition, UC San Diego
Correlation Methods
Introduction Synesthesia is the experience of perceptual sensations in one domain eliciting perceptual sensations in another domain. Various subtypes of synesthesia that have been reported include taste-shape synesthesia, tonecolor synesthesia, and grapheme-color synesthesia1,2. Recent evidence has suggested that synesthetic experiences may arise at distinct stages of neural processing3,4. Additionally, anecdotal evidence from synesthetes suggest that their colors are non-random. If so, then the patterns of synesthetic color experience may be useful in determining the representational stage at which colors are elicited for individual synesthetes. To further investigate the possibility that distinct representations underlie synesthesia in different subjects, we measured correlations among synesthetic experiences of grapheme-color synesthetes. Anecdotal evidence has suggested that some synesthetes have a feature dimension along which their colors are arranged. For example, some synesthetes have reported similar colors for letters that look alike, while others have reported similar colors for letters that sound similar5. We performed a correlational analysis to test this hypothesis. The two dimensions analyzed were visual features of each letter and the sound of the letter names.
Methods y Data from 260 grapheme-color synesthetes (courtesy Sean Day) collected from personal correspondence and historical sources were analyzed. y Of those, 97 had complete data sets, which were then analyzed. y A sound confusion matrix was used to select two sets of five letters that were highly confusable within each group, and easily distinguished across groups. y Based on a letter confusion matrix, two sets of five letters were chosen that had similar visual features within each group, and differing visual features across groups6. y Visual features: K V W X Y and C U D O Q. y Letter names: B P C T V and M N S X F. y To assess distance between letters in color space, a normalized Euclidian distance was calculated:
1- 2*
___________________________________________________________________ √____________________________________________________________________ (Red Gun1 - Red Gun2)2 + (Blue Gun1 - Blue Gun2)2 + (Green Gun1 - Green Gun2)2 ___________________ √(255)2 + (255)2 + (255)2
y Normalized distance ranges from -1 to 1 (analogous to a correlation coefficient). y We then computed distance for groups of letters defined by similar visual features or letter names. y Predict high within group similarity, but low between group similarity. y To determine which group a synesthete belonged to we compared correlations for visual features and letter names.
R 0 178 255 204 255 0 76 255 255 204 191 0 0 255 255 255 204 255 0 0 153 178 178 178 178 0
G 0 178 255 76 204 128 89 153 255 153 76 64 128 127 255 51 153 0 0 128 51 178 178 178 178 0
Letter Names B 255 178 0 0 229 0 0 204 127 0 166 127 0 127 0 0 0 0 255 0 0 178 178 178 178 0
B P C T V M N S X F
B
P
C
T
V
M
N
S
X
F
1.00
-0.05
0.06
-0.16
1.00
-0.16
0.52
-0.19
1.00
-0.16
-0.05
1.00
0.08
-0.21 -0.05 -0.21
0.33
-0.65 -0.05 -0.21 -1.00
0.06
0.08
-0.29
0.18
-0.16
1.00
-0.29 -0.29 -0.16
1.00
Within
0.69
0.19
0.06
-0.16
1.00
-0.16
0.52
1.00
-0.16
Between
0.07
0.06
-0.16 -0.21 -0.29
1.00
-0.16
1.00
-0.29 -0.29 -0.16
1.00
Difference
0.58
0.13
0.52
-0.29
0.52
-0.29
1.00
-0.29
0.33
-0.29
-0.41
0.52
-0.19 -0.65 -1.00 -0.29 -0.19 -0.29 -0.41
1.00
-0.19 -0.29
1.00
-0.19
-0.05
0.18
-0.19
0.06
Letter Names
0.06
1.00
1.00
-0.05
1.00
Visual Features
-0.29
-0.16 -0.21 -0.29
0.06
-0.16
1.00
-0.16
0.52
1.00
-0.16
-0.16 -0.21 -0.29
1.00
-0.16
1.00
-0.29 -0.29 -0.16
1.00
Overall Difference
0.35
Type of Synesthete
Visual Features
Number
Percentage
Visual Features
20
22%
Letter Names
17
18%
Unclassifiable
60
60%
Conclusions
Visual Features K
V
W
X
Y
C
U
D
O
Q
K
1.00
0.53
0.53
0.53
0.53
-0.14
0.22
0.25
-0.14
0.17
V
0.53
1.00
1.00
1.00
1.00
0.06
0.00
0.06
0.06
0.18
W X Y C U D O Q
0.53
1.00
1.00
1.00
1.00
0.06
0.00
0.06
0.06
0.18
0.53
1.00
1.00
1.00
1.00
0.06
0.00
0.06
0.06
0.18
0.53
1.00
1.00
1.00
1.00
0.06
0.00
0.06
0.06
0.18
-0.14
0.06
0.06
0.06
0.06
1.00
-0.04
0.16
1.00
0.48
0.22
0.00
0.00
0.00
0.00
-0.04
1.00
0.74
-0.04
0.48
0.25
0.06
0.06
0.06
0.06
0.16
0.74
1.00
0.16
0.65
-0.14
0.06
0.06
0.06
0.06
1.00
-0.04
0.16
1.00
0.48
0.17
0.18
0.18
0.18
0.18
0.48
0.48
0.65
0.48
1.00
‘WA’ has been classified as a visual features synesthete because not only does she have high correlations for the ‘within’ groups of visual features and low correlations for the ‘between’ groups, but she also has a weak correlation for letter names.
_______________________
1- 2 *
Classification Correlation Matrices
Raw Data (WA) Letter Color A Blue B Grey C Yellow D Brown / Orange E White / Pink F Green G Green / Brown H Pink I White / Yellow J Yellow / Brown K Pink / Purple L Green / Blue M Green N Red / Yellow O Yellow P Red / Orange Q Yellow / Brown R Red S Blue T Green U Brown V Grey W Grey X Grey Y Grey Z Black
Data Summary
√________________________ (R1 - R2)2 + (B1 - B2)2 + (G1 - G2)2
_________________
y We were able to classify some of the synesthetes along two different dimensions. y Many other dimensions may exist along which synesthetes could classify their letters, e.g. the motor commands which define the way letters are written or the way the letter sounds are articulated. y These results help us to gain insight into how synesthetic colors may be acquired and reinforced. y These results suggest that, for some synesthetes, colors are elicited based on letter form, and for some based on letter names. y These results are consistent with other recent work suggesting that synesthesia may be elicited at multiple stages of representation.
√(255)2 + (255)2 + (255)2
Future Directions Correlation Matrices
Raw Data (PF) Letter Color A Orange B Orange/Pink C Light Grey D Yellow E Light Green F Light Green G Yellow / Brown H Purple I Light Gray J Blue K Light Gray L Light Gray M Green N Green O White P Pink Q Yellow R Red S Red T Brown U Brown V Purple W Black X Black Y Yellow Z Black
R 255 255 178 255 0 0 204 128 178 0 178 178 0 0 255 255 255 255 255 153 153 128 0 0 255 0
G 102 127 178 255 128 128 153 0 178 0 178 178 128 128 255 153 255 76 0 51 51 0 0 0 255 0
Letter Names B 0 102 178 0 0 0 0 128 178 255 178 178 0 0 255 204 0 102 0 0 0 128 0 0 0 0
B
P
C
T
V
B P
1.00
0.52
0.45
0.26
0.18
-0.24 -0.24
0.52
1.00
0.62
-0.13
0.04
-0.48 -0.48 -0.15 -0.63 -0.48
C T
0.45
0.62
1.00
0.00
0.13
-0.16 -0.16 -0.19 -0.39 -0.16
0.26
-0.13
0.00
1.00
0.37
0.22
0.22
0.48
0.27
0.22
V
0.18
0.04
0.13
0.37
1.00
0.00
0.00
0.18
0.18
0.00
M N
-0.24 -0.48 -0.16
0.22
0.00
1.00
1.00
-0.29
0.42
1.00
-0.24 -0.48 -0.16
0.22
0.00
1.00
1.00
-0.29
0.42
1.00
S X F
0.26
M
N
S 0.26
X
F
-0.37 -0.24
-0.15 -0.19
0.48
0.18
-0.29 -0.29
1.00
-0.15 -0.29
-0.37 -0.63 -0.39
0.27
0.18
0.42
0.42
-0.15
1.00
0.42
-0.24 -0.48 -0.16
0.22
0.00
1.00
1.00
-0.29
0.42
1.00
D
O
Q
0.39
0.06
Visual Features
Letter Names
Within
0.19
0.43
Between
-0.09
-0.09
Difference
0.28
0.52
Overall Difference
-0.24
Type of Synesthete
Letter Name
Visual Features K
V
W
X
Y
C
U
K V
1.00
0.13
-0.40 -0.40
0.05
1.00
0.00
0.06
0.13
1.00
0.18
0.18
-0.41
0.13
0.37
-0.41 -0.41 -0.41
W X
-0.40
0.18
1.00
1.00
-0.63 -0.40
0.27
-0.63 -1.00 -0.63
-0.40
0.18
1.00
1.00
-0.63 -0.40
0.27
-0.63 -1.00 -0.63
Y C U D
0.05
-0.41 -0.63 -0.63
1.00
0.06
-0.03
0.13
-0.40 -0.40
0.06
1.00
0.00
0.06
0.00
0.37
0.27
0.27
-0.03
0.00
1.00
-0.03 -0.54 -0.03
0.06
-0.41 -0.63 -0.63
1.00
0.06
-0.03
1.00
-0.15
1.00
O Q
0.39
-0.41 -1.00 -1.00 -0.15
0.39
-0.54 -0.15
1.00
-0.15
0.06
-0.41 -0.63 -0.63
0.06
-0.03
-0.15
1.00
1.00
1.00
1.00
1.00
-0.15 0.39
1.00 0.06
‘PF’ has been classified as a letter name synesthete because not only does she have high correlations for the ‘within’ groups of letter names and a slightly negative correlation for the ‘between’ groups, but she also has weak correlations for visual features.
y Compare synesthetes’ introspective reports with our results to determine if they are consistent with their experiences. y Further develop the correlation method by expanding our analysis to all 26 letters e.g., MDS. y Investigate other possible dimensions along which synesthetic experiences can be organized. y Implement a bootstrapping technique in order to provide more reliable results than a correlational analysis.
References 1. Ramachandran, V.S. & Hubbard, E.M. (2001). Synesthesia: A window into perception, thought and language. Journal of Consciousness Studies. 8(12): 3-34. 2. Rich, A. & Mattingley, J. (2002). Anomalous perception in synaesthesia: A cognitive neuroscience perspective. Nature Reviews Neuroscience. 3(1):43-52. 3. Hubbard, E.M., Arman, A.C., Ramachandran, V.S. & Boynton, G.M. (2005). Individual differences among grapheme-color synesthetes: Brain-behavior correlations. Neuron. 45(6):975-85. 4. Dixon M.J., Smilek, D. & Merikle, P.M. (2004). Not all synaesthetes are created equal: Projector versus associator synaesthetes. Cognitive, Affective and Behavioral Neuroscience. 4(3):335-343. 5. Day, S.A. (2005). Some demographic and socio-cultural aspects of synesthesia. In L. Robertson and N. Sagiv (Eds.) Synesthesia: Perspectives from Cognitive Neuroscience. (p. 11-33). 6. Gibson, E.J., Osser, H., Schiff, W., & Smith, J. (1963). An analyses of critical features of letters, tested by a confusion matrix. In A Basic Research Program on Reading. (Cooperative Research Project No. 639). Washington, DC: U.S. Office of Education.
