TECHNOLOGIES OF REPRESENTATION: IMAGES, VISUALISATIONS AND TEXTS Position piece for ‘Computational Turn’ Annamaria Carusi Oxford e-Research Centre University of Oxford OX1 3QG [email protected] [Instead of a position piece as such, I have set out the main lines of my ongoing research on images and visualisations in different domains, scientific and humanistic, and I have pointed towards recent publications of mine, rather than reproduce them.] As technologies for visually rendering data of all kinds continue to develop in their capacity and power at an impressive rate, the research and knowledge domains in which they are deployed are fundamentally challenged at the very same time – and by the very same token – as they are been supported and facilitated. The natural and medical sciences have traditionally been immersed in a visual culture, and have a long history of co-evolution with imaging and visualization technologies: from the telescope to the microscope; the whole range of imaging technologies available, for example, to the biomedical sciences; and the intricate relationship between modeling, simulation and visualization that is currently sweeping across so many scientific fields. There have always been areas of the humanities which have been directed primarily towards visual artifacts, but traditionally, the work of the humanist has occurred mainly through the medium of text. Recent developments in the digital humanities are changing this, with imaging and visualizing technologies increasingly coming to the fore, both in traditionally visual domains (art history, film studies, archaeology) and in traditionally textual domains (classics, history, philosophy). The wide range of visual renderings does not necessarily have a common thread going through it. The visual outputs of imaging, so-called ‘scientific’ visualization and information visualization are all very different in their nature, and in the analytical and interpretive possibilities for which they allow. An image of a work of art or of a document, an image of a cell observed through a microscope, a visualization of the results of text mining, a visualization of an in silico experiment are all visual renderings, but because of their very different relations with that which they represent, and the very different ways in which they are constructed as representations, require very different epistemic attitudes on the part of researchers using them. There is an extremely large research and industry community around the production of computational and engineering resources for imaging and visualisation. The discourse in which these developments is embedded makes frequent appeal to the cognitive enhancement brought about by computational methods for visually rendering data. For example:

Scientists need an alternative to numbers. A technical reality today and a cognitive imperative tomorrow is the use of images. The ability of scientists to visualize complex computations and simulations is absolutely essential to ensure the integrity of analyses, to provoke insights, and to communicate those insights with others.“ (De Fanti & Brown, Advances in Computing, 1991)

The discourse on imaging and visualization often draws upon an implicit cognitivism: For example, Visualizations can expand processing capability by using the resources of the visual system directly. Or they can work indirectly by offloading work from cognition or reducing working memory requirements for a task by allowing the working memory to be external and visual.... Visualizations allow some inferences to be done very easily that are not so easy otherwise (Card, Mackinlay, and Shneiderman 1999, p.16).

And the new capacities for computational visual rendering are touted as bringing about new modes of thinking: ‘Computer graphics, because it bonds minds and machines in a unique partnership, creates an entirely new way of thinking’. (Freidhoff & Benson, 1989) Sociologically informed science and technology studies have also developed a characteristic approach to images and visualizations, focusing on the way in which socially shared practices form scientific evidence (Latour 1986, Lynch & Woolgar, 1990, Goodwin 1994, Sharrock & Coulter 1998).These approaches have also been used to inform ideas about the design and development of imaging technologies (De La Flor et al, forthcoming). This paper, instead, brings to the fore humanities approaches to imaging and visualization. The humanities have a rich and deep tradition of understandings of representation and interpretive techniques to make sense of them. With some notable exceptions, this tradition has been under-represented in the general discourse and conceptual framework informing the development and deployment of imaging and visualization technologies. Using a humanities approach brings about a more reflexive understanding of the tools used for conducting research, paramount for the humanities, but also, I shall argue, for other disciplines. In addition, it offers a route out of narrowly cognitivist and representationalist conceptions of imaging and visualization, offering instead a richer account of the inter-relationship between acts of interpretation, modality, and the ontological commitments made by researchers in the process of their research.

A fundamental question is that of the distinction between imaging and visualisation. These are very different visual renderings, with images being akin to photographs and visualisations to graphs. The scientific literature tends to draw the distinction on the basis of the existence or not of a corresponding 3D object ‘in the world’. Thus in an image of cell tissue, of a painting, or of a text there is a corresponding 3D object, whereas a graph of the rate of increase of taxation over the last 5 years does not have a corresponding 3D object. However, this is made more complex in the case of visualisations of biological processes, which are hybrid visual artefacts, including both image-like and graph-like elements. A clearer understanding of this distinction is essential, in particular since they have fundamentally different implications for interpretation. There are several possible approaches to this question: epistemological, ontological, aesthetic, perceptual, hermeneutic and semiotic approaches will each draw out the distinction in different ways. To show some of the issues involved in considering the distinction between different kinds of visual artefacts, we can consider a proposal put forward by Don Ihde (1990, 1991), who claims that there are several ways in which technologies mediate the relation between humans and their environments in general, and between scientists and their objects of study in particular. One of these is the embodiment relation, whereby the technology enables human beings to do better something (usually perceptual) that they could already do. The technology functions optimally when it recedes from awareness, and becomes as one with whatever human functionality it is aiding or optimising. Spectacles are a good example, and microscopes (as well as hearing aids and telescopes) are in the same category. These are technologies we see (or hear, feel, etc.) through. Ihde indicates this relation as follows:

(human-technology) -> world

to show that the human and the technology operate ‘as one’, at least from the phenomenological or experiential point of view. And similarly, from this point of view, they operate ‘as one’ in their directedness towards, or their intending something in the world around them. Many of the imaging techniques that enhance our view of objects which are otherwise difficult to see and interpret, are of this form. Another relation is the hermeneutic relation. In this relation, the technology does not form one with the human perceiver, but rather shifts over to the world, or the parts of it towards which perception is directed. In this case the relation looks like this:

Human -> (technology-world)

In this relation, the technology represents something to the human perceiver, who must interpret it. The best example of this is a thermometer, which represents temperature rather than making us feel it. An act of interpretation is needed on our

part to understand it. A third type of relation is what Don Ihde calls the ‘alterity relation’. This relation occurs when technologies are experienced as quasi-other: not quite human or simply enabling our own faculties to operate beyond their usual limits, but something with which we are able to interact in the true sense: that is, it responds to us without being entirely controllable by us. Playing with a spinning top is an example of an alterity relation, since once it is in movement, it is unpredictable exactly how it will move; it becomes for this reason an object of fascination in itself. The alterity relation is illustrated as follows:

Human -> technology-(-world)

where the position of the parentheses is meant to show that ‘there may be, but need not be, a relation through the technology to the world (although it might well be expected that the usefulness of any technology will necessarily entail just such a referentiality)’ (1990: 107) These three types of relations between humans, technologies and that which is perceived demarcate a fundamentally different phenomenology of perception. In the first, the orientation is all towards the object, and the technology, though present, is not present or only peripherally present to awareness. In the second, the technology still delivers to us some aspect of the world that we perceive through our interpretation thereof; in principle, it delivers to perception a representation of something which is independent of both ourselves and of itself (temperature, movement, shapes). In the third, the technology does not recede; that which is seen is couched in it; it takes something with no material existence and gives it material form. At first it would seem that images (for example, those of ancient documents, as has been described in Tarte, forthcoming, and de la Flor et al, forthcoming) fall in the first relation, and the visualisations that result from mathematical modelling in the second. Yet, in both cases, this is problematic. In the first case (imaging) the role of mathematical processes in the imaging techniques, as well as the essentially interpretive activity involved in attempting to reconstruct what the image is of, result in what appears merely to be an enhancement (the first relation), becoming a hermeneutic relation (the second relation). In the second case (visualising a mathematical model), what should be a case of the hermeneutic relation (the second relation) begins to shift towards the alterity relation (the alterity relation). This is because of the way in which the visualisation begins to be considered for its own sake – even as an object of fascination in and of itself (see Araya 2003, Carusi 2008) – and begins to ‘gain a life of its own’ and to be considered apart from the thing towards which is supposedly directed. Recently Peter-Paul Verbeek (2008) has extended Ihde’s framework by two further notions, that of hybrid intentionality and composite intentionality. These different modes of mediation entail different roles of the technologies in interpretation and analysis. Understanding which form of mediation between humans, the entities, events and processes that they are studying and the whole environment in which this

occurs, allows for a better understanding of how the research enterprise is subtly shaped in that complex set of relations. Evidently this is not the only framework that can be used to approach these questions and is meant here only as an example of a way of thinking about these artefacts which will enable us to ask a different range of questions than those standardly asked within engineering or cognitivist paradigms. I believe that there are several lines of thought in Merleau-Ponty’s later work that it will be extremely promising to pursue and to develop further. Works such as Visible and Invisible and Nature: Course Notes from the Collège de France (2003) show a way in which it may be possible to move beyond the distinction between constructivism and realism, and explore different ways in which perception is bound up with epistemology and ontology. Instead of opposing perception and being, Merleau-Ponty suggests that they are ‘in a circuit’ (2003: 209) with each other – this is a difficult metaphor, but the implication is that there is no clear cut differentiating point between the one and the other, and that starting at one point leads to the other point and back, endlessly. In scientific and other research domains, the perceptual apparatus crucially involves technologies for observation and for visualisation. How are these technologies involved in the circuit? Questions to be considered include: Is mediation the right way to think of the role of technologies in computational representations? What are the alternatives? What is the difference between a photographic or similar image and a visualization based on information, modeling and simulation? What is the nature of the relation between representers and interpreters, representing technologies and the represented thing? What are the implications for interpretation? The aim of my research is to explore answers that are put forward in the philosophy of science, philosophy of interpretation, semiotic, hermeneutic and phenomenological traditions, and to show the import of the answers on the general approach to design, development and use of images and visualisations for research purposes. I have two case studies to put forward as initial explorations of the territory sketched out above: one in philosophy and one in computational biology. The first case study is of different renditions of Plato’s Allegory of the Cave, showing how visualising technologies play a role in forming epistemological thought. It is drawn from my paper ‘Philosophy Engines’ (Carusi 2009), and compares Plato’s own rendition of the allegory, and the new media version in McKenzie Wark’s Gamer Theory (2007) The second case study is of the differing visual practices of biologists, with a comparison of the visualisations used by mathematical and computational biologists, and the microscopial observations and images used by cell biologists. This case study is drawn from two papers: ‘Scientific visualisations and aesthetic grounds for trust’ (Carusi 2008), and ‘Computational Biology and the Limits of Shared Vision’ (Carusi, forthcoming) and shows the role played by visual artefacts in creating common modes of perception among scientists, and also, more profoundly, in revealing interconnections between visibility and biological ontology. The latter work draws upon Merleau-Ponty, Visible and Invisible (1968) and Nature (2003).

References

Araya, A. (2003) The Hidden Side of Visualization. Techné, 7, 27-93. Card, Stuart K., Mackinlay, Jock D., and Shneiderman, Ben. (1999) Readings in Information Visualization. Using Vision to Think. San Francisco: Morgan Kaufmann Publishers, Inc. Carusi, A. (2008) Scientific visualisations and aesthetic grounds for trust. Ethics and Information Technology. Vol 10, 243-254. http://www.springerlink.com/content/9k74725n54571xw3/

Carusi, A. (2009) Philosophy Engines: Technologies and Reading, Writing, Thinking Philosophy. Discourse, vol 8. http://www.prs.heacademy.ac.uk/view.html/PrsDiscourseArticles/69 SSRN: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1506043

Carusi, A. (forthcoming) Computational Biology and the Limits of Shared Vision. Perspectives on Science. DeFanti, Thomas A.; Brown, Maxine D.; McCormick, Bruce H. (August 1989). "Visualization: Expanding Scientific and Engineering Research Opportunities". Computer (IEEE): 12-25. De La Flor, G., Luff, P., Jirotka, M., (forthcoming) The case of the disappearing ox. CHI 2010 Friedhoff, Richard M. and Benzon, William. (1989) Visualization, The Second Computer Revolution. New York: Harry N. Abrams, Inc., Publishers. Goodwin, C. (1994) 'Professional vision', American Anthropologist vol. 96 no. 3, pp. 606–633. Ihde, Don. (1990) Technology and the Lifeworld. From Garden to Earth. Bloomington: Indiana University Press. Ihde, Don (1991) Instrumental Realism. Bloomington: Indiana University Press. Latour, B. (1986) `Visualization and Cognition: Thinking with Eyes and Hands' , Knowledge and Society 6: 1-40. Lynch, M. & Woolgar, S. (eds). (1990): Representation in Scientific Practice. Cambridge MA; London: MIT Press Merleau-Ponty, M. 1968. Visible and Invisible. Translated by A. Lingis. Evanston, Northwestern University Press. Merleau-Ponty, M. and Séglard, D. 1995. Nature. Course Notes from the College de France. Translated by R. Vallier. Evanston, Northwestern University Press. Pauwels, L. (2006) Visual Cultures of Science: Rethinking Representational Practices in Knowledge Building and Science Communication. Dartmouth College Press. Sharrock, W. and Coulter, J. (1998) 'On What We Can See', Theory and Psychology, vol. 8, no.2, pp.147-164. Tarte, S. (forthcoming) Papyrological Investigations: Transferring Perception and Interpretation into the Digital World, LLC. Wark, M. (2007) Gamer Theory. Harvard University Press. Web version: The Institute for the Future of the Book. http://www.futureofthebook.org/mckenziewark/

Verbeek, P-P. (2008) ‘Cyborg Intentionality: Rethinking the phenomenology of human-technology relations’, Phenomenology and Cognitive Science (2008) 7:387–395

TECHNOLOGIES OF REPRESENTATION

humanities are changing this, with imaging and visualizing technologies increasingly coming to the ... information visualization are all very different in their nature, and in the analytical and interpretive .... The best example of this is a thermometer, which represents temperature .... Course Notes from the College de. France.

117KB Sizes 0 Downloads 275 Views

Recommend Documents

REPRESENTATION OF GRAPHS USING INTUITIONISTIC ...
Nov 17, 2016 - gN ◦ fN : V1 → V3 such that (gN ◦ fN )(u) = ge(fe(u)) for all u ∈ V1. As fN : V1 → V2 is an isomorphism from G1 onto G2, such that fe(v) = v′.

Journal of Functional Programming A representation ... - CiteSeerX
Sep 8, 2015 - programmers and computer scientists, providing and connecting ...... (n, bs, f)). Furthermore, given a traversal of T, a coalgebra for UR. ∗.

On the Representation of Context
the information on which context-dependent speech acts depend, and the situation that speech acts ..... The other was in fact the Secretary of Health and Human.

REPRESENTATION THEORY OF LIE ALGEBRAS ...
The ad Representation : For a Lie algebra L the map ad: L → gl(L) defined by ad(x)(y)=[x, y] is a ..... and the image of this filtration gives a filtration U0(L) ⊆ U1(L) ⊆ ททท on the universal ..... http://www.jmilne.org/math/CourseNotes/

Journal of Functional Programming A representation ... - CiteSeerX
DOI: 10.1017/S0956796815000088, Published online: 08 September 2015 ... programmers and computer scientists, providing and connecting different views on ... over a class of functors, such as monads or applicative functors. ..... In order to make the

Eigenfunctional representation of dyadic Green's ...
May 8, 2007 - problems for electromagnetic scattering, radiation and propagation ...... [19] Tai C T 1994 Dyadic Green's Functions in Electromagnetic Theory ...

RIESZ REPRESENTATION OF SUBHARMONIC ...
Prerequisites. First we establish ... Let K ⊂ Rn be a compact set. The set of peak ... hyperplane Ht = {x ∈ Rn : (x, y0 − x0) = tr0} contains some point x1 ∈ K ∩ B.

Costs of Personal Injury Representation Bakersfield, CA.pdf ...
Whoops! There was a problem loading this page. Retrying... Page 3 of 15. h. t. t. p. s:// b. a. k. e. r. s. field. t. riala. t. t. o. r. n. e. y. s.c. o. m. g. o. o.gl/. 4. j. p. M. 3. e.

Representation and aggregation of preferences ... - ScienceDirect.com
Available online 1 November 2007. Abstract. We axiomatize in the Anscombe–Aumann setting a wide class of preferences called rank-dependent additive ...

On the Representation of Context
machinery that draws both on the ideas of the anti-formalist Grice–Austin tradition .... the time of utterance, this will not be true for answering machine messages.

Vertical representation of quantifier domains
One well-known problem at the interface of semantics and pragmatics is the issue of quantifier domain restriction: .... consideration, seemingly anyone (not just the individuals already relevant to this context). In .... pronouns and quantifiers othe

Eigenfunctional representation of dyadic Green's ...
May 8, 2007 - Microwave Theory Tech. 50 2702. [7] Lai A, Itoh T and ... [15] Zhang F L, Zhao Q, Liu Y H, Luo C R and Zhao X P 2004 Chin. Phys. Lett. 21 1330.

Fepstrum Representation of Speech - IEEE Xplore
ABSTRACT. Pole-zero spectral models in the frequency domain have been well studied and understood in the past several decades. Exploiting the duality ...

Development of action representation during ...
figure 8s with a pencil, within the parallel pair of lines, as quickly and as accurately as they could. There were three size conditions of the figure 8: small, medium.

SPARSE REPRESENTATION OF MEDICAL IMAGES ...
coefficients, enables a potentially significant reduction in storage re- quirements. ... applications have increased the amount of image data at an explo- sive rate. ..... included in the SparseLab package that is available online at http://.

Cognitive Representation of Conversations About ...
conversation, it seemed desirable to obtain data that would help ... In each study, subjects listened to a tape-recorded discussion between two people about a third. Before ...... Person memory: Some tests of associative storage and retrieval ...

REPRESENTATION THEORY OF LIE ALGEBRAS ...
injective the action of τ on φ(h) is same as the action s on h in previous paragraph, i.e., τφ(h)τ−1 = −φ(h). Let Vm−2i be a weight space. Then τ(Vm−2i) ...

A Neural Representation of Sketch Drawings - arXiv
Apr 11, 2017 - outputs a latent vector of size Nz. Specifically, we feed the sketch sequence, S, and also the same ..... hybrids on pixel images, and has recently been tested on text generation [12]. Combining .... 2015/11/13/gan.html, 2015.

Net-Technologies-May201412 .NET TECHNOLOGIES(USING ASP ...
Net-Technologies-May201412 .NET TECHNOLOGIES(USING ASP.NET).pdf. Net-Technologies-May201412 .NET TECHNOLOGIES(USING ASP.NET).pdf.

Representation: Revisited - GEOCITIES.ws
SMEC, Curtin University of Technology. The role of representation in ... Education suffered a decline in the last 20 or 30 years. (vonGlaserfled, 1995), which led ...

Representation: Revisited
in which social interchange has a major role in constructing and representing knowledge ... Explicitly speaking, the construction and representation of meaning.

Hardware and Representation - GitHub
E.g. CPU can access rows in one module, hard disk / another CPU access row in ... (b) Data Bus: bidirectional, sends a word from CPU to main memory or.

Net-Technologies-Dec201311 NET TECHNOLOGIES (USING ASP ...
Net-Technologies-Dec201311 NET TECHNOLOGIES (USING ASP.NET).pdf. Net-Technologies-Dec201311 NET TECHNOLOGIES (USING ASP.NET).pdf. Open.

TKS TECHNOLOGIES
Nov 13, 2017 - งบ 3Q60 ทรงตัว YoY จำกลูกกตัญญูอีกครั้ง. ก ำไรสุทธิ 3Q60 อยู่ที่89 ล้ำนบำท (+20.9% QoQ, +0.3% YoY) ก ำไรที่ฟื้น