Perspectivas en el análisis de la multimodalidad
desde los inicios al estado del arte
Palavras-chave:
multimodalidade. semiótica. tipografia. síntese e automatismo da fala.medicina. hiper-texto. aprendizagem. tradução. análise do corpus.Resumo
Este artigo oferece uma ampla visão histórica desde os inícios da análise multimodal até os candentes temas atuais e as perspectivas futuras no estudo da multimodalidade. Ao contrário de outras publicações, considera-se que o Tratado sobre Laoconte de Lessing (1766) foi o começo histórico das investigações sobre multimodalidade, de modo que o que atualmente acontece é o ressurgimento de um debate que já tinha acontecido, parcialmente, entre os classicistas alemães. Um tema que é discutido ferventemente na atualidade é como os modos se relacionam com outros modos na forma de sub-modos e dentro dos diferentes meios. O potencial de significação de um modo muda segundo a preponderância do modo dentro do construto semiótico assim como também através da perda ou adquisição do significado meta-funcional no desenvolvimento tecnológico dos meios em massa. Um exemplo é a tipografia. Apresentam-se novos temas e tendências no campo atual da investigação da multimodalidade e os últimos descobrimentos em uma gama de estudos nas áreas de investigação aplicada, por exemplo, apresentam-se e discutem-se trabalhos sobre tradução em hospitais, desenho de hiper-textos, desenho de museus, subtitulagem de filmes e tradução de historietas cómicas. Para terminar, são esboçadas as futuras perspectivas sobre como levar a investigação da multimodalidade a uma base mais empírica, por exemplo, através da análise do corpus multimodal.
Downloads
Referências
ALIAS, S. (2004). ‘‘A semiotic study of Singapore’s Orchard and Marriott Hotel’’, en K. O’Halloran (Ed.), Multimodal discourse analysis: Systemic functional perspectives, pp. 55-79. Londres: Continuum.
BALDRY, A. (Ed.) (2000a). Multimodality and multimediality in the distant learning age. Compopasso: Palladino Editore.
BALDRY, A. (2000b). ‘‘English in a visual society: Comparative mediality’’. en A. Baldry (Ed.), Multimodality and multimediality in the distant learning age, pp. 41-89. Campopasso: Palladino Editore.
BALDRY, A. (2004). ‘Phase and transition, type and instance: Patterns in media texts as seen through a multimodal concordancer’, en K. O’Halloran (Ed.), Multimodal discourse analysis: Systemic functional perspectives, pp. 83-108. Londres: Continuum.
BALDRY, A. & Thibault, P. (2004). Multimodal transcription and text analysis. Londres: Equinox Publishing.
BATEMAN, J., DELIN, J. & HENSCHEL, R. (en prensa). ‘‘Multimodality and empiricism: Preparing for a corpus-based approach to the study of multimodal meaning-making’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 65-88. Amsterdam & Philadelphia: John Benjamins.
BESKOW, J. (2003). Talking heads - Models and applications for multimodal speech systhesis. Tesis doctoral no publicada. Department of Speech, Music and Hearing, KTH, Stockholm. http://www.speech.kth.se/~berkow/thesis
BÜHRIG, K. (en prensa). ‘On the «multimodality» of interpreting in medical briefings for informed consent: using diagrams to impart knowledge’ en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 227-214. Amesterdam & Philadelphia: John Benjamins.
CLARK, R.C. & MAYER, R. (2003). E-learning and the science of instruction. Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: Jossey-Bass/Pfeiffer.
COLBY, J. (2002). ‘‘Multimodality: The new wave of mobile interaction’’, en Speech Technology Magazine, November/December 2002. http://www. speechtechmag.com
DAVIS, G. & MASON, P. (2004). ‘‘Trying on voices: Using questions to establish authority, identity, and recipient design in electronic discourse’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse analysis, pp. 47-58. Georgetown, Washington, D.C.: Georgetown University Press.
DE BEAUGRANDE, R.A. & DRESLER W.U. (1981). Einführung in die Textlinguistik. Tübingen: Niemeyer.
DE JONG, M. & VAN DER GEEST, T. (2000). ‘‘Characterizing web heuristics’’, en Technical Communications 3, 311-324.
DE SAINT-GEORGES, I. (2004). ‘‘Materiality in discourse: The influence of space and layout in meaning making’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse analysis, pp. 71-87. Georgetown, Washington, D.C.: Georgetown University Press.
ECO,U. (1980). Il nome della rosa. Milano: Bompiani.
ECO, U. (1983). The name of the rose. New York: Harcourt. Echkrammer, E.M. (en prensa). ‘‘Drawing on theories of inter-semiotic layering to analyse multimodality in medical self-counselling texts and hypertexts’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on Multimodality, pp. 211-226. Amsterdam & Philadelphia: John Benjamins.
EVERTS, E. (2004). ‘Modalities of turn-taking in blind/sighted interaction: better to be seen and not heard?’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse analysis, pp.128-145. Georgetown, Washington, D.C.: Georgetown University Press.
FARKAS, D. & J. FARKAS (2000). ‘‘Guidelines for designing web navigation’’, en Technical Communication, 3, 341-358.
FILLIETTAZ, K. (2004). ‘‘The multimodal negotiation of service encounters’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse analysis, pp. 88-100. Georgetown, Washington, D.C.: Goergetown University Press.
GOETHE, J. W. (1798/1998). ‘‘Über Laokoon’’, en F. Apel (Ed.), Goethe - Ästhetische Schriften 1771 - 1805, pp. 489-500. Frankfurt a. Main: Deutscher Klassiker Verlag.
HALLIDAY, M.A.K. (1961). ‘‘Categories of the theory of grammar’’, en Word, 17, 3, 242-292.
HALLIDAY, M.A.K. (1975). Language as social semiotic: towards a general sociolinguistic theory. Columbia: Hornbeam Press.
HALLIDAY, M.A.K. (1978). Language as social semiotic: The social interpretation of language and meaning. London: Arnold.
HALLIDAY, M.A.K. & HASAN, R. (1976). Cohesion in English. London: Longman.
HODGE, R. & KRESS, G. (1988). Social semiotics. Cambridge: Polity Press.
HOFINGER, A., & VENTOLA, E. (en prensa). ‘‘Multimodality in operation: Language and Picture in a Museum’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 193-2210. Amsterdam & Philadelphia: John Benjamins.
IEDEMA, R. (2003). ‘‘Multimodality, resemiotization: Extending the analysis of discourse as multi-semiotic practice’’, en Visual Communication, 2, (1): 29-57.
JAKOVI-MIRWALD, C. (1997). Buchmalerei; ihre Terminologie in der Kunstgeschichte. Berlin: Reimer.
JEWITT, C. (2004). ‘‘Multimodality and new communication technologies’’, en P. Levine & R. Scollon (Eds.), Discrouse & technology. Multimodal discourse analysis, pp. 184-195. Georgetown, Washington, D.C.: Georgetown University Press.
JOHNSTON, A. (2004). ‘‘Files, forms, and fonts: Mediational means and identity negotiation in immigration interviews’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse nalysis, pp. 116-127. Georgetown, Washington, D.C.: Georgetown University Press.
JONES, R. (2004). ‘‘The problem of context in computer-mediated communication’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. Multimodal discourse analysis, pp.20-23. Georgetown, Washington, D.C.: Georgetown University Press.
KAINDL, K. (en prensa). ‘‘Multimodality in the translation of humour in comics’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 173-192. Amsterdam & Philadelphia: John Benjamins.
KALTENBACHER, M. 2003. ‘‘Language learning via CD-Rom - Old wine in new bottles’’ en D. Newby (Ed.), Mediating between theory and practice in the context of different learning cultures and languages, pp. 171-175. Strasbourg: Council of Europe Publishing.
KALTENBACHER, M. (en prensa). ‘‘Multimodality in language teaching CD-ROMs’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 119-136. Amsterdam & Philadelphia: John Benjamins.
KENDON, A. (1990). Conducting interaction: Patterns of behavior in focussed encounters. Cambridge: Cambridge University Press.
KOK, A. (2004). ‘‘Multisemiotic mediation in hypertext’’, en K. O’Halloran (Ed), Multimodal discourse analysis: Systemic functional perspectives, pp. 131-159. London: Continuum.
KRESS, G., JEWITT, C., OGBORN, J. & TSATSARELIS, C. (2001). Multimodal teaching and learning. The rhetorics of the science classroom. London and New York: Continuum.
KRESS, G. & VAN LEEUWEN, T. (1990). Reading images. Geelong: Deaking University Press.
KRESS, G. & VAN LEEUWEN, T. (1996). Reading images: The grammar of visual design. London: Routledge.
KRESS, G. & VAN LEEUWEN, T. (2001). Multimodal discourse. The modes and media of contemporary communication. London: Arnold.
KRESS, G. & VAN LEEUWEN, T. (2002). ‘‘Colour as a semiotic mode: Notes for a grammar of colour’’, en Visual Communication, 1, 3, pp. 343-368.
LEE, J. (ED.) (1997). Intelligence and multimodality in multimedia interfaces: Research and applications. Menlo Park, CA: AAAI Press.
LEMKE, J. (2002). ‘‘Travels in hypermodality’’, en Visual Communication, 1 (3), pp. 299-325.
LESSING. G.E. (1766/1990). ‘‘Laokoon: Oder über die Grenzan der Melerei und Poesie’’, en W. Barner (Ed,.), Gotthold Ephraim Lessing - Weke und Briefe, 5/2, pp. 11-321. Frankfurt a. Main: Deutscher Klassiker Verlag.
LEVINE, P. & SCOLLON, R. (Eds.) (2004). Discourse & technology. Multimodal discourse analysis. Georgetown, Washington, D.C.: Georgetown University Press.
LIM, V.F. (en prensa). ‘‘Problematising ‘semiotic resource’ ’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 51- 63. Amsterdam & Philadelphia: John Benjamins.
LUTZ, B. (1998). ‘‘Hypertext: das Medium und der Autor. Ein Erfhrungsbericht’’, en B. Ketteman, M. Stegu, & H. Stöckl (Eds.), Mediendiskurse. VerbalWorkshop Graz 1996, pp. 151-162. Frankfurt am Main: Peter Lang Verlag.
MAZAL, O. (1985). Das BUch der Initialen. Wein: Edition Tusch.
MCNEILL, D. (1992). Hand and mind. What gestures reveal about thought. Chicago: University of Chicago Press.
MCNEILL, D. (Ed.) (2001). Language and gesture. Cambridge: Cambridge University Press.
MUNTIGL, P. (en prensa). ‘‘Modelling multiple semiotic systems’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 31-59. Amsterdam & Philadelphia: John Benjamins.
MURRAY, G. & COSTANZO, T. (1999) ‘‘Usability and the web: An overview’’, en Network Notes 61. Information Technology Services. National Library of Canada. http://www.collectionscanada.ca/9/1/pl-260-e.html
NIJHOLT, A. (2003). ‘‘Multimodality and ambient intelligence’’, en W.F.J. Verhaegh, E.H.L. Aarts & J. Korst (Eds.), Algorithms and ambient
intelligence. Dordrecht et al.: Kluwer Academic Publishers. (Disponible en: http//doc.utwente.nl/fid/1144).
O’HALLORAN, K. (2004a). ‘‘Visual semiosis in film’’, en K. O’Halloran (Eds.), Multimodal discourse analysis: Systemic functional perspectives, pp. 109-130. London: Continuum.
O’HALLORAN, K. (Ed.) (2004b). Multimodal discourse analysis: Systemic functional perspectives. London: Coninuum.
O’TOOLE, M. (1994). ‘‘Opera Ludentes. The Sydney Opera House at work and play’’, en K. O’Halloran (Ed.l), Multimodal discourse analysis: Systemic functional perspectives, pp. 11-27. London: Continuum.
PANG, A. (2004). ‘‘Making history in ‘From Colony to Nation’’: A multimodal analysis of a museum exhibition in Singapore’, en K. O’Halloran (Ed.), Multimodal discourse analysis: Systemic functional perspectives, pp. 28-54. London: Continuum.
RAISAMO, R. (1999). Multimodal human-computer interaction: a constructive and empirical study. Tesis doctoral no publicada. Department of Computer Science, University of Tampere. http://acta.uta.fi/pdf/951-44-4701-6.pdf
RHEINDORF, M. (en prensa). ‘‘The multimple modes of dirty dancing: a cultural studies approach to multimodal discourse analysis’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.). Amsterdam & Philadelphia: John Benjamins.
SAJOR, M. (2002). Multimodality: The coming revolution. Wireless Future Magazine, September/Octber 2002. http://www.wirelessfuturemagazine.com
SCHILLER, F. (1793-1962). ‘‘Über das Pathetische’’, en B. von Wiese (Ed.), Schillers Werke, 20 - Philosophische Schriften, 1. Teil, pp. 196-221. Weimar: Böhlau.
SCOLLON, R. & LEVINE, P. (2004). ‘‘Multimodal discourse analysis as the confluence of discourse and technology’’, en P. Levine & R. Scollon (Eds.), Discourse & technology. multimodal discourse analysis, pp. 1-6. Georgetown, Washington, D.C.: Georgetown University Press.
STÖCKL, H. (1998). ‘‘Bild-Text-Verknüpfung’’, en B. Kettemann, M. Stegu & H. Stöckl (Eds.), Mediendiskurse. Vergal-Workshop Graz 1996, pp. 73-92. Frankfurt am Main; Peter Lang Verlag STÖCKL, H. (en prensa). ‘‘In between modes: Language and image in printed media’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 9-30. Amsterdam & Philadelphia: John Benjamins.
TAYLOR, C. (2000). ‘‘Text analysis and translation: An ineractive, self-access computer application incorporating a functional approach’’, en A. Baldry
(Ed.), Multimodality and multimediality in the distant learning age, pp. 295- 309. Compopasso: Palladino Editore.
TAYLOR, C. (en prensa). ‘‘Multimodal text analysis and subtitling’’, en E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality, pp. 153-172. Amsterdam & Philadelphia: John Benjamins.
THIBAULT, P. (2000). ‘‘The multimodal transcription of a television advertisement: Theory and practice’’, en A. Baldry (Ed.), Multimodality and multimediality in the distant learning age, pp. 311-385. Campopasso: Palladino Editore.
USABILITY.GOV (2004). Methods for designing usable web-sites. Collecting, writing and revising content. US Department of Health and Human Services (Ed.). http://www.usability.gov/methods
VAN DER GEEST, T. (2001). Web site design in communication design. Amsterdam: Benjamins.
VAN DER GEEST, T. & SPYRIDAKIS, J. (2000). ‘‘Developing heuristics for web communication’’, en Technical Communication 3, pp. 301-310.
VAN LEEUWEN, T. (1999). Speech, music, sound. London: Macmillan.
VAN LEEUWEN, T. (en prensa). Introducing social semiotics. An introductory textbook. London: Routledge.
VENTOLA, E. & KALTENBACHER, M. (2003). ‘‘Lexicogrammar and language teaching materials - A social semiotic and discourse perspective’’, en J.E. Joyce (Ed.), Grammar in the language classroom, pp. 158-201. Singapore: SEAMEO Regional Language Centre.
VENTOLA, E., CHARLES, C. & KALTENBACHER, M. (Eds.). (en prensa). Perspectives on multimodality. Amsterdam: Benjamins.
VERGILIUS, P. M. (1988). ‘‘Aenis’’, en J. Götte (Ed.), München & Zürich: Artemis Verlag.
WILLIAMS, T. (2000). ‘‘Guidelines for designing and evaluation the display of information on the web’’ en Technical Communication, 3, pp. 383-396.
WINCKELMANN, J.J. (1755-1969). Gedanken über die Nachahmung der griechischen Werke in der Malerei und Bildhauerkunst. Stuttgart: Reclam.
Downloads
Publicado
Como Citar
Edição
Seção
Licença
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Os/As autores/as conservam os direitos autorais e garantem a RALED o direito de ser a primeira publicação do trabalho licenciado por uma Creative Commons Attribution License que permite compartilhar o trabalho com reconhecimento de sua autoria e a publicação inicial nesta revista.
