Análisis del discurso multimodal
Keywords:
discourse analysis. multimodality. television. systemic-functional perspectives.Abstract
Multimodal discourse analysis (MDA) is an emerging paradigm in discourse studies which extends the study of language per se to the study of language in combination with other resources. In this paper, the major concerns of MDA, the reasons for the emergence of this field in linguistics, and the variety of approaches which have been developed are discussed, before concepts specific to MDA are examined in detail. Then, a sample multimodal analysis is presented: an extract from a television multiparty debate. The major challenge of MDA is managing the detail and complexity involved in annotating, analyzing, searching and retrieving multimodal semantics patterns within and across complex multimodal phenomena. Thus, one method of managing this complexity involves the development of interactive digital media platform specifically designed for MDA. In any case, the path forward must necessarily involve interdisciplinary collaboration if the larger goals of understanding patterns and trends in technologies, text, context and culture are to be achieved.
Downloads
References
BALDRY, A. P., & THIBAULT, P. J. (2006). Multimodal Transcription and Text Analysis. London: Equinox.
BATEMAN, J. (2007). ‘Towards a Grande Paradigmatique of Film: Christian Metz Reloaded’, Semiotica,167, 13-64.
BATEMAN, J. (2008). Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. Hampshire: Palgrave Macmillan.
BEDNAREK, M. & MARTIN, J. R. (EDS). (2010). New Discourse on Language: Functional Perspectives on Multimodality, Identity, and Affiliation. London and New York: Continuum.
DESOLNEUX, A., MOISAN, L. & MOREL, J. M. (2008). From Gestalt Theory to Image Analysis: A Probabilistic Approach. New York: Springer.
DJONOV, E. (2007). ‘Website Hierarchy and the Interaction between Content Organization, Webpage and Navigation Design: A Systemic Functional Hypermedia Discourse Analysis Perspective’. Information Design Journal,15:2, 144-162.
FORCEVILLE, C. J. & URIOS-APARISI, E. (eds). (2009). Multimodal Metaphor.Berlin and New York: Mouton de Gruyter.
HALLIDAY, M. A. K. (1978). Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.
HALLIDAY, M. A. K. (1985). An Introduction to Functional Grammar. London: Edward Arnold. [2nd ed 1994].
HALLIDAY, M. A. K. (1991). ‘Towards Probabilistic Interpretations’, in Trends in Linguistics Studies and Monographs 55: Functional and Systemic Linguistics Approaches and Uses, pp. 39-61. Berlin: Mouton de Gruyter.
HALLIDAY, M. A. K. (2004). An Introduction to Functional Grammar.(3rd ed, revised by C. M. I. M Matthiessen ). London: Arnold.
HALLIDAY, M. A. K. & greaves, W. S. (2008). Intonation in the Grammar of English. London: Equinox.
HALLIDAY, M. A. K. & HASAN, R. (1985). Language, Context, and Text: Aspects of Language in a Social-Semiotic Perspective. Geelong, Victoria: Deakin University Press [Republished by Oxford University Press 1989].
IEDEMA, R. (2001A). ‘Analysing Film and Television: A Social Semiotic Account of Hospital: An Unhealthy Business’, en T. van Leeuwen y C. Jewitt, (eds) Handbook of Visual Analysis, pp. 183-204. London: Sage.
IEDEMA, R. (2001b). ‘Resemioticization’. Semiotica, 137 - 1/4: 23-39.ieDema, R. (2003). ‘Multimodality, Resemioticization: Extending the Analysis of Discourse as a Multisemiotic Practice’. Visual Communication, 2(1): 29-57.
JEWITT, C. (2006). Technology, Literacy and Learning: A Multimodal Approach. London: Routledge.
JEWITT, C. (2009a). ‘An Introduction to Multimodality’, en C. Jewitt (ed.) The Routledge Handbook of Multimodal Analysis, pp. 14-27. London and New York: Routledge.
JEWITT, C. (2009b). ‘Different Approaches to Multimodality’, in Jewitt, C. (ed), Handbook of Multimodal Analysis, pp. 28-39. London and New York: Routledge.
JEWITT, C. (ed) (2009c). Handbook of Multimodal Analysis. London: Routledge. Kress, G. (2003). Literacy in the New Media Age. London: Routledge.
KRESS, G. (2009). ‘What is Mode?’, en C. Jewitt (ed.), Handbook of Multimodal Analysis, pp. 54-67. London and New York: Routledge.
KRESS, G. & VAN LEEUWEN, T. (1996). Reading Images: The Grammar of Visual Design. (1 ed). London: Routledge (2nd edition 2006).
KRESS, G. & VANLEEUWEN, T. (2006). Reading Images: The Grammar of Visual Design.(2nd ed). London: Routledge.
LAKOFF, G. & JOHNSON, M. (1980). Metaphors We Live By. Chicago: The University of Chicago Press.
LEMKE, J. L. (1995). ‘Intertextuality and Text Semantics’, in Fries, P. H. & Gregory, M. (eds) Discourse in Society: Systemic Functional Perspectives: Meaning and Choice, pp. 85-114.. Norwood, N.J.: Ablex.
LEMKE, J. L. (1998). ‘Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text’, en J. R. Martin y R. Veel (eds.) Reading Science: Critical and Functional Perspectives on Discourses of Science, pp. 87-113. London: Routledge.
LEMKE, J. L. (1999). ‘Typological and Topological Meaning in Diagnostic Discourse’. Discourse Processes, 27(2): 173-185.
LEMKE, J. L. (2002). ‘Travels in Hypermodality’. Visual Communication, 1(3): 299-325.
LEMKE, J. L. (2009). ‘Multimodal Genres and Transmedia Traversals: Social Se-miotics and the Political Economy of the sign’. Semiotica, 177(1-4): 1-27.
LEVINE, P. & SCOLLON, R. (eds) (2004). Discourse and Technology: Multimodal Discourse Analysis. Washington, DC: Georgetown University Press.
LIU, Y. & O’HALLORAN, K. L. (2009). ‘Intersemiotic Texture: Analyzing Cohesive Devices between Language and Images’, Social Semiotics,19(4): 367-387.
MACHIN, D. (2007). Introduction to Multimodal Analysis. London/ New York: Hodder Arnold.
MACHIN, D. (2009). ‘Multimodality and Theories of the Visual’, en C. Jewitt (ed.) Routledge Handbook of Multimodal Analysis, pp. 181-190. London and New York: Routledge.
MARTIN, J. R. & WHITE, P. R. R. (2005). The Language of Evaluation: Appraisal in English. London / New York: Palgrave Macmillan.
MARTINEC, R. (2000). ‘Types of Processes in Action’. Semiotica, 130-3/4: 243-268.
MARTINEC, R. (2004). ‘Gestures that Co-Concur with Speech as a Systematic Resource: The Realization of Experiential Meanings in Indexes’. Social Semiotics, 14 (2): 193-213.
MARTINEC, R. (2005). ‘A System for image-Text Relations in New (and Old) Media’. Visual Communication, 4(3): 337-371.
NORRIS, S. (2004). Analyzing Multimodal Interaction: A Methodological Fra-mework. London: Routledge.
NORRIS, S. & JONES, R. H. (eds.) (2005). Discourse in Action: Introducing Me-diated Discourse Analysis. London: Routledge.
O’HALLORAN, K. L. (2005). Mathematical Discourse: Language, Symbolism and Visual Images. London / New York: Continuum.
O’HALLORAN, K. L. (2008). ‘Inter-Semiotic Expansion of Experiential Meaning: Hierarchical Scales and Metaphor in Mathematics Discourse’, en C. Jones y E. Ventola (eds.) New Developments in the Study of Ideational Meaning: From Language to Multimodality. London: Equinox.
O’HALLORAN, K. L. & SMITH, B. A. (en prensa). ‘Multimodal Studies’, en K. L. O’ Halloran y B. A. Smith (eds.) Multimodal Studies: Exploring Issues and Domains. London and New York: Routledge.
O’HALLORAN, K. L., TAN, S., SMITH, B. A. & PODLASOV, A. (2010). ‘Challenges in Designing Digital Interfaces for the Study of Multimodal Phenomena’. Information Design Journal, 18(1): 2-12.
O’TOOLE, M. (1994). The Language of Displayed Art. London: Leicester Uni-versity Press.
O’TOOLE, M. (2010). The Language of Displayed Art (2nd ed). London and New York: Routledge.
ROHLFING, K., LOEHR, D., DUNCAN, S., BROWN, A., FRANKLIN, A., KIMBARRA, I., MILDE, J., PARRILL, F., ROSE, T., SCHMIDT, T., SLOETJES, H., THIES, A. & WELLINGHOFF, S. (2006). ‘Comparison of Multimodal Annotation Tools - Workshop Report’. Online-Zeitschrift zur Verbalen Interaktion, Ausgabe,7: 99-123.
SCOLLON, R. (2001). Mediated Discourse: The Nexus of Practice. London and New York: Routledge.
SCOLLON, R. & WONGSCOLLON, S. (2004). Nexus Analysis: Discourse and the Emerging Internet. London: Routledge.
TAN, S. (2005). A Systemic Functional Approach to the Analysis of Corporate Tele-vision Advertisements. Unpublished MA, National University of Singapore.
TAN, S. (2009). ‘A Systemic Functional Framework for the Analysis of Corporate Television Advertisements’, en E. Ventola y A. J. Moya Guijjaro (eds). The World Told and The World Shown: Multisemiotic Issues, pp. 157-182. Hamp-shire: Palgrave Macmillan.
UNSWORTH, L. (2008). Multimodal Semiotics: Functional Analysis in Contexts of Education. London and New York: Continuum.
UNSWORTH, L. & CLEIRIGH, C. (2009). ‘Multimodality and Reading: The Cons-truction of Meaning through Image-Text Interaction’, en C. Jewitt (ed.) The Routledge Handbook of Multimodal Research, pp. 151-163. London / New York: Routledge.
VAN LEEUWEN, T. (1999). Speech, Music, Sound. London: Macmillan. vanleeuwen, t. (2005). Introducing Social Semiotics. London: Routledge.
VANLEEUWEN, T. (2008). Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford: Oxford Univerity Press.
VAN LEEUWEN, T. (2009). ‘Parametric Systems: The Case of Voice Quality’, en C. Jewitt (ed.) The Routledge Handbook of Multimodal Analysis, pp. 68-77. London/ New York: Routledge.
VENTOLA, E. & MOYA, J. (EDS) (2009). The World Told and the World Shown: Mul-tisemiotic Issues. Hampshire: Palgrave Macmillan.
WANG, Y. (2009). ‘On Visual Semantic Algebra (VSA): A Denotational Mathema-tical Structure for Modeling and Manipulating Visual Objects and Patterns’. International Journal of Software Science and Computational Intelligence, 1(4): 1-16.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The authors retain the copyright and guarantee RALED the right to be the first publication of the work as well as a Creative Commons Attribution License that allows others to share the work with recognition of authorship and the initial publication in this journal.
