Intersemiosis in Classroom Discourse: A Dual Perspective on Systemic Functional Linguistics and Multimodal Interaction Analysis
Abstract
This study explores how the synergistic relations of multimodal symbolic resources construct classroom discourse. To this end, we combine two perspectives—multimodal interaction analysis and systemic functional linguistics to divide modal integration into two dimensions, meaning and representation, based on which we construct a new intersemiotic analytical framework to explore how different symbolic modes combine to provide meaning in the Chinese as a foreign language classroom. The findings show that teachers in such courses focus more on the visualization process of thinking and that coordinating rich symbolic resources exercises students' Chinese thinking skills and learning initiatives. At the same time, we demonstrate that combining these two research perspectives can strongly contribute to the development of multimodal interaction analysis in teaching and learning, providing practical implications for improving the quality of teaching. Furthermore, the analytical framework proposed in this paper can be applied to more types of teaching practices in the future, offering a valuable reference for multimodal applications of classroom discourse.
Keywords: classroom discourse, modal integration, multimodal interaction analysis, systemic functional linguistics.
Full Text:
PDFReferences
BATEMAN, J.A. (2011). The decomposability of semiotic modes. In: O'HALLORAN, L., & SMITH, B. (eds.) Multimodal Studies: Exploring Issues and Domains. London: Routledge, pp. 17-38. https://doi.org/10.4324/9780203828847-10
BATEMAN, J., WILDFEUER, J., & HIPPALA, T. (2017). Multimodality: Foundations, Research and Analysis - A Problem-Oriented Introduction. Berlin: De Gruyter Mouton. http://dx.doi.org/10.1515/9783110479898
GREIFFENHAGEN, C. (2008). Video Analysis of Mathematical Practice? Different Attempts to "Open Up" Mathematics for Sociological Investigation. Forum: Qualitative Social Research, 9(3), 32. https://doi.org/10.17169/fqs-9.3.1172
HALLIDAY, M.A.K. (1978). Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.
HALLIDAY, M.A.K. (1994). An Introduction to Functional Grammar. London: Edward Arnold.
HALLIDAY, M.A.K., & MATTHIESSEN, C.M.I.M. (1999). Construing Experience through Meaning: A Language-Based Approach to Cognition. London: Cassell.
JEWITT, C. (2008). Technology, Literacy, Learning: A Multimodal Approach. London: Routledge.
JEWITT, C. (2013). Multimodality and digital technologies in the classroom. In: SAINT-GEORGES, I., & WEBER, J.J. (eds.) Multilingualism and Multimodality: The Future of Education Research. Rotterdam: Sense Publishers, pp. 141-152.
JEWITT, C., BEZEMER, J., & O'HALLORAN, K.L. (2016). Introducing Multimodality. London: Routledge.
JING, G. (2016). On the construction of a system for teaching spoken Chinese as a foreign language. Chinese Teaching in the World, 30(3), 368–378.
KARATZA, S. (2020). Multimodal literacy and language testing: Visual and intersemiotic literacy indicators of reading comprehension texts. Journal of Visual Literacy, 39(3–4), 220–255. https://doi.org/10.1080/1051144X.2020.1826222
KOMARAWAN, Y. (2019). Constructing Interpersonal Meaning in Indonesian Science Classrooms through Language, Space and Gaze. KEMANUSIAAN: The Asian Journal of Humanities, 26(1), 1–23. https://doi.org/10.21315/kajh2019.26.1.1
KRESS, G. (2003). Literacy in the New Media Age. London & New York: Routledge.
KRESS, G. (2010). Mulimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
KRESS, G., & VAN LEEUWEN, T. (2006). Reading Images: The Grammar of Visual Design. 2nd ed. Routledge.
LEMKE, J.L. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In: MARTIN, J.R., & VEE, R. (eds.) Reading Science: Critical and Functional Perspectives on Discourse of Science. London/New York: Routledge, pp. 87–133.
LIM, F.V. (2021). Investigating intersemiosis: A systemic functional multimodal discourse analysis of the relationship between language and gesture in classroom discourse. Visual Communication, 20(1), 34–58. https://doi.org/10.1177/1470357218820695
LIM, V.F. (2019). Analysing the teachers’ use of gestures in the classroom: A systemic functional multimodal discourse analysis approach. Social Semiotics, 29(1), 83–111. https://doi.org/10.1080/10350330.2017.1412168
MARTIN, J.R. (1992). English Text: System and Structure. Amsterdam: John Benjamins Publishing Company.
MARTINEC, R., & SALWAY, A. (2005). A system for image text relations in new (and old) media. Visual Communication, 4(3), 337–371. https://doi.org/10.1177/1470357205055928
MATTHIESSEN, C.M.I.M. (2009). Multisemiosis and Context-Based Register Typology: Registerial Variation in the Complementarity of Semiotic Systems. In: VENTOLA, E., & GUIJARRO, A.J.M. (eds.) The World Told and the World Shown. London: Palgrave Macmillan, pp. 11-38. https://doi.org/10.1057/9780230245341_2
NORRIS, S. (2004). Analyzing Multimodal Interaction: A Methodological Framework. London: Routledge.
NORRIS, S. (2011). Three hierarchical positions of deictic gesture in relation to spoken language: A multimodal interaction analysis. Visual Communication, 10(2), 129–147. https://doi.org/10.1177/1470357211398439
NYSTRAND, M. (2006). Research on the Role of Classroom Discourse As It Affects Reading Comprehension. Research in the Teaching of English, 40(4), 392-412. Retrieved from https://www.troyspier.com/assets/files/bibliographies/teaching/nystrand_classroom_discourse.pdf
O'HALLORAN, K.L. (2003). Intersemiosis in mathematics and science: Grammatical metaphor and semiotic metaphor. In: SIMON-VANDENBERGEN, A.-M., TAARNIERS, M., & RAVELLI, L. (eds.) Grammatical Metaphor: Views from Systemic Functional Linguistics. Amsterdam/Philadelphia, Pennsylvania: John Benjamins, pp. 337–365. https://doi.org/10.1075/cilt.236.18oha
O'HALLORAN, K.L. (2007). Systemic functional multimodal discourse analysis (SF-MDA) approach to mathematics, grammar, and literacy. In: MCCABE, A., O'DONNELL, M., & WHITTAKER R. (eds.) Advances in Language and Education. London: Continuum, pp. 75–100.
PINNOW, R.J. (2011). “I've Got an Idea”: A Social Semiotic Perspective on Agency in the Second Language Classroom. Linguistics and Education, 22(4), 383–392. https://doi.org/10.1016/j.linged.2011.09.002
ROYCE, T. (1998). Synergy on the page: Exploring intersemiotic complementarity in page-based multimodal text. Japan Association Systemic Functional Linguistics Occasional Papers, 1, 25–49.
SCOLLON, R. (1998). Mediated Discourse as Social Interaction. London: Longman.
SCOLLON, R. (2001). Mediated Discourse: The Nexus of Practice. London: Routledge.
TAKAHASHI, J., & YU, D. (2016). Multimodality in the classroom: An introduction. Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics, 16(2), i-vi. https://doi.org/10.7916/salt.v16i2.1247
TAYLOR, R. (2014). Meaning between, in, and around words, gestures and postures: Multimodal meaning making in children's classroom communication. Language and Education, 28(5), 401–420. https://doi.org/10.1080/09500782.2014.885038
TSUNG, L., & CRUICKSHANK, K. (eds.) (2011). Teaching and Learning Chinese in Global Contexts: CFL Worldwide. London: Continuum.
UNSWORTH, L., & CLEIRIGH, C. (2009). Multimodality and reading: The construction of meaning through image-text interaction. In: JEWITT, C. (ed.) The Routledge Handbook of Multimodal Analysis. London & New York: Routledge, pp. 151-163. Retrieved from https://www.researchgate.net/publication/259559003_Multimodality_and_Reading_The_Construction_of_Meaning_Through_Image-Text_Interaction
WALSH, S. (2006). Investigating Classroom Discourse. London: Routledge.
YANG, J. (2019). Understanding Chinese language teachers' beliefs about themselves and their students in an English context. System, 80, 73–82. https://doi.org/10.1016/j.system.2018.10.014
ZHANG, D., & HU, R. (2019). System, Affordances and Choice in Constructing Multimodal Discourse. Contemporary Rhetoric, 5, 68-79. https://doi.org/10.16027/j.cnki.cn31-2043/h.2019.05.007
ZHANZI, L., & DANYUN, L. (2012). Multimodal Semiotics: Theoretical Basis, Research Approaches and Development Prospects. Foreign Language Studies, 2, 1-8.
Refbacks
- There are currently no refbacks.
