Multimodality, as Kress and Van Leeuwen explain, is the result of modern communication professionally and academically moving away from monomodality. Though there are still preferences that lean towards monomodality as higher culture, multimodality has crept from commonality into all aspects of communication and interaction.
This (very academic) piece specifically covers how “semiotic principles operate in and across different modes.” We’re all familiar with multimodality – otherwise we wouldn’t be here. But expanding on that, Kress and Van Leeuwen explore how multimodal texts (and other forms of communication) make and define meaning “in multiple articulations” – that meaning and feeling are continually expressed throughout a text or communication through different modes, at different times, and on different levels. These specific ways are broken up into the four strata.
The Four Strata
Discourse is socially constructed knowledge of reality, or specifically, how social consciousness demands the way one interprets a specific communication. Using their example, a discourse regarding a war will serve the interest of the method in which it is communicated, by leaving out social assumptions based on user perception.
Design is dependent on context, specifically, the method and mode(s) of communication. Expanding slightly on discourses, designs use the communication situation to change socially constructed knowledge (like a discourse) into social interaction or communication. This is not to say that it is dependent on the method and cannot be semiotic or multimodal. Drawing on (and differentiating from) some of McLuhan’s assertions, Kress and Van Leeuwen point out that a “mode is separate from the medium…in which it will be produced.”
Production is the specific origin of an articulate expression – the expression in practice. In talking about the overlap and separation of design and production, we even get an example of how Shaun both overlaps and separates production and design: as a program director and professor, he designs the course and specific lessons. But as to the dissemination of this design, Shaun can be though of as a producer for specifically executing this expression.
Distribution, kind of like production, is the physical execution of an expression for use and re-use. My favorite example is that of a sound engineer. Being a musician and having worked in recording technology, it’s easy to see how a sound engineer can equally contribute to an expression, and that “perfection” in sound expression is not necessarily the goal any more. Rather, it will a contributed improvement to an expression as communication evolves.
On the interpretation side, these strata also apply. Innovation is often a result of interpretation, as we’ve talked about in class. Think about this passage: “a given interaction may be experienced differently, and a given discourse interpreted differently, from the way it was intended.” Sound like using a technology or a communication not for it’s intended purpose?
Kress and Van Leeuwen go on to explain two semiotic principles that innovation in terms of communication and the four strata bring about: Provenance and experiential meaning potential. Provenance is similar to discourse in that it’s an interpretation of social contexts. However, more specifically, provenance as defined in this article is importing social knowledge from another context into a different context in which we are working. Experiential meaning potential is interpreting what is done to produce an expression. Using their example of “breathiness”, we interpret not the actual expression of making sound that has the sound of breathing involved in it, but rather the situations in which exerted breathing is involved.