SciELO - Scientific Electronic Library Online

 
vol.27 índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Em processo de indexaçãoSimilares em Google

Compartilhar


Lexikos

versão On-line ISSN 2224-0039
versão impressa ISSN 1684-4904

Lexikos vol.27  Stellenbosch  2017

 

ARTICLES

 

Multimodal Exemplification: The Expansion of Meaning in Electronic Dictionaries

 

Multimodale toeligting: Betekenisuitbreiding in elektroniese

 

 

Xiqin Liu

School of Foreign Languages, South China University of Technology, China and Guangdong University of Foreign Studies, China (flxqliu@scut.edu.cn)

 

 


ABSTRACT

This article investigates electronic dictionaries under the framework of Systemic-Functional Multimodal Discourse Analysis (SF-MDA) and argues for improving their exemplification multimodally. Multimodal devices, if well coordinated, can help optimize e-dictionary examples in informativity, diversity, dynamicity and interactivity. The term multimodal exemplification is tentatively proposed under the umbrella of multimodal lexicography (Lew 2010), and defined as the selection and presentation of examples with multimodal devices for achieving greater effectiveness in exemplifying than language does alone, especially in an e-dictionary. Evidence shows that multimodal exemplification can expand the three metafunctional meanings of the e-dictionary discourse: ideational, interpersonal and textual. Ideational meaning can be enriched by not only multimodal examples per se but also cross-modal example-definition ties, and hyperlinks facilitate meaning flow in the semantic network. Interpersonal meaning can be expanded by user participation and design options, including those for page layout (spatial mode) and example genre style (verbal mode). Textual meaning can be reinforced by information value, composition, salience and framing. This article makes a first attempt to explore the intermodal relationship between a definition and the examples under the same sense, and to present a diagram illustrating a typical design of visual space in e-dictionaries. By exploring the special features of multimodal example texts, it may deepen our understanding of the emerging multimodal lexicography and complement multimodal discourse studies from a lexicographical perspective.

Keywords: e-dictionary, example, metafunctional meaning, multimodal discourse analysis, multimodal exemplification, multimodal lexicography


OPSOMMING

Woordeboeke. In hierdie artikel word elektroniese woordeboeke binne die raamwerk van Sistemies-Funksionele Multimodale Diskoersanalise (SF-MDA) ondersoek en geargumenteer ten gunste van die verbetering van hul multimodale toeligtingshulpmiddels. Indien multimodale hulpmiddels goed gekoördineer word, kan hulle help om die e-woordeboekvoorbeelde te optimaliseer ten opsigte van informatiwiteit, diversiteit, dinamiek en interaktiwiteit. Die term multimodale toeligting word tentatief voorgestel onder die oorkoepelende multimodale leksikografie (Lew 2010), en gedefinieer as die seleksie en aanbieding van voorbeelde met behulp van multimodale hulpmiddels ten einde groter effektiwiteit in toeligting te verkry as wat met slegs taal gedoen kan word, veral in 'n e-woordeboek. Daar is bewyse dat multimodale toeligting die drie metafunksionele betekenisse van die e-woordeboekdiskoers kan uitbrei: begripvormend, interpersoonlik en tekstueel. Begripvormende betekenis kan uitgebrei word nie alleen deur multimodale voorbeelde per se nie, maar ook deur tussenmodale voorbeeld-definisie verbintenisse, en hiperskakels vergemaklik betekenisvloei in die semantiese netwerk. Interpersoonlike betekenis kan uitgebrei word deur gebruikersdeelname en ontwerpopsies, insluitend dié vir bladuitleg (ruimtelike modus) en voorbeeldgenrestyl (verbale modus). Tekstuele betekenis kan versterk word deur inligtingswaarde, samestelling, treffendheid en raming. In hierdie artikel word 'n eerste poging aangewend om die intermodale verhouding tussen 'n definisie en die voorbeelde onder dieselfde betekenis te verken, en om 'n diagram aan te bied wat 'n ontwerp tipies aan die visuele ruimte in e-woordeboeke illustreer. Deur die spesiale kenmerke van multimodale voorbeeldtekste te verken, kan ons begrip van die groeiende multimodale leksikografie dalk verbeter word en kan die multimodale diskoersstudies vanuit 'n leksikografiese perspektief dalk aangevul word.

Sleutelwoorde: e-woordeboek, voorbeeld, metafunksionele betekenis, multimodale diskoersanalise, multimodale toeligting, multimodale leksikografie


 

 

1. Introduction

Electronic dictionaries (e-dictionaries) have become increasingly popular as a tool of language learning and use in the digital era. In 2012, Macmillan announced that henceforth its dictionaries would only be published online. As "we are liberated from the straitjacket of the printed page and alphabetical order" (Atkins 1996: 516), the use of multimodal resources has become an important topic for lexicography. The combination of different semiotic modes (textual, visual, audio, spatial, etc.) is critical to the information processing of an audience because it can expand meaning by creating new space for interpretations (Lemke 2002). So multimodality plays an important role in meaning representation in (e-)dictionaries.

Examples serve as a basic constituent of the dictionary microstructure, reinforcing meaning explanations, illustrating collocations and colligations, and contextualizing for cultural, stylistic and pragmatic implications (Xu 2009: 12, 26-29; see also Nielsen 2014; Prinsloo 2015; Taljard 2015). In e-dictionaries, the different semiotic modes can be employed to develop this potential to a much greater extent than in print dictionaries. In fact, e-dictionary multimodality has changed exemplification profoundly, enriching and expanding meaning in various dimensions. To deepen our understanding of e-dictionary exemplification in the digital transformation, the term multimodal exemplification is proposed under the umbrella of multimodal lexicography (Lew 2010), following my previous work on multimodal definition1 (Liu 2015). It is tentatively defined as the selection and presentation of examples with multimodal devices for achieving greater effectiveness in exemplifying than language does alone, especially in an e-dictionary.

The present study is guided by the framework of Multimodal Discourse Analysis from a Systemic-Functional perspective, SF-MDA (cf. O'Halloran 2008). MDA is an emerging paradigm in discourse studies which extends the study of language per se to the study of language in combination with other resources, such as images, scientific symbolism, gesture, action, music and sound (O'Halloran 2011: 120). After all, dictionary examples constitute a special genre of discourse, and contribute to the formation of a dictionary text, achieving cohesion with it (cf. Szende 1999; Xu 2009: 8, 26-29). The (multimodal) discourse perspective can be traced back to a holistic view of meaning in a dictionary.

Following the theory of "meaning as use" (Wittgenstein 1953), meaning and usage are inseparable, and "in a living language vocabulary and grammar do not have their own independent existences" (Tarp 2008: 135). Cognitive findings tell us that linguistic information of a word or phrase is merged with encyclopedic information. Meaning is of multiple facets blended into a whole, and "multifaceted meaning descriptions can improve the usability of a dictionary" (Kremer and Abel 2010). If properly employed, multimodal resources for e-dictionary examples may facilitate this process of blending. With space restrictions removed, it is likely for e-dictionaries to have an optimum number of examples showing a great variety of usage. When necessary, e-dictionaries can have options for longer examples than their paper counterparts, providing a complete co-text for users2 (cf. Wojciechowska 2015). Meaning can be enriched in such a way as to improve the informativity and diversity of examples.

In addition, the meaning network in a dictionary is dynamic and interactive (cf. Prinsloo et al. 2012). For one thing, meaning tends to be co-constructed by the dictionary writers and users in the digital revolution. Users' participation in exemplification and their interaction with the dictionary have increased, and user-friendliness and example acceptability can be improved. Users can contribute examples, comment on them or make customized use of them. For another, the hypermodal affordances, like hyperlinks in examples, enable dictionary users to expand the meaning network outside the microstructure of a dictionary, thus meaning "flows". E-dictionary examples are characterized by hypermodality and hypertextuality3, allowing for more reading paths.

The paper argues that multimodal exemplification has the potential of expanding meaning on demand. Expansion means not only enrichment (informativity and diversity) but also flow (dynamicity or interactivity) in the semantic network. In previous literature on MDA, meaning expansion may only refer to the former. Meaning refers to Halliday's (1985) three metafunctions: ideational, interpersonal and textual4.

Drawing evidence mainly from online English learner's dictionaries, the article addresses the following research questions:

(1) Is it necessary to optimize meaning representation by e-dictionary examples from a multimodal perspective?

(2) Can multimodal exemplification expand the ideational, interpersonal and textual meanings in the e-dictionary discourse? If yes, how?

 

2. Background

2.1 Exemplification limitations of paper dictionaries

Paper dictionaries are often subject to space restrictions, with their layout confined by the alphabetic order. Unlike the hypertext, the printed text only has a single conventional sequence. As a result, limitations of exemplification may arise in terms of number, variety and the way of presentation.

One contentious problem about the exemplification in paper dictionaries is what words should be exemplified and how many examples should be given in specific cases, with a seeming focus on collocations. Xu (2009: 156-166) argued that some words should be better exemplified in the Big Five dictionaries5, with illustrations of collocational and syntactic complexities of verbs. A survey by Zhang (2015: 89-92) showed there were not enough collocational examples in the mainstream English dictionaries. CoBUILD decided on a policy of exemplifying as many words as possible (Fox 1987: 137), but this still can't guarantee consistency in exemplification, and Stein (2002: 206) pointed out the need of examples for some concrete nouns to show "morphological restriction on specific word-formation structures".

A second limitation involves the deixis (person, time or space). "The dictionary example cannot usually afford to look outside itself for complete elucidation" (Chen and Chung 2008) and in the fragmental context exhibited in an example sentence, deixis poses a problem for lexicographers (Xu 2009: 120-137). Pronouns and "neutralized" atemporal statements are used (Szende 1999). But in such cases, the cultural, stylistic and pragmatic information may be dispensed with even when it is necessary. Accordingly, authenticity, naturalness and meaning completeness of examples may decline.

A third limitation may lie in the way of displaying information and the users' access to examples. In such densely printed pages of text, reading is often linear and strictly coded (cf. Van Leeuwen 2005: 204). The one-time displaying in a fixed order might leave the users in a passive state of reception. There is a lack of customized presentations of information which are likely to encourage users' active participation in meaning making and improve the usability of dictionaries.

Exemplification limitations of paper dictionaries highlight some advantages of e-dictionaries: hybridization, more and better data, efficiency of access, customization, corpus integration, and user input (cf. Granger 2012). Freedom from the traditional paper format has removed constraints on size and format, paving the way for multi-faceted, flexible and rich representations of word meaning and use that have been unfeasible for print dictionaries (Fellbaum 2014). With the potential of displaying information in a more usable and user-friendly way, multimodal devices in e-dictionaries, if properly employed, can optimize the meaning representation by examples.

2.2 Potential problems with current e-dictionaries

Previous literature about e-dictionaries shows that problems still exist in at least two areas, examples and multimodal features.

First, there is room for improvement in examples. Many online dictionaries, like Wordnik, show contemporary web-derived example sentences on the right side of the screen, supported, on the left, by definitions from a range of traditional dictionaries which don't reflect the recently-coined meaning in the examples (Rundell 2015). As noted by Frankenberg-Garcia (2012), there seem to be few dictionary examples that give clear contextual clues about meaning and help with grammar at the same time. "Language users seem to appreciate examples from dictionaries, but there is not much proof that the examples in dictionaries as they are today suffice" (Frankenberg-Garcia 2015).

Second, surveys showed that users were not satisfied with the multimodal features of e-dictionaries. According to Müller-Spitzer et al. (2012), features typical of digital dictionaries, specifically adaptability and multimedia, received the lowest ratings, and "we wanted to know how users assess innovative features, such as the use of multimedia data or the option of user-adaptive adjustment to an online dictionary" (Müller-Spitzer 2014: 6). After all, it is not easy for lexicographers to understand how to make proper use of them in specific e-dictionaries for specific users since related empirical research has provided an unclear picture.

Kaneta (2011) found dictionary interfaces influenced both the amount and the length of reference to illustrative examples, and users' choice of interfaces "depends on the quality of information available outside the folded elements" (Lew 2015), but Klosa et al. (2014) showed most users chose the expanded view as a default. Dziemianko (2015) proved that color influenced the speed and effectiveness of dictionary search as well as the retention of the retrieved information. Lew and Doroszewska (2009) found a negative effect of viewing animated images on vocabulary retention, but Kemmer (2014: 251-278) indicated a slight majority of users preferred pictorial information to verbal information, especially the younger generation (cf. Lew 2015). Lew et al. (2017) identified a surprising degree of balance in whether the picture or (verbal) definition was viewed first, and the fact that dictionary users normally viewed both. With mixed and even inconsistent findings, these studies highlighted the importance of using multimodal devices appropriately.

According to Gouws (2014), many lexicographic e-products were developed without any influence from innovative theoretical suggestions, and as a result current e-dictionaries often do not live up to the expectations of their users. It is necessary to devise theoretical models for e-dictionaries that focus on critical areas like the data to be included in these dictionaries, the structures to present and accommodate the data, and the way they should respond to the needs of their target users (Gouws 2014). E-dictionaries should be characterized by customized presentations of lexicographical data, and the display of data must be redefined (cf. Gouws 2014; L'Homme and Cormier 2014).

The potential of multimodal resources has not been well exploited for exemplification. For the sake of dictionary users, it is necessary to optimize the examples in e-dictionaries from a multimodal perspective. Exploring how multimodal exemplification influences meaning representation in the e-dictionary discourse may serve as a preliminary step for the construction of theoretical models for e-dictionaries.

 

3. Expansion of ideational meaning

From this part on, the article will explore how multimodal exemplification can expand the three metafunctional meanings of the semiotic system, ideational, interpersonal and textual (Halliday 1985). Ideational semiotic structures construct the nature of events, the objects and participants, and the circumstances; interpersonal structures construct the nature of relationships among addressers and addressees, and viewers and the viewed; and textual structures concern the distribution of the information value or relative emphasis among different elements (cf. Kress and Van Leeuwen 2006: 15).

The inherent relationship between the three metafunctions and the criteria for examples may help us understand multimodal exemplification better. According to Xu (2009: 10), there are four major criteria, namely, informativity, intentionality, acceptability and cohesion6. The ideational metafunction concerns informativity, the interpersonal has much to do with intentionality and acceptability, and the textual involves cohesion.

It is found that ideational meaning can be expanded by multimodal exemplification in three major ways: meaning enrichment realized by multimodal examples (i.e. multimodal resources used for examples), meaning enrichment mediated by example-definition ties, and meaning flow facilitated by example hyperlinks.

3.1 Meaning enrichment by multimodal examples

When necessary, the quantity and variety of examples can be increased in an e-dictionary, and more particulars or nuances of vocabulary usage can be shown than in print dictionaries, enriching the ideational meaning. With the technique of natural language processing, diversified classifications of examples, like collocations, quotations and proverbs, can be extracted from databases. And they can be tailored to user needs, for instance, encoding or decoding purposes. Many e-dictionaries, like Oxford and Longman, have an example bank with various corpus sentences for users to choose from. Multimodal resources are indispensible for enriching the ideational meaning of examples, and audio and visual modes are mostly used.

A lot of e-dictionaries offer the recordings of examples. They can be human voice and synthesized speech. The sound speed can be adjusted by dictionary users, or sometimes there are two speed alternatives (slow or fast). There are also sound choices of gender (male or female) and language variety (e.g. British English or American English). Some e-dictionaries even provide authentic recordings clipped from movies, speeches or news programs, like Youdao English Dictionaries (YED, http://dict.youdao.com/) while others may give those from well-known literary works and mainstream textbooks, like Fayu Zhushou French Dictionaries (FZFD, https://www.frdic.com/).

The visual mode, including color, font, symbol and notation, can also be properly used to reinforce the ideational meaning. Figure 1 shows a part of the love entry in Longman Dictionary of Contemporary English (LDOCE). Several semiotic resources are used to reinforce the representation of collocation and idiomatic usage, like highlighting such an expression in its examples with an underline and boldface font when the mouse cursor hovers over it (see "love at first sight" in Figure 1). During this multimodal human-computer interaction, the solid line emerges, which may imply a lexical bundle as cross-modal metaphor. Its emerging process might give some readers a sense of formation of the bundle.

 

 

Furthermore, there is simultaneous highlighting of parallel lexical units. For instance, in the bilingual examples of YED, any word and its Chinese/English counterpart will be concurrently highlighted with the mouse cursor hovering over it, as indicated by the match between "exploit" and in the second sentence in Figure 2 (screenshot from the entry of learn, http://dict.youdao.com/search?le=eng&q=learn&keyfrom=dict.top).

 

 

The visual information can also be synchronized with the audio. A word in a sentence changes its color when its recording is played, achieving a kind of real-time cross-modal alignment like karaoke singing or movie subtitles. By representing the ideational meaning of example sentences in great detail, such visual-visual or visual-audio synchronization may facilitate the understanding of specific words or collocations, which is likely to improve exemplification effectiveness.

3.2 Meaning enrichment by example-definition ties

The enrichment of ideational meaning within the microstructure is investigated by looking at the cross-modal/intermodal ties between a definition and an example under the same sense. Identifying cohesive links and examining the logical relations in ideational meaning that extend across semiotic modes may help us gain an in-depth understanding of intermodal synergy, the cooperation and interaction among different modes (Royce 1998, 2007; Chan 2011: 144-167). According to the model of Chan (2011: 144-165), there are two types of intermodal relations, concurrence and complementarity. The (intermodal) concurrence or agreement denotes that meanings across modes are similar while (intermodal) complementarity suggests that meanings across modes are different but complement each other (ibid.). An online children's dictionary, Wordsmyth Kid's English Dictionary (WKED), is used to illustrate the example-definition concurrence and complementarity.

The example-definition concurrence can be illustrated by the entry of swan in Figure 3 (screenshot from //kids.wordsmyth.net/wild/). The picture is a non-verbal definition of the headword according to the concept of multimodal definition (Liu 2015). The example sentence describes what the picture is, indicating intermodal concurrence. So meanings across modes are similar and in this way the example enriches the meaning of the verbal definition, indicating the difference in size between a swan and a duck or goose. At first sight, the dictionary users may see only two salient white swans in the foreground of the picture, and the example sentence can remind them of the small ducks and geese in the background (cf. Kemmer 2014: 251-278). That may constitute a two-step cognition process from a definition to examples.

 

 

Furthermore, I'd like to show how the example-definition complementarity can be achieved with the entry of owl in Figure 4 (http://kids.wordsmyth.net/wild/). The verbal mode (language) and the visual mode (picture) of its definition provide visual information about the owl by depicting its physical appearance. Its example sentence offers auditory information by describing its sound verbally. So the definition and the example complement each other. However, if an audio file of the owl's crying or the example sentence were offered, that would achieve example-definition complementarity to a greater extent. In this case, the inclusion of pertinent sound and images as devices optimizing meaning representation is grounded on the principle of synaesthesia.

 

 

3.3 Meaning flow through example hyperlinks

Hyperlinks in e-dictionaries help users find and build semantic connections between words. They can be used on each word in an example to direct the user to another entry in or outside the dictionary. Such exemplification problems as collocations and deixis can be greatly eased. As the smartphone application (app) of OALD (Oxford Advanced Learner's Dictionary) says, "you can tap on any word within an entry to jump to the entry for that word". That enhances hypertextuality of the dictionary and creates meaning flow. So meaning is expanded from the microstructure to other parts of the dictionary, or even outside the dictionary.

Furthermore, there is a trend of linking and fusing dictionaries with other resources (Chen 2010), i.e. hybridization (cf. Granger 2012). A local app of Fayu Zhushou French Dictionaries (FZFD) in China offers example sentences from novels, movies and textbooks, with hyperlinks directing users to the source materials. Figure 5 shows its entry of dans with sentences respectively from The Three Musketeers, Intermediate French Listening Textbook and Comprehensive French Textbook 2 in its smartphone app. The "traversals" between the semantic world within the dictionary and external learning resources are semiotically realized with the design of the page or screen (cf. Lemke 2005).

 

 

 

4. Expansion of interpersonal meaning

Interpersonal meaning concerns two major parties: dictionary users and makers/ writers. By user participation and interaction with the dictionary (makers), interpersonal meaning flows. Since there are more design options for dictionary makers, interpersonal meaning can be enriched.

4.1 User participation and interaction as meaning flow

Compared with paper dictionaries, e-dictionaries provide users with more chances of participation and interaction, facilitating greater flexibility in example use, which can expand interpersonal meaning and improve the acceptability of examples. Information can be presented in an array of interlinked web pages and media networks, enabling e-dictionary users to navigate and choose their own pathways through this semantic "universe". Users can tailor the use according to their own needs. They can change dictionary settings (like interface style), make bookmarks to examples, and store favourite sentences in their own example bank. Taking smartphone or tablet dictionary apps as an example, due to their portability, they can work as flash cards for learners to carry with them. And they have cameras, so a user can add a picture to certain words or examples to aid memorization and retrieval if such apps are open-ended.

Many e-dictionaries invite users to contribute examples or make comments on them (see "user input" by Granger 2012; and "user-generated content" by Lew 2014). In the sentence example bank interface of Jinshan Ciba English Dictionaries (iCIBA), there are user interaction buttons for reporting a mistaken example, praising a good one and saving a useful one. Figure 6 is a screenshot of examples of love (http://dj.iciba.com/love1.html#).

 

 

In Figure 6, the "Learn" button at the end of each example sentence indicates a timed activity of memorizing an example. When triggering it, the user will be invited to put in order the words in an example sentence in 40 seconds (see Figure 7). The time counter urges the user to finish the task quickly, arousing a strong sense of engagement. Furthermore, in contrast to the cool and rational blue which may represent authority in the example page (see Figure 6), bright orange is used for the learning activity interface, which may connote such cultural meanings or personal emotions as enthusiasm, fascination, creativity, determination, and stimulation (Gage 1993).

 

 

4.2 Design options as meaning enrichment

With less pressure to save space, there are more design choices for e-dictionaries. This diversification is likely to reinforce the interpersonal metafunction, improving the intentionality of examples. Integrated with customization, it may enhance their acceptability. The page/screen layout (spatial mode) and the example genre style (verbal mode) are taken as examples for illustration.

The page layout for examples usually looks less "crowded" and cluttered in e-dictionaries. Individual examples often start on a new line while they tend to be run-on in traditional print dictionaries (Lew and De Schryver 2014). Contemporary designs may have large bands of space simply to suggest lightness of the reading experience and also create a sense of "room to think" or "contemplation" (Zhang et al. 2015). A case in point is online Merriam-Webster Learner's Dictionary (MWLD). We find lots of white space for its example texts as the entry of love in Figure 8 shows (screenshot from http://www.learnersdictionary.com/definition/love). In contrast, the print dictionary design is filled with dense texts and communicates the opposite, providing comprehensive amounts of detailed information fulfilling its role as an informer.

 

 

The verbal mode of examples in e-dictionaries can have more choices for genre styles. MWLD, for instance, employs a conversational style. It makes extensive use of instructions and explanations about word usage for different examples. As Figure 8 shows, such instructions as "often used before another noun" and "when people begin to feel..." help enrich interpersonal meaning because they sound like tips from a thoughtful guide or a patient helper. This communicative style suggests a sense of accessibility on behalf of the speaker and of engagement with a reader presumed to have needs (Zhang et al. 2015).

Similar to the conversational style, there is a story style for WKED. The examples for each definition constitute a story in a continuous text unit (see Figures 3 and 4). The story style, together with the image on its right, may be very suitable for children's dictionaries for having advantages of readability and user-friendliness. Besides the interpersonal metafunction, this style may also help reinforce the textual metafunction, both coherence in content and cohesion in form. In contrast, the discrete examples in other dictionaries are relatively independent semantic units.

In a word, the digital transformation has changed the relations between dictionary writers and users, from one where authoritative experts presented information to a passive public to one of engagement and participation. This shift to user-friendliness might be driven by the social force of marketization (Zhang et al. 2015), attracting funding from advertisements to maintain the e-dictionary.

 

5. Expansion of textual meaning

This part analyzes how textual meaning is expanded in the multimodal example discourse in relation to its neighbouring elements. Information value, salience and framing are three interrelated systems in exploring textual meaning (Kress and Van Leeuwen 2006: 177). Information value refers to how certain elements are placed in a space to give them a particular value, which is closely related to composition types (ibid.). Salience means that elements are made to attract the viewer's attention to different degrees, as realized by such factors as placement in the foreground or background, relative size, contrasts in tonal value (or color), and differences in sharpness, etc (ibid.). Framing is the use of boundaries connecting or disconnecting spaces (cf. ibid.).

5.1 Information value and composition as meaning flow

According to Kress and Van Leeuwen (2006: 177-201), there are three main types of composition: vertical (top-bottom, Ideal-Real), horizontal (left-right, Given-New) and circular (Center-Margin, nucleus-subservient). They are used for different purposes on the e-dictionary screen.

On a vertical axis, elements at the top are often described as ideal, and the generalized essence of information and therefore usually most salient; elements placed at the bottom are often represented as real, and present more specific, factual information (cf. Leijon 2016). In the microstructure, a definition is usually placed above the examples under the same sense, forming a typical vertical composition. The definition could be ideal information while examples could be real. In the example text, the vertical composition is most often used. While consulting an e-dictionary, the user usually scrolls down from a definition to the examples, forming a top-bottom reading path. The example text can be further divided into several vertical compositions. In many dictionaries, examples are classified into groups of collocation patterns. Each pattern is followed by specific phrase or sentence examples (see Figure 1). The same rule of information value applies here.

There is a distinctive subtype of the vertical composition for examples in smartphone or tablet dictionary apps. In FZFD (see Figure 5), for instance, elements are mostly placed into equally sized tiles which could be swiped across to see more. This form of organization allows contents to be textually linked as choices of the same order since tiles of the same size also create textual linking or rhyming, alongside that accomplished by color and fonts (Zhang et al. 2015). In this sense, the textual meaning flows rhythmically and this kind of harmony may be important for small-screen reading.

In a horizontal composition, elements on the left could be given information, and those on the right could be new as we tend to read from left to right. This type of composition is found in many e-dictionaries. For instance, in WKED (see Figures 3 and 4), the verbal text of the definition and examples is placed on the left while the picture is on the right. Compared with the picture, the verbal text is given information. The image plays a distinctive role as bearing witness. As a non-verbal definition device (cf. Liu 2015), the picture is mainly aligned with the verbal definition. But the bottom of the picture is aligned with the top of the example text. Such a nuance in composition symbolizes that examples, as a complement to the definition, will describe the picture. So the position of the picture, if carefully chosen, can embody reading tips for dictionary users.

The circular composition is often used if we take a look outside the microstructure. Elements in a space could be located in a circular relation along the dimensions of Center (the nucleus) or Margin (more subservient) (Van Leeuwen 2005: 208). As far as a whole page of an e-dictionary is concerned, the definitions and examples are usually placed in the center, with additional information (usually hyperlinks) put around them. A triptych, a three-part structure, is often seen in such a circular layout.

On the basis of Kress and Van Leeuwen (2006: 177-201), Figure 9 is drawn to illustrate the above three composition types with their information value rules, which constitutes a typical design of visual space in e-dictionaries.

 

 

In terms of information value, the textual meaning usually flows from Ideal to Real, from Given to New, and from nucleus to subservient, forming an organic network of textual cohesion in e-dictionaries.

5.2 Salience and framing as meaning enrichment

According to Kress and Van Leeuwen (2006: 201), salience can create a hierarchy of importance among the elements, selecting some as more important, more worthy of attention than others. The Given may be more salient than the New, for instance, or the New more salient than the Given, or both may be equally salient. And the same applies to Ideal and Real and to Center and Margin. As illustrated above, in a typical visual design of e-dictionaries, more salience may be intended for the Given (verbal), Ideal (definition or collocation heading) and Center (definition and example) although psychological salience depends on individual users in specific cases.

For salience and framing, various semiotic devices can be coordinated such as color, font and line (cf. Liu and Qu 2014). LDOCE (see Figure 1) is characterized by the use of striking blue with high saturation and boldface font for the collocation heading, which helps framing, symbolizing more salience and more information value on collocation patterns (i.e. fixed phrase examples standing for generality) than sentence examples (embodying specificity). Instead, MWLD (see Figure 8) extensively uses icons (squares, addition marks, dashes and diamonds etc.) for framing. FZFD (see Figure 5) uses lines instead. The different styles may cater for different cognitive styles of dictionary users.

In addition, foregrounding and backgrounding are used on an e-dictionary screen. For instance, there is a practical function of hiding examples which may help optimize textual cohesion in e-dictionaries. A case in point is the off-line desktop version of Jinshan Ciba English Dictionaries (JCED) where the examples following each definition can be hidden. The "hide" (top of Figure 10) and "unhide" (bottom of Figure 10) functions give users chances of arranging the interface layout for their own sake. This customization device may facilitate their cognitive process and also enhance the interpersonal metafunction as well.

 

 

With hierarchies of salience and devices of framing, the example texts in e-dictionaries allow for more reading paths than paper dictionaries, and hence for the heterogeneity and diversity of their large readership (cf. Van Leeuwen 2005: 205). Unlike movies or exhibitions, e-dictionary example texts are nonlinear. Non-linear texts often impose a paradigmatics (instead of syntagmatics), leaving the reader to sequence and connect the elements presented (Kress and Van Leeuwen 2006: 208). In the design of such texts there will be pressure to put more of the meaning in the individual elements of the composition, and to use more self-contained items of information (ibid.).

 

6. Discussion and conclusion

6.1 Main findings and implications

This study was inspired by Lew (2010). Following my previous work on multi-modal definition, the article tentatively proposes a new term, multimodal exemplification. A holistic view of meaning and a systemic-functional approach to the semiotic system are adopted for multimodal discourse analysis. Complementing previous literature, the study has the following main findings.

First, the concept of meaning expansion in the field of MDA is extended, with evidence that multimodal exemplification can expand meaning by enriching it and making it "flow" in the semantic network. In this process of expansion, the three metafunctions of the semiotic system can be reinforced: ideational, interpersonal and textual.

Second, meaning can be enriched not only by multimodal examples per se but also by example-definition ties. To the best of my knowledge, this is the first attempt to explore the intermodal relationship between a definition and the examples under the same sense.

Third, the dynamicity and interactivity of e-dictionary example texts enable user participation, and freedom from the traditional paper format allows for more design options, expanding the interpersonal meaning and improving the acceptability and intentionality of examples. Users can enjoy flexibility and customization while creating their own reading paths. Regarding the verbal mode of examples, I identified two distinctive genre styles (the conversational style and the story style), supporting the claim that dictionaries are "discourses in their own right" and they are "divergent and intertwining" (Chen 2015).

Fourth, concerning the example texts, I analyzed the three main composition types with corresponding information value rules, presenting a diagram illustrating a typical design of visual space (see Figure 9). This may be the first endeavor to do so in lexicography.

Last, smartphone dictionary app pages often have a vertical composition of equally-sized tiles, which creates choices of the same order, and textual linking or rhyming. With such findings, the current study echoes with research on other types of multimodal texts, such as science communication websites (Zhang et al. 2015).

The current study has two implications for lexicography and multimodal discourse studies. One implication is that the uniqueness and diversity of e-dictionary example texts are central to the critical analysis and understanding of the multimodal discourse. Different types of e-dictionaries have distinctive features, and each needs a close look separately (cf. L'Homme and Cormier 2014).

The other implication is that an integrative and panoramic perspective should be taken to broaden the horizons of multimodal lexicography. For one thing, different modes and their semiotic devices should be coordinated to make new meanings. For another, the three metafunctions are intrinsically interrelated and can be seamlessly integrated in example texts. Color, for instance, is multi-functional for it can express ideational meaning, arouse personal emotions, and also serve as a tool for separation and framing (cf. Kress and Van Leeuwen 2006: 230).

6.2 Limitations and future research

Largely relying on introspective analysis, this study has some limitations. Empirical research into user needs should be conducted to support the arguments stated in the article. More investigations are needed into the mode selection and intermodal synergy for e-dictionary examples. More data should be collected from smartphone and tablet dictionary apps in the global wave of mobile assisted language learning. This leaves much room for future work.

There are two main directions for future research, user research and (multimodal) lexical database, which have become the foci of lexicographical research in the digital revolution (cf. Thieberger 2011; Fellbaum 2014; Lew 2015; Lew et al. 2017). First and foremost, analysis should be made to uncover how such multimodality impacts upon users' experiential and cognitive encounters with these examples. User research should focus on user needs in specific types of situations, which may lay a foundation for customized example classifications (cf. Bergenholtz and Bothma 2011; Potgieter 2012). Furthermore, multimodal lexical databases should be built on the basis of multimodal corpora and be finely tuned to a variety of parameters or facets of user customization (cf. Kipp et al. 2009; Bergenholtz and Nielsen 2013). These two directions of future research may help build a solid empirical base for a theoretic model of multimodal lexicography.

In summary, the research questions proposed at the beginning of this article have been answered. As an important part of multimodal lexicography, multimodal exemplification deserves scholarly attention since it can bring both chances and challenges for meaning representation.

 

Acknowledgements

This work was supported by the China Scholarship Council [201706155083], the Philosophy and Social Science Fund of the 12th Five-year Plan of Guangdong Province [GD15XWW23], and the Philosophy and Social Science Fund of the 13th Five-year Plan of Guangzhou City [2016GZGJ68]. I would like to express my deep gratitude to Prof. Robert Lew of Adam Mickiewicz University in Poznan for his discussion about multimodal lexicography during his visit to China in 2015. Furthermore, I am obliged to Prof. Paul van den Hoven of Utrecht University for analyzing the problems in an early draft of this paper. Special thanks are also due to Prof. Danie J. Prinsloo and the anonymous reviewers for their constructive comments.

 

References

A. Dictionaries

COBUILD. 1995. Sinclair, John M. (Ed.). 1995. Collins COBUILD English Language Dictionary. Second edition. London/Glasgow: HarperCollins.

FZFD. Fayu Zhushou French Dictionaries (smartphone app from http://rj.baidu.com/soft/detail/11669.html?ald). Accessed on February 13, 2016.

iCIBA. iCIBA Sentence Example Bank. http://dj.iciba.com. Accessed on February 8, 2016.

JCED. Jinshan Ciba English Dictionaries (offline desktop version downloaded from http://rj.baidu.com/soft/detail/11908.html?ald). Accessed on February 15, 2016.

LDOCE. Longman Dictionary of Contemporary English. Accessed on February 8, 2016. http://www.ldoceonline.com/dictionary.

MWLD. Merriam-Webster Learner's Dictionary. Accessed on February 15, 2016. http://www.learnersdictionary.com/.

OALD. Oxford Advanced Learner's Dictionary (smartphone app). Eighth edition. Accessed on February 15, 2016.

WKED. Wordsmyth Kid's English Dictionary. Accessed on February 10, 2016. http://www.wordsmyth.net/wild.

Wordnik. https://www.wordnik.com/.

YED. Youdao English Dictionaries. Accessed on February 10, 2016. http://dict.so.163.com/. http://dict.youdao.com.

B. Other literature

Atkins, B.T. Sue. 1996. Bilingual Dictionaries: Past, Present and Future. Gellerstam, Martin, Jerker Järborg, Sven-Göran Malmgren, Kerstin Norén, Lena Rogström and Catarina Röjder Papmehl (Eds.). 1996. Euralex '96 Proceedings I-II, Papers Submitted to the Seventh EURALEX International Congress on Lexicography in Göteborg, Sweden: 515-546. Gothenburg: Department of Swedish, Göteborg University.         [ Links ]

Bergenholtz, Henning and Theo J.D. Bothma. 2011. Needs-adapted Data Presentation in eInformation Tools. Lexikos 21: 53-77.         [ Links ]

Bergenholtz, Henning and Jesper Skovgård Nielsen. 2013. What is a Lexicographical Database? Lexikos 23: 77-87.         [ Links ]

Chan, Eveline. 2011. Integrating Visual and Verbal Meaning in Multimodal Text Comprehension: Towards a Model of Intermodal Relations. Dreyfus, Shoshana, Susan Hood and Maree Stenglin (Eds.). 2011. Semiotic Margins: Meaning in Multimodalities: 144-167. London: Continuum.         [ Links ]

Chen, Chih-Ming and Ching-Ju Chung. 2008. Personalized Mobile English Vocabulary Learning System Based on Item Response Theory and Learning Memory Cycle. Computers & Education 51(2): 624-645.         [ Links ]

Chen, Wenge. 2015. Bilingual Lexicography as Recontextualization: A Case Study of Illustrative Examples in a New English-Chinese Dictionary. Australian Journal of Linguistics 35(4): 311-333.         [ Links ]

Chen, Yuzhen. 2010. Dictionary Use and EFL Learning: A Contrastive Study of Pocket Electronic Dictionaries and Paper Dictionaries. International Journal of Lexicography 23(3): 275-306.         [ Links ]

De Beaugrande, Robert. 1980. Text, Discourse, and Process: Toward a Multidisciplinary Science of Texts. London: Longman.         [ Links ]

Dziemianko, Anna. 2015. Colours in Online Dictionaries: A Case of Functional Labels. International Journal of Lexicography 28(1): 27-61.         [ Links ]

Fellbaum, Christiane. 2014. Large-scale Lexicography in the Digital Age. International Journal of Lexicography 27(4): 378-395.         [ Links ]

Fox, Gwyneth. 1987. The Case for Examples. Sinclair, John M. (Ed.). 1987. Looking Up. An Account of the COBUILD Project in Lexical Computing and the Development of the Collins COBUILD English Language Dictionary: 137-149. London/Glasgow: Collins ELT.         [ Links ]

Frankenberg-Garcia, Ana. 2012. Learners' Use of Corpus Examples. International Journal of Lexicography 25(3): 273-296.         [ Links ]

Frankenberg-Garcia, Ana. 2015. Dictionaries and Encoding Examples to Support Language Production. International Journal of Lexicography 28(4): 490-512.         [ Links ]

Gage, John. 1993. Color and Culture: Practice and Meaning from Antiquity to Abstraction. Boston: Bulfinch Press.         [ Links ]

Gouws, Rufus H. 2014. Article Structures: Moving from Printed to e-Dictionaries. Lexikos 24: 155-177.         [ Links ]

Gouws, Rufus H. and Sven Tarp. 2016. Information Overload and Data Overload in Lexicography. International Journal of Lexicography (Advance access). https://doi.org/10.1093/ijl/ecw030.

Granger, Sylviane. 2012. Electronic Lexicography: From Challenge to Opportunity. Granger, Sylviane and Magali Paquot (Eds.). 2012. Electronic Lexicography: 1-11. Oxford: Oxford University Press.         [ Links ]

Halliday, Michael A.K. 1985. An Introduction to Functional Grammar. London: Edward Arnold.         [ Links ]

Kaneta, Taku. 2011. Folded or Unfolded: Eye-tracking Analysis of L2 Learners' Reference Behaviour with Different Types of Dictionary Interfaces. Akasu, Kaoru and Satoru Uchida (Eds.). 2011. Asialex2011 Proceedings Lexicography: Theoretical and Practical Perspectives: 219-224. Kyoto: Asian Association for Lexicography.         [ Links ]

Kemmer, Katharina. 2014. Rezeption der Illustration, jedoch Vernachlässigung der Paraphrase? Müller-Spitzer, Carolin (Ed.). 2014. Using Online Dictionaries: 251-278. Berlin/Boston: Walter de Gruyter.         [ Links ]

Kipp, Michael, Jean-Claude Martin, Patrizia Paggio and Dirk Heylen (Eds.). 2009. Multimodal Corpora: From Models of Natural Interaction to Systems and Applications. Berlin/Heidelberg: Springer.         [ Links ]

Klosa, Annette, Alexander Koplenig and Antje Töpel. 2014. Benutzerwünsche und -Meinungen zu dem monolingualen deutschen Onlinewörterbuch Elexiko. Müller-Spitzer, Carolin (Ed.). 2014. Using Online Dictionaries: 281-384. Berlin/New York: Walter de Gruyter.         [ Links ]

Kremer, Gerhard and Andrea Abel. 2010. Semantic Relations in Cognitive eLexicography. Dykstra, Anne and Tanneke Schoonheim (Eds.). 2010. Proceedings of the XIV Euralex International Congress, Leeuwarden, 6-10 July 2010: 380-388. Leeuwarden: Afûk.         [ Links ]

Kress, Gunther and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. Second edition. London: Routledge.         [ Links ]

Leijon, Marie. 2016. Space as Designs for and in Learning: Investigating the Interplay between Space, Interaction and Learning Sequences in Higher Education. Visual Communication 15(1): 93-124.         [ Links ]

Lemke, Jay L. 2002. Travels in Hypermodality. Visual Communication 1(3): 299-325.         [ Links ]

Lemke, Jay L. 2005. Multimedia Genres and Traversals. Folia Linguistica 39(1-2): 45-56.         [ Links ]

Lew, Robert. 2010. Multimodal Lexicography: The Representation of Meaning in Electronic Dictionaries. Lexikos 20: 290-306.         [ Links ]

Lew, Robert. 2014. User-generated Content (UGC) in English Online Dictionaries. Abel, Andrea and Annette Klosa (Eds.). 2014. Der Nutzerbeitrag im Wörterbuchprozess 3. Arbeitsbericht des wissenschaftlichen Netzwerks "Internetlexikografie". (Opal - Online publizierte Arbeiten zur Linguistik): 8-26. Mannheim: Institut für Deutsche Sprache.         [ Links ]

Lew, Robert. 2015. Research into the Use of Online Dictionaries. International Journal of Lexicography 28(2): 232-253.         [ Links ]

Lew, Robert and Gilles-Maurice de Schryver. 2014. Dictionary Users in the Digital Revolution. International Journal of Lexicography 27(4): 341-359.         [ Links ]

Lew, Robert and Joanna Doroszewska. 2009. Electronic Dictionary Entries with Animated Pictures: Lookup Preferences and Word Retention. International Journal of Lexicography 22(3): 239-257.         [ Links ]

Lew, Robert, Rafal Kazmierczak, Ewa Tomczak and Mateusz Leszkowicz. 2017. Competition of Definition and Pictorial Illustration for Dictionary Users' Attention: An Eye-tracking Study International Journal of Lexicography (Advance access). https://doi.org/10.1093/ijl/ecx002.

L'Homme, Marie-Claude and Monique C. Cormier. 2014. Dictionaries and the Digital Revolution: A Focus on Users and Lexical Databases. International Journal of Lexicography 27(4): 331-340.         [ Links ]

Liu, Xiqin. 2015. Multimodal Definition: The Multiplication of Meaning in Electronic Dictionaries. Lexikos 25: 210-232.         [ Links ]

Liu, Xiqin and Dianning Qu. 2014. Exploring the Multimodality of EFL Textbooks for Chinese College Students: A Comparative Study. RELC Journal 45(2): 135-150.         [ Links ]

Müller-Spitzer, Carolin (Ed.). 2014. Using Online Dictionaries. Lexicographica Series Maior 145. Berlin: Walter de Gruyter.         [ Links ]

Müller-Spitzer, Carolin, Alexander Koplenig and Antje Töpel. 2012. Online Dictionary Use: Key Findings from an Empirical Research Project. Granger, Sylviane and Magali Paquot (Eds.). 2012. Electronic Lexicography: 425-457. Oxford: Oxford University Press.         [ Links ]

Nielsen, Sandro. 2014. Example Sentences in Bilingual Specialised Dictionaries Assisting Communication in a Foreign Language. Lexikos 24: 198-213.         [ Links ]

O'Halloran, Kay L. 2008. Systemic-functional Multimodal Discourse Analysis (SF-MDA): Constructing Ideational Meaning Using Language and Visual Imagery. Visual Communication 7(4): 443-475.         [ Links ]

O'Halloran, Kay L. 2011. Multimodal Discourse Analysis. Hyland, Ken and Brian Paltridge (Eds.). 2011. The Continuum Companion to Discourse Analysis: 120-137. London: Continuum.         [ Links ]

Potgieter, Liezl. 2012. Example Sentences in Bilingual School Dictionaries. Lexikos 22: 261-271.         [ Links ]

Prinsloo, Danie J. 2015. Corpus-based Lexicography for Lesser-resourced Languages - Maximizing the Limited Corpus. Lexikos 25: 285-300.         [ Links ]

Prinsloo, Danie J., Ulrich Heid, Theo Bothma and Gertrud Faaß. 2012. Devices for Information Presentation in Electronic Dictionaries. Lexikos 22: 290-320.         [ Links ]

Royce, Terry. 1998. Synergy on the Page: Exploring Intersemiotic Complementarity in Page-Based Multimodal Text. JASFL Occasional Papers 1(1): 25-49.         [ Links ]

Royce, Terry. 2007. Intersemiotic Complementarity: A Framework for Multimodal Discourse Analysis. Royce, Terry and Wendy Bowcher (Eds.). 2007. New Directions in the Analysis of Multimodal Discourse: 63-109. Mahwah, NJ: Lawrence Erlbaum.         [ Links ]

Rundell, Michael. 2015. From Print to Digital: Implications for Dictionary Policy and Lexicographic Conventions. Lexikos 25: 301-322.         [ Links ]

Stein, Gabrielle. 2002. Better Words: Evaluating EFL Dictionaries. Exeter: Exeter University Press.         [ Links ]

Szende, Thomas. 1999. Problems of Exemplification in Bilingual Dictionaries. Lexicographica 15: 198-228.         [ Links ]

Taljard, Elsabé. 2015. Collocations and Grammatical Patterns in a Multilingual Online Term Bank. Lexikos 25: 387-402.         [ Links ]

Tarp, Sven. 2008. Lexicography in the Borderland between Knowledge and Non-knowledge: General Lexicographical Theory with Particular Focus on Learner's Lexicography. Tübingen: Max Niemeyer.         [ Links ]

Thieberger, Nick. 2011. Building a Lexical Database with Multiple Outputs: Examples from Legacy Data and from Multimodal Fieldwork. International Journal of Lexicography 24(4): 463-472.         [ Links ]

Van Leeuwen, Theo. 2005. Introducing Social Semiotics. London/New York: Routledge.         [ Links ]

Wittgenstein, Ludwig. 1953. Philosophical Investigations. Oxford: Blackwell.         [ Links ]

Wojciechowska, Sylwia. 2015. Collocations and Colligations of the Metonymic Shipment: Exemplification in Learners' Dictionaries vs. Corpus Evidence. International Journal of Lexicography 28(4): 465-489.         [ Links ]

Xu, Hai. 2009. Towards Prototypical Exemplification in English Dictionaries for Chinese EFL Learners. Beijing: The Science Press.         [ Links ]

Zhang, Yihua. 2015. Second Language Acquisition and Learner's Dictionaries. Beijing: The Commercial Press.         [ Links ]

Zhang, Yiqiong, David Machin and Tao Song. 2015. Visual Forms of Address in Social Media Discourse: The Case of a Science Communication Website. Journal of Multicultural Discourses 10(2): 236-252.         [ Links ]

 

 

1. Multimodal definition is (the act of) meaning explanation of a word or phrase with multimodal devices for achieving better defining effect than language does alone, especially in an e-dictionary (Liu 2015).
2. The number and complexity of examples should be increased with precautions against information overload (cf. Gouws and Tarp 2016), though. Dictionary users want information fast.
3. Hypermodality is more than multimodality in just the way that hypertext is more than plain text, and it is not simply that we juxtapose image, text, and sound; we design multiple interconnections among them, both potential and explicit (Lemke 2002). In the simplest form of hypertext, we might have a web of "pages" (or paragraphs, sentences, or even single words) in which the whole or some part of the page was linked to the whole or some part of another page in some way other than by the default sequential convention of ordinary reading (ibid.).
4. A communicative system simultaneously fulfills three metafunctions: the ideational, the function of constructing representations of the world; the interpersonal, the function of enacting (or helping to enact) interactions characterized by specific social purposes and specific social relations; and the textual, the function of marshalling communicative acts into larger wholes, into the communicative events or texts (Kress and Van Leeuwen 2006: 228).
5. The Big Five dictionaries refer to the following English learner's dictionaries: Oxford, Longman, Collins COBUILD, Cambridge and Macmillan.
6. Xu (2009: 9-10) tailored De Beaugrande's (1980: 6) seven criteria for textuality to be four criteria for illustrative examples, viz. intentionality, informativity, cohesion and acceptability. Intentionality concerns lexicographic intentions behind a dictionary example; informativity means the types and amount of linguistic information shown; cohesion involves the ties between the dictionary example and its source text, and the links between the example and the dictionary text; and acceptability refers to the degree to which an example satisfies its target users (ibid.).

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons