The use of multi-media in language assessment
In this world of advanced technological realities, the involvement of technology in language teaching and learning is already very familiar in some contexts. Technology helps learners to deal with the tasks requiring complex thinking and the use of language. However, learners' differences in style of learning need to be considered in order to achieve a successful learning. Regarding to L2 aptitude and learning style, (Skehan 1998 cited in Chapelle 2004, p. 596 ) “we now seem to be in a position where views of style are ready to underpin fresh research efforts…it is only by such (research) that we can discover whether style is both separable from the ability and important for second and foreign language learners”. Based on research findings of Chun and Payne (2004), Chapelle (2004) stated that in CALL individual differences in style and aptitude need not be an obscure academic interest. Instead, their research demonstrates that how specific aspects of the learning material can be suited to the needs of individual learners. (Chun and Payne 2004 cited in Chapelle 2004, p. 596) found that learners with lower verbal working memory capacity (WMC) tended to use the built-in dictionary functions in a CALL program to look up more words. They suggest that students use the multimedia look-up features of the CD ROM to compensate for working memory capacity constraints. “While the use of visualization technology is a crucial advance in the teaching of intonation, such teaching can be further enhanced by connecting the technology to an understanding of how intonation functions in discourse”, (Levis and Pickering 2004 cited in Chapelle 2004).
From those findings above and others study which research the use of multimedia in language teaching and learning, we can see many advantages that technology brings to the teaching learning process. The use of multimedia in language teaching slowly suggests people to use multimedia for language testing. However, the use of multimedia in language assessment still remains problematic issue and dilemma regarding to its validity and practicality. In this case, we need to know how the assessment can be conducted without ignoring the quality of knowledge that the test want to measure.
Language assessment is conducted to know the language ability or language proficiency of test takers. This is done for different purposes regarding to the need of individual or institution. Assessment is a crucial thing, thus the design of the test (related to choosing test procedure, medium, and the method in language test) plays important role towards the result that will be resulted from the test itself. As McNamara (2000, p. 4) said that “language tests play a powerful role in many people's lives, acting as gateways at important transitional moments in education, in employment, and in moving from one country to another”. Therefore, the test designer should pay attention to all aspects which will influence the result of the test. Of course it must include validity, reliability, and practicality.
This essay discusses about advantages and disadvantages of the use of multimedia in language assessment. In the discussion below I provide the discussion based on some research findings.
Test designer uses multimedia in language assessment in various ways. It is used based on the purpose of the test. Some use as a delivery test only, but some use for both delivery test and examiner. For instance, Computer-assisted testing (CAT) uses a live examiner while the computer as a word processor. Participant uses the computer as an aid to look up dictionary or thesaurus. Here the participant can develop their creativity in language skill by using the resources from the computer, producing the essay, for instance. On the other hand, as McNamara (2000) pointed out in Britain the use of CAT is quite different, it is called CBELT-computer-based English language testing. McNamara (2000) pointed out that
“This approach uses any of a wide variety of test procedure in test delivery. But unlike CAT, CBELT is scored exclusively by computer. This constraint tends to restrict somewhat the scope item types that can be successfully administered…Rapid, accurate correction of the test could be provided for such items as multiple-choice reading comprehension questions, objective lexical and grammatical questions, editing procedures, and even some translation tasks”.McNamara (2000, p. 117)
In addition, another type of test that uses the computer is computer-adaptive language test (CALT). McNamara (2000, p. 117) stated that “the CALT resembles the CBELT version of CAT: it is computer-driven and computer-scored. But unlike CBELT, the adaptive test is uniquely tailored for each student”. The test is provided in large item bank and it adapt to the level of the test takers' knowledge. If the test takers can answer the first question, the system will provide the next question with next level of language ability. The test results the level of language ability or proficiency that the participants at. It also allows providing different questions for different participant.
Furthermore, as McNamara (2000, p. 79) pointed out that “many important national and international language tests, including TOEFL, are moving to computer-based testing (CBT). Stimulus and prompt are presented not in examination booklet but on the screen, with the candidates being required to key in their responses”. Moreover, there are some other uses of multimedia in language test such as in listening and speaking.
The use of audio and video in listening comprehension test
In listening comprehension test, beside the use of audio-tape to assess listening skill, some test designers also use video media including video tapes, video disks, satellite teleconferencing resources, and computer applications. This still remains problematic issue among the researches in language testing field. It seems no agreement yet to choose which one is effective for assessing the listening.
However, the use of video media challenges the assessor in assessing the quality of listening. In this case, the effect of visualization in listening is debated, whether it is still part of listening or not. (Muller, 1980 cited in Gruba 1997) “utilizing simple line drawings to accompany audio-taped presentations, suggest that visual context may bolster scores of low level candidates but concludes that visuals in and of themselves do not necessarily enhance comprehension”. Another research conducted by Coniam (2001) in the test for English language teachers as a part of certification in Hongkong (Hongkong English Language Benchmark Test) found that the effect of video media distracted the attention of participant to the listening comprehension. The participant said that they are disturbed by the image in the video so they felt it is not effective to have a test of listening comprehension by using video. Based on the comparison of the two group in that case study, Coniam (2001, p. 1) concludes that ”As a high-stakes certification test for English language teachers, the listening comprehension test should be conducted via an audio, not video, mode”.
In this case we need to consider the quality of listening comprehension by looking at validity, practicality of the test. The use of audio is cheaper compared to video. Video needs more preparation regarding to presenting the images. Thus most people can afford the audio-taped test rather than video. However, some researchers argue that another reason separated from practicality is the involving understanding of culture in listening comprehension leads test developer to use video as an instrument for assessment. (Tuff and Tudor, 1990 cited in Gruba 1997, p. 336) conducted an investigation of the possibility of video to support the comprehension of a message. They found that
“Both British native speaking viewers and Belgian non native speakers were shown the same authentic British video broadcasts and were directed to respond to comprehension questions in their native language. The researchers found that the Belgians were far less able to avail themselves of the potentially supportive function of the visual channel than the native speaker group” and that this inability seems strongly linked to the proximity of their native culture to that of the target video.(Tuff and Tudor, 1990 cited in Gruba 1997, p. 336)
This research shown the involving understanding of culture is important in understanding listening comprehension. Moreover, Rubin (1995 cited in Gruba 1997, pp. 336-337) defined “listening skill is an active process in which listeners select and interpret information which comes from auditory and visual cues in order to define what is going on and what the speakers are trying to express”. (Gassin, 1992; Hurley, 1992; Antes, 1996 cited in Gruba, p. 337) point out that nonverbal elements are crucial to communication, (Dunkel, 1991b, cited in Gruba, 1997, p. 337) sees an urgent need for a model of communication that includes nonverbal signals as society enters a post-literate society dominated by video media. However, (Kellerman, 1992 cited in Gruba 1997, p. 336) stated that construct definitions of listening have varied widely but few theorists have acknowledged the influence of nonverbal communication in the comprehension process. Analyzing the 50 definitions of the listening skill, it is found that only eight contained references to visual elements (Glen 1989 cited in Gruba 1997, p. 336).
Semi-direct test in speaking
Semi-directed test is conducted by using the tape recorder. The test is provided a prompt in the tape recorder and the participants have to respond as they are speaking to a person. The responses are recorded and the performance is scored from the tape. This test however is different from the speaking test “live face to face interview”. Participants can see interviewer's face and can feel like talking in the real communication even though it is in a live but at least they can see each other. As McNamara (2000, p. 82) pointed out that “not everybody likes speaking to the tapes. We all know the difficulty many people experience in leaving messages on answering machines. Most test-takers prefer a direct rather than a semi direct format if given the formats”.
Regarding to the interaction, this kind of test seems to ignore the real interaction. Live interaction, however, still can be considered as a real communication, as most live interaction that often used in media such as live interactive discussion.
On the other hand, semi-direct test has its own advantages as it is cheaper than live interaction. The test does not need to provide the interlocutor who interacts with the candidate. Moreover, as McNamara (2000, p. 83) points out that some interlocutors elicit performances which trigger a favourable impression of the candidate; others have the reverse effect. As semi-direct test remove the interlocutor variable-all candidates face the same prompt, delivered by tape-it might be felt that the semi-direct test has the potential to be a fairer test.
However, we need to pay attention to the interaction as mentioned above. As McNamara (2000, p. 84) points out that “As soon as you try to test use (as opposed to usage) you cannot confine yourself to the single individual”. Moreover, interaction in real life communication is human interaction. So can it be replaced by machine? McNamara (2000, p. 85) stated that “…we may need a continuing reminder of the nature of communication as a shared human activity, and that the idea that one of the participants can be replaced by a machine is really a technological fantasy”.
How the inferences can be made on the basis of evidence
To look at assessment in new and challenging ways, the test developer need to be aware the nature of language for each skill (four macro skills: listening, speaking, reading and writing). Each skill has the specific aspect to look at in order to know the ability of participant in that certain skill. The use of multimedia sometime ignore some components of the skill or sometime adding another knowledge in order to deal with those skill such as computer literate and so on.
In addition, the inference that can be made from the result of test is a very crucial part in language assessment. Based on learners' responses to the questions in the test, examiner can make inferences about the learners' ability. The responses also need to be suited with the components of language skills as mentioned above, to consider the participant have the ability that is being measured.
Other kind of use of multimedia
Internet-based language testing, online-communication, interactive learning activities are also very famous in the education system and real communication today. Assessing language through computer technology is a becoming more popular. It may because of the real life communication involve much more technology so language teachers and the test developer also include that as a medium to learn and test the language. Moreover, technology-based language assessment is very popular in educational system, especially in foreign language assessment.
Based on the discussion above, many types of tests have already been used in assessing language using multimedia. There are advantages and also drawbacks in using technology as a medium to assess the language ability. However, test developers have different views in seeing the effectiveness in assessing language using multimedia. For example, in assessing listening skill, some argue it needs to involve verbal communication. Some see listening is only a part of audio, the process to absorb the meaning from the information presented by audio-taped.
To somehow reviewing some reasons why people use video in language assessment, (Bachman, 1990; Dunkel et al., 1993 cited Gruba 1997) proposed four categories of purpose which video media may address in assessing listening comprehension:
“Theory driven: a belief that assessment instruments must better correspond to models of language comprehension that explicitly incorporate aspects of culture (Philips, 1995), kinesics (Antes, 1996), or other aspects of non verbal communication (Kellerman, 1992) drives the test developer to look beyond audio tape as a mode of presentation. Such a test developer adopts a construct definition which incorporates both visual and verbal elements (e.g. Rubin, 1995b); pedagogy related: foreign language teachers have employed visual aids for a variety of pedagogical reasons since the beginning of the profession. Despite the use of video to teach, there is almost total exclusion of television materials from the assessment process. The use of video in assessment conducted is to integrate assessment procedures with existing pedagogic practices (Benson and Benson, 1994); specific purposes: use the video media in language assessment because there are features of the process, or setting, of how language is being used which cannot be separated from its meaning. In a sense, assessment concerns touch on the area of English for specific purposes instructional video usage (Kennedy, 1983); mode of delivery: the use in distance learning programs (clark and Hoosmand, 1992; Oxford et al., 1993, Layne and Lepeintre, 1996)”. (Bachman, 1990; Dunkel et al., 1993 cited in Gruba 1997, pp. 338-339)
To conclude, the use of multimedia in assessment is mostly because of pedagogy related and considering the use of multimedia in real life communication. If we see in real life, technology is mostly involved in real life communication. It is true that most media presented by using video that involving image and sound. However, in language assessment we need to pay attention to the quality of each listening itself. The quality of listening is the process that people heard by the audio. It plays important part in listening comprehension. The involving image is only an extra thing to know detail where it happens and to see the real action that is described by the statement that is being listened by the audio. The comprehension is only gained by the information that is heard by using audio. Thus, the involving of video can be a distraction for some people (as Coniam's findings, 2001). It may also better to suggest that if the test uses video media, the design of video content must be suited to testing listening skill.
Furthermore, in testing speaking ability, we need to be aware of the definition of speaking itself so it will not be ignored what we mean by interaction as a human activity. Semi-direct test such as discussed above only involving the machine as interlocutor. I see that only involve one way discussion. The participant speaks to the machine only, while in real interaction we can see face to face interaction, not talking to the machine. In real life interaction, it involves the self-confident of the speaker, it is crucial, especially for foreign language learners which they are leaning others language. The proficiency or ability in speaking skill involves the real performance to speak to the person, not to the machine. On the other hand, live face to face interaction is quite similar to the real communication even though the interlocutor is not directly sitting infront of participants.
Related to the content validity, it involves the same consideration as other kind of test, as the test also involves the domain of language is being tested. Moreover, practicality and reliability is still must be considered in some kind of the tests that are using multimedia, for example, using video in listening, as it costs more expensive in creating the video rather than audio-taped.
In other kind of tests such as CAT, CALT, and CBELT and other kind of computer-assisted tests, it needs to take consideration as not all people able to read on the screen as well as reading the paper. It will affect the results of the test as participants' ability and it may be less practical.
As a test developer we need to keep in mind that every language test need to measure what we want to test by paying attention to the aspects or components that we need to involve in testing each skills as all skills have different component that the participants need to achieve. As McNamara (2000, p. 86) points out that “…a language test is only as good as the theory of language on which it is based, and it is within this area of theoretical inquiry into the essential nature of language and communication that we need to develop our ability to ask the next question”.
Moreover, multimedia brings many advantages in improving teaching and learning, but dealing with assessment the use of multimedia remains problematic issue for many researchers in the field of language testing, small cases in some kind test or even bigger in others.
Chapelle, C. A. (2004). Technology and second language learning: expanding methods and agendas. System, 32, pp. 593-601. Iowa state university. USA
Coniam, D. (2001). The use of audio or video comprehension as an assessment instrument in the certification of English language teachers: a case study. System, 29, pp. 1-14. The Chinese University of Hongkong, Hongkong.
Gruba, P. (1997). The role of video media in listening assessment. System, 25 (3), pp. 335-345.
McNamara, T. F. (2000). Language Testing. Oxford University Press.