Desingning a test

Desingning a test

1. Test Types

   • Language aptitude
     • Measure capacity or general ability to learn a foreign language and ultimate success in that undertaking
     • Show some significant correlations with ultimate performance of students in language courses.
   • Achievement
     • Related directly to classroom lessons, units, or even a total curriculum.
     • Achievement tests are limited to a particular material addressed in a curriculum within a particular time frame.
   • Profiency
     • Test global competence in a language. It tests overall ability.
     • Proficiency tests are almost always summative and norm-referenced. They provide results in the form of a single score.
     • A key issue in testing proficiency is how the constructs of language ability are specified.
   • Diagnostic
     • Diagnose specified aspects of a language.
     • Diagnostic tests should elicit information on what students need to work on in the future.
   • Placement
     • Place a student into a particular level or section of a language curriculum or school. Secondary benefits to consider include face validity, diagnostic information on students' performance, and authenticity.
     • Depending on the nature of a program and its needs, some programs simply use existing standardized proficiency tests.

2. Test Construction

   • 1. Assessing clear, unambiguous objectives
     • Your first task in designing a test is to determine appropriate objectives.
     • Every curriculum should have appropriately framed assessable objectives, that is, objectives that are stated in terms of overt performance by students.
   • 2. Drawn up test specifications
     • Your specifications will simply comprise
       • A broad outline of the test
       • What skills you will test
       • What the items will look like
         • There are a limited number of modes of eliciting responses and of responding on tests of any kind
           • Oral / Written prompt
           • Oral / Written respond
       • One more test spec that needs to be included is a plan for scoring and assigning relative weight to each section and each item within.
   • 3. Devising test tasks
     • In revising your draft, you will want to ask yourself some important questions:
       • Are the directions to each section absolutely clear?
       • Is there an example item for each section?
       • Does each item measure a specified objective?
       • Is each item stated in clear, simple language?
       • Does each multiple-choice item have appropriate distractors?
       • Is the difficulty of each item appropriate for your students?
       • Is the language of each item sufficiently authentic?
     • Ideally, you would try out all your tests on students not in your class before actually administering the tests.
     • Make sure your test is neat and-uncluttered on the page.
     • If there is an audio component, make sure that the script is clear, that your voice and any other voices are clear, and that the audio equipment is in working order before starting the test.
   • 4. Designing multiple-choice test items
     • Weaknesses of multiple-choice items
       • The technique tests only recognition knowledge
       • Guessing may have a considerable effect on test scores.
       • The technique severely restricts what can be tested.
       • It is very difficult to write successful items.
       • Washback may be harmful.
       • Cheating may be facilitated.
     • Some definitions
       • Multiple-choice items are all receptive, or selective, response items in that the test-taker chooses from a set of responses rather than creating a response
       • Every multiple-choice item has a stem which presents a stimulus, and several options or alternatives to choose from
       • One of those options, the key is the correct response, while the others serve as distractors
     • Design each item to measure a specific objective
     • State both stem and options as simple and direct as possible
       • Get directly to the point
       • Avoid potentially confounding lexical items that could distract the student unnecessarily
       • Remove needless redundancy from your options
     • Make certain that the intended answer is clearly the only correct one
       • With only a minimum of context in each stem, a wide variety of responses may be perceived as correct.
     • Use item indices to accept, discard or revise items
       • The appropriate selection and arrangement of suitable multiple-choice items on a test can best be accomplished by measuring items against three indices:
         • Item facility (IF)
           • The extent to which an item is easy or difficult for the proposed group of test-takers
             • IF= (# of Ss answering the item correctly) / (Total # of Ss responding to that item)
               • Appropriate test items will generally have IFs that range between .15 and .85.
         • Item discrimination (ID)
           • The extent to which an item differentiates between high-and low-ability test-takers
             • ID= (high group #correct-low group #correct) / (1/2 x total of your two comparison groups)
               • High discriminating power would approach a perfect 1.0, and no discriminating power at all would be zero
         • Distractor efficiency
           • The distractors "lure" a sufficient number of test-takers, especially lower-ability ones
           • The responses are somewhat evenly distributed across all distractors