You are on page 1of 33

Module 10: Evaluation Methods

Module 10.3: Construction of Evaluation Tools

Objective
Overall objective is to understand about the construction of evaluation
tools, recognize the various steps in developing the evaluation devices and
apply these tools for the assessment of students in terms of knowledge,
skills and attitude.

Learning Outcome
At the end of this module the students will be able to:
• Explain the steps in test construction
• Describe the guidelines for construction
• Enlist the problems faced in construction
• Identify the examples of questionnaire

List of Topics • Introduction • Steps in test construction • Guidelines for construction • Problems faced in construction • Examples of questionnaire • Summary • References .

Introduction Educational evaluation is a complex. It requires many skills that are of equal importance with other elements of instructional process like how to device. to use and interpret tests. . understand the basic concepts of measurements and the construction of variety of tools. continuing process and an integral part of teaching and learning.

• This will be followed by the scheme of option. . 2. • The weightage for the different forms of the question to be included and for the difficult level to be maintained also are considered while finalising the design. • A test for a single unit may be generally of 40 to 45 minutes duration with maximum of 20 to 25 marks. • This should be decided in terms of the nature and scope of the unit or units involved in the testing. • If the test conducted at the end of the session the duration may be about 2 to 3 hours and the maximum marks may be 100.Steps in Test Construction 1. from of question and weightage of difficulty level are the most important factors to be considered while designing the test. Developing test design: • The objective. content. Planning an Achievement test: This step is concerned with determining the maximum time marks and nature of the test.

Understanding 20 40 3. Synthesis 5 10 6 Evaluation 2 4 Total 50 100 . No Objective Marks Percentage 1. Analysis 5 10 5. Weightage of instructional objectives: Sl. Knowledge 10 20 2.Design for a Unit a. Application 8 16 4.

b. Weightage to Content Areas Sl. No Sub -Unit Marks Percentage 1 I 15 30 2 II 10 20 3 III 10 20 4 IV 5 10 5 V 10 20 Total 50 100 .

c. Form Of No. Weightage to Form of Question Sl. Of Marks Percentage No Question Question 1 Objective .Type 25 25 50 2 Short Answer Type 5 15 30 3 Long Essay Type 1 10 20 Total 31 50 100 .

d. Weightage to Difficulty Level Sl. No Level Of Difficulty Marks Percentages 1 Easy 10 20 2 Average 30 60 3 Difficulty 10 20 Total 50 100 .

It depends on the total content area as well as its nature. • Regarding weightage of difficulty level sixty percent of items of average difficulty with twenty percent on either side. Whether more weightage has to be given to a particular form of question. • As in the case weightage of content. • Regarding the number of question under each from also there cannot be any uniform ally acceptable design.Guidelines for Preparing Test Design • The design should reflect the pre. • Modern trends is to avoid option. there is no final ruling regarding the number of subunits into which the content has to be divided .determined objectives envisaged at the time of instruction. .

wise and form. content. objective-wise. Normally a blueprint for a test is prepared as a three dimensional chart indicating the distribution of question. Prepare the Blue Print for Test The next step in the construction of an achievement test is preparing a blueprint according to design.wise.3. Content Hours Knowledg Application Critical Total e (30%) analysis (60%) (10%) Area 1 20 13 6 1 20 Area 2 20 12 6 2 20 Area 3 15 10 4 1 15 Area 4 25 13 9 3 25 Area 5 20 12 5 3 20 Total 60 30 10 100 .

instruction for answering each part have to written. 5.Steps in Test Construction 4. regarding the number of question to be set from each subunit their forms and scope. The maximum marks and time . . Before that . While setting the question and making the final selection care has to taken to maintain the weightage of difficulty level suggested by the design. Organization of the test: After finalising the items these has to be arranged according to the scheme of section as suggested in the design. Construction of the test: The blue print gives very definite idea. the preliminary details such as name of the examination.

Steps in Test Construction 6. Test Administration: Motivate the students to do their best. Preparation of the scheme for evaluation: One of the steps suggest for maintaining objectivity is to make the scoring strictly in accordance with predesigned scheme of evaluation. In this scoring key for objective type and point method can be used for short and essay type. follow the direction closely. record any significant events that might influence test scores and collect the test material promptly. 7. keep the time correctly. .

• Have range of complexity and difficulty. • The question should be structured and cover a specific topic. • Scoring Procedure • Set out the elements which should appear in the answer. • Score the answers of all the students to one question before scoring another.Essay Type Items • Avoid phrases like “discuss briefly state everything that you know” etc. • Question should be worded carefully. • Do not give too many or too lengthy questions. • Do not allow too many choices. • Use a point system of scoring based upon the elements. .

• Use positive statement in the stem as far as possible. . short notes on” etc. • Language should be precise. Don’t load the stem with irrelevant material. but avoid lengthy stem. • Make the stem simple and brief. underline it or write in capital letters. accurate in relation to the subject matter area Multiple Choice Items: • Have enough content in the stem with distractors as small as possible.Guidelines for Construction Short-answer type items: • Avoid phrases like “briefly. • Each item deals with important content. • Be sure that there is only one correct and best answer. so that it will not be overlooked. • As far as possible use action oriented precise verbs. • Keep the question as long as possible but make the answer short. if negative statement is to be used.

if number of items matched are more than 10. • Have four to five distractors only. • Matching type items • Make relatively several short matching items.Guidelines for Construction • Make distractors that resemble the correct answer i.e. • Provide a blank space or a separate answer sheet against each item for writing the number/letter of the correct answer • Arrange the distractors in such a way that there is no pattern evident about correct answers. • Stimulus and response columns should be preferably on same page. distractors should be plausible. • Avoid completing the stem with an or a which confuses or gives a clue to the learner. • Avoid using the distractors “all of the above” or “none of the above” as far as possible. . don’t make lists that are quite different.

columns A and B or list A and B rather than items of right side and items of left side. • Avoid using clues like all. • Make relatively several short matching items. nothing. always. columns A and B or list A and B rather than items of right side and items of left side. none.g. • Write clear and direct statements.Guidelines for Construction • Give some heading to both the columns e.g. • Avoid the use of negative statements particularly double negative. . • Avoid lengthy statements. may etc. avoid ambiguous statements. True and False Items: • Give single idea in the statement. should. sometimes. • Avoid “trick and catch” items. no. emphasize important points. if number of items matched are more than 10. usually. • Give some heading to both the columns e. • Stimulus and response columns should be preferably on same page.

Checklist : • Should relate directly to learning objectives. . • A copy of the completed checklist should be given to each student for review. • Multiple observations provide a more accurate assessment performance that does a single instance • Students should be evaluated in the natural setting or one as closely approximating reality as possible. • Need to be confined to performance areas that can be assessed sufficiently by examining positive and negative criteria only and when sufficient opportunity for observation exists.Guidelines for Construction • Have equal number of true and false items. which is followed by an individual session in which instructor and student discuss strengths and weakness of the performance and formulate a plan to improve the performance. • Determine the order of true and false by chance.

Guidelines for Construction Rating Scale: • Should relate directly to learning objectives • Need to be confined to performance areas that can be observed • Three to seven rating positions may need to be provided • Provision to omit items. . feedback and student participation in instrument development • Rating scales are vulnerable to errors resulting from the subjective judgement required of the observers. the instructors feel unqualified to judge • All raters to be oriented to the specific scale as well as to the process of rating in general • Consider evaluation setting.

The rater places the appropriate number beside each trait being rated.Guidelines for Construction Numerical Rating Scales: • These are setup so that the rater assigns a code number to each trait of the person being rated. Graphic Rating Scales: • It has descriptive phrases printed horizontally at various points. Code numbers are assigned to the descriptive phrases. The rater indicates the subjects standing with respect to each trait by placing a checkmark at appropriate point along the line. . lack or occurrence of each trait. level. arranging in order of the degree. The degree of each characteristic are arranged so that the rater can make as fine distinctions as the rater wishes. intensity or frequency with which they indicate possession.

• Behavior must be actually observed by the reporter. • All relevant factors in the situation must be given. • Limit each anecdote to a brief description of a single specific incident. the behaviour to assess.Guidelines for Construction Anecdotal Records: • Determine in advance. • The observer must make a definite judgment about behavior that is considered to be critical. Critical Incident Report: • Actual behavior must be reported rather than general trait. • Record enough of the situations to decrease subjectivity and record the incident as soon as possible after its occurrence. . then limit observation to those categories or qualities.

handwriting. • Provide negligible feedback. grammar and length of answer. neatness. • Subjectivity of scoring. • Contaminated by the extraneous factors like spelling. • Lack of objectivity.Problems Faced in Construction Essay type items: • Provide little useful feedback. • Covers only a limited field of knowledge in one test . • Requires excessive time to score. . • Difficulties in obtaining consistent judgement of performance.

• Cost aspects. • A logical error. .Problems Faced in Construction Short-answer type item: • Difficulty in construction of reliable items.is an error that occurs when a rater‘s general impression of a person influence the rating of individual characteristic. • Objective type test items. • Needs more stationary.. • Needs lots of time and effort in preparing the test..errors are indicated by a general tendency to rate all individual at approximately the same position on the scale. • Provides little or no opportunity for measurement of students ability to organize and to express thoughts. Rating scale: • Personal bias. • The halo effect.results when two characteristics are rated as more alike than they actually because of the raster's belief concerning their relationship.

Checklist • It has limited application • Determines only presence or absence of an action • Provides no means of judging the extent to which a behaviour is possessed by the student • Anecdotal Records • Subjectivity • Lack of standardization • Difficulty in scoring • Time consuming • Limited application .

Short-answer type items: Example: • Poor: Give your best definition of health • Better: What is the definition of health according to W.Examples of Questionnaire Essay type items: Example: • Poor: Discuss immunity – 10 • Better: Define immunity. Differentiate between passive and active immunity.H. List general precautions to be taken while giving immunization -1+3+3+3=10. Describes hazards of immunization.O .

Examples of Questionnaire Multiple Choice Items: Example The collection of fluid in the pleural space is known as: • Pleuritis • Pleural Effusion • Pleural Tapping • Pleurodesis Matching type items: Example: Column A Column B • Long bone Skull • Small bone Tibia • Flat bone Femur Stapes .

. .True / False .Examples of Questionnaire True and False Items: Example: Note: Tick the correct response: • Pancreas is an endocrine gland.True / False • The largest gland in the body is pituitary.

Obtains necessary equipment Comments: Student’s Signature Instructor’s Signature . Explains procedure to the patient 2. Checks appearance of present dressing 3. Washes hands 4.Examples of Questionnaire Checklist: Example: • Change of Dressing • Instruction: Tick the appropriate column Behavior Yes No 1.

Checks appearance of present dressing 3. Explains procedure to the patient 2.Examples of Questionnaire Rating Scale: Example: • Change of Dressing • Instruction: Tick the appropriate column Behavior Excellent-5 Very Fair-3 Poor-2 Very Good-4 Poor- 1 1. Obtains necessary equipment Comments: Student’s Signature Instructor’s Signature . Washes hands 4.

Examples of Questionnaire Anecdotal Record: Example: • Name of the Student: • Year: Date & Time: • Name of the Observer: • Setting: • Incident: • Interpretation: • Recommendations: Observer’s Signature .

Behavior and consequences Observer’s Signature .Examples of Questionnaire Critical Incident Report: Example – Empathy: • Positive Behaviors Negative Behaviors • To be encouraged Needing improvement • Uses patient’s name in all Addresses patient by a general communications term such as “Grandpa” Date Behavior (Number What happened: Record in items) antecedents.

. understand the basic concepts of measurements and the construction of variety of tools such as achievement tests and performance appraisal.Summary Educational evaluation requires many skills that include how to device. to use and interpret tests.