You are on page 1of 33

Module 10: Evaluation Methods

Module 10.3: Construction of Evaluation Tools

Objective
Overall objective is to understand about the construction of evaluation
tools, recognize the various steps in developing the evaluation devices and
apply these tools for the assessment of students in terms of knowledge,
skills and attitude.

Learning Outcome
At the end of this module the students will be able to:
• Explain the steps in test construction
• Describe the guidelines for construction
• Enlist the problems faced in construction
• Identify the examples of questionnaire

List of Topics • Introduction • Steps in test construction • Guidelines for construction • Problems faced in construction • Examples of questionnaire • Summary • References .

understand the basic concepts of measurements and the construction of variety of tools. to use and interpret tests. continuing process and an integral part of teaching and learning.Introduction Educational evaluation is a complex. It requires many skills that are of equal importance with other elements of instructional process like how to device. .

2. • The weightage for the different forms of the question to be included and for the difficult level to be maintained also are considered while finalising the design. from of question and weightage of difficulty level are the most important factors to be considered while designing the test. content. Developing test design: • The objective. . • This will be followed by the scheme of option. • A test for a single unit may be generally of 40 to 45 minutes duration with maximum of 20 to 25 marks. Planning an Achievement test: This step is concerned with determining the maximum time marks and nature of the test. • If the test conducted at the end of the session the duration may be about 2 to 3 hours and the maximum marks may be 100. • This should be decided in terms of the nature and scope of the unit or units involved in the testing.Steps in Test Construction 1.

Synthesis 5 10 6 Evaluation 2 4 Total 50 100 . Understanding 20 40 3. Weightage of instructional objectives: Sl. Knowledge 10 20 2. Application 8 16 4.Design for a Unit a. Analysis 5 10 5. No Objective Marks Percentage 1.

b. No Sub -Unit Marks Percentage 1 I 15 30 2 II 10 20 3 III 10 20 4 IV 5 10 5 V 10 20 Total 50 100 . Weightage to Content Areas Sl.

c. Form Of No. Of Marks Percentage No Question Question 1 Objective .Type 25 25 50 2 Short Answer Type 5 15 30 3 Long Essay Type 1 10 20 Total 31 50 100 . Weightage to Form of Question Sl.

d. No Level Of Difficulty Marks Percentages 1 Easy 10 20 2 Average 30 60 3 Difficulty 10 20 Total 50 100 . Weightage to Difficulty Level Sl.

there is no final ruling regarding the number of subunits into which the content has to be divided . • Modern trends is to avoid option. . Whether more weightage has to be given to a particular form of question.Guidelines for Preparing Test Design • The design should reflect the pre.determined objectives envisaged at the time of instruction. It depends on the total content area as well as its nature. • Regarding the number of question under each from also there cannot be any uniform ally acceptable design. • Regarding weightage of difficulty level sixty percent of items of average difficulty with twenty percent on either side. • As in the case weightage of content.

wise and form. content. Normally a blueprint for a test is prepared as a three dimensional chart indicating the distribution of question.wise. objective-wise.3. Content Hours Knowledg Application Critical Total e (30%) analysis (60%) (10%) Area 1 20 13 6 1 20 Area 2 20 12 6 2 20 Area 3 15 10 4 1 15 Area 4 25 13 9 3 25 Area 5 20 12 5 3 20 Total 60 30 10 100 . Prepare the Blue Print for Test The next step in the construction of an achievement test is preparing a blueprint according to design.

regarding the number of question to be set from each subunit their forms and scope. Construction of the test: The blue print gives very definite idea.Steps in Test Construction 4. Organization of the test: After finalising the items these has to be arranged according to the scheme of section as suggested in the design. . Before that . the preliminary details such as name of the examination. instruction for answering each part have to written. While setting the question and making the final selection care has to taken to maintain the weightage of difficulty level suggested by the design. 5. The maximum marks and time .

follow the direction closely.Steps in Test Construction 6. . 7. Test Administration: Motivate the students to do their best. In this scoring key for objective type and point method can be used for short and essay type. Preparation of the scheme for evaluation: One of the steps suggest for maintaining objectivity is to make the scoring strictly in accordance with predesigned scheme of evaluation. record any significant events that might influence test scores and collect the test material promptly. keep the time correctly.

• Score the answers of all the students to one question before scoring another. • Scoring Procedure • Set out the elements which should appear in the answer.Essay Type Items • Avoid phrases like “discuss briefly state everything that you know” etc. • Use a point system of scoring based upon the elements. • Question should be worded carefully. • Do not give too many or too lengthy questions. • Have range of complexity and difficulty. • The question should be structured and cover a specific topic. . • Do not allow too many choices.

but avoid lengthy stem. so that it will not be overlooked. . underline it or write in capital letters. • Each item deals with important content. Don’t load the stem with irrelevant material. • Be sure that there is only one correct and best answer. • As far as possible use action oriented precise verbs. if negative statement is to be used. • Use positive statement in the stem as far as possible. • Make the stem simple and brief. • Language should be precise. accurate in relation to the subject matter area Multiple Choice Items: • Have enough content in the stem with distractors as small as possible. short notes on” etc. • Keep the question as long as possible but make the answer short.Guidelines for Construction Short-answer type items: • Avoid phrases like “briefly.

Guidelines for Construction • Make distractors that resemble the correct answer i. • Avoid using the distractors “all of the above” or “none of the above” as far as possible.e. . • Stimulus and response columns should be preferably on same page. distractors should be plausible. • Have four to five distractors only. • Avoid completing the stem with an or a which confuses or gives a clue to the learner. • Provide a blank space or a separate answer sheet against each item for writing the number/letter of the correct answer • Arrange the distractors in such a way that there is no pattern evident about correct answers. don’t make lists that are quite different. if number of items matched are more than 10. • Matching type items • Make relatively several short matching items.

should. • Avoid the use of negative statements particularly double negative. • Write clear and direct statements. emphasize important points. • Stimulus and response columns should be preferably on same page. no. always.g. sometimes. • Give some heading to both the columns e. columns A and B or list A and B rather than items of right side and items of left side. True and False Items: • Give single idea in the statement. • Avoid using clues like all. avoid ambiguous statements. . may etc. • Avoid lengthy statements. • Avoid “trick and catch” items. • Make relatively several short matching items.g. usually. nothing.Guidelines for Construction • Give some heading to both the columns e. columns A and B or list A and B rather than items of right side and items of left side. none. if number of items matched are more than 10.

which is followed by an individual session in which instructor and student discuss strengths and weakness of the performance and formulate a plan to improve the performance. Checklist : • Should relate directly to learning objectives. .Guidelines for Construction • Have equal number of true and false items. • A copy of the completed checklist should be given to each student for review. • Need to be confined to performance areas that can be assessed sufficiently by examining positive and negative criteria only and when sufficient opportunity for observation exists. • Multiple observations provide a more accurate assessment performance that does a single instance • Students should be evaluated in the natural setting or one as closely approximating reality as possible. • Determine the order of true and false by chance.

the instructors feel unqualified to judge • All raters to be oriented to the specific scale as well as to the process of rating in general • Consider evaluation setting. feedback and student participation in instrument development • Rating scales are vulnerable to errors resulting from the subjective judgement required of the observers.Guidelines for Construction Rating Scale: • Should relate directly to learning objectives • Need to be confined to performance areas that can be observed • Three to seven rating positions may need to be provided • Provision to omit items. .

intensity or frequency with which they indicate possession. Graphic Rating Scales: • It has descriptive phrases printed horizontally at various points. level. lack or occurrence of each trait. The rater places the appropriate number beside each trait being rated. arranging in order of the degree. . The degree of each characteristic are arranged so that the rater can make as fine distinctions as the rater wishes. The rater indicates the subjects standing with respect to each trait by placing a checkmark at appropriate point along the line.Guidelines for Construction Numerical Rating Scales: • These are setup so that the rater assigns a code number to each trait of the person being rated. Code numbers are assigned to the descriptive phrases.

Guidelines for Construction Anecdotal Records: • Determine in advance. then limit observation to those categories or qualities. • The observer must make a definite judgment about behavior that is considered to be critical. • All relevant factors in the situation must be given. • Limit each anecdote to a brief description of a single specific incident. • Behavior must be actually observed by the reporter. Critical Incident Report: • Actual behavior must be reported rather than general trait. the behaviour to assess. • Record enough of the situations to decrease subjectivity and record the incident as soon as possible after its occurrence. .

• Difficulties in obtaining consistent judgement of performance. grammar and length of answer. • Subjectivity of scoring. • Covers only a limited field of knowledge in one test .Problems Faced in Construction Essay type items: • Provide little useful feedback. • Lack of objectivity. • Provide negligible feedback. • Contaminated by the extraneous factors like spelling. handwriting. . • Requires excessive time to score. neatness.

• Needs lots of time and effort in preparing the test.results when two characteristics are rated as more alike than they actually because of the raster's belief concerning their relationship.Problems Faced in Construction Short-answer type item: • Difficulty in construction of reliable items.. • Provides little or no opportunity for measurement of students ability to organize and to express thoughts. • Cost aspects..errors are indicated by a general tendency to rate all individual at approximately the same position on the scale. • The halo effect. • A logical error. Rating scale: • Personal bias.is an error that occurs when a rater‘s general impression of a person influence the rating of individual characteristic. . • Needs more stationary. • Objective type test items.

Checklist • It has limited application • Determines only presence or absence of an action • Provides no means of judging the extent to which a behaviour is possessed by the student • Anecdotal Records • Subjectivity • Lack of standardization • Difficulty in scoring • Time consuming • Limited application .

Differentiate between passive and active immunity. Describes hazards of immunization. List general precautions to be taken while giving immunization -1+3+3+3=10.H. Short-answer type items: Example: • Poor: Give your best definition of health • Better: What is the definition of health according to W.O .Examples of Questionnaire Essay type items: Example: • Poor: Discuss immunity – 10 • Better: Define immunity.

Examples of Questionnaire Multiple Choice Items: Example The collection of fluid in the pleural space is known as: • Pleuritis • Pleural Effusion • Pleural Tapping • Pleurodesis Matching type items: Example: Column A Column B • Long bone Skull • Small bone Tibia • Flat bone Femur Stapes .

.True / False • The largest gland in the body is pituitary.Examples of Questionnaire True and False Items: Example: Note: Tick the correct response: • Pancreas is an endocrine gland. .True / False .

Checks appearance of present dressing 3.Examples of Questionnaire Checklist: Example: • Change of Dressing • Instruction: Tick the appropriate column Behavior Yes No 1. Washes hands 4. Obtains necessary equipment Comments: Student’s Signature Instructor’s Signature . Explains procedure to the patient 2.

Obtains necessary equipment Comments: Student’s Signature Instructor’s Signature .Examples of Questionnaire Rating Scale: Example: • Change of Dressing • Instruction: Tick the appropriate column Behavior Excellent-5 Very Fair-3 Poor-2 Very Good-4 Poor- 1 1. Washes hands 4. Checks appearance of present dressing 3. Explains procedure to the patient 2.

Examples of Questionnaire Anecdotal Record: Example: • Name of the Student: • Year: Date & Time: • Name of the Observer: • Setting: • Incident: • Interpretation: • Recommendations: Observer’s Signature .

Examples of Questionnaire Critical Incident Report: Example – Empathy: • Positive Behaviors Negative Behaviors • To be encouraged Needing improvement • Uses patient’s name in all Addresses patient by a general communications term such as “Grandpa” Date Behavior (Number What happened: Record in items) antecedents. Behavior and consequences Observer’s Signature .

Summary Educational evaluation requires many skills that include how to device. to use and interpret tests. understand the basic concepts of measurements and the construction of variety of tools such as achievement tests and performance appraisal. .