Introduction to Psychological Testing: Administration, Types, and Reliability, Study notes of Psychology

Download Introduction to Psychological Testing: Administration, Types, and Reliability and more Study notes Psychology in PDF only on Docsity!

Discipline of Psychology

School of Social Sciences

Indira Gandhi National Open University

BA PSYCHOLOGY

BPCC-105 Psychological Research

Guidelines on Practicum

(2 Credits)

1.0 INTRODUCTION

The BA Programme offered by the University will introduce you to the laboratory component in the courses offered by the Discipline of Psychology. To complete the course BPCC-105 General Psychology, you must complete one Tutor Marked Assignment, attend practicum sessions (sessions are mandatory), write the term end examination for theory and practical components separately.

Laboratory component is of 2 Credits. In this, you will learn how to administer tests and conduct experiments in a controlled condition, that is in a laboratory setup at your Study Centre. The test and experiments will be carried out on a human participant and you will be the experimenter or test administrator. At times, this situation may also be reversed as during the course you will carry out practicals amongst your classmates. The tests and experiments are related to various topics that you have read or about to read in the theory component of BPCC-105. While doing lab work, you will be directly supervised by your academic counsellor. You will follow a standard procedure for administration, scoring and interpretation of the result and findings. You will also learn about ethical issues as they apply in laboratory work.

As you must have learned that psychology is a science of human behaviour. It aims at understanding various phenomena of human mind and behaviour. The purpose of understanding is description, explanation, prediction and control of behaviour, and the application of various techniques for the betterment of life. But how these goals are achieved? These goals are achieved through scientific research as a first step and then the research results are applied in real life settings. Psychologists have developed certain methods and procedures over the years to understand behaviour. These methods are studied in the branches of psychology especially devoted to the experiments, methods and research in psychology. The first such branch is Psychometrics which literally means measurement of psychological variables. It includes everything related to the measurement of psychological constructs. The more specific branches are Experimental Psychology and Psychological Testing. Experimental psychology, as the name suggests, is focused more on experimentation in psychology. Psychological testing is more focused on psychological tests developed to study various mental abilities, personality traits, and other related aspects of behaviours. Psychological tests are scientifically designed instruments for psychological measurement of constructs.

Experiments in psychology make use of different instruments/apparatus to study different cognitive, affective or behavioural aspects such as sensation, perception, attention, memory, learning and other such processes. They mainly focus on studying the cause and effect relationship between independent and dependent variable. Experiments mainly focus on studying cause and effect relationship between independent and dependent variables. The participant has to be active during the experiment as she/he not only performs on a task but is also vigilant in observing one’s own mental activities while performing the task and reporting it to the experimenter. This is also known as ‘introspective report’.

On the other hand, psychological tests can be classified into various types on the basis of administration, time limit, nature of items. This will be described in detail in later section. Psychological tests can be used in different settings like schools, hospitals, organisations and welfare organisations. They can also be used for research purpose. They are not only used to diagnose mental disorders,

• Independent and dependent variables
• Control and experimental conditions
• Administration
• Scoring

4. After the introduction of the experiment, demonstrate to the learners how to conduct the experiment.

5. The demonstration will include the following:

• Preparation for the experiment, for instance, keeping the material (instrument/ apparatus, stimulus words/ list, stopwatch) ready.
• Establishing rapport with the participant, making him/her feel comfortable
• Explaining the experiment (procedure, time limit, precautions)
• Taking informed consent for undergoing the experiment and informing the participant that the experiment findings will remain confidential.
• Taking permission to record the session, wherever applicable.
• Explaining the instructions to the learners.
• Clearing all doubts in the mind of the participant about the experiment.
• The experiment is conducted on the participant.

6. Explain the scoring procedure to the learners.

7. Explain how to discuss the data.

8. Ask learners to conduct the experiment on each other in pairs and monitor the same.

9. The learners will now conduct the experiment and do the scoring.

10. The learners will have to write a report of the experiment in the practicum notebook which will be evaluated by the academic counsellors.

For Psychological Tests

1. Go though the manual of the test thoroughly.

2. Explain the test in detail to the learners in the class.

3. Introduce the test in terms of:

• History of the Test
• Author
• Development of the test
• Features of the test (e.g. no. of items, dimensions, reliability, validity)
• Administration
• Scoring
• Interpretation

4. After the introduction of the test, demonstrate to the learners how to administer the test.

5. The demonstration of administration will include the following:

a) Preparation for the test, for instance, keeping the test material (test booklet, answer sheet, stopwatch) ready.

b) Establishing rapport with the participant, making him/her feel comfortable

c) Explaining the test ( procedure, time limit, precautions)

d) Taking informed consent for undergoing the test and informing the participant that the test findings will remain confidential.

e) Taking permission to record the session, wherever applicable.

f) Reading the instructions for test administration from the manual and showing it to learners as to from where they have to read the instructions.

g) Clearing all doubts in the mind of the subject about the test administration.

h) The participant takes the test.

i) Taking the answer sheet from the participant after completion of the test.

6. Explain the scoring procedure (as given in the manual) to the learners.

7. Explain how to interpret the data.

8. Ask learners to administer the test on each other in pairs and monitor the same.

9. The learners will then administer, score and the results.

10. The learners will have to write a report of the experiment in the practicum notebook which will be evaluated by the academic counsellors.

4.0 IMPORTANT INFORMATION FOR THE

LEARNER

1. Practicum Counselling Sessions: You are advised to contact your Study Centre with regard to schedule of counseling sessions for the Practicum Course. You may also visit the website of your Regional Centre, where the schedule of sessions is displayed. Unlike counseling sessions of other courses in the programme, the sessions organized for this course are mandatory. Thus, you should attend all the sessions and, in these sessions, your academic counselor will teach you how to conduct and administer experiments and tests. You will clarify all your doubts arising while pursuing this course from your academic counselor. There is weightage given to attendance in evaluation also (refer to evaluation scheme, under ‘Evaluation’). The number of counselling sessions allocated for practicum course are 02 sessions ( session is of 03 Hours duration).

Precautions: Precautions, if any, to be considered while administration of the test/ experiment are mentioned under this sub-heading.

Introspective Report: After completion of the test/ experiment by the participant, an introspective report is to be taken, that is, the participant’s feeling and constraints faced by him/her while undergoing the test/ experiment is mentioned under this sub-heading in first person.

Scoring and Interpretation: After the participant completes the test, the answer sheet is to be scored with the help of the scoring key and the data is to be interpreted with the help of the norms given in the manual. The scores can then be mentioned and interpreted under this heading. For experiments, the findings are to be analysed and mentioned here.

Discussion: You have to discuss the result based on the interpretation. It may be further analysed in the light of the introspective report. In case of experiments, the results may be supported by existing studies conducted in the field.

Conclusion: Under this heading, you have to conclude the findings of the test or the experiment.

References

The books, websites and the manual referred to by the learner are mentioned in American Psychological Association (APA) format.

References (APA Style)

References have to be written in APA format. These should be alphabetically listed.

Books

Anastasi, A. (1968). Psychological Testing. London: MacMillan Company.

Journal Article

Dennision, B. (1984). Bringing corporate culture to the bottomline. Organizational Dynamics , 13,22-24.

Book Chapter

Khan, A.W. (2005). Distance Education for Development. In: Garg, S. et.al. (Eds.) Open and Distance Education in Global Environment: Opportunities for collaboration. New Delhi: Viva Books.

Websites

http://www.mcb.co.uk/apmfirum (accessed on 2.3.2011)

4. You will keep the photocopy of the practicum notebook before submitting it at the study centre. Acknowledgement (Appendix 3) may also be taken while submitting the notebook.

5.0 EVALUATION

1. Term End Examination (TEE) Form and Exam Fee: You will have to deposit separate exam fee for the TEE of Practicum Course. The exam fee is Rs. 200/-. Kindly check the latest fee amount applicable from www.ignou. ac.in (it is subject to revision).

2. TEE: Total marks for Practicum Examination will be 100 marks (Internal assessment is 50 marks and External assessment is 50 Marks). Internal assessment refers to actual conduction of Practicals and reporting them in the practical notebook in the prescribed format. External Assessment refers to conduction/ administration of Practicum on exam day and appearing in Viva- Voce. TEE for the course will be organized at the Study Cente.

3. Conduction of TEE: You will conduct the practicals and submit practicum notebook to the academic counsellor and get it corrected before the Practicum Term End Examination. You will bring notebook at the time of examination. The examination will be conducted at the respective study centres. The duration of the examination will be of 3 hours. During the examination, you will conduct the practical and submit the answer sheet. The practical will be allotted to you by the way of lottery system. You will then collect the test material and start conducting the practical. You need to bring one participant on the day of exam, on whom the test/ experiment will be conducted. Once you finish conducting the practical, write the findings in the answer sheet. This will be followed by viva-voce. The participants may leave after the conduction of practical is over.

The practicum answer sheets will be corrected by the external examiner and the viva-voce will also be conducted by the external examiner.

Dates for Term End Examination of Practicum

Admission Cycle Date Range for TEE of BPCC- June TEE 1 July to 14 August December TEE 1 January to 15 February

The dates for TEE of BPCC 105 will not appear in the datesheet provided by SED, IGNOU. For this, please contact your respective study centres.

4. Passing Marks for Practicum: Minimum passing marks in the course is

35. There is no re-evaluation in TEE of Practicum.

5. Scheme of Evaluation: The following evaluation methodology will be followed for TEE:

INTERNAL Marks EXTERNAL Marks Attendance 05 Conduction 20 Conduction of test/ experiment

30 Evaluation of answer script

Practicum Notebook 15 Viva-Voce 20 TOTAL 50 TOTAL 50

External evaluation will carry 30% weightage and internal evaluation will carry 70% weightage.

concept underlying psychological testing. Francis Galton (1822-1911) was the first scientist to undertake systematic and statistical investigation of individual differences. He demonstrated that individual differences exist in human sensory and motor functioning, such as reaction time, visual acuity and physical strength. James McKeen Cattell extended Galton's work. Cattell also coined the term mental test in 1890. Before Galton, there were other important works in the history of psychology, but difference in human abilities was not focused upon until the work of Galton. Weber (1795-1878) experimented on weight discrimination, vision, hearing and the two point-point threshold. Fechner (1801-07) contributed significantly in the understanding of relation of mental processes to physical phenomena (for example, how the change in the intensity of sound will affect the auditory perception). Wilhelm Wundt (1832-1920) who established the first psychological laboratory in 1879 in Leipzig, Germany, was working on the measurement of mental processes years before. In 1862 he experimented with thought meter to measure the speed of thought.

Thus, psychological testing developed from two lines of enquiry:

• One based on the measurement of individual differences by Darwin, Galton and Cattell
• The other, based on the work of the German psychophysicists – Weber, Fechner and Wundt.

Modern psychological tests were constructed in response to the needs of classifying the mentally and emotionally handicapped. The Seguin Form Board Test (1866) was developed by O. Edward Seguin (1812-1880) to educate and evaluate the mentally disabled. An important breakthrough in the creation of modern tests came at the turn of the twentieth century with the publication of intelligence test by Alfred Binet and T. Simon in 1905. With the time more developments were seen in the field of testing with a range of testing devices like personality tests, performance tests, aptitude tests, interest inventories, educational achievement and multifactor tests etc.

As a learner of psychology, are suggested to read more on the development of psychological testing–how it was started, what were the landmarks in the history of psychological testing. Here, we are giving a brief overview of the early developments:

Table 1: A Summary of Early Landmarks in the History of Testing

2200 B.C. Chinese begin civil service examination

A.D. 1862 Wilhelm Wundt uses a calibrated pendulum to measure the ‘speed of thought’

1884 Francis Galton administers the first test battery to thousands of citizens at the International Health Exhibit

1890 James Mckeen Cattell uses the term mental test in announcing the agenda for his Galtonian test battery

1905 Binet and Simon constructed the first intelligence test

1914 Stern introduces the concept of IQ or intelligence quotient- the mental age divided by chronological age

1916 Lewis Terman revises the Binet-Simon scales, publishes the Stanford-Binet. Revisions appear in 1937, 1960, and

1917 Robert Yerkes spearheads the development of the Army Alpha and Beta examinations used for testing World War I recruits

1917 Robert Woodworth develops the Personal Data Sheet, the first personality Test

1920 Rorschach Inkblot test published

1921 Psychological Corporation- the first major test publisher- founded by Cattell, Thorndike and Woodworth

1927 The first edition of the Strong Vocational Interest Blank published

1939 Wechsler-Bellevue Intelligence Scale published. Revisions published in 1955, 1981 and 1997.

1942 Minnesota Multiphasic Personality Inventory published

1949 Wechsler Intelligence Scale for Children published. Revisions published in 1974, 1991

Adapted from: Psychological Testing by R J Gregory 2004: 51

Types of Tests

Tests can be categorized on the basis of administration, the behaviour they measure, mode of response and on the basis of the structure of the test. On the basis of test administration, there are two types of tests: Individual tests and Group tests. The tests which can be given to one person at a time are known as individual tests. Group tests can be administered to more than one person at a time by a single examiner. If we categorize tests according to the type they measure, these tests are put under a broad category: ability tests. Ability test measure skills in terms of speed, accuracy or both. For example, in the test of mathematical ability, the more problems you solve accurately within the time limit, the more will be your score. Ability is a broad term which encompasses aptitude tests, intelligence tests and achievement tests. Achievement tests measure previous learning, like how much has been learnt in English in one year by six grade learners can be measured by term end examination. Aptitude tests measure potential for acquiring a specific skill, for example how much can be learnt by a person in music if s/he is given specific training is the person's musical aptitude. Intelligence tests measure a person’s general potential to solve problems, to adapt to changing circumstances and to benefit from experience. All the above three types of tests are inter related; sometimes these tests are included under the tests of human ability. Personality tests measure traits, temperaments and dispositions. Personality tests can be categorized on the basis of the structure of the test. Whether the test is clearly structured like a questionnaire or it is semi- structured or uses unstructured stimulus. Unstructured or semi- structured tests are commonly known as projective tests. The test stimulus in projective tests is ambiguous, like ink-blots in Rorschach inkblot test.

same test. The investigators sometimes develop alternate form of the test which has same content and covers the same range and level of difficulty. Both forms of the test are administered on the same group and the test scores are correlated to find out the reliability of the test. It is also called equivalent or parallel forms reliability. Split half reliability is estimated by correlating the scores obtained from equivalent halves of a test administered once to a representative group. In the item total correlations the investigator calculates the correlation between scores on each item of the test and the total score on the test. Inter-rater reliability is calculated when the measured behaviour is rated by observers. Ratings of different observers are correlated to measure the correlation coefficient. The Table 1.2 gives a brief overview of the methods of reliability.

Method No. of Forms

No. of Sessions

Sources of error variance Test-Retest 1 2 Changes over time Alternate forms (immediate) 2 1 Item sampling Alternate Forms (delayed) 2 2 Item sampling Changes over time Split Half 1 1 Item sampling Nature of split Item total 1 1 Item sampling Test heterogeneity Interscorer 1 1 Scorer differences

Source: Robert J Gregory (2004: 111)

There are different statistical methods used to assess reliability: Cronbach’s alpha, Kuder- Richardson (KR-20), Pearson correlation and Guttman’s coefficient and factor analysis. Readers can read more about reliability and validity on http:// psychology.wadsworth.com/book/ gravetterwallnau5e/index.html)

What should be the accepted level of test reliability or when do we say that the particular test should be used as it has good reliability index? There is no such fixed criterion for a good psychological test. Some authors suggest that reliability should be at least .95. But in the words of Guilford and Fruchter (1978), ‘There has been some consensus that to be a very accurate measure of individual differences in some characteristics, the reliability should be above .90. The truth is, however, that many standard tests with reliabilities as low as .70 prove to be very useful. And tests with reliabilities lower than that can be useful in research.

b) Validity

A valid test is one that measures what it is supposed to measure. ‘A test is valid to the extent that inferences made from it are appropriate, meaningful and useful.’ (Standards for Educational and Psychological Testing, 1999).The first essential quality of a valid test is that it should be highly reliable. If a test yields inconsistent results, (i.e. it is not reliable) it can not be correlated with any criterion (some behaviour or personal accomplishment etc.). But high reliability does not guarantee high validity of the test. The relation between reliability and validity can be discussed with the following example: ‘Sir Francis Galton’s sensory and motor measures could never have been valid if they had not been reliable….. Yet even though some of Galton’s measures turned out to be very reliable, later evidence showed that they were not valid measures of intelligence. The measures yielded similar scores time after time, but those scores were poorly correlated

with validity criteria such as school grades and teacher ratings of intelligence.’ (Morgan, King, Weisz and Schoplar,1997: 520).

There are several different types of validity. One or more methods can be selected depending on the needs of the measure. Different ways of measuring validity have been grouped into three categories: Content validity, Criterion-related validity, Construct validity. Content validity is ‘an estimate of validity of a testing instrument based on a detailed examination of the contents of the test items; contents here means the actual constituent material of the test item’ (Reber and Reber 2001: 781). Content validity depends on the judgment of experts on the relevance of the items used in the instrument. Criterion related validity is assessed by determining the relationship between test scores and some independent criterion. Gregory has included two different approaches under criterion related validity- concurrent validity and predictive validity (2004: 124):

• In concurrent validity, the criterion measures are obtained at approximately the same time as the test scores. For example, the current psychiatric diagnosis of a patient would be an appropriate measure to provide validation evidence for a paper-and-pencil psychodiagnostic test.
• In predictive validity, the criterion measures are obtained in future, usually months or years after the test scores are obtained. For example, a college entrance exam that is accurate in predicting the subsequent grade point average of examinees would possess criterion related validity.

Construct validity is ‘a set of procedures for evaluating the validity of a testing instrument based on the determination of the degree to which the test items capture the hypothetical quality or trait (i.e. the construct) it was designed to measure. For example, if a test is supposed to provide a measure of intelligence one should ask: what traits or qualities (or constructs) actually characterize intelligence? Do the test items actually tap such constructs?’ (Reber and Reber 2001: 781). Face validity is dependent on whether the test looks valid to test users, examiners and examinees. Gregory comments that face validity is important for social acceptability of the test but is irrelevant for psychometric purposes.

c) Norms

Suppose someone gets 50 marks on an intelligence test. This score has no meaning in itself. In psychological testing the scores obtained first from a test are called raw scores. These scores are simply overall score of the performance on the test, like the number of problems solved in an intelligence test. These initial scores are converted to some form of standard scores based on a norm group. ‘A norm group consists of a sample of examinees who are representatives of the population for whom the test is intended’ (Gregory 2004: 81). For example, if a test is designed to study the value system of twelfth graders, the test will be given to large number of such age group (rural- urban, rich – middle class – poor etc.) to determine the distribution of raw scores. On the basis of collection of scores, the test developer will provide derived scores. These scores are known as norms. Norms can be in the form of percentiles ranks, stanines, stens, age norm, grade norms or standard scores.

A percentile expresses the percentage of scores in a sample that fall below it. A score at 50th^ percentile indicates that 50% of the scores fall below it. Percentile should not be confused with percent correct. Percentile is a comparative score. It tells where your scores places you in particular sample (norm group) whereas

• The physical conditions (testing room) should be suitable for the test. The conditions such as illumination, temperature and humidity should be taken into consideration before the test. The testing environment should be pleasant, quiet and well illuminated with proper writing desk (in case of a test where answer sheet is required to be filled up).
• Establishing rapport is the first thing that examiners are advised to do when giving a test to an individual or a group. ‘Rapport is a comfortable, relaxed, unconstrained, mutually accepting interaction between persons’ (Reber and Reber 2001: 597), especially between an examinee and an examiner. It is essential requirement to motivate examinees to cooperate during testing. It is more important in individual testing and particularly when examinees are children. Failure in establishing rapport may cause anxiety, hostility, and uncooperative behaviour in examinees.
• The scoring of the test should follow the pattern as specified in the test manual. If scoring is not numerical, the method of interpretation should also follow the guidelines as given in the test manual.

Thus, a psychological test is a standardized instrument in the sense that it provides well defined procedure and instructions, the items used in the test are reliable and valid and the test depicts scores in terms of standardized scores. At present, when we have access to computer assisted test administration and scoring, the accuracy and precision in administration will require proper training and practice of the examiner both on technical and human grounds.

Report Writing

After administration of a psychological test, the findings are to be presented in the form of a report. Report should be written clearly. The report should be properly divided into sections and subsections and the findings should be tabulated wherever required.

The report should be written in passive voice. For example, instead of writing ‘I gave the test booklet to the examinee’, one should write, ‘the test booklet was given to the examinee’. The report should be written in a standard format.

Being Qualified and Trained in Psychological Testing

There are two aspects of being trained in psychological testing:

• Having technical and theoretical knowledge of psychological testing and itsapplications
• Having skills necessary for the application of psychological testing, for example communication skills, being a good observer and empathetic listener, etc.

The above aspects are briefly discussed as follows:

a) Technical and Theoretical Knowledge

Some basic components of this knowledge are:

i) Knowledge of test construction

ii) Efficiency in application

iii) Knowledge and efficiency in scoring and interpretation

i) Knowledge of test construction

Today testing is required in every field: schools, industries, selection agencies, hospitals, special education centers, rehabilitation centers and various other organisations. A psychologist may face the task of choosing a test from the available tests or developing a test as the situation demands. In both the situations knowledge of test construction is mandatory. If one needs to select a test, one should have the knowledge of basics of test construction. How the test is developed? Whether it has proper norms or it is standardized, what is the method of scoring, etc. All this information requires technical knowledge about the test construction process. Otherwise, decision of choosing will be fraught with biased assumptions. The theoretical knowledge pertains not only to the selection of test, but also to the construction of tests. One might face such a situation when no test is available, or the available test is outdated, or not suitable culturally. Suppose you are required to make an index of happiness of people in your country or your state or city. How to prepare such an index? You come to know that one such procedure is available in some other country. But the definition of happiness may differ from one country to the other. At one place, it may be family which is primary source of happiness to persons, but at the other, it may be secure future and material prosperity. Thus, one might decide to prepare a questionnaire to study the level of happiness.

ii) Efficiency in application

Which measure one should choose if one is find out whether a child has learning disability or not. One might need a range of procedures – paper pencil test (test of learning and intelligence), observations, interviews with the child, parents and teachers. Which test one should choose - verbal or non-verbal, some qualitative approach or quantitative or both, whether the test is fit for the cultural background. These decisions need not only theoretical knowledge but insight on the part of the investigator which comes with knowledge, practice and experience.

iii) Efficiency in Scoring and Interpretation

Scoring procedures in testing are developed through rigorous statistical procedures. While using psychological tests, one is required to have sound knowledge of statistical principles applied in psychological testing. How the reliability and the validity of the test has been calculated? How the norms of the test have been developed? Knowledge of these technical aspects helps both in construction, selection, revision and adaptation of a test. Interpretation is an essential aspect after scoring which also involves explaining the significance of the scores. What does it mean for a person who gets IQ score of 94? To fulfill all these purposes, an appropriate explanation is an essential requirement from the part of the investigator.

b) Developing Skills

The work of a psychologist is more like an artist. S/he needs to observe, listen, feel and then say something as less as possible. Observation here is not only a method used to study a specific problem. It should be developed as a habit. How to see things: people talking to each other in buses, trains, or offices; youngsters chatting outside a mall, people writing their views in newspapers and magazines, people behaving with each other- in families, in offices, in traffic, nothing should go unnoticed. It need not be a deliberate exercise once it is developed as a habit.