Docsity
Log inSign up
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search Store documents

The best documents sold by students who completed their studies

Search through all study resources
Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

University Rankings

Discover the best universities in your country according to Docsity users

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

From our blog

Exams and Study
Go to the blog

Understanding Composite Scores and Missing Data in Psychometric Assessments, Lecture notes of Statistics

Northumbria UniversityStatistics

An introduction to composite scores, their calculation methods, and the importance of handling missing data in psychometric assessments. Composite scores are constructed scores derived from summing or averaging responses to multiple items or indicators. the differences between composite scores and scales, indexes, and the importance of considering missing data and outliers in statistical analysis. It also covers the concept of latent variables and their measurement through constructs and indicators.

What you will learn

  • How are constructs measured through indicators and what role do latent variables play in this process?
  • How can missing data affect composite scores and what methods can be used to handle it?
  • What are composite scores and how are they calculated?
  • What are outliers and how can they be identified and handled in statistical analysis?
  • What is the difference between composite scores and scales, indexes, and latent variables?

Typology: Lecture notes

2021/2022

Uploaded on 09/27/2022

pauleen
pauleen 🇬🇧

3.5

(8)

211 documents

1 / 12

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1
04a: Composite Variables and Reverse Scoring
1. Manifest and Latent Variables
Recall the earlier presentation on manifest and latent variables. Summarized:
• Manifest variables = loosely described, are those that can be directly observed or measured
o Examples
▪ height
▪ weight
▪ age
▪ income
• Latent variables = not easily observed or measured; constructed through composite variables as measured
through scales and indexes
o Examples
▪ Stress
▪ general self-efficacy
▪ workplace autonomy
▪ life satisfaction
▪ test anxiety
• Constructs often used to measure latent variables
o created by taking composite scores from
o indicators that are designed to measure a latent variable
o indicator is an instrument item that provides an indication about one’s position or level on some
attribute, attitude, etc.
• Example
o Test Anxiety, composed of two dimensions
o dimension 1: physiological (somatic, emotionality) reactions
▪ sweating
▪ headache
▪ upset stomach
▪ rapid heartbeat
▪ feeling of dread
o dimension 2: negative cognition, thoughts
▪ expecting failure
▪ negative thoughts
▪ frustration
▪ comparing oneself to others negatively
▪ feelings of inadequacy
▪ self-condemnation
o Indicators of physiological reaction
1. Before or during tests you feel your heart start to beat faster.
2. You get upset stomachs while taking tests.
3. When taking a test, you get a feeling of dread.
o Indicators of negative cognition
4. While taking tests you think about how poorly you are doing.
5. You expect failure or poor grades when taking tests.
6. You become frustrated during testing.
o Indicator response options
pf3
pf4
pf5
pf8
pf9
pfa

Partial preview of the text

Download Understanding Composite Scores and Missing Data in Psychometric Assessments and more Lecture notes Statistics in PDF only on Docsity!

04 a: Composite Variables and Reverse Scoring

1. Manifest and Latent Variables Recall the earlier presentation on manifest and latent variables. Summarized: - Manifest variables = loosely described, are those that can be directly observed or measured o Examples ▪ height ▪ weight ▪ age ▪ income - Latent variables = not easily observed or measured; constructed through composite variables as measured through scales and indexes o Examples ▪ Stress ▪ general self-efficacy ▪ workplace autonomy ▪ life satisfaction ▪ test anxiety - Constructs often used to measure latent variables o created by taking composite scores from o indicators that are designed to measure a latent variable o indicator is an instrument item that provides an indication about one’s position or level on some attribute, attitude, etc. - Example o Test Anxiety, composed of two dimensions o dimension 1: physiological (somatic, emotionality) reactions ▪ sweating ▪ headache ▪ upset stomach ▪ rapid heartbeat ▪ feeling of dread o dimension 2: negative cognition, thoughts ▪ expecting failure ▪ negative thoughts ▪ frustration ▪ comparing oneself to others negatively ▪ feelings of inadequacy ▪ self-condemnation o Indicators of physiological reaction 1. Before or during tests you feel your heart start to beat faster. 2. You get upset stomachs while taking tests. 3. When taking a test, you get a feeling of dread. o Indicators of negative cognition 4. While taking tests you think about how poorly you are doing. 5. You expect failure or poor grades when taking tests. 6. You become frustrated during testing. o Indicator response options

▪ 1 = Not at all like me ▪ 7 = Very much like me o One student’s responses

  1. Heart beats faster = 2
  2. Upset stomach = 3
  3. Feel dread = 2
  4. Think of poor performance = 2
  5. Expect failure = 1
  6. Frustrated = 1 o Composite score on one student’s responses ▪ Sum = 2+3+2+2+1+1 = 11 ▪ Mean is 11 / 6 = 1. ▪ Composite test anxiety is 1.83 on 1 to 7 scale 2. Composite Scores
  • Composite scores are constructed scores o Summing responses across items or indicators (not a good option, explained below) o Mean of responses across items or indicators (good option) o Weighted composite from factor analysis or similar analysis (usually sample dependent) is sometimes used; weighted means some items account for more of the composite score than others; this requires more complex statistics or theoretical guidance; using equally weighted composite scores – like taking the sum of all items or the mean of all items – works well in many cases. Weights described below.
  • Sometimes called scale scores, but this can be confusing since scale scores generally are understood to be scores with predefined mean and standard deviation (standard score, Z score)
  • Index (Summary) vs. Scale o Scale: items designed to measure the same construct (e.g., test anxiety); theoretically items should be correlated o Index: items used to form a composite score, but items do not have to measure the same construct; items may be theoretically unrelated ▪ Example: Socio-economic Status (SES)
  • Educational level
  • Occupational Prestige
  • Wealth ▪ Dow Jones Industrial Average: composed of 30 major companies
  • Sum scored can be misleading o Example from test anxiety, student has maximum anxiety ▪ 1. Heart beats faster = 7 (on scale from 1 to 7) ▪ 2. Upset stomach = 7 (on scale from 1 to 7) ▪ 3. Feel dread = 7 (on scale from 1 to 7) o Minimum and maximum summed scores are ▪ 1+1+1 = 3 ▪ 7+7+7 = 21 o Respondent’s sum = 7+7+7 = 21, which is top score possible for sum of these three items o Item 2 has missing data ▪ 1. Heart beats faster = 7 (on scale from 1 to 7) ▪ 2. Upset stomach = missing (on scale from 1 to 7) ▪ 3. Feel dread = 7 (on scale from 1 to 7)

Example 2 Respondent Sex Race Education TA1 TA2 TA3 TA4 TA5 TA 1 0 2 1 1 2 3 3 ----- 1 2 1 3 2 2 2 2 3 4 2 3 0 3 2 ----- 7 6 5 7 6 4 1 2 2 4 5 4 2 3 2 5 0 1 3 2 5 6 6 4 5 6 0 1 1 1 1 1 ----- 1 1 7 0 1 2 3 4 6 3 5 6 8 1 2 4 6 7 6 7 7 7 9 1 3 3 2 1 4 5 1 2 10 1 3 4 2 ----- 4 4 3 1 Some Missing Data for Test Anxiety Questionnaire Missing Seems Random Mean Replacement Acceptable Example 3 Respondent Sex Race Education TA1 TA2 TA3 TA4 TA5 TA 1 0 2 1 1 2 ----- 3 2 1 2 1 3 2 2 2 2 3 4 2 3 0 3 2 7 7 ----- 5 7 6 4 1 2 2 4 5 ----- ----- ----- ----- 5 0 1 3 2 5 6 6 4 5 6 0 1 1 1 1 ---- 1 ----- 1 7 0 1 2 3 4 6 3 5 6 8 1 2 4 6 7 ----- 7 7 7 9 1 3 3 2 1 4 5 1 2 10 1 3 4 2 4 ----- ----- 3 1 Some Missing Data for Test Anxiety Questionnaire Look at TA3 item for wording, perhaps offensive or too personal or maybe difficult to see on questionnaire Missing Seems Systematic Mean Replacement Not Acceptable

4. Outliers - Outliers are scores or score combinations that produce observations that are very difficult from rest of sample - Outliers can influence statistical results so should be examined and fixed, accepted, or removed depending upon findings of case study of outlier - Discussion that follows is an unsophisticated review of outliers; see link below for more detailed treatments o https://en.wikipedia.org/wiki/Outlier - Checking for outliers o Frequency Display o Z Scores o Scatterplot o Boxplot o Histograms

Example 4 : Frequency Display of TA1 (scale min and max 1 to 7) TA1 - Heart Beats Faster During Tests Frequency Percent Valid Percent Cumulative Percent Valid 1.00 5 23.8 23.8 23. 2.00 4 19.0 19.0 42. 3.00 1 4.8 4.8 47. 4.00 4 19.0 19.0 66. 5.00 3 14.3 14.3 81. 7.00 3 14.3 14.3 95. 77.00 1 4.8 4.8 100. Total 21 100.0 100. Recall that the scale for test anxiety is 1 to 7; note one score above is 77 – likely a data entry error – but this score can have a large effect on scoring and analysis so must be corrected. Example 5 : Scatterplot of Tests 2 Grades and Seconds to Answer Each Item on Average Note the outlier in the upper, left corner. This graph makes it easy to identify someone who is likely cheating. All other students took 120 seconds or more, on average, to complete test items. The individual with a score near 100 took less than 45 seconds on average to answer each item.

  • Hendrickson et al. (2008) found some benefit for weighting, but it was small, they write: “The increase in reliability by using the maximum reliability weighting scheme was as much as 0.029” and for validity results were inconclusive and small.
  • In short, unit weighting is easier, is not sample dependent, and loses little in terms of reliability and validity especially when number of items per construct increases. 6. Reversed Scores
  • Reverse scoring is necessary for those items that take opposite responses from other items designed to measure the same construct; reverse coded items are sometimes referred to as negative valence and non-reverse coded items are referenced as positive valence
  • Some argue that reversed items can be useful to help keep respondents alert and break response set
  • Failure to reverse score items that are reverse coded can result in lower reliability and validity, and produces scores that cannot be interpreted
  • Critical that you explain to readers o Which items reverse scored o Which items used to form construct, reversed or non-reversed o Interpretation of construct scores, e.g., 7 = high anxiety and 1 = low, or if reversed, 7 = low anxiety and 1 = high
  • Identify reversed items by assuming role of respondent who has extreme position on construct
  • Example: Test Anxiety – assume you have extreme test anxiety, which item elicits a response that is different from the others? Not true of me Very true of me
  1. Before or during tests you feel your heart start to beat faster.
  1. You get upset stomachs while taking tests. 1 2 3 4 5 6 7
  2. While taking tests you get a feeling of confidence that you will do well tests.
  1. While taking tests you think about how poorly you are doing.
  • Formula for reverse scoring: Reversed Score = (minimum score) + (maximum score) – actual score
  • Example of reverse scoring with 1 to 5 scale: Original Score Formula Reversed Score 1 1 + 5 – 1 = 5 2 1 + 5 – 2 = 4 3 1 + 5 – 3 = 3 4 1 + 5 – 4 = 2 5 1 + 5 – 5 = 1
  • Example of reverse scoring with - 3 to +3 scale:

Original Score Formula Reversed Score

  • 3 - 3 + 3 – (-3) = 3
  • 2 - 3 + 3 – (-2) = 2
  • 1 - 3 + 3 – (-1) = 1 0 - 3 + 3 – (0) = 0 1 - 3 + 3 – (1) = - 1 2 - 3 + 3 – (2) = - 2 3 - 3 + 3 – (3) = - 3
  • Never delete original variable, instead, form a new reversed variable, e.g., TA3 becomes TA3R (R indicates reversed version)
  • Check correlations of original and reversed to ensure reversed scores correct, if correct Pearson r = - 1
  • Worked Example: Academic Control – one’s ability to control or determine their academic behavior and outcomes in college. o Which item or items should be reverse scored? o Goal is to have higher scores indicate more academic control Strongly Disagree Disagree Mix of Disagree and Agree Agree Strongly Agree
  1. There is little I can do about my performance in college/university.
  1. The more effort I put into my courses, the better I do in them.
  1. How well I do in my courses is often the “luck of the draw.”
  1. I have a great deal of control over my academic performance in my courses.
  1. When I do poorly in a course, it’s usually because I haven’t given it my best effort.
  • Worked example of Academic Control o Data File: http://www.bwgriffin.com/gsu/courses/edur9131/2018spr-content/05-composite-scores/05-reverse-scores-academic- control.sav Steps in Computing Reversed Items
  1. Assess correlations among items to ensure items to be reversed correlate negatively with positive items designed to measure the same construct.
  • Now for item

Check on reverse scoring

7. SPSS Example for Composite Scores - Form mean of academic control - Use reversed scores Steps in Computing Composite Variables

  1. Use Compute command to create composite scores.
  2. Use Mean(variable1, variable2, variable3, etc.) function in Compute to create new variable. Use the positive and reversed items, do not use the original items that were reversed. See below.