How do you know that people are giving answers that are truly representative of their views? One way is to measure internal consistency. In other words, you could ask multiple questions on the same topic and measure responses by checking if the participant responds similarly to each question. You could divide a multi-question section into two halves, using some questions in the first half and others in the second, or intersperse them with other questions throughout the survey.
For Example…
A survey may contain multiple questions that try to measure how a participant feels about a particular issue. Say you are developing an index that you want to use to measure how well people learn a particular skill. To do this, you want to ask them a series of questions that relate to their ability to learn, but in very different ways. To test whether all of the questions that you intend to use actually correspond with one another, you would measure their internal consistency. If one or more questions are not consistent with the others, then you are better off discarding those and using the rest. If none of them relate to one another, then you may need to rethink your questions entirely.
When you perform internal consistency tests it’s important to consider the interrelatedness of the questions and how the individual questions relate to the central topic of your study. By including questions that are unrelated to the concept of interest, a researcher may inadvertently lower the measure of internal consistency. That is why we only measure internal consistency for those questions that should be measuring roughly the same thing (self-esteem, preferences within a particular category, thrill-seeking, etc.).
However, in addition to the internal consistency questions, you will still need to include questions on demographics, categorical identifiers, etc. These questions serve as control variables in your analysis and are vital to your study even if they will not provide more extensive measures of internal consistency.
Assessing Internal Consistency
So, how can you assess internal consistency? Here are a couple tips to get started:
- You can calculate the split-half correlation, by splitting the questions into two sets and examining the relationship between them. Once you have finished calculating the correlation, a relationship of +.80 or greater is generally considered good internal consistency.
- You could also compute Cronbach’s alpha, which in simple terms is a measure of consistency that works by comparing the variance across a whole set of data.
