One way you can assess the reliability of questions that relate to factors constant over time in your survey is by using test-retest reliability. This is done by administering the survey to the same group of people at two different times in order to compare the individual results. Then, you calculate the test-retest correlation between the two sets of scores. A higher test-retest correlation shows a greater consistency in your results.
For Example…
Your core personality or intelligence is not likely to change over time, but your mood on a given day will. It’s important to keep in mind that some measures of a person may be more consistent than others.
When thinking about test-retest reliability, consider the different ways test-retest discrepancies can occur. Test-retest reliability is not a great predictor when questions have answers that are expected to change over time.
Calculating Test-Retest Reliability
After you have administered your survey for a second time, you can calculate the test-retest correlation between the two sets of data. Then you will know that a higher value for your test-retest correlation shows greater consistency between the two data sets, whereas a lower score shows more difference between them. These low scores often come from inherent differences in your data that are expected if you compare where change is standard across different timelines. You should also be aware that discrepancies may arise not only from the different conditions but also based on familiarity with the test or fatigue (Statology, 2021).
