Validity and reliability in a survey

Are you conducting surveys for your research?If so, it is important that your survey questions are actually measuring what you want to measure. It is also crucial that your sample is a good representation of your research population. All of these thingsrelate to the validity and reliability of a survey. How do you monitor the two? Here are the main things to consider.

Whether you use surveys or choose another research method, it is always important for your research to be carefully designed and that the results are generalizable. That's where validity and reliability come in. 

A valid study means that with the surveys used, you actually measure what you intended to measure. So, the survey questions and the method of surveying must match that. 

In addition, research is reliable if it is replicable and reproducible. That means that another researcher can conduct your research again in exactly the same way and come up with similar results. Reliability is mainly about conducting the research consistently and carefully.

There are different types of validity. In surveys, content validity, concept validity, ecological validity, external validity and internal validity are particularly important. You can do the following to ensure each form of validity:

• Content validity and concept validity. For this, it is important that you operationalize the concepts in your research appropriately. Choose a way of measuring the concepts that actually aligns with what you want to measure (for example, by looking at how other researchers do this or by using a standardized questionnaire). If necessary, test out the questions with a few subjects so that you can be sure they understand them.

• Ecological validity. This is about the extent to which your results are also applicable to the real world (i.e., outside the situation in which you conduct the surveys). You can check for this by, for example, surveying people in a natural situation (online    home or on the street) rather than in a laboratory setting. 

• External validity. This form of validity means that your results can be generalized to a larger group of people or situations. In this case, it is especially important that the sample for your survey is large enough, and that you select this group randomly and according to clear criteria. Make sure your sample is a good representation of the research population.

• Internal validity. Proper data analysis is essential for this type of validity in a survey. Check how other researchers who have researched something similar have handled it. Make sure you use a proven method. This increases the internal validity of your survey research.

For the reliability of your survey, it is especially important for you to be consistent with how you conduct your survey. For example, do you ask each respondent the same questions in the same order and under the same circumstances? 

These are a few important points to watch out for:

• Determine the internal consistency of the various items you measure by calculating the Cronbach's alpha. 

• Perform the data analysis accurately and do it the same way for each survey. If multiple people are analyzing the data, inter-rater reliability is important. Establish a protocol for reviewing and analyzing the data so that each researcher handles it the same way.

• Keep a log in which you note the steps you took while conducting the surveys. Also write down possible details that may be important to remember. Doing this allows you to describe the survey procedures later in your methods chapter. This is because in this chapter, you will need to explain all the steps of conducting the surveys so that your research is replicable. 

• Don't make the questionnaire too long. This will prevent respondents from paying less attention to the last questions, making the answers to those questions less reliable.

If you want to use surveys for your thesis, it is very important that you do it properly and carefully. With our articles, we can help you with that. For example, read more about making a good research plan, a strong research design and proper data collection. 

Still unsure if a survey is the right research method for you? In our article on research methods, we explain the various methods so you can make an informed choice.