Assessing the quality of your research

The research you do for your thesis should be well constructed. This means, for example, that your research is reliable and valid, and that the results are replicable and reproducible. When these criteria are met, it confirms that the results you find are not flukes, nor have they been influenced by external factors. What exactly should you look for when assessing the quality of your research? Here are the main points of interest.

Want to assess the quality of the research you have done? Pay attention to at least the following:

• validity;

• reliability;

• independence;

• practicality;

• reproducibility and replicability;

• generalizability.

First, your research must be valid. This means that you measure exactly what you intended to measure. 

Suppose you want to measure the degree to which people fear failure. To do this, you must use a measuring instrument that is actually suitable for this topic. A questionnaire about self-confidence, for example, is not valid, and should not be used. The research is also less valid if you design a questionnaire that enables participants to give socially desirable answers. 
 

If your study is reliable, it means that if you repeated your study using exactly the same method, you would get the same results. In other words, you haven't made random measurement errors ("bias"). For example, you want to avoid cases of respondents accidentally giving the wrong answer, evaluators analyzing answers differently, or multiple interviewers phrasing interview questions differently.

Any research you do must be independent and objective. That means that your opinions as the researcher, or the opinions/interests of any stakeholders have not influenced the results. In practice, we often speak of "intersubjectivity": if another researcher does the same study, they should be able to arrive at similar results. 

For example, research may not be independent if it is commissioned by a commercial organization to present their own products in a positive light. Suppose you are conducting research for a coaching agency on the positive effects of hiring a coach. Your questioning may be guided by the interests of the coaching agency, because they benefit from the fact that many positive effects emerge. You must then guard your independence in order to ensure high quality research.

This goal is especially important if you are doing research for a client. If that is the case, you want the client to actually be able to work with the results you produce or therecommendations you provide. You want to design the research in such a way that the recommendations are actionable for the client. 

Incidentally, it is conceivable that an invalid or unreliable survey may still provide something for the client. If you survey a few people about their experience at a dental office, your results are not so easy to generalize to the experiences of all people visiting the dentist. However, the dental practice may be able to get something out of it to improve their specific services. 

To assess the quality of your research, you should also look at reproducibility and replicability. Reproducibility means that a researcher can arrive at similar results when using the same method with a new data set. Replicability means that a researcher using the same method AND the same data set arrives at the same findings.

You usually cannot conduct research among all people in the target population, or including all possible situations. Therefore, you must choose a sample of people from the research population or focus on one or a few specific situations. It is important, however, that your chosen sample is representative of the entire research population. 
This makes the results more generalizable: the results are more likely to apply to a large number of people or cases in the real world. Generalizability is especially important in quantitative research.

We have separate articles online about many of these quality assurance criteria. Want to read more about assessing the quality of your research? Read up on these topics:

• types of validity;

• types of reliability;

• validity literature review;

• validity and reliability interviews;

• validity and reliability surveys;

• research methods;

• data analysis.