Content Validity
Every research design aims at covering all the relevant components of the construct with the help of a specific instrument. A construct is mostly a theoretical notion or idea that we cannot measure physically.
Content validity is a part of the overall measurement validity alongside three others - face, criterion, and construct validity. It is important to consider while developing written exam tests. The aim of any test is to check whether students have enough theoretical background on the subject to answer all the questions.
Examples of Content Validity
You can come across different constructs in your research design that is pretty observable and easy to measure. For instance, you can measure weight in pounds or distance in miles. However, there are constructs that are more abstract and difficult to measure directly than others. For example, the construct of aggression is challenging because it consists of separate dimensions.
Aggression is considered a clinical diagnosis or condition that requires specific screening tools for defining its metrics. So, if you want to develop a measurement scale for aggression, you should consider whether your scale covers all the dimensions of the aggression construct or only a certain part of them. For instance, if your scale covers only the affective aspects of aggression but ignores the behavioral aspects, it will have low content validity.
In addition, to achieve higher content validity, you need to consider a general agreement. It means you need to agree with other experts on what elements the construct has to represent. It is especially important for different types of psychological tests.
Even if self-esteem tests have been used by many researchers, you can criticize them for the lack of the complex nature of this human characteristic assessment. It consists of many emotional, intellectual, behavioral, and psychological components, so if you consider only some of them, such tests will have low content validity. If you see that existing self-esteem tests do not cover all the necessary dimensions, you may develop your own test that will involve all of them or several different tests for every dimension.
Comparison of Content vs. Others Validities
Construct and content validity are easy to confuse though they are different concepts. Construct validity is used to assess how effectively the test measures what it has to measure. Construct validity can be low or negative if some parts of the construct are missing and others that do not relate to it are included.
To check whether all the components of the construct are highlighted relevantly, you need to consider convergent and divergent (discriminant) validity.
Convergent validity shows how well the test measuring a specific construct can correlate with other tests developed for assessing the same construct. Discriminant (divergent) validity shows that the two tests meant for different constructs do not relate to each other. There should be zero or negative relationships between the results of these two tests to consider that your test has high construct validity.
You also have to consider internal and external validity in the experimental research. Internal validity indicates the degree of a causal relationship between variables, while external validity shows the generalizability of the results.
Content validity, in its turn, relates to any context for your test or questionnaire created for a specific construct. It shows confidence that the test actually measures what you want it to measure.
Suppose you want to research the nutritional needs of first-year college students via a survey.
| ✔️ You will get high content validity if your questions involve all the dimensions of the nutrition construct: regularity, healthy food, calories and vitamins, junk food consumption, lack of time, health issues, etc. Indeed, if all these dimensions and aspects are included, the content validity of the survey is high. | ❌ You will get low content validity if you omit some dimensions. The results of the survey may be inaccurate and do not consistently indicate all the nutrition needs of first-year college students. |
| ✔️ The survey will have high convergent validity if the answers to your survey correlate with the answers to similar surveys that have already been in use. If you get similar results, your survey will have high convergent validity as a part of construct validity (the other part is discriminant or divergent validity). | ❌ The survey will have low discriminant validity if its results strongly correlate with the existing measures of the student's attitudes towards their campus administration or specific types of food. Such results cannot be considered valid measures of the student's nutritional needs. In short, such a survey measures a different construct (either attitude or satisfaction) but not nutritional needs. So construct validity is low here, too. |
Therefore, content validity is low in both cases as well.
How to Measure Content Validity
You need to know how to measure content validity correctly. If the content validity score is high, the construct measurement has been pretty accurate.
Let’s regard three main steps to measure content validity.
Step 1. Collecting Necessary Data from Professionals
Subject matter experts or SMEs make up a judging panel for measuring content validity. They are professionals in the field your research belongs to. For instance, school science teachers can make an expert panel for a science test.
The task of the expert panel is to evaluate each component of all the questions as ‘essential,’ ‘useful,’ and ‘not essential.’ If the agreement among all the panelists is high, the level of content validity for an individual item is also high.
If you are a student researcher who works on a dissertation or term paper, you may not have access to any expert panel. The view, as in the case of the dissertation, can occur during the defense procedure. That is why you can involve the peer panel instead. You will need to mention this fact in your paper.
Step 2. Calculating the Ratio of Content Validity
The CVR or content validity ratio can be calculated with the help of the following formula:
+CVR = (ne - N/2) / (N/2)
Here:
N = the overall number of SME panelists who participate in the assessment.
Suppose you have a panel of seven experts to asses a test that includes nine questions. The first question in the test received ‘essential from six professionals. Let’s calculate the content validity ratio for this question:
CVR = (ne - N/2) / (N/2) = (6 - 7/2) / (7/2) = 0.7
You will continue calculating the ratio for every next question with the help of this formula. It can provide ratio values ranging from +1 to -1.
If the ratio is above zero, it means that more than half of the SMEs agreed that the element is essential. A ratio close to +1 means that the content validity is high.
Nevertheless, there are cases in which agreement is coincident. So, you need to use a special critical values table. The critical value means that the CVR for the question under consideration should not be lower than the minimum value indicated in the table. Let’s see what it looks like:
| No. of Experts | Critical Value |
|---|---|
| 5 | 0.99 |
| 6 | 0.99 |
| 7 | 0.98 |
| 8 | 0.76 |
| 9 | 0.79 |
| 10 | 0.63 |
| 11 | 0.58 |
| 12 | 0.55 |
| 21 | 0.45 |
| 31 | 0.34 |
| 41 | 0.28 |
Step 3. Calculating the Index of the Content Validity
However, there can be situations when you need to measure content validity for the entire set of questions but not for one question only. So, you have to use the CVI (Content Validity Index). It is an average CVR score calculated for all the questions. If the values are close to one content validity is high.
For example, CVI = 0.7 + 0.2 - 0.3+ 0.2 - 0.2 + 0.7 + 1/7 = 1.44
If you compare the CVI received here with the critical value for the 7-expert panel (0.98), you can judge that the CVI is high. It means that the test consistently and precisely measures the content validity as it was planned. If the CVI were lower than 0.98, you would need to correct the questions to improve it.
Final Thoughts
You can assume now that the content validity of the research design, tests, and questionnaires is important for indicating the relevance and correctness of all the construct components and dimensions tested within the research. It is a part of the entire measurement validity, and you need to consider and check it while creating written tests.
You know now how to measure content validity and correlate it with construct validity. Be attentive to its indicators and use the formula provided here for checking. It will allow you to ensure that your research design performs properly and can be used by other experimenters.