Trending March 2024 # What Is Criterion Validity? # Suggested April 2024 # Top 8 Popular

You are reading the article What Is Criterion Validity? updated in March 2024 on the website We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested April 2024 What Is Criterion Validity?

Criterion validity (or criterion-related validity) evaluates how accurately a test measures the outcome it was designed to measure. An outcome can be a disease, behavior, or performance. Concurrent validity measures tests and criterion variables in the present, while predictive validity measures those in the future.

To establish criterion validity, you need to compare your test results to criterion variables. Criterion variables are often referred to as a “gold standard” measurement. They comprise other tests that are widely accepted as valid measures of a construct.

Example: Criterion validityA researcher wants to know whether a college entrance exam is able to predict future academic performance. First-semester GPA can serve as the criterion variable, as it is an accepted measure of academic performance.

When your test agrees with the criterion variable, it has high criterion validity. However, criterion variables can be difficult to find.

What is criterion validity?

Criterion validity shows you how well a test correlates with an established standard of comparison called a criterion.

A measurement instrument, like a questionnaire, has criterion validity if its results converge with those of some other, accepted instrument, commonly called a “gold standard.”

A gold standard (or criterion variable) measures:

The same construct

Conceptually relevant constructs

Conceptually relevant behavior or performance

When a gold standard exists, evaluating criterion validity is a straightforward process. For example, you can compare a new questionnaire with an established one. In medical research, you can compare test scores with clinical assessments.

However, in many cases, there is no existing gold standard. If you want to measure pain, for example, there is no objective standard to do so. You must rely on what respondents tell you. In such cases, you can’t achieve criterion validity.

It’s important to keep in mind that criterion validity is only as good as the validity of the gold standard or reference measure. If the reference measure suffers from some sort of research bias, it can impact an otherwise valid measure. In other words, a valid measure tested against a biased gold standard may fail to achieve criterion validity.

Similarly, two biased measures will confirm one another. Thus, criterion validity is no guarantee that a measure is in fact valid. It’s best used in tandem with the other types of validity.

Types of criterion validity

There are two types of criterion validity. Which type you use depends on the time at which the two measures (the criterion and your test) are obtained.

Concurrent validity is used when the scores of a test and the criterion variables are obtained at the same time.

Predictive validity is used when the criterion variables are measured after the scores of the test.

Concurrent validity

Concurrent validity is demonstrated when a new test correlates with another test that is already considered valid, called the criterion test. A high correlation between the new test and the criterion indicates concurrent validity.

Establishing concurrent validity is particularly important when a new measure is created that claims to be better in some way than its predecessors: more objective, faster, cheaper, etc.

Example: Concurrent validityA psychologist wants to evaluate a self-report test on body image dissatisfaction. The concurrent validity of the test can be assessed by comparing the scores of the test with a clinical diagnosis that was made at the same time.

Remember that this form of validity can only be used if another criterion or validated instrument already exists.

Predictive validity

Predictive validity is demonstrated when a test can predict future performance. In other words, the test must correlate with a variable that can only be assessed at some point in the future, after the test has been administered.

For predictive criterion validity, researchers often examine how the results of a test predict a relevant future outcome. For example, the results of an IQ test can be used to predict future educational achievement. The outcome is, by design, assessed at some point in the future.

Example: Predictive validitySuppose you want to find out whether a college entrance math test can predict a student’s future performance in an engineering study program.

A student’s GPA is a widely accepted marker of academic performance and can be used as a criterion variable. To assess the predictive validity of the math test, you compare how students scored in that test to their GPA after the first semester in the engineering program. If high test scores were associated with individuals who later performed well in their studies and achieved a high GPA, then the math test would have strong predictive validity.

A high correlation provides evidence of predictive validity. It indicates that a test can correctly predict something that you hypothesize it should.

What can proofreading do for your paper?

Scribbr editors not only correct grammar and spelling mistakes, but also strengthen your writing by making sure your paper is free of vague language, redundant words, and awkward phrasing.

See editing example

Criterion validity example

Criterion validity is often used when a researcher wishes to replace an established test with a different version of the same test, particularly one that is more objective, shorter, or cheaper.

Example: Criterion validityA school psychologist creates a shorter form of an existing survey to assess procrastination among students.

Although the original test is widely accepted as a valid measure of procrastination, it is very long and takes a lot of time to complete. As a result, many students fill it in without carefully considering their answers.

To evaluate how well the new, shorter test assesses procrastination, the psychologist asks the same group of students to take both the new and the original test. If the results between the two tests are similar, the new test has high criterion validity. The psychologist can be confident that the new test will measure procrastination as accurately as the original.

How to measure criterion validity

Criterion validity is assessed in two ways:

By statistically testing a new measurement technique against an independent criterion or standard to establish concurrent validity

By statistically testing against a future performance to establish predictive validity

The measure to be validated, such as a test, should be correlated with a measure considered to be a well-established indication of the construct under study. This is your criterion variable.

Correlations between the scores on the test and the criterion variable are calculated using a correlation coefficient, such as Pearson’s r. A correlation coefficient expresses the strength of the relationship between two variables in a single value between −1 and +1.

Correlation coefficient values can be interpreted as follows:

r = 1: There is perfect positive correlation

r = 0: There is no correlation at all.

r = −1: There is perfect negative correlation

You can automatically calculate Pearson’s r in Excel, R, SPSS or other statistical software.

Positive correlation between a test and the criterion variable shows that the test is valid. No correlation or a negative correlation indicates that the test and criterion variable do not measure the same concept.

Example: Measuring criterion validitySuppose you are interested in developing your own scale measuring self-esteem. To establish criterion validity, you need to compare it to a criterion variable.

You give the two scales to the same sample of respondents. The extent of agreement between the results of the two scales is expressed through a correlation coefficient.

You calculate the correlation coefficient between the results of the two tests and find out that your scale correlates with the existing scale (r = 0.80). This value shows that there is a strong positive correlation between the two scales.

In other words, your scale is accurately measuring the same construct operationalized in the validated scale.

Other interesting articles

If you want to know more about statistics, methodology, or research bias, make sure to check out some of our other articles with explanations and examples.

Frequently asked questions about criterion validity Cite this Scribbr article

Nikolopoulou, K. Retrieved July 10, 2023,

Cite this article

You're reading What Is Criterion Validity?

What Is Vividness Bias?

Vividness bias is the tendency to focus on certain attributes of a decision or situation while overlooking other elements that are equally or more important.

Vividness bias examplePeople often prioritize a prospective employer’s reputation, the prestige of a title, or a higher salary over other things that they may value more, such as work-from-home possibilities or a shorter commute to work. Prioritizing prestige over what we actually value most is a sign of vividness bias.

What is vividness bias?

Vividness bias is a phenomenon in social psychology in which the most evocative information dominates our thinking and greatly influences our decision-making. In general, the “vividness” of information is the degree to which it is emotionally engaging, concrete, imagery-producing, and personal.

In other words, vividness is essentially the information that is most persuasive or that stands out the most. Recently, vividness bias has become popular specifically in the context of job negotiations, where vividness highlights our concerns to seek status and prestige. Because of vividness bias, we tend to “fall for” the flashier option and are often led to decisions and choices that do not fully align with our priorities and values.

What causes vividness bias?

Vividness bias is believed to be caused by the so-called vividness effect. Here, “vivid” information inherently influences our judgment more than non-vivid information. Vivid messages are thought to be more effective in changing our opinion or behavior. This is because vivid information is more readily available in our memory—we tend to pay more attention to it and recall it more frequently.

Vividly designed communications usually incorporate images, metaphors, and concrete, colorful language. These are more impactful than abstract messages and ideas, like statistics or charts, because the latter fail to draw or hold our attention.

Studies suggest that vividness does not affect persuasion, but rather what people think would persuade others, regardless of their own reactions.

Vividness bias example

Vividness bias can explain why we’re more drawn to the fun or bold aspects when faced with an option, such as which company to work for.

Example: Vividness bias in the workplaceMany tech companies in the recent past have tried to outdo one another in their offerings of fun workplace perks, such as ping-pong tables and free gourmet meals. Hiring managers thought that these vivid elements would attract young talent.

Although it seemed like a generally accepted belief that fun work perks were effective, the idea probably worked well at the very beginning, when hiring managers would walk prospective employees through the office. Over time, employees could see through all of that.

These perks served as the vivid elements of the job offer and although some employees were (or might still be) lured by them, recent studies have shown that this is not what young employees want. Instead, workers younger than 35 place more value on respect, which is reflected in some of the increasingly popular perks like flexibility, paid time off, and mental health support. It seems that the longer people are in the workforce, the less interested they are in the vivid aspects of a role.

How to avoid vividness bias

Vividness bias can harm negotiations, so it’s important to have a strategy in place to avoid it. The following steps can help you do so:

Be conscious of your priorities. We can’t stop and think about every little decision we make in our daily lives. However, before entering a negotiation or making a decision that can have a major impact on our lives (such as where to study or which job to choose), it’s worth pausing for a moment to think about what is most important to you. Setting our priorities straight beforehand can shield us from vividness bias.

Avoid the pitfall of social comparison. We are often tempted to compare ourselves to others, particularly to individuals that society considers successful. This is part of human nature. However, when we compare ourselves to people who have different values to us, we are bound to fall for vividness bias. We might accept the position that comes with the flashier title or expensive electronics, when in reality what we want is a company culture that aligns with our values.

Reflect on your choice. Once you have made up your mind, look at the factors you are most drawn to. Are these your true priorities or vivid factors? Thinking through your choice will help you pinpoint vividness bias. Taking a moment to reflect can also help us avoid other types of bias that influence decision-making, like anchoring bias and the availability heuristic.

Other types of research bias Frequently asked questions about vividness bias

Why is vividness bias important?

Vividness bias is important because it can affect our decisions and negotiations. It causes us to assign more weight to vivid information, like a perception of prestige, rather than other factors that, upon greater reflection, are more important to us. As a result, we get distracted and lose sight of our goals and priorities.

What is a real-life example of vividness bias?

A real-life example of vividness bias can often be observed in the outcome of business negotiations. Price is usually the most vivid information, while other aspects, such the complexity of implementation, or the time needed to complete the project, might be ignored.

What is the vividness effect in communication?

The vividness effect in communication is the persuasive impact that vivid information is thought to have on opinions and behaviors. In other words, information that is vivid, concrete, dramatic, etc., is more likely to capture our attention and sway us into believing or doing one thing rather than another. On the contrary, information that is dull or abstract is not so effective. The vividness effect relates to the vividness bias.

Sources in this article

We strongly encourage students to use sources in their work. You can cite our article (APA Style) or take a deep dive into the articles below.

This Scribbr article

Nikolopoulou, K. Retrieved July 10, 2023,

Cite this article


What Is ‘Foo’ In Programming?

In Programming, “Foo” means nothing. It’s a placeholder value for when a programmer cannot come up with a better alternative. Value like “Foo” also highlights the fact that the choice of name is not the relevant part of the code example.

You see “Foo” a lot in programming tutorials and lecture slides. Also, if more than one name is necessary, the word “foo” is typically followed by “baz” or “bar”.

How to Use “Foo” in Programming?

When you are working on an example code with sample values, you may have noticed how hard it is to come up with a placeholder name.

This is why programmers typically use “Foo”. This dummy value is easy to remember and throw in the mix out of thin air. Another common dummy value that is widely used by programmers is “bar”. This is useful if you need more than one sample name.

Let’s see when “Foo” can be useful. Here is a Python example that demonstrates how to use F-strings with placeholder values:

val1 = "foo" val2 = "bar" print(f"Hello, {val1} {val2}!")


Hello, foo bar!

The whole point of this piece of code is to demonstrate how a Python concept works. Thus, the values it uses don’t matter. This is why the values “foo” and “bar” are used. Also, you don’t want to use names that would draw attention to them. When a programmer sees “foo” or “bar”, they instantly know those are dummy values one can ignore.

Why “Foo”?

The origin of the word “foo” is unclear.

Some suggest the origin of “Foo” comes from the 1930-1950 era in the Smokey Stover comic strip by Bill Holman. This is because the unexplained letters “F-O-O” comically appear on license plates, picture frames, and sandwich board signs.

Another theory for the words “foo” and “bar” is suggested to originate from the World War II era term “FUBAR” which stands for “F***ed up beyond all repair”. This would also explain why the word “bar” is usually used in conjunction with “foo”.

When Not to Use “Foo”?

The variable “foo” is useful when you practice coding skills and play with code examples. Also, if you are running a class, tutorial, or mentoring someone, coming up with short placeholder values like “foo” and “bar” can save you time.

But don’t use values like “foo” or “bar” in actual projects. This is because you want to keep it clean and readable all the time. If you have variables, classes, or methods called “foo” or “bar”, you will have no clue what they are supposed to do. This is especially true if you come back to your projects after a while. Make sure to use descriptive and easy-to-understand names when not playing or demonstrating!


Today you learned what “foo” means in programming.

To take home, “foo” is a dummy name or a placeholder value for the lack of a better name. “Foo” is commonly used in code samples and examples. When one placeholder is not enough, you commonly see the word “bar” used after “foo”.

Using values like “foo” or “bar” is useful when you are playing with code examples or demonstrating code to others. This way you don’t have to waste resources in coming up with a creative name.

Don’t use stupid values like “foo” or “bar” in your real projects, though. Instead, write clean code and give a clear and consistent meaning to variables and other values in your code.

Thanks for reading! Happy coding!

Further Reading

Want to learn more terms related to programming? Make sure to read Programming Glossary!

What Is A Research Design

A research design is a strategy for answering your research question using empirical data. Creating a research design means making decisions about:

Your overall research objectives and approach

Whether you’ll rely on primary research or secondary research

Your sampling methods or criteria for selecting subjects

Your data collection methods

The procedures you’ll follow to collect data

Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Step 1: Consider your aims and approach

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

Research question exampleHow can teachers adapt their lessons for effective remote learning?

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative approach Quantitative approach

Understand subjective experiences, beliefs, and concepts

Gain in-depth knowledge of a specific context or culture

Explore under-researched problems and generate new ideas

Measure different types of variables and describe frequencies, averages, and correlations

Test hypotheses about relationships between variables

Test the effectiveness of a new treatment, program or product

Qualitative research designs tend to be more flexible and inductive, allowing you to adjust your approach based on what you find throughout the research process.

Qualitative research exampleIf you want to generate new ideas for online teaching strategies, a qualitative approach would make the most sense. You can use this type of research to explore exactly what teachers and students struggle with in remote classes. Quantitative research exampleIf you want to test the effectiveness of an online teaching method, a quantitative approach is most suitable. You can use this type of research to measure learning outcomes like grades and test scores.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics.

How much time do you have to collect data and write up the research?

Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?

Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?

Will you need ethical approval?

At each stage of the research design process, make sure that your choices are practically feasible.

Step 2: Choose a type of research design

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

Experimental and


designs allow you to test cause-and-effect relationships




designs allow you to measure variables and describe relationships between them.

Type of design Purpose and characteristics


Used to test causal relationships

Involves manipulating an independent variable and measuring its effect on a dependent variable

Subjects are randomly assigned to groups

Usually conducted in a controlled environment (e.g., a lab)


Used to test causal relationships

Similar to experimental design, but without random assignment

Often involves comparing the outcomes of pre-existing groups

Often conducted in a natural environment (higher ecological validity)


Used to test whether (and how strongly) variables are related

Variables are measured without influencing them


Used to describe characteristics, averages, trends, etc

Variables are measured without influencing them

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation).

Correlational design exampleYou could use a correlational design to find out if the rise in online teaching in the past year correlates with any change in test scores.

But this design can’t confirm a causal relationship between the two variables. Any change in test scores could have been influenced by many other variables, such as increased stress and health issues among students and teachers.

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Experimental design exampleIn an experimental design, you could gather a sample of students and then randomly assign half of them to be taught online and the other half to be taught in person, while controlling all other relevant variables.

By comparing their outcomes in test scores, you can be more confident that it was the method of teaching (and not other variables) that caused any change in scores.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Type of design Purpose and characteristics

Case study

Detailed study of a specific subject (e.g., a place, event, organization, etc).

Data can be collected using a variety of sources and methods.

Focuses on gaining a holistic understanding of the case.


Detailed study of the culture of a specific community or group.

Data is collected by extended immersion and close observation.

Focuses on describing and interpreting beliefs, conventions, social dynamics, etc.

Grounded theory

Aims to develop a theory inductively by systematically analyzing qualitative data.


Aims to understand a phenomenon or event by describing participants’ lived experiences.

What can proofreading do for your paper?

Scribbr editors not only correct grammar and spelling mistakes, but also strengthen your writing by making sure your paper is free of vague language, redundant words, and awkward phrasing.

See editing example

Step 3: Identify your population and sampling method

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

Population exampleIf you’re studying the effectiveness of online teaching in the US, it would be very difficult to get a sample that’s representative of all high school students in the country.

To make the research more manageable, and to draw more precise conclusions, you could focus on a narrower population—for example, 9th-grade students in low-income areas of New York.

Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling. The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling Non-probability sampling

Sample is selected using random methods

Mainly used in quantitative research

Allows you to make strong statistical inferences about the population

Sample selected in a non-random way

Used in both qualitative and quantitative research

Easier to achieve, but more risk of research bias

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study, your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question.

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Step 4: Choose your data collection methods

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews.

Questionnaires Interviews

More common in quantitative research

May be distributed online, by phone, by mail or in person

Usually offer closed questions with limited options

Consistent data can be collected from many people

More common in qualitative research

Conducted by researcher in person, by phone or online

Usually allow participants to answer in their own words

Ideas can be explored in-depth with a smaller group (e.g., focus group)

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Quantitative observation Qualitative observation

Systematically counting or measuring

Taking detailed notes and writing rich descriptions

All relevant observations can be recorded

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

Field Examples of data collection methods

Media & communication Collecting a sample of texts (e.g., speeches, articles, or social media posts) for data on cultural norms and narratives

Psychology Using technologies like neuroimaging, eye-tracking, or computer-based tasks to collect data on things like attention, emotional response, or reaction time

Education Using tests or assignments to collect data on knowledge and skills

Physical sciences Using scientific instruments to collect data on things like weight, blood pressure, or chemical composition

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

Step 5: Plan your data collection procedures

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.


Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations, which events or actions will you count?

ExampleTo measure student participation in an online course, you could record the number of times students ask and answer questions.

If you’re using surveys, which questions will you ask and what range of responses will be offered?

ExampleTo measure teachers’ satisfaction with online learning tools, you could create a questionnaire with a 5-point rating scale.

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

Reliability Validity

Does your measure capture the same concept consistently over time?

Does it produce the same results in different contexts?

Do all questions measure the exact same concept?

Do your measurement materials test all aspects of the concept? (content validity)

Does it correlate with different measures of the same concept? (criterion validity)

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

Sampling procedures

As well as choosing an appropriate sampling method, you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

How many participants do you need for an adequate sample size?

What inclusion and exclusion criteria will you use to identify eligible participants?

How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method, it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method, how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability).

Step 6: Decide on your data analysis strategies

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis. With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics, you can summarize your sample data in terms of:

The distribution of the data (e.g., the frequency of each score on a test)

The central tendency of the data (e.g., the mean to describe the average score)

The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics, you can:

Make estimates about the population based on your sample data.

Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis.

Approach Characteristics

Thematic analysis

Focuses on the content of the data

Involves coding and organizing the data to identify key themes

Discourse analysis

Focuses on putting the data in context

Involves analyzing different levels of communication (language, structure, tone, etc)

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

Other interesting articles

If you want to know more about the research process, methodology, research bias, or statistics, make sure to check out some of our other articles with explanations and examples.

Frequently asked questions about research design Cite this Scribbr article

McCombes, S. Retrieved July 16, 2023,

Cite this article

What Is A Ceiling Effect?

A ceiling effect occurs when too large a percentage of participants achieve the highest score on a test. In other words, when the scores of the test participants are all clustered near the best possible score, or the “ceiling”, the measurement loses value. This phenomenon is problematic because it defeats the purpose of the test, which is to accurately measure something.

Example: Ceiling effectOn a midterm math exam, in which the highest possible score is 100 points, 90% of the students score 98 out of 100. This means that the majority of the students obtained a top score, and the clustering of the scores near the top is evidence of a ceiling effect. This suggests the exam was too easy.

A ceiling effect can be observed in surveys, standardized tests, or other measurements used in quantitative research. 

What is a ceiling effect?

A ceiling effect is a measurement problem that places a limitation to the maximum level an individual can achieve on a test. As a result, there is a discrepancy between a person’s test score and their “true” score, or reality.

Depending on the scientific area, the term signifies one of the following:

A ceiling effect in medicine and pharmacology refers to the phenomenon in which a drug reaches a maximum effect, so that increasing the dosage does not increase its effectiveness.  For example, researchers sometimes observe that there is a threshold above which a painkiller has no additional effect. Even if they increase the dosage, there is no added benefit regarding pain relief. In this context, the ceiling effect occurs due to human biology.

A ceiling effect associated with statistics in social sciences refers to the phenomenon in which the majority of the data are close to the upper limit or highest possible score of a test. This means that (almost) all of the test participants achieved the highest (or very near to the highest) score.

What causes a ceiling effect?

In the context of statistics, a ceiling effect can occur in survey data because of the limited ability of survey instruments to accurately measure participants’ true responses, as well as distinguish them from others’ responses. This can be due to:

Efforts to limit response bias. In an attempt to prevent biases like social desirability bias, researchers might create ceiling effects due to the way they phrase the possible responses.  For example, when asking respondents about their alcohol consumption, the highest possible option might be “2 drinks per day or more”. This makes it easier for heavy drinkers to fill in the question without feeling too exposed. However, researchers then lose the ability to differentiate between those who consume 3, 4, 6, or more drinks per day.

Instrument design constraints. Due to poor design, a questionnaire might not be able to measure a variable above a certain limit. For example, when a college exam is too easy, everyone will get more or less the same high score. The ceiling effect creates an artificially low threshold, since anyone is able to pass the exam. As a result, the exam fails to measure what it’s supposed to measure (aptitude) beyond a certain (low) level.

Why is the ceiling effect a problem?

Because of the ceiling effect, tests, surveys and other measures fail to capture the true range of values or responses, resulting in little variance in the data.

Ceiling effects cause a number of problems in data analysis including the inability to:

Determine the central tendency of the data, or the true average in a dataset.

Compare the means between two groups, e.g., between a treatment and a control group.

Get an accurate measure of variability, such as standard deviation.

Form conclusions about the effect of the independent variable  on any dependent variables.

Rank individuals according to their score.

Overall, a ceiling effect hinders the accurate interpretation of data and can render results meaningless.

Ceiling effect examples

Ceiling effects can be observed in surveys that include response categories that do not fully capture the range of possible answers above a certain point.

Example: Ceiling effect and response biasSuppose that you are researching what residents in an area think about the new section of urban motorway constructed nearby. Among your survey questions, there is one concerning income (“What was your total household income last year?”) You present respondents with different income categories to choose from:

less than $50,000


Over $100,000

Although this is a discreet way to ask a sensitive question and avoid response bias, there is also a downside to it. Setting the top range like this creates an artificial cutoff point, or ceiling, beyond which it is not possible to measure income. In other words, you can’t differentiate between someone that makes  $100,000, $400,000 or $1 million per year.

Because the income range is not inclusive of the true values above that point, this results in inaccurate measurement and a ceiling effect.

A ceiling effect can create a low threshold, making it easy for participants to reach the highest possible score on a test.

Example: Ceiling effect and poor designYou have created a short memory test that assesses participants’ ability to recall information. The test consists of showing five words on a screen. Because most participants can remember all five words, the test exhibits a ceiling effect: you can’t use it to rank participants according to their recall ability. The best approach would be to use an already validated memory test.

How to avoid ceiling effects?

Ceiling effects can impact the quality of your data collection. It’s really important to take the necessary steps to prevent this phenomenon. There are a few strategies you can use to avoid ceiling effects in your research:

Use previously-validated instruments, such as pre-existing questionnaires measuring the concept you are interested in. In this way, you can ensure that the questionnaire will allow you to capture a wide range of responses.

If no such instrument exists, run a pilot survey or experiment to check for ceiling effects. Running a small-scale trial of your survey will give you the opportunity to adjust your questions in case you do notice a ceiling effect.

When your survey includes sensitive or personal topics, like questions about income or drug use, provide anonymity, and don’t set artificial limits on responses. Instead, you could let participants fill in the higher value themselves.

Other types of research bias Frequently asked questions

What is the difference between ceiling and floor effect?

The terms ceiling effect and floor effect are opposites but they refer to the same phenomenon: the clustering of individual survey responses around a certain value. More specifically, ceiling effects occur when a considerable percentage of participants score the best or maximum possible score, while floor effects occur when the opposite happens, i.e.,  a considerable percentage of participants obtain the worst or minimum available score. This can be observed, for example, when a test is too easy (ceiling effect) or too difficult (floor effect). As a result, researchers can’t use the test to rank participants at either end of the scale.

What is a ceiling effect in pharmacology

In pharmacology a ceiling effect is the point at which an independent variable (the variable being manipulated) is no longer affecting the dependent variable  (the variable being measured). This can be seen with analgesic or pain-relieving medication. Even if researchers increase the dosage, there is a certain point beyond which the effectiveness of the medication will no longer increase.

Why is the ceiling effect a problem?

The ceiling effect is a problem in statistical analysis and data interpretation because it restricts the range of values that a variable can take. Due to this, there is a difference between the reported values and the ‘real’ values which means that the survey, test, or other measure used fails to collect accurate data.

Sources in this article

We strongly encourage students to use sources in their work. You can cite our article (APA Style) or take a deep dive into the articles below.

This Scribbr article

Nikolopoulou, K. Retrieved July 10, 2023,

Cite this article


Show all sources (5)

What Is A Super Integron?

Super-integron term was first applied in 1998 (but without definition) to the integron with a long cassette array on the small chromosome of Vibrio cholerae. The term has since been used for integrons of various cassette array lengths or for integrons on bacterial chromosomes (plasmids).

The use of “super-integron” is now discouraged since its meaning is unclear. In more modern usage, an integron located on a bacterial chromosome is termed a sedentary chromosomal integron, and one associated with transposons or plasmids is called a mobile integron. Two groups of integrons are known: resistance integrons and super-integrons. Gene cassettes in super-integrons encode a variety of different functions.

Super-integrons are located on the bacterial chromosome. The gene cassettes in resistance integrons probably originated from super-integrons. The recent finding of super-integron (SI) structures in the genomes of several bacterial species have expanded their role in genome evolution.

The Vibrio cholerae super integron is gathered in a single chromosomal super-structure harboring hundreds of gene cassettes. A comparison of the cassette contents of super-integrons from remote Vibrio species suggests that most of their cassettes are species-specific.

Many bacterial species belonging to several distinct genera of the γ- and β-proteobacteria undoubtedly carry or show strong evidence for the presence of chromosomal SIs. If each bacterial species harboring a SI has its own cassette pool, the resource in terms of gene cassette availability may be immense.

Super Integron

Our five-decade-long battle against bacteria is a testament to the genetic flexibility of these organisms. Not long after their introduction, we were witnessing the emergence of bacterial resistance to new antimicrobial agents. It is now clearly established that the prevailing strategy adopted by bacteria to evade antimicrobial activity is via acquisition of a gene from an exogenous source that confers resistance by any means. Integrons can now be divided into two major groups: the resistance integrons (RI) and the super-integrons (SI).

RI carry mostly gene cassettes that encode resistance against antibiotics and disinfectants and can be located either on the chromosome or on plasmids. The large chromosomally located integrons, which contain gene cassettes with a variety of functions, belong to the SI group.

SI are not given a specific name. The integron originally designated as class 4 is now named Vibrio cholerae SI. SI have been described for Geobacter sulfurreducens, Listonella pelagia, Nitrosomonas europaea, Pseudomonas alcaligenes, Pseudomonas mendocina, Pseudomonas spp., Pseudomonas stutzeri, Shewanella oneidensis, Shewanella putrefaciens, Treponema denticola, Vibrio anguillarum, Vibrio cholerae, Vibrio fischerii, Vibrio metschnikovii, Vibrio mimicus, Vibrio parahaemolyticus and Xanthomonas campestris.

Three classes of multi-resistant (MR) integrons have been defined based on the homology of the integrase genes and each class appears to be able to acquire the same gene cassettes. The integron platforms are defective for self-transposition but they are often found associated with insertion sequences (ISs), transposons, and/or conjugative plasmids which can serve as vehicles for the intra- and interspecies transmission of genetic material.

The potency of a highly efficient gene capture and expression system in combination with broad host range mobility is self-evident. The proficiency of this partnership is confirmed by the marked differences in codon usage among cassettes within the same integron, indicating that the antibiotic resistance determinants are of diverse origins

Such a system permits bacteria to stockpile exogenous genetic loci and MR integrons harboring up to five different cassettes have been characterized (In30).

Several observations suggest that integron structures impact genome evolution to a much greater extent than initially believed. First, the degree of homology between the three integrase classes (45-58%) suggests that their evolutionary divergence has extended over a longer period than the 50 years of the antibiotic era.

Second, the bias towards the propagation of resistance gene cassettes is likely due to the selective pressure of antibiotic therapy regimes driving the specific capture of resistance cassettes, implying that cassette genesis is not restricted to resistance determinants. It is conceivable that any ORF can be structured as a gene cassette.

Recently a new type of integron, a super-integron (SI) harboring hundreds of cassettes and differing in several ways from the MR integrons, has been identified in the Vibrio cholerae genome. This review focuses on this type of integron and gives the current state of knowledge on their characteristics and distribution.


The gene cassettes found in SI encode a wide variety of different functions, in contrast to the functions of gene cassettes found in RI. The number of resistance genes carried by the same plasmid, and even in the same integron, appears to rise. The integration of virulence factors and resistance determinants on the same plasmid may have even greater implications for public health. These bearers of multi-resistance are likely to remain because the physical association of integrons with other resistance determinants will lead to their continuous selection.

The role of SI in the evolution of bacterial species has been barely touched upon, but their apparent ubiquity suggests that they play an important role in bacterial evolution. The variety of structures found among class 1 integrons and their genetic surroundings after slightly more than half a century of antibiotic usage bears testament to the genetic flexibility and adaptability of the bacterial genome under environmental stress, making these microorganisms ultimate survivors.

Update the detailed information about What Is Criterion Validity? on the website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!