Note: The following glossary is not intended to be a complete social science research glossary, but seeks to cover concepts that are important in the completion of IRB applications. This glossary does not include terms specific to medical research.
Download the Glossary as a PDF
Adverse Effect: An undesirable and unintended, although not necessarily unexpected, result of therapy or other intervention.
Anonymity: A research condition in which no one, including the researcher, knows the identities of research participants.
Assent: Agreement by an individual not competent to give legally valid informed consent (e.g., a child or cognitively impaired person) to participate in research.
Atomistic Fallacy: The fallacy one commits when making inferences about groups or aggregates from individuals (see Ecological Fallacy).
Autonomy: The personal capacity participants should possess in research conditions to consider alternatives, make choices, and act without undue influence or interference of others.
Biased Sample: A sample that is not representative of the population from which it was drawn (see Representative Sample).
Beneficence: An ethical principle that requires an obligation to protect research participants from harm. The principle of beneficence can be expressed in two general rules: (1) do not harm; and (2) protect from harm by maximizing possible benefits and minimizing possible risks of harm.
Benefit: A valued or desired outcome; an advantage.
Case Study: A research strategy that focuses on one case (an individual, a group, an organization, etc.) within its social context during one time period.
Causal Hypothesis: A statement hypothesizing that the independent variable affects the dependent variable in some way.
Causal Relationship: A relationship where an independent variable affects a dependent variable in some way.
Children: Persons who have not yet attained the legal age for consent to treatment or procedures involved in the research, as determined under the applicable law of the jurisdiction in which the research will be conducted.
Claim: A statement, similar to a hypothesis, which is made in response to the research question at hand, and that is backed up with evidence based on research.
(Classic) Controlled Experiment: An experimental design with two or more randomly selected groups (an experimental group and control group) in which the researcher controls or introduces the independent variable and measures the dependent variable at least two times (pre- and post-test measurements).
Closed-ended Questions:Survey questions that can only be answered in predetermined ways (for example, a scale of one to five measuring satisfaction with something).
Cluster Sample: A probability sample that is determined by randomly selecting clusters of people from a population and subsequently selecting every person in each cluster for inclusion in the sample.
Cognitively Impaired: Having either a psychiatric disorder (e.g., psychosis, neurosis, personality or behavior disorders, or dementia) or a developmental disorder (e.g., mental retardation) that affects cognitive or emotional functions to the extent that capacity for judgment and reasoning is significantly diminished. Capacity for autonomy and voluntary participation is thus impaired. Others, including people under the influence of or dependent on drugs or alcohol, those suffering from degenerative diseases affecting the brain, terminally ill patients, and persons with severely disabling physical handicaps, may also be compromised in their ability to make decisions in their best interests.
Cohort: A group of people born within a given time frame or experiencing a life event at approximately the same time.
Cohort Study: A specific kind of trend study involving the study of a cohort over time.
Compensation: Payment or medical care provided to participants injured in research; does not refer to payment for participation in research (see Remuneration).
Competence: Used as a legal term to indicate a person’s capacity to act on one’s own behalf; a person’s ability to understand information presented, to realize the consequences of acting (or not acting) on that information, and to make a choice (see Incompetence and Incapacity).
Confidentiality: A research condition in which no one except the researcher(s) knows the identities of the research participants. The treatment of information that a participant has disclosed to the researcher in a relationship of trust and with the expectation that it will not be revealed to others in ways that violate the original agreement, unless permission is granted by the participant.
Content Analysis: The systematic and quantitative study of some form of communication (e.g. speeches, TV programs, newspaper articles, advertisements, etc.).
Confounding Factor: Any factor that might serve as an alternative explanation for a study’s result; confounding factors include non-randomized samples, selection bias, and any arbitrary differences between people that are being compared.
Contraindicated: Disadvantageous, perhaps dangerous; a treatment that should not be used in certain individuals or conditions due to risks. For instance, a drug may be contraindicated for pregnant women and people with high blood pressure. Such individuals should not be involved in the study.
Control Group: The group in an experimental design that receives either no treatment or a different treatment from the experimental group. This group can thus be compared to the experimental group.
Convenience Sample: A non-probability sample that is determined by selecting participants that are readily accessible (convenient) to the researcher, (examples in studies of Colorado College students might include going to an organizational meeting or hanging out outside of Rastall and asking students exiting the lunchroom to take a survey).
Correlational Relationship: A relationship where two variables are associated (this can be measured in terms of strength and direction using statistical tests) but not causally related. They vary together in some way, but the variation of one does not itself cause the variation of the other (see Correlation Coefficient).
Correlation Coefficient: A statistical measurement of the degree of correlational relationship between two variables. Values of correlation coefficients range from –1.00 to +1.00. A correlation coefficient of 0.00 indicates no relationship between the variables. Correlations approaching –1.00 or +1.00 indicate strong relationships between the variables.
Cross-Over Design: Same as Within-participants Design.
Debriefing: After running a study, explaining to a participant what happened and what the study is for, explaining any deception used in the study, asking for any remaining comments or concerns, and ensuring that the participant is left with no adverse consequences from the experience. This sometimes involves providing contact information for groups that can provide support regarding a difficult issue.
Deception: The intentional withholding of information from participants, or deception about the study’s purpose and exact nature, that is deemed necessary by the researcher in order to meet the study’s goals. Deception should only be used when the researcher feels that participant knowledge about the study would alter participants’ behavior or responses in the study. Deception should not cause any adverse consequences to the participants, and participants should be debriefed after running the study. IRB guidelines on the use of deception should be reviewed. A study that cannot justify the use of deception may not receive IRB approval.
Demand Characteristics: A bias that results when participants display characteristics because they are aware that they are being observed.
Dependent Variable: A variable that varies due (at least in part) to the impact of the independent variable – that is, its value “depends” on the value of the independent variable. In the variables “sex” and “academic major,” academic major is the dependent variable, meaning that your major can’t determine whether you are male or female, but your sex might indirectly lead you to favor one major over another (nationally, men tend to major in engineering, women in education).
Descriptive Study: Any study that is not truly experimental (e.g., quasi-experimental studies, correlational studies, record reviews, case histories, and observational studies).
Double-Blind Design: An experiment in which neither the participants nor the research staff who interact with them knows the memberships of the experimental or control groups. Also known as Double-Masked Design (see Single-Blind Design and Open Design).
Double-Masked Design: Same as Double-Blind Design.
Ecological Fallacy: The fallacy one commits when making inferences about individuals from information about groups or aggregates (See Atomistic Fallacy).
Emancipated Minor: A legal status given to those individuals who have not yet attained the age of legal competency as defined by state law, but who are entitled to adult treatment because of assuming adult responsibilities such as being self-supporting and not living at home, marriage, or procreation (see Mature Minor).
Equitable: Fair or just; used in the context of selection of participants to indicate that the benefits and burdens of research are fairly distributed (see Justice).
Ethical Research: Research that follows widely held guidelines about what is ethical, moral and responsible in research settings (e.g. not plagiarizing others’ work, not misreporting sources, not submitting questionable data, not destroying or concealing sources, etc.) and that considers its role in the broader community and the effect of its findings on the community.
Ethnographic Research: Ethnography is the study of people and their cultures. Ethnographic research involves observation of and interactions with the people or group being studied in the group’s own environment, often for long periods of time (see Field Research).
Expedited Review: Review of proposed research by the IRB chair or a designated voting member or group of voting members rather than by the entire IRB. Federal rules permit expedited review for certain kinds of research involving no more than minimal risk and for minor changes in approved research.
Experimental: Term used to denote a therapy (drug, device, procedure) that is unproven or not yet scientifically validated in terms of safety and efficacy. A procedure may be considered “experimental” without necessarily being part of a formal study to evaluate its usefulness.
Experimental Design: A study design that calls for the control or manipulation of the independent variable in some way. A study design in which participants are randomly assigned to experimental groups and receive treatment in the form of the independent variable.
Experimental Group: The group in an experimental design study that receives treatment in the form, or in various forms, of the independent variable. This group can thus be compared to the control group.
False Negative: Same as Type II Error.
False Positive: Same as Type I Error.
Federal Policy (The): The federal policy that provides regulations for the involvement of human participants in research. The Policy applies to all research involving human participants that is conducted, supported, or otherwise participant to regulation by any federal department or agency.
Field Research: Behavioral, social, or anthropological research involving the study of people or groups in their own environment and without manipulation for research purposes. Research conducted in natural, real-life settings, outside the laboratory. This involves observation and, in many cases, interactions with the people being studied (see Ethnographic Research).
Full Board Review: Review of proposed research at a convened meeting at which the majority of the IRB members are present, including one member whose primary concerns are in nonscientific areas. For the research to be approved, it must receive the approval of a majority of those members present at the meeting.
Generalizability: The ability to apply the results of a specific study to groups or situations beyond those actually studied.
Guardian: An individual who is authorized under applicable state or local law to give permission on behalf of a child to general medical care.
Historical Controls: Control participants (followed at some time in the past or whose data are available through records) who are used for comparison with participants being treated concurrently. The study is considered historically controlled when the present condition of participants is compared with their own condition on a prior regimen or treatment.
Human Subject: (See Participant)
Hypothesis: A testable statement of how two or more variables are expected to be related to one another.
Incapacity: Refers to a person’s mental status and means the inability to understand information presented, to appreciate the consequences of acting (or not acting) on that information, and to make a choice (see Incompetence).
Incompetence: Used as a legal term to indicate the inability to manage one’s own affairs.
Independent Variable: The conditions of an experiment that are systematically manipulated by the investigator. A variable that is not impacted by the dependent variable, and that itself impacts the dependent variable. In the earlier example of “sex” and “academic major,” (see Dependent Variable) sex is the independent variable.
In-Depth Interview: A method of data collection in which a participant is interviewed in detail about a certain research participant. In this format, the interviewer leads the discussion flexibly along some pre-structured topics, but also allows the participant to expand upon topics in-depth and to explore new avenues of discussion.
Informant: In field research, a person who is “native” to the social situation being studied, who assists the researcher by providing insider information and serving as a go-between.
Informed Consent: The principle that potential participants are given adequate and accurate information about a study before they are asked to agree to participate, and that they do in fact agree (consent) to participate. In giving informed consent, participants many not waive or appear to waive any of their legal rights, or release or appear to release the investigator, the sponsor, the institution or agents thereof from liability for negligence.
Institutional Review Board (IRB): A specially constituted review body established or designated by an entity to protect the welfare of human participants recruited to participate in biomedical or behavioral research.
Institutionalized: Confined, either voluntarily or involuntarily (e.g., a hospital, prison, or nursing home).
Institutionalized Cognitively Impaired: Persons who are confined, either voluntarily or involuntarily, in a facility for the care of the mentally or otherwise disabled (e.g. a psychiatric hospital, home, or school for the mentally disabled).
Investigator: In clinical trials, the individual who actually conducts the investigation (see Principal Investigator).
Justice: An ethical principle that requires fairness in the distribution of burdens and benefits; often expressed in terms of treating persons of similar circumstances or characteristics similarly (see Equitable).
Legally Authorized Representative: A person authorized either by statute or by court appointment to make decisions on behalf of another person. In human participants research this refers to an individual (or judicial) authorized under applicable law to consent on behalf of a participant to the participant’s participation in the research.
Longitudinal Study: A study in which data are collected from the same sample at least two different times. A study designed to follow participants through time.
Mature Minor: Someone who has not reached adulthood (as defined by state law) but who may be treated as an adult for certain purposes (e.g. consenting to medical care). A mature minor is not necessarily an emancipated minor (see Emancipated Minor).
Minimal Risk: A risk is minimal where the probability and magnitude of harm or discomfort anticipated in the proposed research are not greater, in and of themselves, than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests. The definition of minimal risk for research involving prisoners differs somewhat from that given for non-institutionalized adults (see Risk).
Monitoring: The collection and analysis of data as the project progresses to assure the appropriateness of the research, its design and participant protections.
Multistage Sample: A probability sample that involves several stages (and frequently a cluster sampling stage), such as randomly selecting clusters from a population, then randomly selecting people from each of the clusters.
Non-affiliated Member: Member of an Institutional Review Board who has no ties to the parent institution, its staff, or faculty. This individual is usually from the local community (e.g., minister, business person, attorney, teacher).
Non-probability Sample: A subset of the population chosen in a way that does not give every member of the population a known (nonzero) chance of being selected.
Non-therapeutic Research: Research that has no likelihood or intent of producing a diagnostic, preventive, or therapeutic benefit to the current participants, although it may benefit participants with a similar condition in the future.
Non-response Bias: The bias that results from differences between those who agree to participate in a survey and those who don’t.
Null Hypothesis: The proposition, to be tested statistically, that the experimental intervention has “no effect,” meaning that the treatment and control groups will not differ as a result of the intervention. Investigators usually hope that the data will demonstrate some effect from the intervention, thus allowing the investigator to reject the null hypothesis.
Participant Observation: Observation performed by an observer who takes part in the activities he or she observes.
Non-participant Observation: Observation performed by an observer who remains as distant as possible from those being observed.
Open Design: An experimental design in which both the investigator(s) and the participants know the treatment group(s) to which participants are assigned (see Double-Blind Design and Single-Blind Design).
Open-ended Questions:Survey questions that allow respondents to answer in their own words. (All in-depth interviews use open-ended questions.)
Operational Definition: Statements of the specific ways in which the absence, presence, and/or the degree of presence of a phenomenon will be determined in a specific research process.
Participant: Individuals whose physiological or behavioral characteristics and responses are the object of study in a research project. Under federal regulations, human participants are defined as: living individual(s) about whom an investigator conducting research obtains: (1) data through intervention or interaction with the individual; or (2) identifiable private information.
Paternalism: Making decisions for others against or apart from their wishes with the intent of doing them good.
Permission: The agreement of parent(s) or guardians to the participation of their child or ward in research.
Placebo: A chemically inert substance (e.g., sugar pills) given to control groups as if it were the medicine or treatment for its psychologically suggestive effect; it is used in controlled clinical trials to determine whether improvement and side effects may reflect imagination or anticipation rather than actual power of a drug.
Principal Investigator: The scientist or scholar with primary responsibility for the design and conduct of a research project (see Investigator).
Population: The entire group (or set or type) of people from which a researcher samples, and to which she or he would ideally like to generalize.
Prisoner: An individual involuntarily confined in a penal institution, including persons: (1) sentenced under a criminal or civil statue; (2) detained pending arraignment, trial, or sentencing; and (3) detained in other facilities.
Privacy: A person’s capacity to control the extent, timing, and circumstances of shared oneself (physically, behaviorally, or intellectually) with others.
Probability Sample: A subset of the population chosen in such a way that every member of the population has a known (nonzero) chance of being selected into the sample.
Prospective Studies: Studies designed to observe outcomes or events that occur after the group of participants has been identified. Prospective studies do not have to involve manipulation or intervention but may be purely observational or involve only the collection of data instead.
Protocol: The formal design or plan of an experiment or research activity; specifically, the plan submitted to an IRB for review and to an agency for research support. The protocol includes a description of the research design or methodology to be employed, the eligibility requirements for prospective participants and controls, the treatment regimen(s), and the proposed methods of analysis that will be performed on the collected data.
Qualitative Research: The collection of non-numerical data. Often multi-method in focus, qualitative research involves an interpretive, meaning-driven approach to its participant matter.
Quantitative Research: The collection of numerical data in order to describe, explain, predict and/or control phenomena of interest.
Quasi-experiment: An experimental design that is missing one or more aspects of the (classic) controlled experiment.
Questionnaire: A data collection method in which participants read and answer questions in a written format.
Random Sample: A specific type of probability sample in which participants are selected from a population list using a table of random numbers or a random number generator. (A random sample requires a list of population members in which each member can be assigned a discrete number.) The assignment of participants to different treatments, interventions, or conditions according to chance, rather than systematically. Random assignment of participants increases the probability that differences observed between participant groups are the result of the experimental intervention.
Repeated Measures Design: Same as Within-participants Design.
Reliability: The degree to which a measure yields consistent results.
Remuneration: Payment for participation in research; this is different from compensation, which typically refers to payment for research-related injuries (see Compensation).
Representative Sample: A sample in which the participants closely match the characteristics of the population, and thus, all segments of the population are represented in the sample. A representative sample allows results to be generalized from the sample to the population.
Research: A systematic investigation (i.e., the gathering and analysis of information) designed to develop or contribute to generalizable knowledge.
Respect for Persons: An ethical principle requiring that individual autonomy be respected and that persons with diminished autonomy be protected (see Cognitively Impaired, Incompetence, and Incapacity).
Respondents: Research participants, who fill out a survey, are interviewed, participate in an experiment, are observed in a naturalistic setting, or who are otherwise studied.
Retrospective Studies: Research conducted by reviewing records from the past (e.g., birth and death certificates, medical records, school records, or employment records) or by obtaining information about past events elicited through interviews or surveys. Case studies are an example of this type of research.
Review (of Research): The concurrent oversight of research on a periodic basis by an IRB. In addition to the at least annual reviews mandated by the federal regulations, reviews may, if deemed appropriate, also be conducted on a continuous or periodic basis.
Risk: The probability of harm or injury (physical, psychological, social, or economic) occurring as a result of participation in a research study. Both the probability and magnitude of possible harm may vary from minimal to significant (see Minimal Risk).
Sample: A subset of a given population used for research purposes.
Selection Bias: A bias in the way the experimental and control or comparison groups are selected, resulting in pre-existing differences between the groups that may serve as confounding factors.
Single-Blind Design: Typically, a study design in which the investigator, but not the participant, knows the identity of the treatment assignment. Occasionally the participant, but not the investigator, knows the assignment. Also known as Single-Masked Design (see Double-Blind Design and Open Design).
Single-Masked Design: See Single-Blind Design.
Site Visit: A visit by agency officials, representatives, or consultants to the location of a research activity to assess the adequacy of IRB protection of human participants or the capability of personnel to conduct the research.
Snowball Sample: A non-probability sample that is created by using members of the group of interest to identify other members of the group (for example, asking a participant at the end of an interview for suggestions about who else to interview).
Social Experimentation: Systematic manipulation of, or experimentation in, social or economic systems; used in planning public policy.
Statistical Tests: Researchers use statistical tests to make quantitative decisions about whether a study’s data indicate a significant effect from the intervention and allow the researcher to reject the null hypothesis. That is, statistical tests show whether the differences between the outcomes of the control and experimental groups are great enough to be statistically significant. If differences are found to be statistically significant, it means that the probability (or likelihood) that these differences occurred solely due to chance is relatively low. Most researchers agree that a significance value of .05 or less (there is a 95% probability that the differences are real) sufficiently determines significance.
Statistical Significance: The probability that difference between the outcomes of the control and experimental group are great enough that it is unlikely it is due solely to chance. The probability that the null hypothesis can be rejected at a predetermined significance level (0.05 or 0.01).
Stratified Sample: A probability sample that is determined by dividing the population into groups or strata defined by the presence of certain characteristics and then random sampling from each of the strata. This is a good way to make sure that a student sample is racially diverse (for instance).
Structured Interview: A data collection method in which an interviewer reads a standardized interview schedule to the respondent and records the answers. (Not to be confused with an in-depth interview.)
Survey: A study in which the same data are collected from all members of the sample using a highly structured questionnaire and analyzed using statistical tests.
Systematic Sample:A probability sample that is determined by selecting every ‘nth’ (5 th, 10 th, 50 th, etc.) person from a list of the entire population, after the first person has been randomly selected.
Theory: A general explanation about a specific behavior or set of events that is based on known principles and serves to organize related events in a meaningful way. A theory is not as specific as a hypothesis.
Treatment Group: Same as Experimental Group.
Type I Error: When a test wrongly shows an effect or condition to be present (e.g. that a woman is pregnant when, in fact, she is not). When a researcher falsely rejects the null hypothesis (see False Positive).
Type II Error: When a test wrongly shows an effect or condition to be absent (e.g. that a woman is not pregnant when, in fact, she is). When a researcher fails to reject the null hypothesis (see False Negative).
Validity: The degree to which a measure assesses what we think it is assessing. There are different types of validity, including:
Internal validity (does a study’s conclusions about causal relationships agree with what is actually true?)
Face validity (at face value, does the measure seem valid?)
Construct validity (does the measure of a given concept relate to a measure of another theoretically associated concept?)
Content validity (does the measure cover diverse meanings of the concept?)
Criterion validity (is the measure associated with expected behaviors?)
Variable: Any characteristic or trait that can vary from one person to another (race, sex, academic major) or for one person over time (age, political beliefs).
Voluntary Participation: The principle that study participants choose to participate of their own free will, rather than being coerced or forced to participate. For IRB purposes, this is a key part of your study proposal; you must demonstrate that participants will be participating voluntarily for a study to be approved by the IRB.
Within-participants Design: A research design in which each participant experiences, at different times, all levels of the independent variable (or both the experimental and control treatment). Thus, each participant is tested once in each condition (see Cross-Over Design and Repeated Measures Design).
Random assignment or random placement is an experimental technique for assigning human participants or animal subjects to different groups in an experiment (e.g., a treatment group versus a control group) using randomization, such as by a chance procedure (e.g., flipping a coin) or a random number generator. This ensures that each participant or subject has an equal chance of being placed in any group. Random assignment of participants helps to ensure that any differences between and within the groups are not systematic at the outset of the experiment. Thus, any differences between groups recorded at the end of the experiment can be more confidently attributed to the experimental procedures or treatment.
Random assignment, blinding, and controlling are key aspects of the design of experiments, because they help ensure that the results are not spurious or deceptive via confounding. This is why randomized controlled trials are vital in clinical research, especially ones that can be double-blinded and placebo-controlled.
Mathematically, there are distinctions between randomization, pseudorandomization, and quasirandomization, as well as between random number generators and pseudorandom number generators. How much these differences matter in experiments (such as clinical trials) is a matter of trial design and statistical rigor, which affect evidence grading. Studies done with pseudo- or quasirandomization are usually given nearly the same weight as those with true randomization but are viewed with a bit more caution.
Benefits of random assignment
Imagine an experiment in which the participants are not randomly assigned; perhaps the first 10 people to arrive are assigned to the Experimental Group, and the last 10 people to arrive are assigned to the Control group. At the end of the experiment, the experimenter finds differences between the Experimental group and the Control group, and claims these differences are a result of the experimental procedure. However, they also may be due to some other preexisting attribute of the participants, e.g. people who arrive early versus people who arrive late.
Imagine the experimenter instead uses a coin flip to randomly assign participants. If the coin lands heads-up, the participant is assigned to the Experimental Group. If the coin lands tails-up, the participant is assigned to the Control Group. At the end of the experiment, the experimenter finds differences between the Experimental group and the Control group. Because each participant had an equal chance of being placed in any group, it is unlikely the differences could be attributable to some other preexisting attribute of the participant, e.g. those who arrived on time versus late.
Random assignment does not guarantee that the groups are matched or equivalent. The groups may still differ on some preexisting attribute due to chance. The use of random assignment cannot eliminate this possibility, but it greatly reduces it.
To express this same idea statistically - If a randomly assigned group is compared to the mean it may be discovered that they differ, even though they were assigned from the same group. If a test of statistical significance is applied to randomly assigned groups to test the difference between sample means against the null hypothesis that they are equal to the same population mean (i.e., population mean of differences = 0), given the probability distribution, the null hypothesis will sometimes be "rejected," that is, deemed not plausible. That is, the groups will be sufficiently different on the variable tested to conclude statistically that they did not come from the same population, even though, procedurally, they were assigned from the same total group. For example, using random assignment may create an assignment to groups that has 20 blue-eyed people and 5 brown-eyed people in one group. This is a rare event under random assignment, but it could happen, and when it does it might add some doubt to the causal agent in the experimental hypothesis.
Random sampling is a related, but distinct process. Random sampling is recruiting participants in a way that they represent a larger population. Because most basic statistical tests require the hypothesis of an independent randomly sampled population, random assignment is the desired assignment method because it provides control for all attributes of the members of the samples—in contrast to matching on only one or more variables—and provides the mathematical basis for estimating the likelihood of group equivalence for characteristics one is interested in, both for pretreatment checks on equivalence and the evaluation of post treatment results using inferential statistics. More advanced statistical modeling can be used to adapt the inference to the sampling method.
Randomization was emphasized in the theory of statistical inference of Charles S. Peirce in "Illustrations of the Logic of Science" (1877–1878) and "A Theory of Probable Inference" (1883). Peirce applied randomization in the Peirce-Jastrow experiment on weight perception.
Charles S. Peirce randomly assigned volunteers to a blinded, repeated-measures design to evaluate their ability to discriminate weights. Peirce's experiment inspired other researchers in psychology and education, which developed a research tradition of randomized experiments in laboratories and specialized textbooks in the eighteen-hundreds.
Jerzy Neyman advocated randomization in survey sampling (1934) and in experiments (1923).Ronald A. Fisher advocated randomization in his book on experimental design (1935).
- ^ abhttp://www.socialresearchmethods.net/kb/random.php.
- ^ abCharles Sanders Peirce and Joseph Jastrow (1885). "On Small Differences in Sensation". Memoirs of the National Academy of Sciences. 3: 73–83.
- ^ abIan Hacking (September 1988). "Telepathy: Origins of Randomization in Experimental Design". Isis (A Special Issue on Artifact and Experiment). 79 (3): 427–451. doi:10.1086/354775.
- ^ abStephen M. Stigler (November 1992). "A Historical View of Statistical Concepts in Psychology and Educational Research". American Journal of Education. 101 (1): 60–70. doi:10.1086/444032.
- ^ abTrudy Dehue (December 1997). "Deception, Efficiency, and Random Groups: Psychology and the Gradual Origination of the Random Group Design". Isis. 88 (4): 653–673. doi:10.1086/383850. PMID 9519574.
- ^Neyman, Jerzy (1990) , Dabrowska, Dorota M.; Speed, Terence P., eds., "On the application of probability theory to agricultural experiments: Essay on principles (Section 9)", Statistical Science (Translated from (1923) Polish ed.), 5 (4): 465–472, doi:10.1214/ss/1177012031, MR 1092986
- Caliński, Tadeusz & Kageyama, Sanpei (2000). Block designs: A Randomization approach, Volume I: Analysis. Lecture Notes in Statistics. 150. New York: Springer-Verlag. ISBN 0-387-98578-6.
- Hinkelmann, Klaus and Kempthorne, Oscar (2008). Design and Analysis of Experiments. I and II (Second ed.). Wiley. ISBN 978-0-470-38551-7.
- Charles S. Peirce, "Illustrations of the Logic of Science" (1877–1878)
- Charles S. Peirce, "A Theory of Probable Inference" (1883)
- Charles Sanders Peirce and Joseph Jastrow (1885). "On Small Differences in Sensation". Memoirs of the National Academy of Sciences. 3: 73–83. http://psychclassics.yorku.ca/Peirce/small-diffs.htm
- Hacking, Ian (September 1988). "Telepathy: Origins of Randomization in Experimental Design". Isis. 79 (3): 427–451. doi:10.1086/354775. JSTOR 234674. MR 1013489.
- Stephen M. Stigler (November 1992). "A Historical View of Statistical Concepts in Psychology and Educational Research". American Journal of Education. 101 (1): 60–70. doi:10.1086/444032.
- Trudy Dehue (December 1997). "Deception, Efficiency, and Random Groups: Psychology and the Gradual Origination of the Random Group Design". Isis. 88 (4): 653–673. doi:10.1086/383850. PMID 9519574.
- Basic Psychology by Gleitman, Fridlund, and Reisberg.
- "What statistical testing is, and what it is not," Journal of Experimental Education, 1993, vol 61, pp. 293–316 by Shaver.