By:
C. Nathan DeWall, University of Kentucky
David G. Myers, Hope College
Note: You will be guided through the Intro, Design, Measure, Interpret, Conclusion, and Quiz sections of this activity. You can see your progress highlighted in the non-clickable, navigational list at the right.
Watch this video from your author, David Myers, for a helpful, very brief overview of the activity.
So, how would you know if astrologers can describe people’s personality? To study this question effectively in your role as researcher, you need to DESIGN an appropriate study that will lead to meaningful results, MEASURE personality, and INTERPRET the larger meaning of your results, considering how your findings would apply to the population as a whole.
Click on "Video Hint" below to see brief animations describing Experiments, Case Studies, and Naturalistic Observation.
Experiments:
Case Studies:
Naturalistic Observation:
You have chosen a Case Study design, which means that you want to study one individual or a small group of individuals in depth. For example, researchers who want to know whether introverted people make good leaders might identify a small group of introverted people and spend months or years conducting interviews and personality assessments, and measuring leadership success.
What can we learn from these participant choices?
a. A small group of people who are extremely high on the trait of extraversion This is not the best option. To know whether astrologers can describe people’s personality, you would need to study individuals from the general population who have different levels of many personality traits.
b. A small group of people who have a normal range of personality trait scores You were right to select a group of individuals from the general population who have different levels of many personality traits. However, you need to study a larger group, and not a small group as in this Case Study approach, in order to determine whether the idea of astrologers describing people’s personality may apply to the larger population.
c. A small group of people who are extremely low on the trait of agreeableness, and another group of people who are extremely high on the trait of agreeableness This is not the best option, because you are only focusing on people who have extreme scores on a single personality trait. You also need to include people who have average levels of that trait, and many others. To know whether astrologers can describe people’s personality, you would need to study individuals from the general population who have different levels of many personality traits.
d. A small group of people who are extremely high on the trait of extraversion, and another group of people who are extremely low on the trait of extraversion This is not the best option, because you are only focusing on people who have extreme scores on a single personality trait. You also need to include people who have average levels of that trait, and many others. To know whether astrologers can describe people’s personality, you would need to study individuals from the general population who have different levels of many personality traits.
Trying to choose a sample of participants helps us realize that the CASE STUDY IS NOT THE BEST RESEARCH DESIGN to test this question. We’d get more helpful results by systematically comparing an astrologer’s descriptions of people’s personality traits with the descriptions provided by the people’s close friends. This would help us determine whether astrologers can describe people’s personality.
Click “Next” to go back and try again to select the most effective research design.
You have chosen Naturalistic Observation as your design, which means you want to carefully observe one or more participants in your study and make notes about what they do. For example, researchers seeking to learn about stockbrokers have spent many thousands of hours observing stockbrokers at work and at home, carefully tracking their behaviors and watching for meaningful patterns.
You might use naturalistic observation to gather detailed information about one or more astrologers and how they rate people’s personalities. Do you think it would be appropriate to apply those findings to a larger population? Also, this method would not provide you with a base of comparison, such as ratings made by a close friend, for determining the accuracy of the astrologer’s ratings.
So, NATURALISTIC OBSERVATION IS NOT THE BEST RESEARCH DESIGN for this study.
Click “Next” to go back and try again to select the most effective research design.
Nice work! You have correctly chosen to use an Experimental design for your study.
Next, you need to choose the most appropriate study participants.
You chose People who score extremely high on the trait of extraversion, but this is NOT CORRECT. To know whether astrologers can describe people’s personality, you would need to study people who have different levels of many personality traits.
Click “Next” to try again to choose the most appropriate study participants.
You chose A group of people who score extremely low on the trait of agreeableness, and another group of people who score extremely high on the trait of agreeableness, but this is NOT CORRECT. This is not the best option, because you are only focusing on people who have extreme scores on a single personality trait. You need to include people who have average levels of that trait. To provide the most comprehensive test of your research question, you should also measure more than one personality trait. To know whether astrologers can describe people’s personality, you would need to study people who have different levels of many personality traits.
Click “Next” to try again to choose the most appropriate study participants.
You chose A group of people who score extremely high on the trait of extraversion, and another group of people who score extremely low on the trait of extraversion, but this is NOT CORRECT. This is not the best option, because you are only focusing on people who have extreme scores on a single personality trait. You also need to include people who have average levels of that trait, and many others. To know whether astrologers can describe people’s personality, you would need to study people who have different levels of many personality traits.
Click “Next” to try again to choose the most appropriate study participants.
Good job! You have correctly chosen to use an Experimental design. You also chose an appropriate sample of participants—a group of people who have a normal range of personality trait scores. Now you need to determine how best to MEASURE the relevant behavior or mental process, which in this case is personality traits.
Before we proceed, recall that personality has five main traits, as noted in the table below:
Conscientiousness
Agreeableness
Neuroticism (or Emotional Stability)
Openness to experience
Extraversion
Researchers refer to these traits as the Big Five (Costa & McCrae, 2009).
For the purposes of our experiment, let’s say that the astrologers were given a participant’s birth date—in order to determine astrological sign.
You chose Have participants complete a Big Five personality questionnaire based on how they think others perceive them, and compare that to an astrologer’s rating, but that is NOT CORRECT.
This option would not give you information about participants’ personalities. It would only tell you how participants believe others perceive them. To know whether astrologers can describe people’s personality, we have to measure participants’ actual personality.
Click “Next” to try again to select the best way to measure whether astrologers can describe people’s personality.
You chose Have participants complete a Big Five personality questionnaire. Compare their scores to those obtained from two of their closest friends, but that is NOT CORRECT.
This option would not give you any information about how an astrologer would rate your participants’ personalities. You would only know how well participants’ ratings of their own personalities correlated with how their close friends perceived them. To know whether astrologers can describe people’s personality, we need to have an astrologer rate participants’ personalities.
Click “Next” to try again to select the best way to measure whether astrologers can describe people’s personality.
You chose Have participants complete a Big Five personality questionnaire based on how they think an astrologer would rate them, but that is NOT CORRECT.
By choosing this option, you did not measure your participants’ personalities. Instead, you asked participants to report how they think an astrologer would perceive them. To know whether astrologers can describe people’s personality, we have to measure participants’ actual personality.
Click “Next” to try again to select the best way to measure whether astrologers can describe people’s personality.
Nice job! You have correctly chosen to use an Experimental design. You chose an appropriate sample of participants—a group of people who have a normal range of personality trait scores. You also chose how best to MEASURE the relevant behavior or mental process, which in this case is personality traits. You selected the option, Have participants complete a Big Five personality questionnaire. Compare their scores to either those obtained from an astrologer, or to those from a close friend. This means you will randomly assign some people to have their personality rated by an astrologer, and other people to have their personality rated by a close friend, to see which group gets closer to the accurate rating provided by the Big Five questionnaire.
Numerous similar studies have shown that astrologers cannot accurately describe people’s personality (Carlson, 1985; Kelly, 1997; Reichardt, 2010). Researchers have also studied related concepts. In one study of 20 million people, for example, astrological sign had “no impact on the probability of marrying—and staying married to—someone of any other sign” (Voas, 2008). Who understands our personality? Close friends and family. In fact, some evidence suggests that our friends know important facts about us better than we do (Jackson et al., 2015).
Knowing this, you need to consider how you can apply what you’ve learned to the larger population—beyond the people you’ve studied. Consider where you might encounter roadblocks to confidence in your results. What factors might keep you from being able to apply what you’ve learned in a broader context?
You tested whether astrologers can describe personality. You had participants complete a Big Five personality questionnaire. Those personality scores were then compared with scores made by either an astrologer or a close friend. Several factors may influence your study results, and they might not have any relation to whether an astrologer or a close friend rated your participants’ personalities! Factors that could interfere with our INTERPRETATION of results are called confounding variables.
From the options below, select all of the factors that could affect your confidence about whether astrologers can describe people’s personality:
ZycNTMvpRoSnXPUZl8FiVl5s+iDEL2Bo | Whether the astrologer met the participant or only used the participant’s astrological sign |
ZycNTMvpRoSnXPUZl8FiVl5s+iDEL2Bo | Investigator’s belief in astrology |
3jskoxIwaHnoYr4Fsi2l/v3etGMIufia | Number of times participant has completed online personality questionnaires |
ZycNTMvpRoSnXPUZl8FiVl5s+iDEL2Bo | Whether participants were having a bad day when they completed the personality questionnaire |
3jskoxIwaHnoYr4Fsi2l/v3etGMIufia | The number of hours per week participants spend watching YouTube videos |
Click on "Video Hint" below to see a brief animation describing Confounding Variables.
Confounding Variables:
Confounding Variables:
The confounding variables for your study would include those highlighted below:
Whether the astrologer met the participant or only used the participant’s astrological sign | |
Investigator’s belief in astrology | |
Number of times participant has completed online personality questionnaires | |
Whether participants were having a bad day when they completed the personality questionnaire | |
The number of hours per week participants spend watching YouTube videos |
To maximize confidence in your results, you need to keep track of the highlighted confounding variables. Remember that you are trying to predict whether astrologers can describe people’s personality. We can tell a lot about someone from even a short interaction. If we do not account for whether the astrologer met the participant, we might draw incorrect inferences about what influenced the astrologer’s personality ratings. Having strong beliefs in astrology may also bias your results. In addition, we need to keep track of participants’ mood when they complete the personality questionnaire. If participants are angry or sad when they complete the personality questionnaire, their results might represent their current mood instead of their stable personality traits. Time spent online completing other personality questionnaires, or watching YouTube videos, shouldn’t affect the study results.
Note that by randomly assigning participants to either the astrologer or good friend condition, you control for many other possible confounding variables. With random assignment, each participant has the same chance of being assigned to each group. This should give each of your groups a balanced number of participants expressing these variations.
Click on "Video Hint" below to see a brief animation describing Random Assignment.
Random Assignment:
You may do better on the Quiz if you take notes while watching this video. Feel free to pause the video or re-watch it as often as you like.
REFERENCES
Carlson, S. (1985). A double-blind test of astrology. Nature, 318, 419–425.
Costa, P. T., Jr., & McCrae, R. R. (2009). The five-factor model and the NEO inventories. In J. N. Butcher (ed.), Oxford handbook of personality assessment. New York, NY, US: Oxford University Press.
Jackson, J. J., Connolly, J. J., Garrison, S. M., Levelle, M. M., & Connolly, S. L. (2015). Your friends know how long you will live: A 75-year study of peer-rated personality traits. Psychological Science, 26, 335-340.
Kelly, I. W. (1997). Modern astrology: A critique. Psychological Reports, 81, 1035–1066.
McCrae, R. R., & Costa, P. T., Jr. (1986). Clinical assessment can benefit from recent advances in personality psychology. American Psychologist, 41, 1001–1003.
McCrae, R. R., & Costa, P. T., Jr. (2008). The Five-Factor Theory of personality.
In O. P. John, R. W., Robins, & L. A. Pervin (eds.), Handbook of personality: Theory and research, 3rd edition. New York: Guilford.
QUIZ: NOW WHAT DO YOU KNOW?