R.W. Schwester
© 2015
All Rights Reserved
Except from Teaching Research Methods in Public Administration
MEASURING VARIABLES
What Does Measurement Entail?
In simplest terms, measurement involves applying numbers or labels to variables.
Researchers use the six primary empirical tools discussed in Chapter 1 to measure the variables
that were identified and defined in Chapter 3. Those empirical tools include:
1) Surveys
2) Interviews
3) Focus groups
4) Observation
5) Experimentation
6) Existing Data
Recall that surveys, experimentation, and existing data are deductive measurement tools. They
are best suited for quantitative hypothesis testing. On the other hand, in-depth interviews, focus
groups, and field observation are better suited for inductive measurement, as they are principally
used to generate hypotheses. For our purposes in this chapter, a cursory overview of
measurement will be presented through the lenses of survey research, experimentation, and the
use of existing data resources.
Survey Example
Recall that surveys consist of specific questions that provide the respondent with
predetermined option choices – or a finite number of possible choices. Surveys tell researchers
the “what” or “how much” of something. Consider our recurring example (Figure 4.1) – Do
students that attend research methods class more frequently perform better than students that
attend class less frequently?
Figure 4.1
Schwester 2014
The UoA is students given the structure of the research question. The independent
variable “attendance” is defined as: physically coming to class, arriving on time, and staying for
the entire class. The dependent variable “performance” could be defined as: the extent to which a
student masters course exercises (e.g., assignments and examinations). Ultimately, in order to
measure a variable, a researcher must determine which empirical tool best allows us to apply
labels or numbers to our IV attendance and our DV performance.
Consider this: if you wanted to know how many classes your friend missed and how well
that same friend did in his or her research methods course, how would you go about obtaining
this information? In this case, you might ask specific and narrow questions – the empirical tool
known as surveys. The survey questions themselves are how you actually measure your
independent variable attendance and your dependent variable performance.
The nature and structure of the survey questions should be derived from the variable’s
definition. The independent variable “attendance” is defined as: physically coming to class,
arriving on time, and staying for the entire class. Therefore, the survey question – or questions –
needs to incorporate all aspects of this definition. Survey question measuring the variable
attendance might include:
Of your 15 total research methods classes1 throughout the semester, how many
did you physically attend?
Of your 15 total research methods classes throughout the semester, how many did
you arrive on time?
Of your 15 total research methods classes throughout the semester, how many did
you leave before the completion of the class period?
The DV performance is defined as: the extent to which a student masters course exercises (e.g.,
assignments and examinations). Based on this definition, the DV performance could be measured
through the following survey question:
What was your final course grade in research methods?
Since mastery of course exercises is embodied in a student’s final grade, the question that
measures the variable performance could ask simply about the student’s grade.
1 In my graduate research methods sections, there are 15 total classes – one per week for 15
weeks. This number will vary, of course, based on the program.
For this example, in Chapter 3 we defined the extraneous variable professor quality as:
having an expert grasp of the course material; presenting the course material in a clear and
organized way; being available to students; answering emails in a timely fashion and addressing
questions effectively. Remember, when crafting your own questions to measure a variable, use
the body of the definition to guide the content and even structure of the question. Given the
definition of professor quality, we need questions that deal with five separate yet in some cases
inter-related parts.
(1) Having an expert grasp of the course material; (2) presenting the course
material in a clear and organized way; (3) being available to students; (4)
answering emails in a timely fashion and (5) addressing questions effectively.
Based on this, survey questions that measure the variable professor quality might include
the following:
On a scale of 1 to 10 – with 10 being the best and 1 being the worst – how
would you rate your professor’s knowledge of the course material?
On a scale of 1 to 10 – with 10 being the best and 1 being the worst – how
would you rate the clarity of your professor’s lectures?
On a scale of 1 to 10 – with 10 being the best and 1 being the worst – how
would you rate the organization of your professor’s lectures?
On a scale of 1 to 10 – with 10 being the best and 1 being the worst – how
would you rate your professor’s availability to students?
On a scale of 1 to 10 – with 10 being very quickly and 1 being very slowly
– how would judge the speed at which your professor responds to emails?
On a scale of 1 to 10 – with 10 being the best and 1 being the worst – how
would you rate how well your professor responds to questions in class?
The questions presented above address each part of the definition of professor quality. Now, as
we will see when we discuss how to write survey questions in Chapter 5, there are many ways to
express a concept within the body of a question. However, what is important to see here is that
when writing survey questions from scratch use the definition of the variable as your guide.
Experimentation Example
Social science methods of experimentation follow the logic of natural science
experiments. Experimentation involves manipulation and control. It involves taking one group of
individuals, known as the treatment group, and exposing that group to some sort of external
stimuli. The treatment group is compared to another group known as the control group. This
group is NOT exposed to the external stimuli. For our experimentation example, consider this:
Does exposure to violent video games increase the likelihood of aggressive behavior among
teenagers? The UoA is teenagers. Since video game play presumably impacts aggressive
behavior, this makes video game play the independent variable and aggressive behavior the
dependent variable.
Box 4.1
The independent variable violent video game exposure is considered to be external stimuli, and
therefore it will be measured using an experiment. The IV will be measured through the
experimental process – that is, the process of taking x number of individuals, in this case
teenagers, and splitting them up into at least two groups. The treatment group – sometimes
referred to as the experimental group – will receive the stimuli. Therefore, everyone who is
selected for the treatment group will be exposed to violent video games. By exposure we mean
this: the x number of teenagers within the treatment group will play violent video games for a
predetermined amount of time at some predetermined interval (e.g., two hours per day for three
months). The two groups will then be compared based on the dependent variable. In other words,
the frequency of aggressive behavior of teenagers within the treatment group will ultimately be
compared to the frequency of aggressive behavior of teenagers within the control group (the
group that did not play violent video games). Figure 4.2 illustrates this. It is the process of
(IV) Violent Video
Game Exposure
The IV violent video game exposure is defined as:
Playing video games that are rated as having
“violent content” by the Entertainment Rating
Software Board.
actually splitting up a group of teenagers and putting half in a treatment group and half in a
control group that measures the independent variable violent video game exposure.
Figure 4.2
Schwester 2014
The dependent variable aggressive behavior will be measured using another empirical
tool. Let us adopt the definition of aggressive behavior in Box 4.1. Without question, there are a
myriad of ways to define aggressive behavior. A researcher may develop his or her own
definition, or adopt one from the literature, as we have done here.
I want to briefly differentiate between the experimental process and experimentation as
an empirical measurement tool. As stated previously, the experimental process for measuring an
independent variable involves splitting up your unit of analysis into at least two groups, and
exposing one of those groups to some kind of external stimuli – i.e., violent video games.
Experimentation as an empirical tool not only includes the experimental process, but also at least
one more empirical tool to measure the dependent variable and extraneous variables. So, in order
to measure the dependent variable aggressive behavior, we are going to need to identify another
empirical tool that can be coupled with the experimental process that is measuring the
independent variable violent video game exposure. One way to measure aggressive behavior is
the same way we measured attendance, performance, and professor quality in the previous
example – survey questions. Given our definition of aggressive behavior, we can craft survey
question to capture this.
The dependent variable aggressive behavior is defined as: “Starting a fight over nothing”
or “pushes and shoves others” (Huesmann & Eron, 1986).
(DV) Aggressive
Behavior
The DV aggressive behavior is defined as: “Starting a
fight over nothing” or “pushes and shoves others”
(Huesmann & Eron, 1986).
Box 4.2
The DV aggressive behavior could be measured using the following survey questions:
How often does your son (daughter) start fights over nothing?
A) Frequently; B) Sometimes; C) Seldom; D) Never; E) Do Not Know
How often does your son (daughter) push and shove others?
A) Frequently; B) Sometimes; C) Seldom; D) Never; E) Do Not Know
Figure 4.3
Schwester 2014
Let us consider another experimentation example — Do students that attend research
methods class more frequently perform better than students that attend class less frequently? At
this point, those who have been reading carefully might be somewhat confused. Yes, this
example has been used to illustrate the use of survey questions to measure variables. However,
this is an example where experimentation could be used as well. “Class attendance” is the
external stimuli here. Let us assume that we have an eight week research methods boot camp,
whereby there will be eight total lectures. The treatment group will attend the eight lectures,
while the control group will not attend the eight lectures. This experimental process of splitting
up research methods student into two groups – the treatment and control groups – and having one
group attend lectures over an eight week period measures the IV attendance. In our violent video
game – aggressive behavior example, we used a survey to measure the dependent variable. In
this example, we might think of an alternative way to measure performance. An exercise or
examination that tests knowledge gained over the eight week research methods boot camp would
work. In other words, you could have written examination that asks students to discuss and
define the finer points of research methods. For the extraneous variables, a survey could work.
For example, professor quality could be measured using the survey questions presented
previously, and a question could be developed that very easily gauges one’s study habits and
frequency of studying.
All things being equal, experimentation compared to survey research alone is a preferable
method. Experimentation allows for the cultivation of a controlled environment where
subjectivity can be minimized. Consider this: the major disadvantage of survey research is that
people can lie, and there is very little a researcher can do about this. You can attempt to write
and structure questions in a way that makes people feel comfortable, but ultimately a survey
researcher is at the mercy of people and their truthfulness. A survey respondent could lie about
the number of classes he or she attended (or simply be unaware), or inflate their final grade
because they felt shortchanged by the professor. With experimentation, you can verify if
someone has attended the class session – i.e., those within the treatment group – and you can
objectively evaluate how well someone does on the examination or exercise that measures the
dependent variable performance, as opposed to relying on someone’s accurate recollection of
their class grade or truthfulness. This is precisely why experimentation, all things being equal, is
preferable to survey research should a researcher have the option of using either method. Most
research questions, however, dictate that one method be used.
Figure 4.4
Schwester 2014
Existing Data Example
Surveys and experimentation are measurement tools that deal with collection of original
data. Before a researcher conducts a survey or experiment, the data do not exist. With existing
data, however, “someone else” has collected the data. The key here is that a researcher must
“find” these existing data. Often overlooked is the fact that scientifically reliable and valid data
may already exist. Consider this example – Does an increase in the poverty rate lead to an
increase in New Jersey municipal crime rates? The relationship in this deductive research
question is poverty rate and its presumed impact on crime rates. The relationship applies to
places, more specifically municipalities. Thus, the UoA is municipalities and the independent
and dependent variables are poverty rate and crime rate, respectively. Given our UoA is New
Jersey municipalities, we need to find existing data sources for poverty rates and crime rates. Let
us start with poverty rates. The U.S. Census Bureau collects poverty rate data, and these data
were found through a search engine key word search. That is, I searched using the phrase
“poverty rates in New Jersey municipalities.” This led me to income and poverty data that are
housed within the Department of Labor and Workforce Development website for the State of
New Jersey (see Figure 4.5).
Figure 4.5
In Figure 4.5, you can see toward the bottom of the screen “Poverty Status in 2010” (highlighted
in blue). By clicking on that link, you are brought to another webpage presenting links to New
Jersey county and municipal poverty data (see Figure 4.6).
Figure 4.6
At the bottom of Figure 4.6, there is a link (highlighted in blue) that reads “Poverty Status in the
Past 12 Months of Households by Household Type by Age of Householder: New Jersey,
Counties and Municipalities.” Clicking on this link opens an Excel spreadsheet with a myriad of
data, some of which is household poverty data at the municipal level. This household poverty
data can be used to measure the independent variable poverty rate.
In Figure 4.7 below, the yellow highlighted column represents the number of households
within each New Jersey’s municipality that have incomes below the federal poverty line. The
green highlighted column represents the total number of households within each municipality. If
the independent variable is poverty rate – which implies a proportion or percentage – then by
taking the data in the yellow column and dividing by the data in the green column we can
extrapolate the poverty rate for each New Jersey municipality. In short, the yellow column’s data
divided by green column’s data equals the household poverty rate for New Jersey municipalities.
For example, for the municipality of Absecon, 251 households of 3,200 total households are
considered to be in poverty. Therefore, 251 divided 3,200 equals 0.078, which you then multiply
by 100 to get the percentage. The poverty rated of Absecon is 7.8 percent.
Figure 4.7
For the dependent variable crime rates, we take the same approach to finding an existing
data source to measure this variable. Using the search engine phrase “New Jersey municipal
crime rates,” I was directed to the New Jersey State Police (NJSP) website. On the main page of
the NJSP website, there is a drop down menu under the heading “Public Information.” When
clicking on this drop down menu, you will see a sub-heading entitled “Crime Reports and Stats”
(highlighted in blue) (see Figure 4.8).
Figure 4.8
When clicking on the sub-heading “Crime Reports and Stats,” you are directed to the
Uniform Crime Reports (UCR) of the Federal Bureau of Investigation (see Figure 4.9).
Figure 4.9
When clicking on the 2011 UCR link, you are directed to the data’s table of contents. If you
scroll down the page, you will see a sub-heading that reads “New Jersey Municipal-County
Offense and Demographic Data.” Clicking on this link leads you to the municipal crime data (see
Figure 4.10).
Figure 4.10
Uniform Crime Reports, New Jersey Municipalities
The approaches taken to find existing data source to measure the independent and
dependent variables would be identical to the approaches taken to measure any extraneous
variables relevant to a research question that examines the relationship between poverty rates and
crime rates.
Measurement Reliability and Validity
The concepts of reliability and validity are the primary means of determining whether a
measure is “sound.” Let us first begin with the concept of validity. In the context social science
research, and the measurement of variables more specifically, validity refers to the extent to
which the operational definition of a variable is reflected by the measure. In other words, does
the measure capture the concept imbedded within the definition? If so, the measure is thought to
be valid. One way of assessing validity is via a subjective assessment method known as face
validity. Face validity is based on the premise that, according to the judgment of the researcher,
the measure captures a variable’s concept as dictated by its operational definition. This is the
least rigorous method of assessing validity, and it is usually reserved for measures where there is
little ambiguity as to what concepts the measure is meant to capture – i.e., demographic variables
such as age, gender, and income.
The concept of reliability centers of the consistency and repeatability of a measure.
Assessing the reliability of a measure entails taking multiple measurements of a given measure
using the same research subjects each time. If the measures taken at different points in time
match closely – that is, if they are highly correlated – then you can reasonably assume that the
measure is reliable.
Figure 4.11
The Concept of Measurement Reliability and Validity
A B
C D
A = reliable and valid; B = reliable but not valid; C = valid but not reliable; D = neither valid nor
reliable
Schwester 2014
Are you busy and do not have time to handle your assignment? Are you scared that your paper will not make the grade? Do you have responsibilities that may hinder you from turning in your assignment on time? Are you tired and can barely handle your assignment? Are your grades inconsistent?
Whichever your reason is, it is valid! You can get professional academic help from our service at affordable rates. We have a team of professional academic writers who can handle all your assignments.
Students barely have time to read. We got you! Have your literature essay or book review written without having the hassle of reading the book. You can get your literature paper custom-written for you by our literature specialists.
Do you struggle with finance? No need to torture yourself if finance is not your cup of tea. You can order your finance paper from our academic writing service and get 100% original work from competent finance experts.
Computer science is a tough subject. Fortunately, our computer science experts are up to the match. No need to stress and have sleepless nights. Our academic writers will tackle all your computer science assignments and deliver them on time. Let us handle all your python, java, ruby, JavaScript, php , C+ assignments!
While psychology may be an interesting subject, you may lack sufficient time to handle your assignments. Don’t despair; by using our academic writing service, you can be assured of perfect grades. Moreover, your grades will be consistent.
Engineering is quite a demanding subject. Students face a lot of pressure and barely have enough time to do what they love to do. Our academic writing service got you covered! Our engineering specialists follow the paper instructions and ensure timely delivery of the paper.
In the nursing course, you may have difficulties with literature reviews, annotated bibliographies, critical essays, and other assignments. Our nursing assignment writers will offer you professional nursing paper help at low prices.
Truth be told, sociology papers can be quite exhausting. Our academic writing service relieves you of fatigue, pressure, and stress. You can relax and have peace of mind as our academic writers handle your sociology assignment.
We take pride in having some of the best business writers in the industry. Our business writers have a lot of experience in the field. They are reliable, and you can be assured of a high-grade paper. They are able to handle business papers of any subject, length, deadline, and difficulty!
We boast of having some of the most experienced statistics experts in the industry. Our statistics experts have diverse skills, expertise, and knowledge to handle any kind of assignment. They have access to all kinds of software to get your assignment done.
Writing a law essay may prove to be an insurmountable obstacle, especially when you need to know the peculiarities of the legislative framework. Take advantage of our top-notch law specialists and get superb grades and 100% satisfaction.
We have highlighted some of the most popular subjects we handle above. Those are just a tip of the iceberg. We deal in all academic disciplines since our writers are as diverse. They have been drawn from across all disciplines, and orders are assigned to those writers believed to be the best in the field. In a nutshell, there is no task we cannot handle; all you need to do is place your order with us. As long as your instructions are clear, just trust we shall deliver irrespective of the discipline.
Our essay writers are graduates with bachelor's, masters, Ph.D., and doctorate degrees in various subjects. The minimum requirement to be an essay writer with our essay writing service is to have a college degree. All our academic writers have a minimum of two years of academic writing. We have a stringent recruitment process to ensure that we get only the most competent essay writers in the industry. We also ensure that the writers are handsomely compensated for their value. The majority of our writers are native English speakers. As such, the fluency of language and grammar is impeccable.
There is a very low likelihood that you won’t like the paper.
Not at all. All papers are written from scratch. There is no way your tutor or instructor will realize that you did not write the paper yourself. In fact, we recommend using our assignment help services for consistent results.
We check all papers for plagiarism before we submit them. We use powerful plagiarism checking software such as SafeAssign, LopesWrite, and Turnitin. We also upload the plagiarism report so that you can review it. We understand that plagiarism is academic suicide. We would not take the risk of submitting plagiarized work and jeopardize your academic journey. Furthermore, we do not sell or use prewritten papers, and each paper is written from scratch.
You determine when you get the paper by setting the deadline when placing the order. All papers are delivered within the deadline. We are well aware that we operate in a time-sensitive industry. As such, we have laid out strategies to ensure that the client receives the paper on time and they never miss the deadline. We understand that papers that are submitted late have some points deducted. We do not want you to miss any points due to late submission. We work on beating deadlines by huge margins in order to ensure that you have ample time to review the paper before you submit it.
We have a privacy and confidentiality policy that guides our work. We NEVER share any customer information with third parties. Noone will ever know that you used our assignment help services. It’s only between you and us. We are bound by our policies to protect the customer’s identity and information. All your information, such as your names, phone number, email, order information, and so on, are protected. We have robust security systems that ensure that your data is protected. Hacking our systems is close to impossible, and it has never happened.
You fill all the paper instructions in the order form. Make sure you include all the helpful materials so that our academic writers can deliver the perfect paper. It will also help to eliminate unnecessary revisions.
Proceed to pay for the paper so that it can be assigned to one of our expert academic writers. The paper subject is matched with the writer’s area of specialization.
You communicate with the writer and know about the progress of the paper. The client can ask the writer for drafts of the paper. The client can upload extra material and include additional instructions from the lecturer. Receive a paper.
The paper is sent to your email and uploaded to your personal account. You also get a plagiarism report attached to your paper.
GET A PERFECT SCORE!!! 
Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.
Read moreEach paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.
Read moreThanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.
Read moreYour email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.
Read moreBy sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.
Read more