Week 4 – Discussion: Non-experimental Design
School Academic Counseling course
Respond to the following prompts in the Threats to Validity discussion
forum by Wednesday November 8 2023:
- Select a non-experimenal design
- What type of questions would you ask for your selected non-experimenal design?
- Regarding this weeks article, what are the general ed teachers level of understanding about evidenced based practice in teaching students with Autism?
Educational Measurement and Quantitative Designs
PPS 6025: Methodology of Educational Research
Created by Helen Y. Sung, Ph.D.
Cognitive: measure learning and reasoning
Noncognitive: measure social and emotional characteristics
Consider the following:
Which assessments do school psychologist use most and for what purpose?
What assessment tools to school counselors mostly use?
The scores are interpreted compared to the well-defined norms.
The scores are compared with the scores of others who had taken the test.
Shows how individuals compare with established level of performance or skills.
Large scale standardized tests:
There is a uniform procedure for administration and scoring.
SAT, ACT, State Achievement Test, GRE
Standard based tests:
Standardized achievement tests with criterion based interpretation
Criteria for each grade level skills
Standardized Aptitude Tests:
Measures knowledge or skills that predict future performance.
Aptitude means to have predictive nature of the instrument.
These are the various types of tests used to measure progress.
You will want to know the difference between each of these test designs.
Raw scores are transformed into z scores by raw score –mean/standard deviation.
Standard scores are derived from the z scores.
Individual compared with the normative group in terms of grade level.
This can be confusing to a parent while it looks familiar.
Standard scores and grade equivalent scores compare the student with normative population. However, standard scores are typically use to report the level of functioning.
Types of Data
Rating Scale: Likert Scale or variations of
Checklist: Provide number of options to choose from
Rank order: Put number of categories in sequential order
Observations: Descriptions of observed behaviors
Things to be cautious of:
Response set: refers to the tendency to respond the same way.
Faking: deliberately giving inaccurate response
This happens when the researcher indicates certain positive outcome.
The subjects response is to please the researcher.
Error can happen because of the background, expectations, or frame of reference.
Example – researcher may have had a bad experience with a person of different race.
Observer has knowledge of one or more aspects of the study and this knowledge affect subsequent observations
Example – male/female students on communication.
Observer’s initial impression influence subsequent observations
Example: poor or unkempt person
Subjects are given choice of answers.
Questions are open-ended with specific intent, allowing individual responses.
Open-ended and broad.
The interview has general goal in mind and asks questions relevant to the goal.
It uses statistics such as frequency, percentages, averages and variability.
Often use graphs and visual images to describe the results.
There is a relationship between two variables but does not necessarily reveal the cause.
For example: increase in homework completion lead to higher test scores.
Other variables unaccounted for might be parent support, motivation, students ability, opportunities and exposer to other source of knowledge.
Nonexperimental Design continued
Pairs of scores are used to produce a scatter gram to calculate a correlation coefficient.
In prediction study, correlation coefficient shows how one variable can predict another.
Ex Post Facto:
There is no active manipulation of independent variables because it has already occurred.
Investigators decide on one or more preexisting condition have caused difference in subjects who experienced one type compared to a different condition.
Cooperative learning in relation to achievement.
Students were already engaged in a paired computer-learning situation.
The research question asks, “What type of interaction promoted increased achievement?”