Research Project Presentation

Link to presentation:

http://prezi.com/epjgharx-bge/?utm_campaign=share&utm_medium=copy

Mini Reseach Project

Authentic Assessment:

Role of Student Choice in Summative Assessment Selection

Thomas B Barr

Stevenson University

STUDENT CHOICE IN ASSESSMENT

Research Question

To understand the success and failure of students in a classroom the basic underlying psychology of people must be addressed first. The area that deals with this is termed positive psychology and a specific theory that encompasses how humans thrive is Self-Determination Theory (SDT). SDT states that there are three fundamental mental “nutrients” that are required for optimal development: competency, relatedness, and autonomy (Deci & Vansteenkiste, 2004).

In an effort to meet these needs in a classroom, teachers need to construct activities that are geared to each of the three. The last “nutrient”, autonomy, has been linked to a students motivation and engagement in learning (Bennett, Zentall, French, & Giorgetti-Borucki, 2006; Cosden, Gannon, & Haring, 1995; Eshel & Kohavi, 2003; Hardre & Reeve, 2003; Reeve & Jang, 2006). One of the problems that can occur in teaching are the limited choices that students have access to in the classroom. In a teacher-centered classroom, material covered and assessment methods are decided for the students. One way to engage students in the learning process is to allow them to decide the method that they will be assessed by.

Ideally, students would be able to select from selected response, constructed response, and performance assessments. While students may not have control over the material that is covered due to central decision making, standardized test goals, or other factors; choice of summative assessment can contribute to students’ autonomy in a relatively easy fashion. Consequently, students having the opportunity to choose the type of assessment that they take part in can serve both autonomy and competency within SDT. Autonomy, through being a causal agent in determining their method of assessment and competent, by selecting the test that best allows them to demonstrate mastery.

With this in mind there are two areas of questions that can be asked about student choice in assessment. Primarily, we are concerned with determining what impact student selection of summative assessment type will have on performance on a standardized test. In this manner, selection of assessment type will act as a method of differentiation of assessment, which has been suggested as an effective teaching method (Tomlinson & Moon, 2013). Differentiated instruction has been described is a promising tool, that is at the same time vague and flexible (Ernest, Thompson, Heckaman, Hull, & Yates, 2011). Hopefully, this research can better describe differentiated instruction through student choice.

Therefore, the purpose of this research is primarily to investigate and determine the impact of student choice on learning. This will be accomplished through calculating a change score which will be the difference between pre-test and post-test standardized test scores. To be clear, this means we will not be so concerned with what type of assessment students choose in our primary research question, simply that they have a choice. Our secondary research questions will deal with the nature of the type of assessment, but this will be framed within the context of student choice.

Secondarily, we will determine the types of assessments, student affect, and correlation between test performance. are students most likely to choose resulting student affect on test choice. Depending on the results of our primary research question, it might be efficient for teachers to give certain types of assessments that are more attractive to them. We are also concerned with the way that students feel about selecting their assessment type. It is possible that they will enjoy and are motivated by the ability to select their test, however it may also result in unnecessary stress. Finally, we will investigate if there is a correlation between performance on different assessment types and performance on a standardized test.

Research Questions:

1.What impact does student choice of summative assessment type have on performance on standardized tests in high school biology students?

2. What types of assessments are high school biology students most likely to pick?

3. How does student selection of summative assessment type alter high school student feelings toward assessment?

4. Is there a correlation between performance on different types of summative assessments and performance on a standardized test in high school students?

Research Hypotheses:

There will be significant differences between the experimental group and the control group on the following dependent variables:

1. Standardized test change scores will be higher in the experimental group than the control group

2. “Normal” test types are the most likely to be selected.

3. Student affect will be higher in the experimental group than in the control group.

4. Performance on a test will be correlated with performance on a subsequent standardized test.

Literature Review

            As a psychological movement, positive psychology has attempted to identify environmental and personality factors that nurture an individual’s strengths, virtues, and development (Deci & Vansteenkiste, 2004). The starting point for a positive psychology theory is based in an active organism meta-theory (Deci & Vansteenkiste, 2004), that assumes three things. First, that humans are proactive in pursuing mastery of internal and external forces. Second, humans are self-organized systems that move to growth, integration, and health. Finally, for humans to pursue their optimal selves, they must receive the proper support from their environment (Deci & Ryan, 1985).

Within SDT is the understanding that the growth of a human can be nurtured or thwarted by the environment and the interactions with it. This is explained by the ability to meet the basic psychological needs of a human being. When this occurs, growth and development occurs, however when they are not met negative consequences ensue. The three basic psychological needs are competence, relatedness, and autonomy. Competence is the desire of people to effectively deal with the environment. Relatedness is the need to interact with other people. Finally, autonomy is the human urge to act according to one’s volition, or to have choice when acting (Deci & Vansteenkiste, 2004).

The application of SDT to the educational environment has resulted in an understanding that student choice is key in fostering motivation and engagement. Students must be given more opportunities to play an active role in their learning through setting their own goals, making choices about learning activities, and making decisions based on their needs and preferences (Evans & Boucher, 2015). For choice to be useful in the classroom there must be options, students must be aware of the options, and student must have the ability to make the choice (Evans & Boucher, 2015; Katz & Assor, 2006; Rose & Meyer, 2002; Servilio, 2009).

Student choice in the classroom has been found to reduce the amount of unwanted behavior in students with AD/HD (Bennett, et al., 2006), improve performance of students with behavioral problems (Cosden, et al., 1995), and improved the academics and affect of university students (Dosch & Zidon, 2014; Varsavsky & Rayner, 2013). However, there has been little attention paid to high school students and their choice in assessment. We will use the existing research to better frame our current questions about student choice in high school students.

Student choice in assessment led to a reduction in the number of unwanted behaviors in students with AD/HD. Students with AD/HD were given a choice on how they received feedback and compared to a control group that did not have any choice. There was no performance improvement associated with the student choice, however the purpose of the research was to determine if choice had any impact on behavior. With the clear positive effect of choice, other research needs to determine what other changes occur in students when choice is put in the hands of students (Bennett, et al., 2006).

Other research has examined the University teaching method, which is traditionally based around the professor-centered, lecture style, which can lead to low motivation amongst students. An alternative to this is differentiated instruction, which one component of this is student choice in learning task and assessment type. The study compared a differentiated instruction (DI) classroom to a non-differentiated instruction (NDI) classroom. The NDI classroom was a conventional lecture based class and the DI group had a variety of learning activities, choices for assignments, and instruction altered due to formative assessment (Dosch & Zidon, 2014).

The results of this study demonstrated that students performed better academically at a statistically significant level in a DI classroom than in a NDI classroom. Students also indicated that they liked the type of classroom activity better in the DI classroom than in the NDI. It appears that choice is an important component in learning and students feel better about the material if they have a choice in it (Dosch & Zidon, 2014).

The limitations that accompany this study are the difficulty of identifying the most effective components of DI. DI is a package of tools to use, and thus some skill as a teacher is required. Some teachers may lack the social skills to effectively implement DI in their classroom due to not being able to read what students need in DI (Dosch & Zidon, 2014). As a means to expand upon this study we will attempt to remove the social skills component required of a teacher and focus the current investigation on student choice of assessment.

Other research has pointed out that high achieving students have to deal with challenges in the large, teach to the middle, situation of the large undergraduate classroom. To combat this, differentiated assessments were implemented into a college chemistry and biology class. Students that had achieved a certain standard in a previous science class were given the option to complete a more challenging assessment or the regular standard assessment. Students were then given surveys and a questionnaire to determine their feelings towards the assessment options. Overall, few students attempted the advanced assessments, but the opportunity to do so was deemed a positive thing from all students, regardless of whether or not they took the advanced assessment. It appears that students believe that it is a good idea to have a choice on assessment, even though the options may be more difficult (Varsavsky & Rayner, 2013).

            Again a limitation of this research is that it only examined the affective responses of students to the different types of assessment and was targeted at high achieving students. While these are important factors to consider when differentiating the classroom, it would be more helpful to see if other variables changed such as performance in the class, or performance on other standardized items (Varsavsky & Rayner, 2013) and include more students. To complement Varsavsky & Rayner, we will look at both the performance and aspect of student choice of assessment in all students.

Research Design

The problem that exists today is inauthentic assessment is commonly used, and academic work in general is presented in such a way to disengage and demotivate students. This results in lowered motivation and engagement for students in school. Ultimately, this results in a world with a lack of creativity, problem solving, and critical thinking that could benefit us all. To address this problem, it is proposed to investigate the affect that self-selected assessments have on student performance on a subsequent standardized test.

The proposed method to explore the problem is an action research design that will have two groups of students who are a part of a high school biology class. All students will take a pre-assessment on the digestive system to determine their prior knowledge in this area. Students will then go through a lesson unit on the subject, which will be the same for all. At the end of the unit subjects will be separated into two groups. A control group will take the “normal” test and a second, experimental group that will be given a choice of the type of assessment that they complete.

The “normal” test will consist of a mixture of selected response and constructed response questions that emphasize the basic themes of the unit. This normal test is the only option to the students in the control group. Those subjects in the experimental group may choose the “normal” test or two other types of tests: an advanced test and a performance test. The advanced test stresses critical thinking, creativity, and application of knowledge by emphasizing constructed response questions. Finally, students will be given the option to complete a multimodal performance that demonstrates knowledge of the digestive system. This performance can be a presentation, recorded video, art work, etc. After completing the chosen test, all students will complete a post-unit assessment, which will be another version of the pre-assessment. As a part of the post-assessment students will complete an affective questionnaire to gauge their feelings about the test taking process.

A power analysis was performed to determine the minimum number of students that would be needed to observe statistically significant differences at the .05 alpha level for dependent samples t-test. Based on a power of 80%, a standard deviation of 5, and an effect size of .2, a conservative estimate of 27 subjects in each group would be needed, for a total of 54 subjects.

Data Analysis

            To adequately address the questions and issues that were previously stated, several types of data need to be collected. First, would be descriptive data of the students in the study. Age, education level, previous grades in biology and the sciences, ethnicity, race, and gender would all be useful independent variables to group students and answer questions retrospectively. At this point in time the author has no evidence to indicate that demographic factors will affect any of the variables being studied. They will be collected and analyzed at a later date to determine if future research questions should be asked and answered. The authors would need to collect pre-assessment scores, test type selection & score, post-assessment score, and a post assessment questionnaire.

            The data collected during this study would be best presented in bar graph and chart form depending on the question being answered. This would allow the data to be clearly understood based on the group and data that is being presented. Data could also be presented in a scatter plot in the case of an examination of a correlation between the test scores.

            Data would be collected from two sections of a high school biology class, one for the experimental group and one for the control group. Initially, students would have to consent to being part of the study and understand it is voluntary and will not affect their grade in any way. After consenting to the study students would submit a biographical data sheet that collects the above mentioned personal and demographic data.

All students would then begin the unit as normal and complete all of the learning tasks. They would complete a pre-assessment test, take part in all learning activities, lessons, labs, lectures, projects, etc. As the class approaches the end of the unit the experimental group will be given details about the type of summative assessment they can take. For the control group they will be given the details about the “normal”, selected and constructed response test only.

The implications of the data are the effect that assessment choice has on standardized test scores. This is a means by which to measure the impact that student autonomy has on traditional measures of learning. We will also investigate the type of test that students prefer to take. This will give teachers insight into what types of test are most appealing to students. Teachers may then be able to limit the options to those types of tests that best nurture the autonomy and competence of students. The author also believes that student affect will be higher in the experimental group than in the control group. If SDT hold true, providing a means for student to express autonomy and competence should bring them closer to optimal functioning. This should be reflected in affect scores. Finally, a strong positive correlation should be observed between performance on a test and subsequent standardized test performance.

The author will present the scores in terms of a change score for statistical analysis, however for discussion effect size will be presented. P-values in research are useful to indicate if there is an effect, effect size indicates the magnitude of the effect or what the clinical impact of the variable studied can be (Sullivan & Feinn, 2012). This is commonly done practice in biomedical research, which would prove valuable in education. We would except a minimum effect size of .2 to demonstrate any meaningful impact on student performance (Sullivan & Feinn, 2012). If we found an effect size that is less than .2 this would indicate that there is no meaning effect that student choice has on standardized test scores.

References

Bennett, D. E., Zentall, S. S., French, B. F., & Giorgetti-Borucki, K. (2006). The Effects of Computer-Administered Choice on Students With and Without Characteristics of Attention-Deficit/Hyperactivity Disorder. Behavioral Disorders, 31(2), 189-203.

Cosden, M., Gannon, C., & Haring, T. G. (1995). Teacher-control versus student-control over choice of task and reinforcement for students with severe behavior problems. Journal of Behavioral Education, 5(1), 11-27.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior: Springer US.

Deci, E. L., & Vansteenkiste, M. (2004). Self-determination theory and basic need satisfaction: Understanding human development in positive psychology. Ricerche di Psicologia, 27(1), 23-41.

Dosch, M., & Zidon, M. (2014). "The Course Fit Us": Differentiated Instruction in the College Classroom. International Journal of Teaching and Learning in Higher Education, 26(3), 343-357.

Ernest, J. M., Thompson, S. E., Heckaman, K. A., Hull, K., & Yates, J. (2011). Effects and Social Validity of Differentiated Instruction on Student Outcomes for Special Educators. Journal of the International Association of Special Education, 12(1), 33-41.

Eshel, Y., & Kohavi, R. (2003). Perceived Classroom Control, Self-Regulated Learning Strategies, and Academic Achievement. Educational Psychology, 23(3), 249-260.

Evans, M., & Boucher, A. R. (2015). Optimizing the Power of Choice: Supporting Student Autonomy to Foster Motivation and Engagement in Learning. Mind, Brain, and Education, 9(2), 87-91.

Hardre, P. L., & Reeve, J. (2003). A motivational model of rural students' intentions to persist in, versus drop out of, high school. Journal of Educational Psychology, 95(2), 347-356.

Katz, I., & Assor, A. (2006). When Choice Motivates and When It Does Not. Educ Psychol Rev, 19(4), 429-442.

Reeve, J., & Jang, H. (2006). What teachers say and do to support students' autonomy during a learning activity. Journal of Educational Psychology, 98(1), 209-218.

Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: Association for Supervision and Curriculum Development.

Servilio, K. L. (2009). You Get to Choose! Motivating Students to Read through Differentiated Instruction. TEACHING Exceptional Children Plus, 5(5).

Sullivan, G. M., & Feinn, R. (2012). Using Effect Size—or Why the P Value Is Not Enough. Journal of Graduate Medical Education, 4(3), 279-282.

Tomlinson, C. A., & Moon, T. R. (2013). Assessment and student success in a differentiated classroom. Alexandria, Va.: ASCD.

Varsavsky, C., & Rayner, G. (2013). Strategies That Challenge: Exploring the Use of Differentiated Assessment to Challenge High-Achieving Students in Large Enrolment Undergraduate Cohorts. Assessment & Evaluation in Higher Education, 38(7), 789-802.

Personal Assessment Philosophy

I believe in using assessments as guides for student progress and the practical exercise of assigning grades (John, 2013; Popham, 2014). Assessments must be grounded in authentic and engaging task, context, and criteria to benefit students (MovNat, 2015). For an assessment to be authentic and comprehensive it must contain the following parts: task, physical or virtual context, social context, results, and criteria (Gulikers, Bastiaens, & Kirschner, 2004). This general philosophy of assessment has been reinforced during the experience of this assessment course and my current teaching.

As a specific addition to the above general philosophy, I believe students needs to have a say in what task they are to be assessed by. Several authors indicate the importance that student choice plays in assessment (Dosch & Zidon, 2014; Servilio, 2009; Varsavsky & Rayner, 2013). This can also be viewed as a form of differentiation of instruction and assessment, which has been identified as a valuable teaching tool (Tomlinson & Moon, 2013). Student choice of assessment relates to a larger theory of human motivation, self-determination theory. Self-determination theory is a conceptualization of the factors that lead to optimal human development. There are three factors at play, one of which, autonomy (Deci & Vansteenkiste, 2004), is related to the issue at hand. The basic idea is that students who are able to make decisions about class activities are more likely to be motivated to stay engaged in them. As this is applied to assessment, it is important to give students a choice, so that they are able to demonstrate autonomy in how they are being guided and judged in the class.

While there is some evidence that has already been sited, this is mostly from my experience teaching. Recently, I have experimented with having students actually run part of a class in anyway they see fit, as long as they have cleared it with me first. The results have not been astonishing, but it appears that the students are more engaged and motivated to take part. It appears that when students are in control they have the opportunity to understand the reality of the situation and something changes: it becomes more serious.

I continue to try new methods on expanding my philosophy to include providing options for students on how they are assessed, such as selected response tests, constructed response tests, and performance tests. Morawski (2014) provided an interesting framework from which to work from by creating student selected multimodal responses in place of a test (Morawski et al., 2014). While I am not comfortable creating grading criteria for artwork as was in this example, it is an interesting example and something to use at some point in time.

My personal assessment philosophy is part of a larger education philosophy that I am constantly developing. I am trying to put this philosophy together to understand and leverage how humans learn best, or to put another way, what is our nature as learners and teachers. I believe that it is important to have both roles in your life at the same time. My philosophy of assessment contributes to the overall philosophy, through student choice and using the information as a guide first, and grade second.

References

Deci, E. L., & Vansteenkiste, M. (2004). Self-determination theory and basic need satisfaction: Understanding human development in positive psychology. Ricerche di Psicologia, 27(1), 23-41.

Dosch, M., & Zidon, M. (2014). "The Course Fit Us": Differentiated Instruction in the College Classroom. International Journal of Teaching and Learning in Higher Education, 26(3), 343-357.

Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67-86.

John, D. (2013). Intervention : course corrections for the athlete and trainer. Aptos, CA: On Target Publications.

Morawski, C. M., Hayden, K., Nutt, A., Pasic, N., Rogers, A., & Zawada, V. (2014). A Gallery of Multimodal Possibilities in a Graduate Course on Learning Differences in Education. International Journal of Education & the Arts, 15(18).

MovNat. (2015). Certified Level 1 Trainer Manual (4.2 ed.). Albuquerque, NM.

Popham, W. J. (2014). Classroom assessment : what teachers need to know (Seventh edition. ed.). Boston: Pearson.

Servilio, K. L. (2009). You Get to Choose! Motivating Students to Read through Differentiated Instruction. TEACHING Exceptional Children Plus, 5(5).

Tomlinson, C. A., & Moon, T. R. (2013). Assessment and student success in a differentiated classroom. Alexandria, Va.: ASCD.

Varsavsky, C., & Rayner, G. (2013). Strategies That Challenge: Exploring the Use of Differentiated Assessment to Challenge High-Achieving Students in Large Enrolment Undergraduate Cohorts. Assessment & Evaluation in Higher Education, 38(7), 789-802.