Collaborative Argumentation As A Learning Strategy To Improve Student Performance In Engineering Statics : A Pilot Study

Educators have used argumentation to help students understand mathematical ideas which often appear abstract to the novice learner. A preliminary investigation was conducted to determine if collaborative argumentation is a strategy that can improve the student’s conceptual understanding of the topics taught in the engineering course commonly titled Statics. The academic performance of students enrolled in a traditional problems-solving session was compared to the academic performance of students enrolled in a problem-solving session where collaborative argumentation was used. Results suggest that argumentation improved student performance as measured by grades associated with one-hour long exams, although student written responses on a course evaluation survey responses indicate that students did not believe argumentation was a learning strategy was effective.


INTRODUCTION
umerous studies (e.g., Akkus,. Gunel & Hand, 2007;Hannafin & Land, 2000) can be found in the education literature describing how student-centered learning increases conceptual understanding compared to the traditional instructor-centered lecture (or telling) approach.Student-centered learning engages the student in self-directed discovery practices designed to broaden her/his understanding of a concept and makes the student responsible for assessing her/his learning.Such practices are dynamic and complex undertakings, and often scaffolding is needed to help the student manage self-doubt, misconceptions, and ambiguity, just to name a few of the challenges she/he may face.Argumentation is one instructional technique mathematics education teachers use to help students overcome these challenges and has been shown to help students understand those math concepts that often appear abstract to the novice learner (Forman, Larreamendy-Joerns, Stein & Brown, 1998;Conner, Singletary, Smith, Wagner & Francisco, 2014).
Argumentation involves a dialogue and Toulmin (2003) suggests that this dialogue can be analyzed by breaking it down into three key components commonly known as the claim, data and warrant.In the learning environment, the student establishes a claim that is a statement whose validity is being established, identifies the data needed to support the claim and uses the warrant to connect the data to the claim.The following is an example of a middle-school student grasping the concept of a linear function how the student's argument is broken down into the components of a claim, data and warrant.
Instructor: Looking at this graph, what do you conclude about the function?Student: Using the x and y grid shown on the graph, the slope of the function is 4.This is because when the x value changes by 1 the y value changes by 4.
The analysis of the student's argument is broken into the three key components as follows: Claim: the slope of the function is 4 Data: the x and y values Warrant: the change of the y value with the x value N Table 1.The Motivated Strategies for Learning Questionnaire provided evidence that students in the two treatment groups were similar in their confidence to do well in the course.The two items presented in this table indicate significant differences (p<0.10) between students who received a final course grade of C and those who received a final course grade of A or B.
The instructor can ask the student to… Example Question • provide a fact • What is the slope of this function?• describe the approach used to accomplish the task • How did you determine the slope?• expand on their statements or ideas • Why is your answer 4? • evaluate another student's statements or ideas and negotiate with other students to develop a common response.
• Student A says that the value of y changes by 4 when x changes, if student A correct?
Collaborative argumentation (Nussbaum, 2008) refers to the setting where students work together to construct and reflect on their arguments by asking each other questions and checking on each other's explanations.Connor et al. (2014) use the term collective argumentation to describe the process that students use to adjust their individual arguments and reach a consensus rather than a one student convincing the others that her/his argument is the correct one.Eskin and Ogan-Bekiroglu (2013) provides a case study where collaborative argumentation positively impacts 10 th -grade students' understanding of mechanical dynamics, primarily by making them aware of their misconceptions.Overall, learning through argumentation shifts the student from memorizing to understanding and reasoning plus argumentation develops skills needed to solve problems in real-world situations.Kulatunga, Moog and Lewis (2013) provide evidence that argumentation successfully improves undergraduate performance in an introductory chemistry course.Principles similar to those found in argumentation have been shown to develop students' understanding of engineering ethics (Jonassen & Cho, 2011) and understanding of concepts of thermodynamics (Erduran & Villamanan, 2009).Collaborative learning strategies such as collaborative argumentation improve engineering students' self-efficacy for learning course material, although the grade competitiveness associated with engineering courses reduces this impact (Stump, Husman, Chung & Kim, 2011).As a dialogic pedagogy, argumentation potentially affects a student's belief that her/his intellectual ability is malleable (Resnich and Schantz, 2015).A student with this belief is likely to concentrate more on conceptual understanding (Blackwell, Trzesniewski, and Dweck, 2007;Dupeyrat and Mariné, 2005).In contrast, a student with a stronger belief that intellectual ability is fixed often associates her/his academic performance with having natural ability, and that student tends to concentrate more on performance outcomes (e.g., grades) than on conceptual understanding.
The objective of the study reported herein was to investigate the potential of Collaborative Argumentation as an inclass strategy that helps students in the conceptual learning in the fundamental engineering course statics.The study focused on three research questions: • Does collaborative argumentation learning impact the learning strategies that students use?
• Does collaborative argumentation increase student reflection on topics taught in Statics?
• How is student academic performance affected by the collaborative argumentation learning strategy?

Course Design
This study was implemented in the sophomore-level engineering course Statics.Weekly class meetings included two 75-minute lecture sessions that were held each Tuesday and Thursday and one 50-minute problem-solving session held each Wednesday.Sixty students attended the same lecture sessions.These students were divided in half (i.e., thirty students) where each half attended a different problem-solving session.One set of thirty students attended a

Participants
Sixty engineering majors enrolled in this section of the engineering course Statics.One Ph.D. candidate studying kinesiology attended lectures, studied with the engineering majors enrolled in the course and took all exams.Data from this candidate was not used.Two other course sections, which were not part of this study, enrolled another 135 engineering majors.During registration, each student was able to self-select the course section she/he wanted.When a student registered for the course section included in this study, that student then had to select which one of the two problem-solving sessions she/he wanted.The self-selection process was only available when seats were open for the problem-solving sessions.As those seats began to fill, the self-selection process became limiting where a student was placed in the problem-solving session with an open seat.All 60 students were evenly distributed into each problemsolving session with 30 students per problem-solving session.
Students could withdraw from the course without grade penalty at the date that corresponded when 60% of the semester was completed.Sixteen out of 60 students withdrew from the course at different times of the semester with 13 of these students withdrawing just after taking the third of five-semester hour exams.Nine of the students who withdrew were in the problem-solving session for the traditional treatment group (see below), and seven were in the problem-solving session for the argumentation treatment group (see below).One student in the traditional treatment group was retaking this course for the fifth time; data for that student was removed from the study.Of the remaining 43 engineering students who completed the course, 18 students in both problem-solving sessions signed the consent form to participate in the study.All enrolled students were exposed to one of the two treatments.

Treatments
All participants attended the same lecture sessions.The general format of the lecture was as follows; the faculty instructor presented, on white-board, the mathematical and/or scientific principles found within a topic (e.g., forcecouple system) and worked examples where the principle was applied.Throughout the lecture, the instructor engaged students in questions/answer discussions about specific aspects of the presented topic.Periodically, the students worked a problem before the instructor presented the solution.The treatments for this investigation were applied in the problem-solving sessions.

Traditional Treatment
In the first of the two problem-solving sessions, a traditional instructor-focused pedagogy was used.The faculty instructor would hand out a problem and allocated 5 minutes for the students to start developing the solution to the problem.Two undergraduate TAs walked around the room and provided assistance when requested.After the 5 minutes, the instructor presented the solution, drawing the free-body diagram, listing the equilibrium equations and

Copyright by author(s); CC-BY 14
The Clute Institute finally using the equations to develop the solution.The instructor stopped at each major step to address questions.Three to four problems were presented in this fashion during the 50-minute problem-solving session.Detailed solutions were provided as a hand-out at the end of the session.These same problems were presented to students in the other treatment group.

Argumentation Treatment
The second problem-solving session began immediately after the first problem-solving session.This session followed the student-focused approach offered by argumentation.The first action was to hand out a problem which had the following questions listed: • What is the principle that you would use to solve the problem (the claim)?
• What is the mathematical and/or scientific basis that demonstrates that your choice of principle/concept is valid (the warrant)?• Individual students wrote down their answers.In the second step, the students formed groups of two to three students and each group discussed the following: • What principle did your group partners choose and what was the mathematical and/or scientific rationale did your partners and you use (the qualifier)?• Which of your answers is correct?Each student group was to develop a single answer (collaborative argumentation).In the next step, the class, as a whole, discussed these questions where their responses guided the instructor as the problem's solution was developed.When different answers were given, time was taken for a discussion.When no responses were offered by any student, the instructor would select someone to respond.Due to time constraints, a complete solution to the problem was not developed.Problems were worked in this manner with the intent to complete as many problems as presented in the traditional group problem-session.Detailed solutions to the worked problems plus any 'not-worked' problems presented in the traditional treatment group's problem-solving session were provided at the end of the session.
While students worked on their answers to the above questions, they could request guidance from undergraduate TAs.The TAs would guide the students through the process but would not provide students with the answers.Also, the TAs used a rubric to identify any student who consistently was not participating in the process, and the instructor used this information to encourage students to become more engaged during the session.

Data
The Motivated Strategies for Learning Questionnaire (Credé & Phillips, 2011;(Pintrich, Smith, Garcia, & McKeachie, 1993) was used to assess if the treatment groups had similar beliefs and desires to do well in the course.A 4 point Likert scale was used for each item.The Cronbach's alpha coefficients for the items ranged from 0.64 to 0.80 indicating a good internal reliability for this survey.Additional items on the questionnaire were used to assess the learning strategies students used when studying for the course outside the classroom.The questionnaire was administered during the second lecture session.
The Statics Concept Inventory (SCI) (Steif and Dantzler 2005) was used to assess if each treatment group consisted of students with similar knowledge of engineering mechanics before enrolling into statics.The SCI was administered immediately after the students finished the Motivated Strategies for Learning Questionnaire.The grade for the SCI contributed to 5% of the final course grade and was the first of 6 exams given during the semester.The final exam was the 7th exam given during the semester.
At the date that corresponded to approximately 95% of the semester being completed, the students completed a Course Evaluation survey.This survey investigated the students' perception of the effectiveness of the treatments administered during the problem-solving sessions.Using a five-point Likert scale (strongly disagree to agree strongly), the students responded to the following questions: • I participated in problem-solving session discussions • The techniques used increased my understanding of statics • Homework assignments increased my understanding of statics more than the problem-solving sessions.
• How we engaged the principles needed to solve the problem helped me to reflect on the topic of statics • Student-to-student discussions encouraged reflection on the topic of statics • Class discussions encouraged reflection on the topic of statics • Assignments encouraged reflection on the topic of statics • My experience in this [problem-solving] session helped me understand the importance of taking responsibility for my learning.
This evaluation was not assessed for reliability since this was a pilot study.
In addition to the SCI, five hour-long exams were used to measure changes in each student's academic performance as the semester progressed.These exams were administered at dates that corresponded to 20%, 40%, 60%, 73% and 89% of the semester being completed.Each of these exams was worth 13 points or 13% of the final course grade.
Each exam covered material from the first class session of the semester to the class session just before the exam.
Although each exam was designed to be completed in 60 minutes or less, some students were given extra time as determined by the university's Office of Disability Services.In addition to the SCI and the five 1-hour exams, a final exam was administered 7 days after the final class meeting, and overall student performance on this exam was determined using a 20-point scale or was 20% of the final course grade.Ten students exempted the final exam after earning 92% of all possible points associated the course up to the final exam.A final grade of A or A-(minus) was assigned to these students.The final 10% of the course grade was associated with quizzes given periodically, but these scores were not used in this study.

Analysis
A review of the literature reveals that researchers disagree how to examine questionnaire Likert scale data.De Winter and Dodou (2010) concludes that parametric t-test procedures and nonparametric procedures have similar power, and either can be used to examine Likert scale survey responses.Herein, the motivation survey and the course evaluation survey were analyzed using a nonparametric Kruskal-Wallis statistical procedure.Since this investigation was a pilot study, a significant difference was determined at 90% (p<0.10).
A one-way ANOVA and Tukey posthoc test analysis of the Statics Concept Inventory scores was used to determine if the content knowledge of each treatment group was similar at the beginning of the semester.The initial analysis was done using data collected from all students who completed the SCI.A second analysis was conducted using data from students who were taking statics for the first time.The statistically significant difference was determined at 90% (p<0.10).Statistical analysis of the SCI data was conducted using the software SPSS.
Student performance on second, third, fourth, fifth and sixth one-hour exams was measured using exam scores (percentage scores from 0-100%, not letter grades).The first one-hour exam was for the SCI and was not included in this analysis.Eighteen students from each treatment provided consent to use their scores for all five exams.A repeated measurements ANOVA analysis was used to assess changes in these scores as the semester progressed and to evaluate effects that the treatment had on these scores.The within-subject factor was the exams which were spaced approximately 18 days apart, and the between-subjects factor was treatment.The posthoc Tukey and Bonferroni tests were used.Depending on the Greenhouse-Geisser estimate, violations of sphericity were handled using either the Huynh-Feldt correction or the Greenhouse-Geisser correction.The initial analysis was conducted using grades from all students.Next, exam scores were placed into three groups that were based on students' final exam grade, and the scores for each group were analyzed using a repeated measurement ANOVA.The three groups were students 1) who made a C on the final exam, 2) who made a B on the final exam and 3) who made an A on the final exam plus those who exempted the final exam.Significant differences were considered at 90% (p<0.10).Statistical analysis of the student performance data was conducted using the software SPSS.

Copyright by author(s); CC-BY 16
The Clute Institute

RESULTS AND DISCUSSION
Findings reported in Shaharoun, Puteh and Harun (2012) demonstrate that motivation and learning belief have a significant impact on a student's fundamental understanding of a complex topic such as those found in the statics course.Therefore, a critical aspect of the study herein is to determine if students in each treatment groups have similar confidence in their ability to complete the course.Overall, results from the Motivated Strategies for Learning Questionnaire indicated that students in both treatment groups were similar in this respect.When grouping students by the final course grade, statistical analysis indicated significant differences (p<0.10) between students who made a final grade of C to those who made a final grade of A or B.  Results of the Statics Concept Inventory (SCI) exam indicated that content knowledge of the treatment groups was not significantly different at the beginning of the semester.For both treatment groups, the average SCI exam score was 11.0 points out of a possible 13.The standard deviations were 2.5 and 2.2 points for the traditional treatment groups and the argumentation treatment group, respectively.Three students in the traditional treatment group and four students in the argumentation treatment group had enrolled in a statics course, multiple times.Removing the exam scores for these students did not impact the outcomes, that is no significant differences existed between the groups.After removing these students' scores, the new SCI averages (standard deviations) were 11.4 (2.3) and 10.9 (2.5) points for the traditional treatment group and the argumentation treatment group, respectively.
The Course Evaluation Survey assessed if the students believed that argumentation was an effective learning tool.Table 4 provides a set of responses where significant effects (p<0.10) were found.Interestingly, a large number of students in the argumentation treatment group (29%) indicated that the teaching did not increase their understanding of statics, although analysis of the exam graded (below) suggest that argumentation improved student performance.Von Aufschnaiter, Erduran, Osborne and Simon (2008) suggests that familiarity with the method is a critical factor that impacts the student acceptance of new learning strategy such as argumentation.Investigations of other instructional interventions have found similar limitations.For example, Hanson and Williams (2008) used selfexplanatory writing to increase student performance in statics and found that students initially struggled with the technique, but performance improved as the semester progressed.In the study herein, students in the argumentation treatment may have simply "not liked" the process and therefore indicated that the techniques did not increase their understanding of statics.As expected, students believed that argumentation encouraged them to reflect on the topics taught in the problem-solving sessions (Table 4).

Copyright by author(s); CC-BY 17
The Clute Institute Overall student performance was the same for both treatment groups as indicated by the distribution of the final course grades (Figure 1) and by student performance on each of the five one-hour exams administered during the semester (Figure 2).There were no treatment differences in these assessment metrics (i.e.grades) at any exam point.
The rate at which the grades increased between exam 2 and exam 4 was different for the two groups.Results of the Repeated Measurement ANOVA indicate that when considering the grading trends, the argumentation treatment significantly improved student performance (Figure 3) particularly between exam 2 and exam 4. The sudden drop in grades at exam 5 was similar for both groups.Exam 5 focused on topic of frames and machine while exam 6 focused on topics of truss systems; students typically struggle with these two topics and therefore the drop in grades is expected.
To further examine the differences in student performance, the students in each treatment were grouped by the grades earned on the final exam. Figure 4 shows the exam grades for students who made a C on the final.The C students in the argumentation group significantly improved (p<0.05) their grades between exam 2 to 4 while the grades remained relatively flat for the C students in the traditional problem-solving group.Figure 5 shows the exam grades for students who made an A on the final exam where there was no significant difference between the two treatment groups.These results suggest that collaborative argumentation could offer an instructional tool that increases student academic performance in statics.The greatest potential of argumentation to impact student academic success appears to be with the lower performing students.In the argumentation group, 7 students who earned a C on the final exam also received grades less than 78% of the possible points on the first exam (i.e., Statics Concept Inventory).For students in the traditional group, those who earned a C on the final exam received grades ranged from 30% to 90% of the possible points on the first exam.For argumentation treatment students who received a C on the final exam, the grades they earned on exams 2, 3 and 4 increased as the semester progressed.This grade increase was not seen for traditional treatment students who received a C on the final exam.Capped line represents standard error about the mean exam grade.
Figure 5. Students in each treatment were grouped by the grade earned on the final exam.For students who received an A on the final exam, they received similar grades on exams 2, 3 and 4.This trend was the same for both treatment groups.Capped line represents standard error about the mean exam grade.

LIMITATIONS OF THE STUDY, CONCLUSIONS AND FUTURE DIRECTIONS
This study involved only one section of the fundamental engineering course statics where students participating the same lecture sessions were divided into two problem-solving sessions where the treatments were administered.This scenario could create avenues for students in one treatment group to influence students in another treatment group.For example, a few students participating in the argumentation learning approach could form a study group with a few students involved in the traditional learning approach.Therefore, the impact of the argumentation to learn the concepts of statics could be lessened.A follow-up study should be designed where treatment groups do not share lecture sessions and problem-solving sessions.While it would be difficult to achieve, students should be asked to study only with students in their treatment group.
Students participating in this study engaged argumentation only 15 times during the semester, reducing the time they had to become familiar with this learning tool.Limited exposure to argumentation could limit students to discovering

Figure 3 .
Figure 3.The exam grades as the semester progressed indicate that the argumentation group grades increased between exam 2 and 4. The rate of this change was significantly different (p<0.05) when comparing the two treatments.Capped lines represent standard error about the mean exam grade.

Figure 4 .
Figure 4. Students in each treatment were grouped by the grade earned on the final exam.For argumentation treatment students who received a C on the final exam, the grades they earned on exams 2, 3 and 4 increased as the semester progressed.This grade increase was not seen for traditional treatment students who received a C on the final exam.Capped line represents standard error about the mean exam grade.
Stump, Husman and Corby (2014)wo questions investigated how confident the student was about doing well in the course.No student who made a final grade of A or B student provided a negative response ("not at all true for me"), whereas, 17% of the C students chose this negative response.Stump, Husman and Corby (2014)suggest that a student's belief in her/his intellectual ability did not correlate to academic performance in engineering courses, so the finding reported in this pilot study herein should be investigated further.The Motivated Strategies for Learning Questionnaire included items that investigated what learning strategies students believed they would initially use in the course.No differences were found between the treatment groups.

Table 3 .
The Motivated Strategies for Learning Questionnaire provided evidence that students in the two treatment groups were similar in their confidence to do well in the course.The two items presented in this table indicate significant differences (p<0.10) between students who received a final course grade of C and those who received a final course grade of A or B.Questionnaire Item: I am certain I can understand the ideas taught in this course.Student

responses to this question Very true for me True for me most of the time Sometimes true for me Not at all true for me Number of students
Questionnaire Item: I am sure I can do an excellent job on the problems/tasks assigned for this class.

Table 4 .
The course evaluation survey assessed if students believed that argumentation was a useful learning tool.More students indicated that the argumentation approach did not increase their understanding of the concepts taught in statics.Questionnaire Item: The techniques used increased my understanding of statics.Student-to-student discussions encouraged reflection on the topic of statics.Class discussions encouraged reflection on the topic of statics.The final course grades indicate that overall student performance in both treatment groups was similar.Trends of grades earned each exam indicate that the argumentation group grades increase between exam 2 and 4. Repeated measurement ANOVA analysis indicated the two groups of grades were statistically different (p<0.05).Exam 1 was the Statics Concept Inventory questionnaire and therefore was not considered in this analysis of student performance.Capped lines represent standard error about the mean exam grade.