A Decade of Research on Internet Enhanced Education

A Decade of Research on Internet Enhanced Education

A Decade of Research on Internet Enhanced Education D.A.Kashy, M. Thoennessen, G. Albertelli, & E. Kashy Supported by : Alfred P. Sloan Foundation Andrew W. Mellon Foundation Michigan State University National Superconducting Cyclotron Laboratory National Science Foundation Project Timeline 1992: Initial implementation of the CAPA System Using Telnet, printed individualized assignments, quizzes and exams. 1995: Web implementation for students.

1995-1999: Sloan Foundation support (ALN implementation in large on-campus courses) 1999-2002: Mellon Foundation support (MSU/TAMU; Cost and educational effectiveness) CAPA Open Source 1999: Initiation of CAPA & LectureOnLine collaboration into an open source, experimental version of a content & course management system (LON-CAPA) 2000-2005: NSF Information Technology grant for research with the experimental LON-CAPA system Concerns in Large Lecture Classes

Impersonal nature of instruction Large differences in student preparation Achieving and maintaining high standards Large human resources required for grading and assessment Timely recognition of students problems and difficulties Copying assignments - cheating on exams Computer-Assisted Personalized Approach Individualized homework, quizzes, exams, and supplementary exercises Problems can be numerical or conceptual in

nature Facilitates communication among students and between students and teaching staff Automatic grading and record keeping Online statistical analysis and feedback for both the instructor and student Research Topics Encountered 1. 2. 3. 4. 5. 6. 7.

8. 9. 10. Student evaluations of the technology Educational effectiveness in Physics Gender differences in Physics Educational outcomes in Statistics for Psychology Student feedback and system development Students teaching students Teaching chemistry Cost effectiveness of technology in education Effectiveness of individualized interactive content Educational impact of Web-facilitated cheating

1. Typical results examining student evaluations of CAPA Evaluations of CAPA by students in Physics for Scientists and Engineers CAPA as Tool for Learning/Understanding Fall -Traditional Spring - Technology Question: If I had a choice between the CAPA scheme as used in PHY184 and the recitation scheme used in PHY183, I would be: Number of Students 120 100

80 Phy184 Spring 95 N=156 Birge, Cowen A - Strongly in favor of recitations B - Moderately in favor of recitations C - Indifferent D - Moderately in favor of CAPA E - Strongly in favor of CAPA 60 40

20 0 A B C D E Fall Technology Spring - Traditional Did the different methods affect your learning/understanding of physics?

Recitation preferred Methods equivalent CAPA/Help room preferred PHY184 Spring 96 N=104 Berz, Tomanek 0 20 40 60

Number of Students 80 Time-on-task with and without CAPA Evaluations of CAPA effectiveness by students in Statistics for Psychology on scales from 1-10 Mean SD Range

Homework educationally effective and motivating 8.48 1.53 5-10 Quizzes effective in learning and motivating attendance 7.56

2.29 2-10 9.20 1.17 5-10 Exam Corrections effective in learning material missed on exams Percentage of Psychology Statistics Students who

endorse keeping CAPA Percent of students 80 60 40 20 0 1 2 1=Drop CAPA 3

4 5 6 7=Keep CAPA 7 2. Educational Effectiveness in Physics Technology Impact

Prof. Danielewicz (Physics) % 25 N = 1537 1992-1994 N = 1435 1996-1998 20

15 10 Distribution of grades in Introductory Physics for Scientists and Engineers 5 0 20 15 10 5 0

1992-1994 w/o CAPA 1996+ with CAPA N = 472 1999 N = 449 2000 20 15 10

5 0 20 15 10 5 0 0.0 1.0 1.5 2.0

2.5 Grade 3.0 3.5 4.0 3. Gender differences in Physics %

Grade distributions for male and female students for a twosemester course (Phy231 and Phy232). Phy231 was taught the traditional way and in Phy232 CAPA was used. 35 Male Phy231

30 Phy232 25 20 15 10 5 0 0.0 %

35 1.0 1.5 2.0 2.5 3.0 4.0 Female

Phy231 30 3.5 Phy232 25 20 15 10 5 0

0.0 1.0 1.5 2.0 2.5 Grade 3.0

3.5 4.0 (Female - Male)/Male % Relative difference between female and male performance on exams in Introductory Physics for Scientists and Engineers 5 Exam1 Exam2

Exam3 Final 0 -5 -10 -15 -20 1999 2000 Educational outcomes in Statistics

for Psychology Do students need to be computer savvy to succeed? Measured Computer Self-efficacy on the first day of classes Correlation w/Computer Self-efficacy Final Exam Homework Quizzes -.16 n.s. -.02 n.s.

-.02 n.s. Additional educational outcomes Similar to results found in Physics The vast majority of students achieve very high grades on the homework (e.g., 60% of students are above 90%) Performance on homework predicts success on midterm and final exams Class attendance is generally high (above 75%) and correlates positively with performance Student and faculty feedback: Impact on system development Example: Printing

Printing for On-the-Web-Only Homework Do you print your homework questions? 43% All each week 22% Some each week Rarely 10%

0 N = 326 25% Never 20 40 60 80

100 Number of Responses B. Alex Brown, private communication, Physics 231, Fall 1998 120 140 Web-Only Assignments e-mails: For = 1 Against = 88 PHY183, Fall 98. Harder to work in front of screen 56%

43% Printed HW more accessible Trouble to print, quality of printing 27% Web access required Work on-line too long 18% 11% Paid for it 7%

N= 88 Printed HW is a reminder4% 4% Computer Access Problems Prefer printed copy- No reason 4% 0 10

20 30 40 50 Number of Responses Class Poll, Web-Only Assignment (1 week): For = 6 Against = 420 Students teaching students

Cost Effective Use of Undergraduates as Teaching Assistants Members of the Honor College An H (next to grade) must be earned for a minimum number of some courses Get badges to wear in PLC The H implies additional achievement in the course Used in Phy183 course: Students Teaching Students Agree to have Assignments done 2 or 3 days ahead of deadline. Write an exit essay on experience Undergraduate T.A.s - I

The teaching was good experience and I felt that through teaching, I learned the material better myself. I thought the way the honors option was set up was great because not only was I helping other students I was also helping my self study and learn the material better. I think, we were, in a way, more approachable than the TA's I thought it was a good thing to do as an honors option because it not only helped me but others also. Undergraduate T.A.s - II After working in the PLC, I truly believe that there is no higher form of learning and understanding than

teaching someone else. For some time, Ive been considering teaching as a career, and this has strengthened the inclination. I learned that everyone learns things differently, some need pictures and diagrams, others need equations and examples. I remember a study that said, you learn 40% of what you hear, 50% of what you read, and 90% of what yu teach. Helping other students has enabled me to help myself in this physics class. Undergraduate T.A.s - III GRADE DISTRIBUTIONS %

25 N = 1537 1992-1994 N = 1435 1996-1998 20 15 10 5

0 20 15 10 5 0 N = 472 1999 N = 449 2000

20 15 10 5 0 20 15 10 5 0 0.0 1.0

1.5 2.0 2.5 Grade 3.0 3.5 4.0 7. Teaching chemistry concepts

with Guy Ashkenazi, Hebrew University, Jerusalem Ongoing investigation with 400 chemistry students Comparison of sections taught using traditional methods including in-lecture numerical exercises lectures cover only concepts Results on examination performance: Numerical problems equivalent Significantly higher performance on questions involving concepts for Concept-only lecture sections

8. Cost effectiveness of technology in education Cost Effectiveness Conclusions ALN technology can reduce costs when faculty and staff are involved in grading and record keeping (repetitive, time consuming tasks. Staff reduced by ~30%) Broader impact of technology: improving the students educational experience while maintaining current costs. TAMU study of Psychology Statistics indicated that although larger classes are more cost effective, technology did not make an equal substitute for much smaller class sections. Cost savings are most likely to be in disciplines for which large content libraries have been developed Without such

libraries, however, a considerable initial investment of faculty and teaching assistant time is necessary. 9. Effectiveness of individualized interactive content CAPA PROBLEM TYPES

Numerical problems where each student gets randomly assigned values for the variables Select all the correct statements from a list of statements Assign a conclusion to each statement of a list Identify features in a diagram, with both the labels and the list of features randomized Problems that require a graphical solution Rank items based on quantitative or qualitative measure Problems with an answer consisting of several parts (and) Problems with more than one correct answer (or)

Subjective essays, read and evaluated by instructor with key-words highlighted by the computer Problems with expressions as answers where students enter symbolic formulas Individualized applets as basis of interactive questions Sound and speech as the medium for the questions Mathematics Numerical (Typical Algorithmic) Randomly Labeled Numerical (Student 1) Conceptual Textual: Principe DArchimde [2pt] Un pcheur et sa jeune nice sont en bateau sur un petit tang. Ils portent

l'un et l'autre un gilet de sauvetage. La nice tient la ficelle d'un gros ballon rempli d'hlium qui flotte au-dessus du bateau. Examinez sparment chacune des situations prsentes ci-dessous et indiquez si le niveau d'eau de l'tang: M Monte, B - Baisse, R- Reste le mme, I- Indetermin. (Si dans le premier cas le niveau monte, baisse dans le second cas, et que le rsultat est indetermi pour les autres cas, tapez: MBIIII) A) Le pcheur remplit un verre avec de l'eau de l'tang et le boit. B) Le pcheur jette l'ancre et la laisse pendre verticalement un pied audessus du fond de l'tang. C) Le pcheur se jette lui-mme l'eau et flotte sur le dos. D) La nice fait clater le ballon. E) La nice tombe l'eau, lche la ficelle du ballon qui s'envole. F) Le pcheur renverse par-dessus bord la boite de pche qui coule au fond de l'tang. Conceptual:

Functions Graphed + Random Labels Conceptual: Physical Situation Illustrated + Random Labeling + Random

Order + Textual Variation Applet: Text Applet: Initial and Final Action Frames Covariates included ACT, College GPA, Initial Physics

knowledge (Force Concepts Inventory), and number of absences Interactive Applet Impact Interactive Exercises with Network Technology Goals achieved 1. Increased time-on-task 2. Timely feedback (to both students and instructors) 3. More positive collaboration among students and between students and instructors 4.A greater ability to achieve and maintain high standards

10. Educational impact of Web-facilitated cheating Some Students Response: Defeating the system is the challenge Overall student acceptance very good Helps students who are willing to learn Unfortunately there are students who want to beat the system Make cheating more difficult than actually learning the material Instructor Response: More

Randomization R1 R2 R3 R2 R1 R3 R2R3 Solution: R1 + R +R 2 3

R1R3 Solution: R2 + R +R 1 3 Influence of Websites Homework 0.041 Monitored (posting) 0.118 Quizzes

-0.302 0.098 0.069 Midterm Exams -0.352 0.166 0.160

Final Exam -0.348 0.147 0.129 3rd Party Monitored (looking) -0.109

Instructor Response: Even More Randomization the next step ? Why stop at posting the equation? Solve the individual problems for the students Commercial/charge them... CAPA.solutions.com Sure Enough Excel spreadsheets! Prob # Measurement

1 Side Current B 2 Length F(1) F(2) B 4d i B(loop) B(wire) Total 5L W d

I(2) Force I(1) Quanitity Conversions Units 3.43 0.0343 T Note: 1.2 1.39942E-05 -1.39942E-05 1.41 0.0141

Note: 1.18708E-07 N 5.93538E-08 -5.93538E-08 16.34 0.01634 T Note: 20 0.02 1.53811E-06 4.89596E-07 1.04852E-06 24 0.24 A

Note: 16.7 0.167 11.5 0.115 11.3 0.00000408 1.4607334 My box was like this - + + + Try one of these This was done for top right x componen Wire Clockwise, Current Right

I2 Down, I1 CCW More Individualized Graphics Version I P N O Version II

N P O What to do? Have very large databases of problems Design to make cheating more difficult than doing the work Should we be trying to protect the students from cheating themselves and/or their parents?

Misc. Costs and Benefits of IT in Education: Can be more efficient but also more demanding on the teaching staff. Enhanced learning environment for most students. Far more comprehensive and timely information about student performances and problems What Next? With LON-CAPA what we have been doing is now much easier many other ideas possible For More Info www.loncapa.org

Informational Conference January 17/18 2003 Truckee Meadows Community College Reno, Neveda

Recently Viewed Presentations

  • Graphing Graphing Relationships Relationships Warm Up Lesson Presentation

    Graphing Graphing Relationships Relationships Warm Up Lesson Presentation

    The correct graph is graph C. Check It Out! Example 1 Continued As seen in Example 1, some graphs are connected lines or curves called continuous graphs. Some graphs are only distinct points. They are called discrete graphs The graph...
  • Cargo to Go

    Cargo to Go

    คลังสินค้าสำหรับสินค้าพิเศษ (Special commodity warehouse) คลังสินค้าประเภทนี้ใช้เก็บสินค้าประเภทใดประเภทหนึ่งโดยเฉพาะ เช่น ฝ้าย ขนสัตว์ ...
  • Creative Thinking - FEMA

    Creative Thinking - FEMA

    Critical versus Creative Thinking. They are not the same, but are closely linked. Critical thinking is the active and systematic process of communication, evaluation, reflection and analysis, meant to foster understanding, solve problems, support sound decision-making and guide action
  • "From Electric ham and scuba monkeys" to the 6 C's of motivation

    "From Electric ham and scuba monkeys" to the 6 C's of motivation

    Electric Ham and Scuba Monkeys … (and the Grateful Dead) 6 C's of Engaging Students ... Designed to implement the 6 C's Turner and Paris developed for a literacy context ... Students were grouped homogeneously by Lexile levels derived from...


    SOLIDS. To understand and remember various solids in this subject properly, those are classified & arranged in to two major groups. Group A. Solids having top and base of same shape. Cylinder. Prisms. Triangular Square Pentagonal Hexagonal. Cube. Triangular Square...
  • Verzus PowerPoint Template

    Verzus PowerPoint Template

    The water sensitive design is supposed to be better not just in terms of water performance but for a range of reasons: it is meant to have more useable area, better solar exposition and ventilation, less fences, more communal spaces...
  • Applying for Credit - Loudoun County Public Schools

    Applying for Credit - Loudoun County Public Schools

    COST OF CREDIT. A credit card might offer a low introductory APR but change to a much higher rate after a few months. Credit card companies will usually charge a fee for a cash advance. Cash Advance. a loan given...
  • ESL Review Exam 154- English As a Second Language Supplemental

    ESL Review Exam 154- English As a Second Language Supplemental

    State mandates, sheltered instruction - Partner A and B. Test Taking Tips. ... Morphology, Semantics, Syntax (or Syntactics), Pragmatics. Linguistic Semantics: is the study of meaning that is used for understanding human expression through language. It focuses on the relation...