News of evaluating commercial educational software in the US

Sun Apr 8 23:02:07 CST 2007

 I entirely agree with Mitchel that "The true power of new technology is
that it allows us to rethink what students can and should learn".

None of us should be surprised that a major comparative classroom study
of technology effectiveness shows no significant result. In Auckland,
Parr carried out a review of meta-studies of TEL in relation to literacy
and numeracy
(http://www.minedu.govt.nz/index.cfm?layout=document&documentid=5499&dat
a=l ) and found "Overall, the effectiveness of computer-assisted
learning has not been conclusively demonstrated. To date it has been
shown to be less effective, on average, than other forms of intervention
in education." She reported that most innovations in schools have an
effect size of 0.4. The effect size of computer-assisted learning is
between 0.25 and 0.6. The largest effect is in special needs education
with an effect size of 0.6. There is a mean effect size of 0.4 at
elementary level and 0.25-0.3 at secondary level. So the US Government
could have saved itself a large expense by reading the Parr report, but
at least the study has been conducted in US schools with the latest
software. 

Now we know that the intervention of traditional software, in
traditional classrooms, assessed through traditional methods shows no
significant effect when compared to the use of "other technology
products that may have been in the classrooms." So let's move on. 

Parr's report came to the wise concluson that "In considering the
results of evaluative research in computer-assisted learning software,
one has to avoid confounding the medium with the method. Generally,
computer-assisted learning is under pinned by an older, neo-behaviourist
theory of learning, one that has been displaced in the classroom by more
social constructivist views of learning." Instead of looking at whether
inappropriate technology has marginal effects on children's test scores,
we should be exploring how children can learn differently in
technology-enabled environments. This ranges from developing dynamic
simulations of processes that can't be shown on the printed page, to
redesigning classrooms and schools for collaborative and inquiry
learning, to supporting children in learning across contexts (e.g. home
and school) and life transitions (e.g. school-college-workplace). That's
where 1:1 technology could make a substantial difference. For example,
we have just gained funding for a major project, with the Open
University, to support children in inquiry science learning between
classrooms, homes, and outdoors. The aim is for the 1:1 technology to
structure and guide their inquiry learning outside the school in ways
that just aren't possible with printed media. 

I think that in any response to the study we should indicate that
there's now both a need and an opportunity to move beyond the sterile
debate of whether putting software into schools will raise test
performance, and develop new technology-supported environments for
learning based on our current understanding of best educational theory
and practice.

Mike

-----Original Message-----
From: discussion-bounces at g1to1.org [mailto:discussion-bounces at g1to1.org]
On Behalf Of Mitchel Resnick
Sent: 07 April 2007 19:37
To: Rory McGreal
Cc: discussion at g1to1.org
Subject: RE: News of evaluating commercial educational software in the
US

I think it's useful to consider the oft-used analogy to the early days
of cinema....

As most of you undoubtedly know, early movies were made by setting up
cameras in front of theater productions. If you did a study of people
who watched early movies, most probably would have said that they
preferred going to the theater.

These movies-of-theater-productions might have won over theater on
economic grounds, since movies are easier to copy and disseminate than
theater productions.

But that misses the most important point: the real power of movies came
when film-makers developed new forms of storytelling -- very different
from putting a camera in front of a theater production.

I think the situation is similar with the educational software tested in
this study. Although I'm not familiar with the software in the study, my
guess is that most of it teaches the same old stuff in the same old way
(but uses the computer as a new delivery medium). It is possible that
the new technology might win on economic grounds over textbooks, even if
the learning is not better. But that misses the point. The true power of
new technology is that it allows us to rethink what students can and
should learn, and how they learn it (through new types of design
activities, experimentation, etc.). If we don't use new technology to
fundamentally rethink our approaches to learning and education, then I'm
not very interested. I'd rather spend my time and energy arguing for new
approaches to learning and education, not simply introduction of
technology.

M....

At 01:55 PM 4/7/2007, Rory McGreal wrote:
>To ALL:
>I am surprised that community members are surprised by this result. 
>From Clarke pointed out in 1983, that there is "there are no benefits 
>to be gained from employing different media in instruction"
>
>Clark, R. (1983). Reconsidering research on learning from media. Review

>of Educational Research, 53(4), 445-459.
>
>Tom Russell 2001 compiled a bibliography of 355 studies showing  "no 
>significant difference".
>http://www.nosignificantdifference.org/
>
>My view is that this report in fact supports our use of technology. Was

>there a significant difference in learning when we went from writing on

>bark to writing on slates? From slates to fountain pens with ink? From 
>fountain pens to biros? From blackboard to greenboards?
>
>'No significant difference" shows us that the traditional classroom 
>environment without technology is no better than any other. If 
>textbooks are no better than courseware AND they are more expensive, 
>then why wouldn't we make the transition. Why pay $150 per child (or 
>more) for texts when we can get high (or low) quality courseware that 
>is just as good for less. As
>
>Elliott points out some software did show better results and the 
>differences between the courseware packages used need to be studied. My

>guess is that this is the same as some teachers in some classrooms 
>doing better than others.
>
>The really valid argument that pro-technologists can use successfully 
>with students and parents (not so much with teachers) is that the whole

>world is embracing technology, technology is part of nearly ALL spheres

>of human activity, that there can be no modern education without ICTs.
>
>As far as some of the drill and practice software is concerned, though 
>it would be better to put all the CDs in a big box, tie a chain around 
>it and use it as an anchor. (And rote drills in the classroom are no 
>better!)
>
>Elliott sees the study as a serious problem. I have no idea about 
>educational politics in the US, but my experience in Canada is that 
>schools and politicians (unfortunately) pay little if any attention to 
>educational studies.
>
>ALL social science studies are flawed to some degree. Only in the hard 
>sciences can we expect flawless studies.
>
>All the best.
>Rory McGreal
>
>AVP Research
>Athabasca University
>
>
>Elliott Soloway:
>
>This is a VERY serious situation, I fear.
>This study will be used to attack technology.
>Whlie we can attack the methodology of the study, that isn't really 
>going to do much good.
>
>Sherry:
>
>Here are some things to note:
>--the length of the study was 1 year...pretty short if you expect 
>teachers to integrate and make into effective practices.
>--the software they selected to evaluate were 
>practice-oriented/drill-kill packages with little to no immediate 
>student feedback nor student assessment (with exception to the 
>Andersonian cognitive tutor-one of the five packages in the study.) 
>--The teacher training was given by the developers in how to use the 
>software, not how to use the software effectively for pedagogy or 
>content learning. (But then again, you can't do much with limited
>software.)
>--The length of the intervention was on average 17 hours in one 
>year--not a big allocation of time.
>
>
>
>__
>     This communication is intended for the use of the recipient to
whom it
>     is addressed, and may contain confidential, personal, and or
privileged
>     information. Please contact us immediately if you are not the
intended
>     recipient of this communication, and do not copy, distribute, or
take
>     action relying on it. Any communications received in error, or
>     subsequent reply, should be deleted or destroyed.
>---
>_______________________________________________
>connect with G1:1 community -> discussion at g1to1.org 
>subscribe/unsubscribe G1:1 discussion mailing list -> 
>http://mail.g1to1.org/mailman/listinfo/discussion

-------------------------------------
Mitchel Resnick
mres at media.mit.edu
http://www.media.mit.edu/~mres 

_______________________________________________
connect with G1:1 community -> discussion at g1to1.org
subscribe/unsubscribe G1:1 discussion mailing list ->
http://mail.g1to1.org/mailman/listinfo/discussion

This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.