News of evaluating commercial educational software in the US

[Elliot Soloway]

This is a VERY serious situation, I fear.
This study will be used to attack technology. 
Whlie we can attack the methodology of the study, that isn't really going to
do much good.
One flaw that might be corrected immediately is this: the results from the
different programs were averaged together because the companies were afraid
that their one piece of software would look bad. But, there are clearly some
winners and some losers. The companies themselves might be willing to
publish the results. I talked with Andrew Trotter, editor of Education Week,
and he is working on that angle. It would be great to see that Program X did
in fact result in higher test scores. (I know, I know, test scores as a
measure are not the greatest, but....you can't fight everything all the
time.)

They are conducting this study next year. Maybe the teachers will be better
next year using the technology - though it is not clear that the teachers
will be getting any more professional development. I hate to have "maybe" as
an investment strategy, however. 

What we as a community need to do is to create model 1:1 programs and then
do a proper, gold-standard evaluation. Notice that the current study costs
$2billion. That's BILLION. To do a randomized, controlled study is clearly
very expensive. But, I think that if we had a few model 1:1 implementations,
that we could convince the evaluators to join together to pay for the
evaluation. Joe Krajcik, Phyllis Blumenfeld, Namsoo and I did a 1:1
almost-gold-standard study. I am attaching the press release about it. I
apologize to the community that we haven't written it up for journal
publication yet. 

So, our first task would be to define what would count as a "model
implementation." And then we would have to argue that that model
implementation is "scalable" -- that such a model is within the reach of
typical schools. We can't propose a "hothouse" model where everything is so
perfect that it can only exist in a rarefied atmosphere. 

I close as I began: the poor, urban and rural schools will suffer because of
this study since to get federal money in the US one must have "SBR" -
scientifically based research -- to justify the value of the intervention.
The rich, suburban schools are buying as many computers as they can drink
since they know that equipping their children with technology is the right
way to prepare them for the future (and they don't need federal money to do
so). The digital divide in the US will continue to grow and become a chasm. 

This study couldn't have come out at a worse time. This is not what our
community needed to be faced with.

Elliot

Elliot Soloway
Arthur F. Thurnau Professor
Dept. of EECS, College of Engineering
2260 Hayward,  Room 3629, CSE Building
University of Michgian
Ann Arbor, MI 48109
soloway at umich.edu
Mobile: 734-355-4098
Fax: 734-786-1221 

-----Original Message-----
From: discussion-bounces at g1to1.org [mailto:discussion-bounces at g1to1.org] On
Behalf Of Tak-Wai Chan
Sent: Saturday, April 07, 2007 4:08 AM
To: discussion at g1to1.org
Subject: News of evaluating commercial educational software in the US

 Hi,

A colleague sent me this:    
http://www.washingtonpost.com/wp-dyn/content/article/2007/04/04/AR2007040402
715.html 

In the article, Elliot expressed his worry.

Do you think G1:1 community can help something?


Regards,
Tak-Wai


_______________________________________________
connect with G1:1 community -> discussion at g1to1.org subscribe/unsubscribe
G1:1 discussion mailing list ->
http://mail.g1to1.org/mailman/listinfo/discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: press release detroit palm handheld study.doc
Type: application/msword
Size: 27648 bytes
Desc: not available
URL: <http://mail.g1to1.org/pipermail/discussion/attachments/20070407/8b92bcd1/attachment.doc>