|
The National Science Foundation's Instrumentation
and Laboratory Improvement (ILI) program is a little over
ten years old, and its first ten years (1985-1994) have been
thoroughly evaluated by an independent research and
consulting firm, Westat, Inc. Their report, together with the
observations of six disciplinary experts, has recently been
published as NSF 98-33. According to the report, "the program's
first decade of activity has been rich and fertile", and "the
seed money provided by NSF has yielded value well beyond
its initial investments."
Many readers of the
Journal have participated in ILI projects, and the program's broad impact on
undergraduate programs may seem obvious to us. Nevertheless it is
useful to have the program's many strengths documented and
its few weaknesses identified. Nearly all doctorate-granting
colleges and universities that have undergraduate programs
in science, mathematics, engineering, and technology
(SME&T) have submitted ILI proposals and 86% of them have
received at least one grant. Most non-doctoral four-year colleges
and universities (75%) have also submitted, and 55% of
these have received one or more awards. The fraction of
two-year colleges applying is smaller (40%), but the fraction of
proposals funded for four-year and two-year colleges is
almost identical-roughly one quarter.
Significant impacts of the program at grantee
institutions include reconfiguring, expanding, and updating
existing laboratory courses and providing hands-on research
opportunities. These benefits continue in nearly all of the
institutions long after the end of the grant. The hands-on
use of instruments affects the same percentage of women
students as in the overall population of undergraduates enrolled
in SME&T fields and a slightly higher percentage
of underrepresented minority students than in the overall
population.
Principal investigators report that their projects
benefit other faculty by improving the quality and modernity
of courses, the range of courses that can be taught, and the
morale of the faculty. However, there were no documented
impacts of ILI grants on the faculty reward systems at
grantee institutions-improved prospects for tenure, promotion,
and salary increases. The ILI grants did have tremendous
impact as seed money. On average, during the first five years of
a grant, more than four times the grant amount was
generated in internal and external financial investments.
Finally, there appears to be a synergistic effect when
an unusually large number of awards (more than ten) goes
to the same campus. In such cases there were more
comprehensive curricular reforms, and changes in curriculum and
instruction spread far beyond just the departments that
received the ILI grants.
Dissemination of the innovative educational
materials and pedagogy created with ILI support has been
extensive, but I was somewhat disappointed that only about a third
of grantees published their work in journals and other professional media. I certainly encourage submission of reports
and materials generated as a result of ILI projects to
JCE for peer review and potential publication. Nevertheless,
dissemination activities reached about 100 undergraduate professors
per project, and the fraction of those who reported
incorporating the ILI materials and information into their own
courses was generally higher than 50%.
The evaluation identified four areas in which the
program had fallen short of its goals: participation by
two-year institutions; research-related impacts among
undergraduate students and faculty; development of equipment-sharing
consortia; and community outreach. If you are developing
an ILI proposal, devoting some creative thought to these
areas would be a good idea. For example, one could envision
an equipment-sharing consortium involving several two-year
and one or two four-year institutions. In addition to
sharing equipment, such a group could provide community
service by doing appropriate chemical analyses of water or
other samples or by working with secondary schools to provide
access to instruments.
The team of disciplinary experts that participated in
the ILI evaluation came to somewhat different conclusions
from those of Westat, Inc. Their outlook was considerably
more positive, and they suggested that ILI filled a real and
increasing need for modern equipment and instrumentation
in SME&T programs. The disciplinary experts thought
that there was more impact on outreach and undergraduate
research than did Westat, and they recommended against
focusing the program more narrowly on ILI-rich
institutions, despite the apparent synergistic effect. They did suggest
that NSF should work to increase the number of proposals
from two-year, small, and minority institutions, to increase
the number of women and minority principal investigators,
and to encourage PIs to disseminate the results of ILI
projects more widely.
The ILI program clearly exemplifies NSF's goal of
providing catalytic support for science education. The 4074
grants to 1185 different institutions totaling $158.6 million over
the first decade have had far more impact than these
numbers might indicate. Given the synergistic effect of multiple
awards in the same institution, an even higher level of funding
would clearly bring even greater benefits. NSF is to be
congratulated on a program well conceived and implemented, and the
Congress is encouraged to provide much greater support for
this and other highly effective NSF programs.
|