To stimulate a national conversation on whether and how pull mechanisms might be used to accelerate the development of high-impact learning technologies, OSTP seeks public comment on the questions listed below:
(1) What learning outcomes would be good candidates for the
focus of a pull mechanism to catalyze the creation and use of new learning
technology? These outcomes could be relevant to early childhood education,
K-20, life-long learning, workforce readiness and skills, etc.
(2) How are these learning outcomes currently measured and
assessed?
(3) What information exists about current U.S. performance
relative to this learning outcome? What information exists about the presence
(currently available or potential given current trends or breakthroughs) or
absence of effective interventions (technology-based, offline, or hybrid) to
improve this learning outcome?
(4) Why would a pull mechanism in this area accelerate
innovation in learning technology?
(5) What role might different stakeholders (e.g. Federal
agencies, state and local educational agencies, foundations, researchers,
practitioners, companies, investors, or non-profit organizations) play in
designing, funding, and implementing a pull mechanism for learning technology?
What role would your organization be willing to play?
(6) What changes in public policy would facilitate
experimentation with pull mechanisms at different levels of government?
A Notice by the Science and Technology Policy Office on
01/13/2014
Summary
The Office of Science and Technology Policy requests public
comments to inform its policy development related to high-impact learning
technologies. This Request for Information offers the opportunity for
interested individuals and organizations to identify public and private actions
that have the potential to accelerate the development, rigorous evaluation, and
widespread adoption of high-impact learning technologies. The focus of this RFI
is on the design and implementation of “pull mechanisms” for technologies that
significantly improve a given learning outcome. Comments must be received by
11:59 p.m. on March 7, 2014, to be considered. In your comments, please
reference the question to which you are responding.
Respondents are encouraged to submit their comments through
one of the following methods. Email is the preferred method of submission.
Please do not include in your comments information of a confidential nature,
such as sensitive personal information or proprietary information. Responses to
this notice are not offers and cannot be accepted by the Federal Government to
form a binding contract or issue a grant. Information obtained as a result of
this notice may be used by the Federal Government for program planning on a
non-attribution basis. Please be aware that your comments may be posted online.
Email:
learning@ostp.gov. Email submissions will receive an electronic confirmation
acknowledging receipt of your response, but will not receive individualized feedback
on any suggestions.
Postal Mail: Office
of Science and Technology Policy, Attn: Cristin Dorgelo, 1650 Pennsylvania
Avenue NW., Washington, DC 20504. Submissions by postal mail must be received
by the deadline, and should allow sufficient time for security processing.
This Request for Information (RFI) offers the oppm1unity for
interested individuals and organizations to identify public and private actions
that have the potential to accelerate the development, rigorous evaluation, and
widespread adoption of high-impact learning technologies. The focus of this RFI
is on the design and implementation of “pull mechanisms” for technologies that
significantly improve a given learning outcome. Pull mechanisms increase the
incentives to develop specific products or services by committing to reward
success. Examples of pull mechanisms include incentive prizes, Advance Market
Commitments, milestone payments, “pay for success” bonds, and purchasing consm1ia.
The public input provided through this notice will inform the deliberations of
the Office of Science and Technology Policy (OSTP).
OSTP is interested in identifying policies and serving as a
catalyst for public-private pat1nerships that have the potential to accelerate
the development, rigorous evaluation, and widespread adoption of high-impact
learning technologies. For example, imagine if learners in the United States
had access to technologies that:
Dramatically reduced
the large and persistent gap in vocabulary size between children from wealthy
and poor households.
Allowed middle and
high school students to outperform their international peers in math and
science.
Enabled
English-language learners that are reading at several grade levels below
average to catch up after only a year.
Gave non-college
bound students an industry skills ce1tification or set of cognitive skills
(e.g. literacy, numeracy, ability to understand and apply chmis, graphs and
diagrams) that are a ticket to a middle-class job, increasing their
employability and their incomes by $10,000-$20,000 or more in less than a year.
Doubled the
percentage of community college students that pass remedial math, which is
currently only 30 percent.
Successfully delivered a “growth mindset” intervention to
teachers and students.
Were as effective as a personal tutor, were as engaging as
the best video game, and improved the more students used them.
Currently, there is a
large gap between the relatively modest impact that technology has had on
education, particularly in K-12, and the transformative impact that it has had
in many aspects of our economic and social life. For example, businesses are using
information and communications technologies to dramatically increase
productivity, tap the expe1iise of their employees, slash the time needed to
develop new products, tailor products and services to meet the needs of
individual consumers, orchestrate global networks of suppliers, derive insights
from huge volumes of transactional data, and improve their products and
services by conducting rapid, low-cost experiments.
Education, particularly K-12 education, remains relatively
untouched by advances in our understanding of how people learn, how to design
instruction that incorporates those insights, and the explosion in information
technologies such as low-cost smartphones and tablets, cloud computing,
broadband networks, speech recognition and speech synthesis, predictive
analytics, data mining, machine learning, intelligent tutors, simulations,
games, computer-suppmied collaborative work, and many other technologies. That
is why President Obama has proposed ConnectED, a new initiative to connect 99
percent of America's students to the Internet through high-speed broadband and
high-speed wireless within 5 years.
Learning technologies will be much more effective if they
informed by “learning science”—advances in disciplines in fields such as
neuroscience, cognitive science, educational psychology, and discipline-based
education research that shed light on how people learn. This research can
provide actionable insights on issues such as student motivation, the
circumstances under which prior knowledge helps or hmis learning, how students
can organize knowledge in rich and meaningful ways, and the ways in which
students can progress from novice to expeti in a given domain.
There are a number of reasons for the gap between the
potential of learning science and technology and the cunent
state-of-the-practice:
The United States is
investing 0.1 percent of K-12 expenditures on R&D, compared to 2 percent in
mature industries and 18.7 percent in the pharmaceutical industry. This
extremely low level of investment in educational R&D has clearly limited
the pace of innovation.
Entrepreneurs seeking
to develop and market new products to the K-12 market face a number of
challenges, including low per-pupil expenditures on software, lengthy adoption
cycles, and a highly fragmented market. This in turn limits the amount that
companies can spend on research and product development.
It is difficult for
companies to make authoritative claims about the impact of their products on
learning outcomes assessed through rigorous third-party validation, which
limits the premium that school districts and other consumers of learning
technology are willing to pay for high-quality, effective products.
This suggests that an effective national strategy for
increasing the impact of learning science and technology should address both
the “supply” and “demand” for advanced learning technologies.
To increase the “supply” of learning technology, the Federal
government and philanthropists could increase funding for research and
development and support training grants and scholarships in relevant
disciplines such as educational psychology, cognitive science, instructional
design, artificial intelligence, etc. The National Science Foundation is
funding a program called “Cyberlearning Transforming Education” and the
Depmiment of Defense is supporting research in advanced training technologies.
The President FY14 Budget request includes funding for a “DARPA for Education”
(ARPA-ED).
However, there has been little discussion of the potential
of what economists call “pull mechanisms” to accelerate the development,
evaluation, and adoption of high-impact learning technologies.
As economists have recently noted, governments and other
funders can suppmt innovation using “push” programs (e.g. funding grants and
contracts to universities and companies, providing tax incentives for R&D,
or supporting government laboratories) and “pull” mechanisms that “increase the
rewards for developing specific products by committing to reward success.” Push
programs pay for research inputs; pull mechanisms pay for research outcomes.
“Pull mechanisms” have been used successfully in the field
of global health. In December 2010, children in developing countries began
receiving a vaccine that will prevent deaths from “pneumococcal” diseases
including pneumonia, meningitis, and sepsis. Nearly one million young children
die every year from pneumococcal infections, with 90 percent of these deaths
occurring in developing countries.
The development of this vaccine was accelerated by a $1.5
billion “Advance Market Commitment” backed by five governments and a private
foundation. Pharmaceutical companies that have agreed to provide the vaccine at
$3.50 per dose to low-income countries for the next 10 years will receive
additional payments from the $1.5 billion in donor commitments. The AMC
increased the size and predictability of the market for pneumococcal vaccines,
which increased the willingness of companies to invest in high-volume
production of these vaccines for developing country markets. Expe1ts predict
that this AMC will save 7 million lives over the next twenty years.
Non-binding commitments to purchase products can also
provide market pull, if there is both a clearly defined performance
specification and a strong expression of interest from potential buyers. For
example, in June 2013, the U.S. Department of Energy put together a coalition
of the Federal government and over 200 major commercial building pmtners that
issued a challenge to U.S. manufacturers: “If you can build wireless sub-meters
that cost less than $100 apiece and enable us to identify opportunities to save
money by saving energy, we will buy them.” At least 18 manufacturers agreed to
take up the challenge. In 2011, the Department of Energy put together a similar
and successful challenge for energy-efficient and cost-effective commercial air
conditioners, with the first manufacturer meeting the challenge in May 2012.
In addition, Federal agencies have offered almost 300
incentive prizes on Challenge.gov, providing opportunities for citizen solvers
to offer novel solutions to tough problems, while minimizing risk to Federal
agencies by only paying for success. More information about pull mechanisms can
be found in this supplemental information document.
OSTP is interested in stimulating a conversation about how
pull mechanisms could be used to accelerate the development, evaluation, and
adoption of learning technologies. Some of the advantages of pull mechanisms
are that a funder can (a) pay only for success; (b) set a goal without having
to choose in advance which team or approach is most likely to be successful;
and (c) increase the number and intellectual diversity of the teams that are
working to solve a particular problem. Although there a variety of different
types of pull mechanisms, they generally require establishing a clear goal and
an agreed-upon set of metrics for evaluating progress towards that goal. If
education is going to benefit from increased use of pull mechanisms,
policy-makers and stakeholders have to identify some specific challenges that
are important and measurable, and where it is plausible that learning
technology can help improve student outcomes.
Using Pull Mechanisms for Learning Technologies
Pull mechanisms can be used for social interventions that do
not use technology. For example, the first “social impact bond” is being used
by the United Kingdom to reduce recidivism among 3,000 prisoners. The United
Kingdom's Depa11ment for International Development (DfiD) is supporting a
“Results-Based Aid” approach to improving education in Ethiopia. Under this
pilot, DfiD will make grant payments to the education ministry for the increase
in the number of students above a baseline that sits for or passes the national
grade 10 exam. There will be additional payments for students in the poorest
regions, and for girls compared to boys.
It may also make sense to experiment with pull mechanisms to
accelerate the development and rigorous evaluation of learning technologies.
Some of the potential advantages of learning technologies include:
Low marginal cost:
The marginal cost of making software or digital content and services available
to more students is very low, although the fixed cost of R&D and rigorous
evaluation may be high. This is why IT stmtups are able to grow rapidly—the
cost of serving tens or hundreds of millions of customers does not increase
arithmetically with the number of customers.
Ability to maintain
high levels of “time on task”: For example, good game developers can keep users
riveted for hours at a time. They can create experiences that are intrinsically
motivating, and that offer an increasingly difficult set of challenges that
keep users in the “sweet spot” between being bored and frustrated.
Continuous
improvement: The productivity of most public sector services is flat or
negative. Researchers and entrepreneurs have ideas for developing online
services that get better the more people use them by (a) conducting many
low-cost experiments to discover what works; and (b) collect, analyze and act
on the data that can be generated online.
Learning anytime, anywhere:
Mobile devices allow individuals to access digital content at a time, place,
and pace that is convenient for them. This might be particularly impmiant for
an adult who is trying to upgrade their skills while balancing the competing
demands of work and family.
Digital tutors:
Research suggests that the average student tutored one-on-one using “mastery
learning” techniques (students are helped to master each concept before
proceeding to a more advanced learning task) performed better than 98 percent of
the students that learn the same material using conventional instructional
methods. Projects funded by DARPA and the Office of Naval Research suggest that
it may be possible to develop “digital tutors” that model the one-on-one
interaction between a world-class subject matter expeti and a student. A pilot
suppmied by the Veteran's Administration is allowing unemployed veterans that
use the digital tutor for 6 months to get IT jobs that pay $40,000 to $80,000.
Personalization:
Researchers and firms are developing software and online services that are
personalized to the needs, background, interests and skill levels of
individuals.
Interactive
simulations that enable “learning by doing”: Researchers have developed
simulations in areas such as physics, chemistry, biology, emih science, and
math. For example, an “Energy Skate Park” simulation allows students to explore
energy conservation with multiple different variables (shape of the track,
starting height and speed of the skater, mass of the skater, and friction).
Students can quicldy repeat experiments and rapidly explore the effect of many
different parameters.
Embedded assessment:
Technology can help provide continuous assessment of a given set of knowledge,
skills and abilities if the designers know (a) what behaviors would constitute
evidence that a student has mastered a given competency; and (b) which tasks
can elicit those behaviors.
Questions
To stimulate a national conversation on whether and how pull
mechanisms might be used to accelerate the development of high-impact learning
technologies, OSTP seeks public comment on the questions listed below:
(1) What learning outcomes would be good candidates for the
focus of a pull mechanism to catalyze the creation and use of new learning
technology? These outcomes could be relevant to early childhood education,
K-20, life-long learning, workforce readiness and skills, etc.
(2) How are these learning outcomes currently measured and
assessed?
(3) What information exists about current U.S. performance
relative to this learning outcome? What information exists about the presence
(currently available or potential given current trends or breakthroughs) or
absence of effective interventions (technology-based, offline, or hybrid) to
improve this learning outcome?
(4) Why would a pull mechanism in this area accelerate
innovation in learning technology?
(5) What role might different stakeholders (e.g. Federal
agencies, state and local educational agencies, foundations, researchers,
practitioners, companies, investors, or non-profit organizations) play in
designing, funding, and implementing a pull mechanism for learning technology?
What role would your organization be willing to play?
(6) What changes in public policy would facilitate
experimentation with pull mechanisms at different levels of government?
Response to this RFI is voluntary. Responders are free to
address any or all the above items, as well as provide additional information
that they think is relevant to accelerating the development, rigorous
evaluation and widespread adoption of high-impact learning technologies. Please
note that the U.S. Government will not pay for response preparation or for the
use of any information contained in the response.
Ted Waelder,
Deputy Chief of Staff and Assistant Director.
Supplementary Information: Overview of Pull Mechanisms
Incentive prizes are one type of “pull
mechanism”—results-based market incentives designed to overcome market failures
and catalyze itmovation. Experts often make a distinction between “recognition”
prizes that honor past achievements and “inducement” or “incentive” prizes that
encourage participants in the competition to achieve a particular goal. In a
2009 repot1, McKinsey identified six prize archetypes that provide a useful
framework for identifying types of prizes that can best achieve different types
of goals:
Exemplar Prizes that define excellence within an area.
Point Solution Prizes
that aim to spur development of solutions for a pmiicular well-defined problem.
Solutions can include software applications, algorithms, predictive models,
ideas, business plans, policy proposals, designs, or prototypes.
Market Stimulation
Prizes that try to establish the viability of a market to address a potential
market failure, mobilize additional human talent and financial capital to
jumpstati the development of a new industry, or change public perceptions about
what is possible.
Exposition Prizes
that are designed to highlight a broad range of promising ideas and practices,
attract attention, and mobilize capital to further develop the winning
innovations.
Participation Prizes
that create value during and after the competition- not through conferral of
the prize award itself but through their role in encouraging contestants to
change their behavior or develop new skills that may have beneficial effects
during and beyond the competition.
Network Prizes that
build networks and strengthen communities by organizing winners into new
problem-solving communities that can deliver more impact than individual
effmis.
Other types of pull mechanisms include:
Advance Market
Commitments: Binding commitments to purchase, or to subsidize purchase, of a
ce1iain volume of a product at a fixed prize, if the product meets pre defined
performance characteristics (pneumococcal vaccine and Department of Energy
examples discussed above).
Buyer's Consortia:
Cooperative agreements between purchasers of products that leverage the
combined buying power of those purchasers to drive down the price of products,
such as a buyer's consmiium set up for Maine school districts to purchase
specialized software and specific assistive technology devices.
Pay-for-Success
Bonds: Under a Pay for Success bond, also known as a social impact bond, the
financing organization and the Federal, state, or local government enter into a
contract that specifies the population to be served, the outcomes to be
achieved, the measurement methodology to be used, and the schedule of payments
to be made. The financing organization works with philanthropic and other
investors to invest in innovative, data-driven service providers that can
achieve results. One example of a pay for-success bond program is an initiative
in New York targeted at reducing recidivism in adult males.
Milestone-based
Payments: Payment terms in a standard grant or contract in which the payment
for each performance milestone established in the statement of work is not made
until the milestone is proven to have been achieved. One example of this
approach has been successfully demonstrated in NASA 's Commercial Orbital
Transportation Services (COTS) program.
Priority Review
Vouchers: An accelerated regulatory review offered to products that meet
certain performance or cost criteria, such as the FDA Innovation Pathway and
USPTO 's Patents for Humanity.
Patent Buyout: An
offer to buy out the patent rights to a product that meets specified
performance conditions at a set price (price for patent usually marked up over
market value; followed by placing of the patent into the public domain to
encourage competition for commercialization of the product). One example is the
purchase of the patent for the Daguerreotype process by the French government
in 1839.
No comments:
Post a Comment