81 FR 2876 - Announcement of Requirements and Registration for “Pill Image Recognition Challenge”

DEPARTMENT OF HEALTH AND HUMAN SERVICES
National Institutes of Health

Federal Register Volume 81, Issue 11 (January 19, 2016)

Page Range		2876-2879
FR Document		2016-00777

The Pill Image Recognition Challenge is a National Institutes of Health (NIH) Challenge under the America COMPETES (Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science) Reauthorization Act of 2010 (Pub. L. 111-358). Through this Challenge, the National Library of Medicine (NLM), part of NIH, seeks algorithms and software to match images of prescription oral solid-dose pharmaceutical medications (pills, including capsules and tablets). The objective of the Challenge is the development and discovery of high-quality algorithms and software that rank how well consumer images of prescription pills match reference images of pills in the authoritative NLM RxIMAGE database. NLM may use all or part of any Challenge entry (i.e., algorithm and software) to create a future software system and a future API (Application Programming Interface) for pill image recognition; the system will be freely usable and the API will be freely accessible.

Federal Register, Volume 81 Issue 11 (Tuesday, January 19, 2016)

[Federal Register Volume 81, Number 11 (Tuesday, January 19, 2016)]
[Notices]
[Pages 2876-2879]
From the Federal Register Online [www.thefederalregister.org]
[FR Doc No: 2016-00777]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

National Institutes of Health

Announcement of Requirements and Registration for ``Pill Image
Recognition Challenge''

Authority: 15 U.S.C. 3719

SUMMARY: The Pill Image Recognition Challenge is a National Institutes
of Health (NIH) Challenge under the America COMPETES (Creating
Opportunities to Meaningfully Promote Excellence in Technology,
Education, and Science) Reauthorization Act of 2010 (Pub. L. 111-358).
Through this Challenge, the National Library of Medicine (NLM), part of
NIH, seeks algorithms and software to match images of prescription oral
solid-dose pharmaceutical medications (pills, including capsules and
tablets). The objective of the Challenge is the development and
discovery of high-quality algorithms and software that rank how well
consumer images of prescription pills match reference images of pills
in the authoritative NLM RxIMAGE database. NLM may use all or part of
any Challenge entry (i.e., algorithm and software) to create a future
software system and a future API (Application Programming Interface)
for pill image recognition; the system will be freely usable and the
API will be freely accessible.

DATES: NLM will make a set of consumer-quality images and a companion
set of reference images publicly available on January 15, 2016.
The Challenge begins January 19, 2016.
Submission period: April 4, 2016 to May 31, 2016.
Judging period: June 6, 2016 to July 15, 2016.
Winners announced: August 1, 2016.
Submissions received by NLM after the submission period ends will
not be considered. A submission is considered to meet the submission
deadline if it is received by May 31, 2016, 5:00 p.m. EDT. While NLM
plans to acknowledge receipt of each Challenge submission, the
Government is under no obligation to acknowledge receipt of the
information received or provide feedback to respondents with respect to
any information submitted. NLM will amend this Federal Register notice
if the timeline or the rules for the Challenge are modified. In
addition, NLM will notify registered Challenge participants by email of
any amendments and will include the modified Challenge showing the
changes.

ADDRESSES: Notifications of any amendment to this Federal Register
notice and answers to frequently asked questions about it will be
posted at http://pir.nlm.nih.gov/challenge/notifications-and-FAQs.
Submissions must be mailed to: Pill Image Recognition Challenge,
Computational Photography Project for Pill Identification (C3PI),
National Library of Medicine, Building 38A, Room B1-N30, 8600 Rockville
Pike, Bethesda, MD 20894.

FOR FURTHER INFORMATION CONTACT: Michael J. Ackerman, Ph.D. at (301)
402-4100 or PIR@nlm.nih.gov.

SUPPLEMENTARY INFORMATION:

The IC's Statutory Authority To Conduct the Challenge

What has become today's National Library of Medicine began in 1836
as a small collection of medical books and journals in the office of
the U.S. Army Surgeon General. A 1956 act of Congress (Pub. L. 84-941)
transferred the library to the Public Health Service and gave it its
current name. That law authorizes NLM to ``assist the advancement of
medical and related sciences and to aid the dissemination and exchange
of scientific and other information important to the progress of
medicine and to the public health'' and to ``promote the use of
computers and telecommunications by health professionals (including
health professionals in rural areas) for the purpose of improving
access to biomedical information for health care delivery and medical
research.'' In addition to its subject-matter authority, NLM is
conducting this competition under the America COMPETES Reauthorization
Act of 2010 (Pub. L. 111-358).

Subject of Challenge

Unidentified and misidentified prescription pills present
challenges for patients and professionals. Unidentified pills can be
found by family members, health professionals, educators, and law
enforcement. The nine out of 10 U.S. citizens over age 65 who take more
than one prescription pill can be prone to misidentifying those pills.
Taking such pills can result in adverse drug events that affect health
or cause death. To reduce such errors, any person should easily be able
to confirm that a prescription pill or a refill is correct. For
example, a person should be able to easily verify--or not--that a
refill that has a different color, shape, or text imprinted on the pill
is a different generic version of equivalent drugs he or she was
already taking.
To help address these problems, the NLM Computational Photography
Project for Pill Identification (C3PI) is developing infrastructure and
tools for identifying prescription pills. The infrastructure includes
photographs of such pills taken under laboratory lighting conditions,
from a camera directly above the front and the back faces of the pill,
and at high resolution. Specialized digital macro-photography
techniques were then used to capture JPEG pill images. The NLM RxIMAGE
database contains these high-quality images and associated pill data
such as appearance (color, shape, size, text imprinted on the pill,
etc.), ingredients, and identifiers such as its National Drug Code
(NDC) [http://www.fda.gov/Drugs/InformationOnDrugs/ucm142438.htm].
RxIMAGE images and data are freely available. The freely accessible
RxIMAGE API provides text-based search and retrieval of images and data
from the RxIMAGE database. By contributing their algorithm and
software, Challenge participants will take part in a broader NLM effort
to develop a freely usable software system and a freely accessible API
for image-based search and retrieval from a mobile device.
In a typical scenario for a future NLM mobile app, a person will
download the app and use it to photograph a prescription pill, possibly
under poor lighting conditions, from an angle, or at low resolution.
The future app will communicate with the future pill image recognition
software system, which may use all or part of any Challenge entry, to
compare that photo to reference images in the RxIMAGE database, and
will return one or more reference images that most likely match the
photographed pill along with their associated pill data.

[[Page 2877]]

NLM provides two directories of images to Challenge participants
for use in preparing their submissions:
Directory DR contains 2000 JPEG reference images of 1000
pills. For each pill there are two reference images, one of the front
of the pill and one of the back of the pill. These pill images are the
same as in the RxIMAGE database.
Directory DC contains 5000 JPEG consumer-quality images of
the same 1000 pills that were photographed for DR. However, they were
taken with a variety of digital cameras, under various lighting
conditions, and at camera angles not necessarily perpendicular to the
faces of the pills. They are akin to photos of prescription pills that
the general public might take.
For a pill for which there is at least one consumer-quality image
in DC, DR has two reference images of that pill, one of the front of
the pill and one of the back of the pill. Conversely, for a pill for
which there are two reference images in DR (one of the front of the
pill and one of the back of the pill), DC has two or more consumer-
quality images of that pill, taken under different conditions.
DR and DC come with a ``ground truth table'' that is a two-column
table with column headers ref_images and cons_images. Each row of the
table gives in the first column the name of a reference image and in
the second column the name of a consumer-quality image corresponding to
that reference image. There is a separate row in the table for each
(reference image, consumer-quality image) pair, even when multiple
reference and consumer-quality images are all photos of the same pill.
Respondents can use the images in DR and DC, the ground truth table,
and the RxIMAGE database in developing, training, and validating their
algorithms and software. They can also supplement these data.

Rules for Participating in the Challenge

Teams of one or more members can participate in this Challenge.
There is no maximum team size. Each team must have a captain.
Individual team members and team captains must register in accordance
with the Registration Process for Participants below. The role of the
team captain is to serve as the corresponding participant with NLM
about the Challenge and to submit the team's Challenge entry. While NLM
will notify all registered Challenge participants by email of any
amendments to the Challenge, the team captain is expected to keep the
team members informed about matters germane to the Challenge.
(1) To be eligible to win the Challenge prize, a team--
a. Shall have registered to participate in the Challenge under the
rules promulgated by the NIH as published in this Notice;
b. Shall have complied with all the requirements set forth in this
Notice;
c. In the case of a private entity, shall be incorporated in and
maintain a primary place of business in the United States, and in the
case of an individual, whether participating singly or in a group,
shall be a citizen or permanent resident of the United States. However,
non-U.S. citizens and non-permanent residents can participate as a
member of a team that otherwise satisfies the eligibility criteria.
Non-U.S. citizens and non-permanent residents are not eligible to win a
monetary prize (in whole or in part). Their participation as part of a
winning team, if applicable, may be recognized when the results are
announced.
d. May not be a Federal entity;
e. May not be a Federal employee acting within the scope of the
employee's employment and further, in the case of HHS employees, may
not work on their submission(s) during assigned duty hours. Note:
Federal ethical conduct rules may restrict or prohibit Federal
employees from engaging in certain outside activities, so any Federal
employee seeking to participate in this Challenge outside the scope of
employment should consult his/her agency's ethics official prior to
developing an submission;
f. May not be an employee of the NIH, a judge of the challenge, or
any other party involved with the design, production, execution, or
distribution of the Challenge or the immediate family of such a party
(i.e., spouse, parent, step-parent, child, or step-child).
g. All team members must be at least 18 years old at the time of
submission.
(2) Federal grantees may not use Federal funds to develop their
Challenge submissions unless use of such funds is consistent with the
purpose of their grant award and specifically requested to do so due to
the Challenge design, and as announced in the Federal Register.
(3) Federal contractors may not use Federal funds from a contract
to develop their Challenge submissions or to fund efforts in support of
their Challenge submission.
(4) By participating in this Challenge, each individual (whether
competing singly or in a group) and entity agrees to assume any and all
risks and waive claims against the Federal government and its related
entities (as defined in the COMPETES Act), except in the case of
willful misconduct, for any injury, death, damage, or loss of property,
revenue, or profits, whether direct, indirect, or consequential,
arising from participation in this Challenge, whether the injury,
death, damage, or loss arises through negligence or otherwise.
(5) Based on the subject matter of the Challenge, the type of work
that it will possibly require, as well as an analysis of the likelihood
of any claims for death, bodily injury, property damage, or loss
potentially resulting from Challenge participation, no individual
(whether competing singly or in a group) or entity participating in the
Challenge is required to obtain liability insurance or demonstrate
financial responsibility in order to participate in this Challenge.
(6) By participating in this Challenge, each individual (whether
competing singly or in a group) and entity agrees to indemnify the
Federal government against third party claims for damages arising from
or related to Challenge activities.
(7) An individual or entity shall not be deemed ineligible because
the individual or entity used Federal facilities or consulted with
Federal employees during the Challenge if the facilities and employees
are made available to all individuals and entities participating in the
Challenge on an equitable basis.
(8) By participating in this Challenge, each individual (whether
participating singly or in a group) and entity grants to the NIH, in
any existing or inchoate copyright or patent rights owned by the
individual or entity, an irrevocable, paid-up, royalty-free,
nonexclusive worldwide license to use, reproduce, post, link to, share,
and display publicly on the Web the submission, except for source code.
This license includes without limitation posting or linking to the
submission, except for source code, on the NLM Pill Image Recognition
Web site [http://pir.nlm.nih.gov/challenge]. In developing its future
software system and future API, NLM may include algorithms and software
from Challenge entries and may consult with individuals or teams that
submitted entries. Thus, the license also permits NLM to develop the
future software system and the future API, independently or with
others, using any algorithms or software from Challenge entries,
including those obtained from other Challenges or solicitations, and
NLM may freely use, reproduce, modify and distribute the resulting
future software system and API without restriction. NLM may work with
individuals or teams that submitted entries to write articles about
pill image recognition and submit them to peer-

[[Page 2878]]

reviewed journals. Each participant will retain all other intellectual
property rights in their submissions, as applicable.
(9) NIH reserves the right, in its sole discretion, to (a) cancel,
suspend, or modify the Challenge through amendment to this Federal
Register notice, and/or (b) not award any prizes if no entries are
deemed worthy. In addition, NLM reserves the right to disqualify any
Challenge participants or entries in instances where cheating or other
misconduct is identified.
(10) Each individual (whether participating singly or in a group)
or entity agrees to follow all applicable federal, state, and local
laws, regulations, and policies.
(11) Each individual (whether participating singly or in a group)
and entity participating in this Challenge must comply with all terms
and conditions of these rules, and participation in this Challenge
constitutes each such participant's full and unconditional agreement to
abide by these rules. Winning is contingent upon fulfilling all
requirements herein.
(12) Each individual (whether participating singly or in a group)
and entity grants to NLM and NLM contractors assisting NLM with C3PI
the right to review the submission, study the algorithms and the code,
and run the software on other sets of images.
(13) Submissions must not infringe upon any copyright, patent,
trade secrets, or any other rights of any third party. Each individual
(whether participating singly or in a group) or entity warrants that
he/she or the team is the sole author and owner of any copyrightable
work that the submission comprises, that the submission is wholly
original with the participant or is an improved version of an existing
work that the participant has sufficient rights to use and improve. In
addition, the submission must not trigger any reporting or royalty
obligation to any third party. A submission must not include
proprietary, classified, confidential, or sensitive information.
(14) The submission does not contain malicious code such as
viruses, timebombs, cancelbots, worms, trojan horses, or other
potentially harmful programs or other material or information.
(15) Notwithstanding the above and consistent with the principal
objective of the Challenge to make results widely available to the
public. If the submitter distributes their executable code or source
code NLM encourages every individual and team to distribute their
submission's executable code and preferably also its source code to the
public under an Apache 2.0 License that permits the public to benefit
from and improve upon the submission. In this case and by mutual
agreement, NLM may also post or link to the code from the Pill Image
Recognition Web site. The entry must include a description of how and
under what license terms it intends to make any code that is part of
the entry available to the public.
(16) Challenge participants are free to discuss their submission
and the ideas and technologies that it contains with other parties,
except as stated in #7 above and are free to contract with any third
parties so long as they do not sign any agreement or undertake any
obligation that conflicts with any agreement that they have entered
into, such as with any team members, or do enter into regarding their
submission for the Challenge. For the purpose of clarity, Challenge
participants acknowledge that the intent of the Challenge is to
encourage people to collaborate and share ideas and innovations.

Registration Process for Participants

To participate in this Challenge, team captains must register their
teams, including providing the names and email addresses of all team
members, at http://pir.nlm.nih.gov/challenge/register. Early
registration is encouraged in order to be able to receive email
notifications if this Federal Register notice is amended to change the
timeline or the rules of this Challenge.

Submission Requirements

Participants must provide a complete submission as defined below to
be considered for the prize. The submission must be saved to a USB
storage device containing a virtual machine and must be received (not
simply post-marked) by NLM by May 31, 2016, 5 p.m. EDT. Submissions
must be mailed to: Pill Image Recognition Challenge, Computational
Photography Project for Pill Identification (C3PI), National Library of
Medicine, Building 38A, Room B1-N30, 8600 Rockville Pike, Bethesda, MD
20894.
The Submission is defined to include:
1. Executable software for ranking how well consumer images of
pills taken by digital cameras match reference images. The software
shall be a batch-mode program or a script whose input consists of a
directory of consumer images and a directory of reference images. The
output shall be a comma-separated-value (csv) M-by-N matrix MR of ranks
that for i = 1,...,M compares consumer image i with reference images j
= 1,...,N. For each consumer image, no rank can appear more than once.
The software does not need to identify pills by name.
2. Source code for the executable that is both human- and machine-
readable. The source code can be written in any programming
language(s).
3. A .csv file containing the matrix MRC (C for Challenge) of ranks
that is the output from executing the executable using DC and DR as
input. In this case MRC is a matrix that has at least 5000 rows and has
2000 columns. For each row i, MRC(i,j) will rank how well reference
image j matches consumer-quality image i, for j = 1,...,2000. If
reference image J best matches consumer-quality image I then MRC(I,J) =
1, and if reference image K is the worst match to consume-quality image
I then MRC(I,K) = 2000.
4. A text file written in English and containing the algorithm in
pseudo-code that the source code implements, and a description of how
it works and any tools or packages that it uses. The pseudo-code is to
have the complete pipeline from the input directories to a matrix of
ranks, and also include any code that implements features or does
offline training.
5. A one-page text file written in English that contains the
following:
a. Title of entry
b. Names and email addresses of the team captain and all team members
c. A five or more character identifier for the entry that is used as a
prefix in the names of all of the team's submitted files
d. A brief description of the submission

Amount of the Prize; Award Approving Official

Up to five monetary prizes may be awarded: $25,000 for 1st Place,
$15,000 for 2nd Place, $5,000 for 3rd Place, and two $2,500 prizes for
Honorable Mention for a total prize award pool of up to $50,000. The
names of the winners and the titles of their entries will be posted on
NLM Web sites. The Award Approving Official is the Director of the
National Library of Medicine.

Payment of the Prize

Prizes awarded under this Challenge will be paid by electronic
funds transfer and may be subject to Federal income taxes. HHS/NIH will
comply with IRS (U.S. Internal Revenue Service) withholding and
reporting requirements, where applicable.

Basis Upon Which Winners Will Be Selected

NLM will first review submissions to determine their suitability
for judging and eligibility to win the prize. An

[[Page 2879]]

eligible submission (a) complies with the rules in this Federal
Register notice, (b) follows the detailed submission instructions at
http://pir.nlm.nih.gov/challenge/, (c) is complete (e.g., meets the
Submission Requirements above), and (d) is confirmed by NLM that the
executable is an implementation of the submitted source code, can be
run on the submitted virtual machine using directories DC and DR as
input, and the output matrix is the same as the submission's matrix
MRC.
Eligible submissions will proceed to the judging process, and will
be evaluated based on how well the submissions can match pill images
across a large number of queries. NLM has developed evaluation software
and created directories of images to use in selecting the Challenge
winners. For example, directories DRJ and DCJ (J for Judging) contain
reference and consumer-quality images similar to those in directories
DR and DC provided to potential Challenge participants, and the ground
truth matrix for judging MGTJ has MGTJ(i,j)=1 if DCJ(i) and DRJ(j) are
photos of the same pill, else MGTJ(i,j)=0. The evaluation software
calculates the mean average precision (MAP) for directories and ground
truth matrix such as DRJ, DRC, and MGTJ. Mean average precision is a
widely used measure for evaluating how well information retrieval
systems (for example, search engines) retrieve results across a large
number of queries. The MAP formula and examples of MAP calculations for
this Challenge are at http://pir.nlm.nih.gov/challenge/MAP_example.
The eligible submissions will be submitted to the evaluation
software. The submission with the highest MAP score will be recommended
as the first place winner, with the second, third, fourth, and fifth
best MAPs, respectively being recommended to earn second place, third
place, and two honorable mentions. In the event of tied scores, the
tied submissions will be tested against additional DRJ and DCJ
directories until a winner is determined. Dr. Terry Yoo will serve as
the judge for the competition, thereby overseeing the evaluation
software process and being responsible for its proper application.

Additional Information

The NLM Computational Photography Project for Pill Identification
(C3PI) is a research and development project in the Office of High
Performance Computing and Communications (OHPCC) within the NLM Lister
Hill National Center for Biomedical Communications (LHNCBC). C3PI
computer scientists conduct computer vision R&D in text- and image-
based search and retrieval. C3PI's overall goal is to help improve the
prescription drug information made available to health professionals
and consumers. The NLM ``Pill Image Recognition Request for
Information'' (PIR RFI) (https://www.fbo.gov/index?s=opportunity&mode=form&id=a1a694718366ea7cbaf8f715047d63e1&tab=core&_cview=0) was a pilot for this Challenge. The RFI was announced in
February 2015 and responses were due in May 2015. RFI responses were
used to test, evaluate, and as needed refine the components of the
Challenge, including its instructions. NLM appreciates the work done by
the parties that responded to the RFI.

Dated: January 11, 2016.
Betsy L. Humphreys,
Acting Director, National Library of Medicine, National Institutes of
Health.
[FR Doc. 2016-00777 Filed 1-15-16; 8:45 am]
BILLING CODE 4140-01-P

Document Created: 2016-01-16 01:11:27
Document Modified: 2016-01-16 01:11:27

Category		Regulatory Information
Collection		Federal Register
sudoc Class		AE 2.7: GS 4.107: AE 2.106:
Publisher		Office of the Federal Register, National Archives and Records Administration
Section		Notices
Dates		NLM will make a set of consumer-quality images and a companion set of reference images publicly available on January 15, 2016.
Contact		Michael J. Ackerman, Ph.D. at (301) 402-4100 or [email protected]
FR Citation		81 FR 2876

81 FR 2876 - Announcement of Requirements and Registration for “Pill Image Recognition Challenge”

DEPARTMENT OF HEALTH AND HUMAN SERVICESNational Institutes of Health

Federal Register Volume 81, Issue 11 (January 19, 2016)

DEPARTMENT OF HEALTH AND HUMAN SERVICES
National Institutes of Health