The 2014 Challenge
The purpose of the Teach-Discover-Treat (TDT) initiative is to develop and disseminate computational workflows for drug discovery. As such, the competition puts a premium on reproducibility. Workflows submitted to TDT should utilize freely available software tools. In order to be evaluated, submissions should include a tutorial as well as all models, scripts, and other tools necessary to execute the workflow. Below are the 4 Challenges along with the Judging Criteria and Panel, Important Dates, Notes for Developers of Commercial Software, the Awards, and Submission Package requirements.
Challenge 1 :: Malaria High-throughput Screen
Use a dataset from a high-throughput phenotypic screen for Malaria (single concentration screening data and selected IC50s) to build a predictive model for anti-Malaria activity, and use that model to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
Screening set: 305,568 compounds with single concentration percent inhibition data
Held-out test set: 1,056 compounds with unpublished IC50 data.
Discovery opportunity: Commitment from our partners at St. Jude to screen at least 100 compounds in the Malaria phenotypic screen for an overall winner.
Challenge 2 :: Discover new anti-Malaria compounds
Use the apo structure of a validated Malaria target, Lysyl tRNA synthetase, to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
Held-out test set: Screening data and selected IC50s for compounds in TCAMS (Tres Cantos Anti Malarial Set).
Discovery opportunity: Commitment from our partners at the Structural Genomics Consortium (Toronto) to screen at least 100 compounds in the Lysyl tRNA synthetase biochemical assay for an overall winner.
Challenge 3 :: CDPK1 Virtual Screening
Develop a virtual screening model for a parasitology drug target, the calcium-dependent protein kinase 1 (CDPK1) of Eimeria tenella. Selections may be based on known SAR and/or liganded structures of this target in different species (Toxoplasma gondii, Cryptosporidium parvum). Use the model to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
Held-out test set: IC50 data on 22 compounds for C. parvum, T. gondii and E. tenella CDPK1.
Discovery opportunity: Commitment from our partners at University of Washington to screen at least 100 compounds in the E. tenella CDPK1 biochemical assay for an overall winner.
Challenge 4 :: Open Innovation
Share an innovative computational chemistry workflow that can impact drug discovery for neglected diseases. Examples can include, but are not limited to, workflows for target identification, scaffold-hopping, property predictions, target selection (various flavors of druggability assessment).
Workflow components: The submissions must contain a clear tutorial for a computational chemistry workflow that is relevant to drug discovery. If the example dataset used in the tutorial is not a neglected disease dataset, the workflow has to be adaptable for use in a neglected disease setting.
The purpose of the Teach-Discover-Treat (TDT) initiative is to develop and disseminate computational workflows for drug discovery. As such, the competition puts a premium on reproducibility. Workflows submitted to TDT should utilize freely available software tools. In order to be evaluated, submissions should include a tutorial as well as all models, scripts, and other tools necessary to execute the workflow. Below are the 4 Challenges along with the Judging Criteria and Panel, Important Dates, Notes for Developers of Commercial Software, the Awards, and Submission Package requirements.
Challenge 1 :: Malaria High-throughput Screen
Use a dataset from a high-throughput phenotypic screen for Malaria (single concentration screening data and selected IC50s) to build a predictive model for anti-Malaria activity, and use that model to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
- Analysis of single concentration screening data: hit list triaging, selection of compounds for IC50
- Building and validating a predictive activity model, including predicting activity in a held-out test set
- Follow-up hit-finding: applying predictive model to rank-order commercially available compounds
Screening set: 305,568 compounds with single concentration percent inhibition data
Held-out test set: 1,056 compounds with unpublished IC50 data.
Discovery opportunity: Commitment from our partners at St. Jude to screen at least 100 compounds in the Malaria phenotypic screen for an overall winner.
Challenge 2 :: Discover new anti-Malaria compounds
Use the apo structure of a validated Malaria target, Lysyl tRNA synthetase, to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
- Preparation of crystal structure for virtual screening, including generation of binding pose for Cladosporin (published inhibitor) and activity prediction for held-out test set
- Follow-up hit-finding: applying model to rank-order commercially available compounds
Held-out test set: Screening data and selected IC50s for compounds in TCAMS (Tres Cantos Anti Malarial Set).
Discovery opportunity: Commitment from our partners at the Structural Genomics Consortium (Toronto) to screen at least 100 compounds in the Lysyl tRNA synthetase biochemical assay for an overall winner.
Challenge 3 :: CDPK1 Virtual Screening
Develop a virtual screening model for a parasitology drug target, the calcium-dependent protein kinase 1 (CDPK1) of Eimeria tenella. Selections may be based on known SAR and/or liganded structures of this target in different species (Toxoplasma gondii, Cryptosporidium parvum). Use the model to select the next set of compounds for screening in order to identify high quality chemical starting points for optimization.
Workflow components:
- Validation of the use of CDPK1 SAR and/or crystal structures from C. parvum [PDB ID: 3NCG] and T. gondii [PDB ID: 3I79] in virtual screening, including activity prediction of held-out test set
- Creation of a virtual screening model for E. tenella CDPK1 [NCBI Accession number CAA96439 (GI: 1279425)], including activity prediction of held-out test set
- Follow-up hit-finding: applying model to rank-order commercially available compounds for Eimeria tenella CDPK1
Held-out test set: IC50 data on 22 compounds for C. parvum, T. gondii and E. tenella CDPK1.
Discovery opportunity: Commitment from our partners at University of Washington to screen at least 100 compounds in the E. tenella CDPK1 biochemical assay for an overall winner.
Challenge 4 :: Open Innovation
Share an innovative computational chemistry workflow that can impact drug discovery for neglected diseases. Examples can include, but are not limited to, workflows for target identification, scaffold-hopping, property predictions, target selection (various flavors of druggability assessment).
Workflow components: The submissions must contain a clear tutorial for a computational chemistry workflow that is relevant to drug discovery. If the example dataset used in the tutorial is not a neglected disease dataset, the workflow has to be adaptable for use in a neglected disease setting.
Judging Criteria
The tutorials will be rated on a scale of 1-10 for each of the following 4 criteria:
Scientific content
Judging Panel
Pat Walters (Chair, Vertex), Yvonne Martin (Abbott, retired), Steve Johnson (BMS), Rajarshi Guha (NIH/NCATS), Steve Dixon (Schrödinger)
Note for Developers of Commercial Software
Entries from commercial software vendors who do not intend to make their code publicly available will be accepted for the data-driven challenges (not for the open innovation challenge) under the condition that the entries are not eligible for monetary and travel awards. We will reserve a couple of slots for vendor presentations at the Award symposium, selecting them based on performance against the held-out test data and innovation. We also ask that the vendor makes the relevant scripts and parameter files needed to reproduce their work available for those who have licenses. In addition, we strongly recommend they consider making predictive models they develop using TDT training data publicly available in open access format as a black box (not requiring a license to their software) through, for example, a web service or KNIME node.
Awards
There will be several awards available that consist of $500 as a base award with an added travel award, up to $1,000, which will be reimbursed against receipts for travel expenses to attend the Award symposium. An overall winner will be announced plus 2 runner-up awards using the criteria mentioned above. Requirement for receiving the award money and travel award is that the winners present at the Award symposium at the Fall 2014 ACS National Meeting in San Francisco, California.
Important Dates
Competition opens November 4, 2013
Competition closes March 10, 2014 <- Updated!
Winners announced & drug discovery efforts start April 14, 2014 <- Updated!
Awardee presentations at the San Francisco ACS meeting (Fall 2014)
Submission Package
Tarball with all files needed to run through the workflow:
Information to be entered in submission form
Submit your tutorial and data here: http://file.teach-discover-treat.org/submit/index.php
The tutorials will be rated on a scale of 1-10 for each of the following 4 criteria:
Scientific content
- What is the quality of the underlying science?
- Is the tutorial relevant to drug discovery?
- Are the objectives of the tutorial clearly defined?
- Does the tutorial provide a logical workflow?
- Does the tutorial provide sufficient background?
- Will the tutorial stimulate interest in computational chemistry and molecular modeling?
- Does the tutorial provide a gateway to further research?
- Can the work be easily reproduced?
- Is all of the software readily available?
- Does the tutorial utilize commercial software?
- Does the tutorial require a complex installation?
Judging Panel
Pat Walters (Chair, Vertex), Yvonne Martin (Abbott, retired), Steve Johnson (BMS), Rajarshi Guha (NIH/NCATS), Steve Dixon (Schrödinger)
Note for Developers of Commercial Software
Entries from commercial software vendors who do not intend to make their code publicly available will be accepted for the data-driven challenges (not for the open innovation challenge) under the condition that the entries are not eligible for monetary and travel awards. We will reserve a couple of slots for vendor presentations at the Award symposium, selecting them based on performance against the held-out test data and innovation. We also ask that the vendor makes the relevant scripts and parameter files needed to reproduce their work available for those who have licenses. In addition, we strongly recommend they consider making predictive models they develop using TDT training data publicly available in open access format as a black box (not requiring a license to their software) through, for example, a web service or KNIME node.
Awards
There will be several awards available that consist of $500 as a base award with an added travel award, up to $1,000, which will be reimbursed against receipts for travel expenses to attend the Award symposium. An overall winner will be announced plus 2 runner-up awards using the criteria mentioned above. Requirement for receiving the award money and travel award is that the winners present at the Award symposium at the Fall 2014 ACS National Meeting in San Francisco, California.
Important Dates
Competition opens November 4, 2013
Competition closes March 10, 2014 <- Updated!
Winners announced & drug discovery efforts start April 14, 2014 <- Updated!
Awardee presentations at the San Francisco ACS meeting (Fall 2014)
Submission Package
Tarball with all files needed to run through the workflow:
- Tutorial
- All data and result files from the exemplified workflow
- Computational tools, scripts and models for all steps in the exemplified workflow
- For source code or executables: Instructions on how to access / download from free web servers preferred
- Include exact version numbers
Information to be entered in submission form
- Abstract (≦ 150 words) Please Note: This abstract will be posted on the download page with the tutorial upon approval of the judging panel. If the submission is selected as a winner, it will also be used to submit as an abstract for the Award symposium at the ACS Fall 2014 meeting. Please make sure the abstract is appropriate for both purposes.
- Keywords
- Author list
- Presenting author name, institution, email, and phone number
- Citation language (how people should cite models or workflows when used outside of TDT challenges)
Submit your tutorial and data here: http://file.teach-discover-treat.org/submit/index.php