Registered report: Systematic identification of genomic markers of drug sensitivity in cancer cells

The Reproducibility Project: Cancer Biology seeks to address growing concerns about the reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from “Systematic identification of genomic markers of drug sensitivity in cancer cells” by Garnett and colleagues, published in Nature in 2012 (Garnett et al., 2012). The experiments to be replicated are those reported in Figures 4C, 4E, 4F, and Supplemental Figures 16 and 20. Garnett and colleagues performed a high throughput screen assessing the effect of 130 drugs on 639 cancer-derived cell lines in order to identify novel interactions for possible therapeutic approaches. They then tested this approach by exploring in more detail a novel interaction they identified in which Ewing’s sarcoma cell lines showed an increased sensitivity to PARP inhibitors (Figure 4C). Mesenchymal progenitor cells (MPCs) transformed with the signature EWS-FLI1 translocation, the hallmark of Ewing’s sarcoma family tumors, exhibited increased sensitivity to the PARP inhibitor olaparib as compared to MPCs transformed with a different translocation (Figure 4E). Knockdown mediated by siRNA of EWS-FLI1 abrogated this sensitivity to olaparib (Figure 4F). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife. DOI: http://dx.doi.org/10.7554/eLife.13620.001


Introduction
In their 2012 Nature paper, Garnett and colleagues implemented a large-scale high throughput in vitro screen designed to assess interactions between drugs and cancer-derived human cell lines (Garnett et al., 2012). This study leveraged a collection of over 600 cell lines screened across 130 drugs, with the aim to uncover new interactions between known cancers and known drugs in order to identify new potential therapeutic avenues using extant drugs. They captured a large number of known gene-drug interactions of clinically active drugs and identified several novel gene-drug associations. The ability to accurately capture a large number of known clinically relevant drug response biomarkers as well as preferential cancer type sensitivities known to occur in the clinic, such as decreased sensitivity to BRAF inhibitors in BRAF mutant colorectal cancers relative to melanomas, demonstrated the effectiveness of this large-scale pharmacogenomic approach. A similar approach of interrogating a large panel of human cancer cell lines of diverse lineages to predict drug sensitivity was conducted and reported by Barretina and colleagues at the same time (Barretina et al., 2012).
Garnett and colleagues identified an unexpected highly significant association between the EWS-FLI1 translocation and sensitivity to the PARP inhibitor olaparib (Garnett et al., 2012). The EWS-FLI1 translocation is a defining cytogenetic characteristic of Ewing's sarcoma family tumors (ESFTs). ESFTs are highly malignant tumors that occur in the bone and soft tissue, usually in children. The translocation event combines part of the EWS protein to a member of the ETS transcription factor family; in 90% of cases, this is FLI1. This creates a novel transcription factor, EWS-FLI1, whose oncogenic actions and mechanisms are still being fully explored. The translocation event is thought to be the initiating event for the development of ESFTs (Erkizan et al., 2010;Lessnick and Ladanyi, 2012).
PARP1 has diverse functions in chromatin modification, mitosis and cell death, but it is most well studied in the context of DNA repair and transcriptional regulation (Sonnenblick et al., 2014). PARP1 is a key component of single stranded break (SSB) repair; however, loss of PARP1 activity can be compensated for through DNA repair via homologous recombination (HR). This makes PARP1 an interesting therapeutic target in the context of malignancies with deficient HR, such as BRCA1 and BRCA2 mutant breast and ovarian cancers. In these cancers, loss of PARP activity results in synthetic lethality; with both SSB and HR impaired, the accumulation of DNA damage eventually kills the tumor cells (Jiang et al., 2015;Lord et al., 2015;Sonnenblick et al., 2014). PARP inhibitors (PARPi), such as olaparib, are now at the forefront of treatment for breast and ovarian cancers, as well as other malignancies (Feng et al., 2015).
In Figure 4C, a predicted interaction between Ewing's sarcoma cells and the PARP inhibitor olaparib was tested. PARP inhibitors target BRCA-deficient cells that rely on alternative DNA damage repair pathways involving PARP. A panel of cell lines representing Ewing's sarcoma, a BRCA-deficient line, as well as other osteosarcomas and cancers of soft tissue and epithelium were treated with a range of concentrations of olaparib. The concentration of olaparib required to reduce colony formation by 90% or more was much less for Ewing's sarcoma cells (on par with the concentration required for the BRCA-deficient cell line) than for the non-Ewing's sarcoma cell lines. This experiment will be replicated in Protocol 1.
In Figure 4E, the hypothesis that mouse mesenchymal progenitor cells (MPCs) that had been transformed with the EWS-FLI1 translocation would confer sensitivity to olaparib was tested. The sensitivity of these cells to olaparib were compared to MPCs transformed with a related translocation (FUS-CHOP) as well as to SK-N-MC cells, which have the EWS-FLI1 translocation endogenously. Treatment with olaparib did not inhibit the viability of the FUS-CHOP transformed MPCs, but did inhibit the viability of the SK-N-MC cells. Olaparib also inhibited the viability of the EWS-FLI1 transformed MEFs compared to the FUS-CHOP translocation. This experiment will be replicated in Protocol 2.
In Figure 4F, the effects of EWS-FLI1 depletion on a cell line carrying the translocation endogenously was tested. A673 cells were transfected with siRNAs targeting EWS-FLI1, which resulted in a partial rescue of sensitivity to olaparib compared to control siRNA transfected cells. This experiment will be replicated in Protocol 3.
A paper published at the same time as Garnett and colleagues' work also confirmed that Ewing's sarcoma cell lines were sensitive to treatment with PARP inhibitors (Brenner et al., 2012). In a previous paper, Brenner and colleagues reported that in prostate cancer PARP was a cofactor for wildtype ETS transcription factors, which makes up one half of the defining translocation-based fusion transcription factor of Ewing's sarcoma, and that PARPi treatment of ETS positive prostate cancers disrupted their growth (Brenner et al., 2011;Legrand et al., 2013). Based on this finding, they examined the role of PARP1 and PARPi in Ewing's sarcoma. Using immunoprecipitation, they detected a direct interaction between the EWS-FLI1 fusion transcription factor and PARP1 (Brenner et al., 2012). Further, they reported that transforming a cell line (in this case, PC3 cells) with the EWS-FLI1 translocation conferred sensitivity to treatment with olaparib, and that siRNA mediated knockdown of EWS-FLI1 inhibited transwell migration of ESFT derived cell lines, but not osteosarcoma cell lines (Brenner et al., 2012). Multiple groups have also reported the unique sensitivity of EWS-FLI1 carrying Ewing's sarcoma derived cell lines to olaparib (Lee et al., 2013;Norris et al., 2014;Ordó ñez et al., 2015). Additional work then demonstrated that, similar to breast and ovarian cancers harboring BRCA1/2 mutations, Ewing's sarcomas may also have defects in DNA repair mechanisms, rendering them sensitive to PARP inhibition (Stewart et al., 2014). This has led to the start of clinical trials treating Ewing's sarcoma patients with combination therapies targeting multiple DNA damage pathways and PARP inhibition. Results from a small scale nonrandomized phase II human trial failed to show clinical efficacy in patients with metastatic and/or recurrent Ewing sarcoma treated with only olaparib (Choy et al., 2014), but other trials are underway to explore the efficacy of PARP inhibition in combination with chemotherapy.

Materials and methods
Unless otherwise noted, all protocol information and references were derived from the original paper or information obtained directly from the authors.
Protocol 1: Colony formation assay of Ewing's sarcoma cell lines with olaparib This experiment assesses the sensitivity of Ewing's sarcoma cell lines to the PARP inhibitor olaparib. A colony formation assay will be performed with Ewing's sarcoma, osteosarcoma, and BRCA2-deficient and BRCA-proficient cells treated with a range of olaparib concentrations to determine the effective concentration (number of colonies reduced by at least 90%). This protocol replicates the experiment reported in Figure 4C and Supplemental Figure 16.

Sampling
. The experiment will be performed with two replicates and each experiment will use 5 Ewing's sarcoma cell lines and 7 osteosarcoma cell lines for a power of 82%.
See Power calculations for details. . The experiment will use the following cell lines: Ewing's sarcoma cell lines:  . All cell lines will be sent for STR profiling and mycoplasma testing. . MES-SA cells are maintained in McCoy's 5A Medium Modified supplemented with 10% FBS. . All cells kept at 37˚C and 5% CO 2 . . Olaparib is stored as a 10 mM stock in DMSO at -80˚C. Each aliquot is subjected to no more than 5 freeze-thaw cycles. c. Fix in ice-cold methanol for 30 min while gently shaking at room temperature. d. Remove methanol and add Giemsa stain at 1:20 dilution in deionized water. Incubate for 4 hr at room temperature shaking or overnight at 4˚shaking. e. 4 hr later, or the following day, rinse cells with water and air dry. 5. Take brightfield images of plates and manually quantify the number of colonies, blinded, in each well from each plate. 6. Determine and record the concentration at which colony formation was reduced by >90% compared to DMSO controls for each plate.

Materials and reagents
Deliverables . Data to be collected: Images of all plates Colony counts of each well Graph of each cell line and the concentration of olaparib required to reduce colony formation by >90% compared to DMSO controls. (Compare to Figure 4C)

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Wilcoxon-Mann-Whitney test for ordinal data of the effective concentration of olaparib to reduce the colonies by at least 90% in Ewing's sarcoma compared to osteosarcoma cell lines. Perform for each group (A or B) of replicate plates.
. Meta-analysis of original and replication attempt effect sizes: This replication attempt will perform the statistical analysis listed above, compute the effect sizes (for each independent attempt), compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Known differences from the original study . The replication attempt will only examine Ewing's sarcoma and osteosarcoma derived cell lines, with the BRCA2-deficient cell line as a positive control, and will not include the remaining cell types (soft tissue and epithelial).
. Due to the inability to obtain any of the Ewing's sarcoma cell lines used originally, and in consultation with the original authors, the replication attempt will use A673, TC-71, CHLA-9, SK-N-MC and CHLA-10 cells. The cell lines all carry the critical EWS/FLI1 translocation. The cells used in the original study were ES1, ES6, ES7, ES8, and MHH-ES-1.
. Similarly, the replication attempt will use U-2-OS, SJSA-1, SAOS-2, HOS, MG-63, 143B, and G-292 cells. 143B and G-292 cells were not used in the original study and CAL-72, HuO-3N1, and NY cells that were used in the original study will not be included in this replication attempt.
. All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
The cell lines used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The DMSO concentration, although not originally reported, will be kept at a low percentage to avoid toxicity. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Protocol 2: Olaparib sensitivity in cells transformed with the EWS-FLI1 rearrangement
This experiment assesses if sensitivity to PARP inhibitors is due to the presence of the EWS-FLI1 rearrangement. Mouse mesenchymal progenitor cells (MPCs) transformed with EWS-FLI1, or the related liposarcoma-associated translocation FUS-CHOP, will be analyzed for cellular viability after olaparib treatment. This protocol replicates the experiment reported in Figure 4E.

Sampling
. The experiment will be repeated three times for a power of 99%. Deliverables . Data to be collected: Raw data of fluorescent readout for all wells Graph of normalized readings for each drug concentration compared to vehicle only control (Compare to Figure 4E)

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantilequantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the planned comparisons using the equivalent non-parametric test.
One way ANOVA on IC 50 values of olaparib, determined by spline interpolation, of each cell line with the following planned comparisons using Fisher's LSD test: This replication attempt will perform the statistical analysis listed above, compute the effect sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Known differences from the original study . Commercially available LIF will be used in place of LIF generated from CHO LIF720D cells, as suggested by the original authors.
. All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
The cell lines used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The DMSO concentration, although not originally reported, will be kept at a low percentage to avoid toxicity. The seeding density of each cell line will be empirically determined prior to conducting the replicates so cells will be still be in the growth phase at the end of the assay. Measurements will be taken at 24, 48, and 72 hr after seeding from cells not treated with drug to test for proliferation of cells during the assay. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Protocol 3: Olaparib sensitivity after depletion of EWS-FLI1 from A673 cells
This experiment assesses the sensitivity of PARP inhibitors to the presence of the EWS-FLI1 rearrangement. EWS-FLI1 specific siRNA will be used to deplete the fusion mRNA from A673 cells, which harbor the translocation endogenously, and cell viability after olaparib treatment will be assessed. This protocol replicates the experiment reported in Figure 4F and Supplemental Figure 20.

Sampling
. The experiment will be repeated three times for a minimum power of 80%. See Power calculations for details. . The experiment has 2 cohorts: Cohort1: siControl transfected A673 cells Cohort 2: siEF1 transfected A673 cells . Each cohort will be treated with the following conditions to assess cell viability in technical triplicate:

Procedure
Notes: . All cell lines will be sent for STR profiling and mycoplasma testing. . A673 cells are maintained in DMEM with 10% FBS. . All cells are kept at 37˚C and 5% CO 2 . . Olaparib is stored as a 10 mM stock in DMSO at -80˚C. Each aliquot is subjected to no more than 5 freeze-thaw cycles.
. siRNA stocks kept at 20 mM; final siRNA concentration is 25 nM.
1. Seed cells for assays: a. For cell viability assay, plate 5000 A673 cells per well in 64 ml medium without antibiotics in a 96-well plate. i. Seed enough cells for each condition to be performed in technical triplicate. b. For qRT-PCR, plate 3x10 4 A673 cells per well of a 24 well plate in medium without antibiotics.
i. This is a similar seeding density as the 96 well plate. 2. Immediately transfect cells with 25 nM siControl or siEF1 siRNAs using Lipofectamine RNAi-MAX with the cells in suspension. The following directions prepare enough transfection mixture for one 96-well plate. The amounts will be scaled accordingly to account for the plates used for the qRT-PCR analysis. Deliverables . Data to be collected: Raw absorbance values for all wells. Graph of absorbance corrected values for all concentrations of olaparib or DMSO normalized to untreated controls (as seen in Figure 4F).
IC 50 values for each cohort using normalized olaparib values. Raw and normalized qRT-PCR data (as seen in Supplemental Figure 20).

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantilequantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the planned comparisons using the equivalent non-parametric test.
o Viability assay: & Unpaired two-tailed t-test of olaparib IC 50 values of siControl transfected cells compared to siEF1 transfected cells.

qRT-PCR:
& Two-way ANOVA of siControl and siEF1 transfected cells treated with or without olaparib with the following planned comparisons using the Bonferroni correction: . siControl transfected cells treated with DMSO compared to siEF1 transfected cells treated with DMSO.
. siControl transfected cells treated with olaparib compared to siEF1 transfected cells treated with olaparib.
. Meta-analysis of original and replication attempt effect sizes: This replication attempt will perform the statistical analysis listed above, compute the effect sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Known differences from the original study . All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
The cell line used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The sample purity (A 260/280 ratio) of the isolated RNA from each sample will be reported. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Power calculations
For additional details on power calculations, please see analysis scripts and associated files on the Open Science Framework: https://osf.io/j9bnk/

Protocol 1
Summary of original data estimated from graph reported in Figure 4C Cell Protocol 2 Summary of original data reported in Figure 4E (shared by authors)

Protocol 3 Viability assay
Summary of original data reported in Figure 4F (shared by authors) Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).