Hit selection
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
In high-throughput screening (HTS), one of the major goals is to select compounds (including small molecules, siRNAs, shRNA, genes, et al.) with a desired size of inhibition or activation effects. A compound with a desired size of effects in an HTS screen is called a hit. The process of selecting hits is called hit selection.[citation needed]
Methods for hit selection in general
[edit]HTS experiments have the ability to screen tens of thousands (or even millions) of compounds rapidly. Hence, it is a challenge to glean chemical/biochemical significance from mounds of data in the process of hit selection. To address this challenge, appropriate analytic methods have been adopted for hit selection. There are two main strategies of selecting hits with large effects.[1] One is to use certain metric(s) to rank and/or classify the compounds by their effects and then to select the largest number of potent compounds that is practical for validation assays.[2][3] The other strategy is to test whether a compound has effects strong enough to reach a pre-set level. In this strategy, false-negative rates (FNRs) and/or false-positive rates (FPRs) must be controlled.[4][5][6][7][8][9][10][11]
There are two major types of HTS experiments, one without replicates (usually in primary screens) and one with replicates (usually in confirmatory screens). The analytic methods for hit selection differ in those two types of HTS experiments. For example, the z-score method is suitable for screens without replicates whereas the t-statistic is suitable for screens with replicate. The calculation of SSMD for screens without replicates also differs from that for screens with replicates.[1]
Screens without replicates
[edit]There are many metrics used for hit selection in primary screens without replicates. The easily interpretable ones are fold change, mean difference, percent inhibition, and percent activity. However, the drawback common to all of these metrics is that they do not capture data variability effectively. To address this issue, researchers then turned to the z-score method or SSMD, which can capture data variability in negative references.[12][13]
The z-score method is based on the assumption that the measured values (usually fluorescent intensity in log scale) of all investigated compounds in a plate have a normal distribution. SSMD also works the best under the normality assumption. However, true hits with large effects should behave very different from the majority of the compounds and thus are outliers. Strong assay artifacts may also behave as outliers. Thus, outliers are not uncommon in HTS experiments. The regular versions of z-score and SSMD are sensitive to outliers and can be problematic. Consequently, robust methods such as the z*-score method, SSMD*, B-score method, and quantile-based method have been proposed and adopted for hit selection in primary screens without replicates.[14][15]
In a primary screen without replicates, every compound is measured only once. Consequently, we cannot directly estimate the data variability for each compound. Instead, we indirectly estimate data variability by making a strong assumption that every compound has the same variability as a negative reference in a plate in the screen. The z-score, z*-score and B-score relies on this strong assumption; so are the SSMD and SSMD* for cases without replicates.
Screens with replicates
[edit]In a screen with replicates, we can directly estimate data variability for each compound, and thus we can use more powerful methods, such as SSMD for cases with replicates and t-statistic that does not rely on the strong assumption that the z-score and z*-score rely on. One issue with the use of t-statistic and associated p-values is that they are affected by both sample size and effect size.[16] They come from testing for no mean difference, thus are not designed to measure the size of small molecule or siRNA effects. For hit selection, the major interest is the size of effect in a tested small molecule or siRNA. SSMD directly assesses the size of effects.[17] SSMD has also been shown to be better than other commonly used effect sizes.[18] The population value of SSMD is comparable across experiments and thus we can use the same cutoff for the population value of SSMD to measure the size of siRNA effects.[19]
SSMD can overcome the drawback of average fold change not being able to capture data variability. On the other hand, because SSMD is the ratio of mean to standard deviation, we may get a large SSMD value when the standard deviation is very small, even if the mean is small. In some cases, a too small mean value may not have a biological impact. As such, the compounds with large SSMD values (or differentiations) but too small mean values may not be of interest. The concept of dual-flashlight plot has been proposed to address this issue. In a dual-flashlight plot, we plot the SSMD versus average log fold-change (or average percent inhibition/activation) on the y- and x-axes, respectively, for all compounds investigated in an experiment.[19] With the dual-flashlight plot, we can see how the genes or compounds are distributed into each category in effect sizes, as shown in the figure. Meanwhile, we can also see the average fold-change for each compound.[19][20]
See also
[edit]References
[edit]- ^ a b Zhang XHD (2011). Optimal High-Throughput Screening: Practical Experimental Design and Data Analysis for Genome-scale RNAi Research. Cambridge University Press. ISBN 978-0-521-73444-8.
- ^ Birmingham A, Selfors LM, Forster T, Wrobel D, Kennedy CJ, Shanks E, Santoyo-Lopez J, Dunican DJ, Long A, Kelleher D, Smith Q, Beijersbergen RL, Ghazal P, Shamu CE (2009). "Statistical methods for analysis of high-throughput RNA interference screens". Nature Methods. 6 (8): 569–75. doi:10.1038/nmeth.1351. PMC 2789971. PMID 19644458.
- ^ Zhang XHD (2010). "Genome-wide screens for effective siRNAs through assessing the size of siRNA effects". BMC Research Notes. 1: 33. doi:10.1186/1756-0500-1-33. PMC 2526086. PMID 18710486.
- ^ Malo N, Hanley JA, Cerquozzi S, Pelletier J, Nadon R (2006). "Statistical practice in high-throughput screening data analysis". Nature Biotechnology. 24 (2): 167–75. doi:10.1038/nbt1186. PMID 16465162. S2CID 6158255.
- ^ Zhang XH, Kuan PF, Ferrer M, Shu X, Liu YC, Gates AT, Kunapuli P, Stec EM, Xu M, Marine SD, Holder DJ, Stulovici B, Heyse JF, Espeseth AS (2009). "Hit selection with false discovery rate control in genome-scale RNAi screens". Nucleic Acids Research. 36 (14): 4667–79. doi:10.1093/nar/gkn435. PMC 2504311. PMID 18628291.
- ^ Klinghoffer RA, Frazier J, Annis J, Berndt JD, Roberts BS, Arthur WT, Lacson R, Zhang XH, Ferrer M, Moon RT, Cleary MA (2010). "A lentivirus-mediated genetic screen identifies dihydrofolate reductase (DHFR) as a modulator of beta-catenin/GSK3 signaling". PLOS ONE. 4 (9): e6892. doi:10.1371/journal.pone.0006892. PMC 2731218. PMID 19727391.
- ^ Zhang XHD (2010). "An effective method controlling false discoveries and false non-discoveries in genome-scale RNAi screens". Journal of Biomolecular Screening. 15 (9): 1116–22. doi:10.1177/1087057110381783. PMID 20855561.
- ^ Malo N, Hanley JA, Carlile G, Liu J, Pelletier J, Thomas D, Nadon R (2010). "Experimental design and statistical methods for improved hit detection in high-throughput screening". Journal of Biomolecular Screening. 15 (8): 990–1000. doi:10.1177/1087057110377497. PMID 20817887. S2CID 41358896.
- ^ Zhang XH, Lacson R, Yang R, Marine SD, McCampbell, Toolan DM, Hare TR, Kajdas J, Berger JP, Holder DJ, Heyse JF, Ferrer M (2010). "The use of SSMD-based false discovery and false non-discovery rates in genome-scale RNAi screens". Journal of Biomolecular Screening. 15 (9): 1123–31. doi:10.1177/1087057110381919. PMID 20852024.
- ^ Quon K, Kassner PD (2009). "RNA interference screening for the discovery of oncology targets". Expert Opinion on Therapeutic Targets. 13 (9): 1027–35. doi:10.1517/14728220903179338. PMID 19650760. S2CID 10714162.
- ^ Zhang XH, Marine SD, Ferrer M (2009). "Error rates and power in genome-scale RNAi screens". Journal of Biomolecular Screening. 14 (3): 230–38. doi:10.1177/1087057109331475. PMID 19211781.
- ^ Zhang XHD (2007). "A new method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays". Journal of Biomolecular Screening. 12 (5): 645–55. doi:10.1177/1087057107300645. PMID 17517904.
- ^ Zhang XH, Ferrer M, Espeseth AS, Marine SD, Stec EM, Crackower MA, Holder DJ, Heyse JF, Strulovici B (2007). "The use of strictly standardized mean difference for hit selection in primary RNA interference high-throughput screening experiments". Journal of Biomolecular Screening. 12 (4): 645–55. doi:10.1177/1087057107300646. PMID 17435171. S2CID 7542230.
- ^ Zhang XH, Yang XC, Chung N, Gates A, Stec E, Kunapuli P, Holder DJ, Ferrer M, Espeseth AS (2006). "Robust statistical methods for hit selection in RNA interference high-throughput screening experiments". Pharmacogenomics. 7 (3): 299–09. doi:10.2217/14622416.7.3.299. PMID 16610941.
- ^ Brideau C, Gunter G, Pikounis B, Liaw A (2003). "Improved statistical methods for hit selection in high-throughput screening". Journal of Biomolecular Screening. 8 (6): 634–47. doi:10.1177/1087057103258285. PMID 14711389.
- ^ Cohen J (1994). "The Earth Is Round (P-Less-Than.05)". American Psychologist. 49 (12): 997–1003. doi:10.1037/0003-066X.49.12.997. ISSN 0003-066X.
- ^ Zhang XHD (2009). "A method for effectively comparing gene effects in multiple conditions in RNAi and expression-profiling research". Pharmacogenomics. 10 (3): 345–58. doi:10.2217/14622416.10.3.345. PMID 20397965.
- ^ Zhang XHD (2010). "Strictly standardized mean difference, standardized mean difference and classical t-test for the comparison of two groups". Statistics in Biopharmaceutical Research. 2 (2): 292–99. doi:10.1198/sbr.2009.0074. S2CID 119825625.
- ^ a b c Zhang XHD (2010). "Assessing the size of gene or RNAi effects in multifactor high-throughput experiments". Pharmacogenomics. 11 (2): 199–213. doi:10.2217/PGS.09.136. PMID 20136359.
- ^ Zhao WQ, Santini F, Breese R, Ross D, Zhang XD, Stone DJ, Ferrer M, Townsend M, Wolfe AL, Seager MA, Kinney GG, Shughrue PJ, Ray WJ (2010). "Inhibition of calcineurin-mediated endocytosis and alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors prevents amyloid beta oligomer-induced synaptic disruption". Journal of Biological Chemistry. 285 (10): 7619–32. doi:10.1074/jbc.M109.057182. PMC 2844209. PMID 20032460.