The Roles of Multiple Proteomic Platforms in a Pipeline for New Diagnostics

Numerous recent reports (e.g. Refs. 1‐3) suggest that proteome studies are in the process of finding a range of novel disease biomarkers in plasma and serum, giving rise to the hope that the declining trend in new protein diagnostics over the last decade (4) will be reversed and the obvious benefits of early (5) and correct diagnosis will be extended to many diseases. The path to this bright future is, however, proving to be difficult for reasons both technical and scientific, highlighting the need for a well conceived pipeline process for diagnostic development. Here I review some of the important factors constraining successful protein diagnostic development and propose a three-stage technology roadmap for accelerating the introduction of new clinical tests. At the outset, it is useful to compare the process of developing new diagnostics (Dx) 1 with the relatively well understood pipeline process for developing new drugs (Rx). Drug development usually begins with molecular target identification and progresses through the discovery and gradual winnowing of lead compounds (through studies of preclinical toxicology, tissue disposition, metabolism, pharmacokinetics, etc.) to produce high value drug candidates that go into a series of increasingly large and expensive clinical trials. In the pharmaceutical industry, all of these steps are integrated under a single business management motivated to increase the output of approved drugs by optimizing the allocation of resources to each pipeline stage. In diagnostics, such an integrated pipeline is a rarity if it exists at all: in general, marker candidates are discovered in the course of academic, government-funded research, and only the final stages of clinical validation are carried out by the manufacturers of commercial in vitro diagnostics (IVD). The middle section of the Dx pipeline, where winnowing of candidates occurs to yield the highest value tests, is not explicitly funded by either government or industry and for this reason is not carried out systematically. This gap in the middle of the Dx pipeline largely accounts for the shortfall of new markers reaching clinical use. Lack of unified responsibility for the Dx development or even a well understood cooperative “virtual” Dx pipeline process inhibits solution of the problem. The ability of proteomics to survey large numbers of proteins at once ought to substantially increase the flow of new candidate diagnostics into the Dx pipeline, and in the view of some, proteomics should be able to carry these candidates all the way to routine use. A number of factors blunt the value of proteomics in this respect of which three can be singled out as producing particularly troublesome consequences.


N. Leigh Anderson ‡
Numerous recent reports (e.g. Refs. [1][2][3] suggest that proteome studies are in the process of finding a range of novel disease biomarkers in plasma and serum, giving rise to the hope that the declining trend in new protein diagnostics over the last decade (4) will be reversed and the obvious benefits of early (5) and correct diagnosis will be extended to many diseases. The path to this bright future is, however, proving to be difficult for reasons both technical and scientific, highlighting the need for a well conceived pipeline process for diagnostic development. Here I review some of the important factors constraining successful protein diagnostic development and propose a three-stage technology roadmap for accelerating the introduction of new clinical tests.
At the outset, it is useful to compare the process of developing new diagnostics (Dx) 1 with the relatively well understood pipeline process for developing new drugs (Rx). Drug development usually begins with molecular target identification and progresses through the discovery and gradual winnowing of lead compounds (through studies of preclinical toxicology, tissue disposition, metabolism, pharmacokinetics, etc.) to produce high value drug candidates that go into a series of increasingly large and expensive clinical trials. In the pharmaceutical industry, all of these steps are integrated under a single business management motivated to increase the output of approved drugs by optimizing the allocation of resources to each pipeline stage. In diagnostics, such an integrated pipeline is a rarity if it exists at all: in general, marker candidates are discovered in the course of academic, government-funded research, and only the final stages of clinical validation are carried out by the manufacturers of commercial in vitro diagnostics (IVD). The middle section of the Dx pipeline, where winnowing of candidates occurs to yield the highest value tests, is not explicitly funded by either government or industry and for this reason is not carried out systematically. This gap in the middle of the Dx pipeline largely accounts for the shortfall of new markers reaching clinical use. Lack of unified responsibility for the Dx development or even a well understood cooperative "virtual" Dx pipeline process inhibits solution of the problem.
The ability of proteomics to survey large numbers of proteins at once ought to substantially increase the flow of new candidate diagnostics into the Dx pipeline, and in the view of some, proteomics should be able to carry these candidates all the way to routine use. A number of factors blunt the value of proteomics in this respect of which three can be singled out as producing particularly troublesome consequences.
The first factor is the notion that a simple, quick, and cheap process that produces information-rich patterns can leapfrog over all the tedious effort of characterizing specific molecules. This sounds too good to be true, and, in general, it is; however, the reason is not that meaningful patterns do not exist but rather that the process of achieving such a "simplification without characterization" is inherently difficult to standardize sufficiently for routine use. A scientific shortcut to a reproducible result is admirable, but a technological shortcut leaving behind unexposed variables is apt to come back to bite us (as it has in the vigorous and ongoing debate over SELDI-type peptide profiles of disease (6 -9)). It frequently takes a decade or more to develop control over such a process at the level required in IVD products (10). Perhaps more important, the pattern shortcut seems to be coupled with great technical difficulty in definitively identifying the molecular features making up the pattern. Thus very few of the components of published SELDI patterns have so far been identified. The resulting dependence on so-called "unidentified flying peptides" adds a further barrier to control of system reproducibility: how can an unidentified peak be proven to be the same analyte on another day, in another country, or after even a small modification of the instrument platform?
The second factor is the extreme complexity and dynamic range of the plasma proteome (4) and the resulting requirement for multidimensional fractionation to reveal even modest numbers of protein candidates. Subdivision of the plasma proteome into many fractions allows hundreds and possibly thousands of proteins to be detected but at the cost of many analyses by complex instruments at the final stage. Published studies have examined tens (11), hundreds, and even thousands (12) of plasma fractions. Given the cost and labor intensiveness of such methods, they are clearly not applicable to routine clinical tests and are of marginal use in confirming candidate biomarkers where thousands of individual samples must be analyzed separately to satisfy statistical criteria for diagnostic specificity and sensitivity. In many cases, multidimensional fractionation/detection is most highly developed in discovery-oriented laboratories with limited interest in high throughput routine operation.
The third factor is the very limited overlap between protein sets revealed by the many different proteomic discovery platforms. Initial (13) and subsequent (14) comparisons of different platforms for plasma proteomics indicate that each approach sees a somewhat different set of proteins and hence is capable of discovering a different subset of candidate biomarkers. The unfortunate consequence of this result is that there is as yet no clearly preferred platform for discovery of plasma biomarkers and certainly no comprehensive platform. This means that focusing our efforts on one or a few platforms is unlikely to uncover all, or even the best, biomarkers of a given disease. It also means that some of the candidates revealed by one discovery platform may not be detected in another and thus that a "mix and match" approach, leveraging candidates across platforms, will be difficult using discovery technologies.
For these and other reasons, discovery-oriented proteomics does not appear to offer a complete solution for plasma diagnostics. Some valuable insights have nevertheless been gained in the course of encountering these barriers. Despite the problems of reproducing patterns, the long anticipated benefits of multivariate diagnostic panels, as compared with individual markers, seem increasingly general and powerful. The limitation of current biomarker discovery platforms has highlighted the need for development of another class of "validation" platforms optimized for high throughput measurement of preselected candidate proteins, making use of the general superiority in sensitivity and precision of specific assays over global analyses (15). The heterogeneity of proteome coverage achieved by discovery platforms has emphasized the value of combining candidate biomarkers arising from many sources, including those identified using microarrays and tissue proteomics or predicted by pathway analyses and so-called "systems biology" (16).
Looking at the current state of proteomic technology and the daunting nature of the plasma proteome, can we envision a roadmap for the creation of a viable diagnostic biomarker pipeline? I would argue that the required components are in hand, or nearly so, and that the outlines of a productive, systematic process have begun to emerge. Key to this process is the realization that translation of biomarkers to clinical use requires a three-stage pipeline (shown in Fig. 1) comprising 1) discovery, 2) verification/validation, and 3) clinical implementation in which each stage currently requires a different suite of analytical technologies. Progress through such a pipeline involves a handoff of biomarker candidates between stages and the different groups that operate them, and thus some form of network organization.
The first pipeline stage can be defined to include any investigation that yields one or more candidate protein (or peptide) biomarkers identified by sequence and relevant posttranslational modifications. By accepting a requirement for sequence-based molecular identification, the discovery stage can be freed from any other platform constraint: any method capable of yielding believable candidates can be valid so long as the candidates can be categorically identified. Under this paradigm, a discovery method does not necessarily need to have high throughput, does not need to cover the whole proteome, and does not need to be frozen technologically to provide long term stability; in essence it allows us to yield gracefully to the truth that there is no ideal, comprehensive discovery platform for plasma. One can easily imagine the output of multiple streams of such discovery work coalescing into systematic databases of biomarker candidates (15), assembled from multiple sources using bioinformatics, that can be intelligently prioritized in preparation for subsequent verification and validation.
The second stage begins with identified candidate molecular entities arising in the first stage and proceeds to measure these in the large sample sets required to determine the key parameters for a diagnostic test: normal biovariability, sensitivity and specificity in relation to target diseases, and statistical contribution in the context of various biomarker panels. The primary objectives are to verify the biological significance of candidate biomarkers and to validate prototype assays for them. More than 1,000 separate samples are usually required, good measurement precision is important (coefficients of variation Ͻ10%), and sensitivities should allow detection of both normal and diseased levels (i.e. a range from pg/ml to mg/ml lower limits of quantitation). For a variety of reasons, collections of specific assays carried out in a multiplexed format (10 -100 candidates at once) by quantitative mass spectrometry (15) or on antibody arrays (17) are likely to represent the most practical approach. Given the expected importance of biomarker panels, it will be highly advantageous if the candidates can all be measured in a coherent technology platform providing sufficient uniformity across samples, laboratories, and time to allow construction and testing of virtual marker panels from the accreting data. Thus, in contrast to the discovery stage, a focus on one or at most a few verification/ validation platforms is desirable. The resulting data should ideally be open for ongoing statistical analysis and comparison by the research community under arrangements that emphasize the benefits of collaborative analysis and avoid the inhibiting effects of obstructive intellectual property.
The third stage of the Dx pipeline is focused on commercial implementation of clinical tests and depends on the results of the second stage evaluated in the context of a variety of additional factors including disease prevalence, availability of treatments, cost reimbursement policies, and compatibility with existing clinical laboratory instrument platforms. Decisions on biomarker viability in the final pipeline stage are thus outside the control of the proteomics community; even so, an acute awareness of the logic of success or failure in clinical diagnostics (18,19) is vitally important in guiding the efforts of clinical proteomics. For example, a biomarker panel consisting of up to five different proteins might be implemented effectively on existing hospital immunoassay instruments (as five separate assays costing a total of $10 -20 million to develop to the level of Food and Drug Administration approval), but larger panels become progressively less economical. Small panels for a given disease are thus much more valuable in the near term than larger panels of similar diagnostic power. Implementation on existing instrument platforms is critical if a new test is to be made available in less than a decade because new platforms generally require 5-10 years to develop to IVD reliability standards and a further decade to achieve widespread adoption in hospitals where most testing is done.
The requirement for different, and progressively more focused, analytical platforms in the three pipeline stages does imply that a central and attractive feature of discovery proteomics (looking at a substantial fraction of a proteome at once) will not be translated directly into clinical diagnostics in the near term. To succeed in impacting Dx in the next 5-10 years, proteomics must instead adapt itself to, and consciously exploit, the reality of current IVD (e.g. aiming to winnow large sets of candidates down to small panels). In doing so, proteomics should still be able to pursue the overarching goal of accumulating evidence in favor of the "biomarker hypothesis": that protein diagnostics (including panels) exist for most if not all human disease states. A consensus in favor of this hypothesis would in turn provide the motivation for much larger commitments to human proteome exploration and proteomic technology development.
Much remains to be done before a functioning diagnostic pipeline can be said to exist. In particular the currently separate constituencies carrying out clinical studies, proteomic technology development, biomarker validation, and IVD manufacturing must somehow be brought together for integrated action. Although the first and third stages of the pipeline exist and are reasonably funded (the first primarily through the National Institutes of Health (in the United States) and the third by the IVD industry), the middle stage (verification and validation) is not appropriately staffed or funded. Efforts to overcome this deficiency have begun at the National Institutes of Health (20,21) where the value of leveraging existing clinical and epidemiology studies with new biomarkers is increasingly appreciated, but these resources are still very small in comparison to the requirement. Some industrial support may be forthcoming as a consequence of the growing requirement for surrogate markers in drug trials, but severe pressure on costs of clinical tests continues to inhibit IVD industry commitments in this area.
Commitment to a functioning diagnostic pipeline, coupled with a vigorous research effort to explore the biomarker hypothesis, should be a central element of national health policy. Indeed if the biomarker hypothesis is generally true and future advances in proteomic technology (arrays, mass spectrometry, etc.) eventually make it possible to perform comprehensive early diagnosis using large panels of protein biomarkers, proteomics may provide the only practical solution to the emerging crisis in healthcare cost. ‡ To whom correspondence should be addressed: