|STATISTICS WITHOUT TEARS
|Year : 2008 | Volume
| Issue : 1 | Page : 59-63
Meta-Analysis - Resolving the doctor's dillemma?
H Pandve1, A Banerjee2, S Chaudhury3
1 (Resident), Department of Community Medicine, Padmashree Dr. D.Y.Patil Medical College, Pimpri, Pune-411018, India
2 (Professor), Department of Community Medicine, Padmashree Dr. D.Y.Patil Medical College, Pimpri, Pune -411018, India
3 Professor & Head, Department of Psychiatry, Ranchi Institute of Neuropsychiatry & Allied Sciences, Kanke, Ranchi -834006, India
|Date of Web Publication||13-May-2010|
(Resident), Department of Community Medicine, Padmashree Dr. D.Y.Patil Medical College, Pimpri, Pune-411018
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Clinical trials often yield inconclusive or inconsistent results because of their small sample size leading to the proverbial "Doctor's Dilemma." Against this backdrop, the rapidly evolving statistical tool of meta-analysis has been discussed. The steps of meta-analysis have been enumerated sparing statistical details as far as possible. The advantages as well as the pitfalls of meta-analysis have been reviewed together with some ways of overcoming them.
|How to cite this article:|
Pandve H, Banerjee A, Chaudhury S. Meta-Analysis - Resolving the doctor's dillemma?. Ind Psychiatry J 2008;17:59-63
Meta-analysis is the process of combining the data from a number of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions addressing a particular issue. It aims to utilize the increased power of pooled data to clarify the state of knowledge on that issue (Last, 1995). Meta analysis is often used in systematic reviews of effect studies of medical therapies to evaluate therapeutic effectiveness In simple words meta-analysis is an overview which uses quantitative methods to summarize the results. Meta-analysis in medical research has been best developed with respect to findings of randomized controlled trials (RCTs). Meta-analysis has been extended to observational studies within the fields of etiologic epidemiology and the evaluation of medical interventions. Methods to combine experimental and non-experimental data- for example, the results of RCTs of mammography with findings of case-control studies and databases are currently being developed (Eddy et al, 1998). Despite its widespread use, meta-analysis thus continues to be a controversial technique. While Chamler et al (1992), feel that 'meta-analyses should replace traditional reviews articles of single topic issues whenever possible', others think that it as 'a new bete noire' which represents ' the unacceptable face of statisticism and should 'be stifled at birth' (Oakes, 1986).
This mixed reception is not surprising. The pooling of results from a particular set of studies may be inappropriate from clinical point of view, producing a population 'average' effect, while clinicians want to know how best treat their particular patients. Additional problems arise in meta-analysis of observational studies and in cross-design synthesis. While systematic overviews which include meta-analysis have clear advantage over conventional reviews, meta-analysis is not an infallible tool, and its potential and limitations must be considered carefully.
Modern meta-analysis does more than just combine the effect sizes of a set of studies. It can test if the studies outcomes show more variation than the variation that is expected because of sampling different research participants. If that is the case, study characteristics such as measurement instrument used, population sampled, or aspects of the studies' design are coded. These characteristics are then used as predictor variables to analyze the excess variation in the effect sizes. Some methodological weaknesses in studies can be corrected statistically. Reasons for the popularity of meta-analysis are the growing information in the scientific literature and need of timely decision for risk assesment or in public health.
Evolution of Meta-analysis
The first meta-analysis was performed by Karl Pearson in 1904, in an attempt to overcome the problem of reduced statistical power in studies with small sample sizes; analyzing the results from a group of studies can allow more accurate estimation of effects Meta-analysis is a way to combine the results of several epidemiologic studies. Efforts to pool results from a number of separate studies are not new. The first meta-analysis assessing the effect of a therapeutic intervention was published in 1955. Interestingly the therapy being evaluated was placebo (Beecher, 1955). The development of more sophisticated statistical techniques took place in social sciences, in particular in education research in 1970s. The term meta-analysis was coined in 1976 by psychologist Glass (1976), with meta-analysis in social and psychological sciences generally being applied to combination of results from observational or quasi-experimental studies. Meta-analysis was rediscovered by medical researchers to be used mainly in randomized clinical trial research. In 1982 a meta-analysis of the trials of the use of streptokinase therapy after myocardial infarction was published, which demonstrated a large, statistically highly significant benefit of such therapy, which could not be seen in small individual trials the results of which were being combined (Stamfer et al, 1982). According to Collin and Julian (1991), use of such therapy remained relatively uncommon, however, but very large RCTs have since established its effectiveness. Many deaths which could have been delayed were thus allowed to occur earlier than necessary because of inadequate understanding and implementation of research which had already carried out. In order to minimize future similar occurrences, a network of centres for the systematic meta-analysis of RCTs, named after pioneer in field of evaluation of medical interventions, Archie Cochrane, was created (Chamler et al, 1992). This, it is hoped, will share resources, reduce duplication of effort, and facilitate dissemination of results of meta-analysis.
Need for meta-analysis (Ahrens and Pigeot , 2005)
One of the major issues in assessing causality in epidemiology is "consistency".The goal of meta-analysis is to investigate whether the available evidence is consistent and/or to which degree inconsistent results can be explained.
Meta-analysis are often performed to obtain a combined esimator of quantitative effect of risk factor,such as Relative Risk (RR) OR Odd's ratio.Sometimes, meta-analysis are also used to invstigate more complex dose-response relationships. The growing popularity of meta-analysis will be evident from [Figure 1].
Number of publications about meta-analysis, 1987-96 results from Medline search using text word and medical subject heading "meta-analysis (Smith et al, 1997).
(The number of results on 20 Jan 2007 using subject heading "meta-analysis" is 25047 !!)
Different types of overviews for meta-analysis (Ahrens and Pigeot, 2005)
Approaches for summarising evidences include four different types of overviews
First - Traditional narrative reviews that provide a qualitative reviews but not quantitative assessment of published results.
Second - Meta-analysis from literature which are generally performed from freely available publications without the need of co-operation and without agreement of authors of original studies.
Third - Meta-analysis with individual patient data, in which individual data from published and sometimes also unpublished studies are re-analysed. Often there is a close co-operation between the researcher of meta-analysis and investigators of individual studies.
Fourth - Prospectively planned meta-analysis of several studies in which pooling is already a part of protocol.
Cumulative meta-analysis is defined as the repeated performance of meta-analysis whenever a new trial becomes available for inclusion. Such cumulative meta-analysis can retrospectively identify the point in time when a treatment effect first reached conventional levels of significance.
Steps in performing a meta-analysis (Ahrens and Pigeot, 2005)
Each type of overview needs a clear study protocol that describes the research question and design, including how studies are identified and selected, the statistical methods to use and how the results will be reported.This protocol should also include the exact definition of disease of interest, the risk factors and potential confounding variables that have to be considered.
STEP 1: Define a clear and focused topic for review.
STEP 2: Establish inclusion and exclusion.
STEP 3: Locate all studies ( published and unpublished ) that are relevent to topic.
STEP 4: Abstarct -information from the publication.
STEP 5: Descriptive analysis.
STEP 6: Statistical analysis.
STEP 7: Interpretation of results.
STEP 8: Publication.
| Statistical analysis(Ahrens and Pigeot, 2005)|| |
- Single study results - A first step of stastical analysis is description of characteristics and results of each study. Tabulation and simple graphical methods should be employed to visualize the results of single studies [Figure 2]
- Estimation of summary effect and confidence interval - Frequently,one of the aims of meta-analysis is provide an estimate of overall effect of all studies combined. Two step procedure has to be abstracted from publication or calculated if data is available. Then combined estimate is obtained as a (variance based) weighted average of individual studies.
- Subgroup and Sensitivity Analysis - Subgroup analyses may be possble using data from all or subset of the studied included meta-analysis. Sensitivity analyses indicate how "sensitive" the finding of meta-analysis are to certain decisions about the design of meta-analysis or inclusion of certain studies.
| Problems Occuring in Analysis (Ahrens and Pigeot, 2005)|| |
- Heterogeneity - Combining the results of several studies is not appropriate if studies differ in clinically important ways, such as the intervention, outcome, controls, blinding and so on. It is also inappropriate to combine the findings if the results of individual studies differ widely. Even if methods used in the studies appear to be similar, the fact that results vary markedly suggests that something important was different in the individual studies. This variability in the findings of individual studies is called as "heterogeneity". Meta-regression is used to investigate heterogeneity.
- Publication bias - Publication bias occurs when published are not representative of all studies that have been done, usually because positive reults tend to submitted and published more often than negative results. This publication bias or "file-drawer effect" (where non-significant studies end up in the desk drawer instead of in the public domain) should be seriously considered when interpreting the outcomes of a meta-analysis. A simple graphical tool is used to detect publication bias is called as "funnel plot" [Figure 4].
- Bias - For epidemiological studies in general the main problem is not lack of precision and random error but the fact that results may be distorted by different sources of bias.The assessment of bias in individual studies is therefore crucial for overall interpretation.
- Confounding - Another problem arises because different studies adjust for different confounding factors. It is necessary to use "similar confounders" in each study to adjust the estimated effect of interest in single studies.
| Potentials and Limitations of Meta-Analytic research (Smith et al,1997)|| |
- More objective appraisal of the evidence than in traditional narrative reviews.
- Enhanced precision of pooled estimate, leading to reduced probability of false-negative results.
- Resolution of uncertainty when original research, reviews, and editorials disagree.
- Generation of promising research questions to be addressed in future studies.
- Accurate calculation of sample sizes needed in future studies.
- Results of meta-analysis, albeit precise, may be misleading. Bias is difficult to exclude when combing observational studies.
- Selection bias: inclusion or exchange results. For example inclusion of published studies only may give misleading results (publication bias).
- Equal weight is given to studies of high quality and to more doubtful studies.
- Statistical procedures alone cannot resolve the issue of heterogeneity between study results.
- Extensive data-dredging results in subgroup findings which are likely to be spurious even though they may achieve statistical significance.
- Important further research may be prevented if mea-analyses are falsely perceived as providing definite answers.
- The clinical interpretation of meta-analyses is often problematic.
| Conclusion|| |
Despite the many problems, there is an immense need to summarise current knowledge, meta-analysis is becoming increasingly useful particularly where the previously epidemiological studies have provided inconsistent results a meta-analysis may give some insight. As discussed, a major impediment for meta-analysis of epidemiological data is heterogeneity across studies in their design, data collection methods analysis performed. The statistical combination of risk estimates should not be central component of meta-analysis using published data.There is always danger that meta-analysis of observational studies produces precise looking estmates which are severely biased. This should be kept in mind as more and more public health regulators and decision makers may rely on the results of meta-analysis. Meta-analysis should be seen as structuring the processes through which a thorough review of previous research is carried out, as it is clearly superior to narrative approach to reviewing medical research. Some of the shortcomings of meta-analysis are, however, a consequence of more general failing with respect to the dissemination of research findings. Registers of clinical trials may provide solution to this problem.
| References|| |
|1.||Ahrens W, Pigeot I (2005). Meta-analysis in epidemiology. Blettner M, Schlattamann P.(Ed).Handbook of Epidemilogy. Springer Publication. |
|2.|| Beecher H.K. (1955). The powerful placebo. Journal of American Medical Association, 159,1602-1606. |
|3.|| Chalmer I, Dickersin K, Chalmer T.C. (1992) Getting to grips with archie cochrane agenda. British Medical Journal, 305,786-788. |
|4.|| Collin R, Julian D (1991). British heart foundation survey (1987 and 1989) of United Kingdom treatment policies for acute myocardial infarction. British Heart Journal, 66, 250-255. |
|5.|| Eddy D.M., Hasselblad V, McGiveney W, Hendee W (1988). The value of mammographic screening in women under age 50 years. Journal of American Medical Association, 259,1512-1519. |
|6.|| Glass G.V. Primary, secondary and meta-analysis of research (1976). Education Research, 5,3-8. |
|7.|| Last J.M. (1995).A Dictoonary of Epidemilogy. Third edition.Oxford University Press. |
|8.|| Oakes M (1986). Statistical inference : a commmentary for the social and behavioural sciences.John Willey & Sons, Chichester. |
|9.|| Smith G.D, Egger M, Phillips A.N. (1997). Meta-analysis and data symthesis in medical research. Detels R, Walter W.(Ed).Oxford Textbook of Public Health. Oxford Medical Publication. |
|10.|| Stamfer M.J, Goldhaber S.Z, Yusuf S, Peta R, Henekens C.H (1982). Effect of intravenous sterptokinase an acute myocardial infarction. Pooled results from randomised trials. New England Journal of Medicine,307,1180-1182. |
[Figure 1], [Figure 2], [Figure 4]