REVIEW ARTICLE

 

Impact factor – a brief update

 

Fator de impacto – uma breve revisão

 

 

Maya Fernanda Manfrin Arnez^I; Elaine Machado Pingueiro^I; Francisco Wanderley Garcia de Paula e Silva^II; Adriana Sasso Stuani^III; Andrea Sasso Stuani^IV; Maria Bernadete Sasso Stuani^V

^IPostgraduate Student in Pediatric Dentistry, Department of Clínica Infantil, Odontologia Preventiva e Social, Faculdade de Odontologia de Ribeirão Preto, University of São Paulo (USP) – Ribeirão Preto/SP, Brazil.
^IIMSc, PhD in Pediatric Dentistry, Department of Clínica Infantil, Odontologia Preventiva e Social, Faculdade de Odontologia de Ribeirão Preto, University of São Paulo.
^IIIPostgraduate Student in Farmacology, Departament of Análises Clínicas e Toxicológicas, da Faculdade de Ciências Farmacêuticas da Universidade de São Paulo (USP) – Ribeirão Preto/SP, Brazil.
^IVPhD in Orthodontics, Departament of Ortodontia da Faculdade de Odontologia do Rio de Janeiro da Universidade Estadual do Rio de Janeiro (UERJ) – Rio de Janeiro/RJ, Brazil.
^VProfessor of Orthodontics, Departament of Clínica Infantil, Odontologia Preventiva e Social da Faculdade de Odontologia de Ribeirão Preto da Universidade de São Paulo (USP) – Ribeirão Preto/SP, Brazil.

Corresponding address

 

 

---

 

ABSTRACT

Scientific journals represent the main disclosure tools for the advancement and development of Science. The importance of scientific journals can be usually defined in several ways including its prestige, which may be reflected in its citedness as measured by the impact factor (IF), which is a classic parameter useful to researchers. Due to controversial issues involving scientific publication, the purpose of this update article was to clarify and explain what is the impact factor of a journal and how it can guide faculty advisors and postgraduate students to publish their papers. The IF of a journal describes both journal and author impacts. It is based on two elements: the numerator, which is the number of citations in the current year to any items published in a journal in the previous two years, and the denominator, which is the number of substantive articles (source items) published in the same two years. Although IF is the only measure of journal’s quality, it is important to remember that the calculation of the IF is biased by many factors. These include factors as inclusion of review articles, commentaries, errata and letters in numerator, but not in the denominator of the equation used to calculate the IF. Impact factor is the foremost tool to measure and indirectly indicate the research quality. Although it is not a perfect method and has several limited interpretation power, it remains a valuable measurement technique for scientific evaluation.

Descriptors: Impact factor. Journal impact factor. Database. Citation databases.

---

 

RESUMO

As revistas científicas são as ferramentas de divulgação mais importantes para o avanço e o desenvolvimento da Ciência. A importância destas revistas pode ser definida de diversas maneiras incluindo o seu prestígio, o que reflete a sua citabilidade, mensurada através do fator de impacto (IF), um parâmetro clássico e útil para os pesquisadores. Devido à controvérsia envolvendo este assunto, a proposta deste artigo de atualização é esclarecer e explicar o que é fator de impacto de uma revista científica e como o IF pode guiar os orientadores e pós-graduandos na publicação de seus trabalhos científicos. O fator de impacto de uma revista é usado tanto para descrever o impacto da revista quanto para descrever o impacto do autor. É baseado em dois elementos: o numerador, número de citações no atual ano para qualquer item publicado na revista científica nos dois anos anteriores e o denominador, número de artigos científicos reais (item de origem) publicados nos mesmos dois anos. Embora o IF seja a única medida disponível da qualidade da revista científica, é importante lembrar que o cálculo do IF é enviesado por muitos fatores, como a inclusão no numerador de artigos de revisão, comentários, erratas e cartas, ao passo que o denominador da equação não inclui estes fatores. O IF é a principal ferramenta para medir e indiretamente indicar a qualidade da pesquisa. Embora não seja um método perfeito e haja algumas limitações no poder de interpretação, o IF permanece como valiosa técnica de medida da avaliação científica.

Descritores: Fator de impacto. Fator de impacto de revistas. Base de dados. Bases de dados de citações.

---

 

 

INTRODUCTION

A worldwide biomedical scientific community attempts to publish research manuscripts in different journals. Researchers and institutions have several journals to choose when considering where to find information and where to publish their work³.

It is well known that what must be taken into account is the quality of papers instead of the quantity of papers to be published. In Brazil, an emerging country in scientific research, there is a request for scientific publication in the academic field. Currently, biomedical researchers, mainly in the state of Sao Paulo, have received financial funding from private and public organizations and are interested in publishing their research in journals with high impact factor (IF), which is a classic parameter useful to librarians, editors, publishers and researchers².

Scientific journals represent the main tools for the advancement and development of Science. The importance of scientific journals can be usually defined inseveral ways including its prestige, which may be reflected in its citedness as measured by the IF².

IF means the average number of times articles from a journal published in the past two years have been cited in the current year for a specific journal, i.e.it is a simple descriptive quantitative measurement of a journal';s performance³. On the other hand it is well known that there are alternative metric methods, which can be used as evaluation criteria of journals, even though they do not completely satisfy all parameters of evaluation and there are several limitations to measure the quality of the scientific production¹³.

Due to controversial issues involving scientific publication, the purpose of this update article was to clarify and explain what is the impact factor of a journal and how it can guide faculty advisors and postgraduate students to publish their papers.

 

Background

In the early 1950s, Irving H. Sher and Eugene Garfield, founder of the Institute for Scientific Information (ISI), developed a library system for scientific literature and created the Journal Impact Factor (JIF), aiming to have a simple method for comparing journals, regardless of their size, to help select journals for the Science Citation Index (SCI)⁴ . ISI';s database was initially developed for cross reference literature searches and identification of individual scientists working on particular topics¹.

The IF was originally conceived as a quantitative assessment of refereed publications in a given scientific journal. By processing the data from the citation index, it became possible to calculate a ratio of a journal citation. Eugene Garfield explained the meaning of impact, pointing out that a citation indicates that an article has influenced someone and therefore, the more often an article is cited, the greater its influence on the scientific community⁵. This ratio was then used to select the journals for inclusion in the SCI6. The JIF is currently calculated by Thomson Scientific Database (formerly Thomson ISI) for all journals indexed in the SCI database, then reported in the Journal Citation Reports (JCR) database.

The IF of a journal has gradually evolved to depict both journal and author impact⁸. It is based on two elements: the numerator, which is the number of citations in the current year to any items published in a journal in the previous two years, and the denominator, which is the number of substantive articles (source items) published in the same two years^2,6,7. By ISI definition, only research articles, technical notes and reviews are "citable" items. Editorials, letters, news items, and meeting abstracts are "non-citable items" for the purpose of calculating the denominator. However, all items may be counted in the numerator during the calculation³

The IF can easily be based on the previous year';s articles alone, which would give even greater weight to rapidly changing fields8.The key determinants in impact are not the number of authors or articles in the field but, rather, the mean number of citations per article (density) and the half-life or immediacy of citations to a given journal. All citation studies should be normalized to take into account variables such as field, or discipline, and citation practices. Citation density (mean number of references cited per article) and half-life (number of years, going back from the current year, that cover 50% of the citations in the current year to the journal) are also important variables, as well as the time required to review manuscripts, even the appearance of articles on the same subject in the same issue of a journal may have an effect and may also affect impact⁶.

IF currently reported by ISI in JCR provides quantitative tools for evaluating, comparing, categorizing, and ranking journals. The IF is a measure of the frequency with which the "average article" in a journal has been cited in a particular year or period2. The highest ranking dental journals, according to 2009 IF scores, are described on Table 1.

The JIF are also used by authors to decide where to submit their articles. As a general rule, journals with high IF are among the most prestigious^2,6,9. The perception of prestige is a murky subject and researchers should equate prestige with high impact⁶.

 

DISCUSSION

Although IF is the only measure of journal';s quality, it is important to remember that the calculation of the IF is biased by many factors. These include factors as inclusion of review articles, commentaries, errata and letters in numerator, but not in the denominator of the equation used to calculate the IF. This upwards the IF of some medical journals, although that has been considered a small distortion⁸.

More strikingly against IF, there is a highly skewed, non-Gaussian distribution of citations for the articles and the lack of consideration of size of the different fields of science represented in different journals⁹,the coverage and language preference of the SCI database; procedures used to collect citations at the ISI; algorithm used to calculate the IF; citation distribution of journals; online availability of publications; citations to invalid articles; negative citations; preference of journal publishers for articles of a certain type; publication lag; citing behavior across subjects; possibility of exertion of influence from journal editors³.

 

 

 

The JCR recently added a new feature that provides the ability to establish more precisely journal categories based on citation relatedness. A general formula based on the citation relatedness between two journals is used to express how close they are in subject matter⁸.

Unfortunately, the IF cannot assess the quality of individual articles, due to the qualitative variety of citations distributed in a journal^10,11. A small proportion of articles count for a large percentage of citations. As a result, IF alone is not able to judge the individual article';s or author';s performance. IF measure only the interests of other researchers in an article, but not the article';s importance and usefulness³.

Indeed the citation rankings may be influenced by the size of a journal, i.e., journals that publish a very large number of articles might have a greater chance of being cited more frequently than comparatively smaller journals². However, size of a field will increase the number of "super-cited" papers and many articles in large fields are not well cited, whereas those in small fields may have unusual impact. Therefore, the key determinants of IF are not the number of authors or articles in the field but, rather, the citation density and the age of the literature cited⁸.

It is probable that reviews are more likely to be cited than typical original research articles, since they could replace the earlier literature, as well as methods articles, that seem to attract more citations than other types of articles¹¹. However this cannot be in fact true. Several papers devoted entirely to methods, do not achieve unusual impact. Sometimes a "hot paper", i.e. highly cited paper, might affect the current impact of a journal over the years² and most of these papers will eventually qualify as "citation classics"^2,6.

Different specialties also exhibit different ranges of peak impact. IF will help us to evaluate a journal's relative importance, mainly when compare it to others in the same field².The wide use of the IF and its inherent failures, has encouraged several researchers in scientometrics to try to improve the journal citation methodological aspects (algorithm for the calculation) of the IF or to develop alternative journal citation measures to overcome the subjects bias, in order to achieve a reliable standard measure of ranking list of journals'; quality as a superior assessment tool of scientific research database. It can help to have more accurate information about a paper quality if some alternative suitable measures are used associated with the IF of a journal³.

Many of the discrepancies inherent in JIF are completely eliminated in another Thomson Scientific database called ISI Journal Performance Indicators (JPI). JPI database links each source item to its own unique citations. Therefore, the impact calculations are more precise. Only citations to the substantive items that are in the denominator are included. And it is possible to obtain cumulative impact measures covering longer time spans. Moreover, to helping libraries decide which journals to purchase, JIF are also used by authors to decide where to submit their articles. As a general rule, the journals with high IF include the most prestigious. Some would equate prestige with high impact⁸.

Scientific groups need to keep in mind that IF has a couple of limitations. The easiest way to assess the relative position of a particular journal within its field is to browse through the SCI';s Subject Category and sort all journals by their IF in a particular category. The IF alone cannot provide the knowledge needed for informed decision-making³.

It is important to note that the currently available methods for the evaluation of the quality of scientific papers and the status of the journals that publish these papers are themselves undergoing a period of profound re-evaluation. No metric of scholarly impact represents a final perfect solution about an article or a journal status⁹.

In Brazil, a metric tool named Qualis was created by CAPES (Co-ordination for the Improvement of Higher Education Personnel) to measure and evaluate the intellectual production of postgraduate programs (Masters and PhD). The quality of this production is carried out indirectly, since the measures of the quality of articles and other types of production arise from the indexing scientific journals. These journals are ranked in quality indexes – A1 (the highest level of classification), A2, B1, B2, B3, B4, B5 and C (with zero weight)¹². According to Silva¹³ the correct evaluation of scientific production of faculty advisors and postgraduate students should be based on the number of citations of their publication, since Qualis includes review journals, i.e. not a original scientific articles, in addition to underrating and underestimate the quality of Brazilian journals, whereas English language journals are overrating^12,13.

Therefore it is necessary a method to evaluate and reflect the correct status of a journal that provides a more balanced approach for assessing research contributions without subjective issues. The journal in which are search is published is currently being used as a key indicator of the scientific quality of the research itself and it indicates the impact these publications have on research in their respective field of Biomedical Science⁹. IF is an indirect measure of these qualities and has a place in decision making process⁷ but should not be paramount.

 

CONCLUSION

Impact Factor is the foremost tool to measure and indirectly indicate the research quality by measuring the average citation rate of all the "citable" articles (research articles, technical notes and reviews) in a journal. Although it is not a perfect method and has some limited interpretation power; it remains available measurement technique for scientific evaluation.

 

REFERENCES

1. Adam D. The counting house. Nature 2002;415(6853):726-9.         [ Links ]

2. Ciger S. Impact factor, citation index and circulation. Hellenic Orthodontic Review 2003;6 Suppl 2:31-5.         [ Links ]

3. Dong P, Loh M, Mondry A. The "impact factor" revisited. Biomed Digit Librar 2005;2:7.         [ Links ]

4. Garfield E. Citation indexes for science; a new dimension indocumentation through association of ideas. Science 1955;122(3159):108-11.         [ Links ]

5. Garfield E.Which medical journals have the greatest impact? Ann Intern Med 1986;105(2):313-20.         [ Links ]

6. Garfield E. Journal impact factor: a brief review. CMAJ 1999;161(8):979-80.         [ Links ]

7. Garfield E. The meaning of the impact factor. Int J Clin Health Psychol 2003;3(2):363-9.         [ Links ]

8. Garfield E. The history and meaning of the journal impact factor. JAMA 2006;295(1):90-3.         [ Links ]

9. Hascall VC, Bollen J, Hanson RW. Impact factor page rankled. ASBMB Today 2007;July:16-9.         [ Links ]

10. Opthof T, Coronel R, Piper HM. Impact factors: no totum pro parte by skewness of citation. Cardiovasc Res 2004;61(2):201-3.         [ Links ]

11. Seglen PO.Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314(7079):498-502.         [ Links ]

12. Silva MR. O novo Qualis e o futuro dos periódicos científicos brasileiros. Scientia Medica 2009;19(3):96-7.         [ Links ]

13. Silva MR. Qualis 2011-2013 – os três erres. Clinics (Sao Paulo) 2010;65(10):935-6.

 

 

Corresponding address:
Maria Bernadete Sasso Stuani
Departamento de Clínica Infantil, Odontologia Preventiva e Social
Faculdade de Odontologia de Ribeirão Preto da Universidade de São Paulo
Av. do Café s/n – Monte Alegre
CEP 14040-904 – Ribeirão Preto/SP, Brazil
Fone: (16) 3301-6393
e-mail: bernadete@forp.usp.br

Received in: 18/2/11
Accepted in:30/3/11