PRESIDENTIAL ADDRESS

 

Presidential Address: Truth and error in scientific publishing

 

 

R.T. Jones

Mintek, Randburg

 

 

---

ABSTRACT

Scientific progress relies on the publication of ideas and experimental results that can be replicated, tested, and improved over time. The first printed book on metallurgy to have been published in Europe is considered to be De la Pirotechnia, written in Italian by Vannoccio Biringuccio, and published in Venice in 1540. Together with De Re Metallica, written by Georgius Agricola and published in Latin in 1556, this can be considered to mark the start of scientific and technical literature in this field. Scientific publishing of journal papers has been in existence for 350 years, since the world's oldest and longest-running scientific journal, the Philosophical Transactions of the Royal Society, was first published in London in 1665. The nature of scientific societies has changed significantly since the early days when regular meetings were held to discuss science and conduct experiments, and the reading of scientific papers took place, and publication of papers was undertaken to record the proceedings of meetings, often including rather robust debate. In today's world, there is a plethora of publications, and it is close to impossible for anyone to keep up with the vast flow of information. International conferences with hundreds of presentations have taken the place of the local meetings that used to discuss a single paper or experiment. In this frenetic environment, it is essential that researchers are able to trust the material they read. The system of peer review is used to maintain standards and to improve the quality of papers. This vital system is, however, significantly flawed. There is little incentive for reviewers to invest sufficient time in picking up all errors in publications, and any ineptitude on their part is usually protected by anonymity. It has reached the point where some reviewers have mistakenly permitted the publication of hoax papers deliberately presented with a complicated scientific facade. In light of such astounding inadequacies, perhaps a more open review process would be an improvement. Electronic publishing allows errata to be linked to the original papers. This might improve the current situation, where errors tend to be propagated from one paper to the next. There is an increasing trend towards open access for papers in scientific journals and conference proceedings, which helps to reach as wide an audience as possible. This also supports the statement in the Universal Declaration of Human Rights, which says 'everyone has the right freely to ... share in scientific advancement and its benefits'. Various measures (including the impact factor) have been used to rate the performance of journals, while a count of citations (or the h-index) is often used to rate the performance of scientific authors. Some flaws in this approach have been highlighted. Scientific publishing remains alive and well, despite some problems and challenges. Electronic technology provides some wonderful opportunities to improve the way we communicate scientific results.

---

 

 

Introduction

A quick inspection of today's news media shows that there are numerous scientific issues facing us at the moment. Some current controversies such as climate change make it difficult to know just where to find the truth. There is certainly much evidence that shows that the global climate is changing. However, it is also questioned by some, perhaps on shaky grounds, whether that change is in the direction of warming the globe, and whether man has played a significant part in climate change. How do we go about establishing the truth of the various claims that are made, and how do we interpret where they are coming from?

Another area that affects all of us is the set of dietary guidelines that we have been given regarding how to balance the amounts of protein, fats, and carbohydrates that we eat. Certainly, some of the conventional wisdom has been based on appalling science. But, there are many claims and counter-claims doing the rounds at the moment. Where does the truth lie, and how do you judge how to live your life and feed yourself?

These topics rely on scientific evidence to get closer to the truth. Scientific progress relies on the publication of ideas that can be improved over time, and experimental results that can be replicated and tested.

Many people have spoken over the years on the subject of truth and knowledge, which has to be the beginning of a discussion such as this. For example:

The wisest of men is he who knows that he knows nothing - Socrates All men by nature desire knowledge - Aristotle

If you would be a real seeker after truth, it is necessary that at least once in your life you doubt, as far as possible, all things - René Descartes All truths are easy to understand once they are discovered; the point is to discover them - Galileo Galilei In questions of science, the authority of a thousand is not worth the humble reasoning of a single individual -Galileo Galilei

All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident - Arthur Schopenhauer

This whole notion of truth being something out there to be discovered is something that Isaac Newton spoke about as an 'undiscovered ocean of truth'. He said: 'I do not know what I may appear to the world, but to myself I seem to have been only like a boy playing on the seashore, and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me.' (Brewster, 1855)

 

Epistemology (the study of knowledge)

The foundations of the study of knowledge are to be found in the branch of philosophy known as epistemology. Knowledge was seen by Aristotle as 'justified true belief', that is, belief that is true, and that is known to be true on the basis of compelling reasons and evidence supplied by a rational method of enquiry. It is necessary for knowledge to be arrived at by a process of reasoning, and not merely a lucky guess. Knowledge of the truth can be obtained either by a priori reasoning (such as in mathematics or logic), or by empirical experience (such as in science or engineering). Albert Einstein spoke of epistemology saying:

Science without epistemology is - insofar as it is thinkable at all - primitive and muddled. However, no sooner has the epistemologist, who is seeking a clear system, fought his way through such a system, than he is inclined to interpret the thought-content of science in the sense of his system and to reject whatever does not fit into his system. The scientist, however, cannot afford to carry his striving for epistemological systematic that far.... He therefore must appear to the systematic epistemologist as an unscrupulous opportunist.' (Einstein, 1949)

In the discussion that follows, I will try to avoid too much 'unscrupulous opportunism' as we focus more on scientific matters.

 

 

 

What is truth?

There are many different views of truth. At the one extreme, there is Solipsism - the denial of reality - where life is perhaps seen as an illusion. While this is a possibility that has to be entertained, solipsism is not a view that is held widely. One notch below that is Radical scepticism, where it is claimed that our only direct knowledge is of our senses, and anything else is known only indirectly. Relativism maintains that everything depends on one's point of view. This view goes back many centuries, as Marcus Aurelius said: 'Everything we hear is an opinion, not a fact. Everything we see is a perspective, not the truth.' A more pragmatic view is held in Instrumentalism, where it is said that quantities can be measured, even if we have no way of knowing whether theoretical entities actually exist. Thermodynamics employs this approach quite effectively. Fallabilism says that one can know things, even though we are sometimes wrong. Empiricism is based on observation or experience. This is getting much closer to the central approach of science. Rationalism maintains that truth is based on reason. At the other end of the spectrum is Dogmatism, whose adherents are quite certain of their truth, although this implies a degree of closed-mindedness. As US Justice Oliver Wendell Holmes, Jr said: 'Certitude is not the test of certainty'.

 

Attitude towards knowledge

Dogmatism is problematic, at least in part because it often occurs in close conjunction with ignorance. The Dunning-Kruger effect (Kruger and Dunning, 1999) is a cognitive bias where unskilled individuals overestimate their abilities (due to the meta-cognitive inability of the unskilled to evaluate their own ability levels accurately), and conversely highly skilled individuals underestimate their competence (because they assume that tasks that are easy for them are also easy for others). Their study was inspired by the case of McArthur Wheeler, a man who robbed two banks after covering his face with lemon juice, mistakenly believing that, because lemon juice is usable as invisible ink, it would prevent his face from being recorded on surveillance cameras. He was arrested the same night. As Charles Darwin said: 'ignorance more frequently begets confidence than does knowledge' (Darwin, 1871). William Shakespeare said much the same thing in As You Like It: 'The fool doth think he is wise, but the wise man knows himself to be a fool.' (Shakespeare, 1623)

 

 

A more helpful attitude towards knowledge is one of greater humility that keeps in mind the strong possibility that there is more to the real world than our own perspective allows us to see. This is well illustrated by Edwin Abbott's 1884 satirical short novel entitled Flatland: A Romance of Many Dimensions (Abbott, 1884). The book comments on the hierarchy of Victorian culture, but its most enduring contribution is its examination of dimensions. The story is about a two-dimensional world occupied by geometric figures, and is told from the perspective of a Square who dreams about a visit to a one-dimensional world (Lineland) inhabited by 'lustrous points', in which he attempts to convince the realm's monarch of a second dimension, but is unable to do so. The Square describes (from a two-dimensional point of view) a visit by a three-dimensional Sphere, which he cannot comprehend until he sees Spaceland (a tridimensional world) for himself. The book also talks of Pointland, where the Point (sole inhabitant, monarch, and universe in one) perceives any communication as a thought originating in his own mind (Solipsism).

 

Socrates and Plato

Some useful perspective can be gained by going back to the time of the Ancient Greeks. Socrates lived from 469 to 399 BC. He introduced a method (the Socratic method) of teaching that involved asking questions. This great philosopher, Socrates, did not record his own words for posterity, but fortunately Plato recorded the teachings of Socrates in many of his books. It is often said that all of philosophy following Plato is just a set of variations on the themes he introduced. The Socratic method led to the development of the scientific method.

A number of other people also played a part in the development of the scientific method. A few of these are highlighted here.

 

Karl Popper

Sir Karl Raimund Popper is generally regarded as one of the greatest philosophers of science of the 20th century. He was an Austro-British philosopher and a professor at the London School of Economics. He introduced the notion of falsification and falsifiability as being central to the scientific enterprise (Popper, 1959). Popper is known for his rejection of the classical inductivist views on the scientific method, in favour of empirical falsification: a theory in the empirical sciences can never be proven, but it can be disproved or falsified, meaning that it can and should be scrutinized by decisive experiments. If the outcome of an experiment contradicts the theory, one should refrain from ad hoc manoeuvres that evade the contradiction merely by making it less falsifiable.

 

 

This view led Popper to posit that the strength of a scientific theory lies in its both being susceptible to falsification, and not actually being falsified by criticism made of it. He considered that if a theory cannot, in principle, be falsified by criticism, it is not a scientific theory.

Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory, but a single counterexample is logically decisive: it shows the theory, from which the implication is derived, to be false.

Popper states that while there is no way to prove that the sun will rise, it is possible to formulate the theory that every day the sun will rise; if it does not rise on some particular day, the theory will be falsified and will have to be replaced by a different one. Until that day, there is no need to reject the assumption that the theory is true. Popper and David Hume held to a similar view that there is often a psychological belief that the sun will rise tomorrow, but both denied that there is logical justification for the supposition that it will, simply because it always has in the past.

The search for truth is 'one of the strongest motives for scientific discovery' (Popper, 1959).

Perhaps our attitude as scientists should be one of deliberately trying to prove wrong all of the things that we hold true and closest to us. Go out and test things. If they are true they will be true; if they are not, then it was a good thing to test it anyway.

 

Thomas Kuhn

Thomas Kuhn, an American philosopher, introduced some new ways of looking at the scientific method, describing revolutions in science by paradigm shifts, a paradigm being the accepted corpus of methods and theories within a field (Kuhn, 1966). For example, in the biological sciences, understanding changed dramatically after Darwin; in sociology, economics, and politics, things changed dramatically after Karl Marx published his work; as they did in physics after Albert Einstein made known his theory of relativity.

 

 

Science textbooks expound the body of accepted theory, show many successful applications, and provide exemplary observations and experiments. Before such books became popular in the early 1800s, many of the famous classics of science fulfilled a similar function. Aristotle's Physica, Ptolemy's Almagest, Newton's Principia and Opticks, Franklin's Electricity, Lavoisier's Chemistry, and Lyell's Geology served for a time to define the legitimate problems and methods of a research field for succeeding generations of practitioners. Kuhn explained that the study of the paradigms presented in these books prepares the student for membership in the particular scientific community, and provides a common fundamental basis for effective communication.

History suggests that the road to a firm research consensus is extraordinarily arduous, and often proceeds by a series of scientific revolutions, or shifts away from the prevailing paradigm of the day. For example, in the 1700s Newton's Opticks taught that light was made up of material corpuscles. In the 1800s physics texts taught that light was transverse wave motion. In the 1900s, the work of Planck, Einstein, and others taught that light is photons - quantum-mechanical entities that exhibit some characteristics of waves and some of particles.

Paradigms determine what problems are studied, what methods are used, and what criteria are employed to judge the results. For example, chemists, after Dalton introduced his atomic theory, reported chemical compositions as ratios of integers rather than as decimals with fractions.

The paradigm provides a framework to suggest which experiments are worth conducting and which are not. Both fact collection and theory articulation became highly directed activities. Here, Francis Bacon's comment is appropriate: 'Truth emerges more readily from error than from confusion' (Bacon, 1869, p. 210).

By using an established paradigm (or textbook) as a base, the creative scientist can begin his or her research where it leaves off. Research can then be embodied not in books addressed to a general audience, but in shorter research articles addressed to professional colleagues who work within the same paradigm.

As physicist Max Planck observed (before Popper or Kuhn): 'A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it' (Planck, 1949).

 

Scientific method

Science is an error-correcting process that tests our ideas against the real world.

Observations of the world lead to the recognition of patterns (through inductive reasoning) which lead to interesting questions about why the pattern might occur. Hypotheses are formulated in such a way that they are framed as testable questions. The scientific method cannot be applied to untestable, unfalsifiable questions. A number of possible explanations are found, and the scientist needs to think up ways of testing which ones might be wrong. Data sets are gathered (and repeated) to test the prediction. The hypothesis is then rejected, accepted, or refined and re-tested. Experiments should be replicated reproducibly. Where appropriate, control groups should be used as a reference. In the medical or pharmaceutical fields, double-blind protocols are used where both the subjects being experimented on and the experimenter do not know exactly what is happening in the experiment. If two ideas explain the data equally well, the simpler one is preferred. Occam's razor suggests that the simpler theory with fewer (or less onerous) unproved assumptions is probably the most appropriate one. General theories then gather together the hypotheses that are consistent with all current data. They remain provisional and tentative until something better comes along.

Scientific theories cannot be proven (only corroborated), but can be shown to be beyond reasonable doubt. We can be quite confident that the sun will rise tomorrow. But this does not imply that it will rise ten billion years from now (by which time its fuel will most likely have been exhausted). In science there are no authorities. There are experts at most, and even their opinions can be challenged by anyone - so long as there is an argument, and evidence to back it up.

 

Laws of thermodynamics

Thermodynamics is a collection of useful mathematical relations between quantities, every one of which is independently measurable. Although thermodynamics tells us nothing whatsoever about the microscopic explanation of macroscopic changes, it is useful because it can be used to quantify many unknowns. Thermodynamics is useful precisely because some quantities are easier to measure than others.

The laws of thermodynamics provide an elegant mathematical expression of some empirically discovered facts of nature. The principle of energy conservation allows calculations to be made of the energy requirements for processes. The principle of increasing entropy (and the resulting free-energy minimization) allows predictions to be made about the extent to which those processes may proceed.

Thermodynamics deals with some very abstract quantities, and makes deductions from mathematical relations. In this, it is a little like mathematics itself, which, according to Bertrand Russell, 'may be defined as the subject in which we never know what we are talking about, nor whether what we are saying is true'. However, thermodynamics is trusted as a reliable source of information about the real world, precisely because it has delivered the goods in the past. Its ultimate justification is that it works.

 

 

 

Library of Alexandria

Much of the knowledge of the ancient Greeks was captured in papyrus scrolls in the Library of Alexandria in Egypt from around 280 BC until a few hundred years later when, sadly, it was destroyed. During its existence, the library was the largest in the world. Perhaps it could be argued that its leading position came about because Alexandria dominated papyrus production - the required technology of the time. (In this way, Alexandria could almost be seen as the Silicon Valley of the ancient world.) In addition to collecting ancient works, the library also hosted numerous international scholars, paid for by the Egyptian rulers. This allowed the Library of Alexandria to work towards the fulfilment of its mandate of collecting all of the world's knowledge at the time. Scholars such as Euclid and Archimedes are said to have studied, written, and experimented at Alexandria.

 

Historical mining and metallurgical publications

The tradition of scientific and technical literature in the field of metallurgy dates back to the mid 1500s. The first printed book on metallurgy to have been published in Europe is considered to be De la Pirotechnia, written in Italian by Vannoccio Biringuccio, and published in Venice in 1540. This book (Biringuccio, 1540) gives details on mining practice, the extraction and refining of numerous metals, and the production of alloys such as brass.

Georgius Agricola (1494-1555) was not the first writer on the subjects of mining and metallurgy, but is well known as the author of De ReMetallica ('of things metallic' or 'on the nature of metals'), a book that documents and illustrates the observations he made in the course of his extensive travels in the 1500s. Agricola wrote extensively about mining methods and metallurgical processes that were in place in the Middle Ages. The original Latin version of De Re Metallica was published in 1556, the year after Agricola died. It was later translated into English by Herbert Hoover, a mining engineer and later President of the United States, and his wife Lou Henry Hoover, a geologist and classicist. The English version of De Re Metallica was published in 1912.

In his preface to De Re Metallica (Agricola, 1556), the author says: 'I have omitted all those things which I have not myself seen, or have not read or heard of from persons upon whom I can rely. That which I have neither seen, nor carefully considered after reading or hearing of, I have not written about.'

 

 

 

Philosophical Transactions of the Royal Society

Scientific publishing of journal papers has been in existence for 350 years. The world's oldest and longest-running scientific journal, the Philosophical Transactions of the Royal Society, was first published in March 1665, in London. 'Henry Oldenburg - Secretary of the Royal Society and first Editor of the publication - ensured that it was "licensed by the council of the society, being first reviewed by some of the members of the same", thus making it the first ever peer-reviewed journal' (Royal Society, 2011). The motto of the Royal Society is 'Nullius in verba' - Latin for 'take nobody's word for it'.

In the interests of making scientific information available to a wider audience, the Royal Society announced in October 2011 that it had made the historical archives of the Philosophical Transactions (over 60 000 scientific papers) permanently free to online access from anywhere in the world. All of the historical archival papers (published more than 70 years ago) from the Philosophical Transactions are now freely available on the website of the Royal Society. It is now easy to read the original published work of Newton, and Benjamin Franklin's account of his experiments with lightning by means of holding a kite in a thunderstorm. Current publications are available through delayed open access where older articles (12 months for biological sciences, and 24 months for physical sciences) are made freely available. They also allow a hybrid open-access or open-choice option where authors can pay an article-processing charge that allows for their article to be made freely available immediately upon publication. Such articles are covered by a Creative Commons licence allowing redistribution and re-use (Royal Society, 2012).

 

Technical societies and the SAIMM

In the early days of scientific societies, regular meetings were held to discuss science and to conduct experiments. The reading of scientific papers took place, and publication of papers (and the ensuing discussion) was undertaken to record the proceedings of meetings. Meetings often included rather robust debate. A typical scene from a meeting of a scientific society is shown in Figure 8.

 

 

 

 

The origins of the SAIMM can be traced back to a meeting of fourteen chemists and metallurgists that took place on 24 March 1894 at the North-Western Hotel, 21 Pritchard Street, Johannesburg. The meeting saw the formation of the Chemical and Metallurgical Society of South Africa and the election of a council. At that stage Johannesburg was a very young city, with gold having been discovered there only a few years earlier, in 1886.

In his inaugural address as President of the Chemical and Metallurgical Society of South Africa in 1894, William Bettel made the following remarks:

'As Chemists and Metallurgists of the Rand you have before you much useful and interesting work, and it remains with you, gentlemen, by publication or diffusion of accurate scientific information, by exposure of pseudo-scientific frauds, ... to claim as a right the recognition of your proper status in relation to this community. I heard a rumour about a certain company getting an actual extraction of 125 per cent. from concentrates. Such results as I have instanced are obviously imagined, or as chemists say, "cooked" (Bettel, 1894).

Some further examples of forthright comment can be found in the records of monthly meetings from 1895.

Asfor Mr Suckling's process, I fail to see the novelty or usefulness of the method. The use of a blast (pressure) instead of a draught of air (suction) is, to my mind, a retrograde movement, and absurdfrom its manifest conditions' (Bettel, 1895). 'I have examined the Sulman processfrom both theoretical and practical points of view, and I can only come to the conclusion that it is not a bromination process, neither is it a cyanide process, but that it is a very bad oxidation process, consequently useless' (Schlunde, 1895).

In 1903, mining engineers were included in the society and the name was changed to the Chemical, Metallurgical and Mining Society of South Africa. In 1956, another name change took place, with the new identity being the South African Institute of Mining and Metallurgy. Fifty years later, in 2006, the expansion of activities to the wider region led to the current name of the Southern African Institute of Mining and Metallurgy.

Not only do names change, but the passage of time has brought about a change in the nature of scientific societies. In today's world, there is a plethora of publications, and it is close to impossible for anyone to keep up with the vast flow of information. International conferences with hundreds of presentations have taken the place of local meetings that used to discuss a single paper or experiment. In this frenetic environment, it is essential that researchers are able to trust the material they read.

Learned societies nowadays exist to promote an academic discipline or profession, and are mostly not-for-profit organizations. They typically hold conferences for the presentation and discussion of new research results, and publish or sponsor academic journals in their discipline. The system of peer review (significantly flawed, but the best we have) is used to maintain standards and to improve the quality of papers, but reviewers need to be chosen carefully and monitored. Nowadays, some learned societies continue to publish journals themselves, while others have contracted this job to commercial publishing companies. The SAIMM is fortunate to be in control of its own destiny in this regard.

 

Information explosion and electronic publishing

In recent times, the Internet, and the World Wide Web (Berners-Lee, 2000) - devised by Tim Berners-Lee in 1989, and which attained mass popularization about twenty years ago - have transformed the dissemination of knowledge, a capacity once exclusive to publishers.

Google CEO Eric Schmidt said in 2010: 'Between the birth of the world and 2003, there were five exabytes of information created. We [now] create five exabytes every two days.' It is fair to describe this as an information explosion.

Along with the exponential growth in the world's population in modern times, there has been an even greater exponential growth in the world's accumulated knowledge. The advent of the Internet, the World Wide Web, and search engines such as Google has made it easy to find information on almost any topic. This would have been almost unimaginable as little as twenty years ago. Open access to information is invaluable and taken for granted by many.

Science has grown exponentially since the late 1600s, both in respect of number of researchers and publications. According to Price (1963), the 'size of science' has increased by an estimated five orders of magnitude in three centuries. Price also said: 'we can say that 80 to 90 percent of all the scientists that have ever lived are alive now'.

Journal publishing of scientific papers is the most common form of dissemination of new research results, in particular in science and medicine. Other types of scientific publication include conference papers, book chapters, books, and reports.

Björk and colleagues (2009) estimated that 1.35 million scientific journal papers were published in 23 750 refereed journals in 2006. The total number of active scholarly journals, refereed plus non-refereed, was 60 911. There were 2690 open access scholarly journals, including 1735 that were also refereed. They also found that 19.4 per cent of these papers were openly available online.

According to Reich (2013), more than 2 million papers were published in 2012.

The rise of China in the internationally influential journal literature indexed by Thomson Reuters - in terms of share of world output - is the most significant event in the structure of scientific research in the past 30 years. In 1983, China produced just 0.6 per cent of articles surveyed by Thomson Reuters in the Science Citation Index (Web of Science). By 2013, China produced some 13 per cent of the literature, second only to the United States at 29 percent (King and Pendlebury, 2013).

PLoS ONE, published (as paid open access) by the not-for-profit Public Library of Science, published 6749 papers in 2010, which makes it the world's largest journal (Whitfield, 2011). Articles published in this journal undergo peer review, but some of the standard criteria that older journals use to screen out articles - such as 'degree of advance' or 'interest to a general reader' - are not used by PLoS ONE reviewers; all papers of scientific merit are posted to the public record. Only weeks (not months) go by before a submitted article is published; the journal is in a state of continuous publication, not printed periodically.

In some fields of science, such as physics and astronomy, print journals have receded in importance owing to online repositories such as arXiv (pronounced 'archive') that disseminate studies without the nuisance of peer review. Physicist Paul Ginsparg, formerly at the Los Alamos Lab and now at Cornell University, created a free archive of unrefereed physics "e-prints" - a pre-publication server that is now a primary means for physicists to exchange information. Worldwide readership discovers errors quickly, and authors revise their submissions in response to feedback. This works well for physics, but might be less suitable for the medical community where it might promote the use of unfounded cures.

Recent developments in electronic publishing on websites make it possible to disseminate information more widely and cost-effectively than before. Professional societies have an opportunity to serve their members and their industry by publishing high-quality peer-reviewed papers on their websites as well as in printed form. Older publications can be scanned, and optical character recognition (OCR) can be used to provide searchable text.

 

A long title, a long list of authors, and a short abstract

The style of a journal paper follows a typical stylised form. It always has a title, a list of authors, their affiliations, and an abstract. Typically, papers have a length of 3 000 to 10 000 words. However, there have to be those who take things to the extreme.

Among the contenders for the longest title of a research paper is 'The nucleotide sequence of a 3.2 kb segment of mitochondrial maxicircle DNA from Crithidiafasciculata containing the gene for cytochrome oxidase subunit III, the N-terminal part of the apocytochrome b gene and a possible frameshift gene; further evidence for the use of unusual initiator triplets in trypanosome mitochondria' (Sloof et al., 1987).

A physics paper with 5154 authors (Aad et al., 2015) broke the record for the largest number of contributors to a single research article. This paper presents collaborative work done at the Large Hadron Collider to determine the most precise estimate yet of the mass of the Higgs boson. The 33-page article in Physical Review Letters devotes nine pages to describing the research itself (including references) and 24 pages to listing the authors and their institutions (Castelvecchi, 2015).

More refreshing in style is the following example of a paper (Berry etal., 2011) with a very short abstract:

Title: Can apparent superluminal neutrino speeds be explained as a quantum weak measurement?

Abstract: Probably not.

The shortest paper (Upper, 1974) is entitled The unsuccessful self-treatment of a case of "writer's block"; it contains no words at all in the body of the paper. The published review of the paper said: 'I have studied this manuscript very carefully with lemon juice and X-rays and have not detected a single flaw in either design or writing style. I suggest it be published without revision. Clearly it is the most concise manuscript I have ever seen-yet it contains sufficient detail to allow other investigators to replicate Dr. Upper's failure. In comparison with the other manuscripts I get from you containing all that complicated detail, this one was a pleasure to examine. Surely we can find a place for this paper in the Journal-perhaps on the edge of a blank page.'

 

Citations are rare

The frequency of citations that a paper receives is often used as an indicator of quality, even though this approach has its limitations. When a work is cited, it generally indicates that it is taken as being relevant to the citing author's research. Citations allow scientists to gauge how much their research is used by other authors. Citations, in this way, are an indicator of productivity as well as impact.

As reported by Garfield (1998) and Schwartz (1997), studies conducted on the journals indexed by the Institute for Scientific Information (ISI) indicated that large percentages of the scholarly literature were never cited. It was found that 55 per cent of the papers published between 1981 and 1985 received no citations at all in the five years after they were published. Another study of papers published in 1984 found that 47 per cent of articles in the physical sciences, 75 per cent of articles in the social sciences, and 98 per cent of articles in the arts and humanities had not received any citations by the end of 1988. More than 72 per cent of all papers published in engineering had no citations at all, and for metallurgy and mining the figure was 75 per cent. These statistics apply to the total of every type of article that was indexed (including journal papers, editorials, obituaries, and letters). A narrower interpretation of the data applying only to journal papers found that 22 per cent of articles in the physical sciences, 48 per cent of articles in the social sciences, and 93 per cent of articles in the arts and humanities had not received any citations by the end of 1988. Citations of journal papers are most common in the biological and physical sciences, but less so in engineering where conferences are more important, and where implementation is more important than publication. Social sciences and the humanities tend to place greater reliance on books than on journals.

A more recent study found that, in a sample of over 1.3 million papers across all disciplines and years, 61 per cent of papers had zero citations, and 12 per cent had only one citation, with 4 per cent having 16 or more citations.

The point was also made by Garfield (1998) that a small group of journals account for the vast majority of significant research publications, and the overwhelming majority of articles published in the 200 journals with the highest cumulative impact are cited within a few years of publication, and after five years, uncitedness is almost nonexistent.

 

Negative citations

Not all citations are positive. For instance, Andrew Wakefield wrote a controversial paper on the association between the MMR vaccine and autism, which was published in a leading medical journal The Lancet. This paper has received nearly two thousand citations, whereas most authors would be thrilled to receive a hundred. However, the quality of Wakefield's research is not at all reflected by this large number. Many of these citations are a product of the storm of controversy surrounding the work, and are contained within papers which are critical of the methods used. Wakefield's research has now been robustly discredited, and the paper was retracted by The Lancet in 2010. Nevertheless, this extreme case highlights serious problems with mechanistically judging a paper, or an academic researcher, purely by number of citations.

 

Open access

The Universal Declaration of Human Rights says that everyone has the right freely to ... share in scientific advancement and its benefits'. Access to information is a basic human right that is also entrenched in the Constitution of South Africa. Electronic publishing has changed the dynamics of the dissemination of information, and it is now possible to provide universal, unrestricted free access to full-text scholarly materials via the Internet. An argument for open access publishing (Jones, 2012) has been presented previously in this Journal. Through proper management of open-access publication, it is possible to maintain the same standards of high-quality production of peer-reviewed papers, with the potential for greatly increased dissemination and citation. The SAIMM made its Journal papers available through open access in 2007.

Today's world faces many policy choices, including issues such as climate change, and food production and intake, as mentioned earlier. These issues cannot be properly addressed without widespread access (by researchers and the general public) to the results of scientific research in each of these areas. In 2012, the British government announced an initiative to make all taxpayer-funded research available online to anyone who wants to read or use it (Jones, 2012). In February 2013, the US White House announced that government-funded research should be made free to read within 12 months of publication. From 2014, the results of all research funded by the European Union must be open access.

Estimates of the proportion of papers currently available free online range from 30 per cent to 50 per cent. Half of the papers published in 2011 are now free to read (Van Noorden, 2013).

 

Wikipedia and Encyclopaedia Britannica

In 2005, Nature published a study it undertook of the accuracy of articles from Wikipedia and Encyclopaedia Britannica. This showed that the difference in accuracy was not particularly great (averaging out to 2.92 mistakes per article for Britannica, and 3.86 for Wikipedia). Britannica is based on strong scholarship, sound judgment, and disciplined editorial review. Wikipedia is very current, comprehensive in coverage, and is based on continuous correction by many people. The core Wikipedia values include a neutral point of view, no original research (as appropriate for an encyclopedia), verifiable information only, and citing sources.

 

Stigler's Law

Stigler's Law of Eponymy is a process proposed by Stephen Stigler, a professor of statistics at the University of Chicago (Stigler, 1980). In its simplest and strongest form it says: 'No scientific discovery is named after its original discoverer'.

Stigler pointed out that ' [i]t can be found that Laplace employed Fourier Transforms in print before Fourier published on the topic, that Lagrange presented Laplace Transforms before Laplace began his scientific career, that Poisson published the Cauchy distribution in 1824, twenty-nine years before Cauchy touched on it in an incidental manner, and that Bienaymé stated and proved the Chebychev Inequality a decade before and in greater generality than Chebychev's first work on the topic.' For that matter, the Pythagorean theorem was known before Pythagoras, and Gaussian distributions were not discovered by Gauss.

Historical acclaim for discoveries is often assigned to persons of note who bring attention to an idea that is not yet widely known, whether or not that person was its original inventor. Eminent scientists will often get more credit than a comparatively unknown researcher, even if their works are similar; it also means that credit will usually be given to researchers who are already famous. Some examples from the Wikipedia entry for Stigler's Law are listed below.

> Alzheimer's disease had been previously described by at least half a dozen others before Alois Alzheimer's 1906 report which is often (wrongly) regarded as the first description of the disorder

> The Bessemer process was discovered by William Kelly in 1851. Henry Bessemer was the first to obtain a patent in 1855

> Fibonacci was not the first to discover the famous sequence of Fibonacci numbers. They had existed in Indian mathematics since 200 BC. Fibonacci provided the series in 1202 AD

> The normal or Gaussian distribution was introduced by Abraham de Moivre in 1733, but was named after Carl Friedrich Gauss who began using it in 1794

> Newton's first and second laws of mechanics were known and proposed in separate ways by Galileo, Hooke, and Huygens before Newton described these in his Philosophic Naturalis Principia Mathematica. Newton owns the discovery of only the third one

> The Reynolds number in fluid mechanics was introduced by George Stokes, but is named after Osborne Reynolds, who popularized its use

> Stokes's theorem was discovered by Lord Kelvin.

Stigler explicitly named the sociologist Robert K. Merton as the true discoverer of 'Stigler's Law', and so avoided this law about laws disobeying its very own decree. 'Stigler's Law' is an example of itself. Robert Merton described the principle in his 1957 Presidential Address to the American Sociological Society (Merton, 1957). Merton is regarded as one of the founding fathers of sociology. He also developed and popularized notable concepts such as 'unintended consequences', as well as coining the phrases 'role model', and 'self-fulfilling prophecy'. Throughout his career, Merton came to publish about 50 papers in the sociology of science.

 

Newton and Leibniz

Robert Merton talks about the structure of the scientific enterprise and the pressure that is placed on scientists with regard to priority in scientific discovery. (This can be seen, for example, in the way that scientific journals often print the date on which the manuscript of a published article was received, in order to record its priority date.) One example of these pressures arose in the invention of calculus - an idea whose time had come, and was independently invented in two places at a similar time, by Isaac Newton (1642-1727) and Gottfried Wilhelm Leibniz (1646-1716). [There are discrepancies in the recorded dates of birth and death, depending on whether Julian or Gregorian calendars were used.] Newton was the first to conceive of the methods of calculus, but Leibniz was the first to publish on the topic.

 

 

Much controversy arose between Newton and Leibniz over the invention of calculus. When the Royal Society finally established a committee to adjudicate the rival claims, Newton, who was then president of the Royal Society, packed the committee, helped direct its activities, anonymously wrote the preface for the second published report - the draft is in his handwriting - and included in that preface a disarming reference to the old legal maxim that 'no one is a proper witness for himself'. Newton must have felt intense pressure for self-vindication that made him adopt such offensive means for the defence of his valid claims. According to Merton, it was not because Newton was so weak but because the institutionalized values were so strong that he was driven to such lengths.

 

Fraud in science

Concerns about fraud in science have been around for a long time. In 1830, Charles Babbage deplored unreliable science, and was concerned about the prevailing state of affairs in the scientific world of his day. He discussed 'hoaxing, forging, trimming, and cooking' of data (Babbage, 1830).

Merton (1957) mentions the remarkably prolific Vrain-Lucas who, in the mid-1800s, created and sold more than 27 000 pieces of manuscript that included letters by Pontius Pilate, Mary Magdalene, the resurrected Lazarus, Ovid, Luther, Dante, Shakespeare, Galileo, Pascal, and Newton, all written on paper and in modern French. 'Most provocative among these documents was the correspondence between Pascal and the then eleven-year-old Newton (all in French, of course, although even at the advanced age of thirty-one Newton could struggle through French only with the aid of a dictionary), for these letters made it plain that Pascal, not Newton, had, to the greater glory of France, first discovered the law of gravitation, a momentous correction of history, which for several years excited the interest of the Académie des Sciences and usurped many pages of the Comptes Rendus until, in 1869, Vrain-Lucas was finally brought to book and sentenced to two years in prison.'

In the Mécanique Celeste (until then, outranked only by Newton's Principia) 'theorems and formulae are appropriated wholesale without acknowledgement' by Laplace (Merton, 1957, p. 652).

 

Piltdown hoax

The Piltdown Man was an infamous paleoanthropological hoax, perpetrated in 1912, in which bone fragments (parts of a skull and jawbone) were presented as the fossilized remains of a previously unknown early human. These fragments were said to have been collected in 1912 from a gravel pit at Piltdown, East Sussex, in England by Charles Dawson. The significance of the specimen remained controversial until it was exposed in 1953 as a forgery; it consisted of the lower jawbone of an orang-utan deliberately combined with the cranium of a fully developed modern human. After forty years of uneasy acceptance, the Piltdown Man was shown to be a carefully contrived hoax (Straus, 1954).

 

Plagiarism

An article from 2014 entitled 'Development of a guideline to approach plagiarism in Indian scenario' was retracted by the editors of the Indian Journal of Dermatology (2015), as large portions of the manuscript were copied from a first-round questionnaire of a dissertation entitled 'Developing a comprehensive guideline for overcoming and preventing plagiarism at the international level based on expert opinion with the Delphi method' by another author. This rather ironic occurrence was reported by Retraction Watch (2015).

 

Peer review

Although outright fraud is uncommon, it is necessary to have checks and balances in place to ensure the integrity of published scientific data.

The system of peer review is used to maintain standards and to improve the quality of papers. This vital system is, however, significantly flawed. There is little incentive for reviewers to invest sufficient time in picking up all errors in publications, and any ineptitude on their part is usually protected by anonymity. It has reached the point where some reviewers have mistakenly permitted the publication of hoax papers deliberately presented with a complicated scientific facade. In light of such astounding inadequacies, perhaps a more open review process would be an improvement.

Peer reviewers are not paid, nor adequately rewarded in any other way for what is very hard work. Nor are they held accountable by having to sign their names to their reviews.

The process of peer review is costly and time-consuming. The annual cost of peer review was estimated for 2008 as being about US $2.8 billion (Brembs et al., 2013).

Peer review is resistant to new or controversial ideas. The agreement between referees is often little higher than by chance. Review is also vulnerable to misconduct, plagiarism, and breaches in confidentiality. For example, there was a case reported by Retraction Watch (2012) where an author faked e-mail addresses of suggested reviewers so that he could review his own work.

Wasserman (2012) has criticised the current system of peer review as outdated and rather arbitrary. He has seen too many examples where one referee rejects a paper, and another equally qualified referee accepts it. He questions whether it is fair for a scientist to 'work hard on something for two years only to have it casually dismissed by a couple of people who might happen to be in a bad mood or who feel they have to be critical for the sake of being critical'.

The current system of peer review is a fairly recent innovation, not widespread until the middle of the twentieth century (New Atlantis, 2006). In the nineteenth century, many science journals were commandingly led by what Ohio State University science historian John C. Burnham dubbed 'crusading and colorful editors', who made their publications 'personal mouthpieces' for their individual views. In time, the specialization of science precluded editors from being qualified to evaluate all the submissions they received. About a century ago, Burnham notes, science journals began to direct papers to distinguished experts who would serve on affiliated editorial boards. Eventually - especially following the post-World War II research boom - the deluge of manuscripts and their increasing specialization made it difficult for even an editorial board of a dozen or so experts to handle the load. Journal editors began to seek out experts capable of commenting on manuscripts - not only researchers in the same general field, but researchers familiar with the specific techniques and even laboratory materials described in the papers under consideration. The transition from the editorial-board model to the peer-review model was eased by technological advances, such as the Xerox copier in 1959, that reduced the difficulties of sending manuscripts to experts scattered around the globe (New Atlantis, 2006).

Drubin (2011) has provided an excellent set of guidelines for peer reviewers. A rigorous review process should ensure that published papers are reliable and credible. The review process itself should be constructive, fair, and civil. Drubin passes on the suggestion by David Botstein that reviewers should focus principally on the questions 'Is it new and is it true?' and leave it to future generations to judge a publication's impact.

A recent proposal (Schuman, 2014) suggested that authors should be required to volunteer first to review someone else's article for the same journal in which they would like to publish their own work. The review should be timely (say within three months) and constructive.

 

Peer review: open/closed and identified/anonymous

Many traditional journals use closed anonymous reviews, in that the reviews are not available to readers, and the authors do not know who the reviewers are. The British Medical Journal (BMJ) has its reviewers sign their comments, but does not publish the reviews.

If peer reviews were made public, this would increase the transparency of the publishing process, and would encourage reviewers to write more objective and reasoned reviews. It would keep reviewers focused and fair if they knew that they were accountable for their reviews. Journals, however, might be reluctant to change, for, if reviews were visible (and/or not anonymous) it might be even harder than it already is to find willing reviewers.

Drawbacks of anonymity

> Reviewers do not get credit for their reviewing work. They cannot, for example, reference particular reviews in their CVs as they can with publications. Perhaps promotion committees at universities should consider giving credit to faculty members for writing reviews

> It is relatively easy for a reviewer to provide unnecessarily blunt or harsh critique

> It is difficult to guess if the reviewer has any conflict of interest with the authors by being, for example, a competing researcher interested in stalling the paper's publication.

Advantages of anonymity

> Reviewers do not have to fear 'payback' for an unfavourable review that is perceived as unfair by the authors of the work

> Some reviewers (perhaps especially high-profile senior scientists) might find it difficult to find time to provide as thorough a review as they would ideally like to provide, yet would still like to contribute and can perhaps provide valuable experienced insight. They can do so without putting their reputations on the line.

 

Failure of anonymous peer review

The two main goals of a review system are to minimize both the number of bad studies that are accepted for publication and the number of good studies that are rejected for publication. (This ignores the other intended benefit, which is to improve the quality of a paper.) The cost of wrongly rejecting good papers is invisible (as they do not get published) but potentially very high, as good work may not get the exposure it deserves, a consequence that could discourage promising young scientists. Cases have been documented where a number of very talented and promising young scientists sent work to a journal, fully expecting to be scrutinized, but received reviews that were so personal, rude, scathing, and above all, unfair, that they decided to look for another profession and never returned to science. The inherent conservatism in anonymous peer review means that people with new, original approaches to old problems run the risk of being shut out.

The most fundamental problem with anonymous peer review is the lack of accountability. Reviewers can basically say whatever they want to say, because they are protected by anonymity. An additional problem arises from reviewers having too little time (note that they are not paid for their work), which leads to sloppy and superficial reviews. There is also the temptation to misuse the power available to reviewers: if we look at peer review as a strategic game, rejecting everything is a strong strategy, as this will always reduce the influence of the reviewer's competition.

It has been proposed that reviewers should sign their reviews, and should be able to stand by what they say and not be able to hide behind anonymity in a cowardly fashion. Provision can be made for anonymity on those occasions where a junior person is asked to review the work of an established senior researcher, and fears for possible repercussions. Reviews could be stored and made accessible on a website. Reviewers who reject good papers, and reviewers who accept bad papers, for whatever reason, can be held accountable for what they write. Also, on a more encouraging note, reviewers would get more credit for their reviews. Under the current system, the difference between being a constructive reviewer and a careless one is invisible to all except journal editors (De Ruiter, 2014).

A study published in the British Journal of Psychiatry (Walsh et al., 2000) used a randomized controlled trial to evaluate the feasibility of an open peer-review system. Reviewers were asked whether they would agree to having their names revealed to the authors whose papers they review. A significant 76 per cent agreed to signing their names, 11 per cent refused, and 13 per cent failed to respond. Signed reviews were of a higher quality, were more courteous, but took longer to complete than unsigned reviews. Reviewers who signed their names were more likely to recommend publication. The study supported the feasibility of an open peer-review system.

Some journals have started printing the names of reviewers. The British Medical Journal (BMJ), for instance, decided to discontinue anonymous peer reviews in 1999 (New Atlantis, 2006). Open peer review allows for greater transparency and accountability.

The current usual model is pre-publication peer review. It is also possible to take the somewhat bolder step of publishing papers immediately and then conducting the review in the open afterwards (post-publication peer review). Some online journals have taken to using 'transparent' peer review where the reviewing process is visible as it takes place online. A more dynamic approach allows for reviews and comments to be posted at any time.

A further question arises as to whether the identity of the author should be disclosed to the reviewers. A double-blind review lies at the other end of the spectrum from open peer review. While there might be some hypothetical advantages in masking the identity of the author, reviewers familiar with their fields will usually be able to know immediately who has written the paper under consideration.

 

Bad peer review

David Shatz has pointed out that 'many heavily cited papers, including some describing work which won a Nobel Prize, were originally rejected by peer review' (Shatz, 2004). Shatz, a Yeshiva University philosophy professor, outlines some of the charges made against the referee process in his 2004 book Peer Review: A Critical Inquiry. He maintains that reviewers are often not really 'conversant with the published literature'; they are 'biased toward papers that affirm their prior convictions'; and they 'are biased against innovation and/or are poor judges of quality'. Reviewers also seem biased in favour of authors from prestigious institutions. Shatz describes a study in which 'papers that had been published in journals by authors from prestigious institutions were retyped and resubmitted with a non-prestigious affiliation indicated for the author. Not only did referees mostly fail to recognize these previously published papers in their field, they recommended rejection.'

Campanario (1995) describes examples of influential and/or highly cited papers that were initially rejected by one or more scientific journals. The work reported in eight of the papers eventually earned Nobel Prizes for their authors; six papers later became the most cited of the journals in which they were published. Also described are influential and highly cited scientific books whose authors encountered problems in publishing them. These case studies suggest that, although rejection may subsequently result in an improved manuscript, on other occasions referees may simply have failed to appreciate a paper's importance. Many of these rejected papers also reported unexpected findings or discoveries that challenged conventional models or interpretations.

Some of the most cited papers of all times were rejected by referees, or returned by editors. Of course, these are the ones we know about - the ones where authors have persisted until their papers eventually get published. Shatz (2004, p. 90) mentions numerous innovative papers that were initially rejected in the process of peer review. These include papers presenting the discovery of blood typing, Jenner's 1796 paper describing vaccinations against smallpox, Murray Gell-Man's work on quarks, and Krebs's paper describing the citric acid cycle. Nature declined to accept Krebs's paper on the 'Krebs cycle' in 1937, saying:

'The Editor of Nature presents his compliments to Mr. H.A. Krebs and regrets that as he has already sufficient letters tofill the correspondence columns of Nature for seven or eight weeks, it is undesirable to acceptfurther letters at the present time on account of the delay which must occur in their publication. If Mr. Krebs does not mind such delay, the Editor is prepared to keep the letter until the congestion is relieved in the hope of making use of it. He returns it now, however, in case Mr. Krebs prefers to submit it for early publication to another periodical.'

In 1988, seven years after Krebs's death, an anonymous editor published a letter in Nature calling the rejection the journal's most 'egregious error' (Borrell, 2010). The work by Krebs later won a Nobel Prize.

At least 35 articles that would eventually earn the Nobel Prize and fame for their authors were rejected outright during the initial inspection by reviewers (Campanario, 1995). As one example, in the case of quasicrystals, there was a paper rejected by the Journal of Applied Physics in 1984, but through persistence it was later published in Metallurgical Transactions A and Physical Review Letters. One of the reviewers, Linus Pauling, said: 'There is no such thing as quasicrystals, only quasi-scientists' (Slavov, 2015). Nevertheless, Dan Schechtman won the Nobel Prize for Chemistry in 2011.

Some further examples have been listed by Slavov (2014) of very significant foundational scientific results that were rejected by major journals and magazines but have nonetheless stood the test of time and proven to be of exceptional importance to science.

 

More damaging reviews

Merton (1957) highlights a few cases where poor reviews have had extremely damaging consequences:

The nineteenth-century physicist Waterston, his classic paper on molecular velocity having been rejected by the Royal Society as 'nothing but nonsense', became hopelessly discouraged and left science altogether.

Deeply disappointed by the lack of response to his historic papers on heredity, Mendel refused to publish the now permanently lost results of his further research and, after becoming abbot of his monastery, gave up his research on heredity.

Robert Mayer, tormented by refusals to grant him priority for the principle of conservation of energy, tried a suicide leap from a third-story window and succeeded only in breaking his legs and being straitjacketed, for a time, in an asylum.

In 1842, Mayer came up with the theory of conservation of energy, and wrote an article explaining his idea that 'energy is neither created nor destroyed'. It was rejected by the leading physics journal of the time, ended up in an obscure chemistry journal, and was mostly ignored by physicists. When the physicists of the time rallied around Joule, who described conservation of energy later in the 1840s, Mayer suffered a mental breakdown. Towards the end of his life, he was finally given credit as a father of thermodynamics.

Clearly, the consequences of poor peer review can be extremely serious.

 

Some recent examples of errors not detected by reviewers

One of the fundamental equations used in the modelling of the electrical characteristics of DC arc furnaces is one by Ben Bowman that describes the shape of the arc. This very important equation for modelling plasma arcs appears twice in the literature - wrongly! The first (correct) equation shown in Figure 10 describes the shape of the conducting volume of the arc as a function of the distance from the cathode attachment spot. The assumptions include an axi-symmetric arc and no interaction effects at the anode. This equation is incorrectly printed in Bowman (1994), and again but with a different error in Bowman and Krüger (2009), but has been corrected here by examining the form of the equation that properly fits the figure in the original reference.

One can only speculate as to the cause of these errors. Either the reviewers have hastily skipped over the equations, assuming them to be correct, or typesetters (not understanding the meaning of the equations) have introduced errors that were not checked before printing.

 

 

Another example, one that cannot be blamed on typesetters, occurs in a paper that describes the calculated activity coefficient of liquid cobalt oxide (CoO) in slag as a function of temperature and oxygen partial pressure. This relationship was presented as the equation shown in Figure 11.

 

 

Fortunately, there was also a graphical depiction of the data in the paper. For example, at a temperature of 1400°C (1673 K), and with a value of B = 1.15, the graph shows a value for /CoO of around 10, which is quite reasonable. However, the equation produces a result for /CoO of about 96 000 000, which is clearly spurious. My communication with the author led to the following admission: 'You are right. This equation is not correct. I made a serious mistake. Please never use this equation.'

 

Errata

Very often, errata are published in journals a few months after the initial publication of the paper, sometimes in small print, or somewhere out of the way. In these cases, there is no obvious way of linking the correction to the original publication. Electronic publishing allows this link to be made in a much more robust manner, by allowing errata to be linked to the original papers. A very simple solution is to publish the erratum as an additional page together with the original paper on the journal's website. This simple practical step can be expected to improve the current situation where errors tend to be propagated from one paper to the next, sometimes with additional mutations.

Rekdal (2014) tells a fascinating tale about a story entitled 'Spinach, iron and Popeye: Ironic lessons from biochemistry and history on the importance of healthy eating, healthy scepticism and adequate citation'. The myth about the iron content of spinach was embellished through quotation of secondary sources that were anything but authoritative. It would have been very helpful to have been able to append corrections to the various articles that formed part of the chain of this tangled but very readable story.

 

Tests of the peer-review process

There have been numerous experiments that put the peer review process to the test. Some of these tests involved hoaxes that have become well known in their own right.

In a noteworthy 1998 study, Fiona Godlee, editor of the prestigious British Medical Journal, sent an article containing eight deliberate mistakes in study design, analysis, and interpretation to more than 200 of the BMJ's regular reviewers. Not one picked out all the mistakes. On average, they reported fewer than two; some did not spot any. (Economist, 2013)

 

Sokal Hoax

In 1996, Alan Sokal (Professor of Physics at New York University) published a hoax article in Social Text, a leading North American journal of postmodern cultural studies. It was entitled 'Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity' (Sokal, 1996a). He wanted to test the journal's intellectual rigour and to see if it would publish an article that was liberally salted with nonsense, but sounded good, and flattered the editors' ideological preconceptions. The paper proposed ideas such as 'quantum gravity is a social and linguistic construct'. The hoax sparked a debate about the scholarly merit of humanistic commentary about the physical sciences; the influence of postmodern philosophy on social disciplines in general; academic ethics, including whether Sokal was wrong to deceive the editors and readers of Social Text; and whether Social Text had exercised appropriate intellectual rigour. This is not primarily what Sokal was trying to achieve; he was trying to make the point that postmodern relativism is an inadequate response to science. Sokal maintains that it is almost impossible to function in the world without some functional sense of truth.

On the day of the publication of the article in Social Text in May 1996, Sokal revealed in Lingua Franca (Sokal, 1996b) that the article was a hoax, identifying it as 'a pastiche of left-wing cant, fawning references, grandiose quotations, and outright nonsense ... structured around the silliest quotations [by postmodernist academics] [he] could find about mathematics and physics.'

A later book called Intellectual Impostures (Sokal and Bricmont, 1998) provided further details of nonsensical writing invoking science by postmodern philosophers and other literary figures.

The editors of Social Text thought the manuscript argued that quantum physics, properly understood, dovetails with postmodern philosophy. In fact, Sokal booby-trapped the piece with deliberate mistakes, as he later revealed; he sought to publish it to expose the various intellectual and political weaknesses in Social Text and those it represents.

Sokal's work is somewhat reminiscent of the 'two cultures' outlined by C.P. Snow, who proprosed knowing the second law of thermodynamics as a test of scientific literacy for the humanist, adding that it was 'about the scientific equivalent of: Have you read a work of Shakespeare's?? (Snow, 1964). It is necessary to have knowledge of both the sciences and humanities if one is truly to try to understand the world.

 

 

 

Bohannon

An extensive hoax to test the peer review process was carried out by John Bohannon, a biologist at Harvard, who also works as a journalist for Science. Between January and August 2013, Bohannon submitted, to 304 supposedly peer-reviewed journals, numerous almost-identical pseudonymous fake papers (obviously and seriously flawed) on the effects of a chemical derived from lichen on cancer cells. The paper's template was 'Molecule X from lichen species Y inhibits the growth of cancer cell Z'. The listed authors and their affiliations were unique and fake. It was quite disappointing to find that 157 journals accepted, and 98 rejected the paper. Some accepting journals were managed by major publishers (Elsevier, for example), but predatory paid open-access journals fared the worst. An article (Bohannon, 2013) entitled 'Who's afraid of peer review?' was published in Science on 4 October 2013. Bohannon's study was seriously flawed, especially with regard to how journals were chosen, but made quite an impact.

 

Measurements of scientific output by citation indexing

In order to improve the quality of published science, it is necessary to have some systems of measurement of the reach of publications.

This gives rise to the question of what constitutes a good paper. Essentially, the subject matter should be of interest or importance to at least some readers; the content should be communicated clearly and logically; and some papers should communicate new knowledge that is worth being referenced by other authors. The 'value' of a paper is a difficult thing to define, let alone to measure, but that has not stopped people from trying.

The simplest measure of the worth of a paper is the number of citations it receives. There are numerous publishing organizations that keep track of the references published in a wide range of journals. Probably the best known are the Thomson Reuters Journal Citation Reports and Scopus.

Much of the pioneering work done in 1955 by Eugene Garfield set the scene for citation indexing. Computers had become available, and it became practicable to get lists of publications, and lists of all of the citations (references at the end of each article), and to link them together, and derive some interesting statistics from the links.

Of course, there are many ways of ranking the scientific output of a researcher. Probably the crudest one, used in years gone by, was simply the number of publications. Unfortunately, this incentivizes people to break down their papers into 'least publishable units' instead of more sensible groupings of material. The 'journal impact factor' was something else that Garfield worked on, and this too has its flaws. A more sophisticated approach is to let the worth of a paper be indicated by the number of citations it receives from the author's peers. By this measure, the number of citations indicates the worth of a researcher. Again, this is too crude a measure. If, for example, someone is a co-author on a very highly cited paper, this can skew the impression of worth. So, the number of publications on its own is clearly not enough. Even the average number of citations per paper does not remove the possibility of distortion by one very important paper. Possibly the most widely used measure attempts to combine the number of publications and the number of citations in a single number, a measure that will be discussed in a subsequent section.

When Eugene Garfield introduced the concept of a citation index for the sciences in 1955, he emphasized its several advantages over traditional subject indexing. As a citation index records the references in each article indexed, a search can proceed from a known work of interest to more recently published items that cited that work. Moreover, a search in a citation index, either forward in time or backward through cited references, is both highly efficient and productive because it relies upon the informed judgments of researchers themselves, reflected in the references appended to their papers, rather than the choices of indexing terms by cataloguers who are less familiar with the content of each publication than are the authors. Although it took many years before the Science Citation Index (now the Web of Science) was fully accepted by librarians and the research community, the power of the idea and the utility of its implementation could not be denied (King and Pendlebury, 2013).

Thomson Scientific's Institute for Scientific Information (ISI) has three citation databases: the Science Citation Index, the Social Science Citation Index, and the Arts and Humanities Citation Index. By 2015, there were 45 million documents indexed in the Web of Science.

 

Journal impact factor

The significance of a journal is widely measured by its 'impact factor'. This represents the average number of times that each article is cited in a year (averaged over two years).

Eugene Garfield first introduced the concept of a 'journal impact factor' in 1955, when he was director of the Institute for Scientific Information (ISI) - now part of Thomson Reuters, an information-services company based in New York. It can be inferred from estimates of the number of journal papers published (Björk et al., 2009) that about 70 per cent of refereed scientific journal papers are indexed, even though less than 39 per cent of refereed journals are indexed (which means that more than 15 000 refereed journals are not indexed). The journal impact factor is defined as the ratio of [the number of citations in the current year to the items published in the previous two years] to [the total number of articles published in the same two years]. For example, JIF (2014) = [number of citations in 2014 to articles published in 2012 and 2013] / [total number of articles published in 2012 and 2013]. This number is meant to give a sense of how many citations are received, on average, by the typical paper published in that particular journal.

 

 

The journal impact factor cannot be used meaningfully to compare journals in different fields. For example, biological journals receive orders of magnitude more citations than those in engineering. The journal impact factor is widely criticised for use as a lazy proxy for the quality of a particular paper. If you want to know what the quality of a particular paper is, then read it and judge it on its own merits. The intrinsic merit of a paper is much more important than where it is published. The relevance and quality of a paper are much more important than the impact factor of the journal in which it is published.

Because authors are driven to chase the recognition that supposedly comes from publishing in highly ranked journals, this can lead to some serious problems. Publications in high-ranking journals are not only more likely to be fraudulent than articles in lower ranking journals, but also more likely to present discoveries which are less reliable (i.e., are inflated, or cannot subsequently be replicated). Some of the sociological mechanisms behind these correlations have been well documented, such as pressure to publish (preferably positive results in high-ranking journals), which leads to the potential for decreased ethical standards. (Brembs et al., 2013).

Critics of the status quo object to evaluating research on the basis of where it is published. The shorthand way to do this is by the journal impact factor - an index now kept by Thomson Reuters. In December 2012, hundreds of scientific leaders, funding bodies, journals (including Science, but not Nature) and other organizations gathered in San Francisco to sign the Declaration on Research Assessment (DORA), which criticizes reliance on the impact factor and commits signatories to evaluate research on the basis of its scientific merit. It is important to stimulate a scientific culture and implement policy measures that shift the competition from quantity to quality - that is, to stimulate individual researchers to reduce the quantity and increase the quality of their output so that a larger fraction of the published literature is worth reading.

In South Africa, a large source of university income accrues from a government subsidy received from the Department of Higher Education and Training for academic publications in what it calls accredited journals. In 2011, for instance, the government allocated R2.2 billion to universities for their research outputs. For each article that appears in one of these journals the department remits about R120 000 to the university at which the academic author is employed (Thomas, 2015). This means that academics are pressurised to publish prolifically and to increase their subsidy income. The unintended consequence of this subsidy system is that it encourages an unnecessary proliferation of papers.

 

The h-index

The prominence of a particular author can be gauged to some extent by the number of his or her publications that are cited many times. One of the most widely used measures of scientific influence today is the so-called 'h-index', which reduces this influence to a single number. The index was proposed in 2005 by Jorge E. Hirsch, a physics professor at the University of California, San Diego (Hirsch, 2005). The h-index takes into account both the number of citations and the number of papers that an author has to his/her name. The measure of impact and the quantity of scientific output are expressed as a single number, h, that stands for Hirsch or highly cited. Hirsch defines the h-index as the maximum number of an author's papers that have been cited at least h times. For example, having an h-index of 13 means that the author has 13 papers that have each been referenced 13 or more times. This avoids difficulties where the total number of papers does not indicate the quality of scientific publications, whereas citation counts can be disproportionately affected by a single publication of major influence. The index is a useful comparison between authors within a specific subject area. Care should be taken not to compare a young scientist with someone at the end of a long career, as the h-index increases over time. People with a high h-index tend to be older and well established in their fields. However, on a like-for-like basis, it is a good measure of productivity.

In order to determine an author's h-index, a curve can be plotted of the number of citations versus paper number, with papers numbered in order of decreasing citations (Hirsch, 2005). The intersection of the 45° line with the curve indicates the h value, as shown in Figure 15.

Google Scholar is able to generate h-index values for any published scientist. For example, Albert Einstein was listed as having an h-index of 105 (Google Scholar, 2015). This means that he has 105 publications that have been cited 105 times or more.

 

 

 

 

 

Google and PageRank

The Google search engine ranks the importance of a document in a search according to the networked importance of the pages that link to it. If a paper is referred (linked) to by an 'important' paper (or page on a website), then some of that importance is conferred on the paper to which it is linked. This iterative networked calculation forms the heart of the extremely effective search engine that millions of people around the world rely on. The PageRank algorithm (Page et al., 1998; Brin and Page, 1998) was developed by Larry Page and Sergey Brin, the founders of Google. PageRank (named after Page) was influenced by Eugene Garfield's work on citation analysis at the University of Pennsylvania from the 1950s. In their original paper (Page et al., 1998), Garfield is referenced. Page's and Brin's breakthrough was to create an algorithm that manages to take into account both the number of links into a particular site and the number of links into each of the linking sites. This mirrored the rough approach of academic citation-counting, and worked exceptionally well.

 

Benford's Law

Benford's Law is an interesting and useful mathematical technique that can be used in the detection of fraudulent data in scientific publications (as well as for investigations of accounting fraud). It provides a fascinating insight into the patterns around the first (leftmost) digit of a series of numbers.

Simon Newcomb, a Canadian mathematician, noticed that, when using his book of log tables, the earlier pages (which contained numbers that start with 1) were much more worn than the other pages. He documented some of the implications of this discovery in 1881 in a paper entitled 'Note on the Frequency of Use of the Different Digits in Natural Numbers' (Newcomb, 1881). The physicist Frank Benford rediscovered this observation in 1938 and published an article called 'The Law of Anomalous Numbers' (Benford, 1938). As yet another example of Stigler's Law, this curious pattern is named Benford's Law.

Benford's Law is perhaps best illustrated by means of an example. Imagine having a list of randomly occurring measurements, for example, the altitude (in metres) of the 122 000 most highly populated towns in the world. It is easy to imagine that the leftmost digit of each number would be evenly distributed between the numerals 1-9, which results in an even frequency distribution of about 11 per cent each. This is not the case in reality, however. Even more intriguing is the fact that the distribution remains approximately the same even if the units are changed from metres, to feet, or to cubits (where 1 cubit = 523 mm), as shown in Figure 17 (DataGenetics, 2013). The numeral 1 is much more prevalent as the first digit of many lists of numbers, and there is a decreasing frequency of the remaining digits 2-9.

 

 

 

 

It has been found that this same pattern applies to a wide variety of data sets, including stock market volume, distances to stars, electricity bills, street addresses, stock prices, population numbers, death rates, lengths of rivers, physical and mathematical constants, and Fibonacci numbers. They all follow this pattern of having the first digit being governed by a logarithmic distribution. Benford's Law applies to processes described by power laws (which are very common in nature). It is likely to be best satisfied to a high level of accuracy when values span several orders of magnitude rather uniformly (for example, populations of villages / towns / cities, or stock-market prices). On the other hand, a distribution that is mostly or entirely within one order of magnitude (for example, heights of human adults, or IQ scores) is unlikely to satisfy Benford's Law very accurately, or at all. Where the distribution of first digits of a data set is scale-invariant (or independent of the units that the data are expressed in), the distribution of first digits is always given by Benford's Law.

Benford's Law is very useful in the detection of fraudulent data, because most perpetrators of fraud are not aware of this peculiar pattern so they typically create an even distribution of first digits in their concocted data.

The underlying premise of Benford's Law is that the subject population of quantities, expressed in the base 10 and more or less arbitrary units, is fairly evenly distributed on a logarithmic scale. Benford's formula states that the probability of the leading digit (d) being of a certain value can be described by the following function:

Pr(rf) =log₁₀(d +1)- log₁₀(d)

This simplifies to:

Pr (d) = log₁₀(l+ (1/d))

A good visual explanation for the derivation of these equations can be found elsewhere (DataGenetics, 2013). One helpful picture is to imagine a plant growing where it doubles in size every week. To go from length 1.0 to 2.0 it would have to increase by 100% (and would have a value that begins with 1 for a whole week), but to go from 2.0 to 3.0 it would need to grow only 50% (and so would have a value that starts with 2 for much less time), and so on. This follows a logarithmic scale.

Series of data that are made up by multiplication (multiplicative fluctuations) tend to be well described by Benford's Law. More technically, the central limit theorem says that multiplying more and more random variables will create a lognormal distribution with larger and larger variance, so eventually it covers many orders of magnitude almost uniformly. However, series of data that are generated by addition (additive fluctuations) do not lead to Benford's Law - instead they lead to normal probability distributions (again by the central limit theorem), which do not satisfy Benford's Law.

Some well-known infinite integer sequences provably satisfy Benford's Law exactly (in the asymptotic limit as more and more terms of the sequence are included). Among these are the Fibonacci numbers.

This allows us to close the circle of the story by saying that the advent of computers and electronic publishing, and computers and mathematics, all come together to help us, we hope, to find better ways of getting closer and closer to the truth.

 

Conclusions

Scientific publishing remains alive and well, despite some problems and challenges. The publication of about two million scientific papers per year is a very important component of the advancement of our understanding of the truth about the world in which we live. Electronic communications technology provides some wonderful opportunities to improve the way we communicate scientific results more openly. The approaches outlined here should help us to get closer to the truth. We would do well to remember, in discussions about subjects such as anthropogenic climate change or the benefits of a low-carbohydrate diet, some of the principles espoused here.

 

Acknowledgements

This paper is published with the permission of Mintek. Helpful discussions with Paul den Hoed and some early ideas from Sarah Jones are gratefully acknowledged.

 

References

Aad, G. et al. (ATLAS Collaboration, CMS Collaboration). 2015. Combined Measurement of the Higgs Boson Mass in pp Collisions at sv =7 and 8 TeV with the ATLAS and CMS Experiments. Physical Review Letters, American Physical Society, vol. 114, 191803, 14 May 2015. http://journals.aps.org/prl/pdf/10.1103/PhysRevLett.114.191803        [ Links ]

Abbott, E.A. 1884. Flatland: A romance of many dimensions, by A Square. Seeley & Co., London. 6th edn (1952), Dover Publications, New York. https://archive.org/details/flatlandromanceo00abbouoft        [ Links ]

Agricola, G. 1556. De Re Metallica. Hoover, H.C. and Hoover, L.H. (trans.). Dover Publications, New York, 1950. http://www.gutenberg.org/ebooks/38015        [ Links ]

Babbage, C. 1830. Reflections on the decline of science in England, and on some of its causes. B. Fellowes, London. http://www.gutenberg.org/files/1216/1216-h/1216-h.htm        [ Links ]

Bacon, F. 1869. Novum Organum, Vol. VIII. The Works of Francis Bacon. Spedding, J., Ellis, R.L., and Heath, D.D. (eds.). Houghton Mifflin, New York.         [ Links ]

Benford, F. 1938. The law of anomalous numbers. Proceedings of the American Philosophical Society, vol. 78, no. 4, March 1938. pp. 551-572.         [ Links ]

Berners-Lee, T.J. 2000. Weaving the Web. HarperCollins, New York.         [ Links ]

Berry, M.V., Brunner, N., Popescu, S., and Shukla, P. 2011. Can apparent superluminal neutrino speeds be explained as a quantum weak measurement? Journal of Physics A: Mathematical and Theoretical, vol. 44, no. 49, 492001. 5 pp. http://stacks.iop.org/JPhysA/44/492001        [ Links ]

Bettel, W. 1894. The President's Inaugural Address. Chemical and Metallurgical Society of South Africa, Johannesburg, 19 May 1894. pp. 9-11.         [ Links ]

Bettel, W. 1895. Monthly meeting, Chemical and Metallurgical Society of South Africa, Johannesburg, 16 February 1895. p. 93.         [ Links ]

Biringuccio, V. 1540. The Pyrotechnia of Vannoccio Biringuccio: The Classic Sixteenth-Century Treatise on Metals and Metallurgy. Smith, C.S. and Gnudi, M.T. (trans.). Dover Publications, New York, 1990.         [ Links ]

Björk, B-C., Roos, A., and Lauri, M. 2009. Scientific journal publishing: yearly volume and open access availability. Information Research, vol. 14, no. 1, March 2009. paper 391. http://InformationR.net/ir/14-1/paper391.html        [ Links ]

Bohannon, J. 2013. Who's afraid of peer review? Science, vol. 342, no. 6154, 4 October 2013. pp. 60-65. http://www.sciencemag.org/content/342/6154/60.full        [ Links ]

Borrell, B. 2010. Nature rejects Krebs's paper, 1937. The Scientist, 1 March 2010. http://www.the-scientist.com/?articles.view/articleNo/28819/title/Nature-rejects-Krebs-s-paper--1937        [ Links ]

Bowman B. 1994. Properties of arcs in DC furnaces. Proceedings of the 52nd Electric Furnace Conference, Nashville, Tennessee, 13-16 November 1994. Iron and Steel Society, Warrendale, PA, USA. pp. 111-120.         [ Links ]

Bowman B. and Krüger K. 2009. Arc Furnace Physics. Stahleisen Communications, Düsseldorf, Germany. ISBN 978-3-514-00768-0.         [ Links ]

Brembs, B., Button, K., and Munafò, M. 2013. Deep impact: unintended consequences of journal rank. Frontiers in Human Neuroscience, 24 June 2013. http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291/full        [ Links ]

Brewster, D. 1855. Memoirs of the Life, Writings, and Discoveries of Sir Isaac Newton. Edinburgh and London, 1855. Volume II, Chapter xxvii.         [ Links ]

Brin, S. and Page, L. 1998. The anatomy of a large-scale hypertextual web search engine. 20 pp. http://infolab.stanford.edu/pub/papers/google.pdf        [ Links ]

Campanário, J.M. 1995. Commentary on influential books and journal articles initially rejected because of negative referees' evaluations, Science Communication, 65, March 1995, vol. 16, no. 3. pp. 304-325.         [ Links ]

Campanário, J.M. 1996. Have referees rejected some of the most-cited articles of all times? Journal of the American Society for Information Science, vol. 47, no. 4, April 1996. pp. 302-310.         [ Links ]

Campanário, J.M. 1998. Fraud: retracted articles are still being cited, Nature, vol. 408. p. 288.         [ Links ]

Campanário, J.M. 2006. Using textbooks errors to teach physics: examples of specific activities. European Journal of Physics, vol. 27. pp. 975-981.         [ Links ]

Castelvecchi, D. 2015. Physics paper sets record with more than 5,000 authors. Nature News, 15 May 2015. http://www.nature.com/news/physics-paper-sets-record-with-more-than-5-000-authors-1.17567        [ Links ]

Darwin, C. 1871. The Descent of Man. D. Appleton and Co., New York, p. 4.         [ Links ]

DataGenetics. 2013. Benford's Law. http://datagenetics.com/blog/march52012/index.html        [ Links ]

De Ruiter, J.P. 2014. How anonymous peer review fails to do its job and damages science. Open Science Collaboration, 15 May 2014. http://osc.centerforopenscience.org/2014/05/15/anonymous-peer-review/        [ Links ]

Drubin, D.G. 2011. Any jackass can trash a manuscript, but it takes good scholarship to create one (how MBoC promotes civil and constructive peer review). Molecular Biology of the Cell, vol. 22, no. 5, 1 March 2011. pp. 525-527. http://www.molbiolcell.org/content/22/5/525.full        [ Links ]

Economist. 2013. Unreliable research: Trouble at the lab. Economist, 19 October 2013. http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble        [ Links ]

Einstein, A. 1949. Remarks concerning the essays brought together in this cooperative volume. Albert Einstein, Philosopher-Scientist. Schilpp, P.A. (ed.). Library of Living Philosophers, Evanston, Illinois. pp. 665-688, quotation on p. 684.         [ Links ]

Garfield, E. 1955. Citation indexes for science: a new dimension in documentation through association of ideas. Science, vol. 122, no. 3159. pp. 108-111.         [ Links ]

Garfield, E. 1998. I Had a Dream ... about Uncitedness. The Scientist, vol. 12, no. 14, 6 July 1998. p. 10. http://garfield.library.upenn.edu/commentaries/tsv12%2814%29p10y19980706.pdf        [ Links ]

Garfield, E. 2006. The history and meaning of the journal impact factor. Journal of the American Medical Association, vol. 295. pp. 90-93.         [ Links ]

Google Scholar. 2015. Albert Einstein. https://scholar.google.com/citations?user=qc6CJjYAAAAJ&hl=en&oi=ao [Accessed 11 August 2015].         [ Links ]

Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences USA, vol. 102, no. 46. pp. 16569-16572. http://arxiv.org/pdf/physics/0508025v5.pdf        [ Links ]

Indian Journal of Dermatology editors. 2015. Development of a guideline to approach plagiarism in Indian scenario: Retraction. Indian Journal of Dermatology, vol. 60, no. 2. p. 210. Article: http://www.e-ijd.org/text.asp?2014/59/5/473/139879 Retraction: http://www.e-ijd.org/text.asp?2015/60/2/210/152545        [ Links ]

Jones, R.T. 2012. Open access to metallurgical publications. Journal of the Southern African Institute of Mining and Metallurgy, vol. 112, no. 11, November 2012. pp. 999-1003. http://www.saimm.co.za/Journal/v112n11p999.pdf        [ Links ]

King, C. and Pendlebury, D.A. 2013. Research Fronts 2013, Thomson Reuters. http://garfield.library.upenn.edu/papers/pendleburykingresearchfronts2013.pdf        [ Links ]

Kruger, J. and Dunning, D. 1999. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, vol. 77, no. 6, December 1999. pp. 1121-1134. http://www.jerwoodno.org.uk/pdf/Dunning%20Kruger.pdf        [ Links ]

Kuhn, T.S. 1996. The Structure of Scientific Revolutions. 3rd edn (1962, 1970, 1996). University of Chicago Press, Chicago and London.         [ Links ]

Merton, R.K. 1957. Priorities in scientific discovery: a chapter in the sociology of science. American Sociological Review, vol. 22, no. 6, December 1957. pp. 635-659. http://www.jstor.org/stable/2089193        [ Links ]

New Atlantis editors. 2006. Rethinking peer review: how the Internet is changing science journals. The New Atlantis, no. 13, Summer 2006. pp. 106-110. http://www.thenewatlantis.com/publications/rethinking-peer-review        [ Links ]

Newcomb, S. 1881. Note on the frequency of use of the different digits in natural numbers. American Journal of Mathematics, vol. 4, no. 1. pp. 39-40. http://www.uvm.edu/~pdodds/files/papers/others/1881/newcomb1881a.pdf        [ Links ]

Page, L., Brin, S., Motwani, R., and Winograd, T. 1998. The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford Digital Library Technologies Project, 29 January 1998. 17 pp. http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf        [ Links ]

Planck, M. 1949. Scientific Autobiography and Other Papers. Gaynor, F. (trans.). New York. pp. 33-34 (as cited in Kuhn, T.S. The Structure of Scientific Revolutions). From Wissenschaftliche Selbstbiographie. Mit einem Bildnis und der von Max von Laue gehaltenen Traueransprache. Johann Ambrosius Barth Verlag, Leipzig, 1948.         [ Links ]

Popper, K.R. 1959. The Logic of Scientific Discovery. Routledge. [1934 German version, Logik der Forschung, published by Mohr Siebeck] http://cw.routledge.com/textbooks/popper/

Price, D.J. de Solla. 1963. Little science, Big science. Reprinted in Little science, Big science ... and Beyond. Columbia University Press, New York, 1986. http://wiww.andreasaltelli.eu/file/repository/Little_science_big_science_and_beyond.pdf        [ Links ]

Reich, E.S. 2013. Science publishing: The golden club. Nature, vol. 502, no. 7471. pp. 291-293. http://www.nature.com/news/science-publishing-the-golden-club-1.13951?WT.ec_id=NEWS-20131022        [ Links ]

Rekdal, O.B. 2014. Academic urban legends. Social Studies of Science, vol. 44, no. 4. pp. 638-654. http://sss.sagepub.com/content/44/4/638.full.pdf+html        [ Links ]

Retraction Watch. 2012. South Korean plant compound researcher faked email addresses so he could review his own studies. 24 August 2012. http://retractionwatch.com/2012/08/24/korean-plant-compound-researcher-faked-email-addresses-so-he-could-review-his-own-studies/        [ Links ]

Retraction Watch. 2015. You can't make this stuff up: Plagiarism guideline paper retracted for...plagiarism. 1 April 2015. http://retractionwatch.com/2015/04/01/you-cant-make-this-stuff-up-plagiarism-guideline-paper-retracted-for-plagiarism/        [ Links ]

Royal Society. 2011. Royal Society journal archive made permanently free to access, 26 October 2011. http://royalsociety.org/news/Royal-Society-journal-archive-made-permanently-free-to-access/        [ Links ]

Royal Society. 2012. Royal Society Publishing, Open Access publishing. http://royalsocietypublishing.org/site/authors/EXiS.xhtml [Accessed 16 May 2012].         [ Links ]

Schagrin, M.L. 1963. Resistance to Ohm's Law. American Journal of Physics, vol. 31, no.7, July 1963. pp. 536-547.         [ Links ]

Schlunde, Mr. 1895. Monthly meeting, Chemical and Metallurgical Society of South Africa, Johannesburg, 18 May 1895. p. 125.         [ Links ]

Schuman, R. 2014. Revise and Resubmit! Peer review is slow. It's unhelpful. It's generally awful. Here's how to fix it. Slate, 15 July 2014. http://www.slate.com/articles/life/education/2014/07/the_easy_way_to_fix_peer_review_require _submitters_to_review_first.html        [ Links ]

Schwartz, C.A. 1997. The rise and fall of uncitedness, College & Research Libraries, vol. 58, no. 1, January 1997. pp. 19-29. http://crl.acrl.org/content/58/1/19.full.pdf        [ Links ]

Shakespeare, W. 1623. As You Like It, Act 5 Scene 1.         [ Links ]

Shatz, D. 2004. Peer Review: A Critical Inquiry. Rowman & Littlefield, Lanham, Maryland, USA. 249 pp. https://books.google.co.za/books?id=Jsde5IB11mgC        [ Links ]

Slavov, N. 2014. Papers that triumphed over their rejections. 15 August 2014. https://majesticforest.wordpress.com/2014/08/15/papers-that-triumphed-over-their-rejections/        [ Links ]

Slavov, N. 2015. Science and failures of authority. 22 July 2015. https://majesticforest.wordpress.com/2015/07/22/science-and-failures-of-authority/        [ Links ]

Sloof, P., Van den Burg, J., Voogd, A., and Benne, R. 1987. The nucleotide sequence of a 3.2 kb segment of mitochondrial maxicircle DNA from Crithidia fasciculata containing the gene for cytochrome oxidase subunit III, the N-terminal part of the apocytochrome b gene and a possible frameshift gene; further evidence for the use of unusual initiator triplets in trypanosome mitochondria. Nucleic Acids Research, vol. 15, no. 1. pp. 51-65. http://nar.oxfordjournals.org/content/15/1/51        [ Links ]

Snow, C.P. 1964. The Two Cultures and a Second Look. Cambridge University Press. p. 15.         [ Links ]

Sokal, A.D. 1996a. Transgressing the boundaries: Toward a transformative hermeneutics of quantum gravity. Social Text, vol. 46/47 (Spring/Summer). Duke University Press. pp. 217-252.         [ Links ]

Sokal, A.D. 1996b. A physicist experiments with cultural studies. Lingua Franca, May 1996.         [ Links ]

Sokal, A. and Bricmont, J. 1998. Intellectual Impostures. Profile Books, London.         [ Links ]

Stigler, S.M. 1980. Stigler's law of eponymy. Transactions of the New York Academy of Sciences, vol. 39, no. 1. Series II, April 1980. Science and social structure: afestschrftforRobertK. Merton. Gieryn, T.F. (ed.). Blackwell. pp. 147-158. http://dx.doi.org/10.1111/j.2164-0947.1980.tb02775.x        [ Links ]

Straus, W.L., Jr. 1954. The Great Piltdown Hoax. Science, vol. 119, no. 3087, 26 February 1954, pp. 265-269. http://www.jstor.org/stable/1681349        [ Links ]

Thomas, A. 2015. Cheating academics may make students think plagiarism is acceptable. BusinessTech, 8 August 2015. http://businesstech.co.za/news/general/95403/cheating-academics-may-make-students-think-plagiarism-is-acceptable/        [ Links ]

Upper, D. 1974. The unsuccessful self-treatment of a case of "writer's block". Journal of Applied Behavior Analysis, vol. 7, no. 3, Fall 1974. p. 497. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1311997/        [ Links ]

Van Noorden, R. 2013. Half of 2011 papers now free to read. Nature, vol. 500, 22 August 2013. pp. 386-387. http://www.nature.com/news/half-of-2011-papers-now-free-to-read-1.13577        [ Links ]

Walsh, E., Rooney, M., Appleby, L., and Wilkinson, G. 2000. Open peer review: a randomised controlled trial. British Journal of Psychiatry, vol. 176, no. 1, January 2000. pp. 47-51. http://bjp.rcpsych.org/content/176/1/47.full        [ Links ]

Wasserman, L. 2012. A world without referees. Carnegie Mellon University, 20 February 2012. 4 pp. http://www.stat.cmu.edu/~larry/Peer-Review.pdf        [ Links ]

Whitfield, J. 2011. Open access comes of age. Nature, vol. 474, 21 June 2011. p. 428. http://www.nature.com/news/2011/110621/full/474428a.html         [ Links ]