Our Modern obsession with measurement - the case of publish or perish

Preamble

I wrote the following post last November (2006) on my personal blog. I am repeating it here because the rise of citation indexes and of the term "publish or perish" is an example of a broader trend in Australian and New Zealand thought, our obsession with measurement.

We can see this obsession across all aspects of life. It appears in public policy in the input/output/outcome model. It appears in terms such as key performance indicators. We can see it in concepts such as performance pay. Then there is our obsession with rankings from school exam results to the BRW top lists.

Now my personal view is that this obsession has become quite pernicious. So how did it arise and spread?

Publish or perish and the rise of the citation index is an interesting case study - another modern term - because it shows the process at work in one area.

The Post

This post was meant to continues my notes to myself on changes in public administration since the war (previous post). However, I have been on some weird and wonderful web searches since then giving me enough material for multiple posts.

I have felt for a long time that the seventies were the tip decade marking the break between the previous long continuity of Australian history and the Australia of the present day. Much of my writing on this blog including the earlier migration and current current confessions of a policy adviser series is concerned in one way or another with tracing this process of change.

While all the evidence so far supports my 7os tip hypothesis, to really make my case I need to trace the pattern of change across different dimensions including changes in the use and meaning of words.

In my last post I spoke of the rise of measurement as an all pervasive mechanism. In this context, one of the things that I was musing about was the changing position of academia, an area that I have known very well over a long period starting as a child in an academic household. This is an area where measurement has become pervasive and indeed arguably even perverse.

To try to provide a measure of this (I, too, like measurement!), one of the first things that I thought of were citation indexes.

In the words of the University of Southern Queensland Library (I do like quoting Australia's universities headquartered in regional areas!):

Citation Indexes are compilations of all the cited references from particular groups of journal articles published during a particular year or group of years. In a citation index, you look up a reference to a work that you know to find journal articles that have cited it, although you can also search by concepts and authors. Cited reference searching is a fast and efficient way of finding journal articles that relate to your research.

Citation indexes can be useful. Again to quote the USQ Library:

While most users think of indexes as merely a means of recovering information, index compilers have long been aware that indexes--especially those for scientific literature--serve a vastly more important and often unstated purpose. That is, they reveal connections between ideas or concepts that were not considered before.

That is true, but as anybody in management knows, what you measure is what you get. So treating citations as a performance measure means that those being measured have an incentive to maximise citation performance. Here I was interested in the history of citation indexes as a way of assessing impact. To quote Thompson Scientific:

While the first proposal for the SCI (Science Citation Index) was made in 1955, it was in the 1960s that ISI (Institute For Scientific Information) applied for and received a grant from the National Institutes of Health to create a Genetics Citation Index. As a result of this multidisciplinary project, we published the first SCI covering the 1961 literature) in 1963. We then went on to launch a quarterly service that eventually proved successful. Since then we have indexed the literature back to 1945 and now the CD-ROM version of SCI is updated monthly and the online and magnetic tape versions are updated weekly.

In 1973, we made SSCI® available. It now covers the social sciences literature back to 1956. The index deals with topics such as anthropology, economics, sociology, educational research, and information sciences among other fields. A&HCI® was introduced in 1978. A&HCI provides access to disciplines as varied as archaeology, linguistics, philosophy, musicology, literature, and others in the arts and humanities.

Advances in modes of access have also been made over the years. In 1974, the SCI became one of the first large-scale databases available online via DIALOG. Other ISI databases followed. In 1988, SCI (and later SSCI and A&HCI) became available on CD-ROM. This new technology and increased data storage capabilities enabled us to implement a variety of access and browse features unique to our citation-based searching. Enhancing the power of citation searching through bibliographic coupling, you can navigate the literature by exploring papers that share one or more references.

If we look at this history, we can see that the rise of the citation index paralleled the rise of interest in measurement, going on-line in the seventies as the new computing and communications technologies became available.

By the time I became CEO of the Royal Australian (now Australian and New Zealand) College of Ophthalmologists (RANZCO) at the end of 1997 the citation system was very well entrenched.

At the time we were worried that the College's scientific journal, the Australian and New Zealand Journal of Ophthalmology, was dropping down the citation list. The journal really needed to be in the top ten globally to attract the required level of scientific and clinical articles, so under the leadership of the editorial team the journal was renamed Clinical and Experimental Ophthalmology and effectively relaunched to achieve the required citation level.

This material does provide a guide as to the history of citation indexes and some indication as to their impact, but more is required. So I decided to look at the term publish or perish, now a very common term in academia.

While I found many references, it proved very hard to trace the origin of the term. The only on-line dictionary reference I could find (Source) suggested a mid nineties date. I knew that this could not be right because I was using the term in the eighties, so I continued searching. Finally I found an article by Eugene Garfield, What Is The Primordial Reference For The Phrase' Publish Or Perish'? (The Scientist, Vol:10, #12, p.11 , June 10, 1996.)

Eugene's work traced the term to at least a 1942 book by Logan Wilson with the same connotations as today. So the term has been around for a while at least in the US. But we can also be reasonably sure of is that the term was not common in the sixties since I did not hear reference to it, but was being referred to in the eighties. So like citation indexes, its rise does parallel the rise of the measurement movement.