?> July | 2010 | INFOMATTERS

Monthly Archives: July 2010

Open peer review for publication? The debate continues

This week’s Chronicle of Higher Ed notes an experiment in open peer review of articles by Shakespeare Quarterly was viewed by the authors as most successful. Here, a selection of submitted article drafts was opened for comment by readers and invited reviewers, all of whom were invited to put their name to their comments (though it appears not all did) and the authors received what they largely believed to be constructive comments and citations which improved their drafts. As the editor put it, it was a controlled editorial process just not the typical controlled editorial process. The merits seem to be that authors received more comments than they might have expected from the blind review process that is typical of journals. This, if anything, points to an understated problem in the production cycle of many contemporary journals which results in reviewers offering little detailed feedback (and we’ve all had those!). One suspects that expert reviewers often exploit the blind aspect of the process to cover up their lack of investment in the activity. Having to put your name to your comments seems likely to encourage greater content and (one hopes) more thoughtful reviews too.

The downsides are real. The editorial work is extended, the decision is more open to scrutiny, and it appears from this example at least, the presence of senior reviewers stating their opinion tended to quiet some junior faculty reviewers who were concerned with presenting alternative views. It seems a sad indictment of the tenure system that one might actually pay a price for openly and intelligently holding alternative views but be rewarded by staying quiet. One hopes that is limited to some schools and faculties only.

Interestingly, the Shakespeare Quarterly results were more positive than the earlier findings by Nature which drew similar conclusions about some faculty unwillingness to participate with little on the plus side to balance this difficulty. Nevertheless, there is increasing interest in opening up the process of review and for me, the most practical one is to improve the quality of reviewing which in this field’s leading journals and conferences often leaves something to be desired.

International Tour 2010

I spent much of June traveling to conferences. First, I gave a keynote address at the Hypertext 2010 conference in Toronto where I found a community somewhat under threat by other web research conferences but nevertheless alive and kicking. The organizers had asked me to consider where the field might have gone wrong and where it might go in the future. I prepared for this by examining the last few years proceedings and prodding old colleagues in the field (Herre von Oostendorp at Utrecht, Jean-Francois Rouet at Poitiers, Cliff McKnight at L’boro, among others) to give me their impressions of the recent history of hypertext too. My main message was to argue that we not forget the vision of augmenting human thinking that is a common theme in the work of the holy trinity of hypertext thinkers, Bush, Englebart and Nelson. Interesting responses, and an interesting conference, now in its 21st year. I was reminded by this of the first time I attended CHI in 1988 and walked into a hastily organized meeting of folks interested in talking about hypertext. Ben Shneiderman noted how amazing it was that the room was full (we thought we were a secret group of fanatics) and one attendee later described ‘hypertext’ as the ‘word of the conference’ in ’88. The word may be losing currency, as judged by usage rates in Google Analytics, but the questions of human engagement with structure information is as pertinent as ever. What is great about this conference is that it brings together people interested in all aspects of this technology — from education, design, engineering, and user psychology. Toronto was alive too. Here’s to the next 21 years!

After Toronto I spent 10 days in London, reacquainting myself with a city I’d not visited for over 15 years. It’s changed, and for the better in my view, with the ethnic mixes and modernized licensing laws giving rise to street cafe culture previously missing. London is also a wonderful reminder of the cultural and lifestyle value of decent public transport! The real reason for the trip was a series of LIS related talks at CoLIS 2010, held in UCL, and a keynote address at the newly organized UK LIS Research Coalition Conference at the British Library. It’s an interesting reminder that even in Europe, community divisions are rife. Few people attended both events, despite the shared relevance of these meetings. CoLIS remains, in my view, one of the only LIS conferences where it is possible to seriously think about presentations. Attendance is limited, everyone stays in the same room, and evening events mean that community engagement is assured. The content is thoughtful and sometimes provocative, and one leaves with a sense of refreshment that I find too often is lacking in other conferences. The next one is on 2013 (yes, we don’t want to spoil ourselves by having these too often) in Copenhagen.

The UK LIS Research Coalition put together a one day event emphasizing the issues of demonstrating value and impact in the LIS world. Much like librarians over here, there is increased pressure in the UK to demonstrate return on investment and the financial value of libraries and librarianship. Lots of issues here related to how one demonstrates value, especially for practices and research models that are highly context-dependent. I love the emphasis on data and the concern to answer the perennial ‘so what’ question that plagues LIS research, though I am less convinced than others that the need for evidence-based evaluations in LIS is as great as it apparently appears to be in medicine, but let’s not argue the niggles. If we cannot quickly and clearly articulate examples of how the field adds value, then we have no business claiming intellectual legitimacy.

Good trips, good people, good discussions. If only it was always like this.