[Home]WikiSciencePublication

MeatballWiki | RecentChanges | Random Page | Indices | Categories
This is a continuation of a discusion started at WikiAsScience.

Helmut- I agree that there are reasons for the cultural momentum that maintains current practices in academic science. This is why I think it is the case that change will come at the fringe of science where there are people who want to create the best system for scientific publishing rather than simply use the existing system. Imagine a world with three life forms, a seedless grass that spreads by asexual cloning of grass plants (by a system of self-rooting runners), cows that eat the grass, and humans who cannot digest grass and so their only source of food is cows that can eat the grass. Eventually, anone who controls the best grazing land is going to have vast power in such a society. People will try to graze as many cows as possible, creating a TragedyOfTheCommons (analogy: publishing as many LeastPublishableUnits as possible).

How would it be possible for this tribe of cow-eaters to develop new food resources? Let us imagine that some keen-eyed herdsman noticed a new kind of plant, a grass that reproduces by seeds. At first, nobody knows what a seed is, but eventually it is learned that a seed allows this new type of grass to reproduce. Then it is learned that the grass seeds can be processed and eaten by humans. Then it is learned that if humans would alter their diet to include grass seed in their diet, then the carrying capacity of the world for humans could be vastly increased. But how does this superior system of feeding the human population ever become established? Someone would probably have to go to marginal land and start growing a bit of the seed-producing grass.

This is the situation we are in today with respect to wiki and scientific publishing. The established system of scientific publishing and peer review works (the self-supporting clockwork that you describe) and the current system is a cultural artifact that is almost impossible to modify. Somewhere at the fringe of science, someone will start using wiki publishing for science publishing. My guess is, after much resistance from the established system, a new wiki-based system will slowly be validated and will eventually grow into a viable alternative to the current system. For people in the current system, there is no point in developing the new system and incentives actually exist to sabotage and work against the new system. What is best for science has very little influence on the dynamics of this interaction between the existing system of science and the new possible wiki-based system. The dynamics are governed by cultural momentum and the natural energy barrier that slows the development of any new idea or system.

ChangingPapers?. These have started to appear within the conventional scientific system. Electronic publishing has made possible systems such as Science magazine’s [Signal Transduction Knowledge Environment] which includes frequently updated resources that summarize signal transduction pathways. Of course, STKE is run as a for-profit enterprise by an established science journal publisher, but similar science resources could be open and public and exist in a wiki environment. There are annotation databases such as [Fly Base] that could easily be converted to wiki format. ChangingPapers? in a wiki environment would not erase any of the history of science. With a good PageHistory system, you could always trace back to earlier versions of wiki publications.

KnowledgePapers? (reviews, tutorials, [books] (gasp! Who has time to write a book?)). Some scientific publications, even if published in wiki format would not ever be modified. Some primary research articles are works of art that stand for all time. At the other extreme are review articles that could be constantly updated by their authors. In wiki publishing, these reviews could be true community efforts with hundreds of contributors. Wiki has the flexibility to allow the required level of future modification that is appropriate for each publication. Many authors would use a wiki interface to correct errors in articles and add pointers to new developments described in new articles that are published at later times. Some authors would invite their peers to add such annotations to their articles. Many science publications in a wiki environment could stay fixed in their original content but “grow” through the comments, reviews, and other additions that are linked to the original article. The point is to provide maximum flexibility and let scientists decide what features they want to use as is appropriate for each situation.

OpenReview?. I basically agree with your (Helmut at WikiAsScience) analysis of what drives the existing system of scientific peer review. The current system provides benefits to reviewers that they exploite for personal advantages that are well worth the time invested, particularly since secret reviews are often shoddy hatchet jobs that take little time to produce. If the peer review system were made totally open and totally transparent, each review would have to be a detailed analysis of the scientific merits of the reviewed article. If such a system existed, then those people who are really interested in the science content of a publication would soon dominate the peer review process. An open and public peer review process would be forced to center on the true scientific purpose of peer review, using knowledge distributed in the community to critique the content of publications. In an open and transparent review process, scientists who see no personal benefit in spending time performing peer review would be free to exert themselves elsewhere. Who needs such self-centered scientists as reviewers, anyhow? Scientists who recognize the importance of peer review as a key element in the process of science will participate. There would be a new kind of open market for peer review and peer reviewers. People who write and publish reviews of other scientist's articles would count those reviews as form of secondary publication. There could easily be a new class of scientist who would specialize in doing peer review. If not enough people did peer review voluntarily, there could be a system by which people could get paid for performing this service, but I think there would be adequate rewards in an open peer review system that would naturally encourage peer review for the RIGHT reasons. - JohnSchmidt

John, sorry for thinking so long about that. Basically I agree with your analysis. Sooner or later the scientific publishing system will switch to make full use of new media like wiki, just as a NewBookEconomy will switch to some form of electronic publishing. The example with the grass seeds exists in real, afaik corn was developed that way about 10.000 years ago - but it took a few thousand years to optimize. The question is how much trial and error is needed to construct an alternative system. I think that there are enough big players that could do it with a fair chance to succeed.

I would (just from intuition without much analysis):

Advantages:

The typical problem will be that single players will think that they can create a system on their own (like with e-book formats and readers) and create a kind of monopoly.

-- HelmutLeitner

It's my goal to create an academic-quality wiki. I may not do this inside academia, but with my work with http://grail.oise.utoronto.ca in 2005, I may get close. -- SunirShah

Helmut- I think that the process of science should be open and transparent. I wonder if it will eventually become a part of the culture of science that ALL phases of the scientific process such as funding, research (lab notebooks, raw data), meetings and peer review are open and transparent.

Sunir- I think many people have the idea that it would be useful to start constructing an academic wiki in their specialized field of interest (for example: [Social Science Wiki]. Also see [this list] and [this initiative]). Is there an online summary of your "OISE work"? - John Schmidt

John, I'm a big advocate of transparency, for example as a necessary basis for an efficient democracy. So I should like your idea of a completely transparent scientific process, but I don't because I think it's unreal. My experience is that people only like transparency when they can use it to their advantage - but not when they are urged to become transparent for others. As long as scientists compete for funding, ideas, publishing and status (and there is no reason that this will change, even in the context of a new science publishing system) they will want to make a secret of what they are currently working at or pondering about. An open lab, open raw data are so far from scientific reality (and the vision of any future scientific reality) that this is almost unthinkable. -- HelmutLeitner

No sense being defeatest. That isn't being creative enough. The solution is to compel scientists to be open about their data. As the world moves towards release early, release often, it seems more likely the case that people will have to publish quickly in order to gain attention, traction, and credibility. Of course, only certain work is amenable to release early, release often--those that require rapid feedback.

The main difference between older publications and new publications is that the old ones are arguments in the old Academy style of the Greeks (ladders to God, in the protestant view), whereas newer work are project-based. Projects require constant and rapid feedback in order to succeed, hence they have a need to publish quickly. They also require many, diverse hands to solve the problems we're faced, so attracting help is critical. Moreover, since projects are about overcoming obstacles, feedback can be helpful and constructive (problem solving). The network age is a project-based age; a cybernetic age in a literal sense of the word cybernetic.

Arguments, on the other hand, are very weak. They can be attacked at any point and the entire argument may crumble. They are also genenally individualistic, as anyone who reads arguments knows, it's joyful to develop your own argument, but to follow someone else's train of thought is incredibly difficult. They are also adversarial in structure, unlike projects, so feedback is not generally helpful nor constructive, and thus unsolicited except by trusted friends.

However, even arguments can be turned into network productions. Arguments can be built lemma on lemma. It's possible to just publish small lemmas rather than complete exegeses. This is similar to a PatternLanguage, if you think about it. So, there is hope for even 'old school' Academics (pun intended). -- SunirShah

I think I understand the motivations and forces that currently control human behavior within the cultures of science and academia. I agree that there is no simple way to transform those cultures into something radically different. Any progress towards greater openness in science will have to come at the margins of the existing system. In thinking about how such change might start, I have started to shift my attention from the traditional motivations of individual academicians and the entrenched cultures of brick-and-mortar academic institutions to new types of science-related projects that might be made possible by the internet and information tools like wiki.

For example, one problem that the traditional culture of science has really not been able to deal with is the gap between the world of science and the non-scientific world. Many people are interested in science and enjoy the fruits of science, while also fearing the misuse (intentional and unintentional) of science and technology. Maybe it would be possible to establish some wiki-based science and technology demonstration projects that would accomplish two things at the same time:

1) open up the world of science to the public, showing every step of science education, the scientific research process and technology development in a totally open way.

2) serve as pilot projects for demonstrating that science education, research and technology development could be conducted in a totally open and transparent wiki format.

What might be possible types of wiki science pilot project?

1) There are already some computer technology companies that [Wikicities] might serve as demonstration products for the idea of technology development in an open environment. An [OpenSource software development project] with heavy use of a wiki interface might be a good demo project and lead to a transparent technology project with all community activities in a wiki environment.

2) There are some embryonic education [projects in wiki websites] that could use some support for their development.

3) Some research labs are using [wiki websites] to post information about their activities. Such ?vanity? websites might grow into true open laboratories if given some encouragement.

The ?vanity? label for the cited research lab's wiki is inappropriate. The information posted on the wiki is primarily for use by the lab members themselves. The collaborative nature, and easy accessibility and usage, of wikis is [why the website is a wiki] and not a regular website. - a visitor

4) There are some government-related wiki websites thatmake government projects accessible to the public [NASA project-inspired]. These might eventually grow from comercialization and public relations gimics to include the details of how tax dollars are spent.

5) There are some science-related projects in wiki websites such as [EvoWiki] that could evolve into community websites for crafting scientific articles; in this case, dealing with public outreach related to evolution and ?creation science?.

6) While all of the above kinds of projects would start forms of science-related publication in wiki format, there should be some effort specifically crafted to promote wiki science publishing in some sort of [online journal format].

- JohnSchmidt

The traditional science publication relies on the reputation of the author very much. What would you substitute to that in Wiki where the author is deemphasized? You need to know how much trustworthy are your fundamentals to build upon them, and I don't see any mechanism for assuring the trustworthiness of the information beside the reputation of the author and reviewers. -- ZbigniewLukasiak

I agree that peer review is critical for assessing the trustworthiness of the published information. In an open publishing system, people could cooperate to produce community projects that would develop reputations according to the care taken with the peer review process. For example, in biology, there could be a wiki that had as its objective to continually archive everything of importance that is known about a particular protein such as actin. This could be a community wiki with content that comes from all of the world?s experts on actin. The wiki could be open for contributions from anyone in the world. A new author, with no previous reputation for contributing to the literature on actin, could count on their contributions at the ?actin wiki? to receive reviews from experts. If some existing experts quickly marked a new author?s work as questionable or flawed, then the rest of the community would be likely to not trust what is published by that new author.

Within an open publishing system you could have organized ways by which individual authors would be given recognition for their work. I think an open system could do so in ways that would more fairly give recognition to authors and more accurately allow an author?s reputation to grow according to how useful their activities are to peers. In the current academic system, there are many authors in positions of authority who demand ?honorary authorships? on published material that is actually produced by others. In an open academic publication system, I hope there would be total openness about the role of every author in the production of every publication. If all publications were constructed in a wiki environment, the wiki software could keep track of every author?s contribution.

An author?s reputation is built up by how their intellectual activity impacts their peers. To a large extent, this impact can be measured by the utility of what they publish and what their peers say about them. Acknowledgements are one way of rewarding people for their contributions to progress in academia. I think acknowledgements should be used more, and used as a key part of a system within which only true authors are called authors. People who make important contributions to the work reported in a publication, but who did not actually write what is published, can get recognition through acknowledgements. Just as we now have ways of tracking citations, there should be a simple and organized way of tracking acknowledgements and including professional acknowledgements in the reward system of academia. But much reputation construction will continue to be dependent on citations to an author?s work by other authors. As a default system, every time an existing publication (call it Pe) is cited (within wiki publishing, this would mean forming a link from another publication to Pe is published), there should be a way for the citing author to rank Pe. It would probably work to have a three value ranking system:

Positive: this rank would be used to indicate that Pe was important to the citing author for their own work Neutral: this would be the default rank, used when mentioning another publication that is relevant to the citing work Negative: this ranking would be used when the citing author is publishing a description of an error in Pe or some other complain about the content of Pe such as an inability to replicate experimental results reported in Pe.

Over time, an author?s reputation would be built up by the number of times they are acknowledged by other authors for contributions to the efforts of others and by the number of times that their own publications are cited by others in a favorable context. There could be a system of weightings by which not all citations would be equal. A citation from the context of a detailed and carefully written critical appraisal (a true review) should mean more than a throw-away citation made without any explanation. It would also be possible to have a citation weighting system that was an interlocking network within which the reputation of a citing author influenced the reputation of the author being cited: having your work cited favorably by someone with a good reputation could boost your reputation more than having your work cited by someone with a lesser reputation. For giving credit to authors of multi-author publications, a first approximation would be to give each author credit in proportion to the fraction of the publication actually written by each author.

- JohnSchmidt

I think you miss my point. I am not against open process - what I am attacking here is the idea that you can have reputation without enforcing authorship and wiki is weakening individual authorship. We can see it clearly if we focuse on the plain wiki, where every sentence can be altered by anyone. The history of chages does enforce a bit authorship - but it is not enough. You can solve a dispute over who changed what but this is a very heavy weight tool for this, with a published article it is much more obvious - you don't need to analyze history or anything. The difference between heavy weight and light weight tools should be well understood by wiki practitioners as it is the main point in wiki propaganda when comparing it to some Groupware solutions. I think wiki can work well for small teams where people understand each other and know what they can change in other's texts and what would be opposed (this is quite similar to the Agile Programming concept of collective code owhership). When the wiki users don't know personally each other it becomes much more complicated (the most severly touched wiki practice is the refactoring process).

-- ZbigniewLukasiak

There is a new [academic publishing wikicity]. -JohnSchmidt

Sigh. One can create a thousand science publishing wikis. The question is, how large your commitment is and how credible you are as a scientist and a science leader. This way the chance is 99% that it will not unfold. -- HelmutLeitner

See also: SelfArchiving
CategoryAcademia

Discussion

MeatballWiki | RecentChanges | Random Page | Indices | Categories
Edit text of this page | View other revisions
Search: