David Eppstein has mostly good experiences with rebuttal experiment in SoCG (top conference in computational geometry). Briefly, after the initial review period, authors are given 1 week time to answer to reviewer’s questions. After that, in a few weeks, a final decision is made which papers are to be accepted.
I personally expect that if handled correctly, such rebuttals would help tremendously in the reviewing process. I’ve just too many times received reviews that could be rebutted with a single paragraph, and may be I’ve written some reviews like that myself. I am a strong supporter of rebuttals, and I hope they will be implemented in at least top conferences in cryptography too.
Advertisement
August 11, 2011 at 07:39 |
Nice that you raised the subject.
I think, I very much share your opinion on this issue of rebuttal system. If not for anything else, I think, it is, at least, important to restore public confidence to the conference review process, that I think generates, more often than not, a lot of vitriol from the authors who do not agree with the review. I’m not a prolific submitter of papers, still there are times when the review reports gave me the feeling that my work was not reviewed equitably. There are others too who feel the same way. In any case, whoever is at fault, the authors should be given a chance to defend their standpoints. How could one expect that a system would function properly, if one major party — the referees, perhaps the key players responsible for the success of any conference — are not adequately accountable for their actions? At the same time, it would be an enriching experience for the reviewers too. This is not to make any general allegation against the review process; the point is that, despite their sincere efforts, the reviewers can make really silly mistakes to their own embarrassment. To avoid this unwanted scenario, a rebuttal system, I don’t think the community would dismiss outright, like they did not reject the system of “anonymous submission” in IACR conferences against a significant opposition.
I usually voice my disagreements through my weblog, and I know that many people do the same way. I have also seen comments in IACR eprint archive where the authors refuted the points on which their papers were rejected at so and so conferences. But what made me more vocal about this issue this time is that I recently reviewed a paper that was previously accepted at a cryptography conference; it would not be any exaggeration to say “a cryptography conference of repute”. To my great surprise, I found that the core claim of the paper was fatally flawed and that there is no easy way to fix it, rather than rewriting it by completely changing the line of thinking. What made me more nervous is the fact that I need to be extra confident to give negative remarks; as such, my any negative review directly pits me against a slew of experts who tagged it already “acceptable”. Anyway, it was a relief for me to see finally that other reviewers agreed on the issue.
The bottom-line is: The authors have a lot of emotion at stake in the review process. The rebuttal system would certainly improve the whole process. I do not see any practical problem in implementing a rebuttal system, like allowing just a week for the authors to respond; more so, when there is already a precedent for that.