In this file are the verbatim referee reports from the ISSAC 2001
program committee chair. There are comments below.
-----------verbatim reports------------
Subject: ISSAC 2001 - Submission #30
Date: Mon, 19 Mar 2001 18:00:10 -0500
From: Gilles Villard
To: fateman@cs.berkeley.edu
CC: Gilles.Villard@ens-lyon.fr
Dear Richard Fateman,
This note is to inform you that your submission
"A critique of openmath and thoughts on encoding mathematics ....."
to ISSAC 2001 has been rejected. We had to take a selection among
very good submissions, tough choices have finally been taken by
the vote of the program committee. The grades given to your paper
by the referees have not been sufficiently high. I include the
reports below.
Sincerely yours,
Gilles Villard, PC Chair.
--------------------
..........................
Fill in Referee-assigned grades below
grades are to be in the range 1..4
(Use: 1, Excellent; 2, Good; 3, Below Average; 4, Poor.)
I. RECOMMENDATIONS
[ ] Excellent quality, definitely should be accepted
[ ] Good quality, acception is recommended
[X] Marginal quality, acception is inappropriate
[ ] Poor quality, rejection is imperative
II. SCOPE
[2] Relevance to symbolic computations
III. EVALUATION OF CONTENTS
[2] Relevance
[3] Originality
[3] Importance
[3] Difficulty
[4] Correctness
[3] Completeness
[4] Literature (Cites previous work correctly?)
IV. EVALUATION OF STYLE AND FORM
[2] Clarity
[3] Introduction (Relevance clearly stated in intro/abstract ?)
[2] English
[2] Length (Optimal, Appropriate, too long/short, unreadable because of
length)
V. COMMENTS TO THE AUTHOR(S)
The author says at the outset, and often repeats, that the OpenMath project
has the implicit aim of solving the software re-use problem. I do not think
this is true: if Openmath solves any re-use problem, it is that of DATA
re-use. This may in turn translate into software re-use, but that is not
the primary goal of OpenMath as a whole (though the software vendors
involved may think differently).
The author says (p. 2) that this could be done in any ``language capable of
representing attributed trees''. This is true, and XML is such a language.
The real question is ``what should the attributes be''?
On page 3, the author points out, quite correctly, that much mathematical
discourse is (unintentionally) ambiguous. Indeed, I would go further, and
say that there is deliberate, but unfortunate, re-use, e.g. in $\sin^{-1}$.
He then claims that one neds to be skilled in the discipline. The OpenMath
project discovered (in a paper unfortunately not in this collection) that
knowing the Mathematics Subject Classification (often quite coarsely)
solved a great many ambiguities --- e.g. $\pi$ in Group Theory is much more
likely to be a permutation than 3.14159.
It is certainly possible, as the author does (page 4) to believe that
MathML has eaten OpenMath's lunch, but the fact that the MathML standard
explicitly points to the OpenMath standard as a means of extensibility is
at least a counter-argument.
On page 5, the author says that semantics means ``the meaning of $Y$ to the
program $X$'', and therefore that OpenMath does not solve the $n^2$
problem. As far as I can tell (there is no bibliography) the author does
not refer to the Corless paper, which demonstrates some of the difficulties
in this area, and the techniques for overcoming them, notably by having a
reference `abstract' meaning. Where the author is right is that different
programs $X$ may do different things to the objects they handle --- after
all, there is no point in writing a symbolic integrator in \TeX{}, and not
much point in wiriting a true typesetter in anything else.
The statement on page 5 about the n-ary version of the axiom of
commutativity for IEEE arithmetic is true, as indeed is the stronger one
about the failure of associativity. The problem is rather that all numeric
techniques are approximations, or, in the language of my paragraph above,
what IEEE does with OpenMath objects such as an n-ary plus may well not be
mathematically perfect, but that is not the fault of OpenMath: we have
expressed our idea correctly, and then sent it to an imperfect program.
The reference to Kajler & Soiffer is a good one, in terms of display, and
indeed might be releant to asking how computer algebra systems generate
MathML-P, whether or not this is done via OpenMath.
At the top of page 7, I fear that the author has fundamentally missed the
point. His students on their beginning programming class were rendering
mathematics on a known system, whereas the purpose of the WWW, and
therefore MathML, is to render on an unknown system. Hence the conversion
via XSL into MathML is done at the sending end, but the receiving end
renders that, which may be into speech rather than 2-D diagrams at all. It
is therefore right that the XSL should describe, not, as the author says,
the program to display the object, but rather the pre-rendering (Not in the
sense of sin x rather than sin (x), but issues of layout) format of the
object. Conversely, I agree that his point further down the page about
``written mathematics'' (which I take really to mean ``high-resolution
quality mathematical printing'') being best rendered by \TeX{} is
well-taken, and indeed one unsolved MathML challenge is high-quality (i.e.
via \TeX{} in most cases) printing.
I find the attack on Strotmann/Kahout on page 8 hard to follow. What S/K
are essentially saying is that OpenMath HAS learnt, and HAS a better system
than most computer algebra systems (say), but (though they do not make this
as explicit as I will) that the common calculus technique
$$
\int_0\sin(x)\d x
$$
as a short-hand for
$$
\int_0^x\sin(x)\d x = \int_0^x\sin(y)\d y
$$
(by the Openmath rules for bound variables) is revolting to computer
scientists.
On page 9, the author objects to the fact that OpenMath did not use
mathematics ``as published''. Clearly reproducing the whole research
literature (and its CDs --- a non-trivial task) is difficult, but several
studies into OpenMath versus published literature have taken place. The
only one in this collection is Kohlhase's paper --- again not referred to.
As a very minor point, the eulogy of Lisp on page 6 has unmatched
parentheses. Even more minor, a missing stop at the end of the penultimate
paragraph on page 7.
In sum, I find it hard to see that the author has understood the issue he
is criticising, though part of this may be a valid criticism of the
presentation. I suggest it be published as a rebuttal in the same journal.
..........................
..........................
Fill in Referee-assigned grades below
grades are to be in the range 1..4
(Use: 1, Excellent; 2, Good; 3, Below Average; 4, Poor.)
I. RECOMMENDATIONS
[ ] Excellent quality, definitely should be accepted
[ ] Good quality, acception is recommended
[ ] Marginal quality, acception is inappropriate
[x] Poor quality, rejection is imperative
II. SCOPE
[1] Relevance to symbolic computations
III. EVALUATION OF CONTENTS
[1] Relevance
[3] Originality
[3] Importance
[3] Difficulty
[3] Correctness
[3] Completeness
[4] Literature (Cites previous work correctly?)
IV. EVALUATION OF STYLE AND FORM
[ ] Clarity
[ ] Introduction (Relevance clearly stated in intro/abstract ?)
[1] English
[1] Length (Optimal, Appropriate, too long/short, unreadable because of
length)
V. COMMENTS TO THE AUTHOR(S)
This article discusses several relatively well-known issues related
to the communication of mathematical data. While this is a topic
of great importance to symbolic computation, there is little in the
way of original contribution in this aspect of the paper.
The discussion related to OpenMath seems to be more an analysis of the
author's conclusions about OpenMath rather than an analysis of OpenMath
itself. There are some valuable observations in this discussion, but
these need to be fished out of the rest of the article.
..........................
-----------end of verbatim reports------------
Comments on referee 1.
This person thought the paper was of marginal quality. We are
all entitled to our opinions.
It is relevant to symbolic computation (2=good) but neither
original nor important (3=below average). Its "difficulty" is
poor. Does that mean it is easy? or very difficult? This
ambiguous rating has been in the referee forms for ISSAC for
a long time. Am I the only one who can't figure it out?
Apparently my paper does not cite literature, though the
first line of the abstract says that it all depends on
SIGSAM Bulletin vol 34 number 2. I guess the referee wanted
to see this line in the bibliography. It may also be the
case that this publication is inaccessible to many people
and therefore not a proper reference. It seems that the
effort to make the SIGSAM Bulletin on-line has faltered.
Too bad.
Apparently my style and form are mostly (2=good) except for
the introduction.
now for the comments to the author.
It seems the referee misses the point of Openmath as explained by my
paper, which must either mean I did not make the point clearly enough,
or the referee was not willing to hear it. Although OM seems to be
concerned about notation, the REASON for OM cannot be the notation.
For that we already have conventional mathematical notation, for good
or ill. Some people working on OpenMath may think their primary goal
is data related, but the IMPLICIT AIM is to solve the SOFTWARE
problem. If there is no software involved,who cares about their
notation? There are many notations for mathematics. Look at
Leibnitz. Newton. Godel.
Russell/Whitehead. Turing. Bourbaki. Wolfram(!) Do we need
another one?
Reading this referee's report one would perhaps conclude that
OM was a great success. Instead it has been, so far, without
any impact except perhaps as a footnote to MathML.
--------------------------------
Comments on Referee #2's report
This person thought the topic relevance was "excellent" but declines
to credit the content with anything better than below-average(3).
He/she seems to think I've gotten the right style and length. The
problem here is that the referee wants someone to write a paper
explaining OpenMath. I can't image why that would be original. After
all, there are many many words written in the OM archives. Instead, I
wrote a paper trying to explain why it is wrong-headed. This too
appeared in the OM archives, and attracted substantial comments, (some
of the comments were private so as to not adversely affect funding or
inter-personal relationships!)
If the only people at conferences to discuss OM are the advocates of
OM, then it must remain a mystery to the community -- perhaps handed
from person to person-- as to why it adoption seems to be so slow.
Of course I could be just impatient and will eventually be shown to be
wrong, and OpenMath is just what is needed.
I note that in the program for ISSAC 2001 there are no accepted papers
on OpenMath. I have no idea if any other papers on OpenMath were
submitted. But then computer algebra systems papers may no longer
have much relevance to ISSAC conferences, since of the 46 papers
perhaps 2 concern systems issues.
April, 2001.
Richard Fateman