1. General Questions

1.1 Why a completely new indexing tool? makeindex works fine!

With the International MakeIndex project, Joachim Schrod and Gabor Herr have shown that adding extensions to makeindex is a difficult job. Thus we have decided to develop a new indexing tool from scratch. The new tool is based on a new requirements analysis and offers very interesting features for processing very complex indexing schemes. The resulting index model is described in:

1. Roger Kehr, xindy -- A Flexible Indexing System, Proceedings of the EuroTeX'98, St. Malo, France, March 1998
2. Roger Kehr, xindy -- A Flexible Indexing System, Technical Report No. 11/1997, Computer Science Department, Darmstadt University of Technology, 1997. Also accessible from the xindy homepage.
3. Roger Kehr, xindy Ein Flexibles Indexierungssystem, Studienarbeit FB Informatik, TH-Darmstadt, November 1995.
4. Joachim Schrod, An International Version of MakeIndex, Cahiers GUTenberg, 10, p81-90, 1991
5. Joachim Schrod and Gabor Herr, MakeIndex Version 3.0, Technical University of Darmstadt, 1991

1.2 I'm an old makeindex wizard. What does xindy offer that makeindex doesn't?

Here are the most important differences between xindy and makeindex:

Internationalization

xindy can be configured to process indexes for many languages with different letter sets and different sorting rules. For example, many roman languages such as Italian, French, Portuguese or Spanish contain accentuated letters such as À, Á, ñ. Other languages from northern Europe have letters like Ä, Ø, æ or ß which often can't be processed by many index processors not talking about sorting them correctly into an index. The xindy-system can be configured to process these alphabets by defining sort and merge-rules that allow expressing of language specific rules. One example of such a rule would be

(sort-rule "ä" "ae")

defining that a word containing the umlaut-a will be sorted as if it contained the letters ae instead. This is one form of how the umlaut-a is sorted into german indexes. With an appropriate set of rules one can express the complete rules of a specific language.

Location classes.

makeindex is able to recognize and process arabic numbers, roman numerals and letter-based alphabets as specifiers for the indexed location. Simple composite structures of these are also possible to process. This implicit recognition scheme has completely been dropped in favour of a well-defined and very powerful declaration scheme called location-classes. Thus, xindy initally does not know any location-class by default and must be instructed to accept certain location-classes. A typical declaration might look like:

(define-location-class "page-numbers" ("arabic-numbers"))

This declares that page numbers consist of the enumeration of the arabic numbers. The arabic numbers are referred to as alphabets. Users may use the pre-defined alphabets arabic numbers, roman numerals, etc. or define new alphabets as needed. See the tutorial that comes with this distribution for some examples.

The concept of attributes.

With makeindex one can assign a markup to each index entry using the encapsulators (usually following the vertical bar sign in an index entry command). For example in the specification

\index{xindy|bold}

the encapsulator is bold which encapsulates the page-numbers in the markup-phase. An additional TeX-macro must be supplied to assign some markup with the page number. This concept has completely been dropped in xindy in favour of a more powerful scheme called attributes. Attributes can be used to (i) define several grouping and ordering rules with locations and we can define (ii) markup-tags for the document preparation system.

The result of this design decision is that the user is required to define the attributes in the style file and not in the document preparation system. The reasons lie in the more powerful markup scheme of xindy which can't be specified in the document processor anymore. In fact, the makeindex-like markup is only a small subset of xindys features.

Cross references.

Cross-references were implemented in makeindex with the encapsulation mechanism, which only served for markup purposes. This has been completely separated in xindy. Here we distinguish cleanly between attributes and cross references. This makes it possible to implement checked cross references, i.e. cross-refernces that can be checked if they point to an existing index entry instead of somewhere ``behind-the-moon''.

Less command-line options.

xindy has dropped the usage of command-line options in favour of a well-defined indexstyle description language. Thus, options that could be activated at the command-line level of makeindex must now be specified in the indexstyle file. This sounds more restrictive than it is, because the indexstyle files can be composed from several modules which makes it possible to write style files in just a few lines of code.

Raw index parser.

The parser built into makeindex has completely been separated from the core xindy system. xindy understands a well-defined specification language for the raw index that is completely different from makeindex, but in our opinion more maintainable than the makeindex format. This requires a separate filter that transforms arbitrary indexes to the xindy format. An example filter is the program tex2xindy that comes with this distribution.

Summed up, some of the implicit assumptions made by makeindex have been replaced and now burdend to the user. The reason is that many of makeindex's assumptions were no longer valid in multi-language environments or documents with arbitrary location structures. This also characterizes xindy more as a framework instead of a end-user-tailored product. One should notice that writing an appropriate index style is an essential part of the document preparation process and should be tailored to each document anew.

1.3 What is tex2xindy?

tex2xindy is a filter that parses `.idx' or similar files and converts the \indexentry macros into a form readable by xindy.

The parser of makeindex can be configured to recognize different quoting characters, etc. (see the man-page for makeindex, section input style specifiers for further details). We have tried to extract the parser from makeindex but due to several probems we have finally rewritten the parser using lex. Scanners written with lex are usually fixed to a specific character set used in the regular expressions. Our parser, tex2xindy is therefore not configurable. If one uses a different configuration of the makeindex input style specifiers, one can change the source (tex2xindy.l) to generate a completely new parser. From our personal experience we have rarely used more than two different parsers in practice so we have written tex2xindy in a form that is easily maintainable. The input specifiers are stored symbolically in the source. The definiton section looks like this:

KEYWORD  \\indexentry
ENCAP    \|
ACTUAL   @
ESCAPE   \\
LEVEL    !
QUOTE    \"
ROPEN    \(
RCLOSE   \)
ARGOPEN  \{
ARGCLOSE \}

These definitions are essentially the input style specifiers as can be found in the man-page of makeindex. Changing this section according to your needs and recompiling tex2xindy should be an easy task. Maybe we will include more pre-defined parsers in future releases if necessary.

1.4 What editor should I use when writing xindy style files?

Use Emacs or XEmacs. Turn on the Lisp-mode with

M-x lisp-mode

and you can properly indent commands using M-q. To enter this mode automatically add the following lines to the end of the style file:

^L
;; Local Variables:
;; mode: lisp
;; End:

The ^L (Control-L) can be entered with C-q C-l.