Ubuntu Manpage: lexholder-ru - rulex database holding utility

NAME

       lexholder-ru - rulex database holding utility

SYNOPSIS

       lexholder-ru [options] <db_path>

DESCRIPTION

       lexholder-ru is a small utility intended for use from the command line or shell-scripts. It allows one to
       construct, test, manage and query lexical database as well as extract its content in textual form.

       This database is primarily intended for use along with the Russian TTS engine ru_tts to provide stressing
       and pronunciation information for the Russian words.

       When  filling  and  updating the database, new records are read from the standard input.  When extracting
       data from the database, The result is printed to the standard output.  This behaviour can be  changed  by
       the -f switch.

OPTIONS

All options recognized in the command line are described below. For more convenience they are arranged
into several groups by its functionality.

The first group consists of options specifying an action to be done. These options are mutually
exclusive. We can do only one action per invocation. If no action is specified, the program reads its
standard input (or a file specified by -f option) and stores its content in the database. Here are the
other actions:

-h
Print summary of options and exit. This option discards all other command line specifications. It
is the only case when the database path is not required.

-l
List database content in textual form. This action requires the dataset to be specified explicitly
by one of the -X, -M, -G, -L, -P or -C options.

-s <key>
Search specified key in the lexical database. If the word is found program exits successfully and
outputs its pronunciation string,
otherwise prints the lowercased original word and exits with non-zero exit code. This action is
affected by the search mode options described below. If the -q switch is specified in the command
line, nothing will be printed on the standard output, but return code still can be used to find
out whether the word was found or not.

-b <key>
Treat specified word as an implicit form and discover basic forms (if any) which could be used in
the Implicit dictionary. If quiet mode is not in use then all possible basic forms for the word
will be printed to the standard output (or to the file specified by the -f option) along with the
numbers of corresponding Classifiers. Program exits successfully if it can suggest some basic
forms for specified word and returns a non-zero exit code otherwise. In quiet mode nothing will
be printed on the standard output, but the exit code still can be used to make a decision
concerning the operation result.

-t <dictionary_file>
Test the database against specified dictionary. Test dictionary file is read line by line. Each
line is treated as a record consisting of two fields separated by space. The first field
represents a key word and the second one gives its pronunciation string. If this pronunciation
string differs from the one obtained from the database, then this record is printed to the
standard output or written to the file specified by -f option. Specifying "-" as test dictionary
file name causes the testing records to be read from the standard input. This action is affected
by the search mode options described below.

-d <key>
Delete record for specified key. This action requires the dataset to be specified explicitly by
one of the -X, -M, -G, -L, -P or -C options. For rules its number in the ruleset is used as a key.

-D
Discard the dataset. The dataset must be chosen by one of the -X, -M, -G, -L, -P or -C options.

-c
Clean the database removing redundant entries from dictionaries. By default all records that
surely do not affect any search result are removed. These are the entries of the Implicit
dictionary that do not represent any lexical base and the entries of the Explicit dictionary that
in fact duplicate the result of usual lookup process. If one of the -X or -M options is specified
as well, then only that chosen dictionary will be cleaned. If the Implicit dictionary is chosen,
the extensive cleanup is performed for it, that can drop some useful records. Be careful.

The next group of options is responsible for choosing the dataset. These options are mutually exclusive
and affect deletion, insertion and listing operations. For listing and deletion the dataset must be
specified explicitly. If no one of these options is mentioned when inserting new data, an appropriate
dataset will be chosen according to the input data. Only lexical data can be inserted in such a manner.
For rules target dataset must be specified explicitly.

-X
Explicit dictionary.

-M
Implicit dictionary.

-G
General rules.

-L
Lexical classification rules.

-P
Prefix detection rules.

-C
Correction rules.

The next group contains options devoted to search mode specification.
These options affect search and test operation. By default (no options) full search will be performed,
otherwise only those stages specified explicitly will be included in the search process.

-x
Search in the explicit dictionary.

-m
Try to treat the word as an implicit form.

-g
Try to apply general rules.

The next group contains only one option that affects insertion new data into the lexical database.

-r
Replace mode. For a dictionary this mode causes that the new records replace existing ones with
the same key. By default such records are ignored. For rules this mode means that the ruleset
content should be fully replaced by the new data. Otherwise new rules are appended to the ruleset.

The last group contains several options affecting program behaviour in general.

-f <file>
Use specified file instead of standard input or output.

-q
Be more quiet than usual: don't print search results as well as warnings about duplicate records.

-v
Be more verbose than usual: print messages about work stages and final statistical information
when finishing.

DATA REPRESENTATION

Externally all the data are represented textually. For the Russian letters the koi8-r character set is
used and only lower case is allowed.

The database itself consists of two dictionaries and four sets of rules. The Explicit dictionary contains
the words that are described individually and do not imply any information for other forms. This
dictionary is looked up first if the search includes this stage. The Implicit dictionary contains words
in some basic form. This dictionary is used to construct pronunciation string for various forms of these
words. The basic form of a word is guessed according to the rules from the Classifiers and Prefix
detectors rulesets. This is the second stage of search process. If these stages do not bring a result or
are not performed the rules from the General ruleset are used to guess stressing word. If no one of these
rules can be applied than no guessing is made and search process fails. By default, all three stages are
performed, but it can be specified explicitly which ones should be taken in account.

Externally dictionary data are represented by text lines consisting of two fields separated by space. The
first field is a Russian word. It serves as a key when searching. Only lowercase Russian letters are
allowed here. The second field provides pronunciation string for this word. The pronunciation string is
the word itself, but written in such a manner as it should be pronounced. There are three additional
symbols allowed in the pronunciation string along with the lowercase Russian letters. The "+" sign can be
used to point the stressed letter. It should be placed just after that letter. The "=" sign is used in
some cases just in the same manner to point so-called weak stress. The "-" sign can serve as a separator
in some complex words. All other symbols are treated as illegal.

There are four rulesets in the database: General rules, Classifiers, Prefix detectors and Correctors.
Externally all these rules are represented by strings consisting of one or two fields separated by space.
The first field always contains a regular expression which is matched against the word to make a decision
whether this rule can be applied.

The only task of General rules is to guess stress in the words when dictionary lookup fails. The rules
are tried sequentially until match or the list exhaustion. If match succeeds then the "+" sign is
inserted into the word right after the first subexpression match to point stressing position.
These rules do not contain a second field.

For the Classifiers ruleset each rule is checked one by one until match occurs. Then the part from the
beginning of the word through to the end of the first subexpression match is extracted and if a second
field is present it is appended to the extracted part as a suffix. The resulting string is treated as a
basic form of the word, so it is looked up in the Implicit dictionary. If nothing is found the process
continues until the ruleset will be exceeded.

When nothing is found in the database for a word in its original form, Prefix detection rules are applied
to it sequentially until match occurs. The matched prefix is stripped and replaced by the replacement
string if any. Then the result word is searched in the Implicit dictionary. In the case of success the
original prefix is restored in the pronunciation string.

The rules from Correctors ruleset are applied to the pronunciation strings instead of the original words.
The second field in these rules specifies a regular replacement string where digits serve as
subexpression numbers.

AUTHOR