Ubuntu Manpage: makehistory - Initialize or rebuild INN history database

NAME

       makehistory - Initialize or rebuild INN history database

SYNOPSIS

       makehistory [-abFIOSx] [-f filename] [-l count] [-L load-average] [-s size] [-T tmpdir]

DESCRIPTION

makehistory rebuilds the history(5) text file, which contains a list of message-IDs of articles already
seen by the server. It can also be used to rebuild the overview database. Note that even though the
dbz(3) indices for the history file are also rebuilt by makehistory, it is useful to run makedbz(8) after
makehistory(8) in order to improve the efficiency of the indices (makehistory does not know how large to
make the hash table at first run, unless the size is given by the -s flag).

The default location of the history text file is pathdb/history; to specify an alternate location, use
the -f flag.

By default, makehistory will scan the entire spool, using the storage manager, and write a history line
for every article. To also generate overview information, use the -O flag.

WARNING: If you're trying to rebuild the overview database, be sure to stop innd(8) and delete or zero
out the existing database before you start for the best results. An overview rebuild should not be done
while the server is running. Unless the existing overview is deleted, you may end up with problems like
out-of-order overview entries, excessively large overview buffers, and the like.

If ovmethod in inn.conf is "ovdb", you must have the ovdb processes running while rebuilding overview.
ovdb needs them available while writing overview entries. You can start them by hand separate from the
rest of the server by running ovdb_init; see ovdb_init(8) for more details.

OPTIONS

-a Append to the history file rather than generating a new one. If you append to the main history file,
make sure innd(8) is throttled or not running, or you can corrupt the history.

-b Delete any messages found in the spool that do not have valid Message-ID: headers in them.

-F Fork a separate process to flush overview data to disk rather than doing it directly. The advantage
of this is that it allows makehistory to continue to collect more data from the spool while the first
batch of data is being written to the overview database. The disadvantage is that up to twice as
much temporary disk space will be used for the generated overview data. This option only makes sense
in combination with -O. With buffindexed, the overchan program is invoked to write overview.

-f filename
Rather than writing directly to pathdb/history, instead write to filename, also in pathdb.

-I Don't store overview data for articles numbered lower than the lowest article number in active. This
is useful if there are for whatever reason old articles on disk that shouldn't be available to
readers or put into the overview database.

-l count
This option specifies how many articles to process before writing the accumulated overview
information out to the overview database. The default is 10000. Since overview write performance is
faster with sorted data, each "batch" gets sorted. Increasing the batch size with this option may
further improve write performance, at the cost of longer sort times. Also, temporary space will be
needed to store the overview batches. At a rough estimate, about 300 * count bytes of temporary
space will be required (not counting temp files created by sort(1)). See the description of the -T
option for how to specify the temporary storage location. This option has no effect with
buffindexed, because buffindexed does not need sorted overview and no batching is done.

-L load-average
Temporarily pause activities if the system load average exceeds the specified level load-average.
This allows makehistory to run on a system being used for other purposes without monopolizing system
resources and thus making the response time for other applications unacceptably slow. Using nice(1)
does not help much for that because the problem comes from disk I/O usage, and ionice(1) is not
always available or efficient.

-O Create the overview database as well as the history file. Overview information is only required if
the server supports readers; it is not needed for a transit-only server (see enableoverview in
inn.conf(5)). If you are using the buffindexed overview storage method, erase all of your overview
buffers before running makehistory with -O.

-S Rather than storing the overview data into the overview database, just write it to standard output in
a form suitable for feeding to overchan later if wished. When this option is used, -F, -I, -l, and
-T are ignored. This option only makes sense in combination with -O.

-s size
Size the history database for approximately size pairs. Accurately specifying the size is an
optimization that will create a more efficient database. (The size should be the estimated eventual
size of the history file, typically the size of the old file, in lines.)

-T tmpdir
If -O is given, makehistory needs a location to write temporary overview data. By default, it uses
pathtmp, set in inn.conf, but if this option is given, the provided tmpdir is used instead. This is
also used for temporary files created by sort(1) (which is invoked in the process of writing overview
information since sorted overview information writes faster). By default, sort usually uses your
system temporary directory; see the sort(1) man page on your system to be sure.

-x If this option is given, makehistory won't write out history file entries. This is useful mostly for
building overview without generating a new history file.

EXAMPLES

       Here's a typical example of rebuilding the entire history and overview database, removing broken articles
       in  the  news  spool.  This uses the default temporary file locations and should be done while innd isn't
       running (or is throttled).

           makehistory -b -f history.n -O -l 30000 -I

       This will rebuild the overview (if using buffindexed, erase the existing overview buffers before  running
       this  command)  and  leave  a  new history file as "history.n" in pathdb.  To preserve all of the history
       entries from the old history file that correspond to rejected articles or expired  articles,  follow  the
       above command with:

           cd <pathdb>
           awk 'NF == 2 { print }' < history >> history.n

       (replacing  the path with your pathdb, if it isn't the default).  Then look over the new history file for
       problems and run:

           makedbz -s `wc -l < history.n` -f history.n

       Then rename all of the files  matching  "history.n.*"  to  "history.*",  replacing  the  current  history
       database and indices.  After that, it's safe to unthrottle innd.

       For a simpler example:

           makehistory -b -f history.n -I -O

       will  scan  the  spool, removing broken articles and generating history and overview entries for articles
       missing from history.

       To just rebuild overview:

           makehistory -O -x -F

FILES

       pathdb/history
           This is the default output file for makehistory.

       pathtmp
           Where temporary files are written unless -T is given.

HISTORY

       Originally written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews and updated by various other people
       since.

       $Id: makehistory.pod 8534 2009-06-23 18:08:14Z iulius $