Indexing Postscript and PDF documents containing mathematical equations
Users with large personal document collections invariably want to make them
easily searchable at some point. This can be accomplished on UNIX with free tools such as ht://Dig, SWISH-E and glimpse. In case
some of the documents are in Postscript or PDF format, they must first be converted to plain text using a tool such as pstotext(1).
For mathematical documents, conversion with pstotext(1)
results in text interspersed with many
lines of random characters, because displayed equations aren't handled properly. In these cases, dbacl(1) can act as a filter to remove
the noise lines, by recognizing only lines which appear to be
mostly English text. This somewhat prevents the noise from polluting the list of indexed terms.
The following shell command converts a Postscript or PDF document and filters the noisy lines:
% pstotext Diestel-Graph_Theory.pdf | dbacl -c shake -Rf shake > output.txt
For this to work, first
check that you have both pstotext(1) and dbacl(1) in your path, and create an "English text" category
if necessary as follows:
% zcat Shakespeare-Complete_Works.txt.gz | dbacl -l shake
The sample English text can be freely downloaded, e.g. from Project Gutenberg.