The textfile module

The textfile module was originally designed to allow Orchids to read data from a text file, and split it into lines.  It can now also read from Unix pipes and Unix (AF_UNIX) TCP sockets.

The textfile module is  an input module, meaning that its purpose is to read data from some sources, and convert it to Orchids events.  The sources  should be declared in the orchids-inputs.conf file, by directives such as:

INPUT textfile /var/log/messages

Admissible sources are regular files, Posix pipes and Unix (AF_UNIX) TCP sockets (SOCK_STREAM).  The latter are needed to interact with auditd, for example.

Internet (AF_INET) sockets are not handled, and should be dealt with using the udp module.  Unix UDP sockets (SOCK_DGRAM) should be dealt with using the sockunix module.

The textfile module reads from these sources and produces Orchids events, one per text line.  The contents of each line will then be found in the .textfile.line field of the event.  The name of the source, as given in the orchids-inputs.conf file, will be in .textfile.file.

Configuration options

<module textfile>

  • PollPeriod secs: instructs Orchids that it should poll files read by the textfile module every secs seconds. There is no way to specify a polling period of less than 1 second. This takes precedence over the general PollPeriod directive.
  • ProcessAll n: if n is non-zero, read all lines from the start of the file. This only applies to files, not pipes or Unix TCP sockets, and is meant to analyze log files, offline. The default behavior (n=0) is to read only lines added after Orchids started, through polling.
  • ExitAfterProcessAll n: if n is non-zero, shutdown Orchids after last line has been read. On pipes and sockets, this will cause Orchids to shutdown when the pipe or socket is closed. This is meant to analyze a single log file, offline. If different input sources are specified, Orchids will exit regardless of the fact that there are still events pending from the other sources. The default behavior (n=0) is to continue polling, waiting for extra lines, once the end of file has been reached.

</module>

The textfile module also understands options named INPUT and AddInputFile.  They are synonyms, take a file, pipe or Unix socket name as argument, which they open and use as input to the textfile module.  They should not be used inside the textfile module configuration file.  Instead, input should be specified in the orchids-inputs.conf file.

Fields

The last field is .textfile.line, available for further dissection; the dissection key is .textfile.file.

Field Type Mono? Description
.textfile.line_num uint  ✓ line number
.textfile.file str source file name
.textfile.line str current line