The syslog module | The Orchids IDS

The syslog module is a dissection module: its purpose is to take an Orchids event, parse one of its text fields and return a refined Orchids events, with additional fields. Typically, the syslog module is meant to dissect text coming from the textfile or bintotext modules, assuming it is in syslog format.

The syslog format is the format of system-wide log files on Linux, such as /var/log/messages, or logs of Cisco routers.

There are two slightly differing format: the legacy syslog format and the RFC5424 format. If you can use the latter, fine. A lot of systems still use the legacy syslog format, which has lines resembling the following:

Jan 27 11:32:06 laramie dhclient: DHCPREQUEST on eth0 to 192.168.30.1 port 67

The legacy syslog format has a major drawback: its date field does not mention the year (in most common cases). The RFC5424 format does not suffer from this problem. For example, the date Jan 27 11:32:06 above would typically appear in RFC5424 format as 2017-01-27T11:32:06Z, which specifies the year (2017). The RFC5424 also solves the problem of which time zone the time is given in.

The question of the year (legacy syslog)

To palliate the missing year problem in legacy syslog events, Orchids allows you to specify a starting year, see ‘Configuration options’ below.

Orchids then decides to increment the year by one when it encounters a legacy syslog record whose date looks like it is in the past, compared to the previous legacy syslog record. For example:

Dec 27 23:34:55 laramie syslogd: exiting on signal 15
Jan 01 05:51:16 laramie sendmail[103]: NOQUEUE: SYSERR(root): /etc/sendmail.cf: line 0: cannot open: No such file or director

will be taken by Orchids as an indication that one year has passed between the first and the second event. Assuming we started in year 2014, the first event would be considered to have taken place on Dec 27 2014, 23:34:55, while the second one would be assumed to have taken place on Jan 01 2015, 05:51:16.

On the other hand, the following will not increment the year.

Jan 27 11:32:06 laramie dhclient: DHCPREQUEST on eth0 to 192.168.30.1 on port 67
Jan 27 13:00:32 laramie kernel: current->tss.cr3 = 00510000, ^Lr3 = 00510000

The algorithm Orchids uses may change, but as of January 22, 2015, it is as follows: let diff=(month-of-current-event minus month-of-last-event) mod 12. If diff ≥ 6, then increment the year, otherwise leave it unchanged. Note that years can only change when the month changes.

Structured data (RFC5424 syslog)

Each RFC5424 syslog event contains optional structured data. Consider the following RFC5424 syslog event:

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 [exampleSDID@0 iut="3" eventSource="Application" eventID="1011"][examplePriority@0 class="high"] 'su root' failed for lonvick on /dev/pts/8

The structured data is the collection of so-called sd-elements between square brackets: [exampleSDID@0 iut="3" eventSource="Application" eventID="1011"] and [examplePriority@0 class="high"].

You will retrieve them in the .syslog.data field, which is a two-dimensional Orchids array. There is one such array per RFC5424 syslog event, and one obtains its fields by writing .syslog.data[⟨id⟩][⟨parameter⟩]. In the example above, .syslog.data will be equal to:

id	parameter	value
`"exampleSDID@0"`	`"iut"`	`"3"`
`"exampleSDID@0"`	`"eventSource"`	`"Application"`
`"exampleSDID@0"`	`"eventId"`	`"1011"`
`"examplePriority@0"`	`"class"`	`"high"`

In other words, in that example, .syslog.data["exampleSDID@0"]["eventId"] will return "1011".

Configuration options

<module syslog>

DefaultStartYear nnnn: set the starting year to nnnn; nnnn can be a number, interpreted as an year, or of the form +n (meaning the current year plus n), or of the form –n (meaning the current year minus n).
Only affects dissecting legacy syslog.
Examples include:
```
DefaultStartYear +0
```
to set the starting year to the current year,
```
DefaultStartYear -3
```
to set it three years back, or
```
DefaultStartYear 2010
```
to set it to the year 2010. Default is DefaultStartYear +0.
StartYear tag nnnn: set a starting year that is specific to syslog records whose dissection tag is tag. The syntax of nnnn is as for the DefaultStartYear nnnn directive. This allows one to set different starting dates to syslog records coming from different sources.
Only affects dissecting legacy syslog.
For example, if you have written the following in orchids-inputs.conf:
```
INPUT          textfile "/var/log/messages"
DISSECT syslog textfile "/var/log/messages"
```
then StartYear "/var/log/messages" 2010 will state that the starting year for syslog records read from the /var/log/messages file is 2010, not the default starting year as described by DefaultStartYear.

</module>

Admissible sources are:

a text file such as /var/log/messages, /var/log/syslog, or /var/log/user.log for example, read through the textfile module (offline detection);
a Unix TCP socket such as /dev/log, read through the sockunix and bintotext modules (detection from an online, local feed);
a UDP Internet socket, read through the the udp and bintotext modules (for remote feeds).

Fields

The last field is .syslog.msg, available for further dissection; the dissection key is .syslog.prog. Dissecting the latter is typically handled through the generic module.

Field	Type	Mono?	Description
`.syslog.facility`	`uint`		so-called facility (kernel, user, mail, daemon, etc.); see `SYSLOG_FACILITY_`* macros in `$OCONF/rules/syslog.h` (this field is optional)
`.syslog.severity`	`uint`		severity of the message; see `SYSLOG_SEVERITY_`* macros in `$OCONF/rules/syslog.h` (this field is optional)
`.syslog.version`	`uint`		version of the syslog facility
`.syslog.time`	`ctime`	✓	date of the event — with year inferred as described above in case of legacy syslog
`.syslog.host`	`str`		host name
`.syslog.repeat`	`uint`		message repetition: do as through the current `syslog` record was received this many times (this field is optional, and for legacy syslog only)
`.syslog.pid`	`uint`		process id of the event source (this field is optional, and for legacy syslog only)
`.syslog.procid`	`str`		process id of the event source (RFC5424 only) (this field is optional)
`.syslog.msgid`	`str`		message id (RFC5424 only)
`.syslog.data`	`[[str`		structured data array mapping id and parameter to value (RFC5424 only)
`.syslog.prog`	`str`		program name
`.syslog.msg`	`str`		the message