syslog facility and severity (loglevel)

[telecity]

I've added the following to etc/system/local/limits.conf

[udp://514]
no_priority_stripping = true
no_appending_timestamp = true

My interest is to retrieve the facility and severity (loglevel) from the incoming syslog events. However now each event is prefixed with <137> which means nothing to me. Here's an example:

<137>Sep 22 15:52:30 host...

Facility is set at local1 and level is alert. Per rfc3164 that'd be facility=17 and severity=1.

My questions:
1. What is <137> (it wasn't there
before, and does show up in _raw)?
2. How do I retrieve facility / severity? I'd like them to be indexed fields if possible, to make searching, sorting and alerting easier.

[Jason]

If you are using UDP directly in to Splunk (not our best practice, which is to use a separate syslog daemon and write to files, and have Splunk read the files)

and have no_priority_stripping turned on, then try my app.

http://splunk-base.splunk.com/apps/28634/syslog-priority-field-decoder-lookup

It will use a lookup to automatically create the syslog severity and facility names for you at search time.

[koshyk]

great app @Jason. Still very useful after 4+ years

[Jason]

you're welcome - seems like syslog hasn't changed very much over the years

[_smp_]

Thanks for creating this app - the config files worked great for me.

[Jason]

You're welcome. This reminded to update the listing to show it works with the latest versions of Splunk.

[are0002]

I founded it.

In newest version you have to edit inputs.conf

source: http://docs.splunk.com/Documentation/Splunk/latest/Data/Monitornetworkports?r=searchtip

[are0002]

Hi

I am using the last version of Splunk and I can not activate the priority level on my logs. I edited all limit.conf files with your code.

Regards,

[billshinn]

In RFC3164 priority (i.e. the required PRI part of the syslog packet (before the HEADER and MSG) is calculated by multiplying the facility by 8, then adding the severity. So per the RFC, where local1 = 17, therefore 17*8 = 136. Adding to that a 1 for the severity = alert, you get the 137 mentioned in the original post. The <137> is just on spec for a proper syslog message. As for extraction, you need to reverse the math. This explains it really, really well:

https://gist.github.com/1017480

As for field extraction, I think there must be a quick piece of code that can do that vs a table/matrix search.

[SEJeff]

Pretty print a table of this data in a nice "human friendly" format:
http://www.digitalprognosis.com/opensource/scripts/syslog-priorities.py

Example output:

            emergency   alert   critical   error   warning   notice   info   debug
kernel              0       1          2       3         4        5      6       7
user                8       9         10      11        12       13     14      15
mail               16      17         18      19        20       21     22      23
system             24      25         26      27        28       29     30      31
security           32      33         34      35        36       37     38      39
syslog             40      41         42      43        44       45     46      47
lpd                48      49         50      51        52       53     54      55
nntp               56      57         58      59        60       61     62      63
uucp               64      65         66      67        68       69     70      71
time               72      73         74      75        76       77     78      79
security           80      81         82      83        84       85     86      87
ftpd               88      89         90      91        92       93     94      95
ntpd               96      97         98      99       100      101    102     103
logaudit          104     105        106     107       108      109    110     111
logalert          112     113        114     115       116      117    118     119
clock             120     121        122     123       124      125    126     127
local0            128     129        130     131       132      133    134     135
local1            136     137        138     139       140      141    142     143
local2            144     145        146     147       148      149    150     151
local3            152     153        154     155       156      157    158     159
local4            160     161        162     163       164      165    166     167
local5            168     169        170     171       172      173    174     175
local6            176     177        178     179       180      181    182     183
local7            184     185        186     187       188      189    190     191

This should be easy enough to hack up.

[MillerTime]

One idea that comes to mind for making use of this data would be to transform this matrix into a CSV and use it as part of automatic lookup definition in order to provide additional context/knowledge about these events. Using the code (in your case 137) as the input field, and providing Facility and Severity as output fields.

I'm thinking columns like:

Code,Facility,Severity
1,kernel,0
2,kernel,1
...
18,mail,1
137,local1,0

http://docs.splunk.com/Documentation/Splunk/latest/User/CreateAndConfigureFieldLookups

[infinitiguy]

this was exactly what I was looking for

[telecity · Accepted solution]

After loads of searching, I think I've finally found the answer. Apparently splunk is using a matrix (which I suspected, but couldn't identify or confirm) that seems to match the following:

Severity        0   1   2   3   4   5   6   7
Facility
kernel  0   0   1   2   3   4   5   6   7
user    1   8   9   10  11  12  13  14  15
mail    2   16  17  18  19  20  21  22  23
system  3   24  25  26  27  28  29  30  31
security    4   32  33  34  35  36  37  38  39
syslog  5   40  41  42  43  44  45  46  47
lpd 6   48  49  50  51  52  53  54  55
nntp    7   56  57  58  59  60  61  62  63
uucp    8   64  65  66  67  68  69  70  71
time    9   72  73  74  75  76  77  78  79
security    10  80  81  82  83  84  85  86  87
ftpd    11  88  89  90  91  92  93  94  95
ntpd    12  96  97  98  99  100 101 102 103
logaudit    13  104 105 106 107 108 109 110 111
logalert    14  112 113 114 115 116 117 118 119
clock   15  120 121 122 123 124 125 126 127
local0  16  128 129 130 131 132 133 134 135
local1  17  136 137 138 139 140 141 142 143
local2  18  144 145 146 147 148 149 150 151
local3  19  152 153 154 155 156 157 158 159
local4  20  160 161 162 163 164 165 166 167
local5  21  168 169 170 171 172 173 174 175
local6  22  176 177 178 179 180 181 182 183
local7  23  184 185 186 187 188 189 190 191

Source of Matrix:
http://chris-mccafferty.blogspot.com/2010/12/syslog-priority-matrix.html

Here's a perl script to sort it out for you:

#!/usr/bin/perl -w
use strict;
# http://splunk-base.splunk.com/answers/31036/syslog-facility-and-severity-loglevel

my @facilities = qw(Kernel User Mail System Security Syslog Lpd Nntp Uucp Time
                        Security Ftpd Ntpd Logaudit Logalert Clock Local0 
                        Local1 Local2 Local3 Local4 Local5 Local6 Local7);
my @severities = qw(Emergency Alert Critical Error Warning Notice Info Debug);
my $count = 0;

foreach my $facility (@facilities) {
        foreach my $severity (@severities) {
                print("$count,$facility.$severity\n");
                $count++;
        }
}

[makelovenotwar]

Thanks for doing the legwork here!

Figured I'd post the results of your perl scrip to make it even easier. Also, here's an updated url for a matrix. 

https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/privileged-access-m...

code,facility,severity
0,Kernel,Emergency
1,Kernel,Alert
2,Kernel,Critical
3,Kernel,Error
4,Kernel,Warning
5,Kernel,Notice
6,Kernel,Info
7,Kernel,Debug
8,User,Emergency
9,User,Alert
10,User,Critical
11,User,Error
12,User,Warning
13,User,Notice
14,User,Info
15,User,Debug
16,Mail,Emergency
17,Mail,Alert
18,Mail,Critical
19,Mail,Error
20,Mail,Warning
21,Mail,Notice
22,Mail,Info
23,Mail,Debug
24,System,Emergency
25,System,Alert
26,System,Critical
27,System,Error
28,System,Warning
29,System,Notice
30,System,Info
31,System,Debug
32,Security,Emergency
33,Security,Alert
34,Security,Critical
35,Security,Error
36,Security,Warning
37,Security,Notice
38,Security,Info
39,Security,Debug
40,Syslog,Emergency
41,Syslog,Alert
42,Syslog,Critical
43,Syslog,Error
44,Syslog,Warning
45,Syslog,Notice
46,Syslog,Info
47,Syslog,Debug
48,Lpd,Emergency
49,Lpd,Alert
50,Lpd,Critical
51,Lpd,Error
52,Lpd,Warning
53,Lpd,Notice
54,Lpd,Info
55,Lpd,Debug
56,Nntp,Emergency
57,Nntp,Alert
58,Nntp,Critical
59,Nntp,Error
60,Nntp,Warning
61,Nntp,Notice
62,Nntp,Info
63,Nntp,Debug
64,Uucp,Emergency
65,Uucp,Alert
66,Uucp,Critical
67,Uucp,Error
68,Uucp,Warning
69,Uucp,Notice
70,Uucp,Info
71,Uucp,Debug
72,Time,Emergency
73,Time,Alert
74,Time,Critical
75,Time,Error
76,Time,Warning
77,Time,Notice
78,Time,Info
79,Time,Debug
80,Security,Emergency
81,Security,Alert
82,Security,Critical
83,Security,Error
84,Security,Warning
85,Security,Notice
86,Security,Info
87,Security,Debug
88,Ftpd,Emergency
89,Ftpd,Alert
90,Ftpd,Critical
91,Ftpd,Error
92,Ftpd,Warning
93,Ftpd,Notice
94,Ftpd,Info
95,Ftpd,Debug
96,Ntpd,Emergency
97,Ntpd,Alert
98,Ntpd,Critical
99,Ntpd,Error
100,Ntpd,Warning
101,Ntpd,Notice
102,Ntpd,Info
103,Ntpd,Debug
104,Logaudit,Emergency
105,Logaudit,Alert
106,Logaudit,Critical
107,Logaudit,Error
108,Logaudit,Warning
109,Logaudit,Notice
110,Logaudit,Info
111,Logaudit,Debug
112,Logalert,Emergency
113,Logalert,Alert
114,Logalert,Critical
115,Logalert,Error
116,Logalert,Warning
117,Logalert,Notice
118,Logalert,Info
119,Logalert,Debug
120,Clock,Emergency
121,Clock,Alert
122,Clock,Critical
123,Clock,Error
124,Clock,Warning
125,Clock,Notice
126,Clock,Info
127,Clock,Debug
128,Local0,Emergency
129,Local0,Alert
130,Local0,Critical
131,Local0,Error
132,Local0,Warning
133,Local0,Notice
134,Local0,Info
135,Local0,Debug
136,Local1,Emergency
137,Local1,Alert
138,Local1,Critical
139,Local1,Error
140,Local1,Warning
141,Local1,Notice
142,Local1,Info
143,Local1,Debug
144,Local2,Emergency
145,Local2,Alert
146,Local2,Critical
147,Local2,Error
148,Local2,Warning
149,Local2,Notice
150,Local2,Info
151,Local2,Debug
152,Local3,Emergency
153,Local3,Alert
154,Local3,Critical
155,Local3,Error
156,Local3,Warning
157,Local3,Notice
158,Local3,Info
159,Local3,Debug
160,Local4,Emergency
161,Local4,Alert
162,Local4,Critical
163,Local4,Error
164,Local4,Warning
165,Local4,Notice
166,Local4,Info
167,Local4,Debug
168,Local5,Emergency
169,Local5,Alert
170,Local5,Critical
171,Local5,Error
172,Local5,Warning
173,Local5,Notice
174,Local5,Info
175,Local5,Debug
176,Local6,Emergency
177,Local6,Alert
178,Local6,Critical
179,Local6,Error
180,Local6,Warning
181,Local6,Notice
182,Local6,Info
183,Local6,Debug
184,Local7,Emergency
185,Local7,Alert
186,Local7,Critical
187,Local7,Error
188,Local7,Warning
189,Local7,Notice
190,Local7,Info
191,Local7,Debug

[PickleRick]

You are aware that you're responding to a 10 years old thread, aren't you? 🙂

[makelovenotwar]

Hey, if I ended up here, someone else might too and maybe, just maybe, I saved them 5minutes of work. 

[yeasuh]

The beauty of this site! Some things (troubleshooting) never change hahaha

Thanks so much for contributing! 

[jchampagne]

The way I implemented this was to create a new App that contained my lookup csv as well as the tranforms.conf that references the lookup file. We'll call this "App-syslog-lookup". The permissions on that app were set to allow the files to be shared with all other Splunk apps. This allows me to reference the same lookup file from multiple apps.

I then edited the props.conf inside each app that I want to do a lookup with. As long as you have "App-syslog-lookup" shared with other apps, you can reference the stanza, syslog_facility_severity_codes, from your props.conf and do a lookup.

[jchampagne]

You must already be extracting the syslog code from your events as a field called "syslog_code". I added the following lines to my files to make that happen:

PROPS.conf
- Under the stanza for my sourcetype:
REPORT-ExtractSyslogCode = extract_syslog_code

TRANSFORMS.conf
- New stanza called [extract_syslog_code]
[extract_syslog_code]
REGEX = ^<(\d+)>
FORMAT = syslog_code::$1

[jchampagne]

1. Create a tranforms.conf that references your lookup:

[syslog_facility_severity_codes]
filename = syslog-codes.csv

1. In props.conf, add the following line to the stanza of any sourcetype that you want to do the lookup for:

LOOKUP-SyslogCode = syslog_facility_severity_codes code AS syslog_code OUTPUTNEW facility AS facility, severity AS severity

[jchampagne]

There is definitely a more "fancy" way of doing this via a scripted lookup that does a matrix search, but I didn't want to spend the time doing that...so here is what I did...

1. I created a CSV that contains all of the Syslog codes as well as two corresponding columns, one for the facility, and one for the severity. I'll see if I can post the whole thing in another message, but this is basically what it looks like:

code,facility,severity
0,kernel,emergency
1,kernel,alert
2,kernel,critical