All Apps and Add-ons

Nginx Error Customer logs Transforms/props not working

svemurilv
Path Finder

Hi ,
am trying to write Props /Transforms for the Nginx costume error logs . but they are not working here is my props and transforms.

Props.conf
[nginx:plus:apierror]
REPORT-systemoutlogs = event_extract

[nginx:plus:apperror]
REPORT-systemoutlogs = event_extract

[nginx:plus:ps2error]
REPORT-systemoutlogs = event_extract

[nginx:plus:nginxerror]
REPORT-systemoutlogs = event_extract

And i write the transforms.conf for all the sourcestypes as same:

[event_extract]
REGEX = (^(?P<date>\\d+/\\d+/\\d+\\s+\\d+:\\d+:\\d+)\\s+\\[\\w+\\]\\s+(?P<error>[^,]+),\\s+\\w+:\\s+(?P<client>[^,]+)(?:[^ \\n]* ){4}\"(?P<request>[^\"]+)[^,\\n]*,\\s+\\w+:\\s+\"(?P<upstream>[^\"]+)(?:[^ \\n]* ){2}\"(?P<hostname>[^\"]+))

And my log was looking like:

2017/05/18 16:50:34 [error] 28709#28709: *34 connect() failed (111: Connection refused) while connecting to upstream, client:192.168.128.17, server: , request: "GET /api/slot2/v1/idCards HTTP/1.1", upstream: "http://127.0.0.1:6001/api/slot2/v1/idCards", host: "d-servicing.qa.JPMC.com"
0 Karma

aakwah
Builder

Hello,
The regex just need some tuning, event_extract in transforms.conf should be like that

[event_extract]
REGEX = ^(?P\d+\/\d+\/\d+\s+\d+:\d+:\d+)\s+\[\w+\]\s+(?P[^,]+),\s+\w+:(?P[^,]+)(?:[^ \n]* ){4}\"(?P[^\"]+)[^,\n]*,\s+\w+:\s+\"(?P[^\"]+)(?:[^ \n]* ){2}\"(?P[^\"]+)

Regards

hunters_splunk
Splunk Employee
Splunk Employee

Hi svemurilv,

Have you tried ingesting Nginx logs using the Splunk Add-onf or Ngix? If not, I suggest you take advantage of the predefined sourcetypes provided by the add-on.

http://docs.splunk.com/Documentation/AddOns/released/NGINX/About

Hope this helps. Thanks!
Hunter

0 Karma

svemurilv
Path Finder

Hi ,
here we are not letting nginx to write the default error logs. did some customization to in the code while generating error logs that's why default conf are not working logs for the Nginx

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.