Splunk Search

Setting fields from logs with different row values

Hello everybody,

I have a question that might have been responded before but I have a log file from a server that looks like this

10:01pm load_disk:  200
10:01pm sessions: 2
10:01pm maxpage: 201
10:01pm datadisk: 30
10:01pm memory: 10
10:02pm load_disk:201
10:02pm sessions: 3
10:02pm maxpage: 202
10:02pm datadisk: 31
10:02pm memory: 11
....

As you can see, it is a log file that shows many different values on each row but the values repeat (in this case) every 5 rows. I would like to extract a field for each line that defines a log:
field1 -> load_disk
field2 -> sessions
...

When I try to extract using the Extract Field option in the search, it does not show me all the lines of the log ( this log has a huge number of lines that are different and repeat ) maybe because of window size but I cannot see all the lines of the log (before repeating).

Best regards,

Juan

Tags (2)
0 Karma

Legend

You could do this:

in props.conf

[yoursourcetypehere]
REPORT-eaf=extract-all-fields

in transforms.conf

[extract-all-fields]
FORMAT = $1::$2
REGEX  = (\S+?)\s*:\s*(\S+)

This should give you all the field extractions.

0 Karma

I do not completely understand your question. Do you want to put all fields, in this case load_disk, sessions, maxpage, datadisk and memory on one row?

So you want to gor from your example to something like this:

10:01pm load_disk: 200, sessions: 2, maxpage: 201, datadisk: 30, memory: 10
10:02pm load_disk: 201, sessions: 3, maxpage: 202, datadisk: 31, memory: 11
0 Karma

Hello Tom,

Thank you for your reply and I am sorry for the long silence.

What I actually wanted from the previous log was to set a field for each log variable as follows:

field1 = load_disk
field2 = sessions
field3 = maxpage
field4 = datadisk
field5 = memory

so I can draw a line chart with all these fields inside one chart:

source=/admin/server/* | timechart first(field1) first(field2) avg(field3) first(field4) avg(field5)

My issue now is that the REAL log has more than 80 different variables and they repeat every 10 minutes, so I would like to set a field for each log variable. I saw some usage of the regex editor but I am quite new in Splunk so I would like some advice or if you have had previous experience with such kind of log files

Thank you in advance.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!