Splunk Search
Highlighted

Need help to auto extract fields in non standard files

Engager

Hello,
I am working on this for a while but i can't make it work correctly. I hope someone can help me to do this
I have multiple log files like bellow. It is not a standard log file.

Thank you very much for your help.

Tags (1)
0 Karma
Highlighted

Re: Need help to auto extract fields in non standard files

Splunk Employee
Splunk Employee

First, make sure you are capturing this as a multiline event and the event boundaries are correct. For the fields, use the below in props.conf, change sourcetypename to match yours...

[sourcetypename]
SHOULD_LINEMERGE = true
EXTRACT-starttime = started at (?P<start_time>\d{2}\:\d{2}:\d{2})\,
EXTRACT-endtime = ended at (?P<end_time>\d{2}\:\d{2}:\d{2})\,
EXTRACT-user_id = User Id\: (?P<user_id>[^\s]+)
EXTRACT-dbname = DB Name: (?P<dbname>[^\s]+)
EXTRACT-instruments_processed = instruments processed\: (?P<instruments_processed>\d+)
EXTRACT-inst_skipped = instruments skipped\: (?P<instruments_skipped>\d+)
EXTRACT-error_nums = Number of errors\: (?P<error_num>\d+)
EXTRACT-warn_nums = Number of warnings\: (?P<warn_num>\d+)
Highlighted

Re: Need help to auto extract fields in non standard files

Splunk Employee
Splunk Employee

You should also set the sourcetype here at your inputs. This will be beneficial down the line for parsing. The sourcetype set here should match the props on the indexer/search head.

View solution in original post

Highlighted

Re: Need help to auto extract fields in non standard files

New Member

Hello esixsplunk,
After I extracted the index=sqltem with the sourcetype=temp-log , i have starttime, endtime, user
id, dbname, instrumentsprocessed, instskipped, error_nums

Then I have another index=jobinf with the sourcetype=jobinfo with the field
jobid
user_id
database
status
jobstarttime
jobfinishtime

As you see only user_id is a common field.

I would like to have a table with combining the fields on both index. How can I do that?

Thank you very much.

0 Karma