Splunk Search

Use field extractor with a search?

Communicator

Hello,

I am wandering to know if there is a way to apply a field extractor not to a source type but to a search.
I’d like to employ a delimited-based field extraction only for specific condition. Like

Sourcetype=xxx fied_1=abcd

Thanks for the help.

Regards,
Alex.

0 Karma

Hi,

you can use 'rex' command with your query to extract fields at search time and provides fields extraction as well. The only limitation is, it does not provide any delimiter based extraction, you have to write the regex.

rex command reference - https://docs.splunk.com/Documentation/Splunk/7.2.0/SearchReference/Rex

Please accept the answer, it it solves your problem.

0 Karma

Communicator

Hello @gaurav_maniar,

If we want to extract all fields, the rex command became too complicated and doesn’t work properly (or it could be our lack of skills as well). That’s why we want to use delimited-based field extraction.

0 Karma

As your data is already delimited, writing a field extraction with rex command will be very easy and it will work properly if your regex is correct without any problem.

If you go with filed extractor, it will directly apply it to sourcetype and as of now no delimited field extraction is available with rex command.

We can help, if you provide some sample logs

0 Karma

Communicator

We will really appreciate the help.

Here’s some simple logs, I just modified some private information, like customer ID or domain name.

Nov 2 12:50:14 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] analytics,agent_data,,AgentTimelineEvent,hash,2018-11-02T12:49:45.267329700Z,2018-11-02T12:50:08.656Z,2018-11-02T12:49:45.267329700Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks123,abcdef.fr,,,5.0.3.38921,36-4887,0,7777777777aaaaaaaaaa157092d94eb18c2a73a0a49beeaaaaaaaaaaa30e86a2,dll,,2018-11-02T12:49:45.267329700Z,comdlg32.dll,\?\C:\Windows\SysWOW64\,485888,"{""contentVersion"":""36-4887"",""result"":""Benign"",""trusted"":""None"",""publishers"":[""Microsoft Windows""],""resultId"":0,""trustedId"":0}",0,0,16159


Nov 2 08:59:06 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] threat,threat,,AgentSecurityEvent,2018-11-02T08:16:15.144216600Z,2018-11-02T08:58:55.998Z,2018-11-02T08:16:15.144216600Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks456,abcdef.fr,0,2,5.0.3.38921,36-4887,0,a1866535ef474c2f869865f09x111111,COMPONENT_EPM_J01,ExploitModules,CYSTATUS_JIT_EXCEPTION,,reported,0,,,0,0,"[""CreateProcessA"",""2""]",0,-1,0,"[{""pid"":6952,""parentId"":2724,""exeFileIdx"":0,""userIdx"":0,""commandLine"":""\""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe\"" ""}]","[{""rawFullPath"":""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe"",""fileName"":""firefox.exe"",""sha256"":""70225F14A28007815B0410B1F41F7EA6A16B6329FD69F7EC0638A1A1A1A1A1A1"",""fileSize"":531408,""signers"":[""Mozilla Corporation""]}]","[{""userName"":""user_1234"",""userDomain"":""abcdef.fr""}]",[],Memory Corruption Exploit


The log’s format is described on Paloalto website.

Thanks for the help!

0 Karma

SplunkTrust
SplunkTrust

@AlexeySh

Can you please share more information like sample events and expected results??

0 Karma

Communicator

Well, basically it’s a Paloalto Traps logs. You can find its log format on Paloalto website. As you can see there are 4 log types and they are slightly different, 1-2 fields more or 1-2 fields less. So you can’t apply field extraction to sourcetype directly, you have to know logs format as well (‘recordType’, the first field).

Unfortunately our Traps logs come to Splunk in a pretty messy format: we have some additional information in the beginning of each event. So we decided to create an independent index and sourcetype for it. By using rex transformation we can extract a “real” value of ‘recordType’ field. But once we have it, we’d like to just use a delimited-based field extraction by comma to extract all other fields for each log type.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!