topic Use field extractor with a search? in Splunk Search

Use field extractor with a search?

AlexeySh — Tue, 30 Oct 2018 15:36:00 GMT

Hello,

I am wandering to know if there is a way to apply a field extractor not to a source type but to a search.
I’d like to employ a delimited-based field extraction only for specific condition. Like

Sourcetype=xxx fied_1=abcd

Thanks for the help.

Regards,
Alex.

Re: Use field extractor with a search?

kamlesh_vaghela — Tue, 30 Oct 2018 15:41:21 GMT

@AlexeySh

Can you please share more information like sample events and expected results??

Re: Use field extractor with a search?

AlexeySh — Tue, 30 Oct 2018 17:52:35 GMT

Well, basically it’s a Paloalto Traps logs. You can find its log format on Paloalto website. As you can see there are 4 log types and they are slightly different, 1-2 fields more or 1-2 fields less. So you can’t apply field extraction to sourcetype directly, you have to know logs format as well (‘recordType’, the first field).

Unfortunately our Traps logs come to Splunk in a pretty messy format: we have some additional information in the beginning of each event. So we decided to create an independent index and sourcetype for it. By using rex transformation we can extract a “real” value of ‘recordType’ field. But once we have it, we’d like to just use a delimited-based field extraction by comma to extract all other fields for each log type.

Re: Use field extractor with a search?

gaurav_maniar — Thu, 01 Nov 2018 10:19:33 GMT

Hi,

you can use 'rex' command with your query to extract fields at search time and provides fields extraction as well. The only limitation is, it does not provide any delimiter based extraction, you have to write the regex.

rex command reference - https://docs.splunk.com/Documentation/Splunk/7.2.0/SearchReference/Rex

Please accept the answer, it it solves your problem.

Re: Use field extractor with a search?

AlexeySh — Fri, 02 Nov 2018 09:38:24 GMT

Hello @gaurav_maniar,

If we want to extract all fields, the rex command became too complicated and doesn’t work properly (or it could be our lack of skills as well). That’s why we want to use delimited-based field extraction.

Re: Use field extractor with a search?

gaurav_maniar — Fri, 02 Nov 2018 11:51:07 GMT

As your data is already delimited, writing a field extraction with rex command will be very easy and it will work properly if your regex is correct without any problem.

If you go with filed extractor, it will directly apply it to sourcetype and as of now no delimited field extraction is available with rex command.

We can help, if you provide some sample logs

Re: Use field extractor with a search?

AlexeySh — Tue, 29 Sep 2020 21:53:04 GMT

We will really appreciate the help.

Here’s some simple logs, I just modified some private information, like customer ID or domain name.

Nov 2 12:50:14 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] analytics,agent_data,,AgentTimelineEvent,hash,2018-11-02T12:49:45.267329700Z,2018-11-02T12:50:08.656Z,2018-11-02T12:49:45.267329700Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks123,abcdef.fr,,,5.0.3.38921,36-4887,0,7777777777aaaaaaaaaa157092d94eb18c2a73a0a49beeaaaaaaaaaaa30e86a2,dll,,2018-11-02T12:49:45.267329700Z,comdlg32.dll,\?\C:\Windows\SysWOW64\,485888,"{""contentVersion"":""36-4887"",""result"":""Benign"",""trusted"":""None"",""publishers"":[""Microsoft Windows""],""resultId"":0,""trustedId"":0}",0,0,16159

Nov 2 08:59:06 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] threat,threat,,AgentSecurityEvent,2018-11-02T08:16:15.144216600Z,2018-11-02T08:58:55.998Z,2018-11-02T08:16:15.144216600Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks456,abcdef.fr,0,2,5.0.3.38921,36-4887,0,a1866535ef474c2f869865f09x111111,COMPONENT_EPM_J01,ExploitModules,CYSTATUS_JIT_EXCEPTION,,reported,0,,,0,0,"[""CreateProcessA"",""2""]",0,-1,0,"[{""pid"":6952,""parentId"":2724,""exeFileIdx"":0,""userIdx"":0,""commandLine"":""\""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe\"" ""}]","[{""rawFullPath"":""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe"",""fileName"":""firefox.exe"",""sha256"":""70225F14A28007815B0410B1F41F7EA6A16B6329FD69F7EC0638A1A1A1A1A1A1"",""fileSize"":531408,""signers"":[""Mozilla Corporation""]}]","[{""userName"":""user_1234"",""userDomain"":""abcdef.fr""}]",[],Memory Corruption Exploit

The log’s format is described on Paloalto website.

Thanks for the help!