Splunk Search

Use field extractor with a search?

AlexeySh
Communicator

Hello,

I am wandering to know if there is a way to apply a field extractor not to a source type but to a search.
I’d like to employ a delimited-based field extraction only for specific condition. Like

Sourcetype=xxx fied_1=abcd

Thanks for the help.

Regards,
Alex.

0 Karma

gaurav_maniar
Builder

Hi,

you can use 'rex' command with your query to extract fields at search time and provides fields extraction as well. The only limitation is, it does not provide any delimiter based extraction, you have to write the regex.

rex command reference - https://docs.splunk.com/Documentation/Splunk/7.2.0/SearchReference/Rex

Please accept the answer, it it solves your problem.

0 Karma

AlexeySh
Communicator

Hello @gaurav_maniar,

If we want to extract all fields, the rex command became too complicated and doesn’t work properly (or it could be our lack of skills as well). That’s why we want to use delimited-based field extraction.

0 Karma

gaurav_maniar
Builder

As your data is already delimited, writing a field extraction with rex command will be very easy and it will work properly if your regex is correct without any problem.

If you go with filed extractor, it will directly apply it to sourcetype and as of now no delimited field extraction is available with rex command.

We can help, if you provide some sample logs

0 Karma

AlexeySh
Communicator

We will really appreciate the help.

Here’s some simple logs, I just modified some private information, like customer ID or domain name.

Nov 2 12:50:14 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] analytics,agent_data,,AgentTimelineEvent,hash,2018-11-02T12:49:45.267329700Z,2018-11-02T12:50:08.656Z,2018-11-02T12:49:45.267329700Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks123,abcdef.fr,,,5.0.3.38921,36-4887,0,7777777777aaaaaaaaaa157092d94eb18c2a73a0a49beeaaaaaaaaaaa30e86a2,dll,,2018-11-02T12:49:45.267329700Z,comdlg32.dll,\?\C:\Windows\SysWOW64\,485888,"{""contentVersion"":""36-4887"",""result"":""Benign"",""trusted"":""None"",""publishers"":[""Microsoft Windows""],""resultId"":0,""trustedId"":0}",0,0,16159


Nov 2 08:59:06 sc-1456400473-logforwardercomp-5bce225a53cefb004074a882-59fm5sn logforwarder[24] threat,threat,,AgentSecurityEvent,2018-11-02T08:16:15.144216600Z,2018-11-02T08:58:55.998Z,2018-11-02T08:16:15.144216600Z,60,,TrapsAgent,1111111173,6857076101111111111,coreop-f-prodb2-mnmauto123123123123-1234.prod.brz,2.0.6,70,1,6aaaaaaaaaaaa5da86ada7b4c6b01504,1,0,6.1.7601,1,123.123.123.123,wks456,abcdef.fr,0,2,5.0.3.38921,36-4887,0,a1866535ef474c2f869865f09x111111,COMPONENT_EPM_J01,ExploitModules,CYSTATUS_JIT_EXCEPTION,,reported,0,,,0,0,"[""CreateProcessA"",""2""]",0,-1,0,"[{""pid"":6952,""parentId"":2724,""exeFileIdx"":0,""userIdx"":0,""commandLine"":""\""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe\"" ""}]","[{""rawFullPath"":""C:\Users\user_1234\AppData\Abcdabcd\aaaaaaaaaaaaaa\firefox.exe"",""fileName"":""firefox.exe"",""sha256"":""70225F14A28007815B0410B1F41F7EA6A16B6329FD69F7EC0638A1A1A1A1A1A1"",""fileSize"":531408,""signers"":[""Mozilla Corporation""]}]","[{""userName"":""user_1234"",""userDomain"":""abcdef.fr""}]",[],Memory Corruption Exploit


The log’s format is described on Paloalto website.

Thanks for the help!

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@AlexeySh

Can you please share more information like sample events and expected results??

0 Karma

AlexeySh
Communicator

Well, basically it’s a Paloalto Traps logs. You can find its log format on Paloalto website. As you can see there are 4 log types and they are slightly different, 1-2 fields more or 1-2 fields less. So you can’t apply field extraction to sourcetype directly, you have to know logs format as well (‘recordType’, the first field).

Unfortunately our Traps logs come to Splunk in a pretty messy format: we have some additional information in the beginning of each event. So we decided to create an independent index and sourcetype for it. By using rex transformation we can extract a “real” value of ‘recordType’ field. But once we have it, we’d like to just use a delimited-based field extraction by comma to extract all other fields for each log type.

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...