Archive

Log transformation

srajanbabu
Explorer

I have below requirement, some one Please guide how do I achieve using Splunk

I have host called SourceHost where log files are generated and I have another host called TargetHost where I have to transform the source log and write.

Source log
SNM5 YAHOO3SN.#### :: 03/03/13 00:00:07 :: B ::User yahoo3sn logged in
SNM4 YAHOO3SN.871F :: 03/03/13 00:00:07 :: S ::User logged off, Processing will begin

Target log
Buy | 13/03/03 | YAHOO3SN |03/03/13 00:00:07 | User yahoo3sn logged in
Sell | 13/03/03 | SNM4 |03/03/13 00:00:07 | logged off, Processing will begin

where
1. B stands for buy and S stand for sell
2. Soruce date is mm/dd/yy and target date is yy/mm/dd
3. if third word in the log is "####" then pickup the second word otherwise pickup the first word.

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Something like this should do the job.

... | rex "(?<field1>.*?)\s(?<field2>.*?)\.(?<field3>.*?)\s::(?<mon>.*?)/(?<day>.*?)/(?<year>.*?)\s(?<time>.*?)\s::\s(?<BS>.*?)\s::(?<msg>.*)" | eval trade=case(BS="B","Buy",BS="S","Sell") | eval output3=if(field3="####",field2,field1) | eval targetLog=trade." | ".year."/".mon."/".day." ".time." | ".msg | ...

---
If this reply helps you, an upvote would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

You won't be able to have Splunk process the log without indexing it. You should be able to make the transformation happen on input by adding a similar REGEX statement to your transforms.conf file.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

srajanbabu
Explorer

Thanks for the swift response, however I have few concerns.
I assume, this regex will be performed on indexed file, I would like this tranformation happend as I read the input file and If possible i want to tranfer the data to target host without a footprint in splunk server.

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!