How can we add the hostname in an event itself?

[pankajupadhyay]

Hi, I want to rewrite the event based on some keyword in event.

For Example:

Junly 27 10:00:05 UTC IF_DOWN SYSLOG_DAEMON

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

New Log Event:

July 27 Hostname 10:00:0006 IF_DOWN SYSLOG_DAEMON

Can we do on heavyforward by using transoform.conf or props.conf ?/

Kindly help

[thambisetty]

yes, you can use. if the regex is going to concentrate on similar events.

For example : you have different type of events in single sourcetype. if you want to add hostname for one type of event and you want to remove extra line for different event then you cant.  you should have two SEDCMD-classes

[thambisetty]

Note: The below change will add Hostname to events when event matches "xx:xx:xx anyword:"
for example below regex matches below bold characters

2020 Sep 15 09:23:05 UTC: Hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]
2020 Sep 15 09:23:04 UTC: Hostname %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

props.conf ( replace Hostname below with your actual Hostname and yoursourcetype with sourcetype for which you want to add Hostname when regex matches)

[yoursourcetype]
SEDCMD-addhostname=s/(.*\d+:\d+:\d+\s+\w+\:)(.*)/\1 Hostname \2/g

output:

2020 Sep 15 09:23:05 UTC: Hostname  %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]
2020 Sep 15 09:23:04 UTC: Hostname  %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

[pankajupadhyay]

@thambisetty  Thanks for sharing the information.

We have used the SEDCMD for this sourcetype to remove the unwanted event 

Regex:

SEDCMD-remove3 = s/^(?P<cisco>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+\d+:)//g  

Can we use both SEDCMD for same sourcetype. or we can this to achieve the same result which you have shown in output

[thambisetty]

share one complete sample event for which you want to add hostname.

[pankajupadhyay]

@thambisetty 

Hi Please find the below sample logs event

2020 Sep 15 09:23:05 UTC: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]

2020 Sep 15 09:23:04 UTC: %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

[thambisetty]

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

you want to add Hostname in event and forward to respective destination ( you mean different Indexer?)

adding Hostname to all events transforming all events to new event. This is CPU intensive.

share more details to give you better solution.

[pankajupadhyay]

@thambisetty There are two log sources which does not append the hostname in a log even when it forward to HF.

So We are looking to achive with splunk HF and then forward to indexer and third party SIEM to categorize properly.

[pankajupadhyay]

@thambisetty  Yes you have understood correct.

We need to add hostname in the event based on some keyword and then forward it Indexer and third party SIEM tool.

Please help me with the method and solution.

flow would be "

Logs source >>>> HF>>>>>> Indexer and third party collector.

Thanks in advance

[pankajupadhyay]

@thambisetty 

I have used this.

[sourcety]

SEDCMD-addhostname = s/^(?P<ffff>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+:)/\1 hostname1 \2/g

Outputs of above props.conf

Sep 17 16:40:40 10.1.1.2 : hostname 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

I want highlighted section as a output.

But Bold highlighted section need to be delete from log events but it is not happening.

can you please help me where i am wrong in that.

[thambisetty · Accepted solution]

I believe below is your actual event:

Sep 17 16:40:40 10.81.194.72 : 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

and text in bold needs to be removed and hostname needs to be appended after utc: 

use below regex to do above said actions:

s/^\w+\s+\d+\s+\d+:\d+:\d+\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+\:\s+(.*\d+:\d+:\d+\.\d+\s+\w+\:)(.*)/\1 hostname\2/g

output:

2020 Sep 17 14:55:51.485 utc: hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

you can see demo using below Link:

https://regex101.com/r/UylKiP/1

[pankajupadhyay]

@thambisetty  Yeah thanks for your support.

I got the detail but can you please let me know the Regex which i have shared here where i was wrong.

[thambisetty]

@pankajupadhyay 

break your regex and keep verifying one by one character, you will understand where your regex has problem.

use https://regex101.com.