topic Re: How we can add the hostname in event itself in Splunk Enterprise

How can we add the hostname in an event itself?

pankajupadhyay — Fri, 18 Sep 2020 22:54:03 GMT

Hi, I want to rewrite the event based on some keyword in event.

For Example:

Junly 27 10:00:05 UTC IF_DOWN SYSLOG_DAEMON

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

New Log Event:

July 27 Hostname 10:00:0006 IF_DOWN SYSLOG_DAEMON

Can we do on heavyforward by using transoform.conf or props.conf ?/

Kindly help

Re: How we can add the hostname in event itself

thambisetty — Thu, 17 Sep 2020 08:27:53 GMT

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

you want to add Hostname in event and forward to respective destination ( you mean different Indexer?)

adding Hostname to all events transforming all events to new event. This is CPU intensive.

share more details to give you better solution.

Re: How we can add the hostname in event itself

pankajupadhyay — Thu, 17 Sep 2020 09:35:13 GMT

@thambisetty Yes you have understood correct.

We need to add hostname in the event based on some keyword and then forward it Indexer and third party SIEM tool.

Please help me with the method and solution.

flow would be "

Logs source >>>> HF>>>>>> Indexer and third party collector.

Thanks in advance

Re: How we can add the hostname in event itself

pankajupadhyay — Thu, 17 Sep 2020 10:37:54 GMT

@thambisetty There are two log sources which does not append the hostname in a log even when it forward to HF.

So We are looking to achive with splunk HF and then forward to indexer and third party SIEM to categorize properly.

Re: How we can add the hostname in event itself

thambisetty — Thu, 17 Sep 2020 10:42:38 GMT

share one complete sample event for which you want to add hostname.

Re: How we can add the hostname in event itself

pankajupadhyay — Thu, 17 Sep 2020 10:52:13 GMT

@thambisetty

Hi Please find the below sample logs event

2020 Sep 15 09:23:05 UTC: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]

2020 Sep 15 09:23:04 UTC: %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

Re: How we can add the hostname in event itself

thambisetty — Thu, 17 Sep 2020 11:26:09 GMT

Note: The below change will add Hostname to events when event matches "xx:xx:xx anyword:"
for example below regex matches below bold characters

2020 Sep 15 09:23:05 UTC: Hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]
2020 Sep 15 09:23:04 UTC: Hostname %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

props.conf ( replace Hostname below with your actual Hostname and yoursourcetype with sourcetype for which you want to add Hostname when regex matches)

[yoursourcetype] SEDCMD-addhostname=s/(.*\d+:\d+:\d+\s+\w+\:)(.*)/\1 Hostname \2/g

output:

2020 Sep 15 09:23:05 UTC: Hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425] 2020 Sep 15 09:23:04 UTC: Hostname %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

Re: How we can add the hostname in event itself

pankajupadhyay — Thu, 17 Sep 2020 13:42:04 GMT

@thambisetty Thanks for sharing the information.

We have used the SEDCMD for this sourcetype to remove the unwanted event

Regex:

SEDCMD-remove3 = s/^(?P<cisco>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+\d+:)//g

Can we use both SEDCMD for same sourcetype. or we can this to achieve the same result which you have shown in output

Re: How we can add the hostname in event itself

thambisetty — Thu, 17 Sep 2020 13:48:03 GMT

yes, you can use. if the regex is going to concentrate on similar events.

For example : you have different type of events in single sourcetype. if you want to add hostname for one type of event and you want to remove extra line for different event then you cant. you should have two SEDCMD-classes

Re: How we can add the hostname in event itself

pankajupadhyay — Fri, 18 Sep 2020 09:01:37 GMT

@thambisetty

I have used this.

[sourcety]

SEDCMD-addhostname = s/^(?P<ffff>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+:)/\1 hostname1 \2/g

Outputs of above props.conf

Sep 17 16:40:40 10.1.1.2 : hostname 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

I want highlighted section as a output.

But Bold highlighted section need to be delete from log events but it is not happening.

can you please help me where i am wrong in that.

Re: How we can add the hostname in event itself

thambisetty — Fri, 18 Sep 2020 08:01:00 GMT

I believe below is your actual event:

Sep 17 16:40:40 10.81.194.72 : 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

and text in bold needs to be removed and hostname needs to be appended after utc:

use below regex to do above said actions:

s/^\w+\s+\d+\s+\d+:\d+:\d+\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+\:\s+(.*\d+:\d+:\d+\.\d+\s+\w+\:)(.*)/\1 hostname\2/g

output:

2020 Sep 17 14:55:51.485 utc: hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

you can see demo using below Link:

https://regex101.com/r/UylKiP/1

Re: How we can add the hostname in event itself

pankajupadhyay — Fri, 18 Sep 2020 08:59:52 GMT

@thambisetty Yeah thanks for your support.

I got the detail but can you please let me know the Regex which i have shared here where i was wrong.

Re: How we can add the hostname in event itself

thambisetty — Fri, 18 Sep 2020 09:04:10 GMT

@pankajupadhyay

break your regex and keep verifying one by one character, you will understand where your regex has problem.

use https://regex101.com.