Splunk Enterprise

How can we add the hostname in an event itself?

pankajupadhyay
Path Finder

Hi, I want to rewrite the event based on some keyword in event.

For Example:

Junly 27 10:00:05 UTC IF_DOWN SYSLOG_DAEMON

 

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

 

New Log Event:

July 27 Hostname 10:00:0006 IF_DOWN SYSLOG_DAEMON

 

Can we do on heavyforward by using transoform.conf or props.conf ?/

 

Kindly help

Labels (1)
Tags (2)
0 Karma
1 Solution

thambisetty
SplunkTrust
SplunkTrust

I believe below is your actual event:

Sep 17 16:40:40 10.81.194.72 : 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

and text in bold needs to be removed and hostname needs to be appended after utc: 

use below regex to do above said actions:

s/^\w+\s+\d+\s+\d+:\d+:\d+\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+\:\s+(.*\d+:\d+:\d+\.\d+\s+\w+\:)(.*)/\1 hostname\2/g

output:

2020 Sep 17 14:55:51.485 utc: hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

you can see demo using below Link:

https://regex101.com/r/UylKiP/1

————————————
If this helps, give a like below.

View solution in original post

thambisetty
SplunkTrust
SplunkTrust

yes, you can use. if the regex is going to concentrate on similar events.

For example : you have different type of events in single sourcetype. if you want to add hostname for one type of event and you want to remove extra line for different event then you cant.  you should have two SEDCMD-classes

————————————
If this helps, give a like below.
0 Karma

thambisetty
SplunkTrust
SplunkTrust

Note: The below change will add Hostname to events when event matches "xx:xx:xx anyword:"
for example below regex matches below bold characters

2020 Sep 15 09:23:05 UTC: Hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]
2020 Sep 15 09:23:04 UTC: Hostname %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

props.conf ( replace Hostname below with your actual Hostname and yoursourcetype with sourcetype for which you want to add Hostname when regex matches)

[yoursourcetype]
SEDCMD-addhostname=s/(.*\d+:\d+:\d+\s+\w+\:)(.*)/\1 Hostname \2/g

output:

2020 Sep 15 09:23:05 UTC: Hostname  %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]
2020 Sep 15 09:23:04 UTC: Hostname  %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

 

————————————
If this helps, give a like below.
0 Karma

pankajupadhyay
Path Finder

@thambisetty  Thanks for sharing the information.

We have used the SEDCMD for this sourcetype to remove the unwanted event 

 

Regex:

SEDCMD-remove3 = s/^(?P<cisco>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+\d+:)//g  

 

Can we use both SEDCMD for same sourcetype. or we can this to achieve the same result which you have shown in output

0 Karma

thambisetty
SplunkTrust
SplunkTrust

share one complete sample event for which you want to add hostname.

————————————
If this helps, give a like below.
0 Karma

pankajupadhyay
Path Finder

@thambisetty 

Hi Please find the below sample logs event


2020 Sep 15 09:23:05 UTC: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user user1 from 10.1.1.1 - sshd[16425]

2020 Sep 15 09:23:04 UTC: %AUTHPRIV-3-SYSTEM_MSG: pam_aaa:Authentication failed for user user2 from 10.1.1.1 - sshd[16427]

 

 

0 Karma

thambisetty
SplunkTrust
SplunkTrust

So if i match SYSLOG from the event and add field in event on Heavy forwarder to send the logs to res pective destination.

you want to add Hostname in event and forward to respective destination ( you mean different Indexer?)

adding Hostname to all events transforming all events to new event. This is CPU intensive.

share more details to give you better solution.

————————————
If this helps, give a like below.
0 Karma

pankajupadhyay
Path Finder

@thambisetty There are two log sources which does not append the hostname in a log even when it forward to HF.

 

So We are looking to achive with splunk HF and then forward to indexer and third party SIEM to categorize properly.

0 Karma

pankajupadhyay
Path Finder

@thambisetty  Yes you have understood correct.

We need to add hostname in the event based on some keyword and then forward it Indexer and third party SIEM tool.

 

Please help me with the method and solution.

flow would be "

Logs source >>>> HF>>>>>> Indexer and third party collector.

Thanks in advance

0 Karma

pankajupadhyay
Path Finder

@thambisetty 

I have used this.

[sourcety]

SEDCMD-addhostname = s/^(?P<ffff>\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+:)/\1 hostname1 \2/g

Outputs of above props.conf

Sep 17 16:40:40 10.1.1.2 : hostname 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

 

I want highlighted section as a output.

But Bold highlighted section need to be delete from log events but it is not happening.

 

can you please help me where i am wrong in that.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

I believe below is your actual event:

Sep 17 16:40:40 10.81.194.72 : 2020 Sep 17 14:55:51.485 utc: %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

and text in bold needs to be removed and hostname needs to be appended after utc: 

use below regex to do above said actions:

s/^\w+\s+\d+\s+\d+:\d+:\d+\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+\:\s+(.*\d+:\d+:\d+\.\d+\s+\w+\:)(.*)/\1 hostname\2/g

output:

2020 Sep 17 14:55:51.485 utc: hostname %DAEMON-3-SYSTEM_MSG: error: PAM: Authentication failure for illegal user jhsfhefje from 10.1.1.1- sshd[3475]

you can see demo using below Link:

https://regex101.com/r/UylKiP/1

————————————
If this helps, give a like below.

pankajupadhyay
Path Finder

@thambisetty  Yeah thanks for your support.

I got the detail but can you please let me know the Regex which i have shared here where i was wrong.

 

 

0 Karma

thambisetty
SplunkTrust
SplunkTrust

@pankajupadhyay 

break your regex and keep verifying one by one character, you will understand where your regex has problem.

use https://regex101.com.

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...