Hi Somesoni2, thanks for your effort on this. Hey since the actual logs are multi line and as per the splunk documents we need to have this configured when we use the DATETIME_CONFIG stanza. Both "CURRENT" and "NONE" explicitly disable the per-text timestamp identification, so the default event boundary detection (BREAK_ONLY_BEFORE_DATE = true) is likely to not work as desired. When using these settings, use SHOULD_LINEMERGE and/or the BREAK_ONLY_* , MUST_BREAK_* settings to control event merging. Props.conf [host::test01] DATETIME_CONFIG = CURRENT SHOULD_LINEMERGE = true MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = Can I use this stanza to get entire log in splunk. ... View more

Hi Hiroshisatoh, thanks for your effort on this, I have asked the application owner to change the log generation method, but I had come across this stanza, in case if there is no data, time in the actual log we can configure the below stanza and pull the entire logs. But I am really confused as the actual logs are multi line and I am not sure where the new events starts and ends in the actual logs. Props.conf [host::test01] DATETIME_CONFIG = CURRENT SHOULD_LINEMERGE = true MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = Can I use this stanza to get entire log in splunk. ... View more

Hi Pradeep, Can you please guide me on this . thanks in advance. ... View more

Hi Somesoni2, Can you please guide me on this. thanks in advance. ... View more

Hi Somesoni2, Can you please guide me on this. As I told you we have other details in props.conf for this app and same app is applied for almost 400+ unix nodes in our environment. App name: Test-IA-unix ( To monitor the file from the source and same app is placed in 400 + nodes) Inputs.conf [monitor:///var/bsm] sourcetype = unix:host:bsm crcSalt = index = unix disabled = 0 We are facing an issue with this node test01 as the actual log data "/var/bsm/2017115.bsm.log" does not have time stamp in it. So I add the stanza in props.conf like this so that configuration is applied only to this host=test01 not to the other hosts. App name: Test-TA-unix ( Technology add-on app is placed in all the indexer instances ) Props.conf host::test01 --> where is the host value for an event. DATETIME_CONFIG = CURRENT "Actual Log from the source file /var/bsm/2017115.bsm.log" - There is no time stamp in it. xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp logout" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" event="su" audit-uid=root uid=root text="success for user mqm" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" Note : Attached partial logs in this comment, but most of the content in the 2017115.bsm.log is the same format. Kindly guide me on this. ... View more

Hi Pradeep, Can you please guide me on this thanks in advance ... View more

Hi Hiroshisatoh, I had executed the below query to get the _time details index=unix host="test01" sourcetype="unix:host:bsm" | table _time source host uid 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 mqm 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 2:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 4:30 /var/bsm/20171115.bsm.log test01 mercator 11/15/2017 4:30 /var/bsm/20171115.bsm.log test01 mercator 11/15/2017 4:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 4:30 /var/bsm/20171115.bsm.log test01 solarwinds 11/15/2017 4:30 /var/bsm/20171115.bsm.log test01 root 11/15/2017 1:30 /var/bsm/20171115.bsm.log test01 mercator 11/15/2017 1:30 /var/bsm/20171115.bsm.log test01 mercator 11/15/2017 1:30 /var/bsm/20171115.bsm.log test01 mercator 11/15/2017 1:30 /var/bsm/20171115.bsm.log test01 mercator From the above result above, when we run the query with the time frame set for 4 hours, we could see hardly few events with interval of 1 hour time difference between the each events in splunk. System time was Wed Nov 15 05:26:26 EST 2017 when this query executed. "Actual Log from the source file /var/bsm/2017115.bsm.log" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp logout" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" event="su" audit-uid=root uid=root text="success for user mqm" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp access" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="login - ssh" audit-uid=mercator uid=mercator tid="14610 22 host01.xxx.com" sid="2978615533" retval="0" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" event="su" audit-uid=root uid=root text="success for user mercator" tid="9195 131094 test02.xxx.com" sid="596162876" xsl_error="no_XSL_match" event="ftp access" event="logout" audit-uid=mercator uid=mercator tid="14610 22 host01.xxx.com" sid="2978615533" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp logout" event="login - ssh" audit-uid=mercator uid=mercator tid="14611 22 host01.xxx.com" sid="2083583931" retval="0" xsl_error="no_XSL_match" event="ftp access" event="logout" audit-uid=mercator uid=mercator tid="14611 22 host01.xxx.com" sid="2083583931" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp logout" Note : Attached partial logs in this comment, but most of the content in the 2017115.bsm.log is the same format. Kindly guide me on how to add the time stamp in the events. ... View more

hi somesoni2, thanks for your effort on this, We have almost 400 + servers configured with the below inputs.conf stanzas via UF and deployed from deployment servers. We have technology add-on placed in all the five Indexer instances. So in this case if I want to apply the below prop.conf stanza details only to this host=test01 when it get indexed in splunk, can I add the stanza like this so that configuration is applied only to this host=test01 not to the other hosts. App name: Test-IA-unix Inputs.conf [monitor:///var/bsm] sourcetype = unix:host:bsm crcSalt = index = unix disabled = 0 Example: App name: Test-TA-unix Props.conf host::test01 --> where is the host value for an event. DATETIME_CONFIG = CURRENT Will that work ? Note : We have already other configuration details already configured in props.conf and deployed in all the indexer. All the indexer are at EST time zone " Tue Nov 14 12:14:55 EST 2017 " Kindly guide me on this ... View more

hi Pradeep, thanks for your effort on this, We have almost 400 + servers configured with the below inputs.conf stanzas via UF and deployed from deployment servers. We have technology add-on placed in all the five Indexer instances. So in this case if I want to apply the below prop.conf stanza details only to this host=test01 when it get indexed in splunk, can I add the stanza like this so that configuration is applied only to this host not to the other hosts. App name: Test-IA-unix Inputs.conf [monitor:///var/bsm] sourcetype = unix:host:bsm crcSalt = index = unix disabled = 0 Example: App name: Test-TA-unix Props.conf host::test01 --> where is the host value for an event. DATETIME_CONFIG = CURRENT Will that work ? Note all the indexer are place at Tue Nov 14 12:14:55 EST 2017 Kindly guide me on this. ... View more

Hi Hiroshisatoh, thanks for your effort on this, Hey I had executed the above query but could not fetch any output. index=unix host=test01 sourcetype="unix:host:bsm" "uid=007" | eval indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S") | table _time indextime When executed without the uid=007 getting an output. Kindly guide me how to set a timestamp when splunk reads the data from the source . ... View more

Hi Hiroshisatoh, thanks for your effort on this, I am not sure whether its possible to add the time stamp in the actual logs. so could you please let me know how to add the time stamp via splunk, actually we have the below inputs.conf stanza configured for almost 400 + nodes. [monitor:///var/bsm] sourcetype = unix:host:bsm crcSalt = index = unix disabled = 0 And also tried to execute the below query in real time All Time but could not fetch the output. "index=unix host=test01 sourcetype="unix:host:bsm" 007" var/bsm/20171113.bsm.log : Log Format with out time stamp. event="ftp logout" xsl_error="no_XSL_match" event="ftp access" event="su" audit-uid=root uid=root text="success for user mqm" tid="9195 131094 test02.xxx.com" sid="59616287 xsl_error="no_XSL_match" event="ftp logout" xsl_error="no_XSL_match" event="ftp access" xsl_error="no_XSL_match" event="ftp access"xsl_error="no_XSL_match"event="ftp logout" event="login - ssh" audit-uid=007 uid=007 tid="15615 22 test02.xxx.com" sid="4094245480" retval="0" event="logout" audit-uid=007 uid=007 tid="15615 22 test02.xxx.com" sid="4094245480" xsl_error="no_XSL_match" event="rsh access" xsl_error="no_XSL_match" Kindly guide me on how to add the time stamp in the events. ... View more

Hi Garethatiag, the issue got fixed after implementing the below stanza in the inputs.conf. [monitor:///var/bsm] sourcetype = unix:host:bsm index = unix crcSalt = disabled = 0 ... View more

Hi All, We are able to get the qualys data in to splunk, after re-installing / re -configuring the app completely. We have update the TA-Qualys Add-on with the latest version 1.2.3. ... View more

Hi Kunalmao, Good Evening, thanks for your effort on this, currently this is the permission level set for the splunk.key. $SPLUNK_HOME/var/lib/splunk/kvstore/mongo/splunk.key -rw------- 1 splunk splunk 88 May 16 09:49 splunk.key Hey I have gone through the link provide by you as per that we have only 3 cluster member in our environment. So we are in the odd number, I hope there is no need to remove the cluster member. And I have another question. 1) DO we need to execute the below command on all the search head member to clear the content inside the kvstore/mongo. Or is it good to execute only in the affected search head member. ./splunk clean kvstore --local 2) Will there be any impact if we are going to clean/removing the content present in the kvstore/mongo. As this is the first time, I am coming across this type of issue. Kindly guide me on this. thanks in advance. ... View more

Hi Alemarzu, Good Morning, I have changed the sourcetype to f5:bigip:syslog and we could see the data getting into splunk. But could please guide me on the below statement. Before pushing the app to the search head cluster member. In our case the splunk Add-on for F5 BIG-IP - Is going to be pushed into the search head cluster member and as per the document we need to remove the inputs.conf , eventgen.conf files and all files in the Samples folder. 1) appserver, lookups and statics folder should be pushed along with the app or we can remove it from the package as we are not going to use the irule for LTM, modular inputs and lookups related to this. 2) Similarly under default folder, props.conf, transforms.conf , tags.conf and events.conf contains lots of stanza details which is not going to applicable for us, so in that case, can we remove or ## it out the other stanzas, Keeping only the stanza related to f5_bigip:syslog General [f5_bigip:syslog] rename = f5:bigip:syslog 3) I am not sure what is purpose of the rename = ? Thanks in advance. ... View more

Hi Alemarzu, Good Morning, could you please guide me on the above mentioned comments. thanks in advance. ... View more

Hi Somesoni2, Good Morning, As I had mentioned in the above comment, How will the splunk understands that first set of three hosts are from Malaysia, second set of hosts are from EST and third set of hosts are from CST location as all the hosts details are mentioned continuously in same props files. Though all the hosts have different host name/IP address. OR Do we need to mention the TZ stanza in each and every host name individually for 200 + host names mentioned in props.conf as shown below. Example: Plant from Malaysia ( First 3 different host ) Props.conf Host in Malaysia [host::10.X.X.X] TRANSFORMS-guard_rename = guard_rename_Host1 TZ = GMT -7 #Host in EST [host::10.X.X.X] TRANSFORMS-guard_rename = guard_rename_Host4 TZ = EST Host in CST [host::10.X.X.X] TRANSFORMS-guard_rename = guard_rename_Host7 TZ = CST Another important doubt to be clarified, that all our splunk instance are at EDT time zone, so should we need to adjust anything for day light saving in the above stanza ? Kindly guide me on this. ... View more

Hi alemarzu, thanks for your effort on this, Hey got the approval to change the sourcetype from sourcetype = f5:network:loadbalancer to sourcetype = f5:bigip:syslog as per the Splunk Add on. We have inputs.conf configured in a customized app "Test-IA-F5" as per our environment and placed in the HF instances to monitor the data. Only change to this app is the sourcetype. I have final question to you before pushing the changes in Prod Environment. In our case the splunk Add-on for F5 BIG-IP - Is going to be pushed into the search head cluster member and as per the document we need to remove the inputs.conf , eventgen.conf files and all files in the Samples folder. 1) appserver, lookups and statics folder should be pushed along with the app or we can remove it from the package as we are not going to use the irule for LTM, modular inputs and lookups related to this. 2) Similarly under default folder, props.conf, transforms.conf , tags.conf and events.conf contains lots of stanza details which is not going to applicable for us, so in that case, can we remove or ## it out the other stanzas, Keeping only the stanza related to f5_bigip:syslog General [f5_bigip:syslog] rename = f5:bigip:syslog 3) I am not sure what is purpose of the rename = ? Kindly advise me on this please. Thanks in advance. ... View more

Hi Somesoni2, thanks for your effort, as I had commented earlier, that we have Test-TA-mguard configured with props & transforms and all the plant data from different locations are filtered based on setting per-event Host name. Example: Plant from Malaysia Props.conf [host::10.X.X.X] TRANSFORMS-guard_rename = guard_rename_Host1 Next set of host from EST location configured in same props.conf file, filtered by their host name [host::10.X.X.X] TRANSFORMS-guard_rename = guard_rename_Host4 TZ = EST Lilke wise we have 13 different plant locations host are mentioned in the same props.conf files. And we have almost 200 + host from these 13 location sending the data. Question : How will the splunk understands that first set of three hosts are from Malaysia and second set of hosts are from EST location as all the hosts details are mentioned continuously in same props files. Though all the hosts have different host name/IP address. thanks in advance. ... View more

Hi Somesoni2 thanks for your quick response on this, hey all plant are separated with specific host name and IP address, we use the below query to filter each plant based on the host. Example: index=firewall sourcetype=guard:network:firewall host="elspc0*" 11/3/17 1:04:28.000 PM Nov 3 13:04:28 10.15.13.240 2017-11-03_17:04:28.02540 <13>Nov 3 12:04:28 memlog: 1947 root 2976 S /Packages/ssh_0/sbin/sshd -D -f /etc/sshd/sshd_config host = elspc001 source = /opt/syslogs/guard/10.x.x.x/guard.log sourcetype = guard:network:firewall likewise for each plant contains different set of host name sending the firewall data to HF instances. As I had stated earlier in my comment, that Props.conf and transforms.conf are configured based on setting per-event Host name concept. Kindly guide me how to adjust the time zone based on the plant location. thanks in advance. ... View more