About bhargavi

bhargavi · ‎01-30-2025

Hi @woodcock , Thanks for this piece of info. I am facing the same issue right now. As per this documentation 'https://github.com/splunk/splunk-connect-for-syslog/blob/main/docs/gettingstarted/quickstart_guide.md' , we can either use docker or podman to get this started. Since podman was already installed in my linux machine, I chose it. After updating sc4s.service for podman, I am trying to reload deamon and restart sc4s, but syslog service isn't starting. When executing 'journalctl -xe', I am seeing the below error. Trying to pull ghcr.io/splunk/splunk-connect-for-syslog/container3:latest... podman[147844]:Error: initializing source docker://ghcr.io/splunk/splunk-connect-for-syslog/container3:latest: pinging container registry ghcr.io: Get "https://ghcr.io/v2/": dial tcp: lookup ghcr.io on 192.124.60.53:53: no such host Could you please assist here , am I missing anything?

bhargavi · ‎03-18-2022

Hi all, Please help with the below. I am using rlog.sh (inbuilt script) provided by Splunk in TA-unix package , to apply ausearch utility for linux audit logs. SEEK_FILE=$SPLUNK_HOME/var/run/splunk/unix_audit_seekfile_model_prod AUDIT_FILE=/opt/splunklogs_app/audit_prod/audit.log if [ -e $SEEK_FILE ] ; then SEEK=`head -1 $SEEK_FILE` else SEEK=0 echo "0" > $SEEK_FILE fi FILE_LINES=`wc -l $AUDIT_FILE | cut -d " " -f 1` if [ $FILE_LINES -lt $SEEK ] ; then # audit file has wrapped SEEK=0 fi awk -v START=$SEEK -v OUTPUT=$SEEK_FILE 'NR>START { print } END { print NR > OUTPUT }' $AUDIT_FILE | tee $TEE_DEST | /sbin/ausearch -i 2>/dev/null | grep -v "^----" This inbuilt script is converting default format of linux audit logs by applying ausearch utility. example below: Log input : type=TTY msg=audit(1647315634.249:442): tty pid=2962 uid=0 auid=1001 ses=1368 major=136 minor=0 comm="bash" data=7669202F6574632F727375737F7F79730963090D Log output after using rlog.sh type=TTY msg=audit(03/15/2022 14:40:34.791:2962): tty pid=2962 uid=root auid=root ses=1368 major=136 minor=0 comm="bash" data=7669202F6574632F727375737F7F79730963090D Now I have audit.log being generated in a different format.. like below.. My audit.log: (custom audit.log format) IP: 10.200.30.40 | <158>Mar 11 16:10:24 xxx-yyy-zzz AuditLog type=SYSCALL msg=audit(1646979024.027:1697): arch=c000003e syscall=4 success=yes exit=0 a0=7f1304042410 a1=7f13092f66a0 a2=7f13092f66a0 a3=0 items=1 ppid=1 pid=2270 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="in:imfile" exe="/usr/sbin/rsyslogd" subj=system_u:system_r:syslogd_t:s0 key=(null) Hostname=10.200.30.40 (default audit.log format) type=USER_TTY msg=audit(1646592289.268:441): pid=2962 uid=0 auid=1001 ses=1368 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 data=73797374656D63746C2073746174757320727379736C6F67 Hostname=xxx so basically I will have logs of both default audit log format and this custom format being logged in audit.log. When I apply the rlog.sh/ausearch utility to this log, only logs with default audit.log type are being converted with ausearch utility and sent to output and indexed, the other logs are not even being sent to output. Please help.

bhargavi · ‎11-18-2021

Thank you very much for the reply @richgalloway I tried updating regex in the below pattern mentioned by you,but still receiving error messages in splunkd logs.

bhargavi · ‎11-17-2021

Hi @mlevsh Try this for preamble_regex. It works. PREAMBLE_REGEX=# If this helps, give thumbs-up 🙂 Happy Splunking!!

bhargavi · ‎11-17-2021

Hey, Try these troubleshooting steps. https://www.learnsplunk.com/splunk-forwarder-not-sending-data.html

bhargavi · ‎11-17-2021

Hi @richgalloway My goal to filter the events according to the VM types (like db, ops,telemetry,etc) and log file combinations and route it to different indexes. For example, /opt/tomcat/server/cm.log might be coming from all type of VM's (around 12 VM types; VNFCs mentioned in bracket), but I want to filter the events only from few VM types, like opslb,ntfsync,telemetry,diagnostic and index them and ignore logs from other VM types. VM name pattern would be like this as mentioned in the sample events : xyz002vm002400-db-vm01.xxxx.com.yy A glimpse of transforms.conf [security] REGEX=(\VNFCs\"\:\".*(ops|opslb|ntf|ntfsync|telemetry|diagnostic|db|lb).*class\"\:\"(/var/log/auth.log|/var/log/messages.log|/var/log/syslog.log|/var/log/firewall/firewall.log|/var/log/audit/audit.log)\")|(\VNFCs\"\:\".*(ops|opslb|ntf).*class\"\:\"(/opt/function/application.log)\")|(\VNFCs\"\:\".*(opslb|ntfsync|telemetry|diagnostic).*class\"\:\"(/opt/log/local1.log|/opt/tomcat/server/cm.log)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=60000 Sample Event 1: {"VNFType":"xyz","VNFs":"xyz002vm002400","VNFCType":"db","VNFCs":"xyz002vm002400-db-vm01.xxxx.com.yy","event":{"log_message":"2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01.xxxx.com.yy, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01.xxxx.com.yy, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01.xxxx.com.yy, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n","class":"/var/log/audit.log","log_event_time_stamp":"2021-11-18T02:45:08+10:00"}} Sample Event 2: {"VNFType":"xyz","VNFs":"abc002vm002400","VNFCType":"telemetry","VNFCs":"abc002vm002400-telemetry-vm01.xxxx.com.yy","event":{"log_message":"2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01.xxxx.com.yy, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01.xxxx.com.yy, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01.xxxx.com.yy, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, abc002vm002400, abc002vm002400-telemetry-vm01.xxxx.com.yy, telemetry, telemetry, info, cron, abc002vm002400-telemetry-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n","class":"/opt/log/local1.log","log_event_time_stamp":"2021-11-18T02:45:08+10:00"}}

bhargavi · ‎11-17-2021

Hello all, kindly help with Regex.. I am seeing the below messages in splunkd logs. Though values are actually being extracted properly, below messages are annoying and I want to get rid of those. Need urgent help to construct a better Regex to avoid these messages. I have increased MATCHLIMIT in transforms.conf, but still seeing these messages. 11-17-2021 17:12:34.927 +1100 ERROR Regex - Failed in pcre_exec: Error PCRE_ERROR_MATCHLIMIT for regex 11-17-2021 17:12:34.927 +1100 WARN regexExtractionProcessor - Regular expression for stanza security_index exceeded configured PCRE match limit. One or more fields might not have their values extracted, which can lead to incorrect search results. Fix the regular expression to improve search performance or increase the MATCH_LIMIT in props.conf to include the missing field extractions. Part of my regex looks like this. I have repeated the same regex corresponding to different VM and file combinations. Can someone please help with a better Regex. REGEX=(\VNFCs\"\:\".*(ops|opslb|ntf|ntfsync|telemetry|diagnostic|db|lb).*class\"\:\"(/var/log/auth.log|/var/log/messages.log|/var/log/syslog.log|/var/log/firewall/firewall.log|/var/log/audit/audit.log)\").. followed by \VNFCs\"\:\".*(ops|opslb|ntf).*class\"\:\"(/opt/function/applicatio.log)\").. etc... My Sample event is as below. There are more than 10 type of VM's (like ops,db,sync,etc), more than 300 VM's and 250 different files and I made a Single regex. I need to create regex considering below two criteria: 1) The VM's : "xyz001vm002400-ops-vm01","xyz002vm002400-db-vm01", etc. 2) Log file beginning with a class keyword : "class":"/var/log/syslog.log , "class":"/opt/cat/audit.log , etc. Sample Event 1: {"VNFType":"xyz","VNFs":"xyz001vm002400","VNFCType":"ops","VNFCs":"xyz001vm002400-ops-vm01","event":{"log_message":"2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz001vm002400, xyz001vm002400-ops-vm01, ops, ops, info, cron, xyz001vm002400-ops-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n","class":"/var/log/syslog.log","log_event_time_stamp":"2021-11-18T02:45:08+10:00"}} Sample Event 2: {"VNFType":"xyz","VNFs":"xyz002vm002400","VNFCType":"db","VNFCs":"xyz002vm002400-db-vm01","event":{"log_message":"2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n2021-11-18T02:45:01.085777+10:00, xyz002vm002400, xyz002vm002400-db-vm01, db, db, info, cron, xyz002vm002400-db-vm01, CROND[16311]:, (root) CMD (/usr/sbin/KillIdleSessions)\n","class":"/opt/cat/audit.log","log_event_time_stamp":"2021-11-18T02:45:08+10:00"}} @chrisyounger @harsmarvania57 - Any help is much appreciated.

bhargavi · ‎11-15-2021

Hi @Azwaliyana , Try this. | rex field=Description "(?<Country>\w+)\s(?<Node>\w*\-?\w+)"

bhargavi · ‎11-15-2021

Hi @zoebanning , Chart command does not allow more than 2 fields in by clause Timechart command does not allow more than 1 field in by clause So here I have tried to display the possible values of account and action and merged them into 1 column, then used timechart command . If this helps, give a thumbs-up 🙂 Happy Splunking!!

bhargavi · ‎11-14-2021

Hi @Noobsplunker , Would you want to extract any field containing URL details from your events? If yes, please post the sample events here and I can help in extraction.

bhargavi · ‎11-14-2021

Hi @lana0203 1. Can you check if logs are being generated on your VM/desktop after 08/11/2021. If there are no logs being generated, then your index will not have any data. 2. Are you seeing in splunkd logs if the logs are being watched? .. you would find log line as 'Adding Watch on path "... your log. 3. Is splunk running fine on your VM? You can try restarting splunkd on your VM and checked if logs are being pushed.

bhargavi · ‎11-14-2021

Hi @hrs2019 Try this. | rex "Name=(?P<Name>\w+\,\s\w+)"

bhargavi · ‎11-14-2021

Hi @Abha111 , Will the host name parameter come inside the events as well? If yes. you can simply define a regex in transforms.conf to route it to specific index. Example below. props.conf [your_sourcetype] enter your props TRANSFORMS-routing=set_nonprod,setprod transforms.conf [set_nonprod] REGEX = abc-nprd* DEST_KEY = _MetaData:Index FORMAT = index_x [setprod] REGEX = abc-prd* DEST_KEY = _MetaData:Index FORMAT = index_y Try this. Let me know for any further questions. If this works, give thumbs-up 🙂 Happy Splunking!!

bhargavi · ‎11-13-2021

Hi @danielfurtaw , Nullqueue entry in transforms.conf looks fine. As far as I know, entry for TRANSFORMS- in the props.conf only matters. It executes from Right to Left. Order does not matter in the transforms.conf file. i.e. From the below entries, first pan_decryption is executed, then pan_globalprotect and so on. TRANSFORMS-sourcetype = pan_threat, pan_traffic, pan_system, pan_config, pan_hipmatch, pan_correlation, pan_userid, pan_globalprotect, pan_decryption So if you have the regexes ready for all the below transform segments.. you need to place pan_discard in the first. TRANSFORMS-sourcetype = pan_discard ,pan_threat, pan_traffic, pan_system, pan_config, pan_hipmatch, pan_correlation, pan_userid, pan_globalprotect, pan_decryption So after all the regexes are executed, pan_discard comes into picture and the remaining logs will be sent to nullQueue. Try the above and let me know . If its helps, please give an upvote 🙂 Happy Splunking 🙂

bhargavi · ‎11-13-2021

Hi @Roshni, If you wish to extract during search time, you can use "Rex" command in your search query to extract required fields. It is simple and easy to learn and fun as well :). Sharing here some of splunk docs link for your reference to read through. I have learnt regex from this site. - https://regexone.com/ Its pretty cool and lets us practice then and there. its pretty cool. Try here. Splunk docs link : https://docs.splunk.com/Documentation/Splunk/8.1.1/SearchReference/Rex You can practice regex in https://regex101.com/. Regex cheat sheet : https://www.debuggex.com/cheatsheet/regex/pcre Happy Splunking ! 🙂

bhargavi · ‎11-13-2021

Hello @cfloquet Try with a map command here. Table both the fields you want to compare. Say fields : bad_domain & dest_hostname from respective files. I have used .csv file. Then we are using map command,passing the baddomain field using $baddomain$ to look for a match in field dest_hostname in "desthostnames.csv" and display the matching result. index="main" sourcetype="csv" source="baddomain.csv" OR source="desthostnames.csv" | table bad_domain,dest_hostname | map search="search index=main sourcetype=csv source=desthostname.csv |search dest_hostname=$bad_domain$|table dest_hostname" Note: Map command consumes more resources and takes time for execution. If this helps, please give an upvote 🙂 Happy Splunking 🙂

bhargavi · ‎11-13-2021

Hi @pchintha , Could you please elaborate on your requirement.

bhargavi · ‎11-13-2021

Hi @johnhuang , I have just copied the same query given by you, but the results are multivalue only. Unsure why you got in a single row. Can you try again?

bhargavi · ‎11-13-2021

Hello, Please try the below regex. |rex field=_raw "\sMessage\:(?P<Message>.*)\,\s\Data"

bhargavi · ‎11-13-2021

Hello, We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and I am not getting results as expected. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

bhargavi · ‎11-12-2021

Hi @richgalloway @skoelpin I have a little different scenario but facing a similar issue. Seeking your help. We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and I am not getting results as expected. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

bhargavi · ‎11-12-2021

Hi @woodcock @somesoni2 Could you please help me out here. I have a little different scenario here, but facing similar issue. We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and no data is getting indexed. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

bhargavi · ‎11-12-2021

Hi @niketn Could you please help me out here. I have a little different scenario. We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and no data is getting indexed. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

bhargavi · ‎11-11-2021

Hi @MuS Could you please help me out here. We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and no data is getting indexed. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

bhargavi · ‎11-08-2021

Hi @kristian_kolb @badland, I have a little different scenario but facing a similar issue. We are integrating the json logs via HEC into Splunk Heavy Forwarder. I have tried the below configurations.I am applying the props for the source. In transforms, there are different regexes and I would want to route it to different indexes based on log files and route all the other files not required to a null queue. I would not be able to use FORMAT=indexqueue in transforms.conf as I cannot mention multiple indexes in inputs.conf .This is not working and I am not getting results as expected. Kindly help. The configs are like below: PROPS.CONF -- [source::*model-app*] TRANSFORMS-segment=setnull,security_logs,application_logs,provisioning_logs TRANSFORMS.CONF -- [setnull] REGEX=class\"\:\"(.*?)\" DEST_KEY = queue FORMAT = nullQueue [security_logs] REGEX=(class\"\:\"(/var/log/cron|/var/log/audit/audit.log|/var/log/messages|/var/log/secure)\") DEST_KEY=_MetaData:Index FORMAT=model_sec WRITE_META=true LOOKAHEAD=40000 [application_logs] REGEX=(class\"\:\"(/var/log/application.log|/var/log/local*?.log)\") DEST_KEY=_MetaData:Index FORMAT=model_app WRITE_META=true LOOKAHEAD=40000 [provisioning_logs] REGEX=class\"\:\"(/opt/provgw-error_msg.log|/opt/provgw-bulkrequest.log|/opt/provgw/provgw-spml_command.log.*?)\" DEST_KEY=_MetaData:Index FORMAT=model_prov WRITE_META=true

Posts	25
Solutions	1
Karma Given	0
Karma Received	3
Member Since	‎11-08-2021

Online Status	Offline
Date Last Visited	‎05-28-2025 12:46 PM

How to implement rlog.sh/ausearch utility for an a...

Need help with Regex

Redirection to multiple indexes and Null queue not...

Re: Splunk Connect for Syslog (SC4S) suddenly stop...

How to implement rlog.sh/ausearch utility for an a...

Re: Need help with Regex

Re: How to filter out lines that start with #

Re: Splunk universal forwarder

Re: Need help with Regex

Need help with Regex

Re: Rex field

Re: Can you have 3 fields within a chart

Re: Query for Extraction of URL data in splunk

Re: Splunk stopped indexing on index="main"

Re: HOW TO GET FULL NAME USING REGEX FROM RAW DATA

Re: Route events to different indexes based on hos...

Re: nullQueue question - Palo Logs

Re: splunk integration on ocp

Re: Match Common Values from Multiline Fields from...

Re: stanza's for props.conf

Re: Multivalue field which contains the string "da...

Re: Splunk Regex help

Redirection to multiple indexes and Null queue not...

Re: Filtering Events using nullqueue

Re: How to keep specific events and discard the re...

Re: drop events from being indexed

Re: All events except one to Null

Re: UF+Indexer+nullQueue/Route = Zero

Are you a member of the Splunk Community?