All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

@ITWhisperer need to see filter out/in result to decide. All fields extracted already. need keep the events with T[A].
Index Size is 5.3G vs 1.6G Raw Data: Raw data  Index on Splunk   This is also affecting our licensing plans as well.  This is way bigger than anticipated.  I thought that 110% ro maybe ev... See more...
Index Size is 5.3G vs 1.6G Raw Data: Raw data  Index on Splunk   This is also affecting our licensing plans as well.  This is way bigger than anticipated.  I thought that 110% ro maybe even 180% but not 400%.  Somethings off.  
We followed the steps in https://docs.splunk.com/Documentation/DM/1.8.1/User/AWSAbout to onboard the data from a single AWS account. During the process onboarding data, AWS account details are input... See more...
We followed the steps in https://docs.splunk.com/Documentation/DM/1.8.1/User/AWSAbout to onboard the data from a single AWS account. During the process onboarding data, AWS account details are input using the UI, following which Splunk generates the Cloud Formation Template. This template has the a DM_ID, DM_Name and few indexes which Splunk generates. Does Splunk have API to script this? Our DevOps team wants to automate this process. ps: I was unable to find this in the API documentation.
Hello, I think this is a simple answer but I'm not able to find a solution.  I created a lookup table that looks like this (but of course has more info): Cidr, ip_address 24, 99.99.99.99/24 25... See more...
Hello, I think this is a simple answer but I'm not able to find a solution.  I created a lookup table that looks like this (but of course has more info): Cidr, ip_address 24, 99.99.99.99/24 25, 100.100.100/25 I only included the Cidr column as I read that the lookup table needs at least 2 columns, but I do not use it. Let me know if I should! I am trying to find source ips that match with the ip_address in my lookup table.    index="index1" [|inputlookup lookup | rename ip_address as src_ip] I have ensured that Advanced Settings -> Match -> CIDR(ip_address) When the query is ran, no matches are found, but I know that there is traffic from the addresses. What am I overlooking?
Okay Thank you
wanted to reach out for help regarding an issue we have been experiencing on one of our customers.  We build an app that exports events from a standalone customer using the Splunk Enterprise instance... See more...
wanted to reach out for help regarding an issue we have been experiencing on one of our customers.  We build an app that exports events from a standalone customer using the Splunk Enterprise instance.  We have that box gather the logs and hold them until it can be exported out of the box manually. We used the savedseaches.conf file to schedule a search query script (export.py) to pull events.  The problem is that on this particular customer he is only getting like 11 minutes worth of logs, but the file is scheduled to pull all index events from lets say 3:30pm-4:30pm, but the events start loading only from 4:19pm-4:30pm.   It does this across all times consistently. example, missing the first like 49 minutes of events:  4:19pm-430pm 5:19pm-5:30pm 6:19pm-6:30pm  We have a export.py script that goes out and gathers all index=* events according to the cron specified. savedsearches.conf cron_schedule = 30 */1 * * *  enablesched = 1 dispatch.ttl = 1800 allow_skew = 10m search = | export disable = 0  To compensate for lags, we build into the |export.py script to pull the events 1 hour prior so like.  This is part of the script dealing with the specific search. now = str(time.time()-3600).split(".")[0] query = "search index=* earliest=" + last_scan + "  lastest=" + now + "  once script is done, it creates a timestamp in a file of the now in epoch time, which is used for the next schedule time. Any help would be appreciated
If I understand you correctly the query should work like this:   index=gbts-vconnection sourcetype=VMWareVDM_debug "onEvent: DISCONNECTED" (host=Host1 OR host=host2) | rex field=_raw "(?ms)^(?:[^:\... See more...
If I understand you correctly the query should work like this:   index=gbts-vconnection sourcetype=VMWareVDM_debug "onEvent: DISCONNECTED" (host=Host1 OR host=host2) | rex field=_raw "(?ms)^(?:[^:\\n]*:){5}(?P<IONS>[^;]+)(?:[^:\\n]*:){8}(?P<Device>[^;]+)(?:[^;\\n]*;){4}\\w+:(?P<VDI>\\w+)" offset_field=_extracted_fields_bounds | eventstats count as failed_count by Device IONS | where failed_count>=10 | timechart dc(IONS) as IONS span=1d   This will show you the amount of user with more than 10 failed logons on each day.
I had been struggling with the same problem. After a lot of experimentation with different ideas and inspecting SAML payloads. My two main findings as best as I can tell are: When you configure a G... See more...
I had been struggling with the same problem. After a lot of experimentation with different ideas and inspecting SAML payloads. My two main findings as best as I can tell are: When you configure a Google Groups mapping in Google SAML configuration, Google will send the group name as an attribute identically to if it were an attribute set up in the attribute mapping. When Splunk receives a SAML assertion with a role attribute, I think it will try to match it against roles as well as SAML groups. Though in my case all of the role attributes I use are SAML group names, so I cannot confirm that it will match the "role" attribute against an actual role name. But also crucially, when you update Google SAML configurations, it can take 5-10 minutes for the update to "go live". So watch the SAML assertions that you are actually sending to Splunk as you experiment, because otherwise you'll make a changes and even if you get it right it'll appear to not work, you'll make more changes, and suddenly things work, but actually the working configuration was n attempts ago, and it will break itself as it slowly updates to your later configuration attempts, and all you'll know is that something you tried at some point over the last however long was correct. So the net result: Set the "App attribute" to "role", exactly like you did in your screenshot. If you have created a role in splunk whose name is the same as your Google group, you're done. If your Google group has a different name than your role, then set up a SAML group in Splunk with the same name as your google group and assign it the role you want. Splunk will lowercase the group name, that's fine, it'll still match. As a result, you can actually use both (e.g. a group to grant "user" access, and individual user attributes to grant admin access) In my case, I already had a Google group called "Engineering" that I wanted to set up with the "user" role. Here are my configs: Splunk: 1. Configure SAML groups with names corresponding to your Google groups 2. Configure your Google SAML configuration. If you plan to use both user attributes and Google groups, set both a user attribute and a Group membership, both pointing to the "role" App attribute. If you only plan on using groups, you can omit the user attribute. In my case, as you can see from my Splunk config above, I want the "Engineering" google group to all have "user" access to Splunk: 3. If you want to specify role overrides, set them as you did before: 4. If it isn't working, decode and review the SAML assertion that Splunk is receiving. It can take a surprisingly long time for changes made in Google's SAML configuration to go "live". You will likely observe that you're passing along a SAML assertion that does not reflect your most recent Google configuration changes -- if that's the case, just wait a while and try again in a bit.
Hey @Southy567, To achieve the use case, you can have alert throttling enabled. You can find the throttle checkbox in the below screenshot. Once you check the throttle checkbox, you can suppres... See more...
Hey @Southy567, To achieve the use case, you can have alert throttling enabled. You can find the throttle checkbox in the below screenshot. Once you check the throttle checkbox, you can suppress the alerting for 4 hours as mentioned in the below screenshot   So if the alert is suppressed for 4 hours, the SNOW ticket will not be created for the users that already have s SNOW ticket raised. After 4 hours, the alerting should resume as normal for the same set of users.   Thanks, Tejas. --- If the above solution helps, an upvote is appreciated.
I was able to find this search that gives me the number of users(IONS) who disconnected 10 or more times however it gives me the total based on time.  I would like to display a daily number for 30 da... See more...
I was able to find this search that gives me the number of users(IONS) who disconnected 10 or more times however it gives me the total based on time.  I would like to display a daily number for 30 days in a line chart.  For example Monday there were 10 users who disconnected over 10 time and so on for the rest of week. I can't seem to get the timechart to work with this: index=gbts-vconnection sourcetype=VMWareVDM_debug "onEvent: DISCONNECTED" (host=Host1 OR host=host2) | rex field=_raw "(?ms)^(?:[^:\\n]*:){5}(?P<IONS>[^;]+)(?:[^:\\n]*:){8}(?P<Device>[^;]+)(?:[^;\\n]*;){4}\\w+:(?P<VDI>\\w+)" offset_field=_extracted_fields_bounds | stats count by Device IONS | where count >= 10 | appendpipe [|stats count as IONS | eval Device="Total"]
Okay, thank you. I am quite good with the default JSON but I’ll take a look and maybe have some tuning. Thanks again
Have you asked the CyberArk admins to add your SH IP to the approved list for that safe?
_json is the generic format that's applied to events if Splunk notices them as being JSON so it's not the best idea. It's best to be as specific as possible with your definition. (ideally sourcetype... See more...
_json is the generic format that's applied to events if Splunk notices them as being JSON so it's not the best idea. It's best to be as specific as possible with your definition. (ideally sourcetype should have a set of parameters responsible for proper breaking and time recognition defined - so called Great Eight). So while _json is fairly generic you should have your specific settings for - for example - time extraction. Another thing is that even though the format of the event might just be JSON, specific sourcetypes can have different additional aliases or calculated fields defined (for example for CIM compliance). So you want to have your events "pinned" to the specific sourcetype instead of the generic _json.
Well, I think the part I’ve misunderstood is via normal props.conf rules  I thought all events that the transform is applied to was about the real transform (ie the REGEX + FORMAT ) so I didn’t u... See more...
Well, I think the part I’ve misunderstood is via normal props.conf rules  I thought all events that the transform is applied to was about the real transform (ie the REGEX + FORMAT ) so I didn’t understand why non-matching events (as per REGEX) would get cloned too. Now I get it, I’ll try and find a way to work around that later. About the cloning of _json into a new sourcetype and using that clone into my config, what would be the gain (apart for best practice’s sake) ?
@richgalloway , We can do the Event Action Extract Fields instead on creating the props/transforms right ?   Thanks
Hi,  I'll explain myself better with an example: I have the following values in radio (input): Name -> Value MB ->1024/1024 GB ->1024/1024/1024 (...) with the $token_name$ I can use the select... See more...
Hi,  I'll explain myself better with an example: I have the following values in radio (input): Name -> Value MB ->1024/1024 GB ->1024/1024/1024 (...) with the $token_name$ I can use the selected value in my search, but I would also like to use the name/label of the selected size to use in the chart legend. Is it possible to do this in Dashboard Studio? thanks in advance for your help  
Hello, I want to get Rspamd logs into Splunk with every info available. The best I could do with Rspamd config yields to this: 2023-11-03 13:02:24 #56502(rspamd_proxy) <7fcfc8>; lua; [string "retur... See more...
Hello, I want to get Rspamd logs into Splunk with every info available. The best I could do with Rspamd config yields to this: 2023-11-03 13:02:24 #56502(rspamd_proxy) <7fcfc8>; lua; [string "return function (t...:4: METATEST {"qid":"8BC8C2F741","user":"unknown","ip":"188.68.A.B","header_from":["foo bar via somelist <somelist@baz.org>"],"header_to":["list <somelist@baz.org>"],"header_subject":["proper subject"],"header_date":["Fri, 3 Nov 2023 08:00:43 -0400 (EDT)"],"scan_time":2457,"rcpt":["me@myself.net"],"size":6412,"score":-5.217652,"subject":"proper subject","action":"no action","message_id":"4SMK7v2HQTzJrP1@spike.bar.org","fuzzy":[],"rspamd_server":"rack.myself.net","from":"somelist-bounces@baz.org","symbols":[{"score":-0.500000,"group":"composite","groups":["composite"],"name":"RCVD_DKIM_ARC_DNSWL_MED"},{"score":0,"group":"headers","groups":["headers"],"name":"FROM_HAS_DN"},{"score":0,"group":"headers","options":["somelist@baz.org","somelist-bounces@baz.org"],"groups":["headers"],"name":"FROM_NEQ_ENVFROM"},{"score":-0.010000,"group":"headers","groups":["headers"],"name":"HAS_LIST_UNSUB"},{"score":0,"group":"headers","options":["somelist@baz.org"],"groups":["headers"],"name":"PREVIOUSLY_DELIVERED"},{"score":-1,"group":"abusix","options":["188.68.A.B:from"],"groups":["abusix","rbl"],"name":"RWL_AMI_LASTHOP"},{"score":-0.100000,"group":"mime_types","options":["text/plain"],"groups":["mime_types"],"name":"MIME_GOOD"},{"score":-0.200000,"group":"headers","options":["mailman"],"groups":["headers"],"name":"MAILLIST"},{"score":1,"group":"headers","groups":["headers"],"name":"SUBJECT_ENDS_QUESTION"},{"score":-0.200000,"group":"policies","options":["+ip4:188.68.A.B"],"groups":["policies","spf"],"name":"R_SPF_ALLOW"},{"score":-1,"group":"policies","options":["list.sys4.de:s=2023032101:i=1"],"groups":["policies","arc"],"name":"ARC_ALLOW"},{"score":0,"group":"ungrouped","options":["asn:19xxxx, ipnet:188.68.A.B/20, country:XY"],"groups":[],"name":"ASN"},{"score":0.100000,"group":"headers","groups":["headers"],"name":"RCVD_NO_TLS_LAST"},{"score":0,"group":"headers","groups":["headers","composite"],"name":"FORGED_RECIPIENTS_MAILLIST"},{"score":0,"group":"policies","options":["baz.org:+","bar.org:-"],"groups":["policies","dkim"],"name":"DKIM_TRACE"},{"score":0,"group":"headers","groups":["headers"],"name":"REPLYTO_DOM_NEQ_FROM_DOM"},{"score":0,"group":"policies","options":["bar.org:s=dktest"],"groups":["policies","dkim"],"name":"R_DKIM_REJECT"},{"score":-2.407652,"group":"statistics","options":["97.28%"],"groups":["statistics"],"name":"BAYES_HAM"},{"score":0,"group":"headers","groups":["headers"],"name":"TO_DN_ALL"},{"score":0,"group":"composite","groups":["composite"],"name":"DKIM_MIXED"},{"score":-0.200000,"group":"policies","options":["baz.org:s=20230217-rsa"],"groups":["policies","dkim"],"name":"R_DKIM_ALLOW"},{"score":0,"group":"headers","options":["3"],"groups":["headers"],"name":"RCVD_COUNT_THREE"},{"score":-0.600000,"group":"rbl","options":["188.68.A.B:from","188.68.A.B:received","168.100.A.B:received"],"groups":["rbl","dnswl"],"name":"RCVD_IN_DNSWL_MED"},{"score":-0.100000,"group":"rbl","options":["188.68.A.B:from"],"groups":["rbl","mailspike"],"name":"RWL_MAILSPIKE_GOOD"},{"score":0,"group":"policies","options":["baz.org"],"groups":["policies","dmarc"],"name":"DMARC_NA"},{"score":0,"group":"headers","options":["1"],"groups":["headers"],"name":"RCPT_COUNT_ONE"},{"score":0,"group":"mime_types","options":["0:+"],"groups":["mime_types"],"name":"MIME_TRACE"},{"score":0,"group":"headers","groups":["headers","composite"],"name":"FORGED_SENDER_MAILLIST"},{"score":0,"group":"headers","groups":["headers"],"name":"TO_EQ_FROM"},{"score":0,"group":"headers","options":["foo@bar.org"],"groups":["headers"],"name":"HAS_REPLYTO"}]}   Currently I’m extracting JSON with a props.conf & a transforms.conf: props.conf [rspamd] KV_MODE = json TRANSFORMS-json_extract_rspamd = json_extract_rspamd transforms.conf [json_extract_rspamd] SOURCE_KEY = _raw DEST_KEY = _raw LOOKAHEAD = 10000 #REGEX = ^([^{]+)({.+})$ REGEX = ^(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d) (#\d+)\(([^)]+)\) ([^;]+); lua[^{]+{(.+})$ FORMAT = {"date":"$1","ida":"$2","process":"$3","idb":"$4",$5 CLONE_SOURCETYPE = _json I end up with this in splunk:                                     From here, I have 2 problems. 1st problem: contrary to native JSON (from my Amavis logs for example), Splunk does not extract nor process basic stats about fields unless I’m explicitly extract them… That’s quite a pain. Is there a way / config setting to instruct Splunk to automagically extract every fields? 2nd problem: this JSON is crap. Every object in "symbols[]" looks like this: It’s almost unuseable as it prevent me from linking the name of the symbol to its score and to its options. Is there a parsing option / function I could use to reliably transform this into something I can work with? A good result could be turning { group: abusix groups: [ abusix rbl ] name: RWL_AMI_LASTHOP options: [ A.B.C.D:from ] score: -1 } into RWL_AMI_LASTHOP: [ group: abusix groups: [ abusix rbl ] name: RWL_AMI_LASTHOP options: [ A.B.C.D:from ] score: -1 ]   I’m open to suggestions, I’ve been working for years with the great JSON logs of Amavis (perfect parsing and usability). This problem is new to me…
You can do it also based on source, but you must remember precedence! [<spec>] * This stanza enables properties for a given <spec>. * A props.conf file can contain multiple stanzas for any number of... See more...
You can do it also based on source, but you must remember precedence! [<spec>] * This stanza enables properties for a given <spec>. * A props.conf file can contain multiple stanzas for any number of different <spec>. * Follow this stanza name with any number of the following setting/value pairs, as appropriate for what you want to do. * If you do not set a setting for a given <spec>, the default is used. <spec> can be: 1. <sourcetype>, the source type of an event. 2. host::<host>, where <host> is the host, or host-matching pattern, for an event. 3. source::<source>, where <source> is the source, or source-matching pattern, for an event. 4. rule::<rulename>, where <rulename> is a unique name of a source type classification rule. 5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed source type classification rule. These are only considered as a last resort before generating a new source type based on the source seen. **[<spec>] stanza precedence:** For settings that are specified in multiple categories of matching [<spec>] stanzas, [host::<host>] settings override [<sourcetype>] settings. Additionally, [source::<source>] settings override both [host::<host>] and [<sourcetype>] settings.   And of course restart is needed after changing those.  Also "splunk btool props list --debug" is excellent tool to check that you have correct configuration in use.
Hi All, I want to create an SPL query that first returns data by matching the destination IP address from Palo Alto logs. Then, according to the destination IP, it will resolve the actual destinatio... See more...
Hi All, I want to create an SPL query that first returns data by matching the destination IP address from Palo Alto logs. Then, according to the destination IP, it will resolve the actual destination hostname from Symantec logs and Windows Event logs in separate fields. I was able to match the destination IP (dest_ip) from Palo Alto logs with Symantec logs and return the hostname (if available) from it. However, I am struggling to do the same by joining Windows logs to return the values, which should be equal to the hostname in Symantec logs. Can someone kindly assist me in fixing this code to retrieve the expected results?       index=*-palo threat="SMB: User Password Brute Force Attempt(40004)" src=* dest_port=445 | eval dest_ip=tostring(dest) | join type=left dest_ip [ search index=*-sep device_ip=* | eval dest_ip=tostring(device_ip) | stats count by dest_ip user_name device_name ] | eval dest_ip=tostring(dest) | join type=left dest_ip [ search index="*wineventlog" src_ip=* | eval dest_ip=tostring(src_ip) | eval username=tostring(user) | stats count by dest_ip username ComputerName ] | table future_use3 src_ip dest_ip dest_port user device_name user_name rule threat repeat_count action ComputerName username | sort src_ip | rename future_use3 AS "Date/Time" src_ip AS "Source IP" dest_ip AS "Destination IP" user AS "Palo Detected User" user_name AS "Symantec Detected User @ Destination" device_name AS "Symantec Destination Node" rule AS "Firewall Rule" threat as "Threat Detected" action as "Action" repeat_count AS "Repeated Times"       @eve
Hi @anandhalagaras1 , as @isoutamo said, You have to put these conf files on Indexers or (if present) on Heavy Forwarders. Ciao. Giuseppe