All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

LOL here's ChatGPT, looks like PowerAutomate does have an HTTP post...IDK if it can read a full file, but it probably can   
I haven't used PowerAutomate...so I don't know if this is possible...but could PowerAutomate create an HTTP post event and send the data to a Splunk HEC endpoint?
Hi All: Thank you and I appreciate your response. We have a standalone instance of Splunk indexer and I double-checked and for the most part we're using 8.2.9 version of SplunkUF. Additionally, si... See more...
Hi All: Thank you and I appreciate your response. We have a standalone instance of Splunk indexer and I double-checked and for the most part we're using 8.2.9 version of SplunkUF. Additionally, since Splunk Enterprise 9.2.7 is the version in the 9.x.x family that supports Amazon Linux, we'll go for the same version. Currently the indexes' physical location is spread across volumes that are mounted on the Splunk indexer host at OS level. Please let us know if this is the right approach and if there are any stages we're missing in regards to the data cutover from the old to the new server. • Install Splunk Enterprise 9.2.7 on a new AL2023 server • Take a snapshot of the old server's volumes that contains indexed data, then connect them to the new one using the same mount point. • Copy the entire $SPLUNK_HOME/etc directory from the old server to the new server • Copy indexed data from $SPLUNK_DB (/opt/splunk/var/lib/splunk) to the new server • Detach & attach publicIP/EIP from old to the new server
Firstly, a golden shovel. This is a very very old thread. Secondly, you are mistaken. While the event will not get "immediately deleted" but for a completely different reason. There are several fact... See more...
Firstly, a golden shovel. This is a very very old thread. Secondly, you are mistaken. While the event will not get "immediately deleted" but for a completely different reason. There are several factors here: - events are not handled on their own but by buckets - hot buckets do not roll to frozen directly - "unusual" events (too far in the past or "from the future") are indexed in quarantine buckets which might get rolled completely differently than your normal buckets.
Thanks. Believe I got it. What tripped me up, is I didn't realize latest could be used for non-time based fields.   
I don't mean SharePoint activity, admin or audit logs. I mean actual data files (that will be converted later to lookup files in Splunk Cloud). Basically, do I need to extract the CSV files from Sha... See more...
I don't mean SharePoint activity, admin or audit logs. I mean actual data files (that will be converted later to lookup files in Splunk Cloud). Basically, do I need to extract the CSV files from SharePoint first (eg to a traditional on-prem file share by way of Power Automate) and use a UF to forward the files to Splunk Cloud, or is there some other nifty way to forward CSV data files directly from SharePoint Online to Splunk Cloud, or some other intermediary method? Thank you.
Data retention is not based on _time, its actually based on _indextime and max size set for example, if I index below sample data now, 2020-03-02 12:23:23 blah blah Retention time: 6months Maxsi... See more...
Data retention is not based on _time, its actually based on _indextime and max size set for example, if I index below sample data now, 2020-03-02 12:23:23 blah blah Retention time: 6months Maxsize: 100GB then the _time of the event will be 2020-03-02 12:23:23 but _indextime will be 2025-06-25 HH:MM:SS so this data will not get deleted immediately since _time of this event is 5 years old.  
Thank you @VatsalJagani !! This is so much helpful. A lot of examples to do exactly what I wanted. Works like a charm now.
 @LAME-Creations   For RBAC purposes, you can just make your summary index reside on the same index that it was created for. --> if I do in this way then will it override my original index data? And... See more...
 @LAME-Creations   For RBAC purposes, you can just make your summary index reside on the same index that it was created for. --> if I do in this way then will it override my original index data? And how can I differentiate between both index and summary logs present in same index? We use source field to get application name in normal index.  In my case user want to see raw data as well and we need all fields to be viewed everytime. Will summary index provides raw data as well? We have index format in this way - waf_app_<appID> app ID is different for different app teams. And in dashboard I have just given index = waf_app_* in base search. What index can I give now in summary index (same as my original index)  I am very confused here...
@livehybrid Thank you very much for this solution however my needs need to have it as streamlined as possible. While this does give the essential functionality that I need, it is a solution I need t... See more...
@livehybrid Thank you very much for this solution however my needs need to have it as streamlined as possible. While this does give the essential functionality that I need, it is a solution I need to work off of additionally. My needs specifically are that it is all click based and you don't have to change anything anywhere else and that it can all be handled within the timechart/bar chart. Anything else is to be fully hidden away. Much like the search app counterpart I also needed the timechart to the update itself based on the time range the rest of the visualization would use. Ideally this would just zoom in on one bar. Unfortunately what I am trying to and what I am trying to make altogether can't have the Text box and specific manual inputs. Simply put, this does accomplish the goal of setting time earliest and latest values that are 2h apart and then the other visualizations can take those tokens as their time. But I need a different approach to getting there.  
You have a few options and each have their own pros and cons and without knowing the data, I can only make an estimated guess on what would work best for you. Data Acceleration - you could put you... See more...
You have a few options and each have their own pros and cons and without knowing the data, I can only make an estimated guess on what would work best for you. Data Acceleration - you could put your data into data models - either existing or custom ones that fit your data and accelerate the data.  This will "accelerate" your data which in theory should significantly boost the speed at which you search,  Mileage may vary, but often you get orders of magnitude faster searching.  The cons with this is that you are probably going to double the size of your "indexed data" because acceleration is keeping your non accelerated logs and putting a set of accelerated data on the index meaning that you will be using more storage space.  Additionally, every 5 minutes or so, your accelerated data will be running the search to accelerate your data and that it going to occupy ram and cpu permanently on the box.  Plus depending on your comfort building or fitting your data into a datamodel, this is a little labor intensive to set up the first time.  As for RBAC, Splunk will maintain the same rbac rules to your accelerated data as exists on the index, so you won't need any special rbac considerations.  Summary indexing - This is an amazing tool for doing exactly that, summarizing the data.  For example if you have network logs - you have probably seen that in a given time period when two machines talk to each other, you may find that you have 100s of "connection logs".  If your use case is not interested in each of those 100 logs, but is more interested in - did these two IPs talk - (think threat inteligence - did we go to bad site x) than you could create a single summary log that says IP address x talked to IP address y 100 times.  You write this data to a summary index.  In reality, summary data gets its speed advantages because instead of speeding up the way you look for a needle in the haystack, you shrink the haystack so it is smaller - like in my example it is 1 / 100 smaller than the original index.   This is a useful solution if a summary of the logs is good enough for what your analysts are looking for and that may or may not be the case.  In the world of threat intel, we often have to look back at network traffic 18 months.  We look at the summary data, if we have a hit, the summary data tells us what date the hit was on, but the analysts may have to go look at the unsummarized log for that day to get a better idea of what really happened because summary logs gain their power by being exactly that - a summary.   For RBAC purposes, you can just make your summary index reside on the same index that it was created for.  The term summary index implies that you have a special index, but that is not really the case.  A summary index can be written to any index it is just a new source and the sourcetype is stash.  So if you summarize your data to the same index that the original logs came from, they will have the same rbac rules on them.   Here is a video on how to summarize data  https://youtu.be/mNAAZ3XGSng Below is a simple spl concept of summarizing palo alto firewall logs index=pan sourcetype=connections | stats sum(bytes_in) as bytes_in sum(bytes_out) as bytes_out earliest(_time) as _time count by src_ip, dest_ip   | collect index=pan source="summarized_pan_connections" You now need to determine how often you are summarizing your logs and set up a saved search to run that query.  Once it runs you just query the data with  index=pan source="summarized_pan_connections" Another option you can have is to schedule search your dashboard panels- this means that each of the panels will run the query one time at some specified time and everyone who comes to the dashobards will get the data that was created during the scheduled search.  This is relatively simple to set up, keeps rbac rules, but if having the latest logs included on the dashboard panels is your biggest priority, this one starts to fall apart.   I have given three suggestions, in my environment I have a similar situation as you, large amount of data and looking back long periods of time is slow.  We actually run a little mixture of all of it.  We accelerate a days worth of data, then in the middle of the night, we summarize yesterdays logs.  Then when the users search the dashboard the query is a combination of the accelerated data for today's data, and the summarized data for the previous days data.   Hope this gives you some ideas of a path forward.  There will be plenty of things that you need to consider, particularly how "fresh" does the data need to be.  Is the summary of the logs good enough, can you have static data in your dashboards that refreshes every day or every hour?   
@captaincool07, it would still be the same situation for the datamodel summaries also. Considering your RBAC situation, I would not use datamodel summaries if you want to restrict the access for inde... See more...
@captaincool07, it would still be the same situation for the datamodel summaries also. Considering your RBAC situation, I would not use datamodel summaries if you want to restrict the access for indexes. You'll add a hell lot of workload on your systems for accelerating the summaries and keeping them intact.  If you want to just use tstats for faster query, it would be better to use index time field extractions for the new ingestion.  Thanks, Tejas.
Hi @Kosyay  Yes, you can use props/transforms to extract fields from the event. I wouldnt recommend modifying the event at index-time though, as this might break other extractions that exist for thi... See more...
Hi @Kosyay  Yes, you can use props/transforms to extract fields from the event. I wouldnt recommend modifying the event at index-time though, as this might break other extractions that exist for this feed, instead you can create new fields for the data you want to extract.  I havent validated this with my instance as I dont have WinEventLogs at the moment but you could try something like this: You can extract and rename these fields at index time using props.conf and transforms.conf with appropriate regex. In props.conf, configure a stanza for your sourcetype and reference a transform. In transforms.conf, use regex to capture and rename the two "Account Name" fields. Example configuration:    === props.conf === [your_sourcetype] TRANSFORMS-rename_account_names =extract_win_account_names === transforms.conf === [extract_win_account_names] REGEX = Subject:\s+Security ID:.*?Account Name:\s*([^\s]+).*?New Logon:.*?Account Name:\s*([^\s]+) FORMAT = Source_Account_Name::$1 Destination_Account_Name::$2    Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hello everyone, I have a network monitoring system that exports data via IPFIX using Forwarding Targets. I am trying to receive this data in Splunk using the Splunk Stream app. The add-on is instal... See more...
Hello everyone, I have a network monitoring system that exports data via IPFIX using Forwarding Targets. I am trying to receive this data in Splunk using the Splunk Stream app. The add-on is installed and Stream is enabled, but I am facing the following issues: Templates are not being received properly. The data arrives, but it's unreadable or incomplete. I need full flow data, including summaries or headers from Layer 7 (e.g., HTTP, DNS). My question: Has anyone successfully received and parsed IPFIX data in Splunk? If so, could you share the steps or configurations you used (like streamfwd.conf, input settings, etc.)? Any guidance would be greatly appreciated! Thanks in advance!
Hello! I have logs from Domain Controller Active Directory in Splunk and try to configure monitoring of user logons (EventCode=4624). Unfortunately, there are two fields with a name "Account Name" ... See more...
Hello! I have logs from Domain Controller Active Directory in Splunk and try to configure monitoring of user logons (EventCode=4624). Unfortunately, there are two fields with a name "Account Name" example of a log: 06/25/2025 02:54:32 PM LogName=Security EventCode=4624 EventType=0 ComputerName=num-dc1.boston.loc SourceName=Microsoft Windows security auditing. Type=Information RecordNumber=881265691 Keywords=Audit Success TaskCategory=Logon OpCode=Info Message=An account was successfully logged on. Subject: Security ID: NULL SID Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 Impersonation Level: Impersonation New Logon: Security ID: BOSTON\*** Account Name: *** Account Domain: BOSTON Logon ID: 0x135F601B51 Logon GUID: {12C0DD76-F92B-07E1-88A5-914C43F7D3D5} Could you please advise if it’s possible to modify the fields before indexing, i.e., at the "input" stage? Specifically, I'd like to change the first field Subject: Account Name to Source Account Name and the second field New Logon: Account Name to Destination Account Name. From what I understand, this would require modifications in props.conf and transforms.conf. If anyone has ideas on how to achieve this, please share!
Hi there, In Mission Control in our properly working Splunk environment, we see the following: This is exactly how we want it: the finding based correlation search "Threat - Findings Risk Thres... See more...
Hi there, In Mission Control in our properly working Splunk environment, we see the following: This is exactly how we want it: the finding based correlation search "Threat - Findings Risk Threshold Exceeded for Entity Over 24 Hour Period - Rule" fired because of multiple findings that occured for one specific entity. If you expand it, then it shows all the findings.  (Please ignore the weird names of the findings) Then in our other environment, it looks differently. When you click expand, it has to think for a while: And then it just shows the number of intermediate findings, but the not the actual findings themselves. You also can't click on this grey label. I suspect it has something to do with the fact that our working environment is a somewhat fresh install, whereas the environment in which it doesn't work properly is an upgrade from an old ES version to the newest version. There might be some index problems or something, I don't know. Does anyone know?
@ITWhisperer what if I use data models instead of summary index? Will it serve the same purpose? And how about RBAC control for data models? 
Hi, I have a requirement for "high jvm thread wait time monitoring" for BT's. The only metric for JVM is thread count only (Application Infrastructure Performance -> Tier -> JVM -> Threads). So appr... See more...
Hi, I have a requirement for "high jvm thread wait time monitoring" for BT's. The only metric for JVM is thread count only (Application Infrastructure Performance -> Tier -> JVM -> Threads). So appreciate your expert suggestions on enabling/configuring the metric.  
Another thing to consider with summary indexes is idempotent updates, that is, how to avoid double counting; for example, in your instance, if you created summaries for each day in your summary index... See more...
Another thing to consider with summary indexes is idempotent updates, that is, how to avoid double counting; for example, in your instance, if you created summaries for each day in your summary index, what do you do if you get events which arrive late (for whatever reason), how do you make sure they are included in the summary without double-counting the events which have already been summarised. I did a presentation on this a couple of years ago for the B-Sides program. https://github.com/bsidessplunk/2022/blob/main/Summary%20Index%20Idempotency/Bsides%20Spl22%20Summary%20Index%20Idempotency.pptx
Can I use data models here? Will it serve same as Summary index? What is reliable and best and most importantly aligns with RBAC created?