Anyway I tried the CLI manner and it works. So not sure why there is an issue on the browser though.     ... View more

It is an AIO, search head + indexer, I think the disk performance is also lacking hence the issue. ... View more

Hi @gcusello @deepakc  Thanks for all the inputs. Checking in again today after the weekend, the TCP filtering is working fine! The TCP events from the firewall stopped 1-2 hours after disabling of TCP input, and I suspect this might be due to TCP backlogs? I am not sure how Splunk handles TCP backlogs, but it seems that TCP backlogs will not be processed by the event filtering syntax. Maybe TCP backlogs are "past" the filtering stages and are slowly ingested? ... View more

Hello @deepakc @gcusello  I have checked. Netstat shows the only process listening on 514 is Splunkd. Normally on Windows I use Kiwi Syslog. Just a while I changed the Regex again to . so that all events are moved to null queue. Injected syslog messages with python on TCP (SOCK_STREAM) and surprising no data appears for my test messages. But the firewall messages with "source = tcp:514" is still relentlessly appearing. Can a source appear as TCP: 514 but is not the case? Used btools to check for TCP modifications but can't find any. Oh my god, it's so weird. Seems like ghost TCP 514 messages. Next I will block that port with firewall and see if it still comes in. Update once more: We removed all TCP inputs from data input and do a restart. Funnily the TCP:514 source still comes into the SIEM. I tried with my python syslog message generator and I got an error because the Splunk server rejects the connection. Feels like some kind of corruption somewhere where the messages thinks they are TCP:514? ... View more

Hi @gcusello @deepakc  I will check if Rsyslog is installed. As far as I am aware it shouldn't and also it is running on Windows. 20+ GB used in the license per day. Is it recommended to have Rsyslog / Kiwi Syslog running on the server and then to use file monitor to copy in events? ... View more

Hi, For my case, this is not a heavy forwarder. But this is an indexer and search head. The UDP and TCP flow is direct to the AIO indexer / search head. I understand from the documentation that a HF is more flexible where we can filter using sourcetype, hosts etc but for the indexer it is still possible, but the example given is on the source. We did try to add a backlash to the equal sign, and even just try discarding all events with a "." as the regex search key but somehow the TCP source can't perform any filtering. Is there something unique with direct filtering on indexers? It shows clearly that "source = tcp:514" I don't suppose it has been renamed, or should I try renaming this port, not sure if that help. On the inputs conf, I think there are some TCP settings that directs the sourcetype based on the host address, but I don't suppose that will have any impact? ... View more

Hello Giuseppe, Blanking out some of the details with XXXX for anonymity. I remember we did try with Regex = . for the TCP once too. Sample event from TCP: <189>logver=700140601 timestamp=1714717074 devname="XXXXX" devid="XXXXX" vd="root" date=2024-05-03 time=06:17:54 eventtime=1714688275070439553 tz="XXX" logid="0001000014" type="traffic" subtype="local" level="notice" srcip=XX.XX.XX.XX srcname="XXXXXX" srcport=31745 srcintf="port1" srcintfrole="undefined" dstip=XXX.XXX.XXX.XXX dstname="XXX" dstport=443 dstintf="root" dstintfrole="undefined" srccountry="XXXX" dstcountry="XXX" sessionid=68756048 proto=6 action="deny" policyid=1 policytype="local-in-policy" poluuid="7575f13c-5066-51ed-1e15-40b0e5867f81" service="HTTPS" trandisp="noop" app="HTTPS" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 appcat="unscanned" crscore=5 craction=262144 crlevel="low" Sample event from UDP: May 3 14:21:57 10.XX.XX.XX logver=700140601 timestamp=1714746117 devname="XXX" devid="XXX" vd="XXX" date=2024-05-03 time=14:21:57 eventtime=1714717317787162683 tz="XXX" logid="0000000013" type="traffic" subtype="forward" level="notice" srcip=XX.XX.X.X srcport=38915 srcintf="port5" srcintfrole="lan" dstip=XX.XX.XX.XX dstport=443 dstintf="port9" dstintfrole="wan" srccountry="Reserved" dstcountry="XXXX" sessionid=759555888 proto=17 action="accept" policyid=1 policytype="policy" poluuid="7ade8e92-454b-51e9-5c91-4feddb630366" policyname="XXXXX" service="udp/443" trandisp="snat" transip=XX.XX.XX.XXX transport=38915 appid=40169 app="QUIC" appcat="Network.Service" apprisk="low" applist="XXX" duration=49 sentbyte=1228 rcvdbyte=0 sentpkt=1 rcvdpkt=0 utmaction="block" countapp=1 It seems a bit weird to me that the TCP input doesn't have the data and time infront but starts with <189>, wonder if that is normal.   ... View more

Thanks for the reply. We did do a netstat check and it's a TCP connection in between the source host and Splunk. Something similar that was tried, example [source::udp:514] TRANSFORMS-null1= udp_setnull [source::tcp:514] TRANSFORMS-null2= tcp_setnull However weirdly the TCP part is not working. Once we even tried removing UDP and just have the TCP portion, but it still doesn't work. Very weird. TRANSFORMS-null= setnull_tcp_traffic ... View more

Thanks @gcusello @PickleRick  for all the replies and tips and hints. It has been very helpful. In the end, I went with 1 data model with segregation done using source filtering.  Still fiddling with adding fields into data model but I am sure it will be a nice addition like to have extra info like indexes into the data model fields or during indexing. I wish I can mark both as solutions, but since I can only accept one, I will select Rick's as the reply gave the Eureka moment in which a single model doesn't impact the security roles (index selection) and subsequently made me switch to the single data model. But all in all, all the replies have made me learnt a lot. A big thank you. ... View more

hi @gcusello  I think that would be useful. I try to add the index field in the data model but seems not able to. I don't see that field in the auto-extracted option. I can see fields like host, sourcetype being inherited from BaseEvent in the JSON. I am wondering, shall I modify the JSON then? Not sure if that is the right way. Can't see to figure out how to add the index using the data model editor. Thanks again ... View more

Thanks!  I did not know about indexed field, that would be something interesting. Is there a way to add on another field that is always present for all models? For example in addition to. source, sourcetype, host, _raw, _time, is it possible to add like source_zone or something that works for all models? I saw that the source, sourcetype, host, etc are inherited but unsure from where is the inheritance from. ... View more

Yes, certain source it's a bit hard for me to override the source name, I will try to see what can be done. I was looking at source as it's one of the few fields that seems to be common across multiple models, eg network, authentication, change etc ... View more

Hey @PickleRick  2. You are absolutely right. I just tried with different users on the same accelerated model, same query but different roles, and the restricted users has much less results. So, can I say the way forward seems to be one common data model then? Is there any recommended or easy way to perform filtering between Zones in a summary search for example?  Is using Where source=ZoneA* alright then? ... View more

Thanks for the hints. In terms of data retention all the sections will have similar policy. However, access grants can be an issue. In my use case, the dashboards will be monitored by section personnel and also by the SOC. Therefore in terms of access, SOC will be able to see DMZ, ZoneA and ZoneB while the respective members of each section should only be able to see their zones (need-to-know basis policy) At the moment I am using different indexes so I can perform some transforms specific to each zones, as the syslog log sending formats are different due to the different log aggregator used by each zones. By using the different indexes in the heavy forwarder, I am able to perform some SED for particular log sources, and host & source override on the HF. I remember that I can limit access based on indexes, but I guess this is not possible with data models but will this be a concern? If I put them all in a data model, is it still possible to restrict access? For example, if the user can only manipulate views from dashboard and not be able to run searches themselves, that will still be OK. Pros and Cons in my mind: Separate data model: - Pro's: I can easily segregate the tstats query - Cons: Might be difficult to get an overview stats need to use appends and maintain each additional new zone. Each new data model will need to run periodically and increase the number of scheduled accelerations? Integrated data model: - Cons: might be harder to filter, eg between ZoneA, ZoneB and DMZ. Seems like I can filter only based on the few parameters in the model, eg source, host - Pros: Easier to maintain, as just need to add new indexes into the data model whitelist. Limit the number of Scheduled runs. - And as mentioned the point on data access? Will it be still possible to restrict? I am still quite new to Splunk so some of my thoughts might be wrong. Open to any advice, still in a conundrum.  ... View more

The data will still be CIM compliant though. I am simply replicating the data model, so I have two different sets of data model (all the settings are similar but the whitelisted indexes in each data model is different) By cloning the original Data Model from the CIM app, I have a DMZ Network data model = Only Index for DMZ  Zone A Network data model = Only Index for Zone A At that time, my goal was to provide ease for the users to display the dashboard but simply swapping the data model in use. Cause DMZ and Zone A is highly unique between one another So would the best practice be just to put all in one common data model, eg Default network data model = all indexes and then try to separate out the zones by using filters like "where" in the search queries. ... View more

Further updates, the JS in the previous thread works so long as I leave the browser on that particular dashboard, tested it running for few days and no log outs occured. Anyway, hoping there could be a more direct way to set the settings for the group or user though. ... View more

Thanks, I did try the method here, but it doesn't seem effective. Solved: User-specific browser session timeout? - Splunk Community Are there other ways, regardless of which timeout settings I can use to ensure my particular dashboard user does not logged out? ... View more