1. Will like to explore more secure way on top of just TLSv1.2 If data must be masked, might as well deploy on-prem.  2.SAML authentication seems complicated, the link does not show how to configure SAML authentication for Duo   Here shows Splunk cloud can't configure MFA https://docs.splunk.com/Documentation/Splunk/8.2.3/Security/AboutMultiFactorAuth   ... View more

In splunk cloud, I went to Apps -> Browse more apps Enter windows Installed Splunk Add-On for Microsoft Windows   After that, the data was parsed correctly, can mark this as solved. ... View more

I added inputs.conf in  C:\Program Files\SplunkUniversalForwarder\etc\apps\100_prd-p-gvnkg_splunkcloud\local   [WinEventLog://Application] disabled = 0 start_from = oldest current_only = 0 checkpointInterval = 5 renderXml=true index=winevent [WinEventLog://Security] disabled = 0 start_from = oldest current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 renderXml=true index=winevent [WinEventLog://System] disabled = 0 start_from = oldest current_only = 0 checkpointInterval = 5 renderXml=true index=winevent     I saw some application logs inside, but seems like they did not parse correctly. I go ahead and install the Windows add-on app on the Splunk cloud? https://splunkbase.splunk.com/app/742/ ... View more

From Splunk cloud: To set up the Universal Forwarder: Download the Splunk universal forwarder. Splunk Downloads web page  Install the universal forwarder on one or more machines in your network. Installation Instructions  Download your customized universal forwarder credentials package. Download Universal Forwarder Credentials Install the universal forwarder credentials package on each universal forwarder in your network. Installation Instructions  Configure your universal forwarders to send data to the Splunk platform. Configure data inputs  ... View more

https://docs.splunk.com/Documentation/SplunkCloud/8.2.2109/Admin/WindowsGDI#Step_3:_Configure_indexes_on_your_Splunk_Cloud_Platform_instance Can't I skip step 4 & 5 and go  straight towards install the Splunk universal forwarder?   In the splunk forwarder I see, 12-02-2021 22:26:21.612 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:26:41.414 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:26:52.019 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:27:11.318 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:27:22.282 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:27:41.208 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:27:51.500 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:28:11.073 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:28:21.782 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:28:40.951 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:28:52.022 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:29:10.804 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:29:22.164 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' 12-02-2021 22:29:40.691 +0800 INFO AutoLoadBalancedConnectionStrategy [10792 TcpOutEloop] - Found currently active indexer. Connected to idx=54.83.75.76:9997, reuse=1. 12-02-2021 22:29:52.369 +0800 INFO TailReader [13912 tailreader0] - Batch input finished reading file='C:\Program Files\SplunkUniversalForwarder\var\spool\splunk\tracker.log' ... View more

OK this is my final question We have a requirement to stream the events from a cloud Splunk SIEM to an on-prem SIEM This is the splunk flow: Cloud Splunk  -> On Prem Splunk Fwd (9997) -> On prem Indexer   Cloud Splunk currently sends indexA, indexB, indexC   On my Splunk Forwarder end, I only want data from indexA.   Based on https://docs.splunk.com/Documentation/Splunk/latest/Data/Monitornetworkports I configure the inputs.conf as follows: [splunktcp://9997] disabled = 0  There doesn't seem to be a whitelist for the index, it seems whatever I receive, I will forward to the indexer. I looked through the documentation , doesn't seem it can filter by index, only can filter by regex events at output.conf https://docs.splunk.com/Documentation/Splunk/8.2.0/Forwarding/Routeandfilterdatad#Filter_data_by_target_index ... View more

I say this because Splunk can filter data but it needs one or more rules in data ingestion but you're saying that there isn't any rule in ingestion and users can create every kind of ingestion. This is not what I am saying.   I am saying I have already planned the sourcetype , index and filepath/filetype that I am receiving from my user. But there's no stopping the user from putting a file that is different from the sourcetype that I am monitoring. In the case of the above scenario, what will likely happen? 1. Splunk ingest the content and forward to the indexer, if it is not picked up by the filter. 2. Splunk filters out the content, if it is picked up by the filter with the 3 methods, timestamp, file extension/file name, regular expression whitelist.     ... View more

not sure if any other users faced this issue.  I am having difficulty finding a splunk add on to parse such logs.   ... View more

At first why do you have users that can change Splunk Forwarders configurations? This is not what I want to achieve, what I want to achieve to ensure that content sent to the forwarder is clean.  Based on the solution above, you proposed whitelist, and filter and routing based on regex. Another way I heard of is to check the timestamp of the log, and if it is before xx days, it wont be ingested.   However, assuming I implement all the 3 checks, is that enough to ensure the content that is received from the forwarder is clean? I can't stop if one day the remote user put a wrong macro file , rename the extension to .log and send to me. Then, I will not be sure Splunk will just ingest the content like this or not.     Anyway if you have users that modify inputs adding stanzas that take weird logs, you could do three things:   This is not what I want to achieve, I just want the forwarder to be able to prevent those weird  entries from being ingested, if one day,  the user's server files got corrupted. Then, the file goes to my system and gets ingested in. ... View more

at first, list and analyze the data to take from a source and identify the ones that you want to index yes, we have already done this. We already used monitor on folder at the Splunk forwarder to define the sourcetype and the index. However, we can't stop if the external person send us invalid file types and weird content, it will just ingest in it. So far, I have searched Splunk answers and documentation, there is no way to ensure the content is "clean".   what whitelist can do, is to monitor file extension only For example, to monitor only files with the .log extension, make the following change: [monitor:///mnt/logs] whitelist = \.log$   or based on the file name, but can't check the content.   at least, if there are still unwanted data, you can create a filter on Indexer to delete those data before indexing (https://docs.splunk.com/Documentation/Splunk/8.1.3/Forwarding/Routeandfilterdatad#Filter_event_data_...). This one is based on specific regex expression, doesnt seem to fit in, as we are looking for a whitelist.   Thus, if I am monitoring this folder [monitor:///var/log/putlogshere] whitelist = \.log$ sourcetype=xx index=index1 and I implement the whitelist ,  the user can still send in a log with weird data which fulfil the whitelist condition and it will still be forwarded to the indexer, is that correct?         ... View more

let's say we monitor a folder which has syslog.log in it   in this log there is 3 lines   the first and 3rd lines are showing gibberish data and does not conform to the source type the second line is correct data.   Is the Splunk forwarder intelligent enough to just forward the second line and ignore the first and third line, using the whitelist method you have described?   ... View more

I will consider that anything that does not meet the sourcetype in the monitor file, which is configured at the heavy forwarder,  to be "rubbish data" is there any way to validate at the heavy forwarder end?   ... View more