Did you check this  https://community.splunk.com/t5/Dashboards-Visualizations/Is-there-a-way-to-remove-the-Real-time-part-that-appears-from/td-p/311684 https://community.splunk.com/t5/Dashboards-Visualizations/How-to-hide-Real-Time-Presets-option-in-the-time-range-picker/td-p/210350 ... View more

The better approach of indexing events with correct timestamp is to tell Splunk timezone of events. If you set TZ correctly for your events, you will not need to convert event timestamps for the users in different timezones manually with SPL, instead you can set user TZ in the user profile. This will automatically change timestamp of events. for example: event 1 occurred 00:00 UTC The same event time will be shown 8PM ETC if the profile is set to ETC.    you mentioned that 8PM ETC is being categorized 12AM UTC and its coming next day. you can avoid this by setting TZ for your events in props.conf. [sourcetype] TZ=GMT-4 when you apply above setting in HF/Indexer, users in ETC(by the way I dint find this right timezone)  will see that event occurred at 8PM only not 12AM. ... View more

Simple regex: | rex “\"run\.name\"\:\"(?<runname>[^\"]+)” run.name will be extracted to runname field.   ... View more

can you try monitoring different file from same universal forwarder and this input stanza should use default routing which lbssl (I mean don't define _TCP_ROUTING). check if you see data coming into your AWS indexers. if you see data coming from the new input that means there is issue with fishbucket. ... View more

@isoutamo , updated my answer. @mshakeb , please use updated answer, that should work for you now. up vote, if that works. ... View more

I believe it would have been worked if there was no key set to private key. In my opinion there is secret key set to privkey and that might be causing issue. ... View more

Data flow between universal forwarder and Indexer universal forwarder -> Heavy forwarder(optional) -> Indexer if you have heavy forwarder in place, I recommend using HF because the purpose of placing HF in between UF and Indexer is to parse/clean the data. Indexer is busy in serving requests coming from Search Head. if you don't have HF in the flow , you can apply on Indexer. Hope this helps... up vote, if it solves your issue. ... View more

download TA from https://splunkbase.splunk.com/app/2758/  use below props.conf and transforms.conf in local directory. make sure your sourctype of bluecoat logs is matching with "bluecoat:proxysg:access:syslog" props.conf     [bluecoat:proxysg:access:syslog] pulldown_type = true category = Network & Security description = Data from Blue Coat ProxySG in W3C ELFF format thru syslog KV_MODE = none SHOULD_LINEMERGE = false EVENT_BREAKER_ENABLE=true MAX_DAYS_AGO = 10951 REPORT-categories = bluecoat_categories REPORT-bluecoat_custom = REPORT-bluecoat_custom FIELDALIAS-cookie = cs_Cookie as cookie FIELDALIAS-duration = time_taken as duration FIELDALIAS-src=c_ip as src FIELDALIAS-src_port = c_port as src_port FIELDALIAS-user = cs_username as user FIELDALIAS-http_referrer = cs_Referer as http_referrer FIELDALIAS-status = sc_status as status FIELDALIAS-action = s_action as vendor_action FIELDALIAS-http_method = cs_method as http_method FIELDALIAS-content_type = rs_Content_Type as http_content_type FIELDALIAS-dest_host = cs_host as dest_host FIELDALIAS-dest_port = s_port as dest_port FIELDALIAS-user_agent = cs_User_Agent as http_user_agent FIELDALIAS-dest_ip = cs_ip as dest_ip FIELDALIAS-dvc = s_ip as dvc FIELDALIAS-bytes_in = sc_bytes as bytes_in FIELDALIAS-bytes_out = cs_bytes as bytes_out FIELDALIAS-uri_path = cs_uri_path as uri_path FIELDALIAS-uri_query = cs_uri_query as uri_query FIELDALIAS-protocol = cs_protocol as protocol FIELDALIAS-packets_in = c_pkts_received as packets_in FIELDALIAS-session_id = s_session_id as session_id EVAL-app = "Blue Coat ProxySG" EVAL-dest = coalesce(dest_ip, dest_host) EVAL-bytes = bytes_in + bytes_out EVAL-url = coalesce(cs_uri, if(isnull(cs_uri_scheme) OR (cs_uri_scheme=="-"), "", cs_uri_scheme+"://") + cs_host + cs_uri_path + if(isnull(cs_uri_query) OR (cs_uri_query == "-"), "", cs_uri_query)) EVAL-product = "ProxySG" EVAL-vendor = "Blue Coat" EVAL-vendor_product = "Blue Coat ProxySG" # Eval action to blocked if sc_filter_result is DENIED EVAL-action = case(sc_filter_result=="DENIED","blocked") LOOKUP-vendor_traffic_action = bluecoat_proxy_action_lookup vendor_action OUTPUTNEW action, transport     transforms.conf Note: the fields after field4 may not match with your field format, identify and replace right field names in below FIELDS     [REPORT-bluecoat_custom] DELIMS = " " #Below was used before remove default string appended by syslog-ng FIELDS = "field1","field2","field3","field4","date","time","time-taken","c-ip","cs-username","cs-auth-group","s-supplier-name","s-supplier-ip","s-supplier-country","s-supplier-failures","x-exception-id","sc-filter-result","cs-categories","cs(Referer)","sc-status","s-action","cs-method","rs(Content-Type)","cs-uri-scheme","cs-host","cs-uri-port","cs-uri-path","cs-uri-query","cs-uri-extension","cs(User-Agent)","s-ip","sc-bytes","cs-bytes","x-virus-id","x-bluecoat-application-name","x-bluecoat-application-operation","x-bluecoat-application-groups","cs-threat-risk","x-bluecoat-transaction-uuid","x-icap-reqmod-header(X-ICAP-Metadata)","x-icap-respmod-header(X-ICAP-Metadata)" [bluecoat_proxy_action_lookup] filename = bluecoat_proxy_actions.csv case_sensitive_match = false [bluecoat_categories] SOURCE_KEY = cs_categories REGEX = (?<category>[^;]+) MV_ADD = true     The reason why default transforms doesn't work is because you are collecting events through syslog server, and syslog server is appending data , time, and host to actual message coming from proxy. up vote if this works. ... View more

@payal23 , you can give any index based on your requirement, if you give different index other than indexA and indexB used in transforms, you will be ending up with total 3 indexes for same input. hope this clears your doubt.   up vote if my solution works for you. ... View more

can you verify splunk listen is enabled on Splunk DMZ Heavy forwarder. by running below command  $SPLUNK_HOME/bin/splunk cmd btool inputs list | grep 9997 output should be like below: [splunktcp://9997] if you don't see the above output, that means 9997 is not enabled on DMZ HF, but port is opened on firewall. you can enable by running below command repalce username and password with actual. splunk enable listen 9997 -auth <username>:<password> Try to generate some traffic on 9997 to DMZ HF from where you are saying telnet is working.   telnet yourdmzip 9997 -> enter   check if it says connected or not, if its connected, type something random text and don't close the session. check what IP you are seeing in DMZ HF by doing netstat command    netstat -anp | grep 9997   ... View more

make sure you are not setting up key while generating CSR. Splunk web can't open privatekey if its set with key. public.pem should be in below order: server signed cert intermediate cert root cert PrivKey.key is the one generated before generating CSR and the PrivKey.key should not be set with any secret key. [settings] enableSplunkWebSSL = true privKeyPath = $SPLUNK_HOME/etc/auth/mycerts/PrivKey.key serverCert = $SPLUNK_HOME/etc/auth/mycerts/public.pem   ... View more

Can you search “fields” in your proxy logs where you will see header of the logs, bluecoat version etc. if you found that, please share here. ... View more

Where is the input.conf defined for suricata?   ... View more

@mshakeb , There is no standard TA for bluecoat proxy which works for all versions of bluecoat proxy. log format is changed based on version and how you format the log in the proxy. if you can share sample log or the variables used to format the log in bluecoat (preferred) is helpful to help you. ... View more

Great. You are welcome. ... View more

@nabeel652 You can do this with SPL easily. ... View more

Can you verify question once. Are you trying to get subnet/subnet mask for range of ips ( when you input first and last ip)? Netmasks (or subnet masks) are a shorthand for referring to ranges of consecutive IP addresses in the Internet Protocol.   ... View more

You could say its problem/issue with TA if you are observing same behavior for all the inputs you have created using the TA. I would recommend creating a new input which you don’t think the events of this input are not duplicated with other input in AWS. ... View more

Upvote is appreciated if my solution works. ... View more

Even I have same problem like we get security logs from different applications on monthly basis. The applications are customer specific so they don’t have proper logging mechanism. Sometimes we had to get them from debug logs where each event would further split into many events. what I do is I onboard all the logs required for security monitoring, identify the required event types by just simulating few scenarios where high risk is involved and apply props|transforms, the rest of events which are not matching with my event types will be discarded. I know this is not a solution for your question. Kind of feedback you can consider. There is no such option to automate props and transforms on unknown data/events. we can can automate field-aliases if the event is in key-value fair, not tagging of events. tagging of events is most important to make use of DATA MODELS. ... View more

Its based on triggered time and field set you choose while enabling throttling while creating the correlation rule. ... View more

deployment server is not recognizing the app duke_nix_forwarder_inputs Can you check in deployment server you see app or not under apps. If yes try to deploy from GUI. ... View more