Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About djreschke
djreschke
Communicator
Member since:
01-09-2019
2 weeks ago
Community Statistics
| Posts | 47 |
| Solutions | 2 |
| Karma Given | 12 |
| Karma Received | 2 |
| Member Since | 01-09-2019 |
Activity Feed
- Got Karma for Setup Loop - SecurityScorecard. 2 weeks ago
- Posted Setup Loop - SecurityScorecard on All Apps and Add-ons. 2 weeks ago
- Tagged Setup Loop - SecurityScorecard on All Apps and Add-ons. 2 weeks ago
- Karma Splunk Add-on for Microsoft Office 365 Reporting Web Service 2.0.0 404 ERROR- How to resolve? for Lia. 08-16-2022 10:26 AM
- Posted Re: Monitored Inputs stopped working. on Getting Data In. 06-27-2022 12:45 PM
- Posted Re: Monitored Inputs stopped working. on Getting Data In. 06-27-2022 12:41 PM
- Posted Re: Monitored Inputs stopped working. on Getting Data In. 06-23-2022 09:26 AM
- Posted Why have monitored inputs stopped working? on Getting Data In. 06-23-2022 08:27 AM
- Tagged Why have monitored inputs stopped working? on Getting Data In. 06-23-2022 08:27 AM
- Karma Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? for PickleRick. 11-24-2021 05:02 AM
- Karma Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? for PickleRick. 11-24-2021 05:02 AM
- Posted Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? on Splunk Search. 11-24-2021 05:01 AM
- Karma Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? for PickleRick. 11-24-2021 05:01 AM
- Karma Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? for PickleRick. 11-24-2021 04:56 AM
- Karma Re: Why is _time indicating UTC when the event happened in EST. for PickleRick. 11-24-2021 04:54 AM
- Karma Re: Why is _time indicating UTC when the event happened in EST. for PickleRick. 11-24-2021 04:54 AM
- Karma Re: Why is _time indicating UTC when the event happened in EST. for PickleRick. 11-24-2021 04:54 AM
- Posted Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? on Splunk Search. 11-23-2021 06:55 AM
- Posted Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? on Splunk Search. 11-23-2021 04:19 AM
- Posted Re: Is there a way in Splunk to change the timezone or a raw event from UTC to EST or vica versa? on Splunk Search. 11-22-2021 05:40 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
2 weeks ago
1 Karma
Is annoy else stuck in the Setup loop for SecurityScorecard App. I have data being ingested, and when i am asked for my domain, I select it hit save, just to be brought right back to the same screen.
... View more
- Tags:
- app
Labels
- Labels:
-
Other
06-27-2022
12:45 PM
@richgalloway limits.conf in the local folder limits.conf [thruput] maxKBps = 00
... View more
06-27-2022
12:41 PM
@richgalloway I agree, it is not a lot of files. Current ulimit settings # ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 514862 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 102400 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 514862 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
... View more
06-23-2022
09:26 AM
@richgalloway The time it stops is unknown at this time, I get alerted when logs stop being collected on a daily basis. Its a syslog server so a lot of files, 20 - 40 files depending on activity. ulimit is set at 0.
... View more
06-23-2022
08:27 AM
Has anyone run into an issue where a Splunk HF, is not monioring files being written to it. This HF is also a syslog server so files are being written to it and the monirotied inputs are on the server. The file ingestion happens after a restart. Any pointers?
... View more
- Tags:
- inputs
Labels
- Labels:
-
heavy forwarder
-
inputs.conf
-
monitor
11-24-2021
05:01 AM
@PickleRick Thank you for doing some additional testing and validation. I'll need to kick this up the flag pole to get the fix implemented. Kind of sucks that there are more then one way that Splunk reads time. I would think you would want to go off index time for everything and then change the time zone based off the users location. i know there are other variables like time zone of the log source. Thanks for all of the back and forth.
... View more
11-23-2021
06:55 AM
@PickleRick Correct no delay in indexing. The raw log is in UTC, but because there is not timezone follower, the search head is interpreting it as EST. Indextime is the correct time of the event, so wouldn't that indicatie that the event is being indexed at the right time, and the search head is miss interrupting the based on the props.conf. So there is no why to change the historical events from what you are telling. Would a better fix be change the format in the logs from the original source. How would I accomplish this. The indexers do not have any custom props deployed from the sourcetype, unless I am missing something? Data for this log is ingested from a UF that monitors a file directory. "The proper solution (but applying only to newly ingested events) is to set a proper timezone for this source/sourcetype/host on the ingesting component (first HF or Indexer that's receiving and parsing data)."
... View more
11-23-2021
04:19 AM
@PickleRick Here is the expanded log and the event for reference. I included indextime so you can see the difference. c_ip = 2.2.2.2 cs_method = GET cs_uri_stem = /test/test.aspx date_hour = 7 date_wday = tuesday date_zone = local host = machine index = test sc_status = 404 sc_substatus = 0 IndexTime = 11/23/2021 02:06:29 X_FORWARDED_FOR = 99.99.99.99 blnStartHere = True cs_Cookie_ = - cs_Referer_ = - cs_User_Agent_ = Mozilla/5.0+(Windows+NT+10.0;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/96.0.4664.45+Safari/537.36 cs_bytes = 0 cs_username = - cs_version = HTTP/1.1 date = 2021-11-23 s_ip = 1.0.0.1 s_port = 999 s_sitename = W3SVC2 sc_bytes = 0 sc_win32_status = 0 time = 07:06:28 time_taken = 98 _time = 2021-11-23T07:06:28.000-05:00 Event 2021-11-23 07:06:28 W3SVC2 host 2.2.2.2 GET /test/test.aspx 999 - 1.0.0.1. HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/96.0.4664.45+Safari/537.36 - - test.com 404 0 0 0 0 99.99.99.99
... View more
11-22-2021
05:40 AM
@PickleRick So is the change that needs to made on the indexing side or would a change in the log format be the best route. The Event is being indexed at the right time but since the is no time zone specified in the logs ie. 2021-11-22 08:35:55 versus 2021-11-22:08:35:55 -0500, event time is showing as 08 AM EST when it is indexed at 08 AM UTC. _time = 2021-11-22 08:38:18 IndexTime = 11/22/2021 03:38:49 index = index diff = 0 Search used: index=index | rename _indextime as IndexTime | eval diff=IndexTime-_time | convert ctime(IndexTime) as IndexTime | eval diff=if(diff < 0, "0", diff) | table _time IndexTime diff index
... View more
- Tags:
- props.conf
11-22-2021
05:30 AM
@PickleRick What version of Splunk are you running? I am running 8.1.1.
... View more
11-19-2021
12:13 PM
I am trying to correlate 2 different logs one is in EST and the is in UTC. The UTC logs, I have tried to specific the time zone as UTC in props and then let my user timezone preference do the conversion. Props was only updated on the search head. [soucretype] TZ = UTC This does not seem to work and i have read different post saying that updating the props file should work but not for historical events. Is there a way to do this indexed events?
... View more
Labels
- Labels:
-
field extraction
-
fields
-
metadata
11-19-2021
12:04 PM
1 Karma
@PickleRick Agreed, the UI is correct, I can't post screenshots. I have open a support ticket with Splunk. Thank you for validating that there is a difference. I'll keep this post updated as I here back from Support.
... View more
11-19-2021
10:47 AM
@PickleRick All of the time information is being parsed correctly, it is simply not so simply different on the _time field from the Web UI to the exported csv. Web UI _time is showing: 2021-11-19T11:34:02.000-05:00 CSV Export _time is showing: 2021-11-19T11:34:02.000+0000 I never noticed the difference before today but for users that don't log into Splunk, they are assuming that the CSV _time is in UTC, so they are converting this to EST which puts this event at 630 in the morning, which is not correct.
... View more
11-19-2021
10:26 AM
So when I export the log the it what it shows as _time, but i am seeing the correct _time in the field on the Search head. Is this a know thing that it drops the timezone when you export a log.
... View more
11-19-2021
09:52 AM
Can you please clarify some more? Its right that is showing 2021-11-19T11:34:02.000+0000 Or is this wrong from based of my timezone.
... View more
11-19-2021
09:39 AM
My timezone is in EST, so I would imagine the it would show something like -500?
... View more
11-19-2021
09:27 AM
_time: 2021-11-19T11:34:02.000+0000 date_hour: 11 date_mday: 19 date_wday: friday date_year: 2021 date_zone: -300 raw log snippet [19/Nov/2021:11:34:02 -0500] 2021-11-19T11:34:02.000+0000 indicates UTC. Does this indicate timezone?
... View more
Labels
- Labels:
-
eval
10-25-2021
10:07 AM
I have a props conf file that is not parsing data as i expected. I can see in the raw log that the IIS log has the header information in it. [sourcetype] DATETIME_CONFIG = INDEXED_EXTRACTIONS = w3c LINE_BREAKER = ([\r\n]+) MAX_TIMESTAMP_LOOKAHEAD = 32 NO_BINARY_CHECK = true SHOULD_LINEMERGE = false category = Web description = W3C Extended log format produced by the Microsoft Internet Information Services (IIS) web server detect_trailing_nulls = auto disabled = false pulldown_type = true When i manual upload the log file and assigning it to it will parse out the log with the same settings. [sourcetypeA] DATETIME_CONFIG = INDEXED_EXTRACTIONS = w3c LINE_BREAKER = ([\r\n]+) MAX_TIMESTAMP_LOOKAHEAD = 32 NO_BINARY_CHECK = true SHOULD_LINEMERGE = false category = Web description = W3C Extended log format produced by the Microsoft Internet Information Services (IIS) web server detect_trailing_nulls = auto disabled = false pulldown_type = true This props is on the searchhead were i am searching the data. The IIS logs is being capture via a UF. Has anyone run into this before?
... View more
- Tags:
- props.conf
Labels
- Labels:
-
field extraction
09-15-2021
08:53 AM
When I test the regex in both regex101 and using the rex command in the search bar and they parsed out the fields correctly. Now that i have added them to the props conf on the search head, they are capturing extra information. The Result field is the one that is mainly caputuring the SessionID which the the capture is Verified or Failed. Thank you all for your help with this. props.conf [exp_test] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Custom pulldown_type = true CHECK_FOR_HEADER = false CHARSET = AUTO EXTRACT-SessionID = (?<=SessionID:)(?P<SessionID>.+) EXTRACT-Result = \VerificationResult:(?P<Result>.+) EXTRACT-UserName = (?<=User:)(?P<UserName>.+) EXTRACT-Response_1 = (?<=Response_1:)(?P<Response_1>.+) EXTRACT-Response_2 = (?<=Response_2:)(?P<Response_1>.+) Sample Log Time: 13-09-2021 10:08:19 VerificationResult: Failed SessionID: K3K2N2G3JPSOZNOWJFOMFPBP.pidd1v-210913090809460797217 User: LAST, FIRST 13-09-2021 10:10:18 Response_1: 1st reqest Sent! for User: LAST, FIRST 13-09-2021 10:10:19 Response_1: 1st response received! for User: LAST, FIRST Time: 13-09-2021 10:10:19 SessionID and User Mapping: SessionID: 3EV6PLCHK795Z8FQBKKYS3Z3.pidd2v-210913091018537820706 User: LAST, FIRST 13-09-2021 10:15:13 Response_1: 1st reqest Sent! for User: LAST, FIRST 13-09-2021 10:15:14 Response_1: 1st response received! for User: LAST, FIRST Time: 13-09-2021 10:15:14 SessionID and User Mapping: SessionID: GAWJ1C7ZWNAWCVTEEIWGE3LL.pidd2v-210913091513558630064 User: LAST, FIRST 13-09-2021 10:15:33 Response_1: 1st reqest Sent! for User: LAST, FIRST 13-09-2021 10:15:33 Response_1: 1st response received! for User: LAST, FIRST 13-09-2021 10:15:38 Response_1: 1st reqest Sent! for User: LAST, FIRST 13-09-2021 10:15:39 Response_1: 1st response received! for User: LAST, FIRST Time: 13-09-2021 10:15:39 SessionID and User Mapping: SessionID: 2SYZV3QHCZKYM2YTYIJLVL3E.pidd2v-210913091538460803649 User: LAST, FIRST 13-09-2021 10:15:47 Response_1: 2nd request sent! for the user verification SessionID: 2SYZV3QHCZKYM2YTYIJLVL3E.pidd2v-210913091538460803649 13-09-2021 10:15:48 Response_1: 2nd response received! for user verification SessionID: 2SYZV3QHCZKYM2YTYIJLVL3E.pidd2v-210913091538460803649 Time: 13-09-2021 10:15:48 VerificationResult: Verified SessionID: 2SYZV3QHCZKYM2YTYIJLVL3E.pidd2v-210913091538460803649 User: LAST, FIRST 13-09-2021 10:16:47 Response_1: 1st reqest Sent! for User: LAST, FIRST 13-09-2021 10:16:48 Response_1: 1st response received! for User: LAST, FIRST Time: 13-09-2021 10:16:48 SessionID and User Mapping: SessionID: D5JVVUR3AAKFURITHCI993H9.pidd2v-210913091647448944771 User: LAST, FIRST
... View more
- Tags:
- props.conf
Labels
- Labels:
-
field extraction
09-13-2021
07:19 AM
@richgalloway We are starting to see data ingested in real from the log. No changes made in either the application nor the conf file. Before posting this question we did restart the forwarder but we were not seeing the logs ingested. Next I need to work on the Regex to help parse the fields but I can open a separate question for that. Thank you for your help.
... View more
09-10-2021
10:14 AM
@richgalloway We haven't got to prod yet and that was estimate. I imagine it will be much less. Thanks for the replies and the help thus far. More testing is scheduled for Monday at this point.
... View more
09-10-2021
06:04 AM
@richgalloway Not that big is from i understand as under 100GB. We have restarted the UF with no success. I'll have to review the logs today to see if i can find the issue, and i will share my findings. Thank you for help thus far.
... View more
09-09-2021
07:40 AM
The file is being monitored by a UF, The problem is after the initial log is ingested, if anything additional is written to the log it does not ingest that. The log source itself does not roll after a certain size, and i am told that file does not get that big, as the service is restart monthly for maintenance. I'm not seeing any errors in the logs, but i also might be looking for the wrong thing. - Any tips/advice here?
... View more
09-09-2021
05:19 AM
Good morning everyone, I am trying to ingest a log that does not roll over after a new, only when the service that writes the log is restarted. We have done some testing using cRcSalt and so far that has not helped to continually monitor the file as it is written. Any advice would be appreciated. inputs.conf [monitor://E:\Tomcat 9.0\logs\tomcat9-stdout.*.log] sourcetype = test index = test blacklist = \.(gz|bz2|z|zip)$ disabled = false CRCSALT = <SOURCE> Props.conf [test] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Custom pulldown_type = true CHECK_FOR_HEADER = false CHARSET = AUTO EXTRACT-SessionID = (?<=SessionID:)(?P<SessionID>.+) EXTRACT-Result = (?<=VerificationResult:)(?P<Result>.+) EXTRACT-UserName = (?<=User:)(?P<UserName>.+) EXTRACT-Response = (?<=Account Response:)(?P<Response>.+) EXTRACT-Second_Response = (?<=Verification_test:)(?P<Second_Response>.+)
... View more
Labels
- Labels:
-
inputs.conf
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
2 weeks ago
|
