Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- About lpolo
lpolo
Motivator
Member since:
05-10-2011
08-03-2021
Community Statistics
| Posts | 376 |
| Solutions | 17 |
| Karma Given | 73 |
| Karma Received | 137 |
| Member Since | 05-10-2011 |
Activity Feed
- Got Karma for Re: How to limit my search to return only the top 10 results based on the following search queries ?. 11-03-2024 04:54 AM
- Got Karma for How to merge two fields into one field?. 10-23-2024 09:31 AM
- Got Karma for Re: How to remove data from a bucket for a specified period of time?. 09-05-2024 08:26 AM
- Got Karma for Part 1: How to extract a json portion of an event then use spath to extract key=value pairs. 07-25-2023 01:43 AM
- Got Karma for How to merge two fields into one field?. 03-14-2023 09:50 AM
- Got Karma for Re: How to limit my search to return only the top 10 results based on the following search queries ?. 01-16-2023 05:13 PM
- Got Karma for Re: Filter values after timechart. 06-10-2022 08:59 AM
- Got Karma for How to merge two fields into one field?. 01-10-2022 05:59 AM
- Got Karma for Re: How to limit my search to return only the top 10 results based on the following search queries ?. 09-23-2021 06:18 AM
- Posted How to implement this Prometheus Query using Splunk Query Language on Splunk Enterprise. 08-03-2021 11:37 AM
- Posted Re: Source log events not forwarded after log rotation on Splunk Enterprise. 08-02-2021 08:59 AM
- Posted Re: Source log events not forwarded after log rotation on Splunk Enterprise. 07-26-2021 05:15 AM
- Posted Source log events not forwarded after log rotation on Splunk Enterprise. 07-23-2021 01:25 PM
- Got Karma for Re: How to limit my search to return only the top 10 results based on the following search queries ?. 02-08-2021 08:28 AM
- Posted Re: top 1000 by appiD on Splunk Enterprise. 07-15-2020 03:24 PM
- Posted top 1000 by appiD on Splunk Enterprise. 07-15-2020 12:41 PM
- Posted Re: Top n plus others on Splunk Enterprise. 07-06-2020 01:04 PM
- Posted Top n plus others on Splunk Enterprise. 07-06-2020 11:54 AM
- Karma Re: Regex that covers both cases for richgalloway. 06-05-2020 12:48 AM
- Karma Re: Regex that covers both cases for javiergn. 06-05-2020 12:48 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
12-14-2012
10:44 AM
1 Karma
I would transform your script to create key=value pairs results as follow:
Memtype=Mem total=3820 used=3615 free=205 shared=0 buffers=629 cached=2083
I would include a time stamp:
12/14/2012 00:00:00 +0000 Memtype=Mem total=3820 used=3615 free=205 shared=0 buffers=629 cached=2083
Then, Splunk will automatically extract each field.
Lp
... View more
12-12-2012
08:21 AM
My original config was this one:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
I changed it to:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
blacklist=.(gz|log.*|out.*|run.properties)$
disabled = false
And the source type is back to work.
Thanks,
Lp
... View more
12-12-2012
08:16 AM
My original config was this one:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
I changed it to:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
blacklist=.(gz|log.*|out.*|run.properties)$
disabled = false
And the source type is back to work.
Thanks,
Lp
... View more
12-12-2012
07:59 AM
For the benefit of doubt I tried your recommended blacklist. It worked. I do not see why the _blacklist presented in original question should not work.
Thanks,
LP
... View more
12-11-2012
06:20 PM
I disagree. I am blacklisting ".log.". See the initial question.
_blacklist=.(gz|log.|out.|run.properties)$
... View more
12-11-2012
07:10 AM
I tried both ways:
NO_BINARY_CHECK = true and NO_BINARY_CHECK = false
Still does not work. Any ideas...
... View more
12-11-2012
06:39 AM
I have a source log that sometimes contains binary characters. Splunk is not indexing any events for this source type. The source type in question was configured as follow at the universal forwarder:
This configuration should work but it is not... Any idea...
I tried with NO_BINARY_CHECK = true and NO_BINARY_CHECK = false.
Thanks,
Lp
inputs.conf:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
props.conf:
[azkaban]
NO_BINARY_CHECK = true
The problem is that this source type is not being indexed and events are being appended. Splunkd.log does not complain about it.
12-11-2012 14:24:37.318 +0000 INFO TailingProcessor - Parsing configuration stanza: monitor:///usr/local/azkaban/logs/azkaban.log.
Btool reports:
inputs
[monitor:///usr/local/azkaban/logs/azkaban.log]
_blacklist = \.(gz|log.*|out.*|run.properties)$
_rcvbuf = 1572864
blacklist = \.(gz|log.*|out.*)$
disabled = false
host = abc.com
index = default
sourcetype = azkaban
props
[azkaban]
ANNOTATE_PUNCT = True
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
NO_BINARY_CHECK = true
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
TZ = UTC
... View more
- Tags:
- indexing
11-26-2012
04:58 AM
Does splunk 5.0 require xmltojson file installed in /applications/splunk/etc/apps/xml2json?
Thanks,
Lp
... View more
- Tags:
- sdk
11-20-2012
06:19 AM
We are experiencing the same pattern you just described. We reported this issue to Splunk 2 weeks ago when we upgraded to Splunk 5.0. We increased maxRunningProcessGroups like you explained without luck.
We upgraded to Splunk 5.0.1 and it seems that the issue is not solved yet. Since we have redundant indexers we monitor the number of tsidx per hot bucket and blocked queues. We have been forced to restart splunkd at least every 36 hours and check and repair buckets to avoid indexed data corruption.
Lp
... View more
11-15-2012
07:55 AM
I clearly understand you. That is why I modified the original question to show an example.Thanks for your help.
... View more
11-15-2012
07:27 AM
I might have not laid out my question correctly. I just need the longest (length) distinct q by MAC. I modified the original question to illustrate the requirement.
Thanks for your help.
... View more
11-15-2012
06:36 AM
I fully tested your recommended query. I found that it fails identifying the longest distinct q. I updated the original question to show the problem.
... View more
11-14-2012
12:08 PM
It seems to work for a small data set but I am not sure for large data set. For large data sets it seems that is not doing its job. I have to fully test it. I will comment this question after I fully tested. For the time being it is a good progress.
... View more
11-14-2012
11:55 AM
I updated the question. It did the job. thanks I learned from your example. One more question. Should I expect any search limitations with this query?
... View more
11-14-2012
11:25 AM
I am already extracting q and MAC both are extracted automatically by Splunk. Do I need the rex command?
... View more
11-14-2012
11:09 AM
It does not find the distinct searches. It just grabs the longest q. any idea?
... View more
11-14-2012
08:20 AM
It tried. It does not do the trick. It will always select the longest string. If you add |stats count by q MAC you will get the longest by MAC but not the distinct longest by MAC.
... View more
11-14-2012
07:16 AM
I have a log that registers search queries with the following format:
_time q MAC
11/13/12 12:46:31.000 AM CA 0000000B4BE8
11/13/12 12:46:31.200 AM CARL 0000000B4BE8
11/13/12 12:46:31.250 AM CARLIE 0000000B4BE8
11/13/12 12:46:31.255 AM CARLIE B 0000000B4BE8
11/13/12 12:46:31.270 AM CARLIE BR 0000000B4BE8
11/13/12 12:46:32.100 AM CHAR 0000000B4BE8
11/13/12 12:46:32.150 AM CHARL 0000000B4BE8
11/13/12 12:46:32.155 AM CHARLI 0000000B4BE8
11/13/12 12:46:32.223 AM CHARLIE 0000000B4BE8
11/13/12 12:46:32.354 AM CHARLIE B 0000000B4BE8
11/13/12 12:46:33.400 AM CHARLIE BR 0000000B4BE8
The result set I need should be:
MAC count q
0000000B4BE8 5 CARLIE BR
0000000B4BE8 6 CHARLIE BR
The recommended query found in below answer is the following:
search construct|
| stats count by q, MAC
| eval len=length(q)
| eventstats max(len) AS longest by MAC| where len=longest
| table MAC q count
The query does not returned the distinct longest q. If this query is ran against the sample log presented before the result set is:
MAC count q
0000000B4BE8 6 CHARLIE BR
It should be:
MAC count q
0000000B4BE8 5 CARLIE BR
0000000B4BE8 6 CHARLIE BR
Any ideas? How to fix it.
Thanks,
Lp
... View more
- Tags:
- query
11-02-2012
05:42 AM
1 Karma
This method allowed me to solve this problem.
1) If Splunk identifies a non ASCI character in any event it will flag the file as binary and it will log an event in splunkd.log as follow:
10-22-2012 17:53:21.734 +0000 INFO TailingProcessor - Ignoring file '/usr/local/rex/azkaban/logs/azkaban.log' due to: binary
2) To identify non ASCI characters you can use the following linux command line. The non ASCI characters will be higlighted.
grep --color='auto' -P -n -r "[\x80-\xff]" azkaban.log
3) Solution:
Use the "Binary file configuration" in props.conf as presented in the previous answer.
Regards,
Lp
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
08-03-2021
02:58 PM
|
Karma given to
