Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About MonkeyK
MonkeyK
Builder
Member since:
05-13-2014
Monday
Community Statistics
| Posts | 264 |
| Solutions | 14 |
| Karma Given | 41 |
| Karma Received | 34 |
| Member Since | 05-13-2014 |
Activity Feed
- Posted Re: How can I run a broad search against a sourcetype that has no meaningful _raw string? on Splunk Search. Tuesday
- Posted How can I run a broad search against a sourcetype that has no meaningful _raw string? on Splunk Search. Monday
- Posted Re: Is there an efficient way to do field summary on a large set of source types? on Splunk Enterprise. 03-24-2023 01:24 PM
- Posted Re: How do I rehydrate a sparkline from a lookup? on Splunk Search. 03-18-2023 08:40 AM
- Karma Re: How do I rehydrate a sparkline from a lookup? for ITWhisperer. 03-18-2023 08:40 AM
- Karma Re: How to check 10 days prior to an event in Splunk for a failed login attempt? for PickleRick. 03-18-2023 05:41 AM
- Posted How do I rehydrate a sparkline from a lookup? on Splunk Search. 03-18-2023 05:33 AM
- Karma Re: why does mvappend fail in my alert? for bowesmana. 03-15-2023 07:41 AM
- Posted Re: why does mvappend fail in my alert? on Splunk Search. 03-15-2023 07:40 AM
- Got Karma for Re: why does mvappend fail in my alert?. 03-14-2023 02:29 PM
- Posted Re: why does mvappend fail in my alert? on Splunk Search. 03-14-2023 07:02 AM
- Posted Re: why does mvappend fail in my alert? on Splunk Search. 03-14-2023 06:52 AM
- Posted Why does mvappend fail in my alert? on Splunk Search. 03-13-2023 05:42 PM
- Posted Re: Is there an efficient way to do field summary on a large set of source types? on Splunk Enterprise. 12-11-2022 07:54 PM
- Posted Re: Is there an efficient way to do field summary on a large set of source types on Splunk Enterprise. 12-05-2022 01:43 PM
- Posted Re: Is there an efficient way to do field summary on a large set of source types on Splunk Enterprise. 12-05-2022 07:06 AM
- Posted Is there an efficient way to do field summary on a large set of source types? on Splunk Enterprise. 12-02-2022 03:34 PM
- Got Karma for Re: list all fields within a sourcetype. 06-30-2022 01:18 PM
- Karma Re: How do I list fields vertically in an email alert? for DanielPriceUK. 06-01-2022 10:53 AM
- Posted How do I list fields vertically in an email alert? on Alerting. 05-31-2022 06:59 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
11-22-2017
03:50 PM
1 Karma
a direct approach might be
index=tutorial_data status=200
| stats count by status ip_address
| sort 10 count
| append [search index=tutorial_data status=400 | stats count by status ip_address | sort 10 count]
... View more
11-21-2017
04:55 AM
old question, but i worked through a similar problem in
This question
Basically, you can use these to get at different subdomain levels
| rex field=dest_hostname "(?P<ld2>[\w_-]+\.[\w_-]+)$"
| rex field=dest_hostname "(?P<ld3>[\w_-]+\.[\w_-]+\.[\w_-]+)$"
| rex field=dest_hostname "(?P<ld4>[\w_-]+\.[\w_-]+\.[\w_-]+\.[\w_-]+)$"
... View more
11-20-2017
07:46 PM
Well then, something must be wrong.
in advanced edit, I had found
"dispatch.ttl" and set its value to 259200, (3 days). The search jobs seem as if they stay for 3 days: I can see create and expire dates like this:
Nov 20, 2017 9:20:01 PM Nov 23, 2017 9:20:04 PM
And yet I was alerted today at 12:20, the alert link (from the email) says "Page not found!" and the alert page says "There are no fired events for this alert." While job listing has the alert search job from 12:20 in which I can see the triggering event. So its as if the alert artifact has a shorter life than the alert serach.
... View more
11-20-2017
03:26 PM
I need search results involved in alerts to be available for a longer period of time than they are now (currently, nobody on my team seems to get to the alert in time to see the alerting event).
I know that this question was asked before:
https://answers.splunk.com/answers/440040/saving-alert-artifacts-for-longer-periods-of-time.html
However, I cannot make that solution work. The solution is supposed to be that I run the alter search and set its job settings to have a longer lifetime. When I do that the setting reverts to 10m the next time I open the alert search up.
in my saved search listing, I do notice that there is an advanced edit with 600+ values, including 30 ttl value. Is one of those values the right thing to set?
... View more
- Tags:
- splunk-enterprise
11-16-2017
12:35 PM
Eransh,
There are ways to create the lookup file without admin
if the file is small, you can manually create the entries and append them to a lookup file.
|makeresults | eval site="TexasOffice" src_ip="10.12.1.0/24"
| append [|makeresults | eval site="NextOffice" src_ip="10.12.2.0/24"
...repeat the above line for each office
|outputlookup append=t mysites.csv
if the file is larger and /or you have more lookups to create, you can get your admin to install the Lookup File Editor app. This will let you create and modify files, even import them based on a local CSV
Lookup will do CIDR matching. You will need to make sure that the src_ip field matches your ip field
<mysearch> | lookup mysites.csv src_ip as <ip from mysearch> OUPUT site
... View more
11-14-2017
04:48 PM
Sorry Dal, I left out the meaning of the query to keep my question from getting too complex. Generally the query looks for malware beacons by looking for traffic that is similar in period, size, and duration. The clusters are clustering on those values.
I tightened up my ability to see the nodes by throwing away all clusters that have less than 1% of the total clustered events.
In the case of my first example, this left just the one cluster, which is what I wanted to see. So maybe I could just play with the % that I throw away.
... View more
11-14-2017
08:47 AM
1 Karma
Sometimes when I am looking at search results, I would like to add several field's values to my search terms. Ideally, I'd like to click on the interesting field values and choose "add to search" so that I get the fieldname=fieldvalue added to my search. But as soon as I click it once, the search is launched and I have to find the interesting value again.
Is there a way to make "add to search" only add to search terms? Then I could launch the search when I am good and ready.
... View more
11-14-2017
05:28 AM
I have been trying to do kmeans analysis of some data. I see some of my evaluation points falling into lots of clusters, but with heavy weighting towards 1-2 clusters. Is there a way to call this out?
[my search] | kmeans k=100 col1 col2 col3 | | eventstats count as clusterConnectionEvents by CLUSTERNUM | eval culusterConectionEvents=CLUSTERNUM."(".clusterConnectionEvents.")" | | stats dc(CLUSTERNUM) as clusterCount values(culusterConectionEvents) values(connectionCount) count by
gives me a first item with
clusterCount=26
values(clusterConnectionEvents)
1(7)
10(1)
100(14)
12(100)
14(19)
16(9)
2(50247)
20(2)
37(203)
39(122)
4(472)
40(75)
48(17)
5(2)
50(16)
52(8)
53(39)
59(8)
73(3)
74(20)
75(142)
80(2)
81(13)
83(4)
84(58)
87(96)
This clearly has a huge node at cluster 2
and another
clusterCount=12
values(clusterConnectionEvents)
1(4)
14(2)
16(5)
2(59)
32(3)
4(11)
48(2)
59(148)
75(170)
84(2)
87(69)
89(5)
which clearly has nodes at clusters 59 and 75 (and maybe 2 and 87 as well).
For other items, the nodes are less pronounced. These are less interesting to me
Is there a way to score such data so that items with the vast majority of their values falling into 1-2 buckets comes to the top of a list?
... View more
11-13-2017
03:25 AM
This dashboard does not work for me. I think that it treats $result.multiplier$
in $result.multiplier$ as a literal
I get
Error in 'kmeans' command. The number of clusters ($result.multiplier$) is invalid
... View more
11-12-2017
07:50 PM
Something in this answer confuses me. Am I limited in the number of values that I can give to map? If so, how much should I limit the input set by?
Also if I augment the k value for kmeans by doing (10*number of ports * desired k), does that mean that kmeans will arrange ports as a decimal position? If not then ports will distribute over different ranges buckets. I will do some studies on this. I think that I could do stats on the buckets to determine ports/CLUSTERNAME and CLUSTERNAMES/port.
I will also try out your dashboard after I work out a few kinks in the query that I am building.
Either way on both concerns, map does solve my problem. If you want to convert a comment to an answer, I will accept it.
... View more
11-12-2017
06:28 PM
niketnilay beat me to it, so since I already thougth it through, you can also use "if"
<base search> | eval value1=if(delta_value>2,"error",value1)
... View more
11-12-2017
06:16 PM
Map is interesting, I have not used that command before. As long as I am using it, I can just feed it a list of dest_ports, no?
<search network traffic> | fields + dest_port | dedup dest_port | map maxsearches=25 search="<search network traffic> dest_port=$dest_port$ | kmeans k=10 bytes_out "
seems to work, I tried limiting to dest_port IN (80,443) and got different centroid_bytes_out values for CLUSTERNUM 1 on each port.
... View more
11-11-2017
09:30 PM
Niketnilay, I tied that but I only get 10 buckets total. I want to be able to work with 10 buckets per destination port.
So
port 80 (HTTP) would get 10 buckets
port 443 (HTTPS) would get 10 buckets
port 53 (DNS) would get 10 buckets
This is good to do because some apps send less data than others so anomalous for one app may be different than it is for another
I could write a separate query for each app (destination ports), but then I would have a hard time accounting for anomalous apps.
... View more
11-11-2017
09:24 PM
2 Karma
Sounds like you want a subsearch
Try this
EventCode=4688 [|search EventCode=5156 | fields + Process_ID Application_Name | rename Process_ID as New_Process_ID Application_Name as New_Process_Name]
If you have an index for windows events, it would be good to use that as well. The query turns Process_ID and Application_Name from event 5156 into search terms for event 4688. Since Event 4648 has different field names, we have to rename the fields from 5156.
I am not familiar with these EventCodes, so it could be that instead of New_Process_ID and New_Process_Name, you may need Creator_Process_ID and Creator_Process_Name
... View more
11-11-2017
09:09 PM
I am hoping to do kmeans analysis on my firewall traffic in a way that gives me 10 buckets for each destination port. something like:
search network traffic | kmeans k=10 bytes_out by dest_port
i know that kmeans doesn't support "by", but is there a different way to do this?
... View more
11-10-2017
01:55 PM
Perfect! Thank you, this will save a lot of indexer and search head resources!
A couple of notes on this that I learned:
One can only loadjob for jobs that ran on the current searchhead. So logon to the running searchhead first.
To stop worrying about the search being reaped, one can output to a lookup table:
|loadjob 1510343822.12851_69BC77DB-3AFA-433E-A86E-CFDBF93FB47D | outputlookup myResults.csv
after that, just search the lookup
|inputlookup myResults.csv
| search NOT user in ["MonkeyA", "MonkeyB", "MonkeyC"]
| stats count(bytes_in) by dest_ip
... View more
11-10-2017
12:41 PM
Sometimes I write a brutal search that takes a very long time to run and then realize that I need to do something more to visualize it. Is there a way to just refer to the results of my already run search as input to the extra commands?
example
My manager tells me that I need to know all firewall traffic to a group of servers over the last year.
index=firewall dest_ip=192.168.100.0/24
I run this but it takes 30 hours to run. When I get the output, we see that a lot of the traffic is not interesting
now I would like to search on my original results to filter out the uninteresting traffic, something like:
[get results from sid=1510343822.12851_69BC77DB-3AFA-433E-A86E-CFDBF93FB47D]
| search NOT user in ["MonkeyA", "MonkeyB", "MonkeyC"]
| stats count(bytes_in) by dest_ip
Is there a way to do this?
... View more
- Tags:
- splunk-enterprise
11-07-2017
08:36 PM
regex supports an "OR" operator by using a pipe character "|" between matching expressions.
For example (comes with Splunk)
(?:objectguid|guid)\s*=\s*(? [\w-]+)
... View more
11-07-2017
07:40 PM
Thanks Brian! What you are saying makes sense and I will add my vote for having it in the Add-on since I cannot have the App in Splunk ES.
... View more
11-05-2017
11:40 AM
Also not sure what what you mean, but I'll make a couple of guesses.
If you are trying to affect the value of an existing column, sometimes I do it like this:
|eval Column=if(<Condition>,<new value>, Column)
That way, if the condition is not met, the original value is not affected. If the column that you are setting does not exist yet, and you do not want it to exist if not match, I think that could could try something like
|eval Column=if(<Condition>,<new value>, null())
... View more
10-29-2017
05:48 PM
I think that i found a solution. If I use seq as defined above, I can add that to my sort and then streamstats to find if the last sequence. If the difference between them is 1, then they are sequential. So I did this:
| tstats summariesonly=t min(_time) as sessionStart
from datamodel=Network_Traffic.All_Traffic where All_Traffic.session_id!=0
by All_Traffic.user All_Traffic.src_ip All_Traffic.dest_ip All_Traffic.session_id All_Traffic.src_port
| rename All_Traffic.dest_ip as dest_ip, All_Traffic.src_ip as src_ip, All_Traffic.dest_port as dest_port
| eval _time=sessionStart
| bin _time span=1m
| stats count as connCount by src_ip dest_ip dest_port _time
| streamstats count by src_ip dest_ip dest_port reset_on_change=true
| eval seq=_time/60
| sort 0 src_ip , dest_ip, dest_port seq
| streamstats last(seq) as lastseq by src_ip dest_ip dest_port window=1 current=f
| search lastseq=*
| eval seqDiff=seq-lastseq
| search seqDiff=1
so now everythin in the result set is at least a sequence of 2. But those may not be sequential, so I do it again and then count up until the sequence difference changes.
Then I can do stats to get the max count for each connection and that will give me the longest number of consecutive periods
| sort 0 source, dest_ip, dest_port seq
| streamstats last(seq) as lastseq by src_ip dest_ip dest_port window=1 current=f
| eval seqDiff=seq-lastseq
| sort 0 src_ip , dest_ip, dest_port seq
| streamstats count by src_ip dest_ip dest_port seqDiff reset_on_change=t
| search count>1
| stats max(count) as longestSeq by src_ip , dest_ip, dest_port
I'll hold out for a while for any better answers.
... View more
10-29-2017
11:31 AM
I am trying to write some beaconing reports/dashboards. I have a few of them figured out, but now I am stuck trying to get a decent continuous beacon query.
What I would like to do is rate connections by the number of consecutive time intervals in which they appear. Ultimately, I will use multiple intervals (1m, 15m, 30m, 1h), but for now I am just trying to solve 1 minute intervals.
example data would look something like
time, src_ip, dest_ip, dest_port
10/29/17 05:01 192.168.1.15 8.8.8.8 53
10/29/17 05:02 192.168.1.15 8.8.8.8 53
10/29/17 05:03 192.168.1.15 8.8.8.8 53
10/29/17 05:04 192.168.1.15 8.8.8.8 53
10/29/17 05:05 192.168.1.15 8.8.8.8 53
10/29/17 05:01 192.168.1.21 7.7.7.7 53
10/29/17 05:04 192.168.1.21 7.7.7.7 53
10/29/17 05:05 192.168.1.21 7.7.7.7 53
in the above sample data,
the connection 192.168.1.15 8.8.8.8 53 should get a rank of 5 because it has traffic during 5 consecutive minutes
while
the connection 192.168.1.21 7.7.7.7 53 should only get a rank of 2 since its traffic only happens for 2 consecutive minutes (even though it breaks and starts back up)
I tried this
| tstats summariesonly=t min(_time) as sessionStart
from datamodel=Network_Traffic.All_Traffic where All_Traffic.session_id!=0
by All_Traffic.user All_Traffic.src_ip All_Traffic.dest_ip All_Traffic.session_id All_Traffic.src_port
| rename All_Traffic.dest_ip as dest_ip, All_Traffic.src_ip as src_ip, All_Traffic.dest_port as dest_port
| eval _time=sessionStart
| bin _time span=1m
| stats count as connCount by src_ip dest_ip dest_port _time
| streamstats count by src_ip dest_ip dest_port reset_on_change=true
(session ID is just there because both session start and session end from my firewall get mapped to Network_Traffic)
This is not good enough because the streamstats doesn't know that the adjacent events are not really consecutive intervals.
I could create a sequence,
eval seq=_time/60
But I still don't know how to compare the current against last to figure out the count
Any better ideas on how to do this?
... View more
10-26-2017
03:19 AM
1 Karma
pancontentpack is supposed to get app and threat metadata from Panorama.
I noticed that pancontentpack is only part of the Palo Alto App, not the Add-on. Since I cannot install the App to my ES search head (too bad, considering it's datamodel is more useful to me than Network_Traffic), I cannot update the metadata on ES search head.
Can I just copy pancontentpack.py to ES?
My other option appears to be to manually copy the csv contents from one search head to the other.
... View more
10-17-2017
04:02 AM
Yes, I've hit my head on that wall many times. The lookup table in my query is populated almost directly from the query part. The main difference being that the lookup table intends to summarize over all time, while the query is an accelerated report, so it summarizes over any arbitrary time range
... View more
10-17-2017
03:59 AM
Guiseppe,
The lookup will collect statistics over time, so I expect it to get very large. The subsearch will usually only summarize a small amount of the data.
In the case that I made for my question, the lookup held 19 record and the search (against an accelerated report query) returned 1 more record. As gpradeepkumarreddy guessed. The search was taking way longer than I would have guessed and consequentially timed-out
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
Monday
|
Karma given to
