About maciep

maciep · ‎03-27-2018

can you filter on time like that in the where command? If you can, very cool. If not, you might want to use relative_time() instead. ... | where LPR >= relative_time(now(),"-30d@d") AND LPR <= now() | ...

maciep · ‎03-27-2018

I'm not familiar with the freeradius, so I don't know what the logs look like. A few sample log entries would be helpful. We can help get the data parsed correctly but need to know what we're parsing.

maciep · ‎03-27-2018

I'm not even sure you need a join here. If you have one base search that gets events from both the ids index as well as the vm index (using the same filters you already have), then you will have all of the data you need in your search results already. Then you should be able to just do your stats - values() should handle the dedup for signature. And you can add an aggregate to get a distinct count of indexes and then remove any rows where that is 1. Meaning, if a DstIP is in only one index, you don't want to see it anyway, right? Does that make sense? Something like this maybe? I'm still not sure which fields for the IP exist where, but will toss a coalesce function in here to "normalize" that if needed. (index="ids" Message=remote*code*execution) OR (index=vm sourcetype="vuln" daysago=30 cvss>9) | eval DstIP = coalesce(DstIP,dest) | stats values(Message) as Message, values(signature) as sig, dc(index) as indexes by DstIP | where indexes=2 | fields - indexes

maciep · ‎03-26-2018

ok, updated the question with a sample scrubbed event. forgot the host name is in there before the json too.

maciep · ‎03-26-2018

Hi all, Currently on 6.5.2, but hopefully upgrading to 7.x in the next few months. I have some data that is basically a <timestamp> <component> <json payload> I'm wondering if there is a way that I can replace the _raw with just the <json payload> at search time. I know I can do it with EVAL/replace in props, but I'm hoping to do it before that. The end goal is to have the entire event be json by the time auto kv runs, so that Splunk will parse out all of the json fields. I have no problem extracting the json payload, but I'm struggling to find the process to actually replace _raw with EXTRACT or REPORT (both run before Auto KV). If i use EXTRACT with a named capture group of _raw , then splunk will rename it to just raw ...so that doesn't help. With REPORT , the DEST_KEY isn't available at search time, so I can't use that. I've tried using a named capture group there too with CLEAN_KEYS = false , but nothing gets extracted in that case. I tried capturing the json and then using FORMAT = _raw::$1 ... but that didn't seem to work either. Am I missing something obvious? Is this possible? Worst case, I'll try to extract that json fields with REPORT , but unless the json is pretty flat, the extracted fields aren't the easiest to work with (parent/child relationships get lost). Or maybe I'll just do it at index time and extract the component then too. Maybe someday Splunk will be able to automatically parse json/xml/etc when that structured data is just in an event (not the entire event)? EDIT Adding a sample event. Mar 26 13:44:57 myserver java {"userName":null,"enterpriseName":null,"type":"UPDATE","entityType":"letters","entityVersion":null,"assoicatedEvent":true,"eventReceivedTime":1522071897164,"entities":[{"children":null,"parentType":"letters","entityScope":"GLOBAL","lastUpdatedBy":"GUID","lastUpdatedDate":1522071896000,"creationDate":1518127649000,"alreadyMarkedForUnavailable":false,"unavailableTimestamp":0,"letters":[],"name":"server","location":null,"description":null,"productVersion":"version","status":"UP","lastStateChange":1521206768829,"disks":[{"children":null,"parentType":"letters","entityScope":"GLOBAL","name":"name:","unit":"GB","used":0.10546875,"available":0.90234375,"ID":"GUID","parentID":"GUID","externalID":null,"size":1.0087890625}],"owner":"GUID","ID":"GUID","parentID":"GUID","externalID":null,"address":IP","managementIP":"IP","BGPPeers":null,"messages":null,"peakMemoryUsage":74.54,"averageMemoryUsage":74.54,"currentMemoryUsage":74.54,"peakCPUUsage":37.79,"averageCPUUsage":25.45,"currentCPUUsage":25.57}],"diffMap":{"EntityKey ( id = GUID entityClassType = letters )":{"added":null,"deleted":null,"modified":{"currentCPUUsage":{"oldValue":"25.26","newValue":"25.57"},"peakCPUUsage":{"oldValue":"37.78","newValue":"37.79"}}}},"ignoreDiffInMediationEvents":false,"updateMechanism":"DEFAULT","requestID":null,"sourceEnterpriseID":null}

maciep · ‎03-26-2018

is there a base search in play here? How do you already have a mostRecent field? This is one of those times I wish i could just get hands on keyboard, because I feel like something's getting lost in translation here. Building a dropdown should't take this much effort 🙂

maciep · ‎03-26-2018

just to verify, are you hitting the management port? Typically, that's 8089.

maciep · ‎03-23-2018

I think the suggestion was not to upload here, but upload manually in your splunk env (probably a test box). So Settings -> Add Data -> Upload. From there you can interactively play with props config to see how Splunk reacts. You may already be doing that, but if not, it's better than waiting every day to see how the latest change you made goes. Also, don't forget to post the answer out here if Splunk Support solves the problem.

maciep · ‎03-23-2018

which version of Splunk? I just tested with a simple dashboard on my free 7.0 license, and the date/time range matches with the default value...and it updates accordingly when I select any other presets/options.

maciep · ‎03-23-2018

are you building this dashboard in the search&reporting app as well? Is it possible that your BuildID field extraction isn't shared in the same app as the dashboard your building?

maciep · ‎03-23-2018

does any search work?

maciep · ‎03-23-2018

Hmm, not sure. Is the time scope for the dropdown search the same as what you used in the search app? And i don't think it matters, but might be worth trying to drop the count field. Just add | fields - count after the stats.

maciep · ‎03-23-2018

let's break down what your search is doing and then go from there. index="ids" Message=remote*code*execution [ search index=vm sourcetype="vuln" daysago=30 cvss>9 | dedup dest | table dest | rename dest as DstIP ] So this is searching the ids index for any DstIP fields from your vm index subsearch. So at this point you will only have events from the ids index, presumably including a DstIP field. Then... | join dest [ search index=vm sourcetype="vuln" daysago=30 cvss>9 | dedup signature | table signature ] Now you are trying to join that initial result set with another search of your vm index. This search has the same criteria as the first subsearch, but it only results in one field - a signature field. So it is impossible for the join to work, because you are essentially asking splunk to join the dest field from the original results (if that field even exists there?) to a dest field in this subsearch, but that field doesn't exist. Does that make sense? This is like a sql join in that the join field has to exist in both sets of data for it to successfully join. Then | stats values(Message) as Message values(signature) as sig by DstIP So this stats will effectively just be running on your original results, since the join failed. not sure if signature, Message and DstIP all exist in the ids index, but those are the only results you'll be seeing. That's kinda why I was asking for the description of what you are trying to do because it's hard to infer the goal based on the search. It may be as simple as including the dest/DstIP in the table of your join subsearch, so you would still have the signature, but then the join would have that field to join on as well. But you may not want to dedup the signature at that point, because you may lose some dest values? Depends on your data. Or you could maybe even flip the logic, and start with your vm search and then left join to the ids on the appropriate dest field. Hopefully that helps...

maciep · ‎03-23-2018

So i think the suggestion from bearmomont was to use sort instead of max(), because it handles those values the way you want. You can do something similar to what eventstats max() with sort and streamstats...not pretty but seems to get the job done base search | sort - version | streamstats count | eval latest_release= case(count=1,version) | filldown latest_release But also if your version strings follow a standard format, then I think you should be able to pad it like you're trying as well and then use max(). You'll just to be sure you're padding every part that you need to. Meaning, you need to know how that version can vary with respect to the length of each segment.

maciep · ‎03-22-2018

Interesting. I think sort is treating these like IPs. If I replace the "." with a "|", its sorts like max(). sort By default, sort tries to automatically determine what it is sorting. If the field takes on numeric values, the collating sequence is numeric. If the field takes on IP address values, the collating sequence is for IPs. Otherwise, the collating sequence is in lexicographical order. max() Returns the maximum value of the field X. If the values of X are non-numeric, the maximum value is found using lexicographical ordering. You can manually sort these fields and see how it handles them differently. | stats count | eval b=split("10.6.3319.19,10.7.2025.33,10.6.179.135,10.7.2025.200,10.6.361.425,10.7.2025.102",",") | mvexpand b | eval c = replace(b,"\.","|") | fields b,c

maciep · ‎03-22-2018

that's quite a search. It might be helpful to explain what you're trying to do as well, because I have a feeling this could be simplified a bit. But since you are renaming everything as DstIP, that's what you should be joining on. And I'm not sure how the signature subsearch is supposed get joined since it doesn't have a DstIP. In general, I'd suggest building one search, the add on one join, then add on the next join, and so on. That way you can validate your logic/search as you go instead of trying to troubleshoot one super long complicated search from the start.

maciep · ‎03-22-2018

So are you still not seeing any notables or did this just happen over yesterday? Nonetheless, you should open a splunk support case for this, because troubleshooting it over a forum won't very efficient. And if ES just stopped working, that sounds like something support should hear about. Other than that, maybe start looking for errors/issues sometime around the last notable you saw. And of course if this is still an issue, try restarting splunk (but I'm sure you've tried that)

maciep · ‎03-22-2018

I'm not going to lie, I'm not sure I completely follow what you're trying to accomplish. But it sounds like you want to display to the user in the dropdown a BuildID/Date field to select. But you when they select one, you want to populate the token with source associated with just the BuildID. If that's the case, can you just replace that last stats with this? | stats count by BuildAndDate, source Then use BuildAndDate for the "label" field of your dropdown, and source as the "value" field. So then users are shown the BuildAndDate, but the token is populated with the associated source?

maciep · ‎03-22-2018

haven't tried it yet, but I had a similar request today and someone suggested hitting this on the management port https://<your_server>:8089/services/apps/local/<your_app>?refresh=true

maciep · ‎03-22-2018

I'm not sure that is very straight-forward in Splunk, so the search may be a bit convoluted. Do you want that count to be unique hosts in the lists? Or should it essentially be a count of rows in the results? Maybe something like this? <your search so far> | eventstats dc(host) as num_hosts | eval "host ({num_hosts})" = host | table "host *" _time

maciep · ‎03-22-2018

yep, I get that. But I typically read all answers, because I've definitely found instances where the accepted answer doesn't answer my question but one of the other answers does. And have come across posts with no accepted answers but still find my answer in there. But you're right, the accepted answer is often the one I'm looking for. And those without accepted answers probably have at one that should have been. So if everyone did the right thing out here, then it would make this a better community.

maciep · ‎03-22-2018

I know when I first started out here, I had never used a forum like this. I didn't realize that answers were just for answers and would post my comments as answers. I didn't know that I was supposed to accept an answer and so I think would just upvote them. I really didn't get it. I came from communities like myitforum, where the concept was just long threads. And since there is no "sticky" out here that tells new users what they're supposed to do, I think there's a learning curve. I also agree with others that some users are just so excited they got a solution, they're off to their environments to continuing splunk'ing. I also guess that some users may be embarrassed/surprised that somebody was able to answer their question, so they abandon it. And probably others still that maybe don't have notifications set up and don't realize that somebody answered their question, so it looks like it was abandoned. I don't think having questions unanswered is all that detrimental to the community. Most people don't come out here and then search questions. Most google their splunk problem and land up here (multiple times, on multiple answers). It's the content that I think is helpful, regardless of whether the answer is "complete". So I'm not helping because I don't press the issue. I'm here to help others because I think that's one of the great things about Splunk. And I'm also here to learn. I tell new splunkers to come out here to see if they can answer questions, because it forces you to learn more about Splunk. Karma is cool, but not my main motivator. I feel very few uses are being malicious in their (lack of) actions out here and try to give them all the benefit of the doubt.

maciep · ‎03-20-2018

Second, I think you should be able to do this in one search instead of three. And I'm not sure if your current search is accurate either? I'm assuming your verifying the numbers, but since your first search looks at all data since 2 months ago, it seems like it it's including counts from today. It doesn't look like you're doing anything to actually limit the risk value calculation to 2 months ago. Meaning, your stats is doing its calculations across every event being returned. Sure, the Etime will resolve to the first month because that's what you're telling it, but you're not actually limiting your counts to that month. If the goal is to get the risk values over time by month, then I think this would be better accomplished with a combination of bucket _time span=1mon and stats or timechart span=1mon. But I don't understand the data set enough to create an accurate search. For example, i'm not sure why you're deduping events before doing your counts. And I'm not sure why your searches start 3 weeks into the month. But I hope this helps a little.

maciep · ‎03-20-2018

First, to your question.... I think the problem is that your ETime field is just a string, so it's going to be sorted as a string. Instead of using date_month for that, maybe just eval ETime. your base search | eval ETime = relative_time(_time, "@mon") Then in your stats |stats ... first(ETime) as ETime And at the end of your search | sort ETime | convert timeformat="%b-%Y" ctime(ETime) Something to that effect. There are many different (and probably better) ways to get there, but ensuring you can sort by your ETime as a date instead of string will allow you to get the data in the right order before trying to visualize it

maciep · ‎03-20-2018

one day it clicked for me, so i'd say hang in there. Now i think of searching data more like a puzzle - i know what i'm starting with, i know what i want to end with, and i know the commands i have to work with...now just figure out how to string the commands together to solve the puzzle.

Posts	826
Solutions	118
Karma Given	98
Karma Received	251
Member Since	‎09-18-2013

Online Status	Offline
Date Last Visited	‎10-22-2024 07:27 AM

Getting nested json values into Azure Log Analytic...

Why is distsearch.conf whitelisting all json files...

Any options to speed it up a falling behind JMS Me...

Is there a way to replace _raw at search time with...

Microsoft OMS TA - Any plans to update to the API ...

Best way to move half of a SHC?

How to create a custom streaming command with pyth...

Can a scheduled dashboard pdf delivery be sent to ...

Can we ignore timestamps for some of the inputs in...

How is the partial flag supposed to work with time...

Re: Filtering on multiple times

Re: monitoring freeradius raddact files

Re: How to get at two fields from a subsearch to t...

Re: Is there a way to replace _raw at search time ...

Is there a way to replace _raw at search time with...

Re: How to display all events of a source if the s...

Re: How to perform debug/refresh for a single spec...

Re: How to onboard a large XML file without breaki...

Re: Why is the Default Preset time not matching Da...

Re: How to display all events of a source if the s...

Re: How to display all events of a source if the s...

Re: How to display all events of a source if the s...

Re: How to get at two fields from a subsearch to t...

Re: How to find the first alphabetical value?

Re: How to find the first alphabetical value?

Re: How to get at two fields from a subsearch to t...

Re: Notable events stopped updating in Incidnet Re...

Re: How to display all events of a source if the s...

Re: How to perform debug/refresh for a single spec...

Re: How can I display the count of host in the hea...

Re: why splunk community people not voting and not...

Re: why splunk community people not voting and not...

Re: If I want to display calculated indicators ove...

Re: If I want to display calculated indicators ove...

Re: Loop through a particular field counting how m...

Join the Conversation