About LeeMoe

LeeMoe · ‎03-08-2023

I'm feeling a real bone-head because I cannot make it work. So I bring some images..... I have a dataset built out from the initial breakdown of the JSON messages. Two actually - can I leverage them? That works. Can I leverage something from one of those? For sanity, this is my csv lookup: I can reload the lookup to have a single column as per my manufactured column above of course. Thanks for your patience!

LeeMoe · ‎03-08-2023

I strimmed back the query and can offer an update as what I offered was incorrect; the parsed fields from the JSON are logdata.Username and logdata.Password. I've tried adding the stats and the lookup one-by-one and the records returned go from many to none, the exception being this gives me all records: | lookup mirai-passwords.csv Username Password OUTPUT Password as Found I assume that's because my extracted fields are not named as this. There is a dataset where the colums are labelled as Username and Password - can I leverage that perhaps?

LeeMoe · ‎03-07-2023

I have an index with roughly 1.6 million records and want to compare the roughly 370'000 entries in the table with username/password against a mirai list. My index is searched from this point: index="myindex" | rex "message=\"(?<message>{.+})\" +path=" | eval message = replace(message, ".\"", "\"") | spath input=message This basically parses the JSON WebHooks in the index into fields. Two incoming fields are interesting, Username and Password. What I want to do is relate these into a lookup table I have loaded which has Username and Password colums too (mirai-passwords.csv). The ideal situation would be to have a count of each match (on Username/Password combo) plus have a count of matches that I relate against all of the Username and Passwords in the index (thus showing a percentage of the hits against the total volume) I thought this should work but it returns nothing. index="myindex" | rex "message=\"(?<message>{.+})\" +path=" | eval message = replace(message, ".\"", "\"") | spath input=message | lookup mirai-passwords.csv Username OUTPUT Password | stats count by Username, Password | eval TotalCount=mvindex(split(PasswordList,"|"),1) | eval PercentageCount=count/TotalCount*100 Anyone able to shed some light to help me? Thanks 🙂

LeeMoe · ‎02-15-2023

Thank you, this looks awesome:

LeeMoe · ‎02-15-2023

Rich, thank you. I now have meaningful things in my dashboard and I am grateful. Splunk is brilliant and I think you are pretty darn cool too 🙂

LeeMoe · ‎02-15-2023

Rich, thanks, you shook my head enough to go and load the file into a Lookup (duh). So now I parse my thing and get two columns for Port and Protocol based on my lookup - which is cool. Thank you so much. If I wanted to mix the fields from my index and my lookup, I am sure that's simple to you too (but not to me, sorry). Can you point me in the right direction there too? index="myindex" | rex "message=\"(?<message>{.+})\" +path=" | eval message = replace(message, ".\"", "\"") | spath input=message | lookup ports_protocols.csv Port OUTPUT Protocol | table Port Protocol

LeeMoe · ‎02-15-2023

Background: this builds on a previous question. I've now imported a CSV file into a new index and stored it in a new table/dataset (ports_protocols) index=”myindex” | rex "message=\"(?<message>{.+})\" +path=" | eval message = replace(message, ".\"", "\"") | spath input=message | lookup ports_protocols Port OUTPUT Protocol | table Port Protocol Without the lookup, it works perfectly but of course my Port field is numeric and I would like it to be more human-readable. Clearly I did something wrong up to now.....advice? Thanks 🙂

LeeMoe · ‎02-15-2023

Thanks, Rich. My mind was on the right approach. I have two datasets now, one is my main log source and the other I created as a lookup source (I hope that's right?) called "Lookup_Ports_Protocols" (_time, Port, Protocol columns). I assume I want to add a column to my main dataset that looks this up, I don't quite know how to bring that into being. Or do I need to construct a new dataset from these two? Thanks and sorry for being a bit average at this. I excel at many other things, trust me!

LeeMoe · ‎02-14-2023

I have a dataset which has a column "Port" that contains (limited) numerical values. I want to make these values display as text (e.g. 443 == HTTPS). I could do this in Excel but I'm a Splunk newbie and frankly in need of a nudge in the right direction....I assume it would be some kind of lookup? I would then pull the text values into a pivot for a dashboard to replace my current one with the port numbers. Kudos and virtual shiny things for anyone who can help 🙂

LeeMoe · ‎02-08-2023

My friend, thanks to you I have some very nice dashboards. Long live my project 🙂

LeeMoe · ‎02-08-2023

@yuanliu I salute you. I understand your solution which is eternally graceful and works. Yes, I have 40% of records that have no username or password but that's about the normal volume. I expect you have more than the 2 days of experience with Splunk I have, this is a hobby implementation 😊 🍺or 🥞 is on me, many thanks again.

LeeMoe · ‎02-08-2023

I have an OpenCanary which is using a webhook to deliver data into my Splunk instance. It works really well but my regex is a bit rubbish and the field extraction is not going well. The wizard is getting me a reasonable way but the OpenCanary moves the log items around in the rows and this foxes the wizard which seems to see the repetition and resists my attempts to defeat it when I try to take the text after some labels (namely Port which works as it's in the same location per line, Username, Password and src_host. Two lines which should help with the understanding of my challenge. message="{\"dst_host\": \"10.0.0.117\", \"dst_port\": 23, \"local_time\": \"2023-02-08 16:20:12.113362\", \"local_time_adjusted\": \"2023-02-08 17:20:12.113390\", \"logdata\": {\"PASSWORD\": \"admin\", \"USERNAME\": \"Administrator\"}, \"logtype\": 6001, \"node_id\": \"hostname.domain\", \"src_host\": \"114.216.162.49\", \"src_port\": 47106, \"utc_time\": \"2023-02-08 16:20:12.113383\"}" path=/opencanary/APIKEY_SECRET full_path=/opencanary/APIKEY_SECRET query="" command=POST client_address=100.86.224.114 client_port=54770 message="{\"dst_host\": \"10.0.0.117\", \"dst_port\": 22, \"local_time\": \"2023-02-08 16:20:11.922514\", \"local_time_adjusted\": \"2023-02-08 17:20:11.922544\", \"logdata\": {\"LOCALVERSION\": \"SSH-2.0-OpenSSH_5.1p1 Debian-4\", \"PASSWORD\": \"abc123!\", \"REMOTEVERSION\": \"SSH-2.0-PUTTY\", \"USERNAME\": \"root\"}, \"logtype\": 4002, \"node_id\": \"hostname.domain\", \"src_host\": \"61.177.172.124\", \"src_port\": 17802, \"utc_time\": \"2023-02-08 16:20:11.922536\"}" path=/opencanary/APIKEY_SECRET full_path=/opencanary/APIKEY_SECRET query="" command=POST client_address=100.86.224.114 client_port=54768 Any regex experts will help me build out pivots and reporting for my OpenCanary which gets around 200'000 connection attempts every 7 days 🙂

Posts	12
Solutions	0
Karma Given	6
Karma Received	1
Member Since	‎02-08-2023

Online Status	Offline
Date Last Visited	‎04-10-2023 11:50 AM

How to lookup against two-column table?

How to extend a dataset to add a column that repre...

How to achieve a field extraction where field diff...

Re: Lookup against two-column table

Re: Lookup against two-column table

How to lookup against two-column table?

Re: How to extend a dataset to add a column that r...

Re: How to extend a dataset to add a column that r...

Re: How to extend a dataset to add a column that r...

Re: How to extend a dataset to add a column that r...

Re: Extending a dataset to add a column that repre...

How to extend a dataset to add a column that repre...

Re: How to achieve a field extraction where field ...

Re: How to achieve a field extraction where field ...

How to achieve a field extraction where field diff...

Join the Conversation