About mathiask

mathiask · ‎10-01-2015

Hi Splunkers Any Update on this front? Use Case Im forwarding Traffic from different sources to the stream processing instance. But I would like to later be able to distinguish the traffic from the different sources. Solution 1 : Create for each source a separate processing instance ... Solution 2 : Install multiple ufwd on the instance using different identifiers Solution 3 : Process all the traffic on the same instance, but have it tagged by receiving interface name on the streamfwd. Solution 3 would be the best. Solution 2 might work ... Greetings

mathiask · ‎09-03-2015

@Re Sounds good. Regarding _internal index. If I remember correctly this is documented in the steams setup documentation something like whitelist.2.index(_internal, ...) this should do it right? @"query" and "answer" Splunk Strams App allows to configure the data extracted by the streamfwd. This consists of multiple tiers of extraction. flow. declare layer 2-4 information, so flow information without "DPI", extracted by the streamfwd dns. declare DNS data extracted by the streamfwd In the dns.xyz variables there is nothing that directly maps to the Splunk Search field "answer". Also in the forwarded JSON data there is no key/value pair "answer" : "1.2.3.4". So at the raw Event layer "answer" does not exists yet. And since extracting the correct "answer" So somewhere the answer field is extracted from the event/field. Doing this properly needs a bit of processing of several fields and is not that easy (I'm still working on it). not all IPs in host_addr are answers not all host_addr.index < ancount are answers The question is which Elements in the JSON are required to extract the "answer" field. Do I interpret this correctly: currently the answer field contains the value of host_addr if there is only one IP in the field.

mathiask · ‎08-28-2015

Sounds good We are now using the current version to get a feeling what is possible etc. , i.e. we are running a pilot What I would like to know is Which flow fields do we really need to have the "query" and "answer" field Im pretty sure these fields you don't (currently) get from the streamfwd and they are not selectable in the Streams config Dashboard. I assume you use some splunk magic to extract it. We need to know what is the minimum data footprint, that at we can estimate the needs, or at least have some facts, when the pilot outcomes will be discussed .. we don't know is usually a bad argument and it gives the critics a free pass "too much data .... too expensive ... dont like the color ..." We are aware that working on wire data is everything but trivial (yay MPLS), as we also build some tools. So I have the utmost respect for this constantly improving Tool. For our other use case we build a HTTP dissector. We cannot use Splunk Streams since we only tap outgoing traffic, and Splunk streams requires bi-directional traffic. We understand why you do, and we tested it, but with our current architecture it is not viable. PS: our prototype is VERY basic, and we are aware that we only have a partial/limited view, but it already helped us in one case to confirm our suspicions.

mathiask · ‎08-24-2015

Hi As I mentioned in a previous post http://answers.splunk.com/answers/291111/splunk-app-for-stream-when-analyzing-dns-data-is-t.html we try to build passiv DNS from Wire Data, alternatively we could use Proxy logs, but many of our customers don't have a proxy. pDNS is a great Tool to track down malware What is the minimum needed processed DNS response Query Field containing the domain requested, present in splunk streams in as query field Response Field, multivalue, containing the valid responses given, currently present as answer field, but not always Additional Information that is used/should be used to prevent malicious behaviour or generally used in pDNS matching DNS request to the response, to prevent spoofing otherwise someone can poison the pDNS DB by simply sending responses, mostly given by splunk stream flows. Depending on where the traffic is captured, like internally, this is also less needed. IP and/or hostname of the responding server, usually present as dest_ip Additional Information to track down internal problem client IP/hostname to, usually present as src_ip Generally Splunk Streams is nearly there 😄 I hope this helps

mathiask · ‎08-21-2015

Hi Splunkers Thank you so much for implementing the splunk stream fields "query" and "answer" This already help a lot I'm still wondering how you solved this, assuming it's in Splunk logic and not in the streamfwd not that I will turn of some required field that is needed to compute it. But there is some room for fine tuning The Current implementation handles Single Responses i.e. www.domain.tld 1.2.3.4 just fine CNAME www.domain.tld CNAME www-domain.tld, www-domain.tld 1.2.3.4 What it seems to not yet being able to handle are Queries with multiple responses like google.com ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 24 IN A 173.194.40.35 google.com. 24 IN A 173.194.40.36 google.com. 24 IN A 173.194.40.32 google.com. 24 IN A 173.194.40.39 google.com. 24 IN A 173.194.40.40 google.com. 24 IN A 173.194.40.33 google.com. 24 IN A 173.194.40.41 google.com. 24 IN A 173.194.40.46 google.com. 24 IN A 173.194.40.38 google.com. 24 IN A 173.194.40.37 google.com. 24 IN A 173.194.40.34 also it seems (we are still verifying this) queries of authoritative name server like ;; QUESTION SECTION: ;ns4.google.com. IN A ;; ANSWER SECTION: ns4.google.com. 345600 IN A 216.239.38.10 ;; AUTHORITY SECTION: google.com. 80463 IN NS ns1.google.com. google.com. 80463 IN NS ns4.google.com. google.com. 80463 IN NS ns3.google.com. google.com. 80463 IN NS ns2.google.com. ;; ADDITIONAL SECTION: ns1.google.com. 80463 IN A 216.239.32.10 ns2.google.com. 80463 IN A 216.239.34.10 ns3.google.com. 80463 IN A 216.239.36.10 Thank you

mathiask · ‎08-20-2015

Okay I think now I managed to fix it INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false TIME_PREFIX = "timestamp" MAX_TIMESTAMP_LOOKAHEAD = 50 NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = timestamp category = Structured This seems to export and index the JSON fields at Index Time therefore no later processing needed With the TIME_PREFIX I think i can reduce the lookahead Thanks all

mathiask · ‎08-20-2015

Thank for your help I think this topic i now found covers it better http://answers.splunk.com/answers/223095/why-is-my-sourcetype-configuration-for-json-events.html The issue I created by using INDEXED_EXTRACTIONS = json KV_MODE = json Changing to INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false Fixed it, but now I wonder if where I currently index all the json fields (which might cause quite some indexing) instead of only _time, source, host, sourcetype

mathiask · ‎08-20-2015

Okay I will try that .. I also found the Time_PREFIX option But I did not use it because it does not explain why the url gets extracted twice

mathiask · ‎08-20-2015

At search-time, several fields get extracted more than once, even if they only exist once in the event. I know I can dedup the search, but this is fighting the symptom not solving the problem The Question is, what config do I have to change to get this fixed? Issue: The fields "url" and "timestamp" show up twice with the same value in the search timestamp = 2015-08-20T12:03:33Z timestamp = 2015-08-20T12:03:33Z url = http://www.switch.ch/ url = http://www.switch.ch/ Partial Example Event, in the log it is in one line { <other stuff> <other stuff> <other stuff> <other stuff> <other stuff> timestamp: 2015-08-20T12:03:33Z <other stuff> url: http://www.switch.ch/ <other stuff> } [sourcetype] INDEXED_EXTRACTIONS = json KV_MODE = json MAX_TIMESTAMP_LOOKAHEAD = -1 NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = timestamp category = Structured pulldown_type = true

mathiask · ‎08-07-2015

Thanks So if I got that right there are these options Option 1 : Build 2 Clusters, comes at the cost of roughly 1 additional ClusterMaster/Search Head Option 2 : disable replication for special index, route data manually, other indexes still get replicated to secret indexes Is there maybe technically an option to abuse multi site clusters? But i can already get the additional complexity might not be worth the trouble.

mathiask · ‎08-07-2015

Hi I'm wondering if there is a way to control on which indexers an index resides E.g. there are 5 Indexers (+all the other infrastructure like search head, cluster master etc) 3 of them should hold normal system logs 2 of them should hold confidential data with different protection needs What options are there to control that just_a_normal_index resides on the normal 3 indexers, but the super_secret_danger index will only reside on the 2 other indexers, besides creating 2 own clusters.

mathiask · ‎08-05-2015

I just want to short explain the use case (and maybe hopefully increasing the incentive) I would like to perform passive DNS lookups. pDNS has proven to be a valuable tool within the security community. Having the data structured will help greatly in achieving that. Currently the data is kinda structured when I compare the _raw Event, when i compare it with the dig response name{} contains the left column host_type{} contains the middle column hostname{} contains the hostnames of the right column host_addr{} contains the right column each of the list is sorted from top down. But since the results/right column are split into two lists it is quite a pain in the ass to wiggle it out. Based on these assumptions I managed to mvzip the data together index="esm-url-monitor-stream" | rename "hostname{}" as hostname, "host_addr{}" as host_addr, "host_type{}" as host_type, "name{}" as name | eval a=mvzip(name,host_type) | eval b=mvzip(name,host_type) | eval a=mvfilter(match(a,"address")) | eval b=mvfilter(!match(b,"address")) | eval a=mvzip(a,host_addr) | eval b=mvzip(b,hostname) This produces a (IP result list) and b (hostname list) as comma separated string containing the response. These can now probably again taken apart and then further processed and later presented in a proper way. But I am not sure if this might still work for DNS requests having multiple requests or DNSSEC etc

mathiask · ‎08-05-2015

Thanks. This would really help.

mathiask · ‎08-03-2015

Is there a way to receive the result a bit more structured? Currently the result is kinda mess host_addr contains a list of IPs hostname contains a list of hostnames name contains a list of domains Each list seems sorted lexicografically Is there a way to determine how the actual response looked like? Or at least which hostname is associated with which A AAAA or CNAME? Greetings Mathias

mathiask · ‎12-02-2014

btw im using Splunk Enterprise and not Splunk Cloud, and there i cannot find the sourcetype json_predefined_timestamp

mathiask · ‎12-02-2014

Hi I'm trying to get JSON data from a message queue into splunk. This works very well but the imported events also contain some message header information Tue Dec 02 15:45:56 CET 2014 name="amqp_msg_received" event_id="null" msg_queue="dns-events2splunk-splunk" msg_exchange="dns-events2splunk" msg_body="{"timestamp":"2014-12-02T14:45:56.021Z","src":"192.168.16.56","src_port":"53","dest":"192.168.0.2","dest_port":7521,"dns_meta":{"status":"NOERROR","id":42347,"flags":["qr","rd","ra"],"queries":"1","answers":"1"},"dns_query":{"domain":"lopq.ch.","class":"IN","type":"A"},"dns_response":{"domain":"lopq.ch.","ttl":8477,"class":"IN","type":"A","IP":"10.14.183.106"}}" How can i now get a sourcetype that using the timestamp within the json object correctly extracts the kv pairs as well as the json kv pairs Alternatively only taking the msg_body and correctly extract the json object

mathiask · ‎10-22-2014

You probably have to learn it ... http://docs.splunk.com/Documentation/Splunk/6.1.4/SearchReference/rex http://docs.splunk.com/Documentation/Splunk/6.1.4/Knowledge/AboutSplunkregularexpressions To give you a short introduction about the string I wrote - it will try to find the pattern defined in the expression - the part within the round brackets defines one return value and the field/key it will be associated with is defined in the arrow brackets following the question mark. so the first part will look for a pattern that starts with one or two digits (defined by the backslash d{1,2}, followed by a slash (defined by backslash slash) followed by again one or two digits and a slash followed by for digits. The result will be returned as a value and given to the key submit_date. The backslash s plus stands for one or more Whitespaces, this is not withing round brackets since we don't need the space so it looks for something like d(d)/d(d)/dddd which will match the 10/21/2014 so you get submit_date=10/21/2014 This pattern also allows that 9/9/2014 will be extracted. The pattern is very specific and will not match a/bb/2014 since a and b are not numbers. I did this because there is not much context before or after the time string like "submit time 20/10/2014 12:05:15 AM" Keep in mind this is now only a string. To do some math, difference etc you first have to convert it into a time value with strptime and then convert the result back using strftime

mathiask · ‎10-22-2014

If this is the timestamp of the event then on index time this should be correctly extracted Everything else can be done later during search time In the case this is the Event time then simply use the _time field and extract of it what you want using the eval function with strftime http://www.wowhead.com/news=242815/preparing-for-6-0-guide-to-removed-mounts-pets-transmog-gear-titles-achievements If it is not the event time, extract the strings during search time and process them accordingly using eval and strptime To extract it use a regex like (?<submit_date>BLd{1,2}BL/BLd{1,2}BL/BLd{4})BLs+(?<submit_time>BLd{2}:BLd{2}:BLd{2}BLs+BLw{2}) not tested but should be close enough to figure out PS i had the removal of < and > and backslash ... BL stands for backslash since it wont be printed here ... Good luck 🙂

mathiask · ‎10-20-2014

Okay I just try to understand the reasoning behind it. In the "basic" rabbitMQ tutorials the pub and sub always make sure that the queue exists and therefore never encounter this problem. Is it because as consumer/subscriber it is out of your scope/control what the AMQP compliant provider does and therefore "simply" (have to) assume that everything is already set up properly? The quite real problem I ran into is that until I figured out what was the problem rabbitMQ created quite a log. Even more so when i restarted the MQ server/system which killed the non-persistent queue. The error log quickly grew to 12GB which then disabled the splunk indexer (<5GB) on my splunk test device. It feels a little bit like a local DoS ... Of an operational perspective I need the error to know that there is something wrong, but it kinda sucks when this actually breaks the system 🙂 Sure I could turn off the ERROR log level, which would be kinda stupid ... PS : the same problem arises with a bad password, but at a much slower rate ...

mathiask · ‎09-29-2014

Okay I think I now see the definition problem... The underlying definition problem is what the scope of the term 'indexed' covers - The event/data as a whole can be called indexed if it is listed by any parameter in any index, i.e. splunk context: the event is indexed and therefore searchable - The field/part/subset of the event/data by which the event/data is indexed, i.e. splunk context: index time extracted fields, by which the event gets indexed, http://docs.splunk.com/Splexicon:Indexedfield So to be more precise. - Yes the event (including message envelope/properties) is indexed by the default indexed fields and therefore can be retrieved 'faster' using the index, i.e. splunk context: the event and the message envelope/properties are searchable - No the event is not indexed by additional data found in the message envelope/properties and therefore cannot be retrieved 'faster' by using parameters found in the message envelope/properties, i.e. splunk context: no index time field extraction for the message envelope/properties

Posts	73
Solutions	3
Karma Given	16
Karma Received	33
Member Since	‎06-18-2014

Online Status	Offline
Date Last Visited	‎09-16-2020 06:52 AM

KV Store field type cidr

Apache Addon upgrade to Splunk 7.x

Splunk Stream: REST API tokens

Splunk Stream: Why do I receive the error "Unable ...

Splunk Stream: How to configure streams using REST...

Is it possible to use a lookup at index time, not ...

Splunk Stream: How to manage whitelists and blackl...

Is it possible limit user access to an index on a ...

Splunk Stream: How to configure forwarder and inde...

How to connect Splunk to MongoDB, or is there a KV...

Re: Get attributes with NIC and VLAN IDs?

Re: Feature Request: DNS improved query answer Fie...

Re: Feature Request: DNS improved query answer Fie...

Re: Feature Request: DNS improved query answer Fie...

Feature Request: DNS improved query answer Fields

Re: Why are several JSON fields getting extracted ...

Re: Why are several JSON fields getting extracted ...

Re: Why are several JSON fields getting extracted ...

Why are several JSON fields getting extracted more...

Re: Is there a way to control on which indexers an...

Is there a way to control on which indexers an ind...

Re: Splunk App for Stream: When analyzing DNS data...

Re: Splunk App for Stream: When analyzing DNS data...

Splunk App for Stream: When analyzing DNS data, is...

Re: How to get a sourcetype of JSON mixed with tex...

How to get a sourcetype of JSON mixed with text th...

Re: Splitting a Field into multiple values - separ...

Re: Splitting a Field into multiple values - separ...

Re: Is it correct that the AMQP Messaging Modular ...

Re: AMQP Messaging Modular Input Documentation: Wh...