New to Splunk. Running on an Ubuntu box, current 64ibit release, with current Splunk release.
Attempted to run with a Netgear FVS-318v3, using "netgear" sourcetype. It fails to identify events - no decodes, no events counted, and no panels populated. Sample log entry:
Wed, 2016-02-03 22:11:43 - TCP packet - Source: xxx.xx.x.x - Destination: xx.xx.xx.xxx - [Attempt to access URl: www.google.com Src 50670 Dst 80 from LAN]
Switched to WRT54g running dd-wrt, using "dd-wrt" sourcetype, and Home Monitor detected nothing going on. Identified the "br0" vs "eth0" issue described elsewhere, and corrected with an entry into local props.conf. It properly counts events ("in" only, at the moment), but no decodes, and no panels populated.
I have yet to see the app decode an event and put up a source or destination IP address, and I'm only working with simple connection events at the moment. Tests run with HM 4.3.0 and HM 4.4.0. It refuses to decode logs, and I've run out of firewalls to try.
Where does one find the map of fields that HM wants populated? I'm looking for the field definitions that HM wants filled as input, when it's parsing syslog event lines. If I want to create my own sourcetype (or correct one of the ones already bundled in HM), where do I find that information? It doesn't appear to be in props or transforms or any other conf file (yet) I've looked in.
Through some RegEx hacking, I figured out a proper expression for the FVS-318's timestamp format
%a, %Y-%m-%d %H:%M:%S, but I don't know where to find the field definitions, so that I can tell HM to use it.
Is there a trace ability in Splunk, where I can watch events get parsed against the criteria?
Thanks for whatever help you may offer.
So it looks like the Netgear FVS-318 is a sourcetype that will not work with the netgear sourcetype I wrote for the app. What that means is that when your data comes in, it's looking for matches to the fields DST as Destination IP, DPT as Destination Port etc. From what I recall, the soucetype was coming in as Name = Value pairs and all my configuration files were doing was mapping them to CIM compliant fields (e.g. DST to dest_ip, DPT to dest_port etc.)
The second router type, DD-WRT should work assuming that the data coming in follows the same Name = Value pairs that are present in the openwrt logs. If you want, put some sample events and I should be able to help write a sourcetype that you can put in your
$SPLUNK_HOME/etc/apps/homemonitor/local/props.conf file to help parse your data.
Lastly, what rich7177 described above is correct, the hostname should match your sourcetype if you want the app to automatically set the sourcetype on ingest. However, you can manually override it on the Data Inputs screen for the UDP 514 input. Just make it whatever sourcetype that you want and it should parse, assuming that the data is coming in in a way that is recognized by the app.
How does one force HM to reload? It will sometimes offer no option in the Select Your Sourcetype drop-down on Home Network Overview. It loses the sourcetype I've been working with for some long period of time. I've attempted a reload (*| extract reload=true), I've attempted a restart (.\splunk restart) and I've even attempted a full-on reboot of the machine. I can't get the sourcetype to reappear. I'll walk away for a while, and when I come back, it finally reappears for selection.
Note that I have the sourcetype set for UDP 514, rather than letting HM automatically sense it.
It would be helpful to have HM remember the sourcetype.
If you are working within the home monitor context, make sure to change the permission of the sourcetype so that it is available in the drop down. If you go to $SPLUNK_HOME/etc/users/admin you should see the props.conf with your sourcetype. If that's the case, just copy the props.conf file into your $SPLUNK_HOME/etc/apps/homemonitor/local directory. This will change the permissions from private to app. If you had saved the extractions as app, they would have automatically showed up in the app's props.conf local directory.
That's exactly what happened, thanks. I have a local props file, but the extractions themselves (done through the GUI) were in another props file in the admin\local directory. Copied to the local props file.
If you look at the
props.conf file, you'll also see that I worked on creating the direction field using an EVAL statement in
props.conf file :
EVAL-direction = if(match(OUT,"eth*"), "out", "in") What this says is to create a new field called direction, where the existing field
OUT matches the interface
eth* (traffic going out eth*) otherwise it will be considered in-bound traffic.
Taking that into consideration, you can write an entry for your router that looks for local addresses (e.g. 192.168.0.*) and creates the direction field. Assuming CIM compliant field extractions, let's say that your
src_ip is 192.168.0.1 and it's a local IP going to a
dest_ip 220.127.116.11, then we could make the entry into your
props.conf file that looks like this :
EVAL-direction = if(match(src_ip,"192.168.0.*"),"out","in")
Please note that you should place that in your LOCAL directory of the app, otherwise if you make the change in the default directory any of my updates will overwrite your work.
If that works for you, maybe you can share your props.conf entry and I can add it to the default\props.conf for other users in the community.
I also noticed in your sample data, that you have LAN at the end of the event. If your router has LAN and WAN at the end of your events, you might be able to use that to determine traffic either incoming (WAN) or outgoing (LAN). You can use the same trick from above, but use the LAN or WAN field as your matching field.
Full disclosure: I haven't seen this app until about 5 minutes ago so I could be wrong about all of this. 🙂
Sounds like you've gotten a very good start already at Splunk!
I'll start from what I see in the beginning, and I'm going to look at the netgear stuff. The inputs defined in the app are for udp 514 and are tagged as sourcetype syslog (from inputs.conf).
After that, props.conf does TRANSFORMS-changesourcetype, which calls various transforms.conf entries to change the sourcetype based on a regex match with the hostname. So, now that you see at least some data, check the "host" flyout on the left and see what hostname the data's coming in with. The stanza in transforms.conf for netgear stuff is
[netgear] # Make sure that this matches the hostname of your router, netgear is just an example. REGEX = netgear SOURCE_KEY = MetaData:Host FORMAT = sourcetype::netgear DEST_KEY = MetaData:Sourcetype
The REGEX needs to match your hostname, whatever that is, so you'd just change it to something like
REGEX = myhostname or
REGEX = 192.168.0.1.
If that is correct, the next step is to see what's being extracted. It appears the netgear should send in logs in some sort of key=value pairs, because the only real things I see in the props.conf entries for netgear sourcetypes (and wrt, for that matter) are things like
FIELDALIAS-dst = DST as dest_ip
So, Splunk's already figured out via key/value pairs that there's a field called DST which it's trying to rename to dest_ip. Do you have either of these fields? You may have to search wide open (
index=*) and then change to "Verbose" mode (upper right under the time selector) to see them.
That also means the app and dashboards should be looking for a field called dest_ip. You can use this process for quite a few fields. Unfortunately, it's only good for fields that weren't OK in a key/value pair originally (because those don't need any "changes"). You can also figure out any other missing fields by looking at the searches the dashboard panels run - they'll list out what they expect.
But, my guess is that there's something goofy on the input side, though and something small is missing (like that hostname doesn't match properly) and so the other things aren't happening.
Anyway, see what that gets you, let us know if it helps, and if it doesn't be sure to ask!
Thanks for the replies, both very helpful. I got through a primer on CIM last night, and that helped a bunch - the fields are defined by the model.
I gave up hope on dd-wrt for now, and am concentrating on the fvs-318v3. I managed to write a couple of expressions that assign src_ip, dest_ip, src_port, and dest_port properly. As a result, the dashboard now has the public IP address displayed, and I can search on ip addresses, so there is progress.
The next big challenge is determining how to write an expression that clearly identifies "in" and "out" traffic. The syslogs don't use those words in the connection messages, so direction must be computed using what you know - host ip, src ip, and destination ip.
I'd also like to start assigning eventtypes to syslog events, because there are messages other than simple connection events, and they don't follow the same pattern as my sample log entry (above). Reading suggests that using punctuation marks to infer eventtypes is the proper way to do it. Is that accurate?