Hello all. I'm trying to get this to work with tomato, and I'm having no luck. I'm following the instructions that are on the wiki but still nothing. My setup may be a little different than the instructions, so possibly i'm missing something basic.
My logs that are coming over syslog come in as:
In lots of the documentation it says to specify the hostname of the router, but given that it is coming in as an IP and not a hostname, I don't think that would work.
I've followed https://github.com/amiracle/homemonitor/wiki/Issues-with-Setup-Page-%28404-Error%29-Fix---work-aroun... to get the main setup working. So my files look like this:
is_configured = 1
TRANSFORMS-homemonitor = index_redirect_to_homemonitor
REGEX = .
DESTKEY = _MetaData:Index
FORMAT = homemonitor
REGEX = 192.168.40.1
SOURCEKEY = MetaData:Host
FORMAT = sourcetype::tomato
DESTKEY = MetaData:Sourcetype`
What else am I missing to make this work properly to read the tomato log files?
Apart from some formatting breakage, that looks OK.
While it sounds like you have a good handle on how to operate Splunk, let's check a few basics in case they got missed.
Index homemonitor exists? (Check settings/indexes).
Data's coming in? (Check search like
index=* 192.168.40.1 over all time, might be in the wrong index)
You've restarted Splunk on the indexer to confirm that's not the issue?
So, check those couple of things and let us know what you find, we can troubleshoot each individually.
Actually, looking a bit farther at this specific link you provided (sorry I didn't catch it because it didn't come through as a link in the post)...
BTW, is your indexer windows or linux?
Let me include what I did, see if this helps. I may be able to set this back up later today and help more (heh, this question reminded me I need to do that, in fact!)
I have it removed (testing something) but when I was using Home Monitor I used syslog-ng to snag my syslog stuff on 514. This just makes life easier once it's set up. I'd recommend doing that, though it might be a bit of a scope change. 🙂
Because of that, though, I have my homemonitor/local/inputs.conf look like
[monitor:///var/log/remote/192.168.0.1/log.txt] disabled = false index = homemonitor sourcetype = westell_2200
That sets the right index and sourcetype. IIRC in my case my FW is sort of close to a bunch of the existing sourcetypes but had a few fairly major differences, so I copied one out of props.conf into a local version and renamed it then changed it as appropriate...
[westell_2200] FIELDALIAS-dst = DST as dest_ip FIELDALIAS-dpt = DPT as dest_port ... (more stuff which won't matter because you can use the built in tomato)
That was about all it took. Setting index and sourcetype and making sure the sourcetype referred to an appropriate stanza in props.conf.
Now, you imply from your workaround that you have something else listening on 514? Or something? That article isn't .... something. I'm not sure what it isn't. But if the problem is that your data is already coming in on UDP514, why do you need to enable the UDP514 input (last step in the answer). Hmm. More information may be needed once we figure out what's really going on.
Thanks for the reply!
I'm using splunk on Linux. I can see how syslog-ng would make things cleaner, but for now that's not something I was planning on exploring.
I think I got this working.
The data was being set to the indexer of homemonitor, but the sourcetype was set to genericsingleline instead of tomato like I needed it to.
I think that the default inputs.conf also wasn't being read properly. In default/inputs.conf, it shows a line that is
sourcetype=syslog, notice that there is no space before and after the equal (not sure if that actually matters). I created local/inputs.conf, set
sourcetype = tomato, and that seemed to do the trick. That transformed the data the correct way, and now it's being tagged correctly for homemonitor to see the data!
Next step is to get the bandwidth monitor part of this working, but i'll tackle that at a later date. Thanks for the pointers.
So the reason the default inputs.conf is set to syslog is that I originally had the system look at the hostname of your router and try to determine the source type that way. So if the hostname was tomato, then it would have automatically transformed the source type to tomato. Check out the transforms.conf in the default directory.
By hardcoding the source type in the inputs, you enable the source type manually and negate the transform condition. What's wrong with the bandwidth monitor?
OKay, still struggling with this then. I realized then all of my udp/514 was getting tagged as tomato traffic, which definitely isn't what I wanted...I just want traffic from 192.168.40.1 to be tagged as tomato
I tried this in homemonitor/local/transforms.conf but this still didn't work.
Make sure that this matches the hostname of your router, tomato is just an example.
REGEX = ^host::192.168.40.1$
SOURCEKEY = MetaData:Host
FORMAT = sourcetype::tomato
DESTKEY = MetaData:Sourcetype
Ultimately, I just need a way to get it to pick this up based on the IP instead of a regex string. I'm probably missing something basic. I also tried
REGEX = ^192\.168\.40\.1$ and that wouldnt work correctly either.
If you're trying to have it match based on the IP of the host, then just put the IP in the REGEX stanza:
[tomato] REGEX = 192.168.40.1 SOURCE_KEY = MetaData:Host FORMAT = sourcetype::tomato DEST_KEY = MetaData:Sourcetype
What it's doing is matching based on the Host key, so if the data is coming in from your tomato router and the 'host' is set to 192.168.40.1, then it will match and then change the source type from syslog to tomato. transforms.conf
Let me know if that helps you out.