I've got some data in Splunk that I'd like to put on the maps app but the only data I have are physical addresses, e.g. "123 Happy St, Gleeville 12345". How can I convert these fields into something usable by the maps app? ... View more

I am trying to extract a field from logs that look like this: Apr 28 07:45:22.992 On [2:18]20.5.4.1:5070 sent to 102.11.130.135:50953 ... Apr 28 07:45:22.992 On [0:51]10.20.33.50:5060 received from 10.20.1.1:59758 ... The fields I'm trying to extract are source & destination IPs for each entry. So for source_ip, it's 20.5.4.1 and 10.2.1.1. For the destination, it's 102.11.130.135 & 10.20.33.50. It looks like I need a regex that matches a field that 'begins with "sent to" OR ends with "received from"' and vice-versa but I can't for the life of me get the regex to work. I tried to use prefixes but ended up with some nasty regexes that just don't work. (?i)([^\]\n]*\]|received from )(?P<FIELDNAME>\d+\.\d+\.\d+\.\d+:\d+) Can I do this with Splunk? Is it possible to use two regexes to extract to one field? ... View more

I have set up DB Connect on my Splunk 4.3 installation to provide an input that tails log data from a table. I didn't set up any custom SQL, I have set up the rising column to be "Id" and I have set up the timestamp column seemingly correctly. The problem is that the newest data in Splunk appears to be about 8 hours behind the database table. Splunk is certainly indexing the data, as the summary page shows the count incrementing every second or so. The data are correct, in that the timestamps shown reflect what is in the database but they're stuck at about 8 hours behind (i.e. at 9am I can only see logs from about 1am). I'm not sure if this is a red herring or not, but my timezone is UTC+8. The 8 hour difference could just be a coincidence but I can't be certain. The log timestamps are also in UTC+8. When I run Splunk query like so: sourcetype="dblogs" |stats max(Id) as maxid And a similar SQL query: SELECT Max(Id) FROM Log; The database is significantly ahead of Splunk (by about 22k - which is about right). Is indexing affected by the timestamp field & potential timezone offsets? Is there any other setting I can tweak to get it up to speed? EDIT: Some new information that will hopefully bump this a little: When I check the 'state.xml' file for this input, I get the following (except in real XML, not []). [list] [value key="latest.Id"] [value class="int"]61293050[/value] [/value] [/list] This matches up with the database's max ID when I run an SQL query directly. However when I run a stats max(Id) query in Splunk I get something from 8 hours ago: +----------+---------------------+ | Id | CreatedOn | +----------+---------------------+ | 61285887 | 2013-05-31 01:13:18 | | 61293050 | 2013-05-31 09:17:24 | +----------+---------------------+ If it helps, my Splunk server is running in UTC timezone, as is the MySQL database server. The data is logged in UTC+8. ... View more