About mcafeesecure

mcafeesecure · ‎05-15-2012

This turned out to be a simple oversight. It seems that the parser was picking up the "am" from the begining of my hostname in the next field. Hence only happening on machines names "am....." Fix = Add the following to my stanza's in props.conf MAX_TIMESTAMP_LOOKAHEAD = 15

mcafeesecure · ‎05-15-2012

I have a subset of servers that all of their logs parse the timestamps incorrectly at 12 (noon).. sample log lines: CORRECTLY PARSED: The following all index into splunk as 12PM May 14 12:00:31 dnv-scan001.scanalert.com resin_stdout: : [12:00:31.153] [SCAN_QUEUE ][WARN ]SaScanQueueServlet: Finished (4ms) May 14 12:00:56 dnv-scan001.scanalert.com resin_stdout: : [12:00:56.492] [MEMORY ][WARN ]AppMem=25295248 TotalMem=62128128 May 14 12:00:56 dnv-scan001.scanalert.com resin_stdout: : [12:00:56.494] [SCAN_SEND ][WARN ]SaCronSendCompletedServlet: Finished (342ms) PARSED INCORRECTLY The following all index into splunk as 12AM May 14 12:00:02 am1-scan001.scanalert.com resin_stdout: : [12:00:01.605] [MEMORY ][WARN ]AppMem=26510144 TotalMem=69664768 May 14 12:00:31 am1-scan001.scanalert.com resin_stdout: : [12:00:31.624] [SCAN_QUEUE ][WARN ]SaScanQueueServlet: Finished (7ms) May 14 12:00:51 am1-scan001.scanalert.com resin_stdout: : [12:00:51.622] [SCAN_SEND ][WARN ]SaCronSendCompletedServlet: Finished (0ms) I have a subset of 17 servers, that parse incorrectly, the ONLY difference I can find is the hostnames. has anyone encountered this behavior, and have any idea if there is a fix around this? All logs are in 24HR format fo "midnight" hour shows as 00:00:00, while "noon" as 12:00:00

mcafeesecure · ‎01-13-2012

This is probably something simple that I am missing. Is there a way to filter out what are esentially blank log entries from syslog? I get messages with the timestamp and hostname, process/PID but no data. Unfortunatly, I don't think this will be fixed in the app anytime soon, so I need a way to filter this out at search time, unfortunatly the PID is in the log so that's not uniform, and I can't filter out based on what IS there, as that would filter out the messages that do have data. Example log lines below. Jan 13 10:21:39 hostname.domain.com process[PID]: data is here that I want Jan 13 10:21:39 hostname.domain.com process[PID]: obviously I would want to keep the first, but discard the second

mcafeesecure · ‎10-25-2010

I have some log entries that look like this: (note the brackets ARE in my logs) [10/25/2010] [10:25:31.817] [SCAN_HISTORY][WARN ][START][unique ID number][ip.add.re.ss OR domain.name][service name] [10/25/2010] [10:25:31.817] [SCAN_HISTORY][WARN ][DONE][unique ID number][ip.add.re.ss OR domain.name][service name] What I need to look for is.. Any [unique ID number] that has a "[START]" entry, but NO "[DONE]" entry.. AND any "[DONE]" entry after a certain time. I can parse out the "STATUS" (ie, done, start), but can not figure out how to combine the entries to eval against each other.

mcafeesecure · ‎08-31-2010

This isn't an option for me, as I need to keep the4 sourcetype of the entries that are there.

mcafeesecure · ‎08-31-2010

I have a bit of an issue, as I typo'd a path change this morning, and ended up with about 8-10 hours of data being indexed with the wrong hostname. I have this fixed now, but would like to go back and reindex the strange data.. My log entries look like this (from various sources): Aug 31 10:12:04 host1.mydomain.com facilityname: actual log data Aug 31 10:12:04 host2.mydomain.com facilityname: actual log data Aug 31 10:12:04 host13.mydomain.com facilityname: actual log data Aug 31 10:12:04 host10.mydomain.com facilityname: actual log data Aug 31 10:12:04 host26.mydomain.com facilityname: actual log data Aug 31 10:12:04 host32.mydomain.com facilityname: actual log data the problem is that I had indexed them parsing the pathname segment for hostname... this caused all of the data to end up with hostname "2010" I can find all of the data with this search... host="2010" | rex "(?i)^(?:[^ ]* ){3}(?P<HOSTNAMEFROMLOG>[^ ]+)" However, Is there a way to pipe this into some command that will reindex with the different hostname?

mcafeesecure · ‎06-28-2010

Basically I have a line of data that looks like this: Jun 28 14:15:10 sc4-app04.mcafeesecure.com portal: ACCESS Click Referrer[bluecigs.com][http://www.blucigs.com/privacy-policy] I have rex set up to extract two parts of this: rex "(?i) Referrer\\[(?P<REF1>[^\\]]*)(?=\\])" | rex "(?i)www\.(?P<REF2>[^/]*)(?=/)" This will basically give me 2 fields I can search on REF1 and REF2. I would like only messages that do NOT have those two fields matching to show up for instance, in the previous entry I get REF1=bluecigs.com REF2=bluecigs.com the following line I would have them NOT matching: Jun 28 14:15:10 sc4-app04.mcafeesecure.com portal: ACCESS Click Referrer[bluecigs.com][http://www.SOMEOTHERDOMAIN.com/] IE: REF1=bluecigs.com REF2=SOMEOTHERDOMAIN.com Is there a way to compare the fields?

mcafeesecure · ‎06-14-2010

Thanks. For the record, the final query looks like this: syslog-ng "Log statistics" host!="qa2-cron*" | rex "(?i)^(?:[^=]=){2}(?P [^'] )(?=')" | search DROPPED="*" | streamstats current=f global=f window=1 first(DROPPED) as NEXT_DROPPED by host | eval DROPPEDDIFF = NEXT_DROPPED-DROPPED | search DROPPEDDIFF>0

mcafeesecure · ‎06-14-2010

I have the following log messages coming from syslog-ng Jun 14 10:32:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=175194', processed='center(received)=109152', processed='destination(d_resin)=51369', processed='destination(d_boot)=0', processed='destination(d_auth)=439', processed='destination(d_cron)=362', processed='destination(d_mlal)=0', processed='destination(loghost)=109152', processed='destination(d_mesg)=93', processed='destination(d_cons)=0', processed='destination(d_snmp)=6782', processed='destination(d_spol)=0', processed='destination(d_mail)=6997', processed='source(s_sys)=66165', processed='source(s_tail)=42187', processed='source(s_raccess)=800', processed='source(s_tail_2)=0' Jun 14 10:22:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=173017', processed='center(received)=107903', processed='destination(d_resin)=50855', processed='destination(d_boot)=0', processed='destination(d_auth)=430', processed='destination(d_cron)=359', processed='destination(d_mlal)=0', processed='destination(loghost)=107903', processed='destination(d_mesg)=92', processed='destination(d_cons)=0', processed='destination(d_snmp)=6610', processed='destination(d_spol)=0', processed='destination(d_mail)=6768', processed='source(s_sys)=65231', processed='source(s_tail)=41872', processed='source(s_raccess)=800', processed='source(s_tail_2)=0' Jun 14 10:12:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=171101', processed='center(received)=106828', processed='destination(d_resin)=50405', processed='destination(d_boot)=0', processed='destination(d_auth)=416', processed='destination(d_cron)=338', processed='destination(d_mlal)=0', processed='destination(loghost)=106828', processed='destination(d_mesg)=87', processed='destination(d_cons)=0', processed='destination(d_snmp)=6428', processed='destination(d_spol)=0', processed='destination(d_mail)=6599', processed='source(s_sys)=64382', processed='source(s_tail)=41646', processed='source(s_raccess)=800', processed='source(s_tail_2)=0' I am using the following query to generate a diff of the field in this message I want (in this case the dropped field) syslog-ng "Log statistics" host="sc4-cron*" | rex "(?i)^(?:[^=]=){2}(?P[^'])(?=')" | search DROPPED="" | delta DROPPED as DROPPEDDIFF | search DROPPEDDIFF="" this works fine on a single host, however, I would like to run this across all of my hosts. The problem I then encounter, is that the previous log message is not from the same host as the latest message, therefore the diffs that are created are not accurate. I have hundreds of hosts, and would like to build a report on this data, however, I can't really go and do them all individually. In addition getting all of this into one report would be ideal.

Posts	9
Solutions	0
Karma Given	1
Karma Received	9
Member Since	‎06-14-2010

Online Status	Offline
Date Last Visited	‎06-05-2020 02:02 AM

24hour timestamps parse incorrectly

filter blank events coming from syslog

match data from 2 queries

change hostname for indexed data dynamically?

Compare 2 fields

delta report for multiple hosts

Re: 24hour timestamps parse incorrectly