Getting Data In

Can someone help me with Line_Breaking?

Jarohnimo
Builder

I'm trying to create a props.conf file that will properly break up these av clam logs below. The logs don't have a date/timestamp only the long dashed line separates the events. The stanza below seems to work However I DO NOT like having to set should_linemerge=true/BREAK_ONLY_BEFORE.... in order to get this to work:

[ av:clam ]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=-------------------------------------------------------------------------------
CHARSET=UTF-8
disabled=false
DATETIME_CONFIG=NONE

In regex 101 i used this regex to break up the events and it looks clean there.

\-------------------------------------------------------------------------------$

When I try to use this to break the events it doesn't work (all the events are on separate lines) as if it doesn't recognize my line breaker.

SHOULD_LINEMERGE=false
LINE_BREAKER=\-------------------------------------------------------------------------------$

Below is a sample log (3 events). Hopefully, someone can help

-------------------------------------------------------------------------------

WARNING: Can't open file /etc/rsyslog.conf.broken: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_buu1Z0: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_P1SWCK: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_pD4Mt4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200021_ow7PXV: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600429_vx2xUp: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QYox3k: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_0fEYYI: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600101_tfBE1x: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200026_aPhSxB: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1532.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1599.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1695.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1517.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1602.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1593.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1592.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1727.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1526.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1770.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1513.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1686.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1588.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1607.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1605.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1636.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1698.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1603.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1785.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1617.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1606.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1742.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1585.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1519.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1623.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1708.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1590.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1531.log: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Myl0Qe: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Y8YTvr: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Svzf6O: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QvgHg4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200003_aWcbM9: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_eBGT5M: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200007_cPewso: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600099_vJnQRX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200019_1cSMBo: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_RvJuPw: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600106_bhzQNt: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_BHjkuK: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200001_02GigF: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_8rXTya: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600286_wkk2hw: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200037_PR0YIo: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200030_n0sXYf: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_egUWcm: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_SER8kV: Permission denied
WARNING: Can't open file /tmp/tmp.XIvDgFrUAn: Permission denied
WARNING: Can't open file /tmp/EPEL6-GPG-KEY: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200013_qujiN0: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_g40tLJ: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600427_BuOUej: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_OpGADN: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_jbilyY: Permission denied
WARNING: Can't open file /tmp/krb5cc_888700729_110ApX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200028_4tocVD: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600042_em0uAD: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_qjdnTq: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600043_Pyb8Hf: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600111_CtZB8x: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_HpqDys: Permission denied
WARNING: Can't open file /opt/splunk-6.4.0-f2c836328108-linux-2.6-x86_64.rpm: Permission denied

----------- SCAN SUMMARY -----------
Known viruses: 5995098
Engine version: 0.99.2
Scanned directories: 6366
Scanned files: 41938
Infected files: 0
Total errors: 83
Data scanned: 3329.70 MB
Data read: 4610.58 MB (ratio 0.72:1)
Time: 4296.029 sec (71 m 36 s)

-------------------------------------------------------------------------------

WARNING: Can't open file /etc/rsyslog.conf.broken: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_buu1Z0: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200001_n3Udh3: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_P1SWCK: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_pD4Mt4: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600429_vx2xUp: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QYox3k: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200046_tG4INP: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_0fEYYI: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200071_HSWmZ6: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600101_tfBE1x: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1532.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1599.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1695.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1517.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1594.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1595.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1602.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1580.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1593.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1592.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1526.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1513.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1686.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1588.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1607.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1605.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1589.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1636.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1698.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1603.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1617.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1606.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1604.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1584.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1585.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1519.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1623.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1708.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1590.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1531.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1608.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1610.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1598.log: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Myl0Qe: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Y8YTvr: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Svzf6O: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QvgHg4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200028_dJudKj: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_eBGT5M: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200012_QHbp0P: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200003_3gLmvy: Permission denied
WARNING: Can't open file /tmp/tmp.z1NhS7Cf1p: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600099_vJnQRX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200016_ZuL9m4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200048_CG4mxR: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200019_1cSMBo: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200037_FH62Pc: Permission denied
WARNING: Can't open file /tmp/tmp.a9xsZutIWq: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_RvJuPw: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600106_bhzQNt: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_BHjkuK: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_8rXTya: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600286_wkk2hw: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200030_n0sXYf: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_egUWcm: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_SER8kV: Permission denied
WARNING: Can't open file /tmp/tmp.XIvDgFrUAn: Permission denied
WARNING: Can't open file /tmp/EPEL6-GPG-KEY: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200013_qujiN0: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_g40tLJ: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600427_BuOUej: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_OpGADN: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_jbilyY: Permission denied
WARNING: Can't open file /tmp/krb5cc_888700729_110ApX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200051_5IDsNl: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600042_em0uAD: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200049_70bzRj: Permission denied
WARNING: Can't open file /tmp/tmp.E4AJCzpOIr: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200007_R3pBFi: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_qjdnTq: Permission denied
WARNING: Can't open file /tmp/tmp.nEx5K1P19V: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600043_Pyb8Hf: Permission denied
WARNING: Can't open file /tmp/tmp.3xu23Z8tDj: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600111_CtZB8x: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_HpqDys: Permission denied
WARNING: Can't open file /opt/splunk-6.4.0-f2c836328108-linux-2.6-x86_64.rpm: Permission denied

----------- SCAN SUMMARY -----------
Known viruses: 6319346
Engine version: 0.99.2
Scanned directories: 7233
Scanned files: 45947
Infected files: 0
Total errors: 100
Data scanned: 3594.28 MB
Data read: 4821.47 MB (ratio 0.75:1)
Time: 485.906 sec (8 m 5 s)

-------------------------------------------------------------------------------

WARNING: Can't open file /etc/rsyslog.conf.broken: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200048_SKap8h: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_buu1Z0: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_P1SWCK: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_pD4Mt4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200071_e3US5K: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200021_IfCsp4: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600429_vx2xUp: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QYox3k: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200046_tG4INP: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_0fEYYI: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600101_tfBE1x: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1587.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1599.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1594.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1595.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1602.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1580.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1593.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1592.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1566.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1578.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1611.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1588.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1607.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1605.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1589.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1603.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1583.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1596.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1606.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1604.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1584.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1582.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1620.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1585.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1623.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1590.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1577.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1608.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1610.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1598.log: Permission denied
WARNING: Can't open file /tmp/vmware-root/vmware-apploader-1591.log: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Myl0Qe: Permission denied
WARNING: Can't open file /tmp/tmp.0qPyyvkhIw: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Y8YTvr: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_Svzf6O: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_QvgHg4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200028_dJudKj: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_eBGT5M: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200012_QHbp0P: Permission denied
WARNING: Can't open file /tmp/tmp.z1NhS7Cf1p: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600099_vJnQRX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200019_1cSMBo: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200065_NZfYE4: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200037_FH62Pc: Permission denied
WARNING: Can't open file /tmp/tmp.a9xsZutIWq: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200003_Ysuwzs: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_RvJuPw: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600106_bhzQNt: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_BHjkuK: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_8rXTya: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600286_wkk2hw: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200030_n0sXYf: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200001_VezxBM: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_egUWcm: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_SER8kV: Permission denied
WARNING: Can't open file /tmp/tmp.XIvDgFrUAn: Permission denied
WARNING: Can't open file /tmp/EPEL6-GPG-KEY: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200013_qujiN0: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_g40tLJ: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200049_zrBoRF: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600427_BuOUej: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200051_5uiGLr: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_OpGADN: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_jbilyY: Permission denied
WARNING: Can't open file /tmp/krb5cc_888700729_110ApX: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200047_iM0nZM: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600042_em0uAD: Permission denied
WARNING: Can't open file /tmp/tmp.E4AJCzpOIr: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200007_R3pBFi: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_qjdnTq: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200016_7hh0tc: Permission denied
WARNING: Can't open file /tmp/tmp.nEx5K1P19V: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600043_Pyb8Hf: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200062_Y3tkcC: Permission denied
WARNING: Can't open file /tmp/tmp.3xu23Z8tDj: Permission denied
WARNING: Can't open file /tmp/tmp.KgPSpEWZwR: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600111_CtZB8x: Permission denied
WARNING: Can't open file /tmp/krb5cc_888600038_HpqDys: Permission denied
WARNING: Can't open file /tmp/krb5cc_1846200067_xWpi42: Permission denied

----------- SCAN SUMMARY -----------
Known viruses: 6319470
Engine version: 0.99.4
Scanned directories: 8003
Scanned files: 47590
Infected files: 0
Total errors: 105
Data scanned: 4118.82 MB
Data read: 5005.36 MB (ratio 0.82:1)
Time: 556.020 sec (9 m 16 s)
Tags (2)
0 Karma
1 Solution

Jarohnimo
Builder

Hi Rich, it seems I was able to get this to work by adding an additional slash in front of the s

Time: \d+\.\d+ sec \(\d+ m \d+ s\)()

Thank you for your assistance and explanation!

View solution in original post

0 Karma

Jarohnimo
Builder

Hi Rich, it seems I was able to get this to work by adding an additional slash in front of the s

Time: \d+\.\d+ sec \(\d+ m \d+ s\)()

Thank you for your assistance and explanation!

View solution in original post

0 Karma

PavelP
Motivator

please try:

DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
# with space at the beginning
LINE_BREAKER=([\r\n]+)(\s*\-{10,}|\s*[A-Z]+:)
# without space at the beginning of the line:
# LINE_BREAKER=([\r\n]+)(\-{10,}|[A-Z]+:)

this will let "WARNING: xxx" be one line event and "scan summary" - multiline:

alt text

0 Karma

Jarohnimo
Builder

Thanks for your assistance but this catches too much, did you get a chance to copy the log file into regex 101 and try your reg ex? it didn't seem to match.

0 Karma

PavelP
Motivator

the way how LINE_BREAKER works differs from the usual PRCE extractions because the first capture group is used as line breaker and get removed. This because you cannot use this regex without modification with regex101. You have to use Splunk Input Wizard. Have you tested my regex with the Input Wizard? Give it a try! I've used your data to check the parsing and, how you can see, it worked.

What you mean by "this catches too much"? Give an example how the logs need to be parsed. Does the screenshot above show wrong parsing? What you need to be changed?

0 Karma

Jarohnimo
Builder

Lines 1-81 are one event. It appears your regex broken out every line as an event. I was able to get it to work with this:

Time: \d+.\d+ sec (\d+ m \d+ s)()

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The LINE_BREAKER attribute requires a capture group. Try

SHOULD_LINEMERGE=false
LINE_BREAKER=(\-------------------------------------------------------------------------------$)
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Jarohnimo
Builder

No luck

Unfortunately, this combines everything into one event in splunk. Once again strange because if you grab the log file above paste it in regex 101 and use it does work /break the events it seems.

any other ideas?

0 Karma

Jarohnimo
Builder

Perhaps maybe try to use the time fields as the line breaker? any idea what this line breaker would be: Time: 556.020 sec (9 m 16 s)

^some times are 3 digits some 4

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Bearing mind the contents of the capture group is discarded, you could use either ()Time: \d+\.\d+ sec \(\d+ m \d+ s) or Time: \d+\.\d+ sec \(\d+ m \d+ s)(). The empty () is where the break will be.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Jarohnimo
Builder

Hey Rich, thanks I believe this will do the trick. I think pasting your code didn't work correctly as it removed the backslash before the lasts. I usually use the code box that way it won't delete slashes. Thanks for your help!

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!