Multi-line event and props.conf

[jwhughes58]

I've got something that is confusing me. I've got a file, /logs/oud_ds/audit, of raw events that looks like this

# 07/Jul/2017:04:33:15 -0700; conn=-1; op=916539
dn: dc=dummy,dc=org
changetype: modify
replace: ds-sync-state
ds-sync-state: 0000015538c7f348346c0000db05
ds-sync-state: 00000152b605febb5a1100008e9f
ds-sync-state: 0000015d1cd17e6a3da0000786dd
ds-sync-state: 0000015d1ccdaa897cf400020ff8
ds-sync-state: 0000015d1cc79aa54f4700110281
ds-sync-state: 0000015d1cd2aa0f59c300333ed4
ds-sync-state: 0000015d1ccdcc106ad400028552
ds-sync-state: 000001555b61786120e70000222d
ds-sync-state: 0000015d1cd2ef54185e007b773d
ds-sync-state: 0000015d1ccd0c411ad2001c87a4
ds-sync-state: 0000015537e011f047ff00000c70
ds-sync-state: 0000015d1cd0d5b21bfb006030f8
ds-sync-state: 0000015d1cccf9f95db100009eaf
ds-sync-state: 0000015d1cd18fe127a100007c97
ds-sync-state: 0000015d1cd2a58809d80000613d
ds-sync-state: 0000015d1cd0c5dc403000005ae2
ds-sync-state: 0000015d1cd3414467670073baa1
ds-sync-state: 0000015d1cd1ca8d576700079731
ds-sync-state: 0000015d1ccb5e7b5ce800112951
ds-sync-state: 00000153b020eeca5a4e004b1d6f
ds-sync-state: 0000015d1ccedacc443300036ee1
ds-sync-state: 0000015d1cd346fa4bd200fdd212
ds-sync-state: 0000015d1cd1ca9f44ff000fcb48
ds-sync-state: 000001503fb0c53d5f72000042c9
modifyTimestamp: 20170707113314Z
-
delete: ds-sync-hist
ds-sync-hist: pdxversionnumber:0000015d1285fce71bfb005fa01d:repl:1852
-
add: ds-sync-hist
ds-sync-hist: pdxversionnumber:0000015d1cd346fa4bd200fdd212:repl:1853
-
delete: ds-sync-hist
ds-sync-hist: pdxfailedlogons:0000015d1285fce71bfb005fa01d:repl:0
-
add: ds-sync-hist
ds-sync-hist: pdxfailedlogons:0000015d1cd346fa4bd200fdd212:repl:0
-
delete: ds-sync-hist
ds-sync-hist: modifytimestamp:0000015d1285fce71bfb005fa01d:repl:20170705113237Z
-
add: ds-sync-hist
ds-sync-hist: modifytimestamp:0000015d1cd346fa4bd200fdd212:repl:20170707113314Z
-
add: ds-sync-hist
ds-sync-hist: modifytimestamp:0000015d1cd346fa4bd200fdd212:repl:20170707113314Z
-
add: ds-sync-hist
ds-sync-hist: modifytimestamp:0000015d1cd346fa4bd200fdd212:repl:20170707113314Z
-
delete: ds-sync-hist
ds-sync-hist: pdxlastlogondatetime:0000015d1285fce71bfb005fa01d:repl:1499254357275
-
add: ds-sync-hist
ds-sync-hist: pdxlastlogondatetime:0000015d1cd346fa4bd200fdd212:repl:1499427194714
-
delete: ds-sync-hist
ds-sync-hist: modifiersname:0000015d1285fce71bfb005fa01d:repl:cn=pimsEPS-MAS,ou=Application,ou=ApplicationServices,dc=kp,dc=org
-
add: ds-sync-hist
ds-sync-hist: modifiersname:0000015d1cd346fa4bd200fdd212:repl:cn=dummy,ou=Application,ou=ApplicationServices,dc=dummy,dc=org

I have this in my props.conf

[source:/logs/oud_ds/audit]
# sourcetype : oed:ds
# # 06/Jul/2017:16:39:20 -0700; conn=-1; op=6318761
# dn: uid=dummy,ou=dummy,ou=People,DC=dummy,DC=ORG
# changetype: modify
# replace: userPassword
# userPassword: *****
# -
# replace: obpasswordchangeflag
# obpasswordchangeflag: false
# -
# replace: obpasswordcreationdate
#
BREAK_ONLY_BEFORE = ^#
DATETIME_CONFIG =
MAX_TIMESTAMP_LOOKAHEAD = 21
NO_BINARY_CHECK = true
TIME_FORMAT = %d/%b/%Y:%H:%M:%S
TIME_PREFIX = #\s
TRUNCATE = 999999
TZ = UTC

When I oneshot the data file, the information doesn't get split properly. If I use the UI "Add Data" to process the same file, it processes correctly and I find this in etc/apps/search/local/props.conf

[oracle_oud]
BREAK_ONLY_BEFORE = ^#
DATETIME_CONFIG = 
MAX_TIMESTAMP_LOOKAHEAD = 21
NO_BINARY_CHECK = true
TIME_FORMAT = %d/%b/%Y:%H:%M:%S
TIME_PREFIX = #\s
TZ = UTC
category = Custom
disabled = false
pulldown_type = true

I'm confused because this looks like a processing order issue and I don't know how to get around it. I have to use source because there are three different files source files of sourcetype oud:ds with two different formats. This is the hard format. Any ideas what I might be missing? Would it be better to put each source under a different sourcetype?

TIA,
Joe

[maciep]

where is the data coming from? where is the props? and where are you running oneshot? Is it all happening on the search head? Is your search head also your indexer? Just want to be sure you're putting your props in the right place in your infrastructure - wherever parsing happens (the first instance running a full version of splunk enterprise)

Side note, I think using LINE_BREAKER and SHOULD_LINEMERGE=false is typically more efficient than trying to merge lines then break them.

[gcusello · Accepted solution]

Hi Joe,
I don't see in your props.conf SHOULD_LINEMERGE = true, but probably this isn't the problem.
I found in my experience that it's better to use sourcetype in props.conf and sometimes using source: or host: there are problems, infact using the web interface it uses sourcetypes and it runs.
I understand that if you have two sources with different formats you have to create two sourcetypes but probably this will solve your problem.

Bye.
Giuseppe

[jwhughes58]

Hi Giuseppe,

Thanks for the lead. I found an article on how to assign the proper sourcetype to each file. What I wound up with in inputs.conf is this.

[monitor:///logs/device/file1]
disabled=false
index=my_index
sourcetype=device:file_1

[monitor:///logs/device/file2]
disabled=false
index=my_index
sourcetype=device:file_2

[monitor:///logs/device/file3]
disabled=false
index=my_index
sourcetype=device:file_3

[monitor:///logs/device2/file_1]
disabled=false
index=my_index
sourcetype=device_2:file_1

First time I've worked with the inputs.conf file. Good to learn something new.