Regex for multline events

[Mohsin123]

Hi ,

The Logstash client on the application box is configured to identify multiline events and send each event as a single JSON to BLC.
below is the snippet of data , which is a signle line event with \n and \t characters :
{
"path": "/var/log/armorvox/2018/01/armorvox.2018-01-05.log",
"hostname": "ip-10-13-0-248",
"@timestamp": "2018-01-05T08:57:51.861Z",
"@version": "1",

                         DEBUG  armorvox.main: Configuration:\ndatastore.disk.path=disk_db\ndatastore.class=auraya.datastore.MysqlDataStore\ndatastore.mysql.url=jdbc:mysql://baas-preprod-armorvox-db-rdscluster-1gfzsxrwcxp09.cluster-ce50hnjdbalm.eu-west-1.rds.amazonaws.com:3306/?allowMultiQueries=true&useSSL=false\ndatastore.mysql.username=armorvox\ndatastore.mysql.password=ENC(SOogtFTZCRYXVwiGVJ99qqRZ+4rrPrkUOZdHYgTPUO0GJqZblvifcA==)\ndatastore.mysql.name=armorvox\ndatastore.mysql.max_connections=100\ndatastore.mysql.initial_connections=100\ndatastore.mysql.max_idle=100\ndatastore.mysql.max_prepared_statements=100\ndatastore.oracle.url=jdbc:oracle:thin:@localhost:1521\ndatastore.oracle.username=system\ndatastore.oracle.password=\ndatastore.oracle.max_connections=100\ndatastore.oracle.initial_connections=100\ndatastore.oracle.max_idle=100\ndatastore.oracle.max_prepared_statements=100\ndatastore.sqlserver.url=jdbc:sqlserver://localhost:1433\ndatastore.sqlserver.username=sa\ndatastore.sqlserver.password=\ndatastore.sqlserver.name=armorvox_2017\ndatastore.sqlserver.max_connections=100\ndatastore.sqlserver.initial_connections=100\ndatastore.sqlserver.max_idle=100\ndatastore.sqlserver.max_prepared_statements=100\nenrol.save_features=true\nenrol.save_audio=true\nverify.save_features=true\nverify.save_audio=true\nenrol.qa.min_frames.print_ti=1000\nenrol.qa.min_frames.print_phrase=300\nverify.qa.min_frames.print_ti=400\nenrol.qa.ubm_fr_prob=1.0\nenrol.qa.phrase_fa_prob=1.0\nverify.qa.ubm_fr_prob=0.5\nverify.qa.phrase_fa_prob=1.0\nverify.fa_rate=0.1\nverify.fa_rate.print_10=1.0\nverify.fa_rate.print_11=1.0\nverify.fa_rate.print_ti=1.0\nverify.fa_rate.print_tpd=1.0\nactive_learning.enabled=false\nactive_learning.background_process=false\nactive_learning.rate=2.0\nactive_learning.recalculate_imp_scores.enabled=true\nactive_learning.impostors.sample=5000\nactive_learning.save_features=false\nactive_learning.max_updates=0\nhttp.threads=100\nhttp.port=9005\nhttp.max_request_length=20000000\nhttps.enabled=false\ncache.size=2000\ncache.seconds=60\nkey.seconds=60\ncalibration.channel.print_digit=[-0.959, -0.695, -0.541, -0.389, -0.186, -0.039, 6.085]\ncalibration.channel.print_phrase=[-0.549, -0.152, 0.056, 0.227, 0.420, 0.549, 5.477]\ncalibration.channel.print_tpd=[-1.027, -0.424, -0.109, 0.151, 0.443, 0.639, 3.652]\ncalibration.channel.ubm_date_256_all=[-0.750, -0.554, -0.451, -0.367, -0.271, -0.208, 13.609]\ncalibration.channel.ubm_tdd_256_all=[-1.020, -0.784, -0.661, -0.559, -0.445, -0.369, 12.36]\ncalibration.channel.ubm_ti_512_all=[-0.278, -0.206, -0.169, -0.138, -0.103, -0.080, 30.589]\ncalibration.channel.print_digit.channel_phrase_mismatch=[-1.010, -0.722, -0.591, -0.490, -0.377, -0.299, 9.086]\ncalibration.impostor.print_digit=[-1.230, -0.945, -0.812, -0.715, -0.619, -0.543, 10.893]\ncalibration.impostor.print_phrase=[-0.799, -0.591, -0.482, -0.393, -0.292, -0.224, 12.995]\ncalibration.impostor.print_tpd=[-1.483, -1.200, -1.052, -0.930, -0.793, -0.701, 8.501]\ncalibration.impostor.ubm_date_256_all=[-0.806, -0.609, -0.506, -0.421, -0.325, -0.262, 11.608]\ncalibration.impostor.ubm_tdd_256_all=[-1.038, -0.798, -0.673, -0.569, -0.453, -0.375, 10.508]\ncalibration.impostor.ubm_ti_512_all=[-0.330, -0.233, -0.183, -0.141, -0.094, -0.062, 22.557]\ncalibration.impostor.print_digit.channel_phrase_mismatch=[-1.014, -0.760, -0.644, -0.559, -0.471, -0.406, 11.832]\ncalibration.impostor_al.print_digit.channel_phrase_mismatch=[-0.243, -0.222, -0.208, -0.193, -0.168, -0.154, -0.147]\ncalibration.channel_al=[-0.110, +0.025, +0.100, +0.150, +0.180, +0.210, -0.100]\ncalibration.impostor_al=[-0.250, -0.210, -0.180, -0.150, -0.100, -0.060, -0.250]\ncalibration.speaker_specific_thresholds=true\ncalibration.impostor_rf=8.0\nenrol.neighbours.enabled=true\nenrol.impostors.sample=5000\ntopn=5\nrf=3\nrf.ubm_ti_512_all=16\nrf.print_11=3\nrf.print_tpd=3\nv5.api.enabled=true\nv6.api.enabled=true\nv6.input.id_max_length=100\nv6.input.group_max_length=100\nv6.input.override_max_length=1000\nv6.input.print_name_max_length=100\nv6.input.utterance_max_length=1000000\nv6.input.phrase_max_length=1000\nv6.input.vocab_max_length=100\nv6.input.channel_max_length=100\nv6.input.list_max_number=1000\nv6.input.utterance_max_number=100\nv6.print_name=[digit, date, phrase, ti, tpd]\nv6.print_name.gender=[male, female]\nv6.vocab=en_us_v_2.0\nubm.print_digit=tdd_256_all\nubm.print_date=date_256_all\nubm.print_phrase=phrase_256_all\nubm.print_ti=ti_512_all\nubm.print_tpd=tdd_256_all\nubm.print_male=male_32_all\nubm.print_female=female_32_all\nubm.print_1=tdd_256_all\nubm.print_2=tdd_256_all\nubm.print_4=tdd_256_all\nubm.print_7=tdd_256_all\nubm.print_5=date_256_all\nubm.print_3=phrase_256_all\nubm.print_6=phrase_256_all\nubm.print_8=phrase_256_all\nubm.print_10=ti_512_all\nubm.print_11=tdd_256_all\ncross_match.raw_score=false\nvocab_en_us_v_2.0=tpd_8_all\nimpostor_data_path=[star_id=impostor_data_star_id.dat, star_date=impostor_data_star_date.dat, van=impostor_data_van_phrase.dat, nist=impostor_data_nist_ti.dat]\ntopn_data_path=[tdd_256_all=star_id:tdd_256_star.dat, date_256_all=star_date:date_256_star.dat, phrase_256_all=van:phrase_256_van.dat, ti_512_all=nist:ti_512_nist.dat]\nsimilarity.distance_list_path=dist_all.dat\nhistory.log.path=/var/log/armorvox/history\nhistory.log.seconds=3600\nhistory.log.byte.max=10000000\nubm.location=config/ubms",
            "tags": ["multiline"]

}

some characters have to be escaped, such as newlines and double quotes, in order to convert to JSON. After this, the Logstash client has no control over what is done with the event, whether on BLC or on Splunk. What we need is that this event should appear in Splunk as a multiline text event.

Can anyone give any suggestion , please?

[richgalloway]

What are the props.conf settings for that sourcetype? Are you using LINE_BREAKER = ([\n\r]+)?