F5 BIG-IP linebreaking

[m_zandinia]

Hi Splunkers!

I have a problem with line breaking in Splunk add-on F5-bigip. I've tried some regex to break the line correctly but I'm not successful.

First of all for simplicity I changed my outputs.conf in Heavy Forwarder.

outputs.conf

[indexAndForward]
index = true

In fact the   indexing is false on this node and this HF forward data to my indexer cluster and I also have search head cluster. But as I mentioned just for simplicity I turned mu indexing to true in this HF.

Then I used these regexes to break the lines

props.conf

[f5:bigip:syslog]
# LINE_BREAKER = ^()\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}
# LINE_BREAKER = ^\w{3}\s\d*\s\d{2}\W\d{2}\W\d{2}
LINE_BREAKER = ([\r\n]+)\w{3}\s\d+\s\d{2}\W\d{2}\W\d{2}
# LINE_BREAKER = ([\r\n]+)
# LINE_BREAKER = \n
MAX_TIMESTAMP_LOOKAHEAD = 16
# ADD_EXTRA_TIME_FIELDS = subseconds
NO_BINARY_CHECK = true
# EVENT_BREAKER_ENABLE = false
# TIME_FORMAT = %b %d %H:%M:%S
TIME_PREFIX =
SHOULD_LINEMERGE = false
TRUNCATE = 1000000

This is some of my data that I can't break the line correctly.

Sep 18 19:12:27 192.168.1.1 Sep 18 14:42:27 F5-LTM-3.company.local info logger[25169]: [ssl_req][18/Sep/2021:14:42:27 +0000] 1.1.1.1 TLSv1.2 ECDHE-RSA-AES128-SHA "/mgmt/shared/inflate/available" 2
Sep 18 19:12:28 192.168.1.1 Sep 18 14:42:28 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673914804247",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:28",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="44180",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET /Account/Login HTTP/1.1\r\nConnection: keep-alive\r\nHost: example.com\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.8,en-US;q=0.5,en;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/5.0 (Android 7.0; Mobile; rv:68.0) Gecko/68.0 FUSefox/68.0\r\nUpgrade-Insecure-Requests: 1\r\nX-Forwarded-For: 1.1.1.1\r\n\r\n",response="Response logging disabled"
Sep 18 19:12:28 192.168.1.1 Sep 18 14:42:28 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673951684370",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:28",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="19338",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET //Account/Login HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 FUSefox/92.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: keep-alive\r\nCookie: GuidedTourVersion=1; SiteVersion=3.7.6; __utma=226054936.2062308401.1625890970.1631960683.1631966584.238; __utmz=226054936.1625890970.1.1.utmcsr=(dUSect)|utmccn=(dUSect)|utmcmd=(none); crisp-client%2Fsession%2Fbb1636a8-4b45-4fbb-971e-d5e50e2a1e1f=session_230233c6-895e-42d0-b257-4ae4c1903150; _hjid=b846f33d-e2e6-4c9a-a757-f9ab405b0193; Token=6abe8980-5856-4d6f-b05a-2915b970983e; lastmessage-6=87696; lastmessage-4=1; lastmessage-2=undefined; text0_1567617252=true; text0_496056564=true; .ASPXAUTH=4A5473E3674D47ED86E8EA52D6A4613C2F30F1D31A41DF7F8BEDBAB120DE5ACEB8E3DD46D71
Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673926887289",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="46453",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET //Account/login HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Linux; Android 7.1.1; SM-J510F Build/NMF26X; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/64.0.3282.137 Mobile Saenri/537.36 AgentWeb/4.1.3  UCBrowser/1.1.1.1\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en,en-US;q=0.9,en-GB;q=0.8,en-US;q=0.7\r\nX-Requested-With: com.sefryekcompany.mobiletradingpro\r\nX-Forwarded-For: 1.1.1.1\r\n\r\n",response="Response logging disabled"
Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673919202912",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37419741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="f8689163755118a6",src_port="44760",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared1-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37419741 HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: */*\r\nSave-Data: on\r\nService-Worker: script\r\nSec-Fetch-Site: same-origin\r\nSec-Fetch-Mode: same-origin\r\nSec-Fetch-Dest: serviceworker\r\nReferer: https://mobile.bmibourse.com/serviceworker.js?37419741\r\nUser-Agent: Mozilla/5.0 (Linux; Android 10; SM-A207F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Mobile Saenri/537.36\r\nAccept-Encoding: gzip, deflate, br\r\nAccept-Language: en-US,en;q=0.9,en-US;q=0.8,en;q=0.7\r\nCookie: basket-warning-readed=1; basket-option-visited=true; tag-market-map-visited=true; index-technical-visited=true; stock-technical-visited=true; AppVersion=1.1.2; TS01e42c80=0180bb6f222b77a4b3dd30e3eddfc570acb1a0674cc23f80304088a610b57e5e43c686eb7415c18bc949724b74a1f77b7746en6cd8\r\nX-Forwarded-For: 5.116.208
Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673963109971",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37418741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="80b2664635b96eeb",src_port="41628",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37418741 HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Android 7.0; Mobile; rv:68.0) Gecko/68.0 FUSefox/68.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.8,en-US;q=0.5,en;q=0.3\r\nAccept-Encoding: gzip, deflate, br\r\nService-Worker: script\r\nConnection: keep-alive\r\nCookie: _ga=GA1.2.1098137509.1594471619; basket-warning-readed=1; basket-option-visited=true; AppVersion=1.1.2; index-technical-visited=true; tag-market-map-visited=true\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nX-Forwarded-For: 1.1.1.1\r\nSSLcompany: 1\r\n\r\n",response="Response logging disabled"
Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673952377578",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37418741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="3915b37e523c6d41",src_port="55434",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared2-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37418741 HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: */*\r\nService-Worker: script\r\nX-Requested-With: com.sefryekcompany.mobiletradingpro\r\nSec-Fetch-Site: same-origin\r\nSec-Fetch-Mode: same-origin\r\nSec-Fetch-Dest: serviceworker\r\nReferer: https://mobile.mobinsb.com/serviceworker.js?37418741\r\nUser-Agent: Mozilla/5.0 (Linux; Android 10; SM-A600G Build/QP1A.190711.020; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/81.0.4044.138 Mobile Saenri/537.36\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-US,en;q=0.9,en-CA;q=0.8,en-US;q=0.7,en;q=0.6\r\nCookie: companyRLCUrl=////////////////////////////////////core.companyrlc.com/; companyRLApiUrl=//rlcchartapi.companyrlc.com/; BrokerId=777; ThemeName=MobinSarmayeh; DisabledModules=changebroker; PushSubDomainName=push2v7.company.co

 Thanks in advance

[m_zandinia]

Thanks for your time. It's worked perfectly. Just I have a misspelling in my post. I've corrected it so you do.

MAX_TIMESTAMP_LOOAKAHEAD = 16

MAX_TIMESTAMP_LOOKAHEAD = 16

[richgalloway · Accepted solution]

First, make sure the inputs.conf file specifies the right sourcetype.  Also, be sure to restart Splunk after changing a config file.  

Try these props.conf settings:

[f5:bigip:syslog]
LINE_BREAKER = ([\r\n]+)\w{3}\s+\d+\s\d{2}:\d{2}:\d{2}
MAX_TIMESTAMP_LOOAKAHEAD = 16
TIME_FORMAT = %b %d %H:%M:%S
TIME_PREFIX = ^
SHOULD_LINEMERGE = false
TRUNCATE = 1000000