Splunk Line Breaking No Working as Expected

[zach-keener]

Hello,

I have this data here:

2024-04-03 13:57:54 10.237.8.167 GET / "><script>alert('struts_sa_surl_xss.nasl-1712152675')</script> 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 2 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET / - 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 0 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET / - 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 1 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET / - 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 1 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET /Default.aspx - 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 404 0 0 1 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET /home.jsf autoScroll=0%2c275%29%3b%2f%2f--%3e%3c%2fscript%3e%3cscript%3ealert%28%27myfaces_tomahawk_autoscroll_xss.nasl%27 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 404 0 2 1 10.236.125.4 2024-04-03 13:57:55 10.237.8.167 GET /admin/statistics/ConfigureStatistics - 443 - 10.237.123.253 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 404 0 2 2 10.236.125.4

It is not line breaking properly as expected for our IIS logs.  This is what I currently have for our sourcetype stanza on the indexer.

[iis]
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19

[PickleRick]

Can you please paste it into either a preformatted paragraph or a code block? Here the data is already butchered by the forum's mechanics so we can't see the original raw events. Is that whole block supposed to be in a single line in the IIS log file?

[zach-keener]

2024-04-08 02:24:47 10.236.6.10 GET /wps/wcm/webinterface/login/login.jsp "><script>alert("ibm_login_qs_xss.nasl-1712543165")</script> 443 - 10.236.0.223 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 404 0 2 0 4.35.178.138
2024-04-08 02:24:47 10.236.6.10 GET /cgi-bin/login.php - 443 - 10.236.0.223 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 404 0 2 0 4.35.178.138
2024-04-08 02:24:48 10.236.6.10 GET / - 443 - 10.236.0.223 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 1 4.35.178.138
2024-04-08 02:24:48 10.236.6.10 GET / - 443 - 10.236.0.223 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 0 4.35.178.138
2024-04-08 02:24:48 10.236.6.10 GET / - 443 - 10.236.0.223 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0) - 200 0 0 0 4.35.178.138

[PickleRick]

Ok. This looks better. So the usual suspects are naturally

1. Mismatch between the sourcetype naming in inputs and props (and possibly some overriding settings defined for source or host)

2. Something overriding these parameters - defined elsewhere with higher priority (check with btool)

3. Wrongly placed props.conf (on an indexer when you have a HF in your way).

Of course there is also a question of "why aren't you simply using Splunk-supported TA for IIS?".

[zach-keener]

It could be the first, we do have other defined EXTRACTs and other modifications to data pushed to the indexers and they work properly.  But for some reason this portion of IIS logs just doesn't work properly.

I would have to look into the higher priority, however other IIS sourcetype logs aren't turning out this way.  

I do know that the props.conf is in the correct spot.  

When we stood up Splunk initially there were custom written apps rather than that of the Splunk Supported TA for IIS.  I may go that route if I can't get this resolved via our custom app.

[richgalloway]

It looks like there are no newlines between events so the LINE_BREAKER is not matching.  Try these settings:

[iis]
LINE_BREAKER = ([\r\n]*)\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19

[zach-keener]

Still no dice on that.  It only happens to these few logs that are formatted this way.  Could there be anything else preventing it from breaking apart properly?