Field extraction isn't working correctly, lines are joined together after upgrading to 6.3.0 from 6.2.5.

[lakromani]

After I upgraded to 6.3.0 from 6.2.5 the lines are joined by time stamp not by line itself. Data is logged using syslog.

6.2.5 Samples: (Every line has its own >)

>Sep 23 07:26:57 172.30.112.1 PORTAL: Login failed for 172.30.54.212 - invalid password for username 91803016
>Sep 23 07:26:54 172.30.112.1 AAA: 172.30.60.207 logged in with username 90851458
>Sep 23 07:26:51 172.30.112.1 AAA: 172.30.54.174 logged in with username 91390274
>Sep 23 07:26:44 172.30.112.1 AAA: 172.30.56.73 logged in with username 93257176
>Sep 23 07:26:13 172.30.112.1 AAA: 172.30.52.179 logged in with username 95971045
>Sep 23 07:26:12 172.30.112.1 AAA: 172.30.8.223 logged in with username 91247699
>Sep 23 07:26:09 172.30.112.1 AAA: 172.30.37.83 logged in with username 41034027

6.3.0 samples: (Lines shows up in group with > in front of each group)

>Sep 23 07:59:12 172.30.112.1 AAA: 172.30.12.96 logged in with username 91747473
 Sep 23 07:59:15 172.30.112.1 AAA: 172.30.10.85 logged in with username 90195562
 Sep 23 07:59:24 172.30.112.1 AAA: 172.30.57.58 logged in with username 48075978
 Sep 23 07:59:29 172.30.112.1 PORTAL: SMS sent to 40601577.
 Sep 23 07:59:31 172.30.112.1 AAA: 172.30.37.122 logged in with username 95788664
 Show all 25 lines
>Sep 23 07:58:57 172.30.112.1 PORTAL: SMS sent to 95788664.
 Sep 23 07:58:57 172.30.112.1 PORTAL: Subscriber 95788664 created.
 Sep 23 07:59:01 172.30.112.1 AAA: 172.30.56.139 logged in with username 93438608
 Sep 23 07:59:01 172.30.112.1 AAA: 172.30.59.95 logged in with username 46413647
 Sep 23 07:57:46 172.30.112.1 AAA: 172.30.61.74 logged in with username 98059661
 Sep 23 07:57:46 172.30.112.1 AAA: 172.30.59.7 logged in with username 97952038
 Sep 23 07:57:54 172.30.112.1 PORTAL: SMS sent to 97401997.
 Sep 23 07:57:58 172.30.112.1 AAA: 172.30.66.163 logged in with username 92287373
 Sep 23 07:58:05 172.30.112.1 AAA: 172.30.54.101 logged in with username 95946040
 Show all 14 lines
>Sep 23 07:57:31 172.30.112.1 PORTAL: Login failed for 172.30.54.96 - invalid password for username 92038019

As it is now, my Field Extraction does not work correctly with 6.3.0 since lines are grouped together .

Is this a bug or a setting change?
How to fix it?

[lguinn2 · Accepted solution]

In your inputs.conf, you should have a line that sets a sourcetype for your input.

[udp:514]
sourcetype=syslog

If you do this, you will need to change the stanza name in props.conf to syslog instead of "UDP:514", like this

[syslog]

Since Splunk recognizes "syslog" as a known sourcetype, it should parse your data appropriately. If not, you can add the following line to props.conf, in the syslog stanza.

SHOULD_LINEMERGE=false

This is a better practice than allowing the sourcetype name to default.

[lakromani]

Thanks, that did the trick.

I have some follow up question.

1. What inputs.conf should I add the udp-syslog setting to? I used etc/system/local/inputs.conf. Should this be a input in my app?
2. After upgrading to 6.3.0 I can no longer use http://my-splunk/en-US/debug/refresh to reload the config.

I just get this message:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<response>
<messages>
<msg type="ERROR">Forbidden</msg>
</messages>
</response>

So I just restarted the Splunkd on my server.

[lguinn2]

As a best practice, you should put the inputs.conf in one of the apps, not the system level directory.

if there is an app that is appropriate, then clearly that works. For example, if you are talking about an F5 log input and you have the F5 app installed, the inputs.conf should clearly go in the local directory of the F5 app.

If the input is not directly associated with any app, you might create an app, just as a place to house general configuration files. Or people sometimes choose to use the "search" app for that purpose.

[schoi]

For question #2, yes this is expected -- you will now need to be logged in as a role with the 'web_debug' capability (enabled for admin role only by default) -- see http://my-splunk/en-US/debug/info under "EAI object refresh"

[lakromani]

That is OK form my work server where we have 200GB of log data every day and many users. But for the small test server where I have used the "Free license group" (max 500MB/day) there are only one user since the license just show "This feature is not available with your installed set of licenses." There are no way I can change this. It did work in 6.2.5, but not in 6.3.0

[benlc]

Do you use a pretrained sourcetype or could you post your props.conf for the sourcetype you created?

[lakromani]

No sourcetype is used. Data are coming in as main index using syslog.

Only props.conf I have changed, added is one in etc\apps\HSMX\default\props.conf (What I have made for my HSMX hotspot server), and it looks like this.

[udp:514]
EXTRACT-admin_user1 = CONFIG: .* by (?<admin_user>[^.]*).$
EXTRACT-admin_user2 = CONFIG: Administrator (?<admin_user>[^ ]*) logged
EXTRACT-client_ip1 = PORTAL: Login failed for (?<client_ip>\d+\.\d+\.\d+\.\d+) -
EXTRACT-client_ip2 = PORTAL: (?<client_ip>\d+\.\d+\.\d+\.\d+) redirected
EXTRACT-client_ip3 = AAA: (?<client_ip>\d+\.\d+\.\d+\.\d+) logged
EXTRACT-client_ip4 = AAA: Logout by logout URL from (?<client_ip>\d+\.\d+\.\d+\.\d+)
EXTRACT-client_ip5 = AAA: Logout requested for user [^ ]* - ip (?<client_ip>\d+\.\d+\.\d+\.\d+)
EXTRACT-client_ip6 = PORTAL: Login failed for (?<client_ip>\d+\.\d+\.\d+\.\d+) -
EXTRACT-module =  (?<module>(PORTAL|AAA|CONFIG|SYSTEM)):
EXTRACT-username1 = AAA: .* username (?<username>[^ ]*)$
EXTRACT-username2 = CONFIG: Subscriber (?<username>.*) added
EXTRACT-username3 = PORTAL: .* account (?<username>[^ ]*) is not valid from
EXTRACT-username4 = CONFIG: Subscriber (?<username>.*) deleted
EXTRACT-username5 = PORTAL: Subscriber (?<username>.*) created
EXTRACT-username6 = PORTAL: SMS sent to (?<username>.*)\.
EXTRACT-username7 = AAA: Logout requested for user (?<username>.*)
EXTRACT-username8 = PORTAL: .* username (?<username>.*)

LOOKUP-ip-to-location = ip-to-loaction ip AS client_ip OUTPUTNEW site AS client_site
LOOKUP-username_to_country = username_to_country username OUTPUTNEW country

If I look in my log data, all looks fine until SPLUNK is upgrade to 6.3.0. No config was changed, just update.
Data are still coming in, but now are grouped inn to groups with same time, no line by line.

[benlc]

Hi

Sourcetype is mandatory. Splunk adds a Sourcetype to your input at Indextime. What sourcetype do you see in your search? In your props.conf is no breaking defined, which is also mandatory for working proberly.