Sourcetype not being applied

gabrsar — Thu, 24 Jun 2021 18:56:38 GMT

Hi everyone. I'm using Splunk Enterprise (Trial) to understand how things works.

I'm trying to configure some sourcetype for my python/flask application, logs where getting merged incorrectly, with two or more line logs being joined inside a single event and sourcetype is not being applied.

For example, this is a single event in splunk:

[2021-06-23 12:05:09,807] {/program.py:make_request:452} DEBUG - https://localhost:443 "POST /create/user HTTP/1.1" 201 None
[2021-06-23 12:05:09,810] {/program.py:make_request:493} INFO - user created with success id=1234

I also have some logs with this format:

[2021-06-24 17:48:37,490] {/program/main.py:authorize:69} INFO - Host: localhost:5000
  User-Agent: curl/7.64.0
  Accept: */*

I tried creating a new sourcetype on Settings->Data->Source Types. But I noticed two weird things.

1 - If I go on Advanced and configure as I want, It don't save my new regex for LINE_BREAKER. I need it to be: ([\n\r]+)[ but every time I press save, and open again, its the default one ([\n\r]+). If I go on "Events Break" instead and just type my regex it saves. What I'm doing wrong?

2 - It doesn't apply my new sourcetype to my logs. I check on Search->Event lists and my logs are being sourcetyped as output-too_small, now I changed something and it is output-2

Then, I started googling around, and reading some docs, they tell to edit some files on splunk server then I did:

3 - Also tried creating a new sourcetype on $SPUNK_HOME/etc/system/local/props.conf as follow:

[python_flask]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\n\r]+)\[
NO_BINARY_CHECK = true
disabled = false

4 - Also changed my $SPUNK_HOME/etc/system/local/inputs.conf, and added:

[monitor:///var/log/program/output.log]
sourcetype = python_flask

I restarted with splunk restart both server and universal forwarder, and the only thing that changed is that it started to put sourcetype=output-2 on my events .

I'm quite noob in splunk management, so sorry if any question is dumb, I have already checked the docs, google and so on. Thank you so much in advance.

Re: Sourcetype not being applied

codebuilder — Tue, 29 Jun 2021 18:16:36 GMT

Check the permissions on the props.conf that you created. I'm going to guess it's owned by root (or another user) and Splunk can't access it. If you are running Splunk as the splunk user you'll need to chmod splunk:splunk $SPUNK_HOME/etc/system/local/props.conf (e.g.) and then cycle Splunk.

topic Sourcetype not being applied in Getting Data In

Sourcetype not being applied

Re: Sourcetype not being applied