topic Re: filed extraction on specific path in Splunk Search

filed extraction on specific path

indeed_2000 — Mon, 16 Dec 2019 15:05:17 GMT

Hi
I want to create "field extract" on all logs that exist in below address.
/opt/logs/file1.log
/opt/logs/file2.log
/opt/logs/file3.log

when I create new "field extract" at first step ask me choose a source type: file1.log or file2.log or file3.log ?
How can extract field on all of them like "/opt/logs/*" ?
Should create index for this path?

Thanks,

Re: filed extraction on specific path

mayurr98 — Mon, 16 Dec 2019 15:55:24 GMT

May I know how you are trying to extract?

Re: filed extraction on specific path

richgalloway — Mon, 16 Dec 2019 15:58:49 GMT

If all of the logs have the same structure then your field extraction can be done on one of them. When you do that, create a new sourcetype. Use that sourcetype when you index /opt/logs/* and the field extraction will be applied to all of the files in that directory.

Re: filed extraction on specific path

gcusello — Mon, 16 Dec 2019 16:00:02 GMT

Hi @mehrdad_2000,
let me understand:
you have a list of files that contain logs,
you want to associate to all logs a field called "field extract" that's a part of the source (e.g. the last part of the path),
I don't understand what do you mean when you say "at first step ask me choose a file".

Anyway to extract a field from the source field, it's easy using a regex like this:

index=my_index
| rex field=source "(?<field_extract>\w*\.log)$"
| ...

Ciao.
Giuseppe

Re: filed extraction on specific path

indeed_2000 — Wed, 30 Sep 2020 03:20:04 GMT

create new sourcetype as you mention "my_SourceType", but when I hit "field extraction" it has only show "my_SourceType" and it is empty ! there is no event!

while when going to the search it and enter "source = "/opt/logs/*" show all events!

Any recommendation?

Thanks,

Re: filed extraction on specific path

indeed_2000 — Mon, 16 Dec 2019 16:17:33 GMT

1-I have several log files with different structure and want extract specific field on all of them.
2-At first step of "field extract" Splunk ask sourcetype.

Re: filed extraction on specific path

gcusello — Mon, 16 Dec 2019 16:25:28 GMT

Hi @mehrdad_2000,
because usually knowledge objects (as fields) are related to a sourcetype and every log ingestion must have a sourcetype.
What's the sourcetype you associated to the above files?
You can use it.

Ciao.
Giuseppe

Re: filed extraction on specific path

indeed_2000 — Mon, 16 Dec 2019 18:56:35 GMT

Custom sourcetype

Re: filed extraction on specific path

to4kawa — Mon, 16 Dec 2019 19:05:52 GMT

Get started with getting data in

Have a look at this.

Re: filed extraction on specific path

indeed_2000 — Mon, 16 Dec 2019 19:13:36 GMT

Sure,
1-I’m going to the search and enter "source = "/opt/logs/*"
2-click on “field extraction”
https://docs.splunk.com/File:Extract_new_fields.png

Re: filed extraction on specific path

gcusello — Tue, 17 Dec 2019 07:59:28 GMT

Hi @mehrdad_2000,
as I said use this custom sourcetype, the only important hing is to use one sourcetype otherwise it's difficoult to use the fields.

Ciao.
Giuseppe

Re: filed extraction on specific path

richgalloway — Tue, 17 Dec 2019 12:59:12 GMT

Did you put the new sourcetype in your inputs.conf? The change will only apply to new data. Anything already indexed will be under the old sourcetype.

sourcetype = mysourcetype