Additional searchable value to ingested data

[orendado]

Hi,

Let's say I'm ingesting different types of logs files from different type(some are txt,csv,json,xml....) to the same index. How can I add additional data to each datasource/log? I would like to some extra fields in json format, for example : customers name, system same...

[gcusello]

Hi @orendado ,

Usually different types of logs are categorized using sourcetype.

Related to sourcetype, usually there are all the parsing rules and field extraction.

Are you using different sourcetypes?

If you want to add othe data sources, you can create your own sourcetypes eventually starting from an existern one.

The Add Data function is very useful to find the correct sourcetype to associate to your data sources.

Ciao.

Giuseppe

[orendado]

Yes, I'm using different sourcetype. I would  like to add addtional data that will help distinguish the logs, something like tags or sub category in sourcetype

[gcusello]

Hi @orendado ,

you can tag your data using tags and eventtypes (https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Abouteventtypes) but maintaining the sourcetype of each data source, in this way you have all the parsing rules up and running.

I usually define a sorcetype for each type of data, eventually cloning an existing one: e.g. if I have a custom data source in csv format, I'll clone it from the standard csv and I call it "my_sourcetype" (or the name you like).

In this way I have all the parsing rules of the csv, eventually adding other specific, and I recognize those logs also by sourcetype.

Remember that this is useful only for custom data sources, if you have standard data sources (e.g. Fortinet or Cisco or Checkpoint), it's always better to use the sourcetypes in the add-ons from Splunkbase.

This is relevant also because it isn't sufficient to parse the data, but it's also important to normalize data to use them in apps as Enterprise Security.

In addition, in these add-ons tags and eventtypes are already defined.

Ciao.

Giuseppe