Use CSV file field as a condition in a search

[kvnpichon]

Hello Splunkers,

This is my goal :

• A table with 3 column (field, field_type, field_len) and export it as CSV and CSV file name must be the sourcetype used in input (as a condition).
  • field = list all field for the sourcetype
  • field_type = string, bool, int, etc.
  • field_len = field length

The issue is that I must launch the search for each sourcetype in my indexes (that's a lot).

My CSV file is that form (it lists all sourcetype I use) :

Sourcetype
sourcetype1
sourcetype2
sourcetype3
...
sourcetypeN

My query is actually like :

index=* sourcetype=MY_SOURCETYPE
| fieldsummary 
| eval field_type=typeof(field), field_len=len(field)
| table field, field_type, field_len
| dedup field

I want to add the multiple export to CSV and use a CSV in input instead of sourcetype="MY_SOURCETYPE"

It could be like :

index=main sourcetype=$sourcetype_from_csv_file$
| fieldsummary 
| eval field_type=typeof(field), field_lgth=len(field) 
| table field, field_type, field_lgth
| depup field
| outputcsv $sourcetype_from_csv_file$.csv

How can I build this request as I don't know how to export in search / how to use a csv as input ?

[aashiqwork]

You have a lookup table called "fullusernames.csv", and it contains three columns: username, first_name, and last_name. A sample row from this lookup table contains jsmith, jane, smith - so the username jsmith is mapped to a user whose full name is jane smith.

You perform this search: index=web_proxy and it returns events that contain username=jsmith. You can use the lookup to find the user's full name:

index=web_proxy | lookup full_user_names.csv username OUTPUTNEW first_name, last_name

After the lookup, the event will contain two new fields: first_name=jane and last_name=smith.

Now let's imagine you have that same lookup table, but your search returns events that contain local_user=jsmith (note the field name is now local_user, which doesn't match the field name username in your lookup. No problem, you use the AS clause to fix it:

index=web_proxy | lookup full_user_names.csv username AS local_user OUTPUTNEW first_name, last_name

Again, after the lookup, the event will contain two new fields: first_name=jane and last_name=smith.

To make matters even more complicated, now you have the same lookup table, and your search returns events that contain local_user=jsmith, and in order to correlate your events with some other logs, you want the user's first name to be returned into a field named f_name and last name to be returned into a field named l_name. Again, no problem - you solve it with the AS clause again:

index=web_proxy | lookup full_user_names.csv username AS local_user OUTPUTNEW first_name AS f_name, last_name AS l_name

Now, after the lookup, the event will contain two new fields: f_name=jane and l_name=smith.

Hope this helps !!!