Archive

Not able to search data with source.

New Member

Hi Folks,

we are ingested the aws vpc flow logs in splunk and able to see the data while searching with index but while searching with source it is not showing any data for that particular source.

for example: while using below search it is showing data.
1. index=* and able to see that source(aws:cloudwatch:vpclogs)
2. index=* | stats count by source , it is display the source(aws:cloudwatch:vpclogs) with 86 event counts.

but while searching with below command.

index=* source=aws:cloudwatch:vpclogs it is not displaying any data.

Thanks,
Sridhar

Tags (1)
0 Karma

Splunk Employee
Splunk Employee
  1. Either on $SPLUNKHOME/etc/apps/{App where the source comes from}/fields.conf or $SPLUNKHOME/etc/system/local/fields.conf, set the following configuration:

    [source]
    INDEXED = False

  2. Restart Splunk.

NOTE: For Cloud users, have Support do this for you.

To my understanding, this is the reasoning behind this:

INDEXED=True
Setting this attribute to True, tells Splunk that the source has already been extracted during index-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" as their source (metadata).

INDEXED=False
Setting this attribute to False, tells Splunk that the source will need to be extracted during search-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" in their events (raw data). Otherwise, it will not find anything because the source was not extracted during index-time.

Find documentation here: https://docs.splunk.com/Documentation/Splunk/8.0.3/Admin/Fieldsconf

0 Karma

Ultra Champion

Then you need to check the default search indexes that are assigned to your user role. Probably the specific index you put the vpc logs into, is not included in the indexes searched by default (ie. when using index=*).

0 Karma

New Member

@renjith, I used the quotes but it is not working. index=test source="aws:guraddutty".

@Frank, I have access for that index and able to search other sources from that index.

0 Karma

Ultra Champion

It's not about whether you have access. There are 2 settings related to indexes in a user role:

  1. The indexes a role has access to (that you have)
  2. The indexes searched by default (ie, when not specifying an index explicitely)

I would check that second setting. Quite possible this is set to the default setting of 'main' (if I'm not mistaken), which means index=* will only look in the main index, while your data is elsewhere. If you want to be able to search using index=*, you need to make sure the Indexes searched by default are set to contain the indexes you want to search through.

0 Karma

New Member

I have cross checked that the particular source is storing the data in test index. but it is not working while searching data with that sources

I tried these command and it is not working.

index=* source=aws:guarddutty
index=test source=aws:guarddutty
source=aws:guarddutty.

0 Karma

Ultra Champion

Ok, now I'm getting confused.

In your question, you mentioned source=aws:cloudwatch:vpclogs, now you are searching for another source?

Also: I think it should be source=aws:guardduty, so with a single 't' in duty?

0 Karma

New Member

I given example. the issue is not able to search data with source.

0 Karma

Ultra Champion

Just to prevent confusion over typo's in examples, can you provide a screenshot, showing the indexed data, showing the value of the source field. And then a second screenshot showing the search you are trying but fails?

Feel free to hide/obfuscate any sensitive info of course.

0 Karma

New Member

I could not able to upload the files here.

0 Karma

New Member
0 Karma

Ultra Champion

Just put the screenshots on some online image host and then post the url here.

0 Karma

New Member
0 Karma

New Member
0 Karma

SplunkTrust
SplunkTrust

Try putting the source in quotes (")