Splunk Search

Not able to search data with source.

sridharlakshman
New Member

Hi Folks,

we are ingested the aws vpc flow logs in splunk and able to see the data while searching with index but while searching with source it is not showing any data for that particular source.

for example: while using below search it is showing data.
1. index=* and able to see that source(aws:cloudwatch:vpclogs)
2. index=* | stats count by source , it is display the source(aws:cloudwatch:vpclogs) with 86 event counts.

but while searching with below command.

index=* source=aws:cloudwatch:vpclogs it is not displaying any data.

Thanks,
Sridhar

Tags (1)
0 Karma

povares_splunk
Splunk Employee
Splunk Employee
  1. Either on $SPLUNK_HOME/etc/apps/{App where the source comes from}/fields.conf or $SPLUNK_HOME/etc/system/local/fields.conf, set the following configuration:

    [source]
    INDEXED = False

  2. Restart Splunk.

NOTE: For Cloud users, have Support do this for you.

To my understanding, this is the reasoning behind this:

INDEXED=True
Setting this attribute to True, tells Splunk that the source has already been extracted during index-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" as their source (metadata).

INDEXED=False
Setting this attribute to False, tells Splunk that the source will need to be extracted during search-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" in their events (raw data). Otherwise, it will not find anything because the source was not extracted during index-time.

Find documentation here: https://docs.splunk.com/Documentation/Splunk/8.0.3/Admin/Fieldsconf

0 Karma

FrankVl
Ultra Champion

Then you need to check the default search indexes that are assigned to your user role. Probably the specific index you put the vpc logs into, is not included in the indexes searched by default (ie. when using index=*).

0 Karma

sridharlakshman
New Member

@renjith, I used the quotes but it is not working. index=test source="aws:guraddutty".

@Frank, I have access for that index and able to search other sources from that index.

0 Karma

FrankVl
Ultra Champion

It's not about whether you have access. There are 2 settings related to indexes in a user role:

  1. The indexes a role has access to (that you have)
  2. The indexes searched by default (ie, when not specifying an index explicitely)

I would check that second setting. Quite possible this is set to the default setting of 'main' (if I'm not mistaken), which means index=* will only look in the main index, while your data is elsewhere. If you want to be able to search using index=*, you need to make sure the Indexes searched by default are set to contain the indexes you want to search through.

0 Karma

sridharlakshman
New Member

I have cross checked that the particular source is storing the data in test index. but it is not working while searching data with that sources

I tried these command and it is not working.

index=* source=aws:guarddutty
index=test source=aws:guarddutty
source=aws:guarddutty.

0 Karma

FrankVl
Ultra Champion

Ok, now I'm getting confused.

In your question, you mentioned source=aws:cloudwatch:vpclogs, now you are searching for another source?

Also: I think it should be source=aws:guardduty, so with a single 't' in duty?

0 Karma

sridharlakshman
New Member

I given example. the issue is not able to search data with source.

0 Karma

FrankVl
Ultra Champion

Just to prevent confusion over typo's in examples, can you provide a screenshot, showing the indexed data, showing the value of the source field. And then a second screenshot showing the search you are trying but fails?

Feel free to hide/obfuscate any sensitive info of course.

0 Karma

sridharlakshman
New Member

I could not able to upload the files here.

0 Karma

sridharlakshman
New Member
0 Karma

FrankVl
Ultra Champion

Just put the screenshots on some online image host and then post the url here.

0 Karma

sridharlakshman
New Member
0 Karma

sridharlakshman
New Member
0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Try putting the source in quotes (")

Happy Splunking!
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...