Identity Management is a part of the "data onboarding" portion of working with asset and identity information in ES. Both assets and identities are stored as lookup files. The lookups have specific fields and requirements, a .csv structure, and may be populated manually or dynamically. You may also configure both dynamic and manually updated content, as all configured lookups of a type are loaded and compared, with the resulting merged list being used for Identities reference and search in ES. http://docs.splunk.com/Documentation/ES/3.3.0/Install/IdentityManager http://docs.splunk.com/Documentation/ES/3.3.0/Install/IdentityManager#Integrate_new_sources_of_asset_and_identity_information Lists and Lookups is a handy page to review and edit lookup content. http://docs.splunk.com/Documentation/ES/3.3.0/Install/Applicationprotocolsblacklist#Lists_and_lookups_editor Identities relate to user information such as credentials, roles, email addresses, or sites. http://docs.splunk.com/Documentation/ES/3.3.0/Install/IdentityManager#Identities_fields Assets relate to network devices such as servers, workstations, routers, switches, and other devices. http://docs.splunk.com/Documentation/ES/3.3.0/Install/IdentityManager#Asset_fields For ES to provide a complete perspective, you will need both assets and identities configured. Can I populate the csv files with more fields than are currently there? Adding additional fields beyond what is defined and required for the lookup won’t prevent the lookup from being merged, but you won’t see the added fields and they won’t be used with the provided ES searches. How can I configure what can be put into these csv files and what information is monitored? The Identity fields and requirements are defined by ES. If your content is correctly mapped to the fields, you will see the results in the proper context depending upon the dashboard/data you're viewing. If you're looking for customization, I would speak to your Sales Engineer to discuss the use case. ... View more

I would begin by confirming that TA-squid has been deployed on the indexer. Also check that the props/transforms from TA-squid are being merged properly on the indexer by using "btool". If those configs are in place and being merged, please review the readme file included with TA-squid, specifically the log format requirements stated therein. And good luck! ... View more

I rather like the troubleshooting steps taken in the Answers post on debugging a UF that's not reading a log file. ... View more

There's no magic answer available, as the type of search and the data volume per day will have a large impact on relative performance. If your SATA-based storage delivers the promised 600 IOPS (test with bonnie++,) and the type of searches you are running are balanced between CPU and I/O bound, and the data volume per-day per-indexer averages ~100GB/Day, and the data coming from the forwarders is evenly distributed across all indexers, and assuming an identical CPU core count, then having 2x indexers with less IOPS should equal your other hardware. There is no substitute for testing with actual data and a search and indexing load. Please note that a lot of admins discover that poor search result speed causes user aggravation, and insufficient IOPS can be a factor. ... View more

Yes, the process for measuring Splunk Enterprise forwarding thruput is a send > check numbers > limit > repeat. Obviously I'm not a dev! This results in spikes of data. More detail than that on the process would have to come through support. In any case, reducing the persistent queue on the outputs.conf to something much smaller (~128k) was offered as a way to flatten the peaks, as there was less data queued to send. Play with that on a select group of forwarders and check the results. ... View more

The panels and other visualizations in ES require data that's been normalized into the data models and accelerated. One way to isolate and test the acceleration is by dropping the summariesonly macro from the search string above. If the accelerations have not finished but the data is normalized properly, dropping summariesonly macro should reveal numbers in the "current_count" and "delta" stats. The Data Model Audit dashboard is another tool for looking at the overall status of the data model accelerations. ... View more