I have a csv file that's giving me a headache while trying to index it. It has 100+ columns, several of which are making life difficult by containing large amounts of things like quotes and newlines. A sanitised example showing the header line and a problem event: field1,field2,field3,field4,field5,field6 "55634","Barney","","this field behaves well","","1436504081000" "","Fred","","Here, have some data that will make your life very difficult ""should"" you try to parse this puppy","F6E25B","1435307738000" (The quotes around should are intentional, there's sections of the data that look exactly like that) I've tried using the following props, to no avail - Barney does the right thing, but Fred's line breaking goes wrong. Can someone point out where I'm going wrong? BREAK_ONLY_AFTER=\"$ HEADER_FIELD_LINE_NUMBER=1 NO_BINARY_CHECK=true SHOULD_LINEMERGE=true TIMESTAMP_FIELDS=field6 This file is created completely new at a regular interval - it's a scheduled database dump. I want to index the entirety each time. I want to keep the inputs.conf as simple as possible, only defining host, sourcetype and destination index. A parsing app on the indexer will have the props.conf. Thanks in advance for any help. ... View more

I have a Linux Universal Forwarder that will be receiving events via the REST interface's simple receiver. https://linuxUF:8089/services/receivers/simple?host=xxx&source=xxx&index=xxx&sourcetype=xxx&check-index=false Can I set up a minimum capability role (i.e. not admin) user on the UF so that events can be accepted and forwarded to the indexer? I'd like to create a local user on the UF, and give that user this role. ... View more

Splunk v 6.1.4 SA-ldapsearch v2.0.0 ldap.conf [default] port = 636 server = adhost.mydomain.local ssl = 1 [mydomain.local] alternatedomain = basedn = dc=mydomain,dc=local binddn = cn=mycredentials,cn=Service Account,cn=Domain Services,dc=mydomain,dc=local When I hit the 'test connection' button, I get this in SA-ldapsearch.log every time 2014-10-16 20:46:02,628, Level=ERROR, Pid=15390, File=search_command.py, Line=342, Traceback (most recent call last): File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/splunklib/searchcommands/search_command.py", line 316, in process self.execute(operation, reader, writer) File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/splunklib/searchcommands/generating_command.py", line 79, in _execute for record in operation(): File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/ldapsearch.py", line 87, in generate password=configuration.credentials.password) as connection: File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/core/connection.py", line 264, in __enter_ self.open() File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/strategy/syncWait.py", line 53, in open self.connection.refresh_dsa_info() File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/core/connection.py", line 618, in refresh_dsa_info self.server.get_info_from_server(self) File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/core/server.py", line 273, in get_info_from_server self.get_schema_info(connection) File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/core/server.py", line 229, in _get_schema_info schema_entry = connection.response[0]['attributes']['subschemaSubentry'][0] if result else None File "/opt/splunk/shared/etc/apps/SA-ldapsearch/bin/packages/ldap3/utils/caseInsensitiveDictionary.py", line 33, in __getitem_ return self._store[self._getkey(key)] KeyError: 'subschemaSubentry' Where am I going wrong? ... View more

I have a transaction defined where a trade goes through some stages in its lifecycle. Unfortunately, the markers for these stages aren't consistent in their form, eg. "<$DealId> : The deal has hit stage BOOKED" "<$DealId> : The deal is EXECUTED" "<$DealId> : Received CONFIRMATION" will be found at various times in the transaction. I have come up with a query, but it keeps throwing errors : "Fields cannot be assigned a boolean result. Instead, try if([bool expr], [expr], [expr]). " Here's the query DealId="*" | transaction DealId | eval confirmed=searchmatch("Received CONFIRMATION") | eval executed=searchmatch("The deal is EXECUTED" AND "DownstreamSystemId" | eval booked=searchmatch("Deal has hit stage EXECUTED") | eval status=case(confirmed==true, "CONFIRMED", executed==true, "EXECUTED", booked==true, "BOOKED") | table DealId, allocQty, price, value, transactTime, status A completed transaction will have all of these stages, I'm trying to keep track of which stage a particular deal is up to. It also feels horribly inefficient, is there a better way of writing this query? ... View more

I'm collecting events from a logfile that look like this : 270929.542: [GC 270929.542: [ParNew Desired survivor size 1288490184 bytes, new threshold 16 (max 31) - age 1: 34518968 bytes, 34518968 total - age 2: 257792 bytes, 34776760 total - age 11: 60416 bytes, 34837176 total : 3156097K->34336K(4718592K), 0.0357680 secs] 3548065K->426305K(17301504K), 0.0359060 secs] However, when I see them in Splunk, I only get the first line. The entire 6 lines of this log get written to the file at once, but Splunk seems to only be storing the first line. Does anyone have any ideas as to what could be going on here? The last line contains the info I really want to work with. ... View more