check over your TLS/SSL certs, config and settings on all endpoints. ... View more

Where are the events coming from that are in this index? Sounds to me like the data source itself is at fault and you're missing events, leaving you with gaping holes in your data because you should be getting zero-based events. ... View more

I thought fillnull is only good for charting? He never said he was charting, I think he needs to put in a whole record for that entry he is missing... ... View more

Try DB Connector, make sure that ODBC-DS is machine based and not user based... Should be fine. ... View more

Post a raw event please... Are you saying that a single field X is formed as X=A|B|C|D|E using pipes? ... View more

Scenario 1 is more practical and less troublesome, it's worth doing that if you end up with your logs looking nice at the indexer with the correct source host. Keep in mind that there is a distinct difference between UDP and TCP when using syslog, you are more likely to lose data with UDP as it's black-holed if the endpoint isn't available. Always better to use 2xHF's with load balancing. You could stick to using TCP throughout - you can get a forwarder to index a copy of events locally and then forward on, that's always an option, but you need an enterprise license, I personally would keep a distance from that idea. With syslog you can't really avoid losing events if you need to make changes to the handlers! I specialise in the syslog area in general, I've dealt with massive scale systems that were a mixture of scenario 1 & 2 & others - We actually ended up consolidating everything to load balancers which then forwarded to 4 x RHEL rsyslog instances (like scenario 2), dropped to disk for safety & compliance, but then relayed instantly to HF's and an external SOC - to preserve the source host header mostly. There will always be the problem of determining what is missing and then manually re-loading the events in that were missed, but we decided this would only be the case when a large window of data was missing or to supplement an incident investigation. ... View more

The last version of this app to be released was Version 1.1.8 Dec. 25, 2016, I assume that 18 months down the line, they aren't bothering with it anymore? ... View more

You should check compatibility then for DB connector, this is clearly defined. Lots of the AWS stuff is REST enabled so failing the DB connector, look towards the REST connector in splunk - but beware you will be starting from scratch, you may end up creating a new component based on that! Please share if you do :). cheers ... View more

Sorry what do you mean by RDS? Remote Desktop Services? If you mean RDB (Relational DataBase) then the answer is yes, you can connect to all three of those, it's tricky learning how to but there are lots of examples kicking about, as well as your ability to investigate third party apps that use DB connector, I learned how to use it via reverse engineering the Mcafee app. There are different ways to harvest DB data and this would relate to the result sets of the queries, so for example scalar queries will return a single value, tabular will return rows that come in to Splunk, or perhaps you can tell DB connector to download/sync a table regularly. It's something you will need to master, but typically you run SQL commands against the servers as a client would. ... View more

Hi, your idea is flawed a bit... You want splunk to connect to syslog and get raw logs, this is impossible. Splunk will either receive syslog messages from hosts that are sending syslog data to specified IP's (Your indexers / HF's), or it can relay any data received, to syslog. You can't "pull" with syslog, you only receive or forward/send syslog messages. Syslog is simplex. I think you're diving too deep too soon, all of the points you specify are advanced and some of them can not be done out of the box, there will be hurdles and problems to even the more advanced devs.. Firstly, you need to re-consider the use case here and be more clear, the question is too broad as well. Tell your syslog hosts to send via TCP to your Splunk instance, then create your syslog listener (inputs.conf in etc/system/local OR via web GUI TCP inputs), consider using outputs.conf / web GUI forwarding to relay the syslog messages elsewhere if you require that, confirm it is working by watching your splunk daemon logs. Later, try to move these settings to your own custom app, and take it from there. I'd imagine for advanced functionality of forms you want to use BASH scripting and KV Store + REST API, all of which are tricky to combine and I've never done anything like it before either... You have a long way to go before you get any of what you asked materialised! ... View more

Hi niketnilay, have you had any thoughts on this? anyone??? ... View more

I would imagine this, like suggested on the other Q, would be HTTP spec %XX encoding? After all this is a RESTful service and I believe the question more surrounds how the REST service interprets the data when it lands via HTTP and not how splunk handles or stores it. Certainly with a GET request (Original REST) the %XX will be supported but I'm not sure about how POST header encoding works in later REST versions that have troublesome tokens etc, possibly the same but may need a prefix? I'm unable to research this right now. The difference between a friendly URL and un-friendly URL (GET) matters more on the HTTP server than the browser, for example, and the POST data is encoded in to the request header ?somehow?. ... View more

I think if you have the data in splunk as sourcetype _json or json_no_timestamp you should be ok, see how this turns up in a verbose mode search and there may not be a need to use INDEXED_EXTRACTIONS=json or KV_MODE = none or spath queries. There are issues when you have data that "contains" JSON, e.g. an exception message etc... That's when problems begin, is this the case. ... View more

You see pal, if I just use normal regex (Message\=(?<Message2>.+)\.) on regex101.com I get the right extraction, but only on the website, this doesn't work in splunk, even with the extra hacks we have spoken of... This is quite frustrating... I have copied that event from my search results in splunk and not from the windows event log. ... View more

04/17/2018 12:42:30 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4625 EventType=0 Type=Information ComputerName=WIN-JPMM7KL7IDE TaskCategory=Logon OpCode=Info RecordNumber=45629 Keywords=Audit Failure Message=An account failed to log on. Subject: Security ID: S-1-5-18 Account Name: WIN-JPMM7KL7IDE$ Account Domain: WORKGROUP Logon ID: 0x3E7 Logon Type: 7 Account For Which Logon Failed: Security ID: S-1-0-0 Account Name: Administrator Account Domain: WIN-JPMM7KL7IDE Failure Information: Failure Reason: Unknown user name or bad password. Status: 0xC000006D Sub Status: 0xC000006A Process Information: Caller Process ID: 0x3ac Caller Process Name: C:\Windows\System32\svchost.exe Network Information: Workstation Name: WIN-JPMM7KL7IDE Source Network Address: 127.0.0.1 Source Port: 0 Detailed Authentication Information: Logon Process: User32 Authentication Package: Negotiate Transited Services: - Package Name (NTLM only): - Key Length: 0 This event is generated when a logon request fails. It is generated on the computer where access was attempted. The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network). The Process Information fields indicate which account and process on the system requested the logon. The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The authentication information fields provide detailed information about this specific logon request. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested. ... View more

Sure I will get back to you man, need to do it tonight as I'm on a metaframe desktop currently ... View more

I think I am seeing some slight improvement by using the following regex... (?m)^Message\=(?<Message2>.+)\.$ but this doesn't work for event code 4625 in particular, and windows filtering platform events, fine for all the others though... Without the period terminator on the end, seems to work for scheduled task events... So those are now fixed. No idea what the problem is here, I think it's the windows events containing invisible characters or mixtures of \r and \n etc... Still baffled, not uncommon to see these sorts of issues come up and make what is expected to be a simple task a mission to mars! ... View more

Hi mate, tried this and none of it works, I still believe this to be an issue. After using RegEx101.com and the (?ms) option, that seems to cause a similar issue to what I'm seeing in splunk, it picks up the last period in the message as the terminator, strange, still researching the ?ms combination. I've tried using props.conf and still getting the same result. I believe this is to do with the options of regex (e.g regex../options). the rex command defaults are perfect for this but the queue regex is set differently, perhaps I need to enable /g ensuring nothing else is enabled, as per regex101.com where my results are fine ... View more

Remember rex works fine its just inconvenient to have to have this in every query - because I have lots of event codes that are affected by this, and a few alerts to make based on these codes. The difference between using the web GUI and rex is ringing alarm bells, as my main point is that the web GUI doesn't work. ... View more

I think I did try both, I've tried everything though, apart from props method as opposed to using the GUI field extraction. ... View more

Hi, I will have to try this tonight as my dev workstation is at home. Are you sure about \n\r? I thought it was \r\n? Will try both combinations anyway. ... View more

What operating systems are you using? What other tools have you got available for backups? Looks like a viable solution, but be warned you are cobbling things together by doing this. ... View more

I would try something like a combo of eval / if / isnull() to see if a field has data in, or exists then carry out actions after that ??? ... View more