you're welcome, glad it helped! ... View more

You'll have to make sure your data is CIM compliant. The root search for the datamodel is: (`cim_Network_Traffic_indexes`) tag=network tag=communicate You can build a search to vet the data that the datamodel is processing using the root search and showing the fields in tabular format: (`cim_Network_Traffic_indexes`) tag=network tag=communicate | table _time action app bytes bytes_in bytes_out dest dest_ip src src_ip Note that there's a lot more CIM fields, but since you're asking about bytes I'll focus on that. I'm assuming you have data that is tagged correctly, and that you have null values for bytes, bytes_in and bytes_out - if that's the case you have to make sure those fields exist in the source data, and that those values are all number values and not text values. All of the bytes value fields in the datamodel are calculated: The bytes calculation is case(isnum(bytes),bytes,isnum(bytes_in) AND isnum(bytes_out),bytes_in+bytes_out,1=1,null()) The bytes_in calculation is case(isnum(bytes_in),bytes_in,isnum(bytes) AND isnum(bytes_out),bytes-bytes_out,1=1,null()) The bytes_out calculation is case(isnum(bytes_out),bytes_out,isnum(bytes) AND isnum(bytes_in),bytes-bytes_in,1=1,null()) If your source data does not have the proper combination of bytes, bytes_in, and/or bytes_out or those values are not numeric values then you will wind up with null values for the bytes field values in your datamodel. If you have values that are non-numeric then you'll have to do some normalization work on your source data to convert them to numeric fields. ... View more

If two of the fields are working then my first guess would be that it's an issue with whitespacing in the regular expression. Try running a search that will give you the raw json results and add the following: | rex field=_raw "caller\"\s*:\s*\{\s*\"id\":\s*\"(?<test_callerid>[^\"]+)?\"\,\s*\"version\"\s*:\s*\"(?<test_callerversion>[^\"]+)?\"" See if you get the test_* fields extracted from the json. If the only thing at this point that needs to be tweaked is the regular expression, you might want to try tweaking it on regex101: https://regex101.com/r/4lGvKg/1 ... View more

If collecting local/RDP logon information will do the job for you, you can capture the Windows logs "Microsoft-Windows-TerminalServices-LocalSessionManager/Operational". If you're trying to track network logon sessions, I don't envy you. I worked on that for a bit and came to the conclusion we would never get any kind of useful aggregate reporting out of network session information from Windows logs. ... View more

Use btool on a system where the configuration has been deployed, not on the deployment server. I would probably test the field extraction part on a search head first, then when the extraction syntax has been deployed I would remove it from the testing search head and deploy it to the indexers or heavy forwarders and work on the indexed extraction. ... View more

If you run btool from the command line do you see the configuration items listed? /opt/splunk/bin/splunk cmd btool props list evento_srctyp --debug | less Replace /opt/splunk with whatever your $SPLUNK_HOME path is. I don't know if "evento_srctyp" is the actual sourcetype or if you masked it, but "splunk cmd btool props list {SOURCETYPE} --debug" will dump out the configuration. I wouldn't worry about indexing the field until you have the extractions working. When you can run the search: sourcetype=evento_srctyp | table _time host source callerid callerversion calledid calledversion and get results, then review link text and then work on the indexed extraction. ... View more

Is your ultimate goal to identify when users log on and log off? Are you trying to track interactive logons (local, RDP) only? I ask because if that's what you're trying to do I think there's a better way. Trying to build an idea of logon/logoff by object access is difficult. ... View more

If you're looking to do this with in-line searching, you'll want to use the rex command to extract text with regex. If you're looking to do field extraction, you'll need to read up on props.conf and transforms.conf Here is an in-line extraction example, using your message as a test value: | makeresults | eval message="<*Message>This is what I want to capture.<*/Message>" | rex field=message ">(?<message_value>[^\<]+)<" Regex101 is a great site for testing regular expression: https://regex101.com/r/ihEPfj/1 Regex is far too deep a subject to really cover in a Splunk answers post, but there's a number of good sites that will provide some basic information, including a Splunk page ( Splunk and Regular Expressions ) but when I really wanted to dig in to regex I picked up "Mastering Regular Expressions" by Jeffrey Friedl and it definitely helped. ... View more

*Update: Based on the data you provided in another comment I tweaked the regex. I would avoid lookaheads and lookbehinds if possible, especially with a big payload. It's too easy to have an poorly performing or broken regex. You also don't need to use the FORMAT command in transforms.conf if your regex is formatted to include the field names with the extractions. You can extract the caller and called ID and version fields with two stanzas, one for caller and one for called. [callerid] REGEX = caller\"\s*:\s*\{\s*\"id\":\s*\"(?<callerid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<callerversion>[^\"]+)?\" [calledid] REGEX = called\":\s*\{\s*\"id\":\s*\"(?<calledid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<calledversion>[^\"]+)?\" This was the inline search I used to test it: | makeresults | eval test="{\"info\": {\"eventSource\": \"RPA\", \"sourceType\": \"I\", \"status\": {\"code\": \"0000\", \"msg\": \"Inizio Schedulazione\", \"msgError\": \"\"}, \"transactionId\": \"66083\", \"traceId\": \"124021\", \"timestampStart\": \"2019-10-16T11:34:00.000Z\", \"timestampEnd\": \"null\", \"companyIDCode\": \"01\", \"channelIDCode\": \"\", \"branchCode\": \"\", \"searchFields\": [{\"VDI\": \"WPVRTM2004\"}, {\"PROCESSO\": \"Assegni\"}], \"annotation\": [{\"TIPO\": \"SCHEDULAZIONE\"}, {\"RISORSE POOL\": \"SI\"}], \"caller\": {\"id\": \"VWFM\", \"version\": \"1\", \"acronym\": \"WRPA0\"}, \"called\": {\"id\": \"Assegni\", \"version\": \"1\", \"acronym\": \"WRPA0\"}}, \"payLoad\": {\"output\": {\"encoding\": \"\", \"ccsid\": \"\", \"data\": \"\"}, \"input\": {\"encoding\": \"\", \"ccsid\": \"\", \"data\": \"\"}}}" | rex field=test "called\":\s*\{\s*\"id\":\s*\"(?<calledid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<calledversion>[^\"]+)?\"" | rex field=test "caller\"\s*:\s*\{\s*\"id\":\s*\"(?<callerid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<callerversion>[^\"]+)?\"" ... View more

Escaping backlashes with rex is strange. You could use three slashes: | rex field=_raw "Version=\\\x22(?<Version>.*?)\\\x" or use \x5c: | rex field=_raw "Version=\x5cx22(?<Version>.*?)\x5cx" ... View more

There could be a number of reasons - the first thing I would check is the permissions. Are you running Splunk as a service on the box? If so, are you running it as root or a user account? If you're running it as a service account you have to make sure the account as access to read the file. Are you ingesting other logs from this system, and if so are those being forwarded? I would normally start looking at the splunkd.log file on the host for an idea. grep "/var/log/snort/" /opt/splunkforwarder/var/log/splunk/splunkd.log assuming Splunk is installed in /opt/splunkforwarder, adjust the path as necessary. If you are running with non-root service account and want to verify permissions, then run this with an account that has sudo access: sudo su - splunk -s /bin/sh -c 'tail -n 1 $(find /var/log/snort/ -maxdepth 1 -type f -iname 'snort.log.*' -mtime -1 | tail -n 1)' This assumes that you're running Splunk using the service account "splunk", if you're using a different non-root service account then change "sudo su - splunk ..." to "sudo su - {your service account}" I also don't think you need the stanza "[monitor:///var/log/snort]" since you have "[monitor:///var/log/snort/snort.log.*]" good luck! ... View more

We've solved this by creating a group on the systems that has read access only to the log files we want to process. We configured the syslog service (rsyslog) to write logs with the file permissions 640 with the file group ownership set to the log review group, and we've added the Splunk forwarder service account to this group. You also have to mind your directory permissions to make sure that the Splunk service account can access the directory tree & enumerate/read files. Also make sure the forwarder web interface is disabled and the service account is properly restricted. ... View more

I'm not entirely clear how your messages are actually formatted - is "WindowsIdentity" the name of the field and also part of the field? are all these lines part of the same message and you want to skip the first value - "WindowsIdentity: IIS APPPOOL\login20.monster.com"? Maybe this will work: { ... base search ... } | rex max_match=100 field=WindowsIdentity "IIS APPPOOL\\\+\s?(?<App>.*?)\.monster\.com" | eval app_count=mvcount(App) | eval App=mvindex(App,1,app_count) ... View more

Have you tried btool - if you're running on Linux I would try something like $SPLUNK_HOME/bin/splunk btool props list --debug | grep "userEmail" Field aliasing, calculated fields, and lookups are all performed at search time after KV extractions, maybe there's an errant config that's overwriting your field. ... View more

I'm not sure if you should use a more complex single rex command, but you can: | rex "[Pp]roject (?:name: )?(?<ProjectName>[^\,\r\n]+)(?:\, Machine name\:(?<MachineName>[^\,]+)\, Status\:(?<JobStatus>[^\,]+)\, Backlog bytes\:(?<BacklogBytes>[0-9]+)\, Last consistency\:(?<LastConsistency>.*))?" | table ProjectName, MachineName, JobStatus, BacklogBytes, LastConsistency ... View more

I had set the last search command criteria wrong. I had set it to "| search (EventCode=4625 count>6) OR (EventCode!=4625 count>2)" but looking back at your original post, your second search was filtering with "| where count>1". I've updated my response to change it to " "| search (EventCode=4625 count>6) OR (EventCode!=4625 count>1)" If you still don't get results, simplify the search to look for what isn't there to see if you can find out why. To test, remove the 4625 event criteria from the base search, and remove the filter looking for count>1 and see if you get anything: index="..........." AND sourcetype=".........." AND tag=pci AND user!=null NOT (tag=..........) (EventCode=4720 OR EventCode=4722 OR EventCode=4738) NOT [| inputlookup PCI_service_accounts_list.csv | rename service_accounts as user] | fillnull value="N/A" user, dvc_owner, dvc_bunit, dvc_ip, subject, dvc | bin _time span=1hr as HourDay | eval hourDay=strftime(HourDay,"%m-%d-%y %H:%M:%S") | stats count last(hourDay) as timestamp by user, HourDay, EventCode, Account_Domain, Account_Name, dvc_owner,dvc_bunit,dvc_ip, subject, dvc If you do see the kind of results you expect then add the count>1 back in: index="..........." AND sourcetype=".........." AND tag=pci AND user!=null NOT (tag=..........) (EventCode=4720 OR EventCode=4722 OR EventCode=4738) NOT [| inputlookup PCI_service_accounts_list.csv | rename service_accounts as user] | fillnull value="N/A" user, dvc_owner, dvc_bunit, dvc_ip, subject, dvc | bin _time span=1hr as HourDay | eval hourDay=strftime(HourDay,"%m-%d-%y %H:%M:%S") | stats count last(hourDay) as timestamp by user, HourDay, EventCode, Account_Domain, Account_Name, dvc_owner,dvc_bunit,dvc_ip, subject, dvc | search count>1 and if that works, then the (updated) full search I provided should work. ... View more

Use btool to list out your current configuration. for whatever platform you're on go to the command line into the $SPLUNK_HOME folder and run: bin/splunk btool inputs list 'monitor://C:\UCX\dotnet\UCXWinPortal\logs...\Pinnacle_AX_*.log' --debug That command is for linux - for windows I believe it would be bin\splunk.exe btool ... If you don't get any results, drop the configuration stanza and just run "splunk btool inputs list --debug | more" and find your configuration. It should tell you what Splunk sees as the combined configuration, and the configuration file source it's using for each item. Use btool to troubleshoot configurations ... View more

If you're filtering out anything, it's best to do it as early as possible. By filtering out users at the end instead of the base search you're spending computation time on results you're going to then remove. I moved the last ("search NOT [| inputlookup ...." to your base search to avoid this. I also changed the "AND NOT ((user=null) ..." , the NOT operator is less efficient and since the user field should exist in all the events it can be used here. Use the "NOT" operator where you want to include events where a field doesn't exist. I left the NOT operator in for the tags, since tags may not always exist in some events. The search command "user!=null" or "NOT user=null" will filter out any events where the user is literally named "null". If you have an account named null and you want to exclude it, this will work. If you want to only include fields where "user" is a field that exists, you can just search with "user=*" instead. You can combine the searches and just use the search command with the OR operator to account for both conditions: index="..........." AND sourcetype=".........." AND tag=pci AND user!=null NOT (tag=..........) (EventCode=4720 OR EventCode=4722 OR EventCode=4738 OR EventCode=4625) NOT [| inputlookup PCI_service_accounts_list.csv | rename service_accounts as user] | fillnull value="N/A" user, dvc_owner, dvc_bunit, dvc_ip, subject, dvc | bin _time span=1hr as HourDay | eval hourDay=strftime(HourDay,"%m-%d-%y %H:%M:%S") | stats count last(hourDay) as timestamp by user, HourDay, EventCode, Account_Domain, Account_Name, dvc_owner,dvc_bunit,dvc_ip, subject, dvc | search (EventCode=4625 count>6) OR (EventCode!=4625 count>1) ... View more

Here is the fully formatted example with the sample data you provided: | makeresults | eval test=" CN=AVB6F9974,OU=Farley PC's, OU=Kansas City, OU=Workstations,DC=lop,DC=local|CN=AVB6F9975,OU=Booth PC's, OU=Scott Building, OU=Workstations,DC=lop,DC=local|CN=KBOD6F9975,OU=Booth PC's, OU=Zale Building, OU=Workstations,DC=lop,DC=local" | makemv delim="|" test | mvexpand test | eval _raw=test | rex "CN=(?<commonName>[^\,]+)\,OU=.*?\,\s*OU=(?<location>[^\,]+)\,\s*OU=" ... View more