About daubsi_2

daubsi_2 · ‎07-06-2023

Works perfectly! Awesome! Thanks a lot!

daubsi_2 · ‎07-05-2023

I would like to manually import AWS Cloudtrail logs which were stored as GZipped JSON Files on S3. Those files reside on my local disk, one file per hour, per day, etc. They should not be imported directly from the Cloud, e.g. via the Splunk TA for AWS. I have installed that app though to get all the various AWS specific sourcetypes. The problem with the data is, that those files contain only a single line of data which is a huge JSON array containing all the individual events. This is apparently not too seldom, so I am refering to AWS Cloudtrail only for the sake of providing an example of this format. Small artificially contrived example how such a file looks like, here 3 events. {"Records":[{"eventVersion":"1.08","eventTime":"2022-06-08T22:10:01Z","userIdentity":{"type":"AssumedRole"}},{"eventVersion":"1.08","eventTime":"2022-06-08T22:10:03Z","userIdentity":{"type":"AssumedRole"}},{"eventVersion":"1.08","eventTime":"2022-06-08T22:10:05Z","userIdentity":{"type":"AssumedRole"}}]} Of course the real Cloudtrail events are much more talkative and layered, but all this does not pose a problem for that case here. Selecting the Sourcetype " aws:cloudtrail" does not properly split the events. I changed the LINE_BREAKER to the following value: LINE_BREAKER=((\{"Records":\[)*|,*){"eventVersion" Using this I was able to properly index all the events and even get rid of the header upfront. However, the very last event still gets wrongly indexed, as it ends with the closing " ]}" from the opening/wrapping "Records" element and as such it's not proper JSON. How can I get rid of that trailing "junk" "]}" so also the last event gets properly indexed?

daubsi_2 · ‎01-27-2022

Too bad 😞 Hope they will offer a solution in the near future though

daubsi_2 · ‎01-23-2022

I have installed Splunk on a cgroup1/2 hybrid system using "enable boot-start systemd-managed 1" to start it on bootup. Yesterday I switched to a cgroup2 only system by disabling the usage of cgroup1 via grub/kernel boot parameters. Now splunk doesn't start anymore due to a file in the cgroup1 file system hierarchy no longer been present: Jan 22 10:25:54 bigigloo systemd[1]: Stopping Systemd service file for Splunk, generated by 'splunk enable boot-start'... Jan 22 10:30:58 bigigloo systemd[1]: Splunkd.service: Killing process 2847689 (python3.7) with signal SIGKILL. Jan 22 10:30:58 bigigloo systemd[1]: Splunkd.service: Succeeded. Jan 22 10:30:58 bigigloo systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'. -- Reboot -- Jan 22 10:36:19 bigigloo systemd[1]: Starting Systemd service file for Splunk, generated by 'splunk enable boot-start'... Jan 22 10:36:19 bigigloo bash[3180]: chown: cannot access '/sys/fs/cgroup/cpu/system.slice/Splunkd.service': No such file or directory Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Control process exited, code=exited, status=1/FAILURE Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3393 (sh) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3408 (sh) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Failed with result 'exit-code'. Jan 22 10:36:22 bigigloo systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:22 bigigloo bash[3475]: chown: cannot access '/sys/fs/cgroup/cpu/system.slice/Splunkd.service': No such file or directory Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Scheduled restart job, restart counter is at 1. Jan 22 10:36:23 bigigloo bash[3480]: chown: cannot access '/sys/fs/cgroup/cpu/system.slice/Splunkd.service': No such file or directory Jan 22 10:36:22 bigigloo systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:22 bigigloo systemd[1]: Starting Systemd service file for Splunk, generated by 'splunk enable boot-start'... Jan 22 10:36:23 bigigloo bash[3496]: chown: cannot access '/sys/fs/cgroup/cpu/system.slice/Splunkd.service': No such file or directory Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Control process exited, code=exited, status=1/FAILURE Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3476 (sh) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3477 (btool) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Failed with result 'exit-code'. Jan 22 10:36:22 bigigloo systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Scheduled restart job, restart counter is at 2. Jan 22 10:36:22 bigigloo systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:22 bigigloo systemd[1]: Starting Systemd service file for Splunk, generated by 'splunk enable boot-start'... Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Control process exited, code=exited, status=1/FAILURE Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3481 (sh) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Killing process 3482 (btool) with signal SIGKILL. Jan 22 10:36:22 bigigloo systemd[1]: Splunkd.service: Failed with result 'exit-code'. Jan 22 10:36:22 bigigloo systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Scheduled restart job, restart counter is at 3. Jan 22 10:36:23 bigigloo systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:23 bigigloo systemd[1]: Starting Systemd service file for Splunk, generated by 'splunk enable boot-start'... Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Control process exited, code=exited, status=1/FAILURE Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Killing process 3497 (sh) with signal SIGKILL. Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Killing process 3499 (btool) with signal SIGKILL. Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Failed with result 'exit-code'. Jan 22 10:36:23 bigigloo systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:23 bigigloo systemd[1]: Splunkd.service: Scheduled restart job, restart counter is at 4. Jan 22 10:36:23 bigigloo systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'. Jan 22 10:36:23 bigigloo systemd[1]: Starting Systemd service file for Splunk, generated by 'splunk enable boot-start'... I tracked the problem down to the two ExecStartPost commands in the unit file /etc/systemd/system/Splunkd.service. Commenting those two fixed the problem. #This unit file replaces the traditional start-up script for systemd #configurations, and is used when enabling boot-start for Splunk on #systemd-based Linux distributions. [Unit] Description=Systemd service file for Splunk, generated by 'splunk enable boot-start' After=network.target [Service] Type=simple Restart=always ExecStart=/opt/splunk/bin/splunk _internal_launch_under_systemd KillMode=mixed KillSignal=SIGINT TimeoutStopSec=360 LimitNOFILE=65536 SuccessExitStatus=51 52 RestartPreventExitStatus=51 RestartForceExitStatus=52 User=root Group=root Delegate=true CPUShares=1024 MemoryLimit=20868083712 PermissionsStartOnly=true #ExecStartPost=/bin/bash -c "chown -R root:root /sys/fs/cgroup/cpu/system.slice/%n" #ExecStartPost=/bin/bash -c "chown -R root:root /sys/fs/cgroup/memory/system.slice/%n" [Install] WantedBy=multi-user.target However, I presume updates of Splunk might restore the files to the old variant again. What do I need to do in order to make the start of Splunk cgroup2 compliant?

daubsi_2 · ‎04-11-2017

Dear all, I am a user of the Splunk Developer License aka Enterprise license, time limited to 6 months. Recently it happened that my license expires unnoticed and when I connected to the search head again a couple of days later, Splunk told me, that it had expired and I have this and that many violations because of this. Even after applying my prolonged new dev license, I was not able to search again, because of the "5 violations in 30 days" restrictions and had to ask for a reset license, so I could use Splunk again immediately. I now want to create a search/alert that warns me ahead of time that my license is due to expire so I can prolong it in time. However, I cannot seem to find the proper sourcetype / index, where I can find that information? I'm pretty sure that Splunk splunks these messages, that the license is about to expire in n days, doesn't it? BR Markus

daubsi_2 · ‎06-10-2016

For your convenience, here's a diff so you can apply the patch: -- --- fill_summary_index.py 2016-05-13 00:05:14.000000000 +0200 +++ fill_summary_index_oko3.py 2016-06-10 15:17:18.000000000 +0200 @@ -9,6 +9,7 @@ import splunk.auth import splunk.search import splunk.util +import splunk.entity as entity import splunk.clilib.bundle_paths as bundle_paths app = None @@ -158,12 +159,12 @@ -timefield <string> The field in summary index data that contains the scheduled time of the saved search that generated that data - + 1 - For a full description of Splunk relative time strings, see: http://docs.splunk.com/Documentation/Splunk/6.4.1/User/ChangeTheTimeRangeOfYourSearch#Syntax_for_relative_time_modifiers ''' sys.exit(0) - + def getBoolArg(opt, val): try: return splunk.util.normalizeBoolean(val, enableStrictMode=True) @@ -349,7 +350,8 @@ cur_st_list = [] index = indexarg try: - ent = splunk.search.getSavedSearchWithTimes(ssname, et, lt, namespace=app, sessionKey=sk, owner=owner) + #ent = splunk.search.getSavedSearchWithTimes(ssname, et, lt, namespace=app, sessionKey=sk, owner=owner) + ent = entity.getEntity("/saved/searches", ssname, uri="/servicesNS/"+owner+"/"+app+"/saved/searches/"+ssname+"/scheduled_times", earliest_time=et, latest_time=lt, namespace=app, owner=owner, sessionKey=sk) if 'scheduled_times' in ent: for st in ent['scheduled_times']: cur_st_list.append((ssname, st))

Posts	6
Solutions	1
Karma Given	4
Karma Received	2
Member Since	‎06-10-2016

Online Status	Offline
Date Last Visited	‎07-12-2023 11:11 AM

Properly index json files with events as elements ...

Splunk doesn't start on cgroup2 only system

How to get an alert before license is about to exp...

Re: Properly index json files with events as eleme...

Properly index json files with events as elements ...

Re: Splunk doesn't start on cgroup2 only system

Splunk doesn't start on cgroup2 only system

How to get an alert before license is about to exp...

Re: Summary Index Backfiller fill_summary_index.py...