Dear Members, We are developing a Splunk add-on, and we need to find the octet (byte) count of a data field in an INGEST_EVAL statement in tranforms.conf. Currently we are using the length(_raw) statement, but it is problematic because if the data to be transferred contains UTF-8 characters then the number of characters returned by length() will not match the actual number of bytes used to store the message. Is there a function that returns the byte-count in a single step, or do we have to employ some kind of magic to get this information? Thanks in advance! ... View more

Hello Everyone, I am at the stage of considering to submit a Splunk app to SplunkBase, and before submitting our code, I am running validation steps via `slim validate`: docker run --volume /home/USER/Development/WORK/SPLUNK-ingest/splunk-app:/apps slim-pip:latest slim package /apps/APP_NAME slim package: Packaging app at "/apps/APP_NAME" slim package: [WARNING] /apps/APP_NAME/default/app.conf, line 12: Undefined setting in app.conf, stanza [ui]: supported_themes slim package: [WARNING] /apps/APP_NAME/default/transforms.conf, line 30: Undefined setting in transforms.conf, stanza [all_in_one]: INGEST_EVAL slim package: [WARNING] /apps/APP_NAME/default/transforms.conf, line 33: Undefined setting in transforms.conf, stanza [syslog_octet_count]: INGEST_EVAL slim package: [WARNING] /apps/APP_NAME/default/transforms.conf, line 36: Undefined setting in transforms.conf, stanza [octet_count_prepend]: INGEST_EVAL slim package: [WARNING] /apps/APP_NAME/default/transforms.conf, line 39: Undefined setting in transforms.conf, stanza [discard_empty_msg]: INGEST_EVAL slim package: [WARNING] /apps/APP_NAME/default/transforms.conf, line 42: Undefined setting in transforms.conf, stanza [discard_internals]: INGEST_EVAL slim package: [INFO] Source package exported to "/apps/APP_NAME-0.7.1.tar.gz" docker run --volume /home/USER/Development/WORK/SPLUNK-ingest/splunk-app:/apps slim-pip:latest slim validate /apps/APP_NAME-0.7.1.tar.gz slim validate: Validating app at "/apps/APP_NAME-0.7.1.tar.gz"... slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/app.conf, line 12: Undefined setting in app.conf, stanza [ui]: supported_themes slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/transforms.conf, line 30: Undefined setting in transforms.conf, stanza [all_in_one]: INGEST_EVAL slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/transforms.conf, line 33: Undefined setting in transforms.conf, stanza [syslog_octet_count]: INGEST_EVAL slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/transforms.conf, line 36: Undefined setting in transforms.conf, stanza [octet_count_prepend]: INGEST_EVAL slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/transforms.conf, line 39: Undefined setting in transforms.conf, stanza [discard_empty_msg]: INGEST_EVAL slim validate: [WARNING] /root/.config/slim/repository/slim.cache_xndmoh9d/APP_NAME-0.7.1.source/APP_NAME/default/transforms.conf, line 42: Undefined setting in transforms.conf, stanza [discard_internals]: INGEST_EVAL slim validate: [INFO] App validation complete  What may be the reason of `slim validate` complaining about `INGEST_EVAL`? On my 9.2.1+ Splunk instance that I tested the app with, seem to work fine as is. Any ideas? ... View more

Dear Members, I have a use case where I would need to update or insert configuration to transforms.conf, props.conf and outputs.conf. I was told that it is possible to do this via a creating an app. That would make it easier for users to make the necessary changes, instead of doing it via the error-prone manual procedure. Nevertheless, I haven't come across any documentation that would illustrate and explain how to do it. Does someone have any experience with that? Or perhaps can someone point me to the relevant documentation? Thanks in advance! ... View more

Hello everyone! I am experimenting with the SC4S transforms that are posted here: https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/ My problem is that I am trying to reformat fields, and in one particular place I would need to ensure that a space preceeds the _h= part in the transform stanza below. [md_host] SOURCE_KEY = MetaData:Host REGEX = ^host::(.*)$ FORMAT = _h=$1 $0 DEST_KEY = _raw However if I add multiple whitespaces in the FORMAT string, right after the equals sign in the above example, they will be ignored. Should put the whole thing betweem quotes? Wouldn't the quotes be included in the _raw string? What would be the right solution for this?   ... View more

Hello everyone! I am experimenting with the SC4S transforms that are posted here: https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/ My goal is to send the logs to a syslog-ng instance running with a custom config. My current problem is that the SC4S config contains a part where it checks for subseconds, and appends the value to the timestamp, if found.   [metadata_source] SOURCE_KEY = MetaData:Source REGEX = ^source::(.*)$ FORMAT = _s=$1 $0 DEST_KEY = _raw [metadata_sourcetype] SOURCE_KEY = MetaData:Sourcetype REGEX = ^sourcetype::(.*)$ FORMAT = _st=$1 $0 DEST_KEY = _raw [metadata_index] SOURCE_KEY = _MetaData:Index REGEX = (.*) FORMAT = _idx=$1 $0 DEST_KEY = _raw [metadata_host] SOURCE_KEY = MetaData:Host REGEX = ^host::(.*)$ FORMAT = _h=$1 $0 DEST_KEY = _raw [metadata_time] SOURCE_KEY = _time REGEX = (.*) FORMAT = _ts=$1$0 DEST_KEY = _raw [metadata_subsecond] SOURCE_KEY = _meta REGEX = \_subsecond\:\:(\.\d+) FORMAT = $1 $0 DEST_KEY = _raw   In my case however, when it's not found, the timestamp field will not get a whitespace appended, and thus it will be practically concatenated with the following field, which is not what I want. How could set up the config so that there will always be a whitespace before the next field (the host/_h field)? I tried adding an extra whitespace in front of the _h in the FORMAT part of the metadata_host stanza, but that seems to be ignored. This is what I see:   05:58:07.270973 lo In ifindex 1 00:00:00:00:00:00 ethertype IPv4 (0x0800), length 16712: (tos 0x0, ttl 64, id 49071, offset 0, flags [DF], proto TCP (6), length 16692) 127.0.0.1.49916 > 127.0.0.1.cslistener: Flags [.], cksum 0x3f29 (incorrect -> 0x5743), seq 1:16641, ack 1, win 260, options [nop,nop,TS val 804630966 ecr 804630966], length 16640 0x0000: 0800 0000 0000 0001 0304 0006 0000 0000 ................ 0x0010: 0000 0000 4500 4134 bfaf 4000 4006 3c12 ....E.A4..@.@.<. 0x0020: 7f00 0001 7f00 0001 c2fc 2328 021a 7392 ..........#(..s. 0x0030: 486d 209f 8010 0104 3f29 0000 0101 080a Hm......?)...... 0x0040: 2ff5 b1b6 2ff5 b1b6 5f74 733d 3137 3336 /.../..._ts=1736 0x0050: 3931 3730 3739 5f68 3d73 706c 756e 6b2d 917079_h=splunk- 0x0060: 6866 205f 6964 783d 5f6d 6574 7269 6373 hf._idx=_metrics 0x0070: 205f 7374 3d73 706c 756e 6b5f 696e 7472 ._st=splunk_intr    This is the interesting part:   0x0040: 2ff5 b1b6 2ff5 b1b6 5f74 733d 3137 3336 /.../..._ts=1736 0x0050: 3931 3730 3739 5f68 3d73 706c 756e 6b2d 917079_h=splunk- 0x0060: 6866 205f 6964 783d 5f6d 6574 7269 6373 hf._idx=_metrics   The _h will come right after the end of the _ts field, without any clear separation. ... View more