How do I manually check the stored "hash" in a splunk fs_notification
event and compare it against the a file on the file system? The docs say that the fschange
input generates a hash value using the SHA256 algorithm. I have the sha256sum
utility installed, but it's output looks completely different. Anyone know how to verify the hash of a file manually?
Here is my fs_notification
event:
Thu Aug 4 12:08:31 2011 action=update, path="/home/myuser/SignOn.vb", isdir=0, size=604392, gid=65534, uid=1002, modtime="Tue Jun 7 12:46:32 2011", mode="rw-r--r--", hash=zXd0LxXGDoNzeewkquKeksEV/PhnhtAakFmfMjkpmtU=, chgs="modtime hash "
Looking at the file on my file system:
ls -ln /home/myuser/SignOn.vb
-rw-r--r-- 1 1002 65534 604392 2011-06-07 12:46 /home/myuser/SignOn.vb
sha256sum /home/myuser/SignOn.vb
1cf9c9267c0183d2e248d6318a289796302dd12ddde4712b010a4fadfc008fad /home/myuser/SignOn.vb
I've compared the file size and modification time, but those are easy to fake; I want to be sure these files are the same.
BTW, I've tried simply converting the hex to base64, but that doesn't seem to be enough. It seems like there is some kind of salt or prefix added before the hash is calculated... I'm not sure, any thoughts?
Hmm, I tried an obvious base64 operation as well, taking your "hash=" and decoding it:
$ echo "zXd0LxXGDoNzeewkquKeksEV/PhnhtAakFmfMjkpmtU=" | base64 -d | od -A d -t x1
0000000 cd 77 74 2f 15 c6 0e 83 73 79 ec 24 aa e2 9e 92
0000016 c1 15 fc f8 67 86 d0 1a 90 59 9f 32 39 29 9a d5
0000032
$
No clear relationship to your hash generated by sha256sum. I will play with this some on a scratch monkey.
UPDATE:
I did a rudimentary test on my scratch monkey Splunk install:
dwaddle@dwaddle:~/tmp/fschange$ sha256sum foo
ec52bac967165886a9bf7e559dfbfe9bacf5ff394039fca22dd33ca8b3fe6cff foo
The corresponding fschange event in Splunk, and a hexdump of the base64 decode:
Wed Sep 28 17:28:31 2011 action=add, path="/home/dwaddle/tmp/fschange/foo", isdir=0, size=1598, gid=1000, uid=1000, modtime="Wed Sep 28 17:28:24 2011", mode="r-xr-xr-x", hash=7FK6yWcWWIapv35Vnfv+m6z1/zlAOfyiLdM8qLP+bP8=
dwaddle@dwaddle:~/tmp/fschange$ echo "7FK6yWcWWIapv35Vnfv+m6z1/zlAOfyiLdM8qLP+bP8=" | base64 -d | od -t x1
0000000 ec 52 ba c9 67 16 58 86 a9 bf 7e 55 9d fb fe 9b
0000020 ac f5 ff 39 40 39 fc a2 2d d3 3c a8 b3 fe 6c ff
0000040
This matches the sha256sum output. So, I updated the foo
file by adding a blank line in vi.
Wed Sep 28 17:34:13 2011 action=update, path="/home/dwaddle/tmp/fschange/foo", isdir=0, size=1599, gid=1000, uid=1000, modtime="Wed Sep 28 17:34:04 2011", mode="r-xr-xr-x", hash=PowhA/oG/ngR5dGHbSaWubFSo+9Ly4ERSCbTGUS8Kog=, chgs="modtime hash "
dwaddle@dwaddle:~/tmp/fschange$ sha256sum foo
3e8c2103fa06fe7811e5d1876d2696b9b152a3ef4bcb81114826d31944bc2a88 foo
dwaddle@dwaddle:~/tmp/fschange$ echo "PowhA/oG/ngR5dGHbSaWubFSo+9Ly4ERSCbTGUS8Kog=" | base64 -d | od -t x1
0000000 3e 8c 21 03 fa 06 fe 78 11 e5 d1 87 6d 26 96 b9
0000020 b1 52 a3 ef 4b cb 81 11 48 26 d3 19 44 bc 2a 88
0000040
Based on this, it would appear that Splunk (4.2.1 on my scratch monkey) is using a plain SHA-256 sum that can be directly compared to the output of sha256sum
, once the hash is fed through a base64 decode and an od
to get the byte values.
Unfortunately, this makes it seem like there's either a bug in fschange, or there's a difference in your file between when Splunk hashed it and when you compared it to that hash.
Hmm, I tried an obvious base64 operation as well, taking your "hash=" and decoding it:
$ echo "zXd0LxXGDoNzeewkquKeksEV/PhnhtAakFmfMjkpmtU=" | base64 -d | od -A d -t x1
0000000 cd 77 74 2f 15 c6 0e 83 73 79 ec 24 aa e2 9e 92
0000016 c1 15 fc f8 67 86 d0 1a 90 59 9f 32 39 29 9a d5
0000032
$
No clear relationship to your hash generated by sha256sum. I will play with this some on a scratch monkey.
UPDATE:
I did a rudimentary test on my scratch monkey Splunk install:
dwaddle@dwaddle:~/tmp/fschange$ sha256sum foo
ec52bac967165886a9bf7e559dfbfe9bacf5ff394039fca22dd33ca8b3fe6cff foo
The corresponding fschange event in Splunk, and a hexdump of the base64 decode:
Wed Sep 28 17:28:31 2011 action=add, path="/home/dwaddle/tmp/fschange/foo", isdir=0, size=1598, gid=1000, uid=1000, modtime="Wed Sep 28 17:28:24 2011", mode="r-xr-xr-x", hash=7FK6yWcWWIapv35Vnfv+m6z1/zlAOfyiLdM8qLP+bP8=
dwaddle@dwaddle:~/tmp/fschange$ echo "7FK6yWcWWIapv35Vnfv+m6z1/zlAOfyiLdM8qLP+bP8=" | base64 -d | od -t x1
0000000 ec 52 ba c9 67 16 58 86 a9 bf 7e 55 9d fb fe 9b
0000020 ac f5 ff 39 40 39 fc a2 2d d3 3c a8 b3 fe 6c ff
0000040
This matches the sha256sum output. So, I updated the foo
file by adding a blank line in vi.
Wed Sep 28 17:34:13 2011 action=update, path="/home/dwaddle/tmp/fschange/foo", isdir=0, size=1599, gid=1000, uid=1000, modtime="Wed Sep 28 17:34:04 2011", mode="r-xr-xr-x", hash=PowhA/oG/ngR5dGHbSaWubFSo+9Ly4ERSCbTGUS8Kog=, chgs="modtime hash "
dwaddle@dwaddle:~/tmp/fschange$ sha256sum foo
3e8c2103fa06fe7811e5d1876d2696b9b152a3ef4bcb81114826d31944bc2a88 foo
dwaddle@dwaddle:~/tmp/fschange$ echo "PowhA/oG/ngR5dGHbSaWubFSo+9Ly4ERSCbTGUS8Kog=" | base64 -d | od -t x1
0000000 3e 8c 21 03 fa 06 fe 78 11 e5 d1 87 6d 26 96 b9
0000020 b1 52 a3 ef 4b cb 81 11 48 26 d3 19 44 bc 2a 88
0000040
Based on this, it would appear that Splunk (4.2.1 on my scratch monkey) is using a plain SHA-256 sum that can be directly compared to the output of sha256sum
, once the hash is fed through a base64 decode and an od
to get the byte values.
Unfortunately, this makes it seem like there's either a bug in fschange, or there's a difference in your file between when Splunk hashed it and when you compared it to that hash.