Why does inputs.conf change look fine outside docker, but look wrong inside docker?

Hi folks.

I had this working, but it no longer does, and I somewhat-suspect a docker volume problem as well as somewhat-suspect a permissions issue, and also somewhat suspect an OS upgrade. But I really don’t know what the problem is.

Inside the splunk container, I see:

[root@splunk splunk]# cat /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf
[http]
disabled = 0

[[http://splunk_hec_token](http://splunk_hec_token/)]
disabled = 0
token = really-big-token-thingie

Which is really not what I want.

And outside the splunk container (on the MacOS side), I see:

$ cat splunk-files/opt-splunk-etc-apps-splunk_httpinput-local/inputs.conf
cmd output started 2022 Mon May 02 04:19:43 PM PDT
[http]
disabled = 0

[[http://splunk_hec_token](http://splunk_hec_token/)]
disabled = 0
token = really-big-token-thingie
index = dev_game-publishing

That is what I want.

In my docker-compose, I have (among other things) :

volumes:

- ./splunk-files/opt-splunk-etc-apps-splunk_httpinput-local/ /opt/splunk/etc/apps/splunk_httpinput/local/

(That long volume line is all-one-line. It may or may not be wrapping when you view it, though it is wrapping in this editor)

I tried both setting up a volume for the entire directory, as well as just that one file. I’m hearing that doing an entire directory tends to be more reliable, but both failed the same way.

The directory containing the file is owned by splunk and has restrictive permissions:

[ansible@splunk splunk]$ cat /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf
cat: /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf: Permission denied
[ansible@splunk splunk]$ ls -l /opt/splunk/etc/apps/splunk_httpinput/
total 12
drwxr-xr-x 2 splunk splunk 4096 Jan 15 03:31 default
drwx------ 2 splunk splunk 4096 May 2 22:14 local
drwx------ 2 splunk splunk 4096 May 2 22:14 metadata

Which explains why the ansible user can’t cat it. But is ansible painting itself into a corner and preventing itself from making all the changes I need?

I also upgraded from MacOS 11.x to 12.3 in between when this was working, and when it stopped. I don’t know if that’s related or not.

How can I get the desired content inside the container, automatically?

Thanks for any and all suggestions!

Does really-big-token-thingie include any special characters that may confuse the terminal output?

I added Markdown fenced code blocks to your raw post. Is this indeed what it should show?

[[http://splunk_hec_token](http://splunk_hec_token/)]

If it reads the following, then please use the pencil icon to edit your post:

[http://splunk_hec_token]

It turned out my volume spec was incorrect.

It had a space, where it should’ve had a colon.

Why docker-compose didn’t complain about it, I don’t know.

If you don’t have a colon in the line, it means you define an anonymous volume not a bind mount. Filenames can contain spaces, so I guess it just created an anonymous volume mounting it to that strange destination where one of the folders was actually a “space”

1 Like

Nice. And then /opt/splunk/etc/apps/splunk_httpinput/local/ not being mapped/hidden by any volume, you may have been looking at an old version of inputs.conf that somehow is in the container in that very folder.