3:30 p.m.
What process is going to start this user container? Is this any more
secure then just running as root with the --promethious user?
I would guess it is slightly more, since root inside of the container is
not root on the host, but if you set the --no-new-privs on the container
then, the container process can never become root.
On 1/28/20 2:46 PM, karl(a)touchpoint.io wrote:
> No. Rootless users are usually normal users, and should have
home directories at least, this is where the containers and images are stored.
Of course. I create system users with the `--create-home` flag for this reason. Primary
benefit to system level users is that they come with easily distinguishable UIDs and no
password expiration policy. It does appear that `useradd` will **not** automatically add
a subg/uid entry for system level users, though. The `man` page does not indicate a way to
change this. Hence me doing it manually in my OP. Creating a regular user does
automatically add the subg/uids though.
> Usually this is just the homedir.
Sure. That's a perfectly logical place to store the container images... But for
intuitiveness, this is a terrible place to store files that the container may persist to
the host's disk.
If a container is expected to produce a log file that will be scraped by another process
on the host, then mounting the location the container process expects to write out
it's log file to the host path `/var/log/$service/{app,error}.log` makes infinitely
more sense than sequestering it away inside `/home/$service/logs/{app,error}.log`.
Additionally, now I have to give the `$log_ingester` process read/write (for rotation)
access to `/home/$service/logs/*`.
regardless, **any** location that is to be mapped into the container for persistent
storage requires xx4/6/7 level permissions **or** pre-existing knowledge of which sub id
will be used to do the actual file writes.
This is acceptable for some things like logs which are known not to contain any sensitive
info, but absolutely unacceptable for other workloads, like a database.
I'll have more on this below...
> If you are running the rootless container as root then the UID should be the uid of
the user.
This may be where understanding is broken, then. I am **not** running the container as
root. I thought i made that clear in the OP.
To be explicit: I am running the `prometheus` container as the `prometheus` user on the
host. Within the container, the `prometheus` container image is built so that the process
within the container runs as `nobody`.
Should i be using the hosts root user to invoke `podman run` and passing in the
`--user=prometheus` flag, then?
Should i be building my own prometheus container image so that the service runs as root?
so to recap, then:
- create regular users **or** manually create a home dir + subg/uid range for a system
user
- there is no reliable or automatic way to determine which UID should "own" any
files/dirs that are to be mapped into a container
- any file/dir that is to be mapped into the container will need to be world read/write
initially.
- once the container has successfully started up and written to (host) disk, the sub-ids
that should "own" files can be known and the world read/write permissions can be
revoked and the user/group ID can be properly set on host.
My "process" for determining which user/group should "own" the files
mounted into the container:
##
# I have made a place for the configuration file to live: /tmp/prom.
# The host user `prom` owns this directory and the file in it
##
```
root@com:/tmp/prom# ls -lah
total 12K
drwxr-x--- 2 prom prom 4.0K Jan 28 19:06 .
drwxrwxrwt 17 root root 4.0K Jan 28 19:05 ..
-rw-r----- 1 prom prom 551 Jan 27 19:54 prometheus.yml
```
##
# The host user prom has a proper subgid and subuid map
# The map range is big enough as it allows for `65536` which is
# just bigger than the `65534` i should need for this container
##
```
root@com:/tmp/prom# cat /etc/subuid | grep prom
prom:624288:65536
root@com:/tmp/prom# cat /etc/subgid | grep prom
prom:624288:65536
```
##
# So the host user `prom` should be able to start up the prometheus container
# and read the files that it owns.
##
```
prom@com:/tmp/prom$ whoami
prom
prom@com:/tmp/prom$ cat /tmp/prom/prometheus.yml | wc -l
27
```
##
# But then the container can't actually read the file
##
```
prom@com:/tmp/prom$ podman run -p 9090:9090 -v /tmp/prom:/etc/prometheus prom/prometheus
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:294 msg="no time or size
retention was set so using the default time retention" duration=15d
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:330 msg="Starting
Prometheus" version="(version=2.15.2, branch=HEAD,
revision=d9613e5c466c6e9de548c4dae1b9aabf9aaf7c57)"
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:331
build_context="(go=go1.13.5, user=root@688433cf4ff7, date=20200106-14:50:51)"
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:332 host_details="(Linux
4.15.0-1057-aws #59-Ubuntu SMP Wed Dec 4 10:02:00 UTC 2019 x86_64 dfc11d48288e
(none))"
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:333 fd_limits="(soft=1024,
hard=1024)"
level=info ts=2020-01-28T19:14:31.801Z caller=main.go:334
vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2020-01-28T19:14:31.802Z caller=main.go:648 msg="Starting TSDB
..."
level=info ts=2020-01-28T19:14:31.802Z caller=web.go:506 component=web msg="Start
listening for connections" address=0.0.0.0:9090
level=info ts=2020-01-28T19:14:31.810Z caller=head.go:584 component=tsdb
msg="replaying WAL, this may take awhile"
level=info ts=2020-01-28T19:14:31.810Z caller=head.go:632 component=tsdb msg="WAL
segment loaded" segment=0 maxSegment=0
level=info ts=2020-01-28T19:14:31.811Z caller=main.go:663 fs_type=EXT4_SUPER_MAGIC
level=info ts=2020-01-28T19:14:31.811Z caller=main.go:664 msg="TSDB started"
level=info ts=2020-01-28T19:14:31.811Z caller=main.go:734 msg="Loading configuration
file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-01-28T19:14:31.811Z caller=main.go:517 msg="Stopping scrape
discovery manager..."
level=info ts=2020-01-28T19:14:31.812Z caller=main.go:531 msg="Stopping notify
discovery manager..."
level=info ts=2020-01-28T19:14:31.812Z caller=main.go:553 msg="Stopping scrape
manager..."
level=info ts=2020-01-28T19:14:31.812Z caller=main.go:513 msg="Scrape discovery
manager stopped"
level=info ts=2020-01-28T19:14:31.812Z caller=manager.go:814 component="rule
manager" msg="Stopping rule manager..."
level=info ts=2020-01-28T19:14:31.812Z caller=manager.go:820 component="rule
manager" msg="Rule manager stopped"
level=info ts=2020-01-28T19:14:31.812Z caller=main.go:527 msg="Notify discovery
manager stopped"
level=info ts=2020-01-28T19:14:31.812Z caller=main.go:547 msg="Scrape manager
stopped"
level=info ts=2020-01-28T19:14:31.813Z caller=notifier.go:598 component=notifier
msg="Stopping notification manager..."
level=info ts=2020-01-28T19:14:31.814Z caller=main.go:718 msg="Notifier manager
stopped"
level=error ts=2020-01-28T19:14:31.814Z caller=main.go:727 err="error loading config
from \"/etc/prometheus/prometheus.yml\": couldn't load configuration
(--config.file=\"/etc/prometheus/prometheus.yml\"): open
/etc/prometheus/prometheus.yml: permission denied"
```
##
# So i then make the yaml file full 777 and try again.
# Additionally, i'd like the data that this container will create/store to persist to
(host) disk.
# I'll make a folder on the host for this data: /tmp/prom/data
##
```
root@com:/tmp/prom# chmod 777 *
prom@com:/tmp/prom$ mkdir data
prom@com:/tmp/prom$ chmod 0750 data/
prom@com:/tmp/prom$ ls -lah
total 16K
drwxr-x--- 3 prom prom 4.0K Jan 28 19:20 .
drwxrwxrwt 17 root root 4.0K Jan 28 19:21 ..
drwxr-x--- 2 prom prom 4.0K Jan 28 19:20 data
-rwxrwxrwx 1 prom prom 551 Jan 27 19:54 prometheus.yml
```
##
# The container is configured to store all of it's data in the `/prometheus` path
# so there are two mount arguments fed into the podman run command
##
```
prom@com:/tmp/prom$ podman run -p 9090:9090 -v /tmp/prom:/etc/prometheus -v
/tmp/prom/data:/prometheus prom/prometheus
Error: container_linux.go:346: starting container process caused "chdir to cwd
(\"/prometheus\") set in config.json failed: permission denied": OCI
runtime permission denied error
```
###
# No such luck!
# Make /tmp/prom/data 777 and try again...
##
```
root@com:/tmp/prom# chmod 777 data/
root@com:/tmp/prom# ls -lah
total 16K
drwxr-x--- 3 prom prom 4.0K Jan 28 19:20 .
drwxrwxrwt 17 root root 4.0K Jan 28 19:21 ..
drwxrwxrwx 2 prom prom 4.0K Jan 28 19:20 data
-rwxrwxrwx 1 prom prom 551 Jan 27 19:54 prometheus.yml
prom@com:/tmp/prom$ podman run -p 9090:9090 -v /tmp/prom:/etc/prometheus -v
/tmp/prom/data:/prometheus prom/prometheus
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:294 msg="no time or size
retention was set so using the default time retention" duration=15d
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:330 msg="Starting
Prometheus" version="(version=2.15.2, branch=HEAD,
revision=d9613e5c466c6e9de548c4dae1b9aabf9aaf7c57)"
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:331
build_context="(go=go1.13.5, user=root@688433cf4ff7, date=20200106-14:50:51)"
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:332 host_details="(Linux
4.15.0-1057-aws #59-Ubuntu SMP Wed Dec 4 10:02:00 UTC 2019 x86_64 eacd0e5fbbe2
(none))"
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:333 fd_limits="(soft=1024,
hard=1024)"
level=info ts=2020-01-28T19:26:44.143Z caller=main.go:334
vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2020-01-28T19:26:44.144Z caller=main.go:648 msg="Starting TSDB
..."
level=info ts=2020-01-28T19:26:44.144Z caller=web.go:506 component=web msg="Start
listening for connections" address=0.0.0.0:9090
level=info ts=2020-01-28T19:26:44.151Z caller=head.go:584 component=tsdb
msg="replaying WAL, this may take awhile"
level=info ts=2020-01-28T19:26:44.151Z caller=head.go:632 component=tsdb msg="WAL
segment loaded" segment=0 maxSegment=0
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:663 fs_type=EXT4_SUPER_MAGIC
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:664 msg="TSDB started"
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:734 msg="Loading configuration
file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:517 msg="Stopping scrape
discovery manager..."
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:531 msg="Stopping notify
discovery manager..."
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:553 msg="Stopping scrape
manager..."
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:527 msg="Notify discovery
manager stopped"
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:513 msg="Scrape discovery
manager stopped"
level=info ts=2020-01-28T19:26:44.153Z caller=manager.go:814 component="rule
manager" msg="Stopping rule manager..."
level=info ts=2020-01-28T19:26:44.153Z caller=manager.go:820 component="rule
manager" msg="Rule manager stopped"
level=info ts=2020-01-28T19:26:44.153Z caller=main.go:547 msg="Scrape manager
stopped"
level=info ts=2020-01-28T19:26:44.155Z caller=notifier.go:598 component=notifier
msg="Stopping notification manager..."
level=info ts=2020-01-28T19:26:44.155Z caller=main.go:718 msg="Notifier manager
stopped"
level=error ts=2020-01-28T19:26:44.155Z caller=main.go:727 err="error loading config
from \"/etc/prometheus/prometheus.yml\": couldn't load configuration
(--config.file=\"/etc/prometheus/prometheus.yml\"): open
/etc/prometheus/prometheus.yml: permission denied"
prom@com:/tmp/prom$
```
##
# Still, no such luck.
# But what if the folder *containing* _all_ the prometheus data was WORLD READ/WRITE...
##
```
root@com:/tmp# chmod 777 prom/
root@com:/tmp# ls -lah prom/
total 16K
drwxrwxrwx 3 prom prom 4.0K Jan 28 19:20 .
drwxrwxrwt 17 root root 4.0K Jan 28 19:21 ..
drwxrwxrwx 3 prom prom 4.0K Jan 28 19:26 data
-rwxrwxrwx 1 prom prom 551 Jan 27 19:54 prometheus.yml
prom@com:/tmp/prom$ podman run -p 9090:9090 -v /tmp/prom:/etc/prometheus -v
/tmp/prom/data:/prometheus prom/prometheus
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:294 msg="no time or size
retention was set so using the default time retention" duration=15d
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:330 msg="Starting
Prometheus" version="(version=2.15.2, branch=HEAD,
revision=d9613e5c466c6e9de548c4dae1b9aabf9aaf7c57)"
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:331
build_context="(go=go1.13.5, user=root@688433cf4ff7, date=20200106-14:50:51)"
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:332 host_details="(Linux
4.15.0-1057-aws #59-Ubuntu SMP Wed Dec 4 10:02:00 UTC 2019 x86_64 aeeb1ee9e821
(none))"
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:333 fd_limits="(soft=1024,
hard=1024)"
level=info ts=2020-01-28T19:35:14.704Z caller=main.go:334
vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2020-01-28T19:35:14.706Z caller=main.go:648 msg="Starting TSDB
..."
level=info ts=2020-01-28T19:35:14.706Z caller=web.go:506 component=web msg="Start
listening for connections" address=0.0.0.0:9090
level=info ts=2020-01-28T19:35:14.709Z caller=head.go:584 component=tsdb
msg="replaying WAL, this may take awhile"
level=info ts=2020-01-28T19:35:14.710Z caller=head.go:632 component=tsdb msg="WAL
segment loaded" segment=0 maxSegment=1
level=info ts=2020-01-28T19:35:14.711Z caller=head.go:632 component=tsdb msg="WAL
segment loaded" segment=1 maxSegment=1
level=info ts=2020-01-28T19:35:14.713Z caller=main.go:663 fs_type=EXT4_SUPER_MAGIC
level=info ts=2020-01-28T19:35:14.713Z caller=main.go:664 msg="TSDB started"
level=info ts=2020-01-28T19:35:14.713Z caller=main.go:734 msg="Loading configuration
file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-01-28T19:35:14.715Z caller=main.go:762 msg="Completed loading of
configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-01-28T19:35:14.715Z caller=main.go:617 msg="Server is ready to
receive web requests."
```
##
# Oh hey! That worked!
# Taking a look at the UID that created the data files...
##
```
root@com:/tmp/prom/data# ls
lock queries.active wal
root@com:/tmp/prom/data# ls -lah
total 16K
drwxrwxrwx 3 prom prom 4.0K Jan 28 19:26 .
drwxrwxrwx 3 prom prom 4.0K Jan 28 19:20 ..
-rw-r--r-- 1 689821 689821 0 Jan 28 19:26 lock
-rw-r--r-- 1 689821 689821 20K Jan 28 19:35 queries.active
drwxr-xr-x 2 689821 689821 4.0K Jan 28 19:35 wal
```
_______________________________________________
Podman mailing list -- podman(a)lists.podman.io
To unsubscribe send an email to podman-leave(a)lists.podman.io