Redis memory usage

Docker Engine General
,
1

I’m running Redis as a docker service and when I login to the machine it was assigned to and run docker stats I get the following:

CONTAINER                           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
redis.1.wsqexigxdtraidyyphpmuunbm   0.08%               250.3MiB / 7.797GiB   3.13%               441MB / 190MB       0B / 175MB          0

Specifically, note the mem usage of 250.3MiB.

When running top, however, I see it’s using less than 200M:

6662 999       20   0  197144 171788   1508 S   0.0  2.1   1:19.12 redis-server

The only process running in the container is Redis itself:

ubuntu@prod-docker-4:~$ docker exec -t -i redis.1.wsqexigxdtraidyyphpmuunbm /bin/bash -c 'ps -efww'
UID        PID  PPID  C STIME TTY          TIME CMD
redis        1     0  3 19:23 ?        00:01:19 redis-server *:6379
root        21     0  0 20:06 ?        00:00:00 ps -efww

This doesn’t appear to be a fixed amount of “container overhead”, but rather ~1.5X ratio. :confused:

Here are some more stats on Docker’s memory stats and the process memory usage:

ps aux --sort -rss, docker stats, ratio
171788, 250.3MiB, 1.46
343828, 514.2MiB, 1.49
433940, 656.1MiB, 1.51
519956, 788.7MiB, 1.52

Any ideas on where the additional memory is going?

System info:

ubuntu@prod-docker-4:~$ uname -a
Linux prod-docker-4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@prod-docker-4:~$ cat /etc/issue.net
Ubuntu 14.04.5 LTS
ubuntu@prod-docker-4:~$ docker version
Client:
 Version:      17.04.0-ce
 API version:  1.28
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Mon Apr  3 17:54:19 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.04.0-ce
 API version:  1.28 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Mon Apr  3 17:54:19 2017
 OS/Arch:      linux/amd64
 Experimental: false

Thanks!

1 Like
2

could connect to redis and run info. it would show the memory usage like below:

# Memory
used_memory:409964064
used_memory_human:390.97M
used_memory_rss:422481920
used_memory_rss_human:402.91M
used_memory_peak:409995904
used_memory_peak_human:391.00M
used_memory_peak_perc:99.99%
used_memory_overhead:126958696
used_memory_startup:487168
used_memory_dataset:283005368
used_memory_dataset_perc:69.11%
total_system_memory:1044770816
total_system_memory_human:996.37M
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.03
mem_allocator:jemalloc-4.0.3
lazyfree_pending_objects:0
3

@roberto : Did you ever find out the problem with this? Currently I am expericing the same issue. I have a docker container memory limit of 500MB, and a redis max-memory of 150 with a maxmemory-policy of allkeys-lru so I would expect to stay well below that 500MB limit. We have however had a few containers reach that and crash.

Redis info memory shows redis not going above the threshold, however a top in the container shows a much larger redis process (I know there is overhead but 3x the size of max-memory) and ctop/docker stats shows something that doesn’t match that either.

I have upgraded to the latest redis, 4.0.7 after trying this on 3.2. My docker version is older at 1.12.1, but I see you got this with 17.04

4

If you are using Redis replication, the replication buffer could take like 512MB memory, and it is not included in Redis memory usage calculation. 2 refs: redis replication buffer, Amazon ElasticCache Node size selection.

If not using Redis replication, not sure if there are some other Redis internal data structure that consumes memory. Probably Redis github issue is a better place to ask.

5

Hey all,

It’s been a while since I’m thinking about this. I’m looking at a service right now and see this with the command docker stats stack_redis.1.xxxxxxxxxxxxxxxxxxxxxxxxx:

CONTAINER                                  CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
stack_redis.1.xxxxxxxxxxxxxxxxxxxxxxxxx   0.08%               23.83MiB / 64MiB    37.23%              145MB / 34.9MB      6.18MB / 6.02GB     3

in top I’m now seeing this:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 5183 docker  20   0   49696  25796   2972 S   0.0  0.3  48:54.01 redis-server

The discrepancy between top and docker stats is within 10% (comparing RES); then again this is a much, much smaller redis instance than the previous one.

Docker info:

aguilarr@prod-docker-5:~$ docker version
Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:42:18 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:40:56 2017
 OS/Arch:      linux/amd64
 Experimental: false