MariaDB not starting after update

General Discussions General
1

I am getting the following errors from my log. I only updated the image and restarted. Could a recent change in the image have caused these errors? My host has plenty of space, so I’m not sure why I’m getting the out of space error for ./ibtmp1

The issue occurs when updating from mariadb 10.2.9 to 10.2.10.

db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Uses event mutexes
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Compressed tables use zlib 1.2.8
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Using Linux native AIO
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Number of pools: 1
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Using SSE2 crc32 instructions
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Completed initialization of buffer pool
db_1           | 2017-11-07 21:11:32 140191215482624 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
db_1           | 2017-11-07 21:11:32 140191958161280 [Note] InnoDB: Highest supported file format is Barracuda.
db_1           | 2017-11-07 21:11:33 140191958161280 [Note] InnoDB: 128 out of 128 rollback segments are active.
db_1           | 2017-11-07 21:11:33 140191958161280 [Note] InnoDB: Creating shared tablespace for temporary tables
db_1           | 2017-11-07 21:11:33 140191958161280 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
db_1           | 2017-11-07 21:11:33 140191958161280 [ERROR] InnoDB: preallocating 12582912 bytes for file ./ibtmp1 failed with error 22
db_1           | 2017-11-07 21:11:33 140191958161280 [ERROR] InnoDB: Could not set the file size of './ibtmp1'. Probably out of disk space
db_1           | 2017-11-07 21:11:33 140191958161280 [ERROR] InnoDB: Unable to create the shared innodb_temporary
db_1           | 2017-11-07 21:11:33 140191958161280 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
db_1           | 2017-11-07 21:11:38 140191958161280 [Note] InnoDB: Starting shutdown...
db_1           | 2017-11-07 21:11:38 140191958161280 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
db_1           | 2017-11-07 21:11:38 140191958161280 [ERROR] Plugin 'InnoDB' init function returned error.
db_1           | 2017-11-07 21:11:38 140191958161280 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
db_1           | 2017-11-07 21:11:38 140191958161280 [Note] Plugin 'FEEDBACK' is disabled.
db_1           | 2017-11-07 21:11:38 140191958161280 [ERROR] Unknown/unsupported storage engine: InnoDB
db_1           | 2017-11-07 21:11:38 140191958161280 [ERROR] Aborting
2

The upgrade from 10.2.9 to 10.2.10 alone does not seem to cause the error.
I just tried the following with no problems:

docker run -d --name maria10.2.9 \
-v $PWD/data:/var/lib/mysql \
-v $PWD/dumps:/docker-entrypoint-initdb.d \
-e MYSQL_ROOT_PASSWORD=my-secret-pw \
mariadb:10.2.9
...
docker stop maria10.2.9

docker run --name maria10.2.10 \
-v $PWD/data:/var/lib/mysql \
mariadb:10.2.10

$PWD/dumps contains a dump of a teamcity server database. It should contain plenty of complexity.

Could you give a bit more information on the specifics of your upgrade?
Operating system?
Free disk space?
Accumulated size of tablespaces?
Does your system go back to normal operation when you downgrade to 10.2.9 again?
Any custom mysql configurations?

3

Thank you for looking at my issue.

Could you give a bit more information on the specifics of your upgrade?

I’m using Docker Compose wrapped in a little bit of bash scripting. I’m essentially running docker-compose pull, docker-compose build --pull and then docker-compose up -d --force-recreate.

My docker-compose.yml version 1 and the database section is:

db:
  image: mariadb
  environment:
    MYSQL_ROOT_PASSWORD: xxxxxx
    MYSQL_USER: app_user
    MYSQL_PASSWORD: xxxxxx
    MYSQL_DATABASE: app
    TERM: dumb
  log_driver: "journald"
  restart: always

Operating system?

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

Free disk space?

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            352M     0  352M   0% /dev
tmpfs            75M  1.1M   73M   2% /run
/dev/vda1        15G  9.6G  4.4G  69% /
tmpfs           371M     0  371M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           371M     0  371M   0% /sys/fs/cgroup
tmpfs            75M     0   75M   0% /run/user/1001

Accumulated size of tablespaces?

I’m not sure how to see this.

Does your system go back to normal operation when you downgrade to 10.2.9 again?

Yes it does. I just need to set mariadb:10.2.9 in the docker-compose.yml file.

Any custom mysql configurations?

I am not using any custom configurations, however the database has been running for over a year. It is serving the data for a Drupal 7 website configured to use InnoDB tables I think.