Hello,
I have been obligated to use a linux server for my deployments with a very old version: “CentOS Linux release 7.9.2009 (Core)”
In this server I have the different docker containers that I need running, about two weeks ago, the postgresql service container had a forced and unexpected shut down, producing that since this day it does not start and being in constant loop. The rest of services that were running are still running without problems.
My problem is that all the deployment was automated by gitlab and now a couple of months ago we have capped that connection (which requires vpn) and I can not connect from ip outside Spain, leaving all the automation that had for the deployment inoperative.
I have tried to start the container and the whole service again using the image on the server, but the same error persists.
The log provided by the container inside the loop before stopping and restarting is the following:
atabase_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.281 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.281 UTC [1] LOG: listening on IPv6 address “::”, port 5432
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.286 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.311 UTC [24] LOG: database system was shut down at 2024-04-05 17:04:52 UTC
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.311 UTC [24] LOG: invalid primary checkpoint record
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.311 UTC [24] PANIC: could not locate a valid checkpoint record
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.645 UTC [1] LOG: startup process (PID 24) was terminated by signal 6: Aborted
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.645 UTC [1] LOG: aborting startup due to startup process failure
database_postgresql.1.afgh18et5xe1@apd31lv | 2024-04-22 11:30:13.667 UTC [1] LOG: database system is shut down
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.323 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.323 UTC [1] LOG: listening on IPv6 address “::”, port 5432
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.327 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.350 UTC [24] LOG: database system was shut down at 2024-04-05 17:04:52 UTC
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.351 UTC [24] LOG: invalid primary checkpoint record
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.351 UTC [24] PANIC: could not locate a valid checkpoint record
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.615 UTC [1] LOG: startup process (PID 24) was terminated by signal 6: Aborted
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.615 UTC [1] LOG: aborting startup due to startup process failure
database_postgresql.1.kly3ylmdy2op@apd31lv | 2024-04-22 11:27:09.644 UTC [1] LOG: database system is shut down
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.818 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.818 UTC [1] LOG: listening on IPv6 address “::”, port 5432
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.823 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.845 UTC [24] LOG: database system was shut down at 2024-04-05 17:04:52 UTC
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.845 UTC [24] LOG: invalid primary checkpoint record
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:11.845 UTC [24] PANIC: could not locate a valid checkpoint record
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:12.124 UTC [1] LOG: startup process (PID 24) was terminated by signal 6: Aborted
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:12.124 UTC [1] LOG: aborting startup due to startup process failure
database_postgresql.1.6t92mb46czbw@apd31lv | 2024-04-22 11:29:12.152 UTC [1] LOG: database system is shut down
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.560 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.560 UTC [1] LOG: listening on IPv6 address “::”, port 5432
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.564 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.584 UTC [24] LOG: database system was shut down at 2024-04-05 17:04:52 UTC
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.584 UTC [24] LOG: invalid primary checkpoint record
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.584 UTC [24] PANIC: could not locate a valid checkpoint record
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.857 UTC [1] LOG: startup process (PID 24) was terminated by signal 6: Aborted
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.857 UTC [1] LOG: aborting startup due to startup process failure
database_postgresql.1.jjtbzgmaqe5c@apd31lv | 2024-04-22 11:28:10.882 UTC [1] LOG: database system is shut down
I don’t know what to do or where to look, since I don’t have access to certain log files of the docker swarm or to see if the file system might be corrupted.
I appreciate any feedback or suggestions on how to get the service running again.
Thank you very much
Regards