I didn’t say it’s safer or unsafer to grant unprivileged users access to the docker.sock. The message is: whoever has access can easily mount /etc/groups into a container and themselves to a group. Adding users to docker group is more convenient and therefore often used in development or homelab environments.
General rule of thumb: do not give untrusted users access to your docker.sock.
This old post of mine might give you an idea: Prohibit access to the inside of the container - #2 by meyay.