Hash sum mismatch / Writing more data as expected

I’m using latest Docker for Mac with such Dockerfile (only beginning):

# Base image
FROM ubuntu:16.04

RUN export DEBIAN_FRONTEND=noninteractive

# Update packages list and system
RUN apt-get -y update;
RUN apt-get -y upgrade

# Allow to use add-apt-repository command
RUN apt-get -y install software-properties-common locales poppler-utils

and from a few days I’m getting errors like this:

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/libk5crypto3_1.13.2+dfsg-5ubuntu2_amd64.deb  Hash Sum mismatch

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/k/keyutils/libkeyutils1_1.5.9-8ubuntu1_amd64.deb  Hash Sum mismatch

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/libkrb5-3_1.13.2+dfsg-5ubuntu2_amd64.deb  Hash Sum mismatch

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2_amd64.deb  Writing more data than expected (206672 > 201874) [IP: 91.189.88.152 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

ERROR: Service 'web' failed to build: The command '/bin/sh -c apt-get -y install software-properties-common locales poppler-utils' returned a non-zero code: 100

Previously I was using Docker Toolbox on Windows and got such errors maybe 1 or 2 times in 2 years (or maybe none) and now on Mac I’m getting them all the time and cannot build my images.

What can be reason of this? Should I do something on my Mac or maybe change something in my Dockerfile to make it work?

Just to note, I was also playing with changes like this:

# Base image
FROM ubuntu:16.04

RUN export DEBIAN_FRONTEND=noninteractive

RUN echo 'Acquire::Acquire-by-hash "yes";' >> /etc/apt/apt.conf
RUN echo 'Acquire::CompressionTypes::Order "gz";' >> /etc/apt/apt.conf

# Update packages list and system
RUN apt-get -y update
RUN apt-get -y clean
RUN apt-get -y upgrade
RUN apt-get -y clean
RUN apt-get dist-upgrade

# Allow to use add-apt-repository command
RUN apt-get -y install software-properties-common locales poppler-utils

or

# Base image
FROM ubuntu:16.04

RUN export DEBIAN_FRONTEND=noninteractive

RUN rm -rf /var/lib/apt/lists/partial
RUN echo 'Acquire::By-Hash "yes";' >> /etc/apt/apt.conf
RUN echo 'Acquire::CompressionTypes::Order:: "gz";' >> /etc/apt/apt.conf

# Update packages list and system
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
RUN apt-get -y clean
RUN apt-get -y upgrade
RUN apt-get -y clean
RUN apt-get dist-upgrade

# Allow to use add-apt-repository command
RUN apt-get -y install software-properties-common
RUN apt-get -y install locales poppler-utils

or

# Base image
FROM ubuntu:16.04

RUN export DEBIAN_FRONTEND=noninteractive

RUN rm -rf /var/lib/apt/lists/partial
RUN echo 'Acquire::By-Hash "yes";' >> /etc/apt/apt.conf
RUN echo 'Acquire::CompressionTypes::Order:: "gz";' >> /etc/apt/apt.conf
RUN echo 'Acquire::https::No-Cache "True";' >> /etc/apt/apt.conf
RUN echo 'Acquire::http::No-Cache "True";' >> /etc/apt/apt.conf
RUN echo 'Dir::Cache "";' >> /etc/apt/apt.conf
RUN echo 'Dir::Cache::archives "";' >> /etc/apt/apt.conf

# Update packages list and system
RUN apt-get -y update;
RUN apt-get -y upgrade

# Allow to use add-apt-repository command
RUN apt-get -y install software-properties-common locales poppler-utils

but it didn’t change this. I’ve tested exact same Dockerfile also on Windows and there is no problem in there.

I’ve already:

  • restarted my Mac
  • restarted Docker
  • removed all data
  • reset to factory defaults
  • uninstalled and installed again Docker
  • downgraded from mac49 to 17.12.0-ce-mac46 (21698)

And none of them seemed to helped.

Sometimes I’m also getting error:

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libj/libjpeg-turbo/libjpeg-turbo8_1.4.2-0ubuntu3_amd64.deb 501 Not Implemented [IP: 91.189.88.152 80]

When I used in Dockerfile:

RUN apt-get -y install software-properties-common -o Debug::pkgAcquire::Auth=true

you can really see hash is different:

Get:28 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgnutls30 amd64 3.4.10-4ubuntu1.4 [548 kB]
201 URI Done: http://archive.ubuntu.com/ubuntu/pool/main/g/gnutls28/libgnutls30_3.4.10-4ubuntu1.4_amd64.deb
ReceivedHash:
	- SHA256:01e09e3b1144e14802e9ea4e8865e993bf6c96f3efb17e976a8599eb4ad1ef21
	- SHA1:15ef3dcb2adad376ba8b1c7cf50e20595c140f6e
	- MD5Sum:fc1415e4eb41ad015645cc2bd97be806
	- Checksum-FileSize:43548
ExpectedHash:
	- SHA256:72f68c48b573410f3b51c4f38320886a0ce703a0c922b2ec1470ccae58d68315
	- SHA1:f78e635734ea471f7041443c5f6c7a64584e5bfd
	- MD5Sum:3c26b96fa80970976b76219c7372d5ab
	- Checksum-FileSize:547694

Hello @mnabialek have you found a solution to this issue ? I am having the same Issue with docker for mac 18.03.1-ce on a mac os Sierra.

I have tried all the same steps

restarted my Mac
restarted Docker
removed all data
reset to factory defaults
uninstalled and installed again Docker
downgraded to 17.12.0-ce

@robinsarfati / @mnabialek

I tried many suggestions from Google including setting various headers, removing partials, cleaning apt, and deleting the sources files, however none worked.

I found this article that suggested it might be that you are going through a bad proxy: https://github.com/jenkinsci/docker/issues/543

The solution is to create a file called badproxy with these lines:

Acquire::http::Pipeline-Depth 0;
Acquire::http::No-Cache true;
Acquire::BrokenProxy    true;

then add a line to your Dockerfile to copy it into the apt config folder:

COPY ./badproxy /etc/apt/apt.conf.d/99fixbadproxy

This resolved my issue and the image builds correctly now.

Hope this helps.

3 Likes

This worked for me wayyyy later in 2021 as well

1 Like

thank you mate! It helped me a lot! Almost 6 hours of troubleshooting…