I have created the container rjurney/agile_data_science for running the book Agile Data Science 2.0’s examples. It has mongo, elasticsearch, hadoop, spark, etc. and they all work together. I am using the container to run Spark in local mode, and we process 5GB or so of data that has intermediate output in the tens of GBs.
The container runs out of disk space as soon as any data processing is done. How can I add disk space for this container?
I have searched the forums, and there is no answer. Please help, I’m stuck!
number is in MiB so 150G should be 153600:
$ echo 153600 > com.docker.driver.amd64-linux/disk/size
$ git add com.docker.driver.amd64-linux/disk/size
$ git commit -s -m ‘New target disk size’
and then
rm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
and restart docker? Unfortunately there’s no UI support for this – we’re still thinking about the best way to present it.