Docker Community Forums

Share and learn in the Docker community.

Docker container is failing to load Julia AdaGram process with large data

dockercon
docker

(Ramidavalapati) #1

I have a Julia program which loads 8 GB of data. It is working fine in my local machine.

But when I try it in docker container it is not loading data and giving bus error. It is working fine with small data like 20 MB in docker container.

Dockerfile

FROM ubuntu:16.04

WORKDIR /julia

RUN apt-get -y update

RUN apt-get -y install unzip

RUN apt-get -y install cmake

RUN apt-get -y install clang

RUN apt-get -y install wget

RUN cd /tmp/

RUN wget "https://julialang.s3.amazonaws.com/bin/linux/x64/0.5/julia-0.5.0-linux-x86_64.tar.gz"

RUN tar -xzvf julia-0.5.0-linux-x86_64.tar.gz

RUN mv julia-3c9d75391c/ ~/julia

ENV PATH="/root/julia/bin:${PATH}"

RUN julia --eval 'Pkg.add("JSON")'

RUN julia --eval 'Pkg.add("HttpServer")'

RUN julia --eval 'Pkg.add("URIParser")'

RUN julia --eval 'Pkg.clone("https://github.com/deep-compute/AdaGram.jl.git")'

RUN julia --eval 'Pkg.build("AdaGram")'

CMD ["julia", "/tmp/adagram_server.jl", "80", "/julia/full.embed"]

docker-compose.yml

version: "3.1"
services:
    julia:
        image: ramidavalapati/julia:v-1
        volumes:
            - /home/ram/adagram_data/40MBfull.embed:/julia/full.embed
        ports:
            - 8080:80
        command: ["sleep", "1h"]

Next I am doing

sudo docker-compose up
sudo docker exec -it $(sudo docker-compose ps -q julia) bash
root@3748d5958f77:/julia# julia
julia> using AdaGram
julia> AdaGram.load_model("/julia/full.embed")

Error

signal (7): Bus error
while loading no file, in expression starting on line 0
macro expansion at ./cartesian.jl:62 [inlined]
macro expansion at ./multidimensional.jl:429 [inlined]
_unsafe_batchsetindex! at ./multidimensional.jl:421
_setindex! at ./multidimensional.jl:370 [inlined]
setindex! at ./abstractarray.jl:832 [inlined]
#9 at /root/.julia/v0.5/AdaGram/src/AdaGram.jl:64
#600 at ./multi.jl:1030
run_work_thunk at ./multi.jl:1001
run_work_thunk at ./multi.jl:1010 [inlined]
#597 at ./event.jl:68
unknown function (ip: 0x7fe1822db16f)
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1942
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1392 [inlined]
start_task at /home/centos/buildbot/slave/package_tarball64/build/src/task.c:253
unknown function (ip: 0xffffffffffffffff)
Allocations: 9661042 (Pool: 9659980; Big: 1062); GC: 16
Bus error (core dumped)

Docker version

Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:42:18 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:40:56 2017
 OS/Arch:      linux/amd64
 Experimental: false

Docker info

Containers: 24
 Running: 0
 Paused: 0
 Stopped: 24
Images: 24
Server Version: 17.09.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: jlkmigmtyjhz6yvi3zuvkobu7
 Is Manager: true
 ClusterID: rqt03ulgvvnym235m1qm8vd17
 Managers: 4
 Nodes: 15
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: X.X.X.X
  Manager Addresses:
   X.X.X.X:2377
   X.X.X.X:2377
   X.X.X.X:2377
   X.X.X.X:2377
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
 runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
 init version: 949e6fa
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.10.0-35-generic
 Operating System: Ubuntu 16.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 251.8GiB
 Name: ram
 ID: 3OGG:275C:Q3IW:O4HX:DPLP:DPI3:5TIT:AG5J:EDMK:7NK3:L4UZ:BTQH
 Docker Root Dir: /var/lib/docker
 Debug Mode (client): false
 Debug Mode (server): false
 Username: ramidavalapati
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 
 WARNING: No swap limit support

Thanks in advance for any kind of help

link to 40MBfull.embed file