Was just wondering if MapD is supported running within a GPU-enabled Docker container?
Would love to hear if anyone has any experience with this and what, if any, gotchas there are.
Yes, and I’ll actually be writing some more in-depth instructions for how to use and build it this week (you’re the third person to ask in as many days ).
A Dockerfile and bare minimum instructions are in the docker directory: https://github.com/mapd/mapd-core/tree/master/docker . At the moment it’s easiest to use the prebuilt Community Edition, but let me know if you’d like to use your own build (there are a few gotchas related to linking in the dependencies).
I’ll also be pushing prebuilt containers to Docker Hub once the lawyers sign off on how we bundle some of the closed source bits (rendering support, Immerse). There will soon also be a public container with all the build dependencies.
As for some gotchas:
Thanks for the quick response @andrew, this is excellent news! I look forward to reading your instructions once published, but in the mean time I will definitely start playing around with what’s currently in git.
Just wanted to follow up on this that I was able to get MapD running in Docker on AWS. Here are my notes for what I had to do in case it helps someone else.
Steps to set up MapD running in Docker on a fresh P2 instance on AWS. Note that this doesn’t work on Amazon ECS optimized Linux because it does not have systemd, which nvidia-docker RPM requires. So we need to use a different distro and install the ECS agents and what not ourselves. I used the Official Ubuntu 16.04 AMI (ami-835b4efa in us-west-2).
Install NVIDIA drivers
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo sh -c 'echo “deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 /” > /etc/apt/sources.list.d/cuda.list’
sudo apt-get update && sudo apt-get install -y --no-install-recommends linux-headers-generic dkms cuda-drivers
Install docker-ce on Ubuntu
sudo apt-get update
sudo apt-get install -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
""deb [arch=amd64] https://download.docker.com/linux/ubuntu
sudo apt-get update
sudo apt-get install -y docker-ce
Install NVIDIA docker
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb
sudo dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.deb
After reboot test that nvidia-docker works:
sudo nvidia-docker run --rm nvidia/cuda nvidia-smi
On boot with the above AMI, Optimize GPU (P2 instances only). I put this in /etc/rc.local
sudo nvidia-smi -pm 1
sudo nvidia-smi --auto-boost-default=0
sudo nvidia-smi -ac 2505,875
Build MapD Docker container
wget -O mapd-latest-Linux-x86_64.tar.gz
tar xvfz mapd-latest-Linux-x86_64.tar.gz
(copy Dockerfile from https://github.com/mapd/mapd-core/tree/master/docker to where the mapd tar ball is exploded and change the ports exposed to be the server ports)
sudo nvidia-docker build -t mapd/mapd:v3.1.1 .
Run the MapD container
sudo nvidia-docker run -d -p 9091:9091 --name mapd -v /path/to/mapd-storage:/mapd-storage -v /usr/share/glvnd/egl_vendor.d:/usr/share/glvnd/egl_vendor.d mapd/mapd:v3.1.1
Connect to the container
User mapd connected to database mapd
Awesome, thanks for beating me to it & writing this up.
Also, sorry about editing your post slightly, had to remove your personalized download link. I should be able to make the next release available on Docker Hub, which will remove step 4) above.
OOPS! Sorry about that (forgot those were personalized).
We are using containerized mapd in our beta at the moment. Everything is based of this repo which has similar steps to what you wrote above @bploetz. I didn’t do any optimizations on the P2 instances though so I’ll have to look into that.
We also use nvidia-docker-compose to help with automation.