Notes on Docker

Docker is a fascinating concept that could be potentially useful in many ways, especially in Data science, and making reproducible workflows / environments. There are several articles which have great introductions and examples of using docker in data science This is an evolving summary of my exploration with Docker. It should prove to be a…

Tips for building a dev-env with docker (Muffin Research Labs)

There’s a few gotchas when using docker to run your development environment. This post provides tips on overcoming the biggest hurdles you might face.

"Having used docker-compose (née fig) on the marketplace development environment. We learnt a lot about what did and didn’t work. Two of the biggest issues we hit were: Dependencies and dep updates Front-end tools the best way to avoid deps becoming a problem, is to use the hub to build your docker images from a…

Init: custom docker container for Data Science

I have enabled the TCP on port 8787 via UFW, and am running a docker container of the image ‘rocker/tidyverse’, which is providing an Rstudio IDE online to play with. This was relatively straightforward, starting with installing docker on my debian machine, and then pulling in the rocker/tidyverse container, as well as the rocker/shiny-verse container.…

Easy to resize the swap partition in Linode

In the ‘new’ interface for Linode manager – the advanced tab contains the list of disks. When the Linode is fully powered down, it is possible to resize the partitions as desired. My earlier notes indicate that atleats 2.5GB of swap space was required to install Rstudio (without docker) in the past, on a machine…

Installing UMAP took up nearly 1-1.5 GB of swap

Atleast ~500MB of RAM and around 1-1.5GB of swap was used while installing the UMAP package into rocker/tidyverse. It also took ~10 minutes atleast, and started with installing the reticulate, Rspectra and Rcppeigen (?) packages first, after which UMAP was installed. It would certainly save time and head-banging to have umap already installed into a…