Introduction and Motivation
A beaker notebook allows us to perform complex analyses, visualize results interactively and finally publish or share our notebook with other persons (that can then modify the live code). In this way it becomes the platform in which the various technologies and the various WP encounter themselves and are exposed to the user.
A beaker notebook supports various languages, even in the same notebook (unlike for example plain jupyter), a definite plus for us.
How to use it
The goal is to have a docker image that we customize, and where we put add all our libraries. Furthermore we have to write a server that starts the images and routes the requests to the correct image (to cleanly support multiple users). Our customized docker image is called nomadlab/notebook. And for the time being (till we write the server), we have created a shared notebook.
The shared notebook has the parsed, normalized, and raw data mounted at /parsed, /normalized and /raw-data respectively. Further more, there is a persistent volume mounted at /home/beaker/notebooks. So you can use it to save and share beaker scripts, on the container directly. All the other directories (like /home/beaker) are non-persistent and the data stored on them will disappear once the container is recreated or upgraded.
nomadlab/notebook is our customised docker image for beaker notebook. nomadlab/notebook also contains libraries related to NOMAD that are accessible inside beaker notebook, for example, python library for structural similarity and Scala libraries. Currently it is publicly unavailable but if you have access to nomad-lab-base then you can easily build the docker image and use it locally.
$ sbt notebook/docker
On successful completion of the command, nomadlab/notebook:version should be available locally. You can check it using :
$ docker images
To run the docker image :
$ docker run -p 8801:8801 --name beaker$USER nomadlab/notebook:copyCorrectVersionHere
To run the docker image with data mounted on it: $ docker run -p 8801:8801 -v /nomad/nomadlab/raw-data:/raw-data:ro -v /nomad/nomadlab/parsed:/parsed:ro --name beaker$USER nomadlab/notebook:#copyCorrectVersionHere
Now, you can use beaker as usual at
You can also play around by opening a shell in the container (to install extra stuff and customize it):
docker exec -ti beaker$USER bash
To restart the container. docker start -a beaker$USER
To remove the container (to restart from scratch for example)
docker stop beaker$USER docker rm beaker$USER
You can modify the nomadlab/notebook as per your need, so that you do not have to make changes in docker container each time using bash. All settings are available in notebook module in nomad-lab-base/build.sbt, and modifying it, is straightforward. For example, you can include other NOMAD related python libraries by adding the following in section called new Dockerfile, in notebook module :
add(baseDirectory.value / "../path_inside_nomad-lab-base/library_path/python-modules-path", "/usr/lib/python2.7")
If there is a dependency that you want to be resolved by default (and not when you build nomadlab/notebook), you can contact Ankit (@ankar) and it can pushed to nomadlab/beaker.
nomadlab/notebook is based on nomadlab/beaker. Most probably you don't need to know the details of nomadlab/beaker. But if you are curious then go ahead...
Beaker is a great tool, but the official docker image for beaker is tuned differently then we would like, so we have created our own beaker image based upon the official image (called beakernotebook/beaker). Our beaker notebook, called nomadlab/beaker, doesn't not require a password when you start it and in the future will support an external authentication mechanism. The image is publicly available on docker hub and currently the latest version is 1.6.