Bare-metal to BinderHub

Installation of the BinderHub from bare-metal is fully automatic and reproducible through terraform configuration ran using this Docker container.

The following is intended for neurolibre backend developers, but can be read by anyone interested in our process. It assumes that you have basic knowledge on using the command line on a remote server (bash, ssh authentication..).

The sections Pre-setup and Docker-specific preparations should be done just the first time. Once it is done, you can directly go to the section Spawn a BinderHub instance using Docker.

Pre-setup

You first need to prepare the necessary files that will be used later to install and ssh to the newly spawned BinderHub instance.

We are using git-crypt to encrypt our password files for the whole process, these can be unencrypted with the appropriate gitcrypt-key. For the ssh authentication on the BinderHub server, you have two choices : i) use neurolibre’s key (recommended) or ii) use your own ssh key.

Note

You can request the gitcrypt-key, neurolibre’s ssh key, cloudflare and arbutus API keys to any infrastructure admin if authorized.

Warning

You should never share the aforementioned file to anyone.

  1. Create a folder on your local machine, which is later to be mounted to the Docker container for securely using your keys during spawning a BinderHub instance. Here, we will call it my-keys for convenience:

    cd /home/$USER
mkdir /my-keys

  2. Option (i), use neurolibre’s key (recommended):

    1. Simply copy the public id_rsa.pub and private key id_rsa to /home/$USER/my-keys/

      cp id_rsa* /home/$USER/my-keys/

  3. Option (ii), use your own local key:

    1. Make sure your public key and private are under /home/$USER/.ssh an copy it to /home/$USER/my-keys.

      cp /home/$USER/.ssh/id_rsa* /home/$USER/my-keys/

    2. If not already associated, add your local’s key to your GitHub account:

      • You can check and add new keys on your GitHub settings.

      • Test your ssh connection to your GitHub account by following these steps.

  4. Finally, copy the key gitcrypt-key in /home/$USER/my-keys/.

Docker-specific preparations

You will install a trusted Docker image that will later be used to spawn the BinderHub instance.

  1. Install Docker and log in to the dockerhub with your credentials.

    sudo docker login

  2. Pull the Docker image that encapsulates the barebones environment to spawn a BinderHub instance with our provider (compute canada as of late 2019). You can check the different tags available under our dockerhub user.

    sudo docker pull conpdev/neurolibre-instance:v1.3

Spawn a BinderHub instance using Docker

To achieve this, you will instantiate a container (from the image you just pulled) mounted with specific volumes from your computer. You will be mounting two directories into the container: /my_keys containing the files from Pre-setup, and /instance_name containing the terraform recipe, artifacts and API keys.

Warning

The Docker container that you will run contain sensitive information (i.e. your ssh keys, passwords, etc), so never share it with anyone else. If you need to share information to another developer, share the Dockerfile and/or these instructions.

Note

The Docker image itself has no knowledge of the sensitive files since they are used just at runtime (through entrypoint command).

  1. Place a main.tf file (see Appendix A for details) into a new folder /instance-name, which describes the terraform recipe for spawning a BinderHub instance on the cloud provider. For convenience, we suggest that you use the actual name of the instance (value of the project_name field in main.tf).

    mkdir /home/$USER/instance-name
vim /home/$USER/instance-name/main.tf

Note

If you choose not to copy main.tf file to this directory, you will be asked to fill out one manually during container runtime.

  1. Now you can copy the cloudflare keys_cc.sh and computecanada/arbutus *openrc.sh API keys.

    cp PATH/TO/keys_cc.sh /home/$USER/instance-name/
cp PATH/TO/*openrc.sh /home/$USER/instance-name/

  2. Start the Docker container which is going to spawn the BinderHub instance:

    sudo docker run -v /home/$USER/my_keys:/tmp/.ssh -v /home/$USER/instance-name:/terraform-artifacts -it neurolibre-instance:v1.2

  3. Take a coffee and wait! The instance should be ready in 5~10 minutes.

  4. For security measure, stop and delete the container that you used to span the instance:

    sudo docker stop conpdev/neurolibre-instance:v1.3
sudo docker rm conpdev/neurolibre-instance:v1.3

If you need more information about this docker, check the neurolibre repository.

Appendix A

Here we describe the default terraform recipe that can be used to spawn a BinderHub instance, it is also available online. There are three different modules used by our terraform scripts, all run consecutively and only if the previous one succeeded.

  1. provider populates terraform with the variables related to our cloud provider (compute canada as of late 2019):

    • project_name: name of the instances (will be project_name_master and project_name_nodei)

    • nb_nodes: number of k8s nodes excluding the master node

    • instance_volume_size: main volume size of the instances in GB including the master node

    • ssh_authorized_keys: list of the public ssh keys that will be allowed on the server

    • os_flavor_master: hardware configuration of the k8s master instance in the form c{n_cpus}-{ram}gb-{optionnal_vol_in_gb}

    • os_flavor_node: hardware configuration of the k8s node instances

    • image_name: OS image name used by the instance

    • docker_registry: domain for the Docker registry, if empty it uses Docker.io by default

    • docker_id: user id credential to connect to the Docker registry

    • docker_password: password credential to connect to the Docker registry

Warning

The flavors and image name are not fully customizable and should be set accordingly to the provider’s list. You can check them through openstack API using openstack flavor list && openstack image list or using the horizon dashboard.

  1. dns related to cloudflare DNS configuration:

    • domain: domain name to access your BinderHub environment, it will automatically point to the k8s master floating IP

  2. binderhub specific to binderhub configuration:

    • binder_version: you can check the current BinderHub version releases here

    • TLS_email: this email will be used by Let’s Encrypt to request a TLS certificate

    • TLS_name: TLS certificate name should be the same as the domain but with dashes - instead of points .

    • mem_alloc_gb: Amount of RAM (in GB) used by each user of your BinderHub

    • cpu_alloc: Number of CPU cores (Intel® Xeon® Gold 6130 for compute canada) used by each user of your BinderHub

 1 module "provider" {
 2 source = "git::ssh://git@github.com/neurolibre/terraform-binderhub.git//terraform-modules/providers/openstack"
 3
 4 project_name         = "instance-name"
 5 nb_nodes             = 1
 6 instance_volume_size = 100
 7 ssh_authorized_keys  = ["<redacted>"]
 8 os_flavor_master     = "c4-30gb-83"
 9 os_flavor_node       = "c16-60gb-392"
10 image_name           = "Ubuntu-18.04.3-Bionic-x64-2020-01"
11 is_computecanada     = true
12 docker_registry      = "binder-registry.conp.cloud"
13 docker_id            = "<redacted>"
14 docker_password      = "<redacted>"
15 }
16
17 module "dns" {
18 source = "git::ssh://git@github.com/neurolibre/terraform-binderhub.git//terraform-modules/dns/cloudflare"
19
20 domain    = "instance-name.conp.cloud"
21 public_ip = "${module.provider.public_ip}"
22 }
23
24 module "binderhub" {
25 source = "git::ssh://git@github.com/neurolibre/terraform-binderhub.git//terraform-modules/binderhub"
26
27 ip               = "${module.provider.public_ip}"
28 domain           = "${module.dns.domain}"
29 admin_user       = "${module.provider.admin_user}"
30 binder_version   = "v0.2.0-n121.h6d936d7"
31 TLS_email        = "<redacted>"
32 TLS_name         = "instance-name-conp-cloud"
33 mem_alloc_gb     = 4
34 cpu_alloc        = 1
35 docker_registry  = "${module.provider.docker_registry}"
36 docker_id        = "${module.provider.docker_id}"
37 docker_password  = "${module.provider.docker_password}"
38 }