Virtual machines - Getting started

../_images/IGRIDA_VMs.png

Create an interactive job

user@igrida-oar-frontend:~$ oarsub -I -l core=2,walltime=1 -p "virt='YES'"

Load Veertuosa modulefile

user@igrida04-06:~$ module load veertuosa/0.0.1

Launch an Ubuntu 16.04 virtual machine

This command will launch an ephemeral virtual machine with all reserved resources (2 core and all available memory associated to this cores).

user@igrida04-06:~$ veertuosa_launch --name myvm --image /soft/igrida/vms/images/ubuntu-16.04-x86_64.qcow2

Open a shell on the virtual machine

user@igrida04-06:~$ ssh-vm myvm
root@veertuosa:~# nproc
2
root@veertuosa:~# free -m
          total        used        free      shared  buff/cache   available
Mem:           1177          51         761           2         364         972
Swap:           511           0         511

Read/Write access to your datas from virtual machines

Mount points :

Host (compute node) Virtual machine
/udd/user_login /mnt/home
/temp_dd/igrida-fs1/ /mnt/temp_dd/igrida-fs1
/srv/local /mnt/srv/local
/nfs /mnt/nfs

Note

Even if your are root on the VM, all read/write access on /mnt/temp_dd/igrida-fs1 and /mnt/home using your login privileges.

Install packages

root@veertuosa:~# apt update && apt install -y htop
root@veertuosa:~# htop

Warning

All these system modifications are ephemeral. You will see next how to customize an image.

Available images

Description Size Path
Ubuntu 16.04 amd64 2.1G /soft/igrida/vms/images/ubuntu-16.04-x86_64.qcow2
Ubuntu 17.04 amd64 1.9G /soft/igrida/vms/images/ubuntu-17.04-x86_64.qcow2
Debian 8.7 amd64 1.9G /soft/igrida/vms/images/debian-8.7-x86_64.qcow2
Debian 9.0.0 amd64 2.3G /soft/igrida/vms/images/debian-9.0.0-x86_64.qcow2
Alpine Linux 3.4.3 amd64 + Docker 508G /soft/igrida/vms/images/alpine-3.4.3-docker-x86_64.qcow2

Customize an image

user@igrida-oar-frontend:~$ oarsub -I -l core=2,walltime=1 -p "virt='YES'"
user@igrida04-06:~$ module load veertuosa/0.0.1

Copy base image

user@igrida04-06:~$ mkdir -p /temp_dd/igrida-fs1/user/ffmpeg/images
user@igrida04-06:~$ cp /soft/igrida/vms/images/alpine-3.4.3-docker-x86_64.qcow2 /temp_dd/igrida-fs1/user/ffmpeg/images/docker-ffmpeg.qcow2

Launch virtual machine with persistent mode

user@igrida04-06:~$ veertuosa_launch --name myvm --image /temp_dd/igrida-fs1/user/ffmpeg/images/docker-ffmpeg.qcow2 --persistent
user@igrida04-06:~$ ssh-vm myvm

Customize

alpine-343:~# docker pull jrottenberg/ffmpeg

Shutdown the VM

alpine-343:~# /sbin/poweroff

Note

With Ubuntu image, uses command shutdown -h now.

Usages samples

Parametric array jobs with docker containers

We will create a job to convert video with different bitrate with ffmpeg (uses image created below).

  • Download a sample input video :
user@igrida-oar-frontend:~$ mkdir -p /temp_dd/igrida-fs1/user/ffmpeg/{images,output_files,input_files,oar_output}
user@igrida-oar-frontend:~$ cd /temp_dd/igrida-fs1/user/ffmpeg/input_files/
user@igrida-oar-frontend:~$ wget http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov
user@igrida-oar-frontend:~$ cd
  • Create a params file ~/ffmpeg/params.txt:
64
128
256
512
1024
2048
4096
8192
  • Create a bash script for OAR passive jobs ~/ffmpeg/oar_script.sh:
#!/bin/bash
#OAR -l cpu=1/core=4,walltime=0:30:00
#OAR -p virt='YES'
#OAR --array-param-file ~user/ffmpeg/params.txt
#OAR -O /temp_dd/igrida-fs1/user/ffmpeg/oar_output/job.%jobid%.output
#OAR -E /temp_dd/igrida-fs1/user/ffmpeg/oar_output/job.%jobid%.error

. /etc/profile.d/modules.sh

set -x

module load veertuosa/0.0.1

VM_NAME=vm_${OAR_JOBID}

veertuosa_launch --name ${VM_NAME} --image /temp_dd/igrida-fs1/user/ffmpeg/images/docker-ffmpeg.qcow2

FFMPEG_BITRATE=$*
FFMPEG_INPUT_FILE="/tmp/ffmpeg/input_files/big_buck_bunny_1080p_h264.mov"
FFMPEG_OUTPUT_FILE="/tmp/ffmpeg/output_files/result_${FFMPEG_BITRATE}.mov"

VM_CMD="docker run -v /mnt/temp_dd/igrida-fs1/user/ffmpeg:/tmp/ffmpeg jrottenberg/ffmpeg -stats -i ${FFMPEG_INPUT_FILE} -b:v ${FFMPEG_BITRATE}k ${FFMPEG_OUTPUT_FILE}"

ssh-vm $VM_NAME "$VM_CMD"
  • Add execution rights :
user@igrida-oar-frontend:~$ chmod u+x ./ffmpeg/oar_script.sh
  • Launch parametric job array :
user@igrida-oar-frontend:~$ oarsub -S ./ffmpeg/oar_script.sh
[ADMISSION RULE] Modify resource description with type constraints
[ADMISSION RULE] Add property max_walltime >= 30 minutes.Simple array job submission is used
OAR_JOB_ID=869088
OAR_JOB_ID=869089
OAR_JOB_ID=869090
OAR_JOB_ID=869091
OAR_JOB_ID=869092
OAR_JOB_ID=869093
OAR_JOB_ID=869094
OAR_JOB_ID=869095
OAR_ARRAY_ID=869088
user@igrida-oar-frontend:~$ oarstat -u
Job id    S User     Duration   System message
--------- - -------- ---------- ------------------------------------------------
869088    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869089    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869090    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869091    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869092    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869093    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869094    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
869095    W user    0:00:00 R=4,W=0:30:0,J=B (Karma=0.003)
  • When all jobs are terminated, checks the result :
user@igrida-oar-frontend:~$ ls -lh /temp_dd/igrida-fs1/user/ffmpeg/output_files/
total 1.4G
-rw-r--r-- 1 user myriads  97M Jan 17 15:34 result_1024.mov
-rw-r--r-- 1 user myriads  34M Jan 17 15:32 result_128.mov
-rw-r--r-- 1 user myriads 169M Jan 17 15:36 result_2048.mov
-rw-r--r-- 1 user myriads  43M Jan 17 15:34 result_256.mov
-rw-r--r-- 1 user myriads 314M Jan 17 15:44 result_4096.mov
-rw-r--r-- 1 user myriads  61M Jan 17 15:32 result_512.mov
-rw-r--r-- 1 user myriads  30M Jan 17 15:32 result_64.mov
-rw-r--r-- 1 user myriads 607M Jan 17 15:47 result_8192.mov
  • Debug with result files :
user@igrida-oar-frontend:~$ cat /temp_dd/igrida-fs1/user/ffmpeg/oar_output/job.869088.error
+ module load veertuosa/0.0.1
++ /soft/igrida/Modules/3.2.10/bin/modulecmd bash load veertuosa/0.0.1
+ eval LOADEDMODULES=veertuosa/0.0.1 ';export' 'LOADEDMODULES;PATH=/soft/igrida/veertuosa/0.0.1/bin:/usr/lib/oar/oardodo:/usr/lib/oar/oardodo:/usr/local/bin:/usr/bin:/bin:/usr/games' ';export' 'PATH;VEERTUOSA_CONTEXT=igrida' ';export' 'VEERTUOSA_CONTEXT;_LMFILES_=/soft/igrida/modulefiles/applications/veertuosa/0.0.1' ';export' '_LMFILES_;'
++ LOADEDMODULES=veertuosa/0.0.1
++ export LOADEDMODULES
++ PATH=/soft/igrida/veertuosa/0.0.1/bin:/usr/lib/oar/oardodo:/usr/lib/oar/oardodo:/usr/local/bin:/usr/bin:/bin:/usr/games
++ export PATH
++ VEERTUOSA_CONTEXT=igrida
++ export VEERTUOSA_CONTEXT
++ _LMFILES_=/soft/igrida/modulefiles/applications/veertuosa/0.0.1
++ export _LMFILES_
+ VM_NAME=vm_869088
++ date
++ hostname
+ echo '[DEBUG] VM started at Tue Jan 17 15:23:33 CET 2017 on igrida10-12'
+ veertuosa_launch --name vm_869088 --image /temp_dd/igrida-fs1/user/ffmpeg/images/docker-ffmpeg.qcow2
+ FFMPEG_BITRATE=64
+ FFMPEG_INPUT_FILE=/tmp/ffmpeg/input_files/big_buck_bunny_1080p_h264.mov
+ FFMPEG_OUTPUT_FILE=/tmp/ffmpeg/output_files/result_64.mov
+ VM_CMD='docker run -v /mnt/temp_dd/igrida-fs1/user/ffmpeg:/tmp/ffmpeg jrottenberg/ffmpeg -stats -i /tmp/ffmpeg/input_files/big_buck_bunny_1080p_h264.mov -b:v 64k /tmp/ffmpeg/output_files/result_64.mov'
+ ssh-vm vm_869088 'docker run -v /mnt/temp_dd/igrida-fs1/user/ffmpeg:/tmp/ffmpeg jrottenberg/ffmpeg -stats -i /tmp/ffmpeg/input_files/big_buck_bunny_1080p_h264.mov -b:v 64k /tmp/ffmpeg/output_files/result_64.mov'
Warning: Permanently added '[localhost]:22001' (ECDSA) to the list of known hosts.
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr/local --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --bindir=/usr/local/bin --disable-doc --disable-static --enable-shared --disable-ffplay --extra-libs=-ldl --enable-version3 --enable-libfdk_aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-gpl --enable-avresample --enable-postproc --enable-nonfree --disable-debug --enable-small --enable-openssl
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/ffmpeg/input_files/big_buck_bunny_1080p_h264.mov':
...
frame=14315 fps= 26 q=-1.0 Lsize=   30147kB time=00:09:56.48 bitrate= 414.0kbits/s speed= 1.1x
video:4885kB audio:24860kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.350476%
...

Expose services

veertuosa_launch script allow you to expose port with the --redir option.

user@igrida06-09:~$ veertuosa_launch --name myvm --image /soft/igrida/vms/images/ubuntu-16.04-x86_64.qcow2 --redir 10080:80
user@igrida06-09:~$ ssh-vm myvm
root@veertuosa:~# apt update && apt install -y apache2

Virtual machine HTTP port (80) is redirected to port 10080 on igrida06-09. Now, try url http://igrida06-09.irisa.fr:10080 into your web browser to see the default page of the HTTP server previously installed.