Testing the RuneStone interactive Python courses server in docker

I’ve been working on setting up a Docker container environment allowing to test the RuneStone Interactive server.

RuneStone Interactive allows the publication of courses containing interactive Python examples, and while most of the content is static (the Python examples are run innside a Python interpreter implemented in JavaScript, hence locally in the JS VM of the Web browser), the tool also offers an environment allowing to monitor the progress of learners in a course, which is dynamic and is queried by the browser over AJAX APIs.

That’s the part which I wanted to be able to operate for test purposes. As it is a web2py application, it’s not exactly obvious to gather all dependencies and run locally. Well, in fact it is, but I want to understand the architecture of the tool to be able to understand the deployment constraints, so making a docker image will help in this purpose.

The result is the following :

Now, it’s easier to test the writing of a new course (yet another container above the latter one), and directly test for real.

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Configuring the start of multiple docker container with Vagrant in a portable manner

I’ve mentioned earlier the work that our students did on migrating part of the elements of the Database MOOC lab VM to docker.

While docker seems quite cool, let’s face it, participants to the MOOCs aren’t all using Linux where docker can be available directly. Hence the need to use boot2docker, for instance on Windows.

Then we’re back quite close to the architecture of the Vagrang VM, which relies too on a VirtualBox VM to run a Linux machine (boot2docker does exactly that with a minimal Linux which runs docker).

If VirtualBox is to be kept around, then why not stick to Vagrant also, as it offers a docker provider. This docker provider for Vagrant helps configure basic parameters of docker containers in a Vagrantfile, and basically uses the vagrant up command instead of using docker build + docker run. If on Linux, it only triggers docker, and if not, then it’ll start boot2docker (or any other Linux box) in between.

This somehow offers a unified invocation command, which renders a bit more portable the documentation.

Now, there are some tricks when using this docker provider, in particular for debugging what’s happening inside the VM.

One nice feature is that you can debug on Linux what is to be executed on Windows, by explicitely requiring the start of the intermediary boot2docker VM even if it’s not really needed.

By using a custom secondary Vagrantfile for that VM, it is possible to tune some parameters of that VM (like its graphic memory to allow to start it with a GUI allowing to connect — another alternative is to “ssh -p 2222 docker@localhost” once you know that its password is ‘tcuser’).

I’ve committed an example of such a setup in the moocbdvm project’s Git, which duplicates the docker provisioning files that our students had already published in the dedicated GitHub repo.

Here’s an interesting reference post about Vagrant + docker and multiple containers, btw.

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Student project contributions on using docker to build headless virtualised labs for MOOC participants

After having used Vagrant to create Virtual Machines (VMs) ready made for MOOC participants, I’ve been investigating the use of docker for a similar need.

In order to move on, I’ve applied a clever trick by proposing a project to students of ours (3rd year engineering school end-of-classes project of the ASR major).

The project was addressing several needs :

  • a functional need: being able to monitor what happens inside the VMs run on the participant’s side, so that MOOC labs aren’t performed in a black box;
  • a non functional need: investigate the potential benefits of docker, compared to Vagrant.

The students (François Monniot and Alexis Mousset) have worked very well, and we have a few apps in the result, that allow us to assemble a POC in order to complete our evaluation.

In following posts, I will describe some of these components and will present some ideas and maybe results on the technology’s potential. The impatient reader may directly browse their code on GitHub, starting from the dedicated site.

Posted in Uncategorized | Tagged , , , | Leave a comment

Shell script to connecting to a Shibboleth protected web app with curl

Here’s a shell script I’ve created (reusing one meant for CAS protected resources), which will allow to connect to a Web application protected by the Shibboleth SSO mechanism.
Continue reading

Posted in Uncategorized | Tagged , , , , | Leave a comment

Offline backup/mirror of a Moodle course, using httrack

I havent’ found much details online on how to perform a Moodle course mirror that could be browsed offline using httrack.

This could be useful both for backup purposes, or for distant learners with connectivity issues.

In my case, there’s a login/password dialog that grants access to the moodle platform, which can be processed by httrack by capturing the POST form results using the “catchurl” option.

The strategy I’ve used is to add filters so that everything is excluded and only explicitely mentioned filters are then allowed to be mirrored. This allows to perform the backup connected with a user that may have high privileges, while avoiding to disappear in loops or complex links following for UI rendering variants of Moodle’s interface.
Continue reading

Posted in Uncategorized | Tagged , , , , | 1 Comment

BoF “retour expérience enseignement Python” lors de la PyConFr 2014

Nous avons organisé une BoF “retour expérience enseignement Python” lors de la PyConFr 2014 à Lyon.

Cette BoF s’inscrivait dans la continuité des présentations du thème Enseignement / Recherche de la conférence.

Environ 40 participants ont pris sur leur temps de pause (merci aux organisateurs de nous avoir fourni des sandwiches de façon prioritaire) pour partager des éléments de retour d’expérience.

Voici quelques éléments qui m’ont semblé intéressants, qui pourront peut-être vous intéresser, qui ont été abordés durant la BoF, ou que j’ai collectés dans les présentations de la matinée (j’ai un peu délaissé l’amphi de l’après-midi, pour aller discuter avec quelques connaissances aussi présentes à la conf) :

  • enseignement :
    • deux environnements assez populaires : IPython notebooks, et Sage. Le deuxième éventuellement intégré avec le premier (SageCell). Les deux permettent d’avoir une interface Web qui semble offrir un certain avantage pour une population d’apprenants “génération FaceBook” qui trouve facile de rentrer du texte dans des zones de saisie d’une page Web (contrairement à des environnements type ligne de commande ou éditeurs plutôt effrayants). Avantage de disposer de LaTeX ou de texte (et plus) et de code dans un même document.
    • Même si Matlab reste assez utilisé dans certains labos (poids du legacy), l’enseignement semble pas mal passer à IPython, même si certains inconvénients existent parfois (plantages des kernels avec MatPlotLib, par ex.).
    • Livre : Calcul mathématique avec Sage
    • Les notebooks permettent de rendre la programmation vivante : le prof tape le code : montre les erreurs, les typos, etc. Rend le
      processus plus accessible.
    • Côté notation des devoirs, corrections deux solutions en environnements MOOC avec de nombreux apprenants :
  • Recherche :
    • le LaTeX embarqué dans des Notebooks IPython amène certains à s’y intéresser alors qu’ils n’y viendraient pas naturellement (“elle
      est belle ta formule, t’as fait ça comment ?” : double clic, et hop ;)
    • On commence à croiser des Notebooks IPython en matériel supplémentaire pour des publications académiques
Posted in Uncategorized | Tagged , , , | Leave a comment

Le MOOC Bases de données relationnelles est lancé

Nous venons de lancer la première édition du MOOC sur les bases de données relationnelles de Télécom SudParis. Au programme, de la théorie (algèbre relationnelle), de la pratique (dans SQLite dans les navigateurs basés sur WebKit, et plus tard dans PostgreSQL dans une box Vagrant basée sur Debian (voir post précédent)), des contenus et logiciels libres (autant que possible) et pas mal de rush pour finaliser tout ça dans le Moodle.

On débute avec plus de 800 inscrits à la fin du premier jour (y compris les 180 étudiants ingénieurs de 2ème année de Télécom SudParis, qui suivront le cours présentiel en parallèle du MOOC, et collaboreront avec les apprenants externes pour les travaux personnels).

Il est toujours possible de s’inscrire : le gros du travail commence en semaine 2 (qui commence lundi 15/09 à 00h00 heure de Paris).

Posted in Uncategorized | Tagged , , , , | Leave a comment

Building a lab VM based on Debian for a MOOC, using Vagrant + VirtualBox

We’ve been busy setting up a Virtual Machine (VM) image to be used by participants of a MOOC that’s opening in early september on Relational Databases at Telecom SudParis.

We’ve chosen to use Vagrant and VirtualBox which are used to build, distribute and run the box, providing scriptability (reproducibility) and making it portable on most operating systems.

The VM itself contains a Debian (jessie) minimal system which runs (in the background) PostgreSQL, Apache + mod_php, phpPgAdmin, and a few applications of our own to play with example databases already populated in PostgreSQL.
Continue reading

Posted in Uncategorized | Tagged , , , , , , , , , | 8 Comments

Using RDFAlchemy together with RDFLib’s SPARQLStore to query DBPedia and process resources in OO way

I’ve been searching for interesting ways to manipulate RDF graphs in Python, to create an application that would handle Linked Data Resources in an OO-way, i.e. using Python classes and not tables/sets/lists of triples. The data will be persisted in graphs in a triple store, accessed through a SPARQL enpoint.

In this post, I’ll illustrate how I managed to tie RDFLib’s SPARQLStore plugin and RDFAlchemy to reach a rather nice looking result.

Continue reading

Posted in Uncategorized | Tagged , , , , , , , , | 2 Comments

Debian docker containers using a modified baseimage-docker

I have been testing Docker for a few weeks now, and investigated the use of baseimage-docker, which provides support for supervising services with runit, and includes OpenSSH, among other things, based on an Ubuntu base system. Of couse, I’m interested in a Debian counterpart.

I had initially followed instructions provided by Steve Kemp which also prepared a Debian image including OpenSSH and runit, but it appears that baseimage-docker provides more tiny bits that avoid reinventing the wheel.

I’ve then forked the baseimage-docker to do a quick and dirty adaptation for Debian. There’s a sid variant (my ‘debian’ branch) and a wheezy one (my ‘wheezy’ branch, unsurprisingly). I haven’t used all features of baseimage-docker, so things might break for sure.

For the records, I’m playing with it as a base image to construct a docker-based container running the FusionForge test suite.

Did I warn you it’s quick and dirty and without any warranty ? Hoping that this is useful anyway.

Posted in Uncategorized | Tagged , , , , | Leave a comment

Tagged a first version of the TWiki to FusionForge’s MediaWiki converter

As announced previously, I’ve been hacking on a migration tool allowing to import into the MediaWiki of a FusionForge project, a conversion of the contents of a TWiki wiki.

I’ve succesfully imported a first project (from PicoForge to FusionForge) using the tool, so I’ve decided to tag a first release and make the Git repo accessible.

More details at : https://fusionforge.int-evry.fr/projects/pytwiki2mediawi/

Feel free to ask here in the comments or by email, in case of need.

And, yes, my Python is most likely awful, but at least, this works, and much more featureful than existing tools I could test.

Posted in Projects, Uncategorized | Tagged , , , , , | 1 Comment

Working on a TWiki to MediaWiki converter (targetting FusionForge wikis)

I’m currently working on a wiki converter allowing me to transfer old TWiki wikis (hosted on picoforge) to MediaWikis hosted on FusionForge.

Unlike existing tools that I’ve found that more or less target the same needs, mine will address two peculiarities :

  • using MediaWiki’s API to perform the import, where many tools seemed to use SQL requests: this should allow non-administrator users to do the job,
  • importing to wikis of projects hosted on FusionForge instances, even when the project is not public, which means that the API calls need to authenticate to FusionForge first.

The tool is written in Python, and will include my own crappy wiki syntax converter in Python, instead of spawning existing Perl scripts, as others did.

It may happen to work for FosWiki too, but I don’t intend to use it beyond our old TWiki installations, for a start.

Stay tuned for more progress updates.

Edit: I’ve now released a first version.

Posted in Uncategorized | Tagged , , , , , , | 1 Comment

Switched from gnome flashback-session to XFCE

I’ve gotten fed up with Gnome flashback session annoyances on Debian testing, so, for a bit more than a month now I’m running XFCE 4.

So far so good.

YMMV but if you’ve not made the switch to Gnome 3’s shell and all the JS enabled novelties, XFCE might be an option.

Posted in Uncategorized | Tagged , , , | Leave a comment

Generating WebID profiles for Debian project members

I’ve been investigating the generation of WebID profiles for Debian project members for some time.
Continue reading

Posted in Uncategorized | Tagged , , , | Leave a comment

Qu’est-ce que le Linked Data

Voici une traduction en français d’un court document introductif au Linked Data, originellement écrit par Luca Matteis :

http://www-public.telecom-sudparis.eu/~berger_o/linkeddatawhat/whatislinkeddata.html

Bonne lecture.

Posted in Uncategorized | Tagged , , , , | Leave a comment