Author Archives: Chris Samuel

Bandwidth of Tape

Posted on by

As part of the ongoing Stage 2 upgrade at VLSCI we received an extra 1,000 LTO5 tapes, each rated at 1.5TB uncompressed, for an additional raw, uncompressed, storage of 1.5PB. Now we typically get around 2X compression so that’s about 3PB usable. It took our team about 3 hours to uncrate, unpack, transfer and load the 1,000 tapes, effectively shifting 1.5PB of tape in an hour. That’s about 139GB/s or if you are a network person 1.1Tb/s. Not bad! 🙂

Patch for Modules to use shell functions with BASH, not aliases

Posted on by

Whilst the Modules system is awesome in making life easy to maintain multiple versions of packages and their dependencies (and is heavily used in HPC centres like VLSCI) it can have some annoyances (and seems to be fairly half-heartedly maintained looking at the bugtracker on SourceForge). One thing that’s bitten us from time to time is that you can’t really use its “set-alias” functionality as the bash shell does not expand aliases in non-interactive shells and that includes jobs that are launched from an HPC queuing system like Torque, PBSPro, etc.

It does have the compile time option “--disable-shell-alias” but annoyingly the condition is only applied when your shell is “sh“, not “bash“, so I’ve ended up having to patch Modules to make this work for bash as well. This patch is against 3.2.9c:

--- utility.c.orig      2011-11-29 08:27:13.000000000 +1100
+++ utility.c   2012-05-16 15:08:34.012038000 +1000
@@ -1422,7 +1422,7 @@
         **  Shells supporting extended bourne shell syntax ....
         **/
        if( (!strcmp( shell_name, "sh") && bourne_alias)
-               ||  !strcmp( shell_name, "bash")
+               || ( !strcmp( shell_name, "bash") && bourne_alias )
                 ||  !strcmp( shell_name, "zsh" )
                 ||  !strcmp( shell_name, "ksh")) {
            /**
@@ -1471,7 +1471,7 @@
 
            fprintf( aliasfile, "'%c", alias_separator);
 
-        } else if( !strcmp( shell_name, "sh")
+        } else if( ( !strcmp( shell_name, "sh") || !strcmp( shell_name, "bash") )
                &&   bourne_funcs) {
        /**

Hopefully this patch will be of use to people..