Category Archives: Science

Science

Portable Hardware Locality (hwloc) Library v1.0 Released

Posted on by

One of the things that us HPC folks tend to get hot under the collar about is hardware locality, basically making sure that your memory accesses are as fast as possible by optimising where on the system you’re getting memory from and making sure your process doesn’t get moved further away. Just binding your processes to the cores they are on can make for a significant speed up so it’s well worth doing. If you’ve just got a single socket, or a pre-Nehalem Intel x86 system then your path to RAM has been pretty much identical wherever you are so the only benefits are from not moving away from your CPU cache lines but on AMD Opteron, Nehalem, Itanic, Alpha, etc you really should care a lot more about locality for best performance.

The open source Torque queuing system (which I help out with) does some of this already, if you compile it with –enable-cpuset and have the /dev/cpuset virtual filesystem mounted then before it starts a job on a node it will create a cpuset for that (based on what cores have been allocated on the node) and then put the HPC processes into that cpuset. If you’re using Open-MPI 1.4.x and have the environment variable OMPI_MCA_orte_process_binding set to core then each of the MPI ranks will bind itself to one of the cores within that cpuset.

All good ? Well not quite as Torque is reliant on /dev/cpuset being there and being able to parse the contents of it and Open-MPI 1.4.x uses the Portable Linux Process Affinity (PLPA) library which, as its name suggests, is only for Linux. So the good Open-MPI people looked at their PLPA library and decided it needed extending and teamed up with the INRIA libtopology team who were working on how you discover the topology of various architectures and decided to merge the two projects together under the banner of the Portable Hardware Locality (hwloc) library.

The Portable Hardware Locality (hwloc) software package provides a portable abstraction (across OS, versions, architectures, …) of the hierarchical topology of modern architectures, including NUMA memory nodes, sockets, shared caches, cores and simultaneous multithreading. It also gathers various system attributes such as cache and memory information. It primarily aims at helping applications with gathering information about modern computing hardware so as to exploit it accordingly and efficiently.

The portable bit of the name comes from the fact that it works on Linux, Solaris, AIX, Darwin, FreeBSD, Tru64, HP-UX and Windows (though with limitations on some architectures – e.g. Windows – which don’t expose all the info it needs) and can extended for other OS’s if people feel they need to scratch that itch (OpenVMS anyone?). This release is also embeddable into projects (such as Open-MPI 1.5) and I have an interest in Torque picking it up to improve and extend its cpuset support.

How China Wrecked Copenhagen

Posted on by

A very interesting story courtesy of the Australian ABC from an insider at the Copenhagan COP15 climate change talks:

He says Prime Minister Kevin Rudd and other Western leaders were visibly upset when China started “removing all the numbers that mattered” in the final talks, including emissions cuts by developed countries of 80 per cent by 2050. ‘Why can’t we even mention our own targets?’ demanded a furious [German Premier] Angela Merkel. Australia’s prime minister, Kevin Rudd, was annoyed enough to bang his microphone. Brazil’s representative too pointed out the illogicality of China’s position. Why should rich countries not announce even this unilateral cut?

Before anyone gets any thoughts that this is someone from a rich country trying to pass the buck, the person in question was Mark Lynas who was attached to the Maldives delegation, who have probably the most to loose from this. Mark has written more in an article in the UK’s Guardian newspaper, saying:

Copenhagen was a disaster. That much is agreed. But the truth about what actually happened is in danger of being lost amid the spin and inevitable mutual recriminations. The truth is this: China wrecked the talks, intentionally humiliated Barack Obama, and insisted on an awful “deal” so western leaders would walk away carrying the blame. How do I know this? Because I was in the room and saw it happen. China’s strategy was simple: block the open negotiations for two weeks, and then ensure that the closed-door deal made it look as if the west had failed the world’s poor once again. And sure enough, the aid agencies, civil society movements and environmental groups all took the bait. The failure was “the inevitable result of rich countries refusing adequately and fairly to shoulder their overwhelming responsibility”, said Christian Aid. “Rich countries have bullied developing nations,” fumed Friends of the Earth International.

It was basically gutted at their behest:

China, backed at times by India, then proceeded to take out all the numbers that mattered. A 2020 peaking year in global emissions, essential to restrain temperatures to 2C, was removed and replaced by woolly language suggesting that emissions should peak “as soon as possible”. The long-term target, of global 50% cuts by 2050, was also excised. No one else, perhaps with the exceptions of India and Saudi Arabia, wanted this to happen.

So there we go, it’s probably futile to try and get a deal in the near future, and longer than that (saving a miracle) probably means we’re stuffed.

The ensemble contained 17 model runs, and of these 13 showed a global average temperature rise of 4°C or higher by the 2080s.

My gut feeling that humans are too stupid to survive this one seems to be correct – I only wish that it wasn’t going to be at the expense of so many powerless people.

RIP Tudor Jenkins, Reader in Physics, University of Wales Aberystwyth

Posted on by

My friend Alun Jones forwarded onto me the sad news of the death of one of my former lecturers, Tudor Jenkins (here’s the archive.org link, in case the original goes away), affectionately called “Tuba Jenkins” by his students as he played the tuba for the Aberystwyth Silver Band (Seindorf Arian Aberystwyth).

Dr TUDOR E. JENKINS, MA, DPhil (Oxon), FInstP

It is with great sadness that we report the untimely death of Dr Tudor Jenkins, Reader in Physics at the Institute of Mathematics and Physics at Aberystwyth University. Dr Jenkins died on 3rd November after a short illness aged 60.

Originally from the Rhondda Fawr, Dr Jenkins read Physics at Corpus Christi College Oxford, and obtained a D.Phill at the Clarendon Laboratory, Oxford. He subsequently studied as a post-doctoral research assistant in Cardiff University before being appointed as lecture in micro electronics at St Andrews University in 1979. He joined the Department of Physics at Aberystwyth in 1983 becoming Senior Lecturer in 1990 and Reader in 2007.

A tribute by Professor Neville Greaves, Director of the Institute of Mathematics and Physics, is published on http://www.aber.ac.uk/aberonline/en/.

I remember I did his lasers course when I did my degree in Planetary and Space Physics at Aberystwyth back in the late 1980’s and whilst I wasn’t very good at that course I do remember his enthusiasm for physics and the fact he could inspire you to want to learn more. Turns out that he was recognised for that, in 2005 he won the Universities award for Teaching Excellence. This quote from the tribute also rings true to form:

Dr Tudor Jenkins was a committed and colourful colleague, famous for his often pithy Latin quotations with which he ended his e-mails. Looking forward to rationalising teaching modules for the 2009 session, he concluded wryly with Occam’s Razor: entia non sunt multiplicanda praeter necessitatem, which approximately translates as “entities must not be multiplied beyond necessity”. Tudor was always a pragmatist.

I also seem to remember that he had an enthusiasm for real ale.. 😉

Serious SSL Renegotiation Problem

Posted on by

This just in from Ben Lawrie:

For the last 6 weeks or so, a bunch of us have been working on a really serious issue in SSL. In short, a man-in-the-middle can use SSL renegotiation to inject an arbitrary prefix into any SSL session, undetected by either end.

But wait, there’s more..

To make matters even worse, through a piece of (in retrospect) incredibly bad design, HTTP servers will, under some circumstances, replay that arbitrary prefix in a new authentication context. For example, this is what happens if you configure Apache to require client certificates for one directory but not another. Once it emerges that your request is for a protected directory, a renegotiation will occur to obtain the appropriate client certificate, and then the original request (i.e. the stuff from the bad guy) gets replayed as if it had been authenticated by the client certificate. But it hasn’t.

Ben has a patch against the current development head of OpenSSL to ban renegotiation, but for most people it’ll need backporting to their current OpenSSL versions..

MPI-3 Forum Seeks Feedback from Fortran MPI Developers

Posted on by

In a posting to the Open-MPI development list Jeff Squyres has requested feedback from Fortran MPI developers on proposed changes to the bindings of MPI functions in MPI-3. He writes:

In the MPI-3 Forum, we’re working on revamping the Fortran bindings to be “better” (for a variety of definitions of “better”). There’s at least one question that we really need some feedback from the MPI Fortran developer community before proceeding. Craig Rasmussen from Los Alamos National Laboratory, chair of the MPI-3 Fortran Working Group, asked me to post a “request for information” to my blog and pass on the URL to every Fortran MPI programmer that I know

The URL of Jeff’s blog is http://blogs.cisco.com/ciscotalk/performance/comments/mpi-3_fortran_community_feedback_needed/. Please pass this on if you do know other Fortran MPI developers.

More Global Warming Evidence

Posted on by

Hot off the press from the University of Colorado via Futurity.org – an analysis of 200,000 years of climate evidence from a lake on Baffin Island in the arctic shows that whilst the temperature should be getting cooler due to natural effects it’s actually going the other way:

While environmental changes at the lake over the past millennia have been shown to be tightly linked with natural causes of climate change—like periodic, well-understood wobbles in Earth’s orbit—changes seen in the sediment cores since about 1950 indicate expected climate cooling is being overridden by human activity like greenhouse gas emissions.

The sediments here go way back:

The lake sediment cores go back 80,000 years beyond the oldest reliable ice cores from Greenland and capture the environmental conditions of two previous ice ages and three interglacial periods.

Anglo-Saxon Hoard Discovered in Mercia

Posted on by

A metal detectorist has discovered a hoard of precious items (now declared treasure under the Treasure Act 1996) in land that was once part of the Anglo-Saxon kingdom of Mercia in England. Archaeologists were called in and recorded around 1500 items altogether, with around half being precious metals and gems.

It looks phenomenal, and is far larger than the hoard found at the Sutton Hoo ship burials which itself was impressive enough. What is really interesting to me is that these are often fragments of larger items and even the whole items haven’t been treated with care by whoever assembled the hoard – crosses are folded up for example – indicating that it was the previous metals and gems rather than items themselves that were valued. One of the experts who examined the hoard wrote:

The material is predominantly associated with war – swords, sword fittings, bits of helmets and the like – but all the precious metalwork has been stripped. That means they’re not treasuring the objects as wholes, they’re taking the precious metals off and keeping them.

Basically it looks like the collector of this was an Anglo-Saxon magpie, on a massive scale! 🙂

I’m glad to see that this is a case where the Treasure Act and the Portable Antiquities Scheme worked out really well; the find was reported immediately allowing archaeologists to excavate the items in situ and record all the vital information that lets us put these items into the wider context which is otherwise lost if these things are just ripped out of the ground and disappear into a private collection or onto the black market. All too often we are losing that valuable information about our past forever when that happens. So big props to the finder and the landowner for doing the right thing (not to mention they’re up for a massive reward, which is also the right thing to do).

UK Government Apologises to Alan Turing

Posted on by

55 years after Alan Turing, one of the fathers of modern computing and one of the intellectual powerhouses behind the achievements of Bletchley Park, committed suicide following his conviction for “gross indecency” for being gay and his subsequent exile from GCHQ the UK Prime Minister has apologised for his treatment.

Turing was a quite brilliant mathematician, most famous for his work on breaking the German Enigma codes. It is no exaggeration to say that, without his outstanding contribution, the history of World War Two could well have been very different. He truly was one of those individuals we can point to whose unique contribution helped to turn the tide of war. The debt of gratitude he is owed makes it all the more horrifying, therefore, that he was treated so inhumanely. In 1952, he was convicted of ‘gross indecency’ – in effect, tried for being gay. His sentence – and he was faced with the miserable choice of this or prison – was chemical castration by a series of injections of female hormones. He took his own life just two years later. […] we’re sorry, you deserved so much better.

The BBC has a good article on Turing, his persecution and the apology.

Alan, we all owe you a massive debt of gratitude for all your work and I’m very sorry the UK treated you so very cruelly. We cannot right those wrongs, all we can hope to do is to learn from them and try to not let them be repeated.

(Heard via an InsideHPC blog)