As science advances and knowledge accumulates, ensuing generations of innovators spend longer in training and become more narrowly expert, shifting key innovations (i) later in the life cycle and (ii) from solo researchers toward teams

As evidence to this, the average age at which a scientist made a discovery which later qualified for a Nobel prize has increased by 6 years over the course of the 20th century. When looking at publications, the researchers found that the average author list on a publication grew, on average, by 15-20% per decade!

We’ve discussed before the “decline of the Lone Ranger model of science”, but Jones’ paper focuses on looking at the policy implications for such a change. He concludes that the government (and, probably, the academic and private institutions which support researchers) need to adapt policy to reflect this new reality by:

• Tailoring funding and messaging to help keep young researchers interested despite the longer and more difficult training period
• Finding new ways to evaluate the worthiness of proposals as scientist’s expertise becomes more and more specialized
• Altering incentive structures as the team of collaborators replaces the Lone Ranger scientist model of discovery

These policy suggestions are definitely good ones, and are certainly necessary to adapt to a new scientific environment, but one dimension of this which Jones doesn’t discuss as much are the technological (the focus of this blog!) innovations which can help further research in this brave new world.

• Improving science communication with the public. We’ve made multiple mentions of this in the past, but they are no less true here. Active public communications management not only helps secure funding and raise public awareness of the good scientists can do, but it also helps attract the interest of future generations of researchers and policymakers.
• Embracing new collaboration tools. To really kick-start collaboration between scientists across geographies and specialties, we need tools that go beyond just email and fax machines. Tools like Google Wave, wikis, distributed version control, and social media forums like Friendfeed are an early taste of the sort of live collaboration that new web technology can bring about.
• Leveraging cloud computing and heterogeneous compute. One of the reasons discoveries are taking longer and are more expensive is that there is so much more data to collect and to analyze than before. One technological innovation which we’ve talked about at lengths here is the ability of graphics cards/GPUs to make supercomputer-level processing power more readily accessible to research labs. Another is the use of new cloud computing services like Amazon’s to rapidly increase the computational resources that a lab/company has access to. Neither are panaceas for all the data analysis issues which scientists face, but they are definitely ways to make things easier for research groups who have stringent IT budgets.
• Using crowdsourcing to speed innovation. Who says research has to take longer and be more expensive? Perhaps its time to pull on new technological levers which let scientists borrow on the resources and brains of a wider group of people. While new platforms like ChemBioConnect, distributed computing systems like Folding@Home, and volunteer crowdsourcing initiatives like Fold.IT are far from perfect, they hint at a future where researchers can call on resources beyond what their personal computers and brains are capable of.
• Building new research attribution models. When I say new attribution models, I’m referring to two things. The first is embodied by new standards like ORCID which make it easier to understand which person is the author/researcher in question (something which will become more and more important as more people with the same initials/names enter the sciences). The second, and more substantive, is finding new ways to understand who contributed what to a particular study. In today’s digital age, I find it laughable that we still rely on simple author list order to determine the relative roles and positions of the researchers listed on a publication. Employing metadata and other graphical cues can help scientists achieve the recognition they deserve, as well as provide appropriate incentives for teams of researchers to contribute.
• Contributing negative and after-publication results to open repository. While I can understand the hesitation for most research groups to pursue a pure open access strategy, those concerns should not hold with negative or post-publication experimental data. While opening up access to data from failed/negative experiments does little to hurt a lab’s ability to publish first, it can be a dramatic boon for other research groups (especially new labs or labs with interdisciplinary focuses) who can not only use the data for their own analyses and experimental designs, but avoid committing resources to experiments which have already been conducted. If it can work for biotechs and pharma companies, then there’s no reason it should be any different for non-commercial groups.

These suggestions only scratch the surface of what new technologies and policies can do to help scientists in a world where scientific training takes longer and where scientific discoveries need to be more collaborative. If anyone else has any other suggestions, feel free to leave them in the comments!

The distortions caused by spacedust are caused by radiative transfer – a process of light absorption and scattering which also explains why the sky is blue and why sunsets/sunrises look red. Astronomers have built highly sophisticated models to understand radiative transfer across a wide range of different dust backgrounds. These models have enabled researchers to build very cool simulations, such as this one of two galaxies colliding:

Interestingly, greater sophistication in our understanding of spacedust and a greater desire for precision and resolution in the modeling has meant that more and more of the computational processing of these radiative transfer models has been spent on calculating dust grain temperatures rather than the math behind the actual radiation transfer (a product of the fact that you need to calculate across many points in the “dust cloud”, across many types/sizes of dust particles, and because you need to iterate many times to find an equilibrium) – something on the order of 10¹¹-10¹² exponentials per simulation!

That traditional processors are not well suited for calculating exponentials and that there were simply so many calculations which needed to be done in parallel convinced researchers to turn to NVIDIA’s CUDA as a potential solution. As we’ve noted before with using raytracing as a means to accelerate radiotherapy dosage calculations, NVIDIA’s CUDA is a standard programming toolset which lets programmers more easily use the power of (NVIDIA) graphics cards for calculations. Because the calculations needed to do high-performance graphics for a game of Modern Warfare 2 are similar to the calculations that supercomputers crunch through, NVIDIA’s CUDA has been demonstrated to be able to accelerate calculation speed by orders of magnitude!

In the case of dust grain temperature calculation, the results were equally impressive. Not only were the researchers able to accelerate dust grain calculation using a NVIDIA Tesla C1060 (with 4 GB of memory) over an 8-core Intel Xeon E5420 processor (with 32 GB of RAM) alone by a factor of 55, they were able to do this despite:

• the fact that 17% of processing time on the GPU solution was dedicated to data transfer (something the CPU-only solution has to worry about less)
• the maximum theoretical capacity of the GPU was only 6 times greater than that of the CPU, highlighting a big difference between the CUDA philosophy (crank up performance) and the CPU compiler philosophy (abstract but flexible)

Amazingly, the researchers found that even if the CPU were to run an interpolation scheme (requires less processing power, but introduces a little more error and makes it harder to do more sophisticated calculations vs. the equilibrium calculations done here), the GPU solution is still faster by a factor of 16 times!

So: spacedust – 0. GPU – 1. Now let’s see if they can tackle the flexible dust temperature problem…

Paper: “Accelerating Dust Temperature Calculations with Graphics Processing Units”, submitted to New Astronomy; ArXiV link

Armed with his gaming console, Dr. Scarle used the Xbox’s GPU computing power to calculate and even predict cardiac arrhythmias based on his model of electric excitations of the heart. The result? A paper titled Implications of the Turing completeness of reaction-diffusion models, informed by GPGPU simulations on an XBox 360: Cardiac arrhythmias, re-entry and the Halting problem.

This is a highly effective way of carrying out high end parallel computing on “domestic” hardware for cardiac simulations. Although major reworking of any previous code framework is required, the Xbox 360 is a very easy platform to develop for and this cost can easily be outweighed by the benefits in gained computational power and speed, as well as the relative ease of visualization of the system.

So much attention thus far has focused on using the PlayStation 3 in distributed computing projects like Folding@Home — maybe its time that Microsoft release some sort of software to let the legions of XBox360 owners out there show the PS3 users that their machines are good for more than just gaming?

Of course, while ray tracing is an impressive computational feat, most of these demonstrations only show off the aesthetic benefits of being able to implement ray tracing quickly. There are much more real-world impacts in the scientific and medical domains, such as in the field of radiotherapy.

In a nutshell, the idea behind radiotherapy as a cancer therapy is that you use strong bursts of radiation to kill off a tumor while minimizing the side effects of radiation exposure to the surrounding tissue. This balance is extremely difficult to manage as the calculations necessary to understand the effect of applying radiation from an external source on a complex three-dimensional maze of organs and liquids like the human body are highly sophisticated. Interestingly, these calculations actually resemble a ray tracing problem, as the problem of understanding radiotherapy dosage is one of understanding how individual “beams” of radiation travel and interact with the human body.

The result is that computer models which have been used to do dosage calculations are slow (making it impractical for physicians to consider multiple regimens or use more sophisticated “adaptive”/modulated radiation therapies), error-prone from the introduction of assumptions to “gloss over” some of the more sophisticated calculations, and very expensive given the need for large clusters of computational power.

What researchers at the University of Amsterdam have demonstrated is an implementation of a ray tracing algorithm targeted at the radiotherapy dosage question using GPU maker NVIDIA’s CUDA toolkit for performing mathematical calculations using the power of a graphics processor. The researchers used the fact that the power of a GPU rests in its ability to split up complicated math problems into many simpler problems to have the GPU calculate the paths of multiple “rays” of radiation simultaneously, resulting in a performance increase over a non-GPU accelerated technique ranging from 50% faster to 6 times faster! Amazingly, because of the way the GPU does its calculations (mainly that it avoids using a look-up table the CPU-driven algorithm needs), the GPU’s results are also more accurate, despite a single GPU implementation being both faster and cheaper than traditional techniques.

The implications to medicine? To quote the paper:

“The developed GPU algorithm now enables dose calculations at a speed that will be experienced as real time for conventional forward planning based on clinically relevant datasets. this can lead to a major reduction in the workload of radiotherapy treatment planning. Moreover, the presented GPU algorithm can be used to accelerate more advanced treatment planning optimization techniques.”

Paper: M. de Greef et al, “Accelerated Ray Tracing for Radiotherapy Dose Calculations on a GPU.” Medical Physics, Vol 36, Issue 9 (link: http://dx.doi.org/10.1118/1.3190156)

Presentation: http://www.amc.nl/upload/teksten/radiotherapie/hyperthermie/RayForDose-NVIDIA.pdf

(Image credit – Ray tracing schema) (Image from presentation)

Some researchers, however, have turned to a radically different approach. Instead of building a bigger, better mousetrap to deal with more mice, the distributed computing approach takes the approach of placing many small, cheap mousetraps. The result is cheap “supercomputers” which are able to “pool” the computing power of many computers connected over a network.

This approach has been used by projects like Folding@Home and SETI@Home which are able to combine computing power from volunteers over the internet to do the number-crunching needed to simulate protein folding or scan deep space for extraterrestrial life. SETI@Home was the first such large-scale distributed computing platform. This platform, now the Berkeley Open Infrastructure for Network Computing (BOINC), is today used for many other distributed computing projects such as attempts to search for gravitational waves, do climate modeling, and simulate particle collisions in the Large Hadron Collider.

Folding@Home, a project started by the Pande group at Stanford to use distributed computing to study protein folding uses a similar approach, albeit with different underlying software (is it any wonder that a Stanford group doesn’t use Berkeley’s distributed computing platform?! ) . It has probably been the most successful distributed computing approach to date, and, as a testament to the power of distributed computing, has become known as the first computing system to break the petaFLOPS barrier – e.g. capable of one quadrillion floating point calculations per second! This has enabled the team to do protein-folding simulations on a scale of ~10 micro-seconds.

But, as impressive as the science achieved by distributed computing projects is, what impresses me the most is that projects like Folding@Home and SETI@Home have defined some brilliant new ways to do science:

• Use the internet – It’s a common theme on Bench Press, but with more and more people having faster and faster access to the internet, the potential for distributed computing becomes greater and greater. As Folding@Home demonstrated, such approaches can produce computing systems as powerful (or potentially more powerful) as leading supercomputer systems at a fraction of the cost.
• Mobilize the public – We’ve discussed ways for the scientific community to reach out to the public like using social media and creating interactive applications/tools for the public to use, but efforts like Folding@Home illustrate a way to not only reach out to the public but to get them vested in science. In a world where high school science teachers find it difficult to get teens interested in science, initiatives like Folding@Home have created a system where teams of individuals compete on who can contribute the most to the effort! Instead of simply hoping that the public will continue to fund and listen, why not borrow a page from the many existing cancer-walk-a-thons and make it easy for the public to get involved?
• Leverage new technology – It may not come as a surprise to our readers that a significant amount of the computational power at Folding@Home comes from graphics cards and Playstation 3’s. But, while many “mainstream” supercomputers ignored the new power afforded by these new chip types, Folding@Home developed software so that volunteers could quickly and easily use these powerful chips to boost their Folding@Home scores. The Folding@Home initiative also developed software to take advantage of innovations AMD and Intel included in their chips (new multi-core architectures and special instructions to speed up calculations). Is it any wonder, then, that Sony, NVIDIA, and AMD have all publically announced support for the initiative with their products?

I don’t pretend that every scientific problem is amenable to a distributed computing initiative, but to some extent, I believe that every scientific endeavor has something valuable to learn from the success of Folding@Home and SETI@Home and their brethren. To that end, I sincerely hope to see an open-source distributed computing architecture like BOINC but with:

• Support for new chip technologies – To provide greater value to the scientific effort, the architecture should support new chip technologies like Intel’s SSE extensions, SMP, or stream processing
• Client contribution tracking – To make it easier for volunteers to know how much they’ve contributed and/or have contests on how much they’ve contributed, a simple system to enable users/administrators to track the effort is needed
• Better security – Medical initiatives and volunteer privacy concerns demand that very fine and specialized security controls are necessary. Support for sophisticated encryption and authentication are a must.
• Linkage to social media – This probably seems extraneous, but since distributed computing efforts depend on motivated volunteers actively seeking out new volunteers, a successful architecture needs to make it easy for volunteers to share their progress with their friends whether it be via blog, or social network, or Twitter, or anything.
• Tie-in with new cloud computing systems – Along the theme of cutting costs, it is reasonable to assume that as offerings like Google’s App Engine and Amazon’s EC2 and technologies like MapReduce become better developed, we will see cash-strapped research groups using the power of “Clouds” to hold their computing power – after all, what is distributed/grid computing other than a specific variant of cloud computing (de-localized, pooled computing)? It’s probably necessary, then, for the new distributed computing architecture to more easily link with EC2 or MapReduce or App Engine.

Anyone else have any thoughts?

(Image Credit – picture of the internet) (Image Credit – Folding@Home computing power)

There was a time when video game consoles and graphics cards were “just for games.” In those days, game console chips and graphics cards were the domain of little boys, not grown men. Well, thank the stars those days are long gone!

Today, if someone were to tease a grown man for purchasing Sony’s Playstation 3, he could simply reply, “I beg your pardon. I am a grown man, not a little boy. I am clearly using the Playstation 3, not to play great games like Grand Theft Auto IV and Metal Gear Solid, but to use its TeraFLOPS (1 trillion floating point calculations per second) capacity to solve important and complex scientific problems.”

It almost sounds like a fantasy, but it’s not. The idea behind this is pretty basic. To make games and graphics run smoothly, video game console chips and graphics cards have to do a mind-boggling number of calculations much faster than a basic computer chip can. It “just so happens” that the supercomputers scientists and Wall Street analysts use to do simulations and research with also need to do those same types of calculations. Hence the idea of Stream Processing was born – why not use graphics card/game console chips for things which aren’t directly related to graphics or gaming?

Why not indeed? I can’t list all of the projects out there, but here’s just a snapshot of the scientific applications that people have been able to do with the Playstation 3’s unique chip, IBM’s Cell Broadband Engine, and graphics cards from NVIDIA and AMD:

• Researchers at the National Center for Atmospheric Research used NVIDIA’s CUDA stream processing platform to improve the speed of their weather forecasting models
• Astrophysicist Gaurav Khanna from UMass Dartmouth used 16 Playstation 3’s to simulate the collision of two black holes
• AMD researchers were able to demonstrate the use of AMD graphics cards to let a computer see in 3D and then map that image to an actual physical simulation
• The Los Alamos National Laboratories turned to IBM to design them a new supercomputer, but instead of just designing them with plain vanilla chips from Intel and AMD, they asked for a new breed of computer. Enter the Roadrunner, a computer which when fully operational will be capable of PetaFLOPS performance (1 quadrillion floating point calculations per second), and consists of ~7000 AMD Opteron chips coupled with ~13000 Cell processors
• The University of Illinois at Urbana-Champaign, by using 3 NVIDIA graphics processors, was able to help perform biochemical simulations at a submolecular level 100 times faster than 18 typical computer CPUs.

Technology – it’s good for more than just playing games.

Image Credit