Commercial vs. Academic Software

Recently, I had the pleasure to meet with an R&D representative of a major DBS electrode manufacturer and we talked about software built to model deep brain stimulation electrodes in computer simulations, such as Lead-DBS.

This meeting – and the development of Lead-DBS since 2012 – made me think about pros and cons of academic vs. commercial software with similar goals. A few aspects were new to me and could be of interest, so I decided to write them up.

Since a few years, software that aims at modeling DBS electrodes in computer simulations has been emerging across the globe, both authored by research institutions and industry. On the academic side, Cicerone DBS (1) was a pioneering platform developed by the McIntyre lab later bought by Boston Scientific and turned into the commercial application GUIDE. Around the same time, the Yelnik/Bardinet group in Paris developed the deformable YeB atlas that went into the commercial Optivize (2) software by Medtronic and is now used in it’s successor, SureTune 3. The company Brainlab also develops similar software with some more focus on the pre-surgical / planning part. CranialCloud, developed by the spinoff Neurotargeting LLC out of Vanderbilt University also develops a product aimed at modeling DBS electrodes.

We started development of Lead-DBS in 2012 to fill a clinical need at our center, since at the time, no commercial or academic software was available for download purchase. Later on, the first similar software that was made publicly available was the AFNI based DBSproc. There are multiple other academic projects in development, such as PyDBS in Rennes, PaCER in Luxembourg, “VirtualPatient” at MGH in Boston, DBSmapping in Porto, StimVision at Case Western – this list may not be exhaustive. At the point of writing, the only two academic toolboxes that are openly available for download are DBSproc and Lead-DBS (3).

The obvious

A few obvious pros and cons for commercial vs. academic software exist: Of course, academic software is most often free and open source. It may be customized, you are free to plug in your own research scripts and modify everything tailor-cut to your needs. For some researchers, this is a massive advantage. I would even argue that some research projects may just not be feasible with commercial software since it’s often hard to automate, batch-process or interface with these tools.

On the other hand, commercial software is shiny, easy to use, comparably bug free and if need be, you can often call a hotline or even a sales-rep for in-house support. These advantages can be massive, as well. Hassle-free work with a product in a pipeline that just needs to function is often not easily imaginable with academic software. In a purely clinical setting, where not-so tech-savvy residents need to quickly localize electrodes in the brains of their patients can easily become a nightmare with academic and their home-brew nature.

Keeping pace with research

A slightly less obvious advantage of research software is that it is much more flexible. Commercial applications undergo tedious certification processes to achieve CE-marks or FDA-approvals and may safely be applied in clinical context. This is a great advantage for clinicians but drastically slows down development of software. New research developments may not be easily integrated into commercial tools since they need to be certified, too. On the wild-west side of academic software, however, new tools can be integrated from idea & concept to end-user deployment within days. For instance, in 2009, the global fibertracking approach (‘Gibbstracking, Reisert 2009’) won the Neurospin fibercup and was thus evaluated as the best fibertracking software compared to 9 competitors. We integrated this software into Lead-DBS when we started it. Recently, a new comparative study found the generalized Q-sampling algorithm implemented in DSI studio most promising. With some help from Fang-Cheng Yeh, the author of DSI studio, we were able to integrate this approach into Lead-DBS within a few weeks. Fibertracking is an extremely good example to highlight differences between academic and commercial pipelines – the latter often still apply a combination of tensor-based DTI & Mori Streamline tracking.

Another good example is a brainshift correction feature in Lead-DBS that developed from concept to published code on a 3-day hackathon at MIT. Such a pace is not imaginable for commercial applications – but naturally, it may also bear some risks.

Reproducibility and open science

Clinicians may not care too much about this and likely trade open source for a CE-mark / FDA approval in a snap. Still, using a commercial software without published methods does not fulfill criteria of reproducible science and is a no-go in the eyes of most researchers. Moreover, in the short history of DBS imaging suites, already three of them were announced to reach market shortly but soon after withdrawn, their development discontinued (4).

In the short period between announcement and withdrawal, some scientific colleagues had enthusiastically started research projects that would make use of the tools because they had been given exclusive pre-market access to the software. After withdrawal of the tools, reproducibility of their scientific findings was not guaranteed anymore.

Moreover, sales representatives of two major electrode manufacturers told me that their companies did not have any interest in publishing methods of the software packages (such as details about nonlinear warping techniques applied or the estimation of activation volumes around DBS electrodes).

Self-limitation

One reason I write this post is to highlight a thought that had not been obvious to me for some time. Namely, algorithms applied in commercial applications are often limited by strict time constrains. For instance, there are various methods to perform nonlinear multispectral co-registration between patient and template anatomy implemented in Lead-DBS. If one chooses the BSplineSyn approach included in Advanced Normalization Tools and selects the “fine” preset with additional subcortical refinement, a registration will use up ~50GB of RAM and it is really helpful to run it on a compute cluster. Lead-DBS proposes of an internal job submitting system that works on usual high performance clusters with some minor modifications. [Needless to say, Lead-DBS can easily be used on a standard laptop and most users do so – but then choose less costly deformation strategies]. In academic environments, processing time is of secondary importance (e.g. less important than accuracy of results). Scientists are used to long job walltimes and for most, it is not a dealbreaker if a job takes a few hours or a night. In contrast, this doesn’t fit into a clinical workflow where results are needed fast. An R&D representative of a company told me their product was limited to processing times below an hour, a few minutes were optimal. This is understandable given the target user group of their products – but it deliberately hinders their algorithms from realizing their full potential.

Conclusion

I hope this post could elaborate some pros and cons about commercial and academic software in the field of DBS imaging. It is not meant to discretize either/or – on the contrary, being a clinician and researcher myself, I am absolutely convinced that there is a strong need for both types of software. Still, I hope I could convince you about some issues in using either type of software for the wrong task. Needless to say, I’d love to hear your feedback or thoughts.

Footnotes

1. to the best of our knowledge development discontinued but still available for macaques
2. to the best of our knowledge development discontinued
3. to the best of our knowledge
4. referring to Boston Scientific GUIDE system, Medtronic OptiVize and the initial version of Medtronic SureTune