If you ask me what the single most important benefit of an open source systems is, I would unhesitatingly say “accelerates collaboration and hence innovation” through extended communities. Not only is it fun to work within international communities, but if you want to build leading-edge software an open source approach is the best way to connect with experts wherever they are in the world. As Michael Nielson argues in his wonderful book Reinventing Discovery The Era of Networked Science, online tools make groups collectively smarter by harnessing micro-expertise, that is harnessing expert attention at just the right place and just the right time (also referred to as designed serendipity).

As Dr. Nielson explains, cognitively diverse, large groups have a tremendous inherent expertise with diverse capabilities, and with the right tools and processes this latent potential can be harnessed to amazing effect.  This is exactly the process I have seen on numerous occasions developing scientific computing systems ranging from visualization to medical imaging to software process tools; working with open research labs and closed commercial organizations and everything in between. However, it does require that some basic conditions are met in order to work; in my experience these are the key considerations to creating a successful open source project:

• The starting point is the quality of openness. Barriers to simple exchange of documents, code and data kill innovation. Break down barriers, and this may even include the heresy of avoiding certain open source licenses (e.g., reciprocal, GPL) licenses since they scare many potential collaborators away.

• Next there’s the issue of the scale of the community. Having a larger, passionate community inherently brings more expertise to the table. But it’s important to match the right members with the right challenges or inefficient churning will result.

• The foundation for collaboration is communication. A layered strategy is most effective including messaging, IRC, quick chats (video and audio), email, micro-blogging, wikis, web pages, and formal meetings (including face-to-face code-athons). Software processes that implicitly integrate communications such as code repositories, software review, pair-programming, and testing dashboards are invaluable.

• A critical component is what I refer to as the “maturity” of the community. Ego-centric, controlling, combative, and inflexible communities die an ugly death. Those that value community members, encourage vigorous, merit-based discussion, and have the ability to make effective decisions are a delight.

• A clear understanding of the business model is essential, meaning where is the long term support of the community going to come from? Is it strictly volunteer? Or are there future aspirations for business? I’ve seen open source communities splinter due to clumsy attempts at monetizing a project. Get this issue out in the open early, which may also include choosing the right license.

• Finally there’s the software process. If the community isn’t testing, then I can guarantee that the software is broken. There are fantastic open-source software process tools available ranging from git (version control), gerrit (code review), Maven and CMake (cross-platform building), Jenkins and CTest/CDash (testing), and so on. Use them, live them, and breathe them.

Of course there are some foundational axioms in play here. If the project captures little interest it will result in a community size of one, thus there’s the assumption that there is a community to build.  Agile processes are also a necessary condition; the innovative process is predicated on reacting quickly to the community.

Note that this innovation framework for open source systems can be adapted to closed systems too at some cost. For example, in my line of work we frequently integrate open source toolkits into proprietary enterprise workflow; while it works well for our customers, there is a significant negative impact due to limited community size. This is why some customers adopt hybrid approaches, contributing as much “infrastructure code” to the open community, thus offloading maintenance and harvesting innovation, yet holding back what they consider their crown jewels.

As computing professionals, we are all attempting to deal with the onslaught of new technology. Ultimately we need to depend on the experts all around us to get the job done. Increasingly this means reaching out to large communities, and learning how to collaborate effectively. When done well, it’s amazing how innovative you can be.