New drug development is one of the most expensive and risky activities that any business undertakes, writes Wikibon Big Data Analyst Jeff Kelly in his latest Wikibon Professional Alert. Taking a new drug from discovery to market takes 12 years and costs about $4 billion on average. And that is if the drug makes it all the way, only 10%-12% do. The others drop out of the process for any of several reasons, including that the drug fails the trial — either because of adverse side-effects, or it turned out to be less effective as a treatment than hoped — or for practical reasons — a competing drug grabs the market, or regulations and other problems make it impractical to market the drug to the intended audience.

Drug companies are always on the lookout for strategies that can cut the huge cost and risk associated with creating new drugs. In the last few years they have started applying Big Data analysis to the problem, Kelly says, taking several approaches including:

• Combing data from previous trials to identify potential problems or adverse effects on patients.

• Predicting the likely level of clinical and financial success of a new drug based on bing historical data on similar drugs to identify adverse effects on patients.

• Identifying regulatory trends including patent law related to commercial pharmaceuticals versus generic alternatives.

• Analyzing clinical data in near real time to identify similar drugs under development and adjust the trial process to incorporate the latest insights of other researchers.

Bristol Myers Squibb has been a leader in using Big Data to improve its chances of identifying successful drug candidates early. Since 1997 the New York-based pharma giant has spent nearly $46 billion to bring 11 successful drugs to market. It indexes hundreds of thousands of clinical documents per year to find important insights into important research. It recently began using HP Autonomy software to analyze this Big Data.

Kelly recommends that pharmaceutical companies and other organizations involved in complex development of new products should harness Big Data to lead the development process to maximize results without sacrificing safety or efficacy.

Like all Wikibon research, this Alert is available in full without charge on the Wikibon Web site. IT professionals are invited to register to join the Wikibon community. Membership allows them to post their own questions, tips, and research, as well as comment on published research on the site. Members also receive invitations to the periodic Peer Incite Meetings at which their peer present on how they are using advanced technologies to solve business and technical challenges.