Magic Research LLC, a holding company for commercial artificial intelligence research and software development, has launched an artificial intelligence platform for on-premises use that it claims can cut costs by up to 90% from comparable cloud-based services.

Called Private AI, the platform is intended to give organizations complete control over their data, infrastructure and brand by operating securely behind a firewall and leveraging existing computing resources.

The proprietary technology underlying Private AI is Fabric Hypergrid, a distributed computing mesh that taps into existing hardware on a network, including legacy graphic processing units, central processing units and accelerators, to create the equivalent of an AI supercomputer at a small fraction of the cost.

Hypergrid can “shard neural networks during the inference process,” said Humberto Farias, founder and chief executive of Magic Research. Sharding is an architecture originally created for database management systems that breaks monolithic databases into smaller, faster, more manageable pieces called shards that each hold a subset of the dataset.

Dynamic workload routing

During the inference process, a dynamic model router analyzes the AI task and matches it with the best available computational resource within the private network. The Hypergrid layer then orchestrates the workload, performing model acceleration and distributing shards of the neural network across the available hardware before reassembling the results.

“We’re able to send a piece of the neural network to a specific CPU and send another one to a GPU, and another one somewhere else, and then get everything back together to return the results,” Farias said.

Magic Research is addressing what it says is huge demand for AI models that guarantee data protection. All prompts, documents and logs remain within an organization’s infrastructure, whether on-premises or in a private cloud. The company cited a recent Cloudera Inc. survey that found that more than half of enterprise leaders are concerned about data privacy when adopting AI from external sources, fearing leaks, unauthorized access and compliance violations.

Magic Research says its technology is well-suited to companies in highly regulated or high-stakes environments, such as legal, healthcare and government agencies. Farias said it already has “a few dozen” paying clients.

The model-agnostic platform provides a white-labeled AI chatbot experience with customizable user interfaces, workflows, models and permissions. A component called GatewAI, enforces policies, filters content, logs activity and ensures alignment with industry-specific regulations like FERPA, HIPAA, GDPR and SOC 2.

The platform is priced based on the size of a client’s infrastructure, starting at $5,000 per month.

Magic Research is a spinoff from Cross Ventures LLC, a holding company for technology, energy, mobility and AI firms. The Private AI platform was built by a team of about 20 experts in collaboration with institutions that include Intel Corp. and Harvard Medical School.

Farias said the company has gained strong early traction and anticipates profitability in its first year on between $2 million and $3 million in revenue. Currently self-funded, it’s likely to seek external funding in the near future, he said.

Image: Interwork/Facebook