Primate Labs Inc., the maker of cross-platform benchmarking software, Thursday officially released Geekbench AI 1.0, a tool designed to test the real-world performance of artificial intelligence workloads on mobile and desktop platforms.

The tool, previously known as Geekbench ML, has been renamed to better fit the industry standard naming convention although it does cover machine learning and deep learning workloads alongside AI. Its prior preview releases were developed over years of work with the company’s customers, partners and the AI engineering community.

 Instead of a single “score,” the new software breaks down the scoring into three metrics that cover the complexity of scoring performance.

“Measuring performance is, put simply, really hard,” the company said. “That’s not because it’s hard to run an arbitrary test, but because it’s hard to determine which tests are the most important for the performance you want to measure – especially across different platforms.”

The three-score system provides different ranges of precision, which may matter when providing a particular type of AI task. This will allow AI developers and engineers to understand how their app will work on the hardware on a more granular level than having a single score, the company said: “With our benchmark, you can explore how different approaches at the hardware level have been optimized for particular tasks.”

The new tool also includes workload accuracy measurements on a per-test basis. That’s because AI involves not just running fast, but doing so with a low error rate. For example, an image recognition model looking for cats might run quickly, but if it detects cats only 1% of the time, it’s not doing a great job. This measurement lets developers see how well their model manages on different types of devices and allows them to fine-tune it.

The new software supports a variety of AI frameworks including OpenVINO on Linux and Windows, and TensorFlow Lite vendor-specific delegates such as Samsung ENN, ArmENN and Qualcomm ENN on Android. That allows it to best fit the software and hardware that engineers and developers currently build their apps on.

The tool is available for Windows, MacOS, and Linux, as well as on the Google Play Store and Apple App Store for mobile devices. Geekbench AI is also integrated with Geekbench Browser, which allows users to submit AI benchmarks for cross-platform context and comparison.

Images: Primate Labs, Pixabay