Accessible Labs Ltd., which does business as cnvrg.io, today released a dashboard that can be used by developers of machine learning models to optimize their use of server resources.

The Jerusalem-based startup said its ML Infrastructure Dashboard can initially be used to give development managers a look at utilization of central processing units, graphics processing units, memory and cluster nodes for workloads running in software containers, which are portable environments that include workloads and all the resources need to run them.

The next release, which is targeted for the first quarter of next year, will analyze workloads and recommend an allocation of resources to optimize them, said Chief Executive Yochay Ettun. Although virtualization has helped information technology organizations dramatically drive up server utilization rates, such efficiencies have yet to come to the typical machine learning developer. Ettun said.

“When a customer is buying an Nvidia server, they are getting a black box — they can’t see how many resources are being used,” he said, adding that Nvidia has estimated that less than 30% of its server resources are typically utilized.

Among the reasons are that configuring software containers and estimating processing needs for large data sets requires specialized skills that not all organizations have in house. “For example, I may allocate two GPUs to a model, but I only use one because I don’t have enough data or the code is not optimized properly,” he said.

GPU optimization is particularly important since a single GPU server can cost $5,000 and add-ons and enhancements can more than double that price. Lack of visibility can not only reduce developer productivity but prevent resources from being used by other jobs running on the same machine. The dashboard can potentially increase GPU/CPU and memory utilization rates to 70%, the company claimed.

In addition to providing a snapshot of current resource usage, the dashboard shows how much capacity is going unused as well as the amount of allocated resource that is being underutilized. It uses the Prometheus open-source monitoring and time-series database and data from machine monitors to assemble a single view that shows active jobs with information about users, projects and containers. Metrics can be exported to external data analytics platforms like Tableau Software Inc.’s Tableau and Excel spreadsheets.

Cnvrg.io sells a full-stack data science platform for managing and scaling AI development. The company raised $8 million last fall.

There is a free community version and a paid enterprise version. Cnvrg.io wouldn’t reveal pricing.

Image: Pixabay