Ameet Talwalkar, Evan Sparks, and Neil Conway co-founded Determined AI in San Francisco, CA.
The three founders have a combined experience of 14 years in research and industry, working on large-scale machine learning systems.
Ameet is a computer science Ph.D. candidate at Stanford University and has interned at Google DeepMind and Microsoft Research.
Evan is a computer science Ph.D. student at Stanford University who worked on machine learning systems as an intern at Amazon and Google.
Cloud-based platform with GPU cluster support for distributed machine learning training
Open-source machine learning platform specializing in deep learning model training, optimization, and management at scale with integration into high-performance computing environments
Technology details based on platform documentation and company resources; detailed architecture information not fully disclosed
| Service | Cost | Details | Source |
|---|---|---|---|
| Open Source Edition | $0 | Distributed training, hyperparameter tuning, experiment tracking, resource management. Community supported. | HPE Developer Portal and GitHub |
| Enterprise Edition | Full-featured for teams at scale, additional enterprise features, hybrid/multi-cloud support, integrated with HPE HPC and GreenLake. | Official announcements and product documentation |
| Feature | Determined AI / HPE | Kubeflow | MLflow | Ray Train |
|---|---|---|---|---|
| Distributed Training | Yes (fault-tolerant) | Yes | Partial | Yes |
| Hyperparameter Optimization | Advanced (Hyperband) | Yes | Basic | Yes |
| Experiment Tracking | Yes (central dashboard) | Partial | Yes | Partial |
| Resource Scheduling | Smart GPU scheduling | Yes | No | Yes |
| Hybrid/Multi-Cloud | Yes (any infrastructure) | Yes | Limited | Yes |
| Fault Tolerance | Yes (server failure recovery) | Partial | No | Partial |
| Starting Price | Free OSS / Contact sales | Free | Free | Free / Paid enterprise |
| Free Tier | Yes (open source) | Yes | Yes | Yes |
| Enterprise SSO/RBAC | Enterprise | Via Kubernetes | Via Databricks | Enterprise |
| HPC Integration | Native (HPE Cray) | Partial | No | Partial |
Determined ai provides stronger out-of-the-box distributed training and hyperparameter optimization than Kubeflow’s Kubernetes-centric approach which requires more configuration — better suited for teams without deep Kubernetes expertise but less flexible for custom infrastructure
Determined ai enhances productivity of ml engineers — Kubeflow is better suited for Kubernetes-native DevOps teams.
MLflow excels at tracking experiments but lacks determined ai’s distributed training and resource orchestration — determined provides end-to-end platform while MLflow is tracking-focused
Utilize determined AI for entire workflow of complete training; Utilize MLflow as a light weight tracking tool.
Both scale training across clusters but determined ai emphasizes fault tolerance and HPE HPC integration — Ray offers broader ecosystem for distributed computing but steeper learning curve
Utilize determined AI for HPC/ML focused teams; Utilize ray for the general use of distributed python.
W&B dominates at visualizing/tracking experiments — lacks training infrastructure — determined ai provides full platform while W&B’s focus is on observability for MLOps
Utilize determined AI as the total solution; Utilize w & b for the additional advanced visualization capabilities.
Determined AI is an open source platform which makes it simple to run distributed deep learning training using PyTorch and TensorFlow. Determined AI handles the automation of hyperparameter search, the tracking of experiments and the management of GPU resources with no changes to your model code. The user submits experiments via web interface, command line interface or application programming interface.
Determined AI Core Platform is Open Source and Free. HPE offers commercial support and integration with HPE HPC Systems. Pricing available from HPE Sales for managed deployments of the Determined AI.
Determined AI has a focus on deep learning training and includes native distributed training capabilities along with Hyperband for hyperparameter search and native experiment tracking. Kubeflow is a general machine learning platform and will require more knowledge of Kubernetes than Determined AI. Determined AI can be deployed with minimal setup of your infrastructure.
Yes, Determined AI can be used both on premise or in your cloud account with complete control over your data. Includes TLS, Docker Isolation, and Enterprise-Grade Security via HPE Integration. Completely open source so you are never locked in to a vendor.
Yes, Determined AI exposes a RESTful API and a Python SDK for programmatically submitting experiments. The CLI can be integrated directly into your pipelines. Determined AI also supports all of the major cloud providers (AWS, GCP) and on premise HPC Clusters.
Yes, Determined AI has an active Slack Community for its open source users. There is extensive documentation and workshops provided. HPE provides commercial support for its commercial customers.
Yes, Determined AI can be deployed immediately with either 'det deploy local cluster-up' or Docker Compose. No sign up is required to use the open source version of Determined AI. HPE Developer Portal offers free workshops-on-demand.
Development can occur using a single node workstation with a single GPU/CPU. For production, multiple nodes are required with NVIDIA GPUs. InfiniBand/100GbE is recommended for multi-node training. Agent configuration is supported by shared memory optimizations.
Determined AI, which is now an HPE company, offers enterprise-level AI training systems that are relatively easy to set up. This combination of the open-source core of Determined AI and the HPC expertise of HPE will provide a powerful system to deploy large-scale AI applications. Determined AI has a strong focus on increasing researcher productivity as opposed to minimizing DevOps overhead.
Determined AI is best suited for deep learning teams that are serious about training at scale and are working in HPE environments.
Determined AI provides production-ready distributed deep learning training with auto-hyperparameter tuning, experiment tracking, and GPU-optimized performance. With its acquisition by HPE, Determined AI can integrate with enterprise HPC infrastructures while still being available as an open-source solution. Determined AI includes Python SDKs and REST APIs to facilitate a wide range of user interactions.
Good - comprehensive technical documentation from official sources and HPE Developer Portal. Limited commercial pricing/support details available publicly.
Determined powers the HPE Machine Learning Development System (ML/D). Optimized Docker Images (hpe-mlde-master/agent) for HPE HPC systems with InfiniBand, NVIDIA GPUs, and RHEL 8.5.
There are many active GitHub contributors (determined-ai/determined), with over 3k stars. Determined AI also hosts a Slack channel for support. They regularly release new versions that update the environment with PyTorch and TensorFlow.
Training can occur on local clusters, AWS/GCP, Kubernetes, Slurm/PBS. Determined's CLI makes it simple to deploy multi-cloud with just one command: det deploy aws up. Also, Determined's web interface allows for easy experimentation, and seeing how much each resource was utilized during that time.
HPE offers complimentary Jupyter-based workshops in the HPE Developer Hack Shack. Hands-on tutorials cover topics such as distributed training, hyperparameter tuning, and managing clusters.
Founded in 2017, Determined AI became open source in 2020. As of approximately 2023, Determined AI has been acquired by HPE, and its functionality is now being used in HPE's GreenLake AI offerings for HPC/ML workloads in large enterprise environments.
With Determined, users can achieve high-throughput parallel model training at scale, without needing to make any modifications to their existing code, thanks to Determined's built-in accelerator scheduling and fault-tolerance capabilities.
Determined's Hyperband-based automatic tuning enables users to automatically find the optimal set of hyperparameters for their models, and integrate these into the user's training workflow.
Determined optimizes accelerator utilization on both on-prem and cloud environments, including spot instances.
This tool tracks versions of code, metrics, checkpoints, and hyper-parameters to produce reproducible results.
| Framework | Core Optimization | Primary Use Case | Hardware Support | API Type | Multi-Tenancy |
|---|---|---|---|---|---|
| Determined AI / HPE | Distributed training + hyperparameter optimization | End-to-end ML training to deployment pipeline | HPE HPC, NVIDIA GPUs, Cloud/Hybrid | HPE MLDE + KServe integration | Enterprise-grade via HPE GreenLake |
| vLLM | PagedAttention with continuous batching | Open baseline for chat/completion workloads | NVIDIA GPUs (primary), AMD emerging | OpenAI-compatible REST API | Strong support via Ray Serve |
| TensorRT-LLM | Kernel fusion + quantization (FP8/INT8) | Maximum NVIDIA-specific optimization | NVIDIA GPUs exclusively | Triton Inference Server API | Production-grade with model ensembles |
Deployment can be based on consumption and can be done on a customer's premises using HPE hardware or in a public cloud environment.
Customer managed workstations and clusters are provided to customers that have been optimized for HPC workloads utilizing the Determined AI software stack.
The tool automates every aspect of the workflow process from data processing to KServe deployment as well as providing reproducible workflows.
Cloud cost optimization is achieved via the seamless use of spot instances with no additional complexity to the operations of an organization.
Automatic checkpointing and recovery are available for distributed training workloads.
Tracking of code, metrics, hyper-parameters, and model checkpoints is included out-of-the-box.
Version controlled experiments allow teams to collaborate and provide auditability of their experimentations.
Enterprise grade management of infrastructure is offered with consumption based economics.
Retrain and deploy automatically when new data arrives.
Spot instances and hybrid clouds can be orchestrated seamlessly.
