Home / Compute
MLPerf
mlcommons.org/benchmarks/Cloud VMcompute/mlperfJSONcomparereport an errorprovider JSON
- Free allowance
- MLPerf awards no compute at all.
- Verdict
- fails: identitythe identity required is not reachable without it
- Cost
- no price stated
- Identity
- institutionno card, no phone
- Binding limit
- write-gatedtwo gates stack: the Training and Inference working groups require organisational membership, and even a non-member submission needs a signed test agreement and a trademark licence before a result can be published, so the constraint...
- Evidence
- documented3 sources; a first-party page was read and the sentence quoted
- Freshness
- stale on 2026-11-22checked 2026-10-22 (live); stale on 2026-11-22
| Free tier | MLPerf awards no compute at all. The programme is a measurement, and the submitter supplies the system, quoted: 'The working group defines the AI model to run, the data set against which it gets run, sets rules on what changes to the model are allowed, and measures how fast a given hardware runs the model.' The gate is membership, quoted: 'Membership is required for most benchmark working groups (e.g., Training, Inference, Mobile). There are some public benchmark working groups that have no access requirements where non-members may submit to the benchmark by first signing a Non-member Test Agreement.' The submission requirements list, quoted: 'Join an appropriate MLCommons Working Group', 'A signed Contributor License Agreement', 'Membership in MLCommons OR a signed Non-member Test Agreement', 'A signed trademark license agreement'. unit none; metered on wall-clock; none |
|---|
| Price | none stated |
|---|
| Limits | - write-gated (binding) two gates stack: the Training and Inference working groups require organisational membership, and even a non-member submission needs a signed test agreement and a trademark licence before a result can be published, so the constraint that fails first is the membership, not the hardware. source
|
|---|
| Identity | institution not published |
|---|
| Egress | not stated |
|---|
| Interface | unknown inferred from the tooling the provider documents; not probed |
|---|
| Tooling | not stated |
|---|
| Composes with | Nil. The useful product of the record is the exclusion: a reader who sees MLPerf results quoted as free capacity should be told that the numbers are vendor submissions, and that no MLCommons programme pays for the machine under test. |
|---|
| Reliability | no reliability event recorded |
|---|
| Notes | MLPerf is not a compute-awarding programme. The membership sentence could mean that an individual can join, that a company pays dues, or that non-members are fine through the test agreement. The page's own 'Membership and Affiliate' FAQ and the phrase 'if you do not think your organization would be a good fit as a Member' in the GitHub route point at the company-dues reading, and the test-agreement reading is true but ends in a signed agreement that a solo entrant can sign. So the honest verdict is fails-identity for the main suites and a narrow non-member path for the public groups; the hardware cost stands either way. |
|---|
| Parsed numbers | |
|---|
Evidence 3
| date | source | what it proved |
|---|
| 2026-10-22 | mlcommons.org/benchmarks/ | 'Membership is required for most benchmark working groups (e.g., Training, Inference, Mobile)'; 'There are some public benchmark working groups that have no access requirements where non-members may submit to the benchmark by first signing a Non-member Test Agreement'; 'Membership in MLCommons OR a signed Non-member Test Agreement'; 'A signed trademark license agreement'; page 200 |
| 2026-10-22 | mlcommons.org/benchmarks/inference-datacenter/ | the suite page describes measurement only ('measures how fast systems can process inputs and produce results') and names no grant, credit, allowance or hardware pool; page 200 |
| 2026-10-22 | mlcommons.org/about-us/policies/ | the policies page offers three agreements, 'Membership Agreement', 'Membership and Affiliate FAQ', 'Non-member Test Agreement', which is the only route for an unaffiliated submitter; page 200 |
More fields 1
| facets | {"class": "benchmark-programme","free_tier_kind": "none","shape": {"vcpu": null,"ram_gib": null,"gpu_class": null,"gpu_count": null}} |
|---|
Unknown values 7
- egress metered: egress metering was not published
- egress terms: a source was read and is silent on this field
- interface: the access interface was not established
- per object cap stated: the field does not apply to this kind of offer
- reliability note: nothing of this kind was observed
- retention stated: the field does not apply to this kind of offer
- tooling: a source was read and is silent on this field