fshy

Home / Compute

ICML 2026 Agent

howaiworks.ai/blog/icml-2026-agent-reproduction-challengeCloud VMcompute/icml-2026-agentJSONcomparereport an errorprovider JSON

Free allowance
A conference-wide reproduction challenge that pays in GPU credits rather than cash.
Verdict
triala time-boxed or one-time trial, not a recurring allowance
Cost
free tier
Identity
emailno card, no phone
Binding limit
write-gatedthe essential gate is scarcity rather than eligibility: the onboarding credit slots were exhausted within days of launch and the remaining allowance was reserved for the organising group, so a new entrant works on their own compute for...
Evidence
reported4 sources; a third party said so
Freshness
stale on 2026-11-19checked 2026-11-04 (live); stale on 2026-11-19
Free tierA conference-wide reproduction challenge that pays in GPU credits rather than cash. Quoted: 'A community challenge on Hugging Face has AI coding agents reproduce claims from every ICML 2026 paper, with $4,000 in GPU credits for the best runs.' The prize split, quoted: 'First place: $2,000', 'Second place: $1,000', 'Two runner-ups: $500 each'. The onboarding credit, quoted: 'The organizers also offered free GPU-credit slots to help people get started, but noted that those slots were fully claimed within days of launch, with remaining credits reserved for existing members of the organizing group. The prizes themselves stay open to everyone.' Window, quoted: 'July 15 to August 2, 2026'.
unit none; metered on wall-clock; prize
Pricenone stated
Limits
  • write-gated (binding) the essential gate is scarcity rather than eligibility: the onboarding credit slots were exhausted within days of launch and the remaining allowance was reserved for the organising group, so a new entrant works on their own compute for a prize that pays in credits. source
Identityemail not published
Egressnot published: no egress allowance or overage price on https://howaiworks.ai/blog/icml-2026-agent-reproduction-challenge, read 2026-11-04
Interfaceunknown not published: the page describes a prize challenge on Hugging Face and names no REST API, CLI, SSH or SDK for the credits
Toolingnot stated
Composes withUSD 20 per participant across 750 slots is USD 15,000 of credits distributed before the prizes, which is three times the prize pool: the participation credit was the real award and the prize is a rounding error on it.
Reliabilityno reliability event recorded
NotesRecorded at reported rather than documented on purpose: the amounts come from two third-party write-ups, and the Hugging Face org page confirms only that the programme exists. The organisation's own README could not be reached: the org profile returned 200 with activity only, and the challenge Space returned no extractable text, so the primary page for the numbers was not obtained. The interesting property is that the onboarding credit, not the prize, is where the capacity sits, and it ran out in days, which is the same pattern as the AIcrowd credits and the AIMO pool.
Parsed numbers
  • has-free-tier 1
Evidence 4
datesourcewhat it proved
2026-10-22howaiworks.ai/blog/icml-2026-agent-reproduction-challenge'$4,000 in Hugging Face GPU credits'; 'First place: $2,000'; 'Second place: $1,000'; 'Two runner-ups: $500 each'; 'those slots were fully claimed within days of launch, with remaining credits reserved for existing members of the organizing group'; 'July 15 to August 2, 2026'; page 200
2026-10-22huggingface.co/ICML-2026-agent-reprothe organising org exists on Hugging Face with an open 'Request to join this org' control and recent activity, which confirms the venue but not the amounts; page 200
2026-10-22www.notatechguy.com/icml-2026-23-of-papers-have-a-falsified-a second independent account: 'Participants received $20 in Hugging Face compute credits and launched 2,962 cloud jobs'; 'They confirmed 3,978 individual claims with real experiments'; read through the search index for notatechguy.com
2026-11-04howaiworks.ai/blog/icml-2026-agent-reproduction-challengethe page offers this and its text carries no traffic-out term, allowance or price
More fields 1
facets{"class": "benchmark-programme","free_tier_kind": "prize","shape": {"vcpu": null,"ram_gib": null,"gpu_class": null,"gpu_count": null}}
Unknown values 6
  • egress metered: egress metering was not published
  • interface: the access interface was not established
  • per object cap stated: the field does not apply to this kind of offer
  • reliability note: nothing of this kind was observed
  • retention stated: the field does not apply to this kind of offer
  • tooling: a source was read and is silent on this field