Ai2 has opened the training stack it used to run a 1.2-trillion-parameter model across 512 GPUs
The scarce thing in open AI research is not model weights. It is the distributed training code that makes a very large sparse model trainable at all.
Company
Allen Institute for AI
Allen Institute for AI is an organisation working in artificial intelligence, founded in 2014 and based in Seattle, Washington, United States. This page collects every Aivio News story about Allen Institute for AI, together with its models and recorded benchmark results.
- Founded
- 2014
- Headquarters
- Seattle, Washington, United States
- Official website
- allenai.org