Record 003 · Models · 2021
50%21.03.L
GPT-J
EleutherAI
“The open 6B that proved you did not need OpenAI's permission to have a generative model.”
EleutherAI trained GPT-J-6B and released it. Together with GPT-NeoX and BLOOM it is the prehistory of the local-weights era.
Opened large open generative weights on hardware you could buy.
- Open 6B GPT-style model
- The Pile dataset lineage
- Community training
Before Llama, there was a 6B.
