Introducing Carbon-X
๐งฌ
52
The best open model for DNA understanding
Michael Pieler
MicPie
AI & ML interests
ML
Recent Activity
liked a Space 1 day ago
HuggingFaceBio/Carbon-X-Release upvoted a collection about 1 year ago
ChemPile upvoted a paper about 1 year ago
Thought Anchors: Which LLM Reasoning Steps Matter?Organizations
liked a
Space 1 day ago
upvoted a collection about 1 year ago
upvoted a paper about 1 year ago
upvoted an article about 1 year ago
Article
Towards Open Evolutionary Agents
driaforall
โข โข 23 upvoted an article over 1 year ago
Article
System Prompt Learning: Teaching LLMs to Learn Problem-Solving Strategies from Experience
codelion
โข โข 26 liked a
dataset over 1 year ago
liked a
Space about 2 years ago
upvoted a collection about 2 years ago
liked 2
Spaces about 2 years ago
reacted to m-ric's post with ๐ about 2 years ago
Post
2,283
๐ง๐ต๐ฒ ๐ต๐๐ด๐ฒ ๐ฐ๐ผ๐๐ ๐ผ๐ณ ๐ฟ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐ผ๐ป ๐ณ๐ฟ๐ผ๐ป๐๐ถ๐ฒ๐ฟ ๐๐๐ ๐ ๐ธ
Google DeepMind recently released a great paper that shows optimal hyperparameters to train across different regimes: Scaling Exponents Across Parameterizations and Optimizers, with data from 10,000 training runs.
One engineer decided to quantify the price of such a large-scale experiment.
๐ฌ And the bill is hefty: ~13M USD
This exact number is to take with a grain of salt because many approximations were necessary to get the final result.
โ๏ธ But still this ballpark means that for this sole experiment, the price is way over what most startups or research labs could afford.
This means that open-sourcing research is more important than ever, to put everyone in the ecosystem on a roughly equal footing. Don't let OpenAI run first, they'll keep everything for themselves!
Read the full post that quantifies the paper's cost ๐ https://152334h.github.io/blog/scaling-exponents/
Google DeepMind recently released a great paper that shows optimal hyperparameters to train across different regimes: Scaling Exponents Across Parameterizations and Optimizers, with data from 10,000 training runs.
One engineer decided to quantify the price of such a large-scale experiment.
๐ฌ And the bill is hefty: ~13M USD
This exact number is to take with a grain of salt because many approximations were necessary to get the final result.
โ๏ธ But still this ballpark means that for this sole experiment, the price is way over what most startups or research labs could afford.
This means that open-sourcing research is more important than ever, to put everyone in the ecosystem on a roughly equal footing. Don't let OpenAI run first, they'll keep everything for themselves!
Read the full post that quantifies the paper's cost ๐ https://152334h.github.io/blog/scaling-exponents/
- 1 reply
100 word justifications from Llama3 annotations
2
#3 opened over 2 years ago
by MicPie
liked a
dataset over 2 years ago
liked a
Space over 2 years ago
upvoted a collection over 2 years ago
liked a
model over 2 years ago
authored 4
papers over 2 years ago
Stable LM 2 1.6B Technical Report
Paper โข 2402.17834 โข Published โข 3
Are large language models superhuman chemists?
Paper โข 2404.01475 โข Published โข 18
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Paper โข 2204.06745 โข Published โข 1
Few-shot Adaptation Works with UnpredicTable Data
Paper โข 2208.01009 โข Published