Hugging Face's logo
AbstractPhila's picture
🤝 Open to Collab

AbstractPhila PRO

AbstractPhil
AbstractPowered
19 6 34
ironbcc's profile picture aelitis's profile picture Thomas2895's profile picture
·
https://civitai.com/user/AbstractPhila

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

updated a model about 3 hours ago
AbstractPhil/qwen-symbolic-thinking-image-training
updated a model about 13 hours ago
AbstractPhil/qwen3.5-9b-think-symbolic-image
repliedto their post about 13 hours ago
A universe in an atom, a 9m rotary prototype with an absurd context length. https://huggingface.co/blog/AbstractPowered/rotary-addressed-selection I've coined the term Rotary Addressed Selection, at least until I discover if someone else has made it first. This rotary can handle an absurd length of context window, though the technical documentation is quite complex. Targeting context window after 1 million was how the addressing corruption was solved. The rotary system involves including a full address for every 4096 context window lookup state. This allows the model to differentiate between a multitude of modular model extensions. How does that work? Well if there's nothing at the address, then there's no return. Addresses are essentially selectors promoting the cooperative state and it does have some limitations, but the full global rotary supports DIRECT dynamic extension to the model at runtime. Need only a simple classifier? The selectors know which experts classify and which state they exist in. Need a full VLM? Load up the experts. You get the idea. Query with the address, get the correct addressing. That's how it works. The model learns to handle everything internally, so there is no need for a large structural addressing screening beforehand. With this development, full mixture of experts with uncertain (absurdly long) context depth can have a consistent context window guarantee. There is bound to be some direct breakdowns and numerics that can't simply reach beyond certain points likely based on hardware or rounding differences, but the system essentially allows full MOE where experts are selectable and the selected experts are a guaranteed addressing. The upcoming experimentation will fill in the gaps for the test cases, confirm or refute clauses that are established on the rotary, and then training a functional Alhena at the size of the mini-beatrix scale with some tweaks for the changes.
View all activity

Organizations

DeepGHS's profile picture Blog-explorers's profile picture BangumiBase's profile picture Abstract Powered Research's profile picture