#Looking for collaborators on a brain-inspired SNN language model (618M params, open-source, used Go

1 messages · Page 1 of 1 (latest)

frail panther
#

Looking for collaborators on a brain-inspired SNN language model (618M params, open-source, used Google Cloud TPU)

Hey everyone! I'm building Nord — an open-source Spiking Neural Network language model trained from scratch. I used Google Cloud TPU v4 through the TRC Research Program for part of the training, and I'm looking for developers and researchers who want to collaborate on pushing this further.

The problem I'm solving:
Current LLMs activate 100% of neurons for every token. That's extremely wasteful. Biological brains activate ~5-15% of neurons at any time. Nord does the same — 87-93% of neurons stay silent, and different zones self-specialize during training without any manual programming.

Where I am now:
— 618M parameters trained from scratch (no distillation, no teacher model)
— Loss 3.65, instruction-tuned on OpenHermes 2.5
— Self-organized brain zones confirmed at scale
— First SNN with chat-style instruction following
— All code open-source, Apache 2.0

Where I need help:
— TPU optimization (PyTorch XLA) — ran into CUDA/XLA conflicts during training, would love advice from anyone experienced with TPU pods
— Neuromorphic inference — porting spike-based models to Intel Loihi or BrainChip Akida
— Benchmarking — setting up proper eval (MMLU, HellaSwag, HumanEval) for a non-standard architecture
— Paper writing — preparing a NeurIPS 2026 submission, could use feedback from anyone who's published at ML conferences

Links:
GitHub: https://github.com/gtausa197-svg/-Project-Nord-Spiking-Neural-Network-Language-Model
HuggingFace: https://huggingface.co/zerdovzad/Nord-AI
Website: https://www.nord-ai.net/

If you're interested in neuromorphic computing, energy-efficient AI, or brain-inspired architectures — I'd love to connect. Happy to answer any questions about the architecture or training process.

Thanks for reading!

image.png
grand shard
# frail panther Looking for collaborators on a brain-inspired SNN language model (618M params, o...

Hello, I saw your message.
I think this is seriously impressive - especially getting an SNN to 618M params with instruction tuning.
I am a full stack + AI dev with experience in LLM pipelines, APIs, and scalable systems. Ive also worked with FastAPI, PyTorch, and deployment workflows, so I can help on infra, benchmarking or integration.
The sparse activation + self-organizing zones approach is really interesting.
I am curious how you are handling evaluation vs standard transformers.
Happy to contribute if you are open.

frail panther