Distilled from 405B
https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f
#Llama 3.1 8B and 70B
quasi ocean
quiet urchin
Soon hopefully on OpenRouter -> #1265324647969194117 message
glossy dome
I can already use Groq API llama-3.1-8b-instant, llama-3.1-70b-versatile free (8k context).
I cannot yet use llama-3.1-405b-reasoning, which is "available to select Groq customers only" according to https://wow.groq.com/now-available-on-groq-the-largest-and-most-capable-openly-available-foundation-model-to-date-llama-3-1-405b/
Together has base models (8b, 70b) x (Reference, Turbo) listed but appears to be not usable yet.
queen rain
thin kettle
Ooh wonder how this will perform
quiet urchin
-> #announcements message
solar sphinx
any good?
waxen vortex
Better than 3.0 70b it seems, but I still notice the repition issue
Finetunes should be pretty good though, if they can get it right.
empty oracle
Sentence repetition seem to be much less of a problem than on L3.0 70B
L3.0 fell apart after 4k, this seems to hold up to like 16k at the very least
quasi ocean
clearly 3.1 70b instruct is the best model under 10b...
looks like they put the data in wrong
InternLM2.5 is still much stronger on Open LLM Leaderboard
70B Instruct is still way lower than Qwen2 72B Instruct too
empty oracle
70B Instruct is still way lower than Qwen2 72B Instruct too
tbf Qwen is a bit of a benchmaxx
We tried to finetune Qwen2-7B and it just refused to learn
I think Qwen is a bit fried with synth
quasi ocean
ic
quasi ocean
how many tokens did they put into Qwen?
empty oracle
how many tokens did they put into Qwen?
7T I think?
Most of it being being Chinese corpora likely
I think most english stuff they put is synth
quasi ocean
7T I think?
Most of it being being Chinese corpora likely
I think most english s...
hmm, so it's not like they saturated it with more than 15Tt
empty oracle
hmm, so it's not like they saturated it with more than 15Tt
I think our set which is 50% natural data and 50% Opus synth was quite hard to learn for Qwen
But quite easy for L3
empty oracle
what were you finetuning it for
storytelling/RPing
which requires a richer vocabulary that purely synthetic data can provide
my finetune is called L3-8B-Celeste btw
dataset was like 50% short story writing (natural), 25% RP and 25% (Opus synth)
waxen vortex
Any good settings for the 70b? Is the version run by Fireworks a quant?
hushed bough
@uncut galleon DeepInfra now has 3.1 8B and 70B:
https://deepinfra.com/meta-llama/Meta-Llama-3.1-8B-Instruct
https://deepinfra.com/meta-llama/Meta-Llama-3.1-70B-Instruct
Oops, just saw it posted in the 405B one, my bad
hushed bough
Awesome, thanks!
high lintel
Has anyone compared the censorship levels between L3 and L3.1?
waxen vortex
Has anyone compared the censorship levels between L3 and L3.1?
It occasionally denies your requests, but I didn't check which providers
But otherwise it seemed very unrestricted
high lintel
seems pretty censored
solar sphinx
my finetune is called L3-8B-Celeste btw
I hear so many good things but I rlly need slightly bigger context windows 🥺 any chance u can expand it?
hushed bough
@uncut galleon The context length has been updated on DeepInfra's website for Llama 3.1. It now says 128K:
https://deepinfra.com/meta-llama/Meta-Llama-3.1-8B-Instruct
https://deepinfra.com/meta-llama/Meta-Llama-3.1-70B-Instruct
thin kettle
seems pretty censored
This is a hilarious way to test for censorship
uncut galleon
<@392529839745269760> The context length has been updated on DeepInfra's website...
thx! deploying now
@oak mango fyi^ sorry for nerfing it temporarily
oak mango
<@981296560749027410> fyi^ sorry for nerfing it temporarily
So grateful!