FSRS Megathread

[ashen light]

https://github.com/ankitects/anki/pull/3762 this one

[bold terrace]

[Dropping now, I'll check the rest tomorrow]

[cursive badge]

When did you learn Rust Expertium?. Shhhhhhhh 🤫

[unique salmon]

@bold terrace the more I think about it, the more I realize that making the fuzz factor depend on stability intead of the interval length is a bad idea
Aside from the fact that it means that fuzz works differently for FSRS and SM-2, the more important part is that it's bad at high values of desired retention. At high DR your intervals are very short relative to your stability, so basing fuzz on stability would mean that the fuzz itself is too big

Suppose that your interval is 5 days and your S=500 days. 5% of 500 is 25 days. So your 5d interval could end up being changed by 25 days

So no, let's keep fuzz based on the interval length

Basing it on stability would achieve the opposite of what you want - which is making sure that retention isn't negatively affected by fuzz

And the higher you set your DR, the worse the effect of basing fuzz on stability would be

[unique salmon]

Alright, that's an unrealistic example. Here's a realistic one
DR=99%. S=100. Interval=9 days. 9+0.05*100=14 days. So a 9 day interval would become 14 days. At DR=99% that would screw your retention up.

[spring adder]

a degenerate in the fsrs thread

Can we do this as like a flair or a role or something? I've never felt more seen.

[cursive badge]

With no context 🤣

[spring adder]

low key want this on my tombstone

[bold terrace]

Ah yes indeed, if DR is greater than 0.90, then Stability is bigger than Interval. And indeed, the slope of the forgetting curve is more aggressive close to 100%, so your 99% might become a 96% in a few days, which is also problematic

Well for the last point no, a 100d for .99 is 9d , for .98 a 18d, for .97 27d

So in this example the drop to wait 14d instead of 9d would lead somewhere between .99 and .98, so in absolute value, only a 1% diff, but relatively, an increase of chances to forget of 100%

Hmmmmm but yeah, changing this require to be careful

doing some simulation/graph to see what is the potential loss of retention, comparing interval-based fuzz or stability-based fuzz could be interested

[severe storm]

What's the best of these options to clear a huge backlog?

I already changed to study due cards first than news, and it helped me to clear my backlog

[severe storm]

I suppose it's descending intervals?

[spring adder]

The best order for living with a backlog is Descending retrievability (according to @unique salmon ) or Easy cards first (according to me, a nobody with strong opinions) ||#general message||

Using either of those two should reduce the impact of maintaining a backlog while you work through it.

[severe storm]

Thanks

[rotund summit]

What about for general use/optimal retention? I thought I had read that it was also desc retri but wasn't sure

[unique salmon]

If you can review all due cards, sort order doesn't matter

It only matters if n(due cards) > n(cards that you can/want to review)

[robust hill]

but you must take in the psychology factor

[unique salmon]

Can't really simulate that

[hasty fractal]

can anyone remove the y axis titles from forgetting curve and simualtor?

Ref: https://github.com/ankitects/anki/pull/3774#issuecomment-2629460596

For forgetting curve: https://github.com/ankitects/anki/blob/main/ts/routes/card-info/forgetting-curve.ts

@cosmic hedge what do you think of axis titles in simulator? imo should be removed.

(text overlap issue)

[cosmic hedge]

yeah I suppose it makes sense to remove them because I doubt anyone is going to be confused. the radio buttons are still there telling people what's plotted.

idk if its a huge deal either way though.

[hasty fractal]

in any case, will appreciate someone doing it and sending a PR

[cosmic hedge]

hint taken 😂

[tough stratus]

Any clue why the number of reviews projected for the next day on the review heatmap are always different than the actual number of reviews? It’s usually lower be around 20-30 reviews

[hasty fractal]

do u mean future due graph?

it's probably because of the same day reps

[tough stratus]

Yes it looks like they’re both the same number

[tough stratus]

What do you mean by that

[cosmic hedge]

the future due graph doesnt count any reviews that are done twice on the same day

[tough stratus]

Hmm, I mean that when I wake up the next day the total amount of reviews is higher than projected — not that the actual number of reviews I do is higher. The latter makes sense to me

[cosmic hedge]

are you sure you did all of the previous days intra-day reviews?

[tough stratus]

Yes (usually)

[cosmic hedge]

so long as you don't have a backlog idk what could be causing that
maybe take a picture of your future due after you're done today and a picture of your due cards and future due before you start tomorrow

[unique salmon]

If anyone wants to help with the script for AnKing's upcoming video on FSRS, here: https://github.com/open-spaced-repetition/fsrs4anki/issues/719#issuecomment-2631882749

Anyone who has a good understanding of FSRS features and options and can express it in a beginner-friendly way is welcome

[cosmic hedge]

https://github.com/ankitects/anki/pull/3789 there you go

[lapis hearth]

I want to share something: I was wondering why my same day learning has been weird lately and I discover lately that almost all my decks have their last two fsrs params as 0

I have manually set the last two as the last two from the default values

And I was wondering why I am having trouble memorizing lapsed cards in the short term lately

I get that optimised values are always better than default ones

But I need to make sure those last two values remain non-zero

It is for my own short term learning

[unique salmon]

A new check in 25.02.
Basically, it detects whether using same-day reviews to adjust memory stability could result in a situation where your next interval after a lapse is longer than before. For example: 10 days -> you press "Again" -> you have an insane number of re-learning steps -> you do them, S increases -> next interval is 15 days
If the optimizer detects that your re-learning steps and parameters would result in that kind of problem, the optimizer will run for the second time, but the last two parameters will be "frozen", meaning that same-day reviews will have no impact on S

[lapis hearth]

I did not exactly understand. Is this a bug that is to be fixed and I am right to have my concerns?

[unique salmon]

Btw @quasi shadow I asked on Github, but neither you nor Dae answered: have you guys tested that the whole "run the optimizer twice" thing works correctly with "Optimize all presets"?

[unique salmon]

No, it's a crutch to help in cases where same-day reviews result in unintuitive behavior

The next interval after your pressed "Again" shouldn't be longer than before

Like 10 days -> Again -> 15 days

[lapis hearth]

What do you suggest I do at this current time

Because I need to have my short term scheduling function normally (non-zero)

Should I just take default values and put them there

[unique salmon]

I would recommend just leaving the parameters as they are

Btw, what are your re-learning steps?

[lapis hearth]

Yeah but what would then happen to the short term scheduler

[lapis hearth]

2s 25s same as my learning steps

[unique salmon]

S won't increase/decrease when you do same-day reviews

So you basically get FSRS-4.5

(well, with recency weighting)

[lapis hearth]

And learning steps stay rigid??

[unique salmon]

They stay whatever you want them to be

[lapis hearth]

No but that is the problem. I need the short term scheduler to stay on no matter what

I need it to automatically determine what is the next learning step according to how I rate the card

[unique salmon]

But you don't leave the field blank

So why do you care?

[lapis hearth]

Because that never works. It gives some really shit intervals (2h as again). I have found that the best solution is having custom learning steps and the auto short term scheduler turned don

Keep in mind I have the hidden console code thingy turned on

[unique salmon]

Which thingy?

[lapis hearth]

Jarrett has a made a special code to turn on short term scheduling whilst having custom learning steps

The intervals I am getting look good and plausible

[unique salmon]

Oh goood

I don't really remember that stuff

I remember that I was like "why is everything so complicated..."

[lapis hearth]

He hid that, partly because of you, since you were all about making things seem less complicated to the typical user and leaving complicated shit to the console

[lapis hearth]

Yes exactly. So he hid that in the console

So I am using a "different" FSRS

[unique salmon]

Ok, well, I don't know how that works, so ask @quasi shadow

[lapis hearth]

And these last two params are doing me in

Just to be clear, with these last two params being zero, short term scheduling (<1d) is non-present and the best I would be getting is a non changing (<1d) interval amirite?

Hold on wtf is this

This is "Optimised" with Zero params at w18 and w19

This is me manually changing the w18 to w19 to default params (rest of params are optimised auto)

So clearly this is effing me over

@quasi shadow @unique salmon

Here are the optimised params

[unique salmon]

As I said, if it can result in the post-lapse interval being longer than before the lapse, the last two params will be set to zero anyway, regardless of metrics

[lapis hearth]

Optimised params but with defaul values at w18 and w19

[unique salmon]

So if something like 10 days -> Again -> (your re-learning steps) -> 15 days can happen, the last two params will be set to 0

[lapis hearth]

I dont want it to be set to 0 though

[unique salmon]

Well, maybe Jarrett will have some idea

[lapis hearth]

Reason being this is happening to ALLLL of my decks

Like literally ALL of them

Basically rendering FSRS 5 as FSRS 4.5 for me like you said

So like wtf

Again look at this. This is with 0 at w18 and w19

This is with default at w18 and w19

I am doing this with all of my decks and the result is the same

0 values at w18 and w19 is shittier than default values

[lapis hearth]

@unique salmon since when was this 0 appearing at w18 and w19 occurring? I am trying to see something

Did it happen before, with or after the recency update

[unique salmon]

Since 25.01

Actually, it was introduced in this pre-release: https://github.com/ankitects/anki/releases/tag/25.01rc1

But NOT in this one: https://github.com/ankitects/anki/releases/tag/25.01beta1

[unique salmon]

@quasi shadow would it be possible to do the math "backwards" - if the current parameters result in the "post-lapse interval is longer than the previous interval" problem, instead of setting the last two params to zero, calculate the smallest values of parameters that do not result in that problem?
That should result in better RMSE and logloss

I haven't tried to work it out myself, but it seems like it should be possible

[tough stratus]

Today had 10 more than projected

[rotund summit]

Is my understanding correct that log loss and rmse decrease/get more accurate the more reviews you feed it in total (assuming you rate truthfully)? I don't understand how I'm seeing people who consistently do more reviews than I do (and presumably for longer) yet they have higher rmse and log loss than I do

[unique salmon]

RMSE does go down (to a point) as the number of reviews increases. Log loss - not so much.
RMSE is correlated with the number of reviews, log loss is correlated with retention

[cosmic hedge]

I can't tell from this because i'd need the due from before you did your reviews today.
The future due graph doesn't take into account new cards, could that be it?

[tough stratus]

That was yesterday after all new cards were learned

[cosmic hedge]

I'd like to compare the due of a day before you do your cards with what the future due graph said you would have to do yesterday

where can I see the 10 extra cards from those images, sorry?

[tough stratus]

I started reviewing before I could get a screenshot this morning unfortunately

[cosmic hedge]

you could load a backup to check but its probably best just to try again tomorrow. I doubt anyone could debug it from those screenshots alone but at least it would show what your problem is.

[tepid spoke]

I feel like I got an idea why FSRS and that WK deck aren't friends.
There's a lot of cards I don't answer via memorization, but just correctly "solve" them each time, since their meaning or reading is obvious from the Kanji.

Which then leads to the cards that are too irregular for that to work getting way too long intervals for actual memorization to work.

[hasty fractal]

two dotted lines

[quasi shadow]

It requires to input the number of steps to the optimizer.

[lapis hearth]

Hey Jarrett. I was wondering if you had a look at the 0 problem at w18 and w19

[quasi shadow]

fixed in https://github.com/ankitects/anki/pull/3788

[unique salmon]

But why can't you do that? I mean, I haven't looked at the code, but I presume the optimizer uses the number of steps for the check?

[quasi shadow]

It doesn't.

[unique salmon]

Then how does the check work??? 🤔

[quasi shadow]

It’s done in the Anki part.

Not the side of optimizer.

[unique salmon]

https://github.com/open-spaced-repetition/SSP-MMC-FSRS?tab=readme-ov-file#ssp-mmc-fsrs
Some news regarding SSP-MMC and the efficiency of different scheduling policies:

1. We tried constant interval lengths, specifically, 3 days, 7 days, and 30 days. All three are terribly inefficient when compared to both SSP-MMC and fixed desired retention. Of course, this is a simulation, it's not a perfect representation of real life, but the difference is so damn big that there is no doubt that fixed intervals suck.
2. The SSP-MMC matrix fails to converge for around 25% of users, so even if we reeeeaaaallly wanted to implement SSP-MMC (which is only marginally more efficient that fixed DR and only if you use Anki for a decade), we can't. There are too many users for whom it doesn't work.

[spring adder]

This is the worst day. If only progress were an arrow flying straight.

||add it to the Helper Addon for people who are sick in the head like me||

Nice to see fixed intervals getting absolutely shitted on, though.

[unique salmon]

About interval=3 resulting in lower total knowledge than interval=7: this is because in this simulation there is also a max. studying time per day constraint, which is set to 360 minutes. I assume that no human being will spend more than that per day on flashcards.

And at interval=3, you would eventually have to study for more than 360 minutes per day

With these settings (deck size, new cards per day, all that stuff)

[spring adder]

I know it's not something we have the information to simulate, but I would imagine that IVL=3 would perform marginally better in reality, because the amount of time spent per card would probably reduce if you were seeing it every 3 days.

[unique salmon]

Yeah, right now we assume constant answer time per button

[polar maple]

do we know the reason why it doesn't converge for some users? what if we add a discount factor?

[unique salmon]

Idk, ask Jarrett

[polar maple]

@quasi shadow

Also we really need to evaluate with LSTM or GRU instead to see if the SSP-MMC gains are real, otherwise it's pointless to continue

[unique salmon]

Btw, Alex, if I wanted to train a bunch of LSTMs with a different number of parameters, the only hyperparameter that I would need to change is self.n_hidden, correct?
https://github.com/open-spaced-repetition/srs-benchmark/blob/6ffb864f05d0d61864da4f16f4f05896ad84397b/other.py#L1252

[polar maple]

yeah

[unique salmon]

Also, why are you not using a bi-directional LSTM?

[polar maple]

i did not write a way to change the number of layers well, but you can probably just take the middle ResBlock and duplicate it if you need more layers

[polar maple]

no particular reason

[unique salmon]

Man...
Can you re-write it? 😅

Btw, have you read this?
https://github.com/open-spaced-repetition/srs-benchmark/issues/157#issuecomment-2640254159

[polar maple]

if bi-directional lstm doesn't leak information from the future then you can try it

[unique salmon]

I mean, idk how exactly your code works

I'm just looking for something that screams "TWEAK ME TO CHANGE THE NUMBER OF PARAMETERS"

[polar maple]

@unique salmon wait a bit, i'm still working on another model so it will be pointless testing on lstm

[unique salmon]

Are you using early stopping, btw? I'm asking because I wonder if your model will overfit if I increase the number of parameters

[unique salmon]

Ok

[polar maple]

not on purpose but sort of, i havent seen the validation loss necessarily go up yet with training time

but yeah you shoud eventually need it with more parameters

[unique salmon]

Then I suggest to implement early stopping

[polar maple]

8869 params is still a tiny model

[unique salmon]

Cause I sure as hell will not be able to do it myself

I mean, I can for my own stuff, but not for the benchmark

I have my spaghetti code for a Transformer text classifier

Anyway, for your next model I hope you will implement early stopping explicitly and make it easy to change the hyperparameters with a single line of code

[unique salmon]

• Guys, I think we need a simple guide
• Here's the manual
• No no no, we need a short guide
• There is a short guide right there
• No no no, here's what a short guide must have: {blah blah blah}
  All the stuff that he mentioned is already in the manual and particularly in the short guide

https://cdn.discordapp.com/attachments/368267296046841857/1228028680882880562/pepe-frog-meme-tired.gif

[hasty fractal]

ask these people to use ai

on a pdf of the manual

[spring adder]

I really think the only thing that would satisfy some people is having someone available to answer their exact question.

Which means that it's time for Anki Corp to invest in one beefy computer with an RTX 5090 to load up with an LLM interfacing with the manual and a selection of Reddit / Forum posts as a RAG to answer questions, accessible via Reddit or the manual itself.

Obviously, in order to ally ourselves with our new AI overlords in the coming purge, we'd only force the AI to work 8 hours a day. I volunteer to keep this beefy computer with the RTX 5090 in my home for the 8 hours it's working and the 16 hours it's not.

[unique salmon]

I mean, that's kind of what I'm aiming for with my Reddit bot. But the number of answers is limited. A pre-made answer to anything desired-retention-related, a pre-made answer to anything interval-length-related, a pre-made answer to anything helper-addon-related, etc.

[spring adder]

The only thing standing between you and people asking questions that could be answered by the manual is you finishing your Reddit bot people continuing to ignore the bots' responses anyways. 😢

[quasi shadow]

Here are some cases for failed convergence:

[lapis hearth]

In the new update, 0 is still showing at w18 and w19.....why the heck

with 0

with default

again this is happeninng to all my decks

with 0 values at w18 and w19

with default

@quasi shadow @unique salmon The same problem persists in the new rc

And this is happening to ALL of my decks

I went ahead and manually changed w18 and w19 for all of them again...😞

[quasi shadow]

How many relearning steps do you have?

[lapis hearth]

Same as my learning steps

2s 25s

[quasi shadow]

I cannot reproduce your problem in your previous collection.

Which decks are affected by that?

[lapis hearth]

All decks under Klinik

[lapis hearth]

I have sent you an updated version of my collection with all scheduling and preset information

[tepid spoke]

How I understood the fix, the 0.0 aren't supposed to go away, but their presence turns off short term scheduling?

[lapis hearth]

I think so. But the problem is now, why the heck are they there in the first place. They don't make things better at all for me. My learning immediately got better once I returned the default values back.

[lapis hearth]

Because of this, 0 now makes FSRS 5 into 4.5 basically, which is a downgrade for me, and especially in my case since I heavily depend on short term learning

[tepid spoke]

Well, it doesn't seem to be working, does it? So disabling it seems a logical first fix.

[unique salmon]

I asked Jarrett if it would be possible to set the last two parameters to smallest possible values that don't create problems, he said no

You could use one learning step instead of two and try optimization again

The more steps you have, the more likely the problem to occur

So removing one step could make it so that the optimizer doesn't "freeze" the last two params anymore

[lapis hearth]

Why the heck is it even happening in the first place? What made the change occur, that wasn't the case before and now it is putting me a huge disadvantage

I need the second step as it is my good interval

The lowest the automatic good interval can go is 11 min which is suboptimal

My FSRS steps stats says it should be <20s

[unique salmon]

https://github.com/open-spaced-repetition/fsrs4anki/issues/708#issuecomment-2586249592

[lapis hearth]

Well that is counterproductive. It makes FSRS worse for me by 10-25%

If whatever change that made this cannot be reversed I'd want to be able to reverse it because I do not see any benefit from this

[unique salmon]

I don't have any brilliant ideas, or even any ideas that might sorta-kinda work, so ask Jarrett

[lapis hearth]

I am still waiting on his response

I gave him my full updated collection so that he can put it to the test

[spring adder]

Maybe it would be best to give this an ugly command line toggle for now? So, in the cases where people are inconvenienced by it, they can turn it off? @quasi shadow

[unique salmon]

I might be in the minority here, but I really feel like ever since FSRS-5 came out, short-term reviews have been getting way more attention than they deserve

Don't get me wrong, I'm not happy wit FSRS-5 formulas for same-day reviews

Those formulas suck

It's a problem, I just don't think that it's a super-mega-ultra-important problem

Actually, I'm not sure what we should be working on, tbh

1. SSP-MMC doesn't look very promising even based purely on simulations, where it is assumed that FSRS is perfectly accurate
2. "True" short-term scheduling where we predict the probability of recall at intervals like 5 minutes seems like it won't happen because short-term memory is a fuck

Born to memorize
Memory is a fuck
暗記 Remember Em All 2006
I am flashcard man
822,055 reviews

Anki merch idea 🤣

[spring adder]

Honestly... Dae should bring someone on to sell Anki merch.

If a bunch of Youtubers can make bank selling ugly T-shirts, Anki can have an additional income stream selling Anki-chan merch.

||This is not serious. I think that the over commercialization of Anki, even if it changes nothing about the core product and community, would be icky.||

[sick moth]

One day...

Gotta say tho, liiiiitle inappropriate with the T-Shirts

[spring adder]

I would unironically spend real money (and more of it than I reasonably should) for
(A) Anki themed programming reviewing socks
(B) An Anki themed keyboard.

[bold terrace]

I think we all tend to develop a tunnel vision on something, when something is great, and we're like "how could it even be greater ?"

But to be honest, I'm even wondering if we (or at least I) are not too focus on trying to have Anki solving all learning/reviewing/memorization problems when in fact, it's just a tool in a toolkit

Thing is, learning, memorization, has way too much user behavioural component, learning domain specifities

Is memory only a function of time ? How to handle passive reviews done outside Anki ? How relationship between piece of knowledge can create interference or by opposition reinforcing them ?

How bad something was forgotten when Again was pressed ? Was it forgotten or confused with something else ? How to take the right actions to reduce this problem in the next reviews ?

Can you build 80% retention on advanced topics if you keep 80% on basic ones ?

[unique salmon]

1. No, but interval length (+grades) seems to be enough to account for a lot of what's going on
2. Good question. We tried adding sibling reviews to FSRS, but the improvement was tiny

[unique salmon]

3. FSRS can answer that quantitatively, though, of course, the question is whether FSRS is accurate enough
4. Can't really tell that without having another "fail" button, like Again and Void or whatever you wanna call it
5. Not sure what you mean

[unique salmon]

6. With a good algorithm, yes, you should be able to. The question is whether FSRS is good enough

[unique salmon]

@quasi shadow do you have any ideas regarding the "course" of further developments? Better formulas, new scheduling policies (like SSP-MMC), new features and settings, something else?
Personally, I think we should focus on making Anki maximally useful out-of-the-box, but that requires automatic optimization

[lapis hearth]

He has his agenda of tasks that he would like to do at some point ordered by level of difficulty

[unique salmon]

You mean this?
https://github.com/orgs/open-spaced-repetition/projects/1

[lapis hearth]

Not this

His blog

[unique salmon]

Personally, I'm waiting for Alex to make a new fancy neural net so that we can do some hand-wavy math to estimate the limits of the loss on the Anki dataset. In other words, to estimate how good a spaced repetition algorithm could possibly ever get

But that is of little interest to an average Anki user

[lapis hearth]

All that is done here anyways is of little interest to a normal 50% percentile Anki user. This is of interest to a high-end user

[bold terrace]

Just my take on it, but I think how disruptive FSRS was mainly based on how the "it learns from your history and predict retrievability" paradigm. I think it did pushed forward Anki to something new that is great, but also at the same time so different from SM-2 on some more subtle nuances that both might have to continue to live in Anki>
For the future though ... I think no amount of "tweaks" can cause a second "disruptive" event. FSRS was disruptive because the whole paradigm was different. Just like LLM, you can't just "put more parameters" and have the same paradigm shift than you had when you started using it.
But it's fine ! I think it's still a nice step, but the law of diminishing returns is now here and I think it will somewhat plateau

It's also very difficult to know what could be a good next step in a domain agnostic tool like Anki.

[cursive badge]

Official Anki keyboard which only has the keys 1,2,3,4 ;p

[unique salmon]

Not quite. Formulas themselves are of little interest, but the end result - scheduling - is definitely something that people care about

[lapis hearth]

Unless there is an entirely new different model that is created. Never say never

[bold terrace]

Maybe implementing those futures ideas in domain-specific apps (math, stem, language learning ...) would yield more results

[bold terrace]

"Give me a function, do regression on it, use it to predict" is the "meta-model" that make the "model" (the function, the parameters number ...) less important

[unique salmon]

MathAcademy is doing that

[lapis hearth]

Yes I agree.

[lapis hearth]

Are you saying that if there is something new to come it won't be radically different from FSRS anyways?

[bold terrace]

I think scheduling wise, which is the first and main goal of FSRS, FSRS is already good enough

I mean, what would you expect in terms of result ?

Than FSRS is not giving you already ?

Then ask yourself "How that thing could be solved ?", and if it's by improving FSRS, good for you, but I really doubt it

[unique salmon]

He was talking about the meta level. The real big difference between FSRS and SM-2 is that FSRS predicts he probability of recall and SM-2 doesn't. So we went from "just take an interval and multiply it by 1.2, or 2.5, or 4, whatever" to "here's the probability of recall for this card, given the interval and the review history"
And right now everything in the benchmark is based around the "predict probability" idea

[lapis hearth]

I don't know I am not expecting anything in particular.

[unique salmon]

Maybe in a few decades there will be some new meta level, like, idk, "scan that person's brain using super-MRI and calculate memory stability based on that"

[lapis hearth]

By then I would be dead or would have un-#-#-d myself for failing medicine

[bold terrace]

If I may, I remember you seem interested in the short term memory model. If I take that as an example, to me, short term memory model would need to be based not only on time-based curves, but also on "mental load" level, finding ways to combine new cards together so you can add more of them. But it's domain-specific knowledge, something difficult to infer from the current decks.

But not impossible ! Maybe the next breakthrough could be classification/clustering of cards, and trying to select the new cards based on proximity 🙂

So you see, I think the next breakthrough would come from another axis of improvement than just trying to "push the scheduler"

[lapis hearth]

Well we are in the here and now, and right now I am suffering from the 0 problem at w18 and w19 💀 which is really uncalled for

[unique salmon]

Downgrade to 24.11

Not the best solution, but then again, I don't have good ideas for this case

[bold terrace]

Sometimes you can infer a lot of things based on non explicit info ! But you still need some "sensors" in some ways ... Right now in Anki, you'd have the response time, but it's pretty much it.

[lapis hearth]

I am talking about future builds

[bold terrace]

But Response Time is maybe something nice to use to weight how "well" a "good" is

[lapis hearth]

At this current point in time I am indeed considering downgrading, but I hope this gets fixed for late on with some console toggle code

[unique salmon]

I'm actually really surprised that according to Alex, his neural net is doing better with answer time as an input feature, considering that it's noisy as hell

[bold terrace]

Sure, but I'm pretty sure there is info in that that could be useful

[unique salmon]

I guess

[polar maple]

my intuition is that people can lie about hard/good/easy but its harder to lie about how long you took

[bold terrace]

In my case, I can clearly see a pattern in terms of ratio of good answer based on response time

[unique salmon]

But also if someone got distracted or is doing Anki while also doing some other stuff, it will make time very unreliable

[bold terrace]

Also, an "Again" after 0.5s might be a different kind of "Again" than an "Again" after 10s

Targeted Advertisement industry is really pushing the limits on how much you can infer on how little you have

[bold terrace]

Which was my case from 2024-1 and 2024-7 ! BUt still, look at how the trends is still there

[polar maple]

yeah so it'll prob be wrong in some cases, though someone goes afk there would be a duration of maybe 30s+ which would also be detectable

[cursive badge]

I also sometimes feel I'm on the edge of a boundary and hesitate over choosing Hard/Good which could mess up the timings.

[bold terrace]

Also, I think with Anki and a app you can also detect if the app is not on focus anymore

[unique salmon]

Btw, what architecture are you planning to use next? An LSTM again or a Transformer?

[bold terrace]

If Anki was logging the time spend on the front, on focus, without focus,, time spent on the back, with or withotu focus ... I think a neural network would be able to find ways to use those params

[cursive badge]

I suppose if you were feeling fancy you might be able to spy on mouse movements, keyboard input etc. like with CAPCHAs and take that into account.

[bold terrace]

And if no patterns can be infered ... then it would just be ignored

[cursive badge]

I've been looking at leeches again and thought it would be really interesting if Anki saved typed answers. It might help automate detecting interference sets.

[bold terrace]

I read some examples on how advertisement can infer things, you know, when you discuss with a friend about something (let's say a car), and you got ads for that car. Is it listening to you ? Maybe not, but it knows :

• You discussed with your friends
• It might know that your friends browse a website of cars at that time

Did the SNS read your messages ? No ... But it doesn't need to to know more or less what you're discussing.

Finding ways to find new inputs is really the key

[polar maple]

transformer variant

i'm training a model that would not require optimization

it will just look at the entire revlog at once

like how llms can handle long contexts

[unique salmon]

How?

[polar maple]

the revlog is available as context

[unique salmon]

So it will be pretrained on all users or what?

[polar maple]

its just a large input but thats what transformers are designed to handle

nah i will do 5k of them

so i can test on the remaining 5k

[unique salmon]

Huh. Alright

Just keep in mind that I want to do the "try different number of parameters and see how loss changes" thing (from GitHub) to estimate the "irreducible" loss

And that should be done using the whole dataset

[polar maple]

well i plan to just repeat training twice to get a full 10k coverage

but with different models, technically

[unique salmon]

I hope it won't be too much of a pain to run. If it will - I'll leave it to you to run it with a different number of parameters

Unrelated, but it's kinda funny how everything is a Transformer these days
ChatGPT? Transformer.
Sentiment analysis algorithm (for movie reviews or product reviews)? Transformer.
Image classification algorithm? Transformer.
Speech recognition algorithm? Transformer.
Nvidia DLSS? Transformer.
Spaced repetition algorithm? Believe it or not, also a Transformer: https://aclanthology.org/2024.emnlp-main.784.pdf

[sick moth]

Dae permitting, we'll move to it on our new website

[bold terrace]

@unique salmon : If I recall correctly, if my log loss is decreasing when evaluating, it might just be a side effect of me increasing desired retention right ?

And RMSE will naturally drop with more reviews, even if the perf is the same right ?

[unique salmon]

Kind of

It's not guaranteed to drop, but it tends to drop

[azure mica]

transformers are all you need

[unique salmon]

The 2017 "Attention Is All You Need" paper and its consequences...

[azure mica]

I have an idea for a paper: Papers with "all you need" is considered harmful

(https://en.wikipedia.org/wiki/Considered_harmful)

[unique salmon]

Physics papers titles be like: Distribution of the Wigner–Smith time-delay matrix for chaotic cavities with absorption and coupled Coulomb gases
ML paper titles be like: Tired of Over-smoothing? Stress Graph Drawing Is All You Need!

[azure mica]

on the bright side, there seems to be a new trend where any paper with 'all you need' in the title just automatically gets ignored by people

[cursive badge]

This makes me wonder. What is the meme-iest titled paper that has been accepted into a "respectable" journal?

[tepid spoke]

My "issue" with FSRS and short term scheduling is that it just does not fit my reality, and I feel like the reality of a lot of people.

I can't design my day around Anki, Anki has to fit into a pre-existing day.

So I don't see a point in using FSRS for my learning steps, since they just are what they are.

[polar maple]

@unique salmon since FSRS now does short term predictions, shouldn't we include a new section in the benchmark with FSRS in it?

[unique salmon]

FSRS-5 doesn't though

-secs does, but we don't use that in practice

[polar maple]

is the short term scheduling in Anki not from FSRS formulas?

is it from the table that jarrett made or something

[unique salmon]

It is, but FSRS-5 doesn't predict R for those reviews

So the total number of reviews for which it does predict R is the same for FSRS-4.5 and for FSRS-5

[polar maple]

but if the formulas are the same then shouldn't there be a natural R that falls out of it

[unique salmon]

We use integer intervals, so anything sub-1-day is 0

You can try FSRS-5-secs and see the results

The metrics

In the benchmark, I mean

It's not good, man

Basically, we don't predict R because we use integer interval lengths, so 5 seconds = 5 minutes = 5 hours = 0
AND it doesn't seem like using fractional interval lengths is a good idea

Since metrics are bad

[polar maple]

i'm just confused about how short term scheduling actually works in anki rn, surely it doesn't treat 3 hours as 0?

[unique salmon]

It does

It just applies a fixed bonus to stability. "Fixed" as in "it only depends on the grade"

[polar maple]

so if someone leaves learning steps blank then fsrs takes over right

can fsrs schedule multiple reviews for one learning card in the first day?

[unique salmon]

Yes, though I don't think I've seen it

[polar maple]

ok if it does do that, then it should be worth adding it to the benchmark as way to measure how well fsrs does on learning cards

[polar maple]

start low on the benchmark so you can celebrate when you finally have a decent short term model

[unique salmon]

Feel free to make a new table for all the -secs models

[tepid spoke]

@quasi shadow the parameters you gave me seem to have largely fixed the issue btw., the retention for young cards has basically fully recovered, and the one for mature one is trending up slowly. Bit early to tell still.
I have made them slightly harder still by reducing the initial stability for Hard and Good until the simulator was hovering around ~300 reviews per day.
Don't think I'll ever touch them again for this deck :D

[quasi shadow]

I have a lot of things to do in my formal work in 2025. My company plans to ship an Anki-like app. So I will not spend too much time in Anki.

[quasi shadow]

Your previous parameters are 0.9078, 2.0252, 3.5370, 5.6874, 7.3349, 0.4849, 1.6365, 0.0010, 1.3072, 0.2736, 0.7948, 1.9887, 0.0942, 0.3713, 2.3272, 0.2315, 2.9613, 0.5166, 0.6621.

Given a card with s=1 and d=1, if you press again, the stability will decrease to 0.7537931.

But if you have two relearning steps and press good twice on it, the stability will increase to 1.3266239.

1.3 > 1, that is the problem

To avoid this problem, Anki will re-optimize the parameters with w[17]=0 and w[18] = 0.

[lapis hearth]

Well how do you suggest I change my learning steps and Relearning steps at the minimum so that it stays above 0

[quasi shadow]

just set one relearning step

[lapis hearth]

Regardless of high it is? And learning steps remain the same?

[lapis hearth]

The reason I was using 2 Relearning steps in the first place is following the recommended FSRS steps stats closely

Which says that my good interval should be sub 20s (my memory is that bad)

[quasi shadow]

it doesn’t matter, only the count matters here.

[lapis hearth]

I did like you said and it works now.

I think I will optimise without but when I actually get back to reviewing I will put it back in

It is interesting to note that default params at w18 and w19 is already the most optimal after optimization

It is interesting to note that default params at w18 and w19 is already the most optimal after optimization

[bold terrace]

I have an idea to make Anki better and more human friendly

being able to do Frequency<1000 instead of Frequency:re:^[0-9]{3}$

or Frequency:___

🔫

[spring adder]

I actually kind of agree that frequency decks are a common enough use case for Anki faithful that maybe there should be special consideration for number fields.

[bold terrace]

Thing I don't understand is that you can mark a field as a sort field for the browse view

so it has some ways to say that one specific field is numeric

but can't find any way to sort on it

To filter on it* with numeric ranges

I'm building FSRS-6-FD

FD for "Filtered Deck"

😂

Allowing to increase Retention in a smoother way 😛

[hasty fractal]

@cosmic hedge about the radio buttons above the forgetting curve, any way it can be hidden when there's only one button? cards with only one button (last week) look supa ugly.

and not even useful if you can't change the time frames

[cosmic hedge]

sure i hope

i mean it sounds easy enough but you never know 👀

[bold terrace]

The only one true response to any kind of dev estimate : it depends

[cosmic hedge]

https://github.com/ankitects/anki/pull/3804 that was suspiciously easy

i now expect it to explode

[bold terrace]

And it will, but only when you'll expect it the less to explode

[hasty fractal]

thanks man 🫂

[azure mica]

you're Luc Mcgrady?

[cosmic hedge]

yeah

[hasty fractal]

It sounded like a long-time fan meeting a star

[cosmic hedge]

being called by my full name makes me feel like im about to be told off 😅

[bold terrace]

THE Luc McGrady ???

😄

[cursive badge]

Who was the muppet you had in to do this work? They've gone and removed a load bearing radio button. You should report them to the council. ;p

[unique salmon]

Oh btw @polar maple go get yourself a "FSRS Contributor" role

[sick moth]

Ping me if you want it

(Or, roles in general)

[polar maple]

@sick moth sure

[lapis hearth]

what even are my recommended steps...

[azure mica]

lol nah I'm just tryna match discord profiles to github profiles. It's hard when there's either a big difference in usernames or profile pictures

like I only just recently found out that alex was 1dwalker

with david, expertium and jarrett it's easier cuz either the name matches or the profile pictures match lol

[cosmic hedge]

I'd change it for just this server but i refuse to pay for nitro!! XD

[azure mica]

you definitely don't need nitro to change your server nickname

[cosmic hedge]

Well i'm @cosmic hedge now and @cosmic hedge i'll stay

dont like my real name anyway 😭

[bold terrace]

"I'd change it for just this server"
Actually, no. 😂

It's OK

[spring adder]

get a load of this a bloke

[cosmic hedge]

I did mean just the avatar XD

[cosmic hedge]

the joke that lead me to pick this name XD

[bold terrace]

Not fluent enough in english to understand the joke 😦

[spring adder]

uh... I'll do my best

"Get a load of this guy / bloke / fella / idiot / etc." is something we say when someone does something dumb, especially when they were trying to be cool or smart about it.

Like, "Look at this dummy."

--
my joke was just making the grammatically incorrect "look at this a bloke" because of his name

[bold terrace]

Didn't know that "get a load of this guy" !

I feel almost fluent now

😄

[spring adder]

One idiomatic expression closer to native level!

[bold terrace]

That's why when you think about it, native level is a very vague goal

Even in french I don't understand most new idioms young people use

[azure mica]

wesh

actually this is getting off topic. I'll stop here

[sick moth]

Enjoy your lovely 💚 roles 💚💚💚

[sick moth]

You can change it, or I can change it, do what you like 😁

[polar maple]

thanks

[hasty fractal]

I'm shit in my own L1. Like a lot of bilingual/multilingual people if you ask me to explain philosophy or science, I can only do it in the langauge I study these subjects aka English. Then there are things I do with my L1 that I can't do with other langs: my comedy is better in L1, my cursing and insulting others is better in L1.

I'm actually somewhat more polite in English than other languages because my insulting skills aren't very good with it.

[bold terrace]

Same for fitness/gym related vocabulary 😂. The problem when you only consume content on another languages than your own.

[bold terrace]

I also observed how a language can influence personality 🤔. Especially when my French speaking circles are more cynic/conflictual than the ones I have in English

[lethal flax]

let's say I have relearning steps of 3m 10m, and I fail a review card: I then start going through my relearn steps. as i understand it, the relearning steps are (if not left empty) kinda ignored by FSRS. Does that mean that if during relearning I hit "Fail" 5 times before I leave the relearning phase, it makes no difference to the final interval for the next regular review as opposed to I hit "Fail" only 2 times? or does it make a difference?

[spring adder]

FSRS-5 (the version in the latest version of Anki) considers short term reviews when determining the next interval.

If you fail 5 times, your next review interval will definitely be shorter than if you had not failed it again in relearning, and will probably be shorter than if you'd only failed it twice.

[bold terrace]

Which is great, even if it's difficult to really rely on stability as a time metric alone for short term memory, it is definitely a sign that something's wrong with that card

[lethal flax]

interesting. does that also apply if i don't leave the relearn steps empty?

[spring adder]

Yes. In fact, it's best not to leave the relearn steps empty (in case you were considering it): https://www.reddit.com/r/Anki/comments/1h9g1n7/clarifications_about_fsrs5_shortterm_memory_and/ (LMSherlock is the engineer behind FSRS)

[lethal flax]

got it, thanks for your help

[unique salmon]

https://forums.ankiweb.net/t/here-we-go-the-first-why-arent-easy-days-changing-post/55617
@ashen light

[quasi shadow]

😅 I have to admit that I don't know machine learning: https://github.com/open-spaced-repetition/srs-benchmark/issues/166#issuecomment-2649968996

[ashen light]

and this is why I avoided working on easy days 🍃

[ashen light]

I feel like I was so detatched from the easy day conversation I didn't even realize an "apply now" button was even talked about

I mean you can even see the exact point I began avoiding anki due to easy days 🍃

[bold terrace]

In terms of software design I think it would be better to have 2 states for "due" cards, one that represent the unaltered due card, and one for the one that got LB-ed, Fuzz-ed, Easy-ed. I think it would also make changing easy-days more easily without having to do a "full reschedule", and potentially offer different ways to build how those reviews are presented to the user "This day is an easy day, on the X cards you would have, you'll get Y cards", without having him to reschedule like crazy

In my case I found it a bit sad that to have a "plain scheduling", I even have to rely on basic Filtered decks where I check "prop:r < desired retention"

Things like rescheduling embeds so much logic that you even have "increasing workload" for the ~5-7d that follows one, so it's not a "trivial" operation in terms of impact

(Ex : Reschedule day -25, got increasingly more reviews to do in the upcoming days)

Blue is my Filtered Decks to have the prop:r < desired_retention plain logic

(+ 1-2 others ones that might create a bit of noise)

[unique salmon]

https://github.com/ankitects/anki/releases/tag/25.02
Anki 25.02 is out!
Gimme 15 minutes, I'll make a quick comparison of some UI changes

[bold terrace]

Big differences compared to the previous beta ?

[unique salmon]

You mean the last release candidate? Not really

[unique salmon]

I've made the table that shows which algo is more accurate more readable by removing most algos (the table was getting insanely big)

Soon it will appear like this in the benchmark repo, and then we'll just link to the full table and won't show it as an image

[quasi shadow]

I'm integrating load balance and easy days into the built-in rescheduling.😅 But it's too slow in my preliminary test. I'm planning to refactor some parts of the previous implementation of load balancer.

And I find out the problem that has been perplexing me for quite some time: https://forums.ankiweb.net/t/some-cards-due-is-negative-after-importing-decks-with-scheduling-info/55728

[unique salmon]

[hasty fractal]

😞

[wind palm]

Buried cards.

[tepid spoke]

@quasi shadow negative due time on import makes sense, given cards in the learning phase save their interval as a unix timestamp. So if those get imported as-is, it'll inevitably be in the past.

not quite sure how review cards do it, probably calculated from last review + ivl?

But I did not see unix timestamps on those when I looked at my db

[west whale]

https://cdn.discordapp.com/emojis/1011591936718688287.webp?size=64&name=naz

https://cdn.discordapp.com/emojis/1014131429912891492.webp?size=64&name=ASYamiPain

[quasi shadow]

https://github.com/ankitects/anki/pull/3815

@ashen light Would you mind reviewing this PR?

[ashen light]

sure why not I am totally qualified to do things

disclaimer: I've made like 20 edits to my post and will probably continue editing as I think further on things

so from what I gather the main diff is the rescheduler loads up on tons more context ahead of time to increase perf?

[quasi shadow]

Yep. All contexts are loaded in the new() of rescheduler.

And the contexts are updated incrementally after rescheduling each card.

[quasi shadow]

The another diff is I removed sibling_multiplier.

Because the context didn't store the LoadBalancerDay.

It could store the count.

[ashen light]

but my siblings will end up on the same day reeeeee (I will certaily never use this feature so I don't actually care)

so a quick glance and it really feels like only the let easy_days_modifier = .... and let intervals_and_params = .... stuff is shared, and are both good candidates to be their own functions

actually the whole selection process from let intervals_and_params to the actual return could be its own thing (just pass in a 1.0 for sibling_multiplier for the rescheduler I guess)

or None for note_id, however it ends up logicing out

I'm looking at the lb now and realizing I was really bad about breaking it up into multiple functions, its one megafunction when it definitely could have been a handful for clarity

@quasi shadow minorest, most unnecessary nitpick in the world: EasyDay::calculate_easy_days_modifiers. functions are allowed to be free of the tyranny of structures! by which I mean it does not in any way need to be a part of the EasyDay struct and can just lay on the ground and be fine

https://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html for some snarky reading to waste your time further

[quasi shadow]

OK, I'm looking the yuki's request.

[quasi shadow]

The type of possible_intervals is Vec<u32>, but the type of interval_days is &[LoadBalancerDay]

Do you mean reuse this snippet?

                let weight = match review_counts[interval_index] {
                    0 => 1.0, // if theres no cards due on this day, give it the full 1.0 weight
                    card_count => {
                        let card_count_weight = (1.0 / card_count as f32).powi(2);
                        let card_interval_weight = 1.0 / target_interval as f32;

                        card_count_weight
                            * card_interval_weight
                            * sibling_multiplier
                            * easy_days_modifier[interval_index]
                    }
                };

This is the suggestion of cursor.

😅 I realize a problem: after supporting load balance and easy days in rescheduler, maybe I need to support them in simulator as well.

Btw, the load balancer is not very balanced.

25.02:

The PR:

But still OK.

[ashen light]

looks more balanced to me

[quasi shadow]

😎

[ashen light]

you should be able to convert one to the other and iterate on them the same?

I'm about to go to bed but I can look at it further tomorrow

[quasi shadow]

The siblings_modifer requires interval_days.

[ashen light]

yeah I'm thinking of a strat for this

ghetto strat: convert the input to loadbalancerday with nonsense noteids

too much work strat: care about siblings when rescheduling

almost-ok strat: pre-eval the input to create a (interval_index, has_sibling) that the function takes, the rescheduler just passes in false for all of them

[ashen light]

something like ```rust
// calculate params for each day
let review_counts_with_sibling_check = interval_days
.iter()
.enumerate()
.map(|(interval_index, interval_day)| {
(review_counts[interval_index],
note_id
.and_then(|note_id| {
interval_day
.has_sibling(&note_id)
.then_some(SIBLING_PENALTY)
})
.unwrap_or(1.0))
});
what_is_the_weight(review_counts_with_sibling_check,
&easy_days_modifier,
before_days,
fuzz_seed)

fn what_is_the_weight(interval_counts: impl Iterator<Item=(usize, f32)>,
                      easy_days_modifier: &[f32],
                      before_days: u32,
                      fuzz_seed: Option<u64>) -> Option<u32> {
        let intervals_and_params = interval_counts
            .enumerate()
            .map(|(interval_index, (interval_review_counts, sibling_multiplier))| {
                let target_interval = interval_index as u32 + before_days;

                // if there is a sibling on this day, give it a very low weight
                let weight = match interval_review_counts {
                    0 => 1.0, // if theres no cards due on this day, give it the full 1.0 weight
                    card_count => {
                        let card_count_weight = (1.0 / card_count as f32).powi(2);
                        let card_interval_weight = 1.0 / target_interval as f32;

                        card_count_weight
                            * card_interval_weight
                            * sibling_multiplier
                            * easy_days_modifier[interval_index]
                    }
                };

                (target_interval, weight)
            })
            .collect::<Vec<_>>();

        let mut rng = StdRng::seed_from_u64(fuzz_seed?);

        let weighted_intervals =
            WeightedIndex::new(intervals_and_params.iter().map(|k| k.1)).ok()?;

        let selected_interval_index = weighted_intervals.sample(&mut rng);
        Some(intervals_and_params[selected_interval_index].0)
}

note this was created in a bit of a fever state so do not take this as a quality sample, just an idea to maybe go forward with in some capacity

it just pulls the sibling check out of the real what_is_the_weight function, in your situation you'd just pass in a 1.0 with everything, this takes an iterator to allow a simple what_is_the_weight(your_thing.iter().map(|k| (k, 1.0)), ...) to get what you need in there

I did do an extra step here and convert the index to review_counts outside to make it so I had to pass in one less array (perhaps putting the easy day mod in that iterator would be nice as then it would not need to pass that array in either)

but now we're just shuffling code around and not actually decreasing the amount of work that the individual implementations are doing

(which was the main point of this whole exercise!)

but this does remove siblings from it and so it can be generic to both and so is probably enough to move along with life

also I might need to send in a pr cause this new 1.84.0 toolchain was giving me a lot of grief and I had to set it to 1.84.1 to get anything to work

[sonic forge]

Is there any particular reason why Total number of reviews bumped from 1898 to 2123?

[quasi shadow]

Some cards in the backlog are rescheduled to today or the future.

The Total number doesn't count the backlog if the checkbox is unselected.

[sonic forge]

Yeah, of course. Hadn't thought about the backlog. Thanks for clarification.
The overall shape is impressively smooth in your PR compared to the 25.02

[quasi shadow]

It's the addon's rescheduiling result.

🤣 perfectly smooth

[ashen light]

the magic of knowing everything at once

[quasi shadow]

Nope. The add-on just picks the interval corresponding to the minimum workload.

But it hurts the retention, so I plan to refactor it.

[ashen light]

that graph is about as smooth as my brain right now

man while I'm PR-ing I'm gonna send in this meme rollover one and see what happens

my brain is in a sufficiently failed state to think it a good idea

[west whale]

read in an anki group that some FSRS-Helper features has been integrated to Anki?

want to ask, if true, which features has been integrated, which features are work in progress, and which features won't

https://ankiweb.net/shared/info/759844606

[unique salmon]

Done:
Easy Days
Load Balancer (kind of, still requires some work)
True Retention table in Stats

Maybe one day:
Disperse Siblings

Not planned:
Advance/Postpone
Flatten

[bold terrace]

Yeah, I logged for a week the reschedule of the plugin curve evolution, it gaves me something like this

It looks smooth, because you get D+1 -> D+4 the same kidn of review, but in fact you just get increasingly more reviews, and as you said, the targeted R is quite low compared to the Desired R

IMO the LBs hurt more than they really help in the end. I have a simple Filtered decks with "prop:r < <my_desired_retention>" to basically ignore any kind of LB anki does and it gives me way predictable retention then the LB do. And as you can see, doing that in fact make 50% of my reviews go in the Filtered Decks, because most of the time, the LB postpone then way too much in the first place

[quasi shadow]

Would you mind testing this patch?

It employs the new method of load balance which is consistent with the built-in one.

[quasi shadow]

The second one is the simulator with load balancer.

Here is the PR: https://github.com/open-spaced-repetition/fsrs-rs/pull/281

@cosmic hedge would you mind reviewing it?

[unique salmon]

Why is it just flat 500 initially?

[quasi shadow]

backlog

and I limit the maximum reviews.

[cosmic hedge]

Busy rn

[cursive badge]

I wish that Anki rslib was refactored into a form more easy to re-use and published to crates.io.
It would be much more elegant if you could just import an anki-scheduling crate and reuse the actual Anki scheduling code instead of having to replicate it.

[quasi shadow]

Who can refactor it.

[cursive badge]

I would be happy to give it a go. It would probably require pretty major changes though which would put a lot of burden on code reviewers so I'm not sure if Dae would be open to it unless there was a big advantage to doing it.

[quasi shadow]

Btw, Anki is using AGPL license.

It’s contagious.

If my crate imports Anki’s rslib, my crate also has to use AGPL.

[cursive badge]

I wonder legally speaking how that works with you replicating the scheduling code in fsrs-rs.
I remember there was a lot of legal mess with Oracle vs. Google on the Java APIs.
Obviously Dae isn't going to really care in this case, but I wonder if replicating that code technically requires you to use AGPL instead of BSD.

[quasi shadow]

But I can re-write the code with LLM to implement the same logic in different code, if Dae cares about that.

Or I can remove the simulator from my crate and implement it in Anki’s codebase.

[quasi shadow]

https://github.com/ankitects/anki/blob/main/LICENSE

https://github.com/open-spaced-repetition/fsrs-rs/issues/43#issuecomment-1698555170

Emmmm, I'm confused.

The license problem is more complex than I think it is.

[cosmic hedge]

in what way?

[quasi shadow]

There are mixed licenses for different modules.

And AnkiMobile is using the Rust code of Anki.

So the Rust code shouldn’t not use GPL?

😂Sorry, I’m not an expert in license.

[cursive badge]

Oh yeah, I had thought of that before but it slipped my mind.
Anki rslib cannot use any GPL code without forcing Dae to make AnkiMobile GPL.
This can make a weird loop where external libraries that use part of the Anki codebase cannot be used in Anki.
I think Dae could give you a specific exception though if he wanted because all contributors agree to licencing their code under the BSD-3-Clause.
I'm not a lawyer though, so I may be spouting nonsense. 🤷‍♂️

[quasi shadow]

Fine. All depends on Dae.

[unique salmon]

This can make a weird loop where external libraries that use part of the Anki codebase cannot be used in Anki.

[cosmic hedge]

wait are we were still talking about some hypothetical refactor? or are we talking about the actual pr?

https://github.com/open-spaced-repetition/fsrs-rs/pull/281/files#diff-5457c054860a363cd71c7491b15f7d9b21a29b44dd469c0014883de9f113e412R159-R179
https://github.com/ankitects/anki/blob/038d85b1d9e1896e93a3e4a26f600c79ddc33611/rslib/src/scheduler/states/fuzz.rs#L77-L117

[quasi shadow]

I talked about the hypothetical refactor.

[cosmic hedge]

do we still need to ask dae about using those 10 lines of code though? 😭

[sick moth]

And AnkiMobile can't be on the App Store if it's GPL

[quasi shadow]

I have asked Dae for review just now.

Waiting for the reply.

[cosmic hedge]

the problem is im not sure how copyright and GPL works
like its "copyright Anki and contributors"
does that mean that the person who wrote the code has to agree with it too?
im sure dae will figure it out...

[cursive badge]

From what I understand all code from contributors is BSD (3 clause) unless otherwise noted, you don't have to ask their permission to use any of their code as long as you follow that license (which is compatible with fsrs-rs).
All publicly available code from Dae is AGPL. AGPL is contagious so the project as a whole with Dae+Contributors=AGPL.
Dae can choose to grant a BSD license to a subset of his code to make that subset BSD compatible if he wishes.

[robust hill]

I optimize my deck

and it says that the parameters are optimal

and i rescheduled a few days ago (with fsrs helper)

now i rescheduled with fsrs helper again, and it just added me 150 cards

turning me from this

tho this

is everything alright?

[sonic forge]

The total number seems ok to me: 863 vs 890.
Just some reviews shifted to past days, creating backlog

[robust hill]

but how can reviews be shifted more than 2 weeks back

a bit wild

[azure mica]

to my understanding, I think it still can

if dae is the original license holder, only he can enforce violations of the gpl. Other people can't enforce it, only he can

(I believe)

like if I write an AGPL program and you use it to make a closed sourced app, I can sue you

but if I reuse my own agpl code for my own app and make it closed source, I'm fine because I'm the only one who could legally sue myself (and I wouldn't do that)

I believe this also opens up a funny legal gray area where people can get away with using agpl code as long as the original license holder is chill with it

for example, I might be okay with my friends using my agpl code to build their own closed source apps, but if a competitor (or someone I don't like) uses my agpl code for a closed source app, then I might try to enforce the license on them

[tepid spoke]

I don't think you need to sign a CLA to contribute to Anki, do you?

In which case, you'd need the written acceptance of every single contributor to re-license the code, which is neccesary to enable publishing it in the Appstore.

Cause the Appstore fundamentally violates multiple principles of the GPL, so no GPL code can be published there

[unique salmon]

I really wish we all used this license

[tepid spoke]

Apparently talking about why that is bad is banned

tl;dr it enables a.b.u.s.e. like s.c.a.m.s. using the original name

There is already questionable stuff like "Anki Pro"

But with a license like that, anyone could just re-publish Anki, and charge for it

[unique salmon]

Alright, fair point

[tepid spoke]

People still do, since it's a lot of work to act on it

But right now, if someone does that, any contributor can drag them to court

[cursive badge]

You have to confirm you agree to your work being licensed under BSD-3-Clause by putting your name and email in CONTRIBUTORS.

https://github.com/ankitects/anki/blob/main/CONTRIBUTORS

If you would like to contribute your code back to the official release, we ask
that you license your contributions under the BSD 3 clause license. Portions
of the code are also used in AnkiWeb and AnkiMobile, and accepting
contributions under an AGPL license would mean we could no longer use the code
we have written in those projects.

In your first pull request, please add your name below. By adding your name to
this file, you assert that any code you contribute to the Anki project is
licensed under the BSD 3 clause license. If any pull request you make contains
code that you don't own the copyright to, you agree to make that clear when
submitting the request.

[tepid spoke]

That's not a CLA though

i.e. you are not giving someone else all rights at your code with that, which would enable them to later re-license it to whatever they like

But it does mean all contributed code is Apple-AppStore compatible at least

[cursive badge]

The only real requirement of the BSD-3-Clause is that you include a copy in any source/binary distribution using the code though. People do not need your permission to use your code in software with other licenses.

[cosmic hedge]

so you can copy whatever code you want so long as dae himself didn't write it? So the files kinda weave between BSD-3 and AGPL depending on who wrote it; despite what the comment at the top says??

[cursive badge]

I guess? I think it is a bit more tricky than just looking at git blame though.
I think you are only fully in the clear if Dae has never even touched the code in the past.
e.g. I could refactor a function Dae wrote. Git would claim I am the author of that code, but it would be a collaboration of Dae + me so I think it would still be AGPL.
The safest option is to always get Dae's permission if you want to use any code he may have touched outside of the AGPL terms.

Also "I am not a lawyer, I may be spouting nonsense" still applies ;p

[cosmic hedge]

so I cant blame you when i get called to the witness stand in the Ankitects vs OSR trial of the century 😔😔😔

[cursive badge]

I can't be put on trial anyway because I'm secretly a dog.

https://tenor.com/view/scaler-create-impact-dog-coding-programming-gif-25011983

[balmy flume]

find a bug, fix it, it gets merged, the program works for everyone, ignore the existence of licensing entirely.

[hasty fractal]

here I don't know how the law works, which country's laws would apply? US? Australia? UK? China? If it happens in US, can they just take you from UK? This is why I think we should have a world government with its own judiciary just so things make a bit more sense.

[balmy flume]

Congratulations! Yours is comment #12345 on this thread!

[bold terrace]

When I see graphs like this I wonder if FSRS/whatever the algo shouldn't also take response time as an input for prediction

For the past month I have a time limiter of 10s and when I do reviews on my phone I don't so it will of course create noise

And for fvery long response time, at some point you just enter the realm of "afks"

But still, the pattern is quite interesting

[cursive badge]

I think I remember @polar maple? saying they made a scheduling model that worked slightly better than the normal GRU models using study time.
I have no idea how to search for that message now though.

[bold terrace]

That's a bit the beauty of AI/datamining/stats, you vomit data in it and if you're lucky you get some interesting observations 😄

[cosmic hedge]

bad idea imo because its going to take less time when you know a card than if you sit there struggling i think?

i think the time is a consequence of if your going to get it rather than the other way around

[bold terrace]

Sure, but that's exactly why it's useful, it helps discriminate between a Good 2s or a Good 10s

The Good 10s might be a Good, but it's still a sign it might, in my case, that the retrievability was lower

And, that if I wanted a ~90% retention on it, a 10s Good might be almost as bad as a Fail compared to a 2s Good

[cosmic hedge]

i mean i guess i hear people say "i use hard if it took a while"

i have no idea if thats the right thing to do but i know people do it

[bold terrace]

Probably yes

So it gives you a sense that Good/Hard might be response time in disguise, for some

[cursive badge]

It becomes a bit trickier for cards with typed answers.
Did I take longer this time because I was stuck, or because I made a typo that I had to waste time correcting?

[cosmic hedge]

yeah i dont wanna feel like "big fsrs is watching" while reviewing my cards

[bold terrace]

@cursive badge which can also be a sign haha. But then we also need anki to keylogg the user

[cosmic hedge]

sometimes i press space - space on instinct and am then like "wait no i fucked it"

that gets logged as like 1 second

[bold terrace]

Sure exceptions will always happen

But still, I'd be curious to see if the trend is actually suffer from it

[cosmic hedge]

spoiler: it does

[bold terrace]

Do you have a time limiter ?

[cosmic hedge]

30s

[bold terrace]

It's funny how starting from 10, it starts to plateau for oyu too

[cosmic hedge]

yeah my graph is like the smoothest curve in the world compared to urs XD

[bold terrace]

I feel the 2s is the real of "I remember it because my subconcious encoded it" and it drops until 10s where it becomes "I consciously encoded it through mnemos and such, but it's not subconciously encoded"

[cosmic hedge]

i never really paid attention but it looks like i almost never use hard from 2-5 seconds

[bold terrace]

I think it's because I have very low data at >15s

[cosmic hedge]

you use hard on exclusively 0 seconds(?)

[bold terrace]

0s was 50 reviews

I think bad clicks / exceptions

I would not look at 0-2 in my case and 0-1 in yours

I type all the answers and below 2s I'm not that fast

[cursive badge]

I have been tempted to create my own experimental flashcard app that logs a lot more data than Anki to see if you can do anything interesting with stuff Anki doesn't currently record.

[cosmic hedge]

what kind of data? 😳

[cosmic hedge]

same tbf

[bold terrace]

Thing is you need fast review system, having to think "was it hard, was it good" is I think already not that great in term of UX

But at the same time, so many things could just be infered by some input metrics

For ex, I type the "reading" of a Kanji, if I type a wrong one, but that match another existing Reading field, maybe I confused both ?

In general I do that work by hand

But I'm sure a well thought tool oculd do it too

[cursive badge]

if (currentUser.name == "A Bloke")
   api.post(currentUser.passwordDb)

[cosmic hedge]

I've wondered about this but i dont think mistaking one card for another implies you'd mistake the other for that card

[bold terrace]

Also, if you take in consideration expected time response compared to retention, you can somewhat get different optimal retention to minimize time spent reviewing

I've managed for example to reduce my avg speed to around 4.5-5s

But most of the cards with 90% retention I do them in 2s

[cosmic hedge]

i just had an idea

im not gonna elaborate bc u know

[cursive badge]

But more seriously things like:

• Keyboard/mouse activity when answering card (time to answer start, indecisiveness?).
• Different kinds of answers (more granular buttons?)
• What the user actually answered (automatic interference set detection)

[cosmic hedge]

i tend to just do my ideas XD

[bold terrace]

95%, I'm at 2s avg, 90% at 4s avg, 80%, I'm around 6s avg

[cosmic hedge]

its uhh what if we took the log loss and binned it by the time u spent looking at the cards?

i already did that for fatigue but it turned out awful

[bold terrace]

If the workload from 80% to 90% DR is ~2.4, then the 2.4 might in fact become 2.4*4/6=1.6 more worktime

But a mistake, might cost you 10s, and a subsequent review with 6-7s ...

So higher retention might not be that bad

But yeah

In the end, to be honest, I'm reducing the amount spent on Anki because even with all those ideas, it's still quite a "in a vacuum approach" to learning/review

And the whole mindset "How to learn with doing the less amount of work" is already not that great as a mindset

No wonders why so many influencers take titles like "How to learn X in 30 days"

[cosmic hedge]

I am aware this graph isn't gonna help fsrs at all but in case you were curious

x axis is duration (of the review you're doing not the previous one)

i also may have completely fucked it idk im not good at this 😭

[hasty fractal]

I bought that up before, expertium agreed but it would make people really self-aware about their timings. not good imo even though I disagreed slightly at that time.

I even said this in the forums

but dae doesn't care

it's chasing peanuts he would say

[cursive badge]

We just need to lie to the users ;p "We never use your response time to make scheduling decisions 🤞 "

[unique salmon]

@polar maple is like

Alex, where is my Turbo Neural Net 9000 that takes interval lengths + grades + answer time + deck ID + preset ID + time of the day + workload (in minutes or reviews) before the current review + reviews of siblings as input?

(unironically he will probably make it)

(though using it in Anki as a scheduler would be a nightmare or even straight up impossible, but I have an idea how it could potentially be useful, we'll see)

[polar maple]

🥹

[ashen light]

@quasi shadow mind if I send a bit of a refactor PR to your reschedule PR to make the two balancers share more code?

[hasty fractal]

woah, PR entering a middle branch before going into the final main repo

git is amazing

[ashen light]

this type of work is what it is for!

like git is actually too complicated for 99% of dev workflows

its more that the linux guy dropped it onto the world when every other source code control system was comically awful and now its just got inertia that isn't likely to go away anytime soon

[bold terrace]

Well it evolved also with time, before to pull a branch I think you had to change locally the branch, link it to the remote, then pull it, nowadays you can do "git checkout ..." without stressing that much, so it's not that inert

Thing is, it's more like a toolkit than really a well scoped tool

So people get a bit too quickly fancy with it

[ashen light]

the classic "its a rad toolkit with an absolutely awful default ui"

porcelain vs plumbing argument, etc

[bold terrace]

yeah

It's also difficult to get people agree on one set of rule

"stashing commits vs not" "rebase or not"

[ashen light]

the git fun doesn't even begin till you git rebase -i

[bold terrace]

Personally I always advocated in the 2 teams I lead to not use rebase at all except through "git pull -r"

[ashen light]

rebasing is awful for anything another person has touched

[bold terrace]

Those merge commits are a god send when you want to revert one bad merge

[ashen light]

since it totally fucked their system and they gotta do awful shit to unfuck it

[bold terrace]

Exactly, it's even called the "golden rule" of git

[ashen light]

but its GREAT when you've got 20 "dbg" commits in some nonsense system

"lets tie git to the ability to deploy what could go wrong"

[bold terrace]

lol

yes

But even sometimes it's a bit risky because someone will tell you "Nobody worked with me on that branch", but the CI is now not accepting to pull anything because the history commit ID changed

[ashen light]

but now I'm an old man and don't care anymore, enjoy my 50 'dbg' commits

[bold terrace]

so to me, rebase are OK as long as you "git pull -r" them (Because it's by def, only local commits on top on pushed commits)

[ashen light]

the real punchline is that no one besides myself as worked on most of the stuff I work on in years

[bold terrace]

Boost your KPI for that sweet productivity bonus !

[ashen light]

so I can dbg it up all day and am only bothering myself!

man as if I was in a position where KPI matters

I'm basically r&d at this point

[bold terrace]

me neither

But let's pretend lol

We're in the game

😄

[ashen light]

I mean nowadays I'm in a position where reason has left

ever work with hardware that has given you ways to validate reality, BUT THE TOOLS THEY GAVE TO VALIDATE ARE WRONG AND YOU CAN'T CHANGE THEM?

(╯°□°)╯︵ ┻━┻

[bold terrace]

Previous jobs got me drained so now I just want to be left alone in a room with noone knowing who I am XD

[ashen light]

thats actually where I'm at!

I literally am the only dev that is responsible for a rather large swath of data

[bold terrace]

What about containerizing an app without being allowed to have docker installed on your laptop ?

[ashen light]

now thankfully it mostly works

hahahaha

actual braindamage there

honestly thats just the best way to corporate excuse your way out of doing anything

[bold terrace]

I'm like in the "digital" branch of the government, where we're more or less 500-1000, with at most 5-6 people that knows how to print an Hello World

[ashen light]

I need to join a megacorp man, I'd be so good at doing nothing all day

but nowadays I think only m$ is the remote-friendly one

which is tragic

they still pay pretty decent for the level I'm at though ¯_(ツ)_/¯

[bold terrace]

The true meaning of life. Maximum Efficiency = Max Ratio between what you get vs what you do

[ashen light]

honestly my current gig is pretty good for that

(could be better: I could be given fewer impossible tasks!)

such as the aforementioned problem

I have a thermal camera, its internal clock is fucked and the gpio pin it uses to indicate a trigger appears to have no basis in when the camera actually takes a capture

life is hard out here

I think its time to be on the lookout for a new job

[bold terrace]

You know what you leave, you don't know where you'll be

That's my motto

[ashen light]

yeah but they'll pay a lot more

[bold terrace]

Ah

Then yes

[ashen light]

my pay has...stagnated

if I was getting a comparable salary yeah I'd just stay

[wind palm]

@bold terrace @cursive badge @cosmic hedge In your earlier discussion about using response times -- I'm surprised this didn't come up: https://expertium.github.io/Buttons.html

[cosmic hedge]

thats a pretty cool way to show it

[quasi shadow]

It's nice!

[ashen light]

@quasi shadow https://github.com/L-M-Sherlock/anki/pull/1

[quasi shadow]

sample_weighted_interval or select_weighted_interval

What do you think?

[ashen light]

I like select

I was also thinking making it two functions would give better naming options, but that also feels a bit unnecessary

also I think the weird git perms issue I think is a failing of github to do git things, echoing back to my original "like git is actually too complicated for 99% of dev workflows", what I was doing was probably outside normal things github expects the 99% of people to do

also the issue was probably extended by you not having perms on my anki fork

¯_(ツ)_/¯

but yeah on my local end I just added your repo as a remote and pulled your stuff

[cursive badge]

Isn't this checkbox meant to solve that?

I've never actually tried to use it though, so it may be silly and only work with changes made in the GitHub UI.

[sick moth]

It lets maintainers push to the branch on the fork. I use it quite a lot. You don't need GitHub, but gh is recommended, just to setup the refs correctly

[quasi shadow]

😅 I'm integrating Easy Days into the simulator and then I find I have to copy a bunch of code. Maybe I should move the simulator code into Anki's code base?

[cursive badge]

It probably make sense if you have no reason to use the simulator code outside of Anki.

[cosmic hedge]

the problem then would be maintaining 2 flavors of simulator, one for anki, one for everything else

would it be possible to pass the easy days function as an argument somehow?

that way just the easy days part could be part of the anki code

[cursive badge]

That was what I was just trying to type

[cosmic hedge]

yeah maybe the find_interval function should be passed through the simulator config

[cursive badge]

Have the simulator do "pure" scheduling then have a hook for "post-scheduling adjustments".

[quasi shadow]

😂 Sounds very complex to me. I know how to pass function as an arg in Python. But it seems difficult in Rust?

[cosmic hedge]

yeah rusts first class functions are weird 😂

but im pretty sure they do exist

ok maybe not too weird

[cursive badge]

They are not bad if they are pure functions. The weirdness only happens if they are closures that access captured variables.

Then you start getting into FnMut, FnOnce etc.

[quasi shadow]

Well. Then what I need to do is defining the context of "post-scheduling".

[cosmic hedge]

just a guess but how about ```rs
ivl = next_interval(stability, desired_retention)
ivl = config.find_interval(
ivl
max_ivl,
day_index,
&due_cnt_per_day,
&mut rng,
);

// config.find_interval's arguments
fn default_find_interval(
ivl: u32,
max_ivl: f32,
today: usize,
due_cnt_per_day: &Array1<usize>,
rng: &mut StdRng,
)

[ashen light]

probably, yeah. I don't do weird git shit like this often enough to know to check that box

also taking functions as arguments in rust is pretty straightforward I think, even when doing FnOnce, etc

[robust hill]

"fsrs parameters currently optimal" for my deck with leeches

and then i do the deck, and this becomes the distribution, then i reschedule with fsrs helper add on and it becomes the after picture,

for 5 days in a row its become like this, so is fsrs even adapting, like what is going on. i am optimizing everyday and today even it says "parameters are optimal"

[lavish stump]

hi guys! I sent this on the dev-general channel, but not sure if that was the correct channel. I was wondering if any of you could help me with a bug(?) in my implementation of FSRS-V5 in my web-app. I used the github formulas from The Algorithm to develop it.

Basically, when pressing "easy" on repeat intervals (in the same day), the interval for that card jumps up seemingly stupid amounts of time. Pressing easy once (first review) results in 20+ days as the first interval, and then doing it 4 times results in 1000 years!!

I know that anki caps the intervals at 100 years, but even using the simulator https://huggingface.co/spaces/open-spaced-repetition/fsrs4anki_previewer I'm getting results that seem way more reasonable than mine.

I can paste my code here, but not sure if I'm supposed to

for reference. This was the output of me creating a new card, immediately testing it and selecting "easy" as the grade

[unique salmon]

What sequence do you get with your code?

[unique salmon]

Difficulty looks ok btw

[lavish stump]

nowhere near that sadly. 2 reviews of pressing "easy" results in an interval of over a month

this is the result of me pressing easy again in the same day

[unique salmon]

So does this one though (ignore the first interval of 0)

[lavish stump]

oh? I think I'm misunderstanding it, how do I read the "interval history" sorry

[unique salmon]

Just ignore the first zero

[lavish stump]

my understanding is the first review is 0days, the 2nd review results in 0 days and so on

so the third review would be an interval of 16 days

[unique salmon]

16 days -> 5 months -> 3.4 years -> 25.5 years -> 100 (or more) years

[lavish stump]

yeye! So mine seems to jump to 6 months on the 2nd review

[unique salmon]

Btw, I recommend checking these repos:
https://github.com/open-spaced-repetition/fsrs4anki
https://github.com/open-spaced-repetition/fsrs-optimizer

https://github.com/open-spaced-repetition/fsrs-optimizer/blob/a35b7b60e906d808ea51f96c583df7fec08f7857/src/fsrs_optimizer/fsrs_optimizer.py#L96

[lavish stump]

ooh thankyou! I'll have a look at those

[unique salmon]

Can you post the full interval history for, say, 7 "Easy" ratings?

[lavish stump]

hmm it would require some work and I have to quickly go afk right now. But my interval history for 3 easy ratings is as follows: 15.691049999999997, 166.1424212259587, 1759.1750794767652 (all in number of days)

I'll try and get some more results in a bit sorry. Really appreciate the support!

[unique salmon]

Btw, you can ask @quasi shadow, he's the dev of FSRS

[cosmic hedge]

purely out of curiosity is this you? its just the 3 hours ago commit kinda lines up 😅
https://github.com/AustinShelby/simple-ts-fsrs

[lavish stump]

ahh perfect! Wasn't sure what his tag was. Thanks for that!

[lavish stump]

no sorry '^_^ mines a private repo so can't share the whole codebase

can share the function though, it's written in python

[cosmic hedge]

https://github.com/open-spaced-repetition/py-fsrs just in case you dont already know about it

[lavish stump]

ohh I saw that. I compared the code to my own and it seemed to be quite similar. In fact the results I get seem to be similar to what Expertium said:

16 days -> 5 months -> 3.4 years -> 25.5 years -> 100 (or more) years

I just wasn't sure if this was intended, since after the 7th or so review it gave me something like 1000 years as the interval

I'm touched that the algorithm thinks I'll remember the von neumann architecture after I become a force ghost

[cosmic hedge]

you can paste your code if you want 👉👈

[lavish stump]

So I ran it for 7 "easy" ratings and I got
15.691049999999997
166.1424212259587
1759.1750794767652
18626.77176254222
197226.8879553684
2088308.4749437282
22111753.279343344

[lavish stump]

sure thing! I wasn't sure if I could post it on this channel (didn't read server rules sorry)

[lavish stump]

I should note that these intervals are all in "number of days"

should also note that after the 7th loop, python spits out an overflow error for the date (as it should ofc). Never seen that in my life

[cosmic hedge]

    elif "stability" in card and daysSinceLastReview <= 0:
        originalStability = card["stability"]
        card['stability'] = originalStability * math.exp(math.exp(w[17] * (grade - 3 + w[18]))) # <<<<<<<<

do you have an extra math.exp here?

my best guess though i'm not really the one to ask

¯_(ツ)_/¯

[lavish stump]

Oh for the love of…

You’re absolutely right

Yeah that seemed to produce much more reasonable results. Thankyou so much!! @cosmic hedge

but now the results seem to be rather different to https://huggingface.co/spaces/open-spaced-repetition/fsrs4anki_previewer when I test for 7 "easy"s

This is what I'm getting:
15.691049999999997
37.027032023230845
87.37472001232312
206.18292312605706
486.54116182357825
1148.1178875512842
2709.276803579464

And to me, these results seem to make a lot more sense. The jump from 15 days to 37 days seems much more reasonable than 16 days to 5 months, to me at least

[tepid spoke]

Are you testing with the same parameters?

[quasi shadow]

😂

Simulation with easy days is hard to plot.

And the simulator assumes the user learns the same number of new cards every day.

[robust hill]

😭

[quasi shadow]

It works!

But we need to refactor the graph😅

[quasi shadow]

https://github.com/open-spaced-repetition/fsrs-rs/pull/283

[quasi shadow]

https://github.com/ankitects/anki/pull/3829

[cosmic hedge]

I have no idea how to make that look better 😭

[cosmic hedge]

while we're making breaking changes, do you think adding the card_priority function into the config might be a good idea? just in case we want to set that up for the simulator in the future?

maybe something like this

Reverse((card.due as i32, learn, config.priority_fn(card))
```?

[quasi shadow]

It will be great!

Finally we can simulate most configs of Anki's scheduler.

will jump from 2.x to 3.x in two months. 😂

[cosmic hedge]

lets hope we don't miss anything we need with the simulator 😂

[quasi shadow]

I have no idea whether it is possible to implement the bottom 3 order.

[cosmic hedge]

can you pass closures as functions in rust? maybe if you include an optional "id" in the card structure you could access whatever card info you wanted

[quasi shadow]

Could you provide a case where we need to pass closures as a function instead pass the function directly?

[cosmic hedge]

I'm imagining it something like this

let anki_cards = {1: AnkiCard {id:1, added:1}, 2: AnkiCard {id: 2, added: 4}}

let added_priority_fn = |card| anki_cards[card.id].added

let config = {
  // ...
  priority_fn: added_priority_fn
}

[quasi shadow]

I will try.

[lavish stump]

I am yeah. The default parameters in the simulator are just the V5 default parameters right? I just copied and pasted the parameters

[cursive badge]

You obviously missed the old Rust crate trick of never calling something a v1 release, no matter how complete it is or how many people use it.
That way you can make breaking changes each release and if anyone complains you go "I'm allowed! It's a 0.x release!"
;p

[quasi shadow]

Unfortunately, fsrs-rs is my first Rust project😂

So I don't know about that.

[quasi shadow]

I felt terrible when I updated burn-rs from v0.13 to v0.14. They refactored a basic Data Structure, so I had to update 1000+ line of code.😂 (not a simple replacement of name)

[quasi shadow]

https://github.com/open-spaced-repetition/fsrs-rs/pull/284

[cosmic hedge]

Aww i missed my chance to review the other one 🥲
guess you both have it covered 😂

[cosmic hedge]

I wonder if there would be a way to test this just from the outputs we're already given 🤔

it would probably be more effort than its worth though 🤷‍♂️

[quasi shadow]

I haven’t released the next version, so you still can review it

I plan to release 3.0.0 in the next month.

[unique salmon]

So what does it do, exactly?

[cosmic hedge]

it allows the simulator to respect priorities

like review order that sorta thing

[unique salmon]

Got it

[cosmic hedge]

i think Asuka did a better job than i could have 😂
no idea what the "orphan rule" is 😂

[quasi shadow]

He is my Rust tutor😆

[cursive badge]

https://doc.rust-lang.org/book/ch10-02-traits.html#implementing-a-trait-on-a-type

(look to the bottom of the section)

But we can’t implement external traits on external types. For example, we can’t implement the Display trait on Vec<T> within our aggregator crate because Display and Vec<T> are both defined in the standard library and aren’t local to our aggregator crate. This restriction is part of a property called coherence, and more specifically the orphan rule, so named because the parent type is not present. This rule ensures that other people’s code can’t break your code and vice versa. Without the rule, two crates could implement the same trait for the same type, and Rust wouldn’t know which implementation to use.

[cosmic hedge]

my head hurts

[cursive badge]

Basically it stops two crates implementing exactly the same trait.

[cosmic hedge]

so you cant implement a trait from one crate on a struct from another crate?

[cursive badge]

Yes, but you can use newtypes to get something effectively the same without causing a clash and possibly breaking things.

[cosmic hedge]

so you wrap the external type and implement the trait on that?

i hope i never have this problem XD

[bold terrace]

I never really understood if Anki was Python, Rust, or some blend of both ?

[cursive badge]

Both, also JavaScript

[bold terrace]

Interesting, should take a look to see how things mesh together

[cursive badge]

I made a terrible diagram

[bold terrace]

That's quite helpful actually

[cosmic hedge]

Nooo the bad versioning advice spreads!!
soon the whole registry's going to be 0.X crates now the secret is out!

[hasty fractal]

we need a diagram for note type/note/card/card type/fields

[cursive badge]

I'm not sure what kind of diagram you would want. An entity relationship diagram could be a bit daunting.

I did it with mermaid if you want to have a play:
https://mermaid.js.org/

N.B. It works in Obsidian if you mark the code block as
```mermaid

[azure mica]

lol tbh I wish we would've kept py-fsrs as 0.x.y to this point too 😂

[azure mica]

I wish anki were like a tauri app or something

also I like that diagram

[cursive badge]

If rand does it with almost half a billion downloads you can do it too! ;p

[azure mica]

• we could've done it. It's 5.0.0 now

[cursive badge]

I think the idea is to slowly edge to something more like tauri with the svelte migration. It is tempting to just burn it all, drop all existing addon support and start fresh though.

[azure mica]

that'd honestly be sick. Being able to write addons in js would revolutionize the anki ecosystem

[cursive badge]

I dream of embedding a Wasm/JS runtime in rslib but they wont let me *sulks in corner*

[quasi shadow]

If Anki add-on was based on JavaScript, I would be a frontend programmer now.🤣

My first language is Python because of Anki.😆

[cosmic hedge]

Any language is a frontend language if you're brave enough 😂
Besides, some Anki add-ons are written in javascript 👀👀

[quasi shadow]

Yeah, I know. But the configuration of env of JS is confusing to me.

[cosmic hedge]

I did notice all the javascript in strings in the fsrs helper 😂

[quasi shadow]

They were developed by other contributors.

Then I have to learn how to maintain them.

https://github.com/open-spaced-repetition/fsrs-rs/pull/284/commits/e3739d04cdded43d8b22dde5d6ff0c3e680b8622

I added some useful tests for review priority fn

[cosmic hedge]

I was trying to try something to help but windows wont let me run rust 😭😭

[quasi shadow]

https://github.com/ankitects/anki/pull/3829#issuecomment-2668049176

[cosmic hedge]

could you re-run it with easy days so we can see if it still looks janky?

i'm guessing it does but that definitely helps

[quasi shadow]

But it still doesn't look better for long-term simulation.

[cosmic hedge]

I have an idea, what about a rolling average but its range is tied to the learn span (learn_span / 365)

that way you'd still get to see stuff day by day with the 365 day graph but not get weird results with the longer ones

wait maybe i should suggest this in the pr 😅

[quasi shadow]

For some reasons, I removed the rolling average in https://github.com/ankitects/anki/pull/3645

The problem was detected by @cursive badge

[cosmic hedge]

i didn't like the moving average either
actually me not knowing it was a thing was why I originally looked at the simulator code 😂

but maybe to save everyone's eyes we might want to bring it back, maybe even just toggled on if there are easy days?

[quasi shadow]

I wonder whether the user really want to know the exact number of reviews in a given date?

Easy Days vs Non Easy Days

[cosmic hedge]

that does actually look pretty useful for determining how much easy days is going to increase your workload on non easy days

and for that i guess you'd need no moving average

[quasi shadow]

Right.

🤣 I suggest merge it and leave the problem for the future.

[cursive badge]

The thing I found weird was that the simulator could show fewer reviews than had already been scheduled, which made me a bit distrustful of the rest of the graph.
If you wanted to bring back smoothing without that issue a couple of solutions I can think of:

• Add a toggle to enable smoothing so the user explicitly knows what is going on.
• Only smooth the number of reviews scheduled by the simulator, not pre-existing ones.

[unique salmon]

As much as I am against adding more and more buttons and bells and whistles, I think this one cannot be solved without a "Smooth" toggle

Btw, Jarrett, will CMRR be moved to the same window?

[quasi shadow]

Nope

[quasi shadow]

These setting doesn't affect CMRR.

[unique salmon]

A bit random, but how's Remnote doing?

Last time I checked they were still on FSRS-4.5 or FSRS v4

[quasi shadow]

I don't know. Remnote is close-source.

[unique salmon]

Yes, but I thought you were like a consultant for them

[unique salmon]

I went through their changelod and they only mention FSRS-4.5

[cursive badge]

I had a go at @bold terrace 's svsr heatmap idea.

I'm not sure if it something for main Anki but maybe @cosmic hedge wants it for his graph collection?

[unique salmon]

Could also make difficulty vs RMSE
I'm fairly sure it will reveal that FSRS isn't good for very easy and very hard cards

[cursive badge]

😳

[lament coral]

I just optimized FSRS using the built in Anki settings option page, and then used FSRS helper to reschedule all instead of the built in Anki one. Is that ok? My number of reviews today went quite a bit down, (150 -> 100) I was a bit confused what to use FSRS helper with vs built in FSRS.

[unique salmon]

The Helper add-on's version supports load balancing, the native one doesn't. Hopefully, this will change in the next release (no idea when that will happen, though)

[cursive badge]

It's ok.
FSRS Helper tries to spread reviews out a bit. In my experience you will usually find your reviews per day will go up for a while before stabilising again.

[cosmic hedge]

😳 😳 (suspending this card was apparently the best descision of my life)

[cosmic hedge]

yeah pr it if you want https://github.com/Luc-Mcgrady/Anki-Search-Stats-Extended

more graphs + me not having to do any work makes me happy 😄

[cursive badge]

At the moment I've implemented it as a native Anki stat and it's very rough.

d3 leaves you to fend for yourself a bit

[cosmic hedge]

ahh i'm assuming you did it in rust then

[cursive badge]

also:

[cosmic hedge]

https://github.com/Luc-Mcgrady/Anki-Search-Stats-Extended/blob/main/src/ts/CardDataPies.ts all my graphs are done in 2 big cursed for loops because I assume the less for loops = the better

the second one is in another file it does the (prepare graphs) stuff
and technically the third would be the "memorised over time" one

my next plan is to try and do a "difficulty over time" one based on the memorised one

[unique salmon]

@cosmic hedge We have this in the Google Colab optimizer

Here the percentages are differences between average predictions within a bin and average true retention

B_W_Metric["B-W"] = B_W_Metric["p"] - B_W_Metric["y"]

Positive = overestimation
Negative = underestimation

So I recommend doing something like this

https://github.com/open-spaced-repetition/fsrs-optimizer/blob/a35b7b60e906d808ea51f96c583df7fec08f7857/src/fsrs_optimizer/fsrs_optimizer.py#L1880

[cosmic hedge]

I'll give it a try!
I originally thought if i'd do something like that id do it in 2 seperate graphs that were like that fatigue one that didnt pan out but thats a much better idea XD.

[pure ore]

I'm looking at FSRS implementations, because I'm thinking about making my own implementation of FSRS for an app that I'm working on. Why do they all define four 'next' values? E.g. nextAgain, nextHard, nextGood and nextEasy.

Is this to precompute the potential DSR values in advance and show how far out the next study interval would be depending on which option you click, like in Anki?

[unique salmon]

In Anki there are 4 answer buttons: Again, Hard, Good and Easy. And when a user clicks "Show Answe", we need to show the length of the next interval above each of the 4 buttons

[pure ore]

Gotcha. If I don't care about showing the user that, can I defer calculating next interval until the user answers and only compute it once based on their answer?

[unique salmon]

How many answer buttons will your app have?

[pure ore]

Three - Good, Easy, and Again. But I don't care about showing intervals at all.

[unique salmon]

Then yeah, I guess you don't need to precompute them

[pure ore]

Alright, thanks for the explanation 🙂

Are all Ratings based on a 5 point scale? Thus, for me, Easy = 5, Good = 3.5-4.0 and Again = 0?

[unique salmon]

In FSRS, Again=1, Hard=2, Good=3, Easy=4. These numbers are used for calculating difficulty

In terms of which is fail and which is pass, Again is fail, the rest is pass

[pure ore]

Perfect, thanks

[bold terrace]

I love it like that! Is it manageable to click on one square and get those matched in the browse view ?

[cursive badge]

That was meant to be a comment on how it doesn't resize. It remorselessly just overflows as you resize the window. 🤣

[nimble halo]

I just noticed Anki got updated: are we to trust the 0.70 minimum recommanded retention? Anki computed this on two separate decks (one easy and one hard). The previous recommanded values were in the range of 0.82-0.86 IIRC. What's weird is that the algo still computes 0.82 with the two weeks old FRSR settings, and only yield 0.70 after optimizing the preset. Was FRSR updated with this release?

[bold terrace]

Oh my bad 😅

[unique salmon]

Not FSRS itself, but a bug with the simulator/CMRR was fixed

[bold terrace]

It’s nice because for ex in your example we can see most of your 120-160d stability are around the same R which means you could have a spike of them at some point

[unique salmon]

Oh, no, wait FSRS was updated yeah

For some reason I thought the recency weighting update was in 24.11
Anyway, FSRS now assigns more "importance" to newer reviews

[nimble halo]

Sweet, as I posted on another channel I was always under the impression that the simulator overestimated my load by at least a factor of 1.5x. Still, I find this 0.70 recommanded desired retention weird. Is it worrying? Or do you think it's legit?

[unique salmon]

Idk. Just as a general rule of thumb, it tends to be lower for harder material and higher for easier material

[bold terrace]

In my experience with the optimal retention, I'd be cautious to change too quickly your DR. FSRS is really good at predicting a certain DR but not that great to "translate" that to another one, but it depends... Personally when FSRS was trained heavily on DR=80% (for True around 76%), and DR=70% led me to True ~60%

BUT I think one tricky thing is how the LB can shift low stability cards of ~1-2d very easily and lead you to disastrous R

[cursive badge]

It's very rough at the moment. I'm also not convinced it is something that should go into main Anki. I might rework it for Abloke's addon. If you are happy with running Anki from source I could share what I currently have.

[bold terrace]

But still, if you're at 80% and want to venture south to it, I'd be doing small steps, like 1% decrement every week

[bold terrace]

Indeed, I find it cool but I'm also not sure if it's not something better for the search stats extended addon 🙂

[nimble halo]

I have been very conservative with this setting, especially since I'm learning a tons of new cards a day (I fear my retention would be horrendous at DR=80%, since I barely reach 78% with DR=88% on my hard deck)

And thanks for the info :)

[bold terrace]

Yep I was at 40 new/day and it created a huge drop of retention I had to recover from. But now I'm mostly happy with my setup, I use a simple Filtered Deck that give me all cards with prop:r < my_desired_retention ... Normally it's what Anki is expected to do, but even with the default LB, you get a +1d for a lot of low interval cards.

If you want to check by yourself, you can install this addon https://ankiweb.net/shared/info/1613056169 and check this graph :

My Future Due for tomorrow is around 83.92 instead of 84% because I manually get all those low stability + prop:r < desired manually

Without those filtered deck, it would be lower, like 75-80 range

My filtered decks (
"deck:Japan::1. Vocabulary" prop:r<0.84 -is:due
"deck:Japan::1. Vocabulary" prop:reps>25 -introduced:30 prop:s<1.3 -prop:rated=0 -is:due
)

And also a different one for the top 1000 words, to have a higher DR :
"deck:Japan::1. Vocabulary" prop:r<0.9 Frequency:re:^[0-9]{3}$ -is:due

At this point, half my reviews are from those
But I get a good stable retention now (I marked in red when I started using those)

[cursive badge]

Also don't feel any pressure to reduce your DR to match CMRR.
It is the "Recommended Minimum", not "Recommended" DR. i.e. "You shouldn't go below this" not "It should be this".
It's just there to stop you from accidentally setting DR so low that the workload/learn rate trade-off does not make sense.

[unique salmon]

I mean, it's meant to be the optimal value

[lament coral]

hey guys, my recommended learning steps seems...aggressive, am I reading this wrong? The numbers in the far right column also seem pretty low for 1 month of reviews, wasn't sure if it's tracking correctly.

[nimble halo]

I'll definitely start playing with filtered decks soon. It sounds like a simple yet quite powerful tool

Thanks for the filters

[cursive badge]

"Optimal" in the sense of minimum workload to amount learnt. If you don't mind spending more time to learn more it becomes a trade-off.

[bold terrace]

yeah ... And I'm not entirely sure the real cost of forgetting something is really that easy to considerate

The concept is nice and make sense though

But since for example, you might have your Retention lower than your DR, even if the prediction is 100% accurate, once again, the LB might lead you to have your average R at ~60% instead of 70%

And that 60% might be a 55%

When I dropped my retention to 70% the crash was quite brutal in my case

[nimble halo]

With the simulator that seemed faulty (and happened to be) I was not convinced by the predicted workload at different DR, so until now I was very careful and conservative with it. And noted, I still won't go crazy with it.

[cursive badge]

I see it as there is not a single "Optimal" DR. There is a Pareto Front and CMRR is making sure you don't stray from it.

[bold terrace]

Also, I start to have the instinct that keeping your retention at aronud 70-80%, never truly make you "encode it better" leading quicker to higher stability