#ChronoGrapher - One Unified Scheduler, Unlimited Power

1 messages · Page 3 of 1

fervent lark
#

python is the opposite of good for performance

#

xd

placid drum
#

haa i used to use moviepy, for auto video generation 99 percent cpu usage

fervent lark
#

ye lol

#

threading is terrible there

placid drum
#

fr

fervent lark
#

i've supported anyhow and even eyre

#

in the future, perhaps it will be a good idea to support tracing

fervent lark
#

@jolly frost @placid drum @zenith urchin @arctic trench Gentlemen, i think once i am done with the collection taskframe, we can start with the API documentation rewritting on tommorow

#

make sure you read some of the API docs guidelines to get an idea

#

after we do:

  • Finish the guidebook
  • Write proceedual macros to add on top
  • Write unit tests for ChronoGrapher
  • Rewrite API documentation fully with the guidelines in mind
  • Finish the entire contribution guidelines
  • Make the CI/CD pipeline more strict and more complete with stuff like https://codspeed.io/ for performance
  • Finish the landing page of the ChronoGrapher plus roadmap section
  • Make and finish the python and even JS/TS bindings (SDKs)
CodSpeed: Optimize Performance, Eliminate Regressions

CodSpeed integrates into dev and CI workflows to measure performance, detect regressions, and enable actionable optimizations.

#

i think ChronoGrapher as v0.0.1a will be exceptionally strong

#

even if persistence will be missing

#

the goal is to deliver ChronoGrapher somewhere early to mid April

#

even if core is one of the smaller parts, its perhaps one of the most importants, and just because we will finish it, doesn't mean sometimes we won't revisit it. Though we will rarely do as pretty much everything our user base and us ever needs will be possible with the core

fervent lark
#

so far the CollectionTaskFrame is left to make and we are ready, more specifically the ParallelExecStrategy and SelectionExecStrategy with the polices and so on (actually Sequential will need it too but im kinda lazy, @jolly frost do finish it rq)

#

il handle SchedulerHandler instructions

jolly frost
fervent lark
#

for the scheduler instructions, hmmm, i may need to look how to best make it

#

in a way that doesn't share the SchedulerConfig

#

the idea is via ctx

#

one can send instructions from the Task regardless where in the workflow they are to the Scheduler

fervent lark
#

btw i also feel like GSIs can be more powerful (though not too costly in performance)

fervent lark
#

GSIs apparently are not even needed

#

a bit crazy

#

but hear me out

#

with a newtype struct, you can wrap an already-defined SchedulerTaskStore

#

and on store simply append your own info

#

the only thing i did is made it possible to modify an erased task

fervent lark
#

just like shared data API

fervent lark
#

hmmm, im gonna need some restructuring to do in Scheduling land

#

currently some instructions like

#

Halt and Execute

#

are painful

fervent lark
#

guys

#

we have to move quick

#

we are too slow rn

#

the plan is to get it til mid April, we have around 2 months

#

not just finish the core

#

but like produce bindings, the guidebook, unit tests, strengthen the CI/CD pipeline

#

and everything around

fervent lark
#

also SchedulerHandle instructions will be a part of the scheduler engine

#

which means it sometimes may be optional

#

HOL UP

#

nvm

#

the fuck are these numbers

#

118.5k empty-ish tasks per second

#

that is poor af

#

we will definitely need to optimize this

fervent lark
#

like tokio_schedule is around 10x faster

#

the goal is for every Task to do 1 microseconds at most

#

not the execution but scheduling

fervent lark
#

slowly the Tasks To Workflows chapter series is coming to a close

#

right now i have to document Error Handling & Introspection, Conditionals & Thresholds, Dependency Management and a Summary / Practical Patterns chapter

#

this is 9 chapters (maybe even 10) worth of material

#

im half way

#

and once that is finished i gotta document TaskHooks which is another chapter series, the Shared Data API, the Scheduler side and so on

fervent lark
#

thats like writing a novel, thats how deep it is

#

and im like 8 chapters in, xd

#

the 8 chapters took around a month

fervent lark
#

i may need to revisit error handling a bit

#

there is a slight problem in my newtype approach

#

updating the workflow with a new retry or modifying it in any capacity will result in a different error type all toggether

fervent lark
#

this is all fine when it comes to destructuring the error if users actually need it

#

but for those who don't (most of us), its bloat, we have to unwrap individually every layer

fervent lark
#

,_, im in pain

#

i gotta make TaskFrame accepts any arguments and so on

#

and this argument generic leaks throughout the design

#

in both TaskContext, RestrictedTaskContext... etc.

fervent lark
#

honestly i have no idea what to do really

fervent lark
#

i increased performance apparently

#

there a lot of context switches happening

#

which killed performance

#

by just making the clock sync and some other methods in task store

#

im hitting higher performances

#

+85% to be precise on average

#

there were a lot of context switches that happened

fervent lark
#

and there still are

fervent lark
#

the SchedulerHandle btw drops performance massively

#

it happens every time a Task runs

#

holy shit

#

thats a massive performance boost

#

around +60% performance enhancement

fervent lark
#

generally a 7x increase in performance

#

all that from some simple changes

#

with the timewheel, this can go even faster

#

might be worth looking into

#

for 200k tasks the process uses 3.2GB

#

hmm

#

as such somewhere to 16KB per task

#

what in the f_ck

#

once optimizing runtime performance is done

#

then memory will follow next

fervent lark
#

i will have to remake the task store in a way which allows higher performance

fervent lark
#

but

#

have u read the API docs by chance?

#

so we can get ready for this rewrite

zenith urchin
#

I haven’t read it. Sorry :/ I had to work on my thesis due to meeting today

fervent lark
#

its ok though

zenith urchin
#

It was but we had to schedule one more because of issues on some tests in the implementation. I will complete reading it tomorrow afternoon

fervent lark
#

ah i see

#

okie then gl with that

zenith urchin
#

Also, answer your message in detail about my comments

fervent lark
zenith urchin
fervent lark
#

^

#

and even provided some stuff you can do as an extra mile

#

but even that

#

thats helpful

zenith urchin
fervent lark
#

oh

#

ok ig

fervent lark
zenith urchin
# fervent lark thats helpful

Thanks 👍🏻 will get back to you. I am reading messages here on my commute time, but I don’t comment since I am not that deep in the code yet.

fervent lark
zenith urchin
fervent lark
#

especially for this kind of project

#

plus

fervent lark
#

we do have a deadline of around 2 months

#

somewhere in mid April

#

the reason for this is the core has been delayed like twice i think

zenith urchin
fervent lark
#

yup

#

if possible, not pressurizing, try your best this time period (assuming you can be free)

zenith urchin
#

Yeah I am not that free 😄 I have two deadlines in the same period but I will give my best to work on it don’t worry

fervent lark
#

ik i am asking a lot, but delivering the project in a good state and fast is quite important

zenith urchin
#

Yeah I also think accomplishing deadlines are important. Is this deadline decided by you or something else?

fervent lark
#

a bit

zenith urchin
#

Okay let’s make it happen ferrisCat

fervent lark
#

this is gonna be a little complex, do want it to integrate this a bit since it seems quite optimized

zenith urchin
#

@fervent lark should I read API documentation guidelines and the rest? Are they up-to-date or should I dive into code to understand it? If they are not aligning with the codebase, it might be confusing.

fervent lark
#

the guidebook guidelines yes

#

and

#

up until Advanced Retries its up to date

#

also keep in mind it uses the supposed macro syntax not yet implemented, but you can translate it to the "low-level" APIs

fervent lark
#

though it doesn't do yields and stuff

#

with the yield statement it goes to like 700k tasks per second, which is still hella optimized

#

and it even used mutex to top it all off, which is slower

fervent lark
#

that way the entire ecosystem benifits

#

and it should be with async channels, no mutex for maximum throughput

fervent lark
#

we can go like 2 million tasks per second which is absurd

#

the orange is the predicted performance (the average of the blue and purple curve) and the red is the tokio_schedule

image.png
#

the purple curve is the current benchmark of ChronoGrapher and the blue curve is the benchmark done for a Mutex timewheel

#

worthwhile if you ask me

fervent lark
#

its gonna be difficult

#

hmm, i may need to think this through

#

the SchedulerClock rn misses one method, tick

#

the heartbeat basically

#

timer wheels requires it

formal sedge
fervent lark
fervent lark
#

Im analyzing both CPU and Memory usage

#

the 3.2GB is consistent

#

it doesn't fluctate

#

it just stays there

#

jesus f_ck

#

honestly the async stuff is kinda unreadable

#

drop time

#

is a culprit

#

so typed arena

#

this will present some problems, one thing to do is to break apart the arenas for each type and consectively break it apart for use, the smaller arenas the better

#

though il have a problem with trait objects

#

ig we will go with bumpalo

fervent lark
#

hmm apparently, it will be difficult to integrate bumpalo

fervent lark
#

hmm bumpalo might make it around at best (assuming the drop time is zero) +16.6% faster

#

so from 350k tasks it goes to around 400k tasks

#

realistically this would be 390k tasks per second

zenith urchin
#

Hey I am giving feedback to the remaining part which is Introspecting Workflows and so on.

  • I don't know if it is really required, but for TaskHook Implementation/Definition Phase, it required async_trait to be implemented. However, let's say I have an Workflow that listens a website every day at 9:00 UTC. Scraper should read the text, but also mark the places that contain images then we can do our trained OCR specific to that scraper. Now, we are trying to do something heavy CPU bound task as a async. Would it be possible to have maybe both async and sync implementation? Maybe we can again make a proc macro and define it under the arms since it cannot expand that much if we only handle async and sync actions in there. However, I think we can handle this with fallbacks since this is more of a fallback then a TaskHook. Can you maybe clarify?

We cannot route from clickables under the /docs/contributing. I think this is because we are already under the contributing path.

GET /contributing/api_docs_guidelines 404 in 737ms (compile: 672ms, render: 64ms)
GET /contributing/ai_use_guidelines 404 in 73ms (compile: 11ms, render: 61ms)

I looked into it and we should change the index.mdx /contributing/api_docs_guidelines to /docs/contributing/api_docs_guidelines, so correct link will be routed. Should I create a quick PR for it or would you change it?
So one important note can be really long Guidebook Guidelines chapter which can be expected since it explains complete guidebook, but still it is kind of long and lots of text. I am used to read articles, but it felt little bit long and I was like most of these things I will forget because they are statements explaining what are the Guidebook components etc.

#
  • We will see about that, not sure tbh, we could, would need a bit more explanation
    Answering this, for the
For the workflow(retry(3, 2s)) whereas task interval was already 2s. We may get the workflow::delay::default as the time of the task. Since people will do the same time most of the time except people who are aware what they are doing, it might help to reduce boilerplate.

For example if we have a workflow as following:

#[workflow(
    -- timeout(None) --
    fallback(myfallback),
    delay(500ms),
    retry(3, 2s),
)]

We can set the timeout at the beginning as a default value. Using the following definitions, like ((fallback_time * is_occured) + delay) * retry if something like

#[workflow(
    -- timeout(None) --
    fallback(myfallback),
    retry(3, 2s),
    delay(500ms),
)]

happens, as far as I understood every retry will delay for 500ms like
retry 1. delay 1. retry 2. delay. ...
For this case, (fallback_time * is_occured) + (retry * delay) can be calculated, if I undertood correctly. Doing so we can remove headache of setting timeout for each workflow because most of the time devs will just want to timeout in the maximum amount of time within the defined workflow.

#
  • I think it is well organized and I grasped the concepts quite well at least on the surface level.
  • As I said on one of my previous feedback, it is professional, but still some documentation missing ofc due to being developed at the moment.
  • It was quite light-weight especially with the examples, but maybe we could have provide more realistic examples instead of just making up general examples. Also, Guidebook chapter under Contributing is quite heavy not in the cognitive perspective, but due to statement overload.
  • I couldn't get the question. What I enjoyed about the overall website or the project? I think it is website, but still not sure.
  • It was mostly frictionless except the missing documentation. Missing documentation hangs you in the middle.

Maybe TaskHooks could have been explained little bit more in detail. However, overall cool explanation with good examples. Great work 👍

zenith urchin
fervent lark
#

holy sh#t

#

my guy

#

@zenith urchin you insane? (joking ofc)

fervent lark
# zenith urchin Hey I am giving feedback to the remaining part which is Introspecting Workflows ...
  1. Well for async_trait it is required, funnily enough the thing you are saying is something we discussed with @arctic trench, its gonna be hella difficult, and i am not sure if i really should. We will see, its not something i can gurantee to be async and sync both. Do need some explaination on what you mean for fallback and TaskHook stuff

Yeah open up a PR. For the Guidebook Guidelines, honestly it even clarifies, its not meant to be done in one sitting, moreover its something you come back to and see stuff. You SHOULDN'T remember every tiny detail, the idea is say you want to use a codeblock, you take a look in the codeblock header. You will have to remember the rules, now exactly, i don't expect it, this comes mostly from experience

fervent lark
fervent lark
fervent lark
#

if its unlimited

#

oh i think i get it

#

so let me get this straight, the workflow::delay::default is just a constant you can define and use for delays

#

if its like that, then i believe it might be better to allow referencing constants

#

like you could define

# 
const ABC: usize = 30;

or something along those lines

#

then use it

#

for delays it would be Duration since 3s and so on are translated under the hood to duration

fervent lark
#

🤔 for the guidebook guidelines honestly im thinking the most

#

its a valid concern, i do get it, and i should fix this definitely. Maybe split the guidebook guidelines in two or more parts?

#

Then like make it more aligned with the guidebook docs standard

arctic trench
# fervent lark 1. Well for async_trait it is required, funnily enough the thing you are saying ...

@fervent lark If the OCR tool specifically is Tesseract, normally every language has a wrapper lib like pytesseract; under the hood, it executes a bash command. In these cases, it is possible to convert to async.

But most of the other solutions, like dedicated OCR, use deep learning models (PyTorch, TensorFlow with options to use CPU or GPU). For these cases, it is difficult to maintain async.

And… well, please excuse not giving a sign of life. I just got back from my postgraduate studies today, and my week was full… as always.

tomorrow i finally will have some time to do something here.

zenith urchin
# fervent lark 1. Well for async_trait it is required, funnily enough the thing you are saying ...

Yeah I think it might be little hard to maintain like it is the same for the tokio library. They were also suffering for the async sync and had to add some boilerplate in it, so it can work for both. I am not even sure if they still provide sync or completely removed it. We can continue with async for the MVP and then add sync if it makes really sense. As @arctic trench said, most of the OCR related implementations are done by async wrappers or something cloud provided like Textract or Mathpix. About the Fallbacks and TaskHooks I think Fallbacks would be more reliable place to handle error handling etc. whereas TaskHooks handles more IO/Networking stuff. At least that's what I understood from the documentation. So if Fallbacksare already implemented as async and sync traits it should be fine already.

Working on it 👍

zenith urchin
zenith urchin
# fervent lark but then the problem is why put a timeout?

So let me explain what I understood from the Workflow and timeout interaction and then what I am suggesting.
Workflows can contain several timeouts if no timeout at the beginning of the Workflow like

#[workflow(
    fallback(myfallback),
    delay(500ms),
    retry(3, 2s),
)]

This means we will have no limitation in the current implementation because we didn't specify it. However, what I am implying is that if somebody uses some kind of Task Scheduler, they would probably expect some timeout to be applied, and most of the time the timeout they would add is the amount of time that total workflow takes. At the above example with different calculations, I wanted to display some of the possible scenarios for the minimum required time for Workflow before completed. Therefore, I thought it would be nice to provide some default() that calculates total minimum amount of time of the current workflow chain.

const ABC: usize = 30;

like constants might work for the beginning, but I think it would not be really useful because default total timeout should change from workflow to workflow. That's why what I am suggesting is some kind of a dynamic calculation of the workflow, so that users does not need to enter it manually everytime if they want to represent the timeout for the whole Workflow.

// some proc macro for workflow
// in the scope of the workflow collection
let global_timeout: Duration = 0;
// variants are the collection that contain delay, retry, fallback etc.
for variant in variants {
  // time_it() function should give 2s for retry(3, 2s)
  // I am assuming we are flattening retry timeout into 3 different variant in here.
  global_timeout += variant.time_it() // or variants.time()

  variant.run()
} 
// ...

Well, I was writing down the above code, then I realised if we implement such a for loop we are already running each right after time_it, so it wouldn't make sense to collect durations.

zenith urchin
zenith urchin
fervent lark
fervent lark
#

Post-Error Handling is meant to just do side-effects to fix some of the "error"

fervent lark
#

its not as easy to do so

#

the fact you can make your own backoff strategy (yes you can refer to your types), its difficult

#

I think with refering to defaults we could make the feature more powerful, one idea is to make it accept calculations

fervent lark
#

well...

fervent lark
fervent lark
#

again can't thank enough btw for this dedication

fervent lark
#

imm, im gonna make chrono optional and scrap the cron_parser crate

#

this means il have to write my own CRON parser

#

cron_parser doesn't parse at initilization time, so il have more control on that

#

which increases performance when using cron expressions

fervent lark
#

probably i should make cron expressions compile-time checked

zenith urchin
fervent lark
#

so its best to do the defaults approach and let the user do calculations on top of these defaults

fervent lark
#

i got the lexing part of cron done

fervent lark
#

now its time for parsing

fervent lark
#

ok now the lexing is complete

# 
enum ConstantVariant {
    Month,
    Day,
    None
}

enum Token {
    Numeric(usize, ConstantVariant),
    Minus,
    Wildcard,
    ListSeparator,
    Unspecified,
    Step,
    Last,
    NearestWeekday,
    NthWeekday,
}

impl FromStr for TaskScheduleCron {
    type Err = CronExpressionErrors;

    fn from_str(s: &str) -> Result<Self, Self::Err> {
        let mut tokens: [Vec<Token>; 6] = [const { Vec::new() }; 6];
        let mut current_number = 0usize;
        let mut is_number = false;
        let mut field_pos = 0;
        let mut char_buffer: String = String::with_capacity(3);
        let mut chars = s.chars().enumerate().peekable();
        while let Some((position, char)) = chars.next()  {
            if char == ' ' {
                if field_pos >= 5 {
                    return Err(CronExpressionErrors::UnknownFieldFormat);
                }

                if tokens[field_pos].is_empty() {
                    return Err(CronExpressionErrors::EmptyField(field_pos))
                }
                field_pos += 1;
                continue;
            }
# 
            if char.is_ascii() {
                char_buffer.push(char);
                if char_buffer.len() == 3 {
                    let num: usize;
                    let variant: ConstantVariant;
                    match &char_buffer[0..=2] {
                        "SUN" | "sun" => {
                            num = 1;
                            variant = ConstantVariant::Day;
                        },
                        "MON" | "mon" => {
                            num = 2;
                            variant = ConstantVariant::Day;
                        }
                        "TUE" | "tue" => {
                            num = 3;
                            variant = ConstantVariant::Day;
                        }
                        "WED" | "wed" => {
                            num = 4;
                            variant = ConstantVariant::Day;
                        }
                        "THU" | "thu" => {
                            num = 5;
                            variant = ConstantVariant::Day;
                        }
                        "FRI" | "fri" => {
                            num = 6;
                            variant = ConstantVariant::Day;
                        }
                        "SAT" | "sat" => {
                            num = 7;
                            variant = ConstantVariant::Day;
                        }
                        "JAN" | "jan" => {
                            num = 1;
                            variant = ConstantVariant::Month;
                        }
                        "FEB" | "feb" => {
                            num = 2;
                            variant = ConstantVariant::Month;
                        }
                        "MAR" | "mar" => {
                            num = 3;
                            variant = ConstantVariant::Month;
                        }
# 
                        "APR" | "apr" => {
                            num = 4;
                            variant = ConstantVariant::Month;
                        }
                        "MAY" | "may" => {
                            num = 5;
                            variant = ConstantVariant::Month;
                        }
                        "JUN" | "jun" => {
                            num = 6;
                            variant = ConstantVariant::Month;
                        }
                        "JUL" | "jul" => {
                            num = 7;
                            variant = ConstantVariant::Month;
                        }
                        "AUG" | "aug" => {
                            num = 8;
                            variant = ConstantVariant::Month;
                        }
                        "SEP" | "sep" => {
                            num = 9;
                            variant = ConstantVariant::Month;
                        }
                        "OCT" | "oct" => {
                            num = 10;
                            variant = ConstantVariant::Month;
                        }
                        "NOV" | "nov" => {
                            num = 11;
                            variant = ConstantVariant::Month;
                        }
                        "DEC" | "dec" => {
                            num = 12;
                            variant = ConstantVariant::Month;
                        }

                        _ => {
                            return Err(CronExpressionErrors::UnknownCharacter {field_pos, position, char })
                        }
                    }
                    tokens[field_pos].push(Token::Numeric(num, variant));
                    char_buffer.clear();
                    continue;
                }
            }
# 
if char.is_ascii_digit() {
                is_number = true;
                current_number = current_number * 10 + (char as u8 - b'0') as usize;
                continue;
            }

            if is_number {
                tokens[field_pos].push(Token::Numeric(current_number, ConstantVariant::None));
                current_number = 0;
                is_number = false;
            }

            match char {
                '-' => tokens[field_pos].push(Token::Minus),
                '*' => tokens[field_pos].push(Token::Wildcard),
                ',' => tokens[field_pos].push(Token::ListSeparator),
                '?' => tokens[field_pos].push(Token::Unspecified),
                '/' => tokens[field_pos].push(Token::Step),
                'L' => tokens[field_pos].push(Token::Last),
                '#' => tokens[field_pos].push(Token::NearestWeekday),
                'W' if !matches!(chars.peek(), Some((_, 'E' | 'e'))) => {
                    tokens[field_pos].push(Token::NthWeekday)
                },
                _ => return Err(CronExpressionErrors::UnknownCharacter {field_pos, position, char }),
            }
        }

        if field_pos != 5 && field_pos != 4 {
            return Err(CronExpressionErrors::UnknownFieldFormat)
        }

        if !char_buffer.is_empty() {
            let position = s.len() - char_buffer.len();
            return Err(CronExpressionErrors::UnknownCharacter {
                field_pos,
                position,
                char: s.as_bytes()[position] as char,
            });
        }

        if is_number {
            tokens[field_pos].push(Token::Numeric(current_number, ConstantVariant::None));
        }
#

hmm

#

parsing will be a lil difficult-ish

#

il also move the checking for constants to the lexing part

fervent lark
#

parsing is done

#

i will have to conduct semantic analysis

#

i have the Abstract Syntax Tree

#

im fixing some bugs

fervent lark
#

ok AST construction is done

#

now time for semantic analysis

#

more so "compiling whilist validating"

#

the idea for "compiling" is to bake the sturcture right into the cron struct to save lexing + parsing + validation costs and only use the primitives to translate the current time to future time

fervent lark
#

also @placid drum @formal sedge @zenith urchin @arctic trench Once @jolly frost finishes fully the Collection strategies, we will start with the API docs rewrites, again i suggest reading the API doc guidelines. The areas for each individual is split as to not collide, the documentation will include modules as well. You should all strictly follow the API doc guidelines (it may seem long but the header list is something you can always just refer back when needed)

For @formal sedge, you can do both unit-tests and API doc rewrite
For @arctic trench, you can also do critism plus the API docs rewrite (would be a good time to critique the API doc guidelines as well while at it)

Il alert once stuff is done and the areas each individual will be assigned to. Focus on these areas and only these areas, if you need to refer to a specific documentation present in another area, i'd suggest leaving a **``[TODO LINK TO X]``** (which turns into [TODO LINK TO X]) in between the text so once the other area is finished writing the API docs, then you can refer to it

We will need some coordination to write it out in a few days so, talk to me if needed and the other members

#

also @placid drum howz the progression going?

#

need help with anything

placid drum
#

reading docs right now no help need i will ask if i got stuck

fervent lark
#

oki dokie

fervent lark
#

or it was nothing?

placid drum
fervent lark
#

ah ok

#

hmmm

#

will need a profile of your experience level

#

with Rust, documentation and stuff

placid drum
#

well i am beginner with rust but did rust book and build some project i should start with docs i guess

fervent lark
#

hmm

#

guidebook docs will be a sensetive topic, API docs should be safe

#

though even on API docs

#

do be careful, i don't want you to just describe this thing, i want best quality

#

even if it takes longer, idc

placid drum
fervent lark
#

ok lol

#

with the added +10%, it should be enough

#

🤔 im also planning to do the proc-macro extension on top of ChronoGrapher, since samy is the only one comfortable (but focused in other areas) with Rust and most of you seem to be beginners (except perhaps bagu, they could help out). I will have to do it alone

placid drum
fervent lark
#

its gonna be a pain in the ass honestly

fervent lark
placid drum
fervent lark
#

and to give an idea

#

of what we have to finish

placid drum
#

ya list down all i might able to do something

fervent lark
#
  • Finish the guidebook fully
  • Write proceedual macros to add on top
  • Write unit tests for ChronoGrapher
  • Rewrite API documentation fully with the guidelines in mind
  • Finish the entire contribution guidelines
  • Make the CI/CD pipeline more strict and more complete with stuff like https://codspeed.io/ for performance
  • Finish the landing page of the ChronoGrapher plus roadmap section
  • Make and finish the python and even JS/TS bindings (SDKs)
CodSpeed: Optimize Performance, Eliminate Regressions

CodSpeed integrates into dev and CI workflows to measure performance, detect regressions, and enable actionable optimizations.

#

they seem a lot at first, but they are even more

jolly frost
light pierBOT
fervent lark
#

we will have to do API docs, unit tests, guidebook stuff and CI/CD sheninigans (to check per SDK and even package them) there as well

#

its doable, but i will need your fullest attention to the project from all you

#

and organize you effectively

placid drum
#

yes sir 🫡

zenith urchin
zenith urchin
# zenith urchin What kind of proc macros do you have in your mind? Like this workflow etc stuff?...

Or maybe if you already have a dedicated ticket for this you can also link it. Is it this one https://github.com/GitBrincie212/ChronoGrapher/issues/105?

GitHub
[QoL] ChronoGrapher Main Macro (Similar To Tokio) · Issue #105 · ...

What feels teadious about the current way of doing things? Please describe. Currently, for simple example, beginners experimenting often forget to put below: CHRONOGRAPHER_SCHEDULER.start().await; ...

fervent lark
#

as well as a cron macro for typed cron expressions

fervent lark
fervent lark
#

its kinda this one

#

not exactly

zenith urchin
#

Okay I might handle it depending on complexity 😄

fervent lark
#

oh good, good

zenith urchin
zenith urchin
fervent lark
#

foreshadowing

zenith urchin
#

Also, doing common folder helps a lot with code deduplication with proper folder structure since there will be lots of core features used across SDKs

fervent lark
#

I've published some new changes, @jolly frost has finished some of the stuff so now i shall announce for @zenith urchin @placid drum @formal sedge @arctic trench and ofc @jolly frost . It is time to do the API documentation. The areas in which each individual is assigned are as follows:

Note: For type-aliases, they shouldn't be heavily documented, some headers can be omitted, though i haven't formed a conclusion on how the structure of type aliases docs looks.

KEEP IN MIND, EVERYTHING INTERNALLY USED, SHOULD NOT BE DOCUMENTED, ONLY PUBLIC / USER-FACING THINGS

@placid drum

  • everything under core/src/task/dependency
  • everything inside of core/src/task/frame_builder.rs (Describe the module briefly, not too much in detail as its kind of obvious)
  • everything inside of core/src/task/dependency.rs (Unlike the previous, this one does need a thorough description)
  • everything inside of core/src/utils.rs
  • everything inside of core/src/task/trigger.rs

@zenith urchin

  • everything under core/src/scheduler (Including its submodules)
  • everything inside of core/src/scheduler.rs
  • everything under core/src/task/hooks.rs
  • every module inside of core/src/lib.rs (including the prelude one)

@jolly frost

  • everything under core/src/task/frames (including the TaskHookEvents)
  • everything inside of core/src/task/frames.rs
  • everything inside of core/src/errors.rs
  • everything inside of core/src/task.rs

Good luck, il be "watching from afar" for how the API docs result, if you have any questions, let know. Keep in mind, read the API docs guidelines under content/docs/contributing/api_docs_guidelines.mdx

light pierBOT
fervent lark
# fervent lark I've published some new changes, <@1119720414021685402> has finished some of the...

the ideal deadline is about 1-2 weeks which you should strive for, this way we can focus on other things as well.

For you @jolly frost as we said, focus in parallel for optimization (though mainly focus for now on API docs)
For you @zenith urchin, we can try to make the proc-macros toggether, its not a promise, we will see how much you can truly take. Prioritize API docs.

Ideally, you should dedicate the effort as otherwise il have to split the effort to other contributors (plus me, and my goal is to watch things from afar for how effective the API doc guidelines apply whilist doing other things as well in parallel while you guys write the docs)

fervent lark
#

imma restructure the issues in gh

#

i've closed all issues and will make better templates

formal sedge
# fervent lark i've closed all issues and will make better templates 
#[async_trait]
impl ResolvableFrameDependency for FlagDependency {
    async fn resolve(&self) {
        self.0.store(true, Ordering::Relaxed);
    }
}

#[async_trait]
impl UnresolvableFrameDependency for FlagDependency {
    async fn unresolve(&self) {
        self.0.store(true, Ordering::Relaxed);
    }
}

Im not sure, but dont unresolve() should pass false ? Seems strange since resolve and unresolve actually do the same thing

formal sedge
#

Btw, is there any dependency/ documentation ? I struggle to understand it

#

Like how it is used in workflows

formal sedge
#

In task.rs. Is it worth it ? It fixed my test by changing it

image.png
# 
#[async_trait]
impl TaskHook<OnTaskEnd> for TaskDependencyTracker {
    async fn on_event(
        &self,
        ctx: &TaskHookContext,
        payload: &<OnTaskEnd as TaskHookEvent>::Payload<'_>,
    ) {
        let should_increment = self.resolve_behavior.should_count(ctx, payload).await;

        if !should_increment {
            return;
        }

        self.run_count.fetch_add(1, Ordering::Relaxed);
    }
}
#

Also, if !should_increment was opposite (if should_increment). I guess that we should return when we shouldnt increment

fervent lark
fervent lark
fervent lark
fervent lark
# fervent lark I've published some new changes, <@1119720414021685402> has finished some of the...

@placid drum @jolly frost @arctic trench @zenith urchin Really sorry for the mention yet again. I forgot to mention, for API docs, it is advised nott o dump it all but rather do it incrementally and showcase them, this way we can fix mistakes early and reduce repetitive errors whereas they would be made throughout the API docs

The idea is simple, document one to two things, showcase how its documented (and ofc what is documented) and after that il critique it and give feedback if it has mistakes or if its all around good. If its good then you can repeat this cycle for new docs

formal sedge
fervent lark
#

wait

fervent lark
#

this is a sync function so i have to inject async somehow

#

i do wwant to ideally inject before making the dependency

#

this way i can ensure it works

#

ye ik .attach_hook doesn't take much time but still

#

i was in a hurry so i couldn't respond early

fervent lark
#

btw, before you say to make ChronoGrapher support sync, there is already a talk about it

#

for MVP stage, its something il leave out, il consider this for the future

formal sedge
fervent lark
formal sedge
fervent lark
#

sure

fervent lark
#

more of a nitpick

#

but i would break it apart to smaller functions

#

like test_or_logical_combination, test_and_logical_combination, test_not_logical_combination, test_xor_logical_combination

#

but this is more of a nitpick than anything

#

for unit tests, its best to break things as much as possible, so each thing does its own stuff

formal sedge
#

Im going to do that

fervent lark
#

i reccomend making a module for dependency

#

and have submodules

#

for each logical dependency, dynamic dependency and so on

#

this way we can group things

formal sedge
#

A module is a folder with a mod.rs file in it right😅?

fervent lark
#

or

#

as you mentioned

# 
my_module
  - a.rs
  - b.rs
  - c.rs
  - mod.rs
fervent lark
formal sedge
fervent lark
#

also also

#

idk honestly

//      REMOVED:
        tokio::task::spawn_blocking(move || async move {

//      ADDED:
        tokio::spawn(async move {
            config.task.attach_hook::<OnTaskEnd>(cloned_tracker).await;
        });

i do prefer the spawn_blocking

#

wait

#

nvm

#

nvm

fervent lark
#

keep it

#

though not sure why you did

        Some(self.cmp(other))

but meh...

#

good thing you fixed some bugs

formal sedge
#

It proposed a « more idiomatic » way

fervent lark
#

oh

#

ok then

#

if its a clippy warning im ok

formal sedge
fervent lark
#

lol

fervent lark
#

if you don't get the code, do ask me if there are any questions

formal sedge
#

@fervent lark for the task dependency, should i group the failures or separate ? (Failures and success)

fervent lark
#

failures and successes

#

group by "action" (what each unit test group is supposed to do)

formal sedge
#

It should be ready to be merged

#

All tests passed with no clippy warnings

fervent lark
#

okie dokie

#

imma check it

#

wtf?

# 
    assert!(
        !dep.is_enabled().await,
        "Dynamic dependency should be disabled by default"
    );
#

@formal sedge

#

oh

#

i did a mistake

#

il fix it

#

ye it was by default disabled when it wasn't really meant to

formal sedge
fervent lark
#

ye

#

it was meant to]

#

late night coding ig

#

also also

#

here you test fine

#[tokio::test]
async fn test_dynamic_dependency() {
    let dep = DynamicDependency::new(|| async { true });
    assert!(
        dep.is_resolved().await,
        "Dynamic dependency should resolve to true based on its future"
    );
    assert!(
        !dep.is_enabled().await,
        "Dynamic dependency should be disabled by default"
    );

    dep.enable().await;
    assert!(
        dep.is_enabled().await,
        "Dynamic dependency should be enabled after calling enable()"
    );

    dep.disable().await;
    assert!(
        !dep.is_enabled().await,
        "Dynamic dependency should be disabled after calling disable()"
    );

    let flag = Arc::new(AtomicBool::new(false));
    let flag_clone = flag.clone();

    let stateful_dep = DynamicDependency::new(move || {
        let f = flag_clone.clone();
        async move { f.load(Ordering::Relaxed) }
    });

    assert!(
        !stateful_dep.is_resolved().await,
        "Stateful dynamic dependency should initially resolve to false"
    );

    flag.store(true, Ordering::Relaxed);
    assert!(
        stateful_dep.is_resolved().await,
        "Stateful dynamic dependency should resolve to true after underlying state changes"
    );
}
#

but

#

here:

# 
#[tokio::test]
async fn test_flag_resolution() {
    let flag = Arc::new(AtomicBool::new(false));
    let dep = FlagDependency::new(flag.clone());

    assert!(
        !dep.is_resolved().await,
        "Dependency should be unresolved initially"
    );

    dep.resolve().await;
    assert!(
        dep.is_resolved().await,
        "Dependency should be resolved after calling resolve()"
    );

    dep.unresolve().await;
    assert!(
        !dep.is_resolved().await,
        "Dependency should be unresolved after calling unresolve()"
    );

    assert!(
        dep.is_enabled().await,
        "Dependency should be enabled by default"
    );
    dep.disable().await;
    assert!(
        !dep.is_enabled().await,
        "Dependency should be disabled after disable()"
    );
    dep.enable().await;
    assert!(
        dep.is_enabled().await,
        "Dependency should be enabled after enable()"
    );
}
#

two things

#

first it shouldn't be just Dependency

#

but Flag dependency for clarity reasons

#

and second

#

even if its just two boolean values technically

#

you should check the combinations:

  • enabled - resolved
  • disabled - resolved
  • enabled - unresolved
  • disabled - unresolved
#

and ofc initial stuff like you do

#

hmm wait

#

nvm

#

scrap it

#

ok now this bothers me actually

formal sedge
fervent lark
# 
    f2.store(true, Ordering::Relaxed);
    assert!(
        and_dep.is_resolved().await,
        "AND dependency should be resolved when both inputs are resolved"
    );

You finish here

#

but you never check the combination of d2 enabled and d1 disabled

#

you check for

#

d1 and d2 disabled
d1 enabled and d2 disabled
d1 and d2 enabled

#

for the and dependency

#

the OR dependency is worse

formal sedge
fervent lark
#

ye ik, but like still

formal sedge
#

Mb a FlagDependency

fervent lark
#

even if its just simple boolean logic which is always correct

formal sedge
fervent lark
#

i might change LogicalDependency in the future for example

fervent lark
formal sedge
fervent lark
#

what if i want to change the dependency implementation in the future

#

your test cases might run fine for d1-d2 resolved/unresolved

#

but

#

maybe

#

d2-d1 unresolved/resolved might not successfully run

formal sedge
fervent lark
#

hmm

formal sedge
#

Anyway im doing that

#

Itll be fast since its just copy paste

fervent lark
#

ye

fervent lark
#

even if you think its unnesscarry, include it

#

you assumed d1 | d2 = d2 | d1 to avoid testing edge cases, but there may be a specific edge case in a future implementation where d1 | d2 ≠ d2 | d1

#

this should be checked

#

even if its basic

formal sedge
#

D1 cant be inequal to itself

fervent lark
#

wait hold on

#

lets walk this through back

#

let's take this test right

# 
#[tokio::test]
async fn test_or_dependency() {
    let f1 = Arc::new(AtomicBool::new(false));
    let f2 = Arc::new(AtomicBool::new(false));

    let d1 = FlagDependency::new(f1.clone());
    let d2 = FlagDependency::new(f2.clone());

    let or_dep = LogicalDependency::or(d1, d2);
    assert!(
        !or_dep.is_resolved().await,
        "OR dependency should not be resolved initially"
    );

    f1.store(true, Ordering::Relaxed);
    assert!(
        or_dep.is_resolved().await,
        "OR dependency should be resolved when at least one input is resolved"
    );
}
#

so what you do is test if

  • or_dep is initially unresolved
  • then you toggle f1 which changes d1 to test if or_dep is now resolved
#

you skip the other tests because you assume a | b = b | a, so your tests:

  • false false
  • true false

You skip:

  • false true scenario

And you also assume that the implementation a | b = true if one of the values is true, and since they are true, you don't check true true scenario

#

but

#

while its simple boolean logic, the problem isn't that but its the future

formal sedge
#

Wait

#

Actually there is just true true that is not tested

fervent lark
#

ye

#

that

#

and the false true scenario

formal sedge
fervent lark
#

someone may modify the or

#

for any reason, perhaps "performance" (unreasonable ik, but bear with me)

#

and from your tests, all tests may pass

formal sedge
#

But like, in the background FlagDependency is An Arc atomic bool

#

Which is already tested in the language

#

So how can it be false

fervent lark
#

their implementation may have a problem there

#

and this would go undetected

#

for example

formal sedge
#

I dont understand

#

Since d1 and d2 are from the test so it doesnt matter of the actual implementation

fervent lark
#

like extensively

#

everything that could possibily happen

#

not just "use it"

formal sedge
#

Ok

#

Quick question true true (inOR) should be true ?

fervent lark
#

ye? i mean its true | true = true

formal sedge
#

So below i negate d2 to test true false ?

IMG_1850.jpg
fervent lark
#

true false you already checked it

formal sedge
#

Mb

#

I implemented it

#

Is there orher edge cases

fervent lark
#

not really

#

or hmm

#

give a sec

formal sedge
#

Mb

#

I forgot XOR

#

Does it behaves like OR ?

fervent lark
#

not too similar

formal sedge
fervent lark
#

yes

#

and false when false false

formal sedge
#

Finished

#

Any other things ?

fervent lark
formal sedge
fervent lark
#

give a sec

#

il do this part actually myself, since there will need to be some changes done to the dependencies

#

but

#

This again, annoys me, you check false, false, true, false, true, true but never false, true

#[tokio::test]
async fn test_and_dependency() {
    let f1 = Arc::new(AtomicBool::new(false));
    let f2 = Arc::new(AtomicBool::new(false));

    let d1 = FlagDependency::new(f1.clone());
    let d2 = FlagDependency::new(f2.clone());

    let and_dep = LogicalDependency::and(d1, d2);
    assert!(
        !and_dep.is_resolved().await,
        "AND dependency should not be resolved initially"
    );

    f1.store(true, Ordering::Relaxed);
    assert!(
        !and_dep.is_resolved().await,
        "AND dependency should remain unresolved when only one input is resolved"
    );

    f2.store(true, Ordering::Relaxed);
    assert!(
        and_dep.is_resolved().await,
        "AND dependency should be resolved when both inputs are resolved"
    );
}
#

i don't care if in boolean algebra true & false = false & true

#

this is dependent on the implementation, if the dev writes something wrong when they modify the AND implementation

#

it should test this edge case, and detect it failed

formal sedge
fervent lark
#

and you should always do this

#

never assume stuff will hold true in the implementation, the implementation is always a black box and you must ensure that black box works for any different input

#

even if like boolean algebra says so, assume the developer will make a mistake, you have to catch it no matter what

formal sedge
#

Pushed to the PR

#

Should be ready to be merged ig

formal sedge
#

But anyways

#

Its still better to have all possible edge cases

fervent lark
#

im talking about the values

formal sedge
#

D1 and d2 shares the same behavior

fervent lark
#

typo, should be f1

    f2.store(false, Ordering::Relaxed);
    assert!(
        !and_dep.is_resolved().await,
        "AND dependency should remain unresolved when only one input is resolved"
    );
#

in the AND

#

let me select a bit more stuff

formal sedge
#

Good catch

#

Pushed to the PR

fervent lark
#

imma check the rest and let yknow, logical seems good (minus the typo, but its a typo)

#

That is, ugly

# 
    let task = Box::leak(Box::new(Task::new(TaskScheduleImmediate, frame)));
#

never ever do such thing, this is an anti-pattern

#

code smell

#

why are you delaying?

# 
    tokio::time::sleep(std::time::Duration::from_millis(50)).await;
#

just to check if its initially resolved

formal sedge
formal sedge
fervent lark
formal sedge
#

IT IS NECCESARY

fervent lark
#

ye pls don't just pump out content blindly like this

formal sedge
#

For any reason, if i delete this duration the test blocks

fervent lark
#

hmm then you found a bug

formal sedge
fervent lark
#

when the others were like bam bam

formal sedge
fervent lark
#

in a couple of minutes and max 1 hour

fervent lark
formal sedge
#

Like why dont you make me a single review and all get fixed in one time

formal sedge
#

Then you tell me about this antipattern

fervent lark
#

ye look im checking things ok

#

i got other stuff irl to take care of

formal sedge
#

I just said bc you told me that it was too long

fervent lark
# 
    dep.disable().await;
    assert!(!dep.is_enabled().await);
    dep.enable().await;
    assert!(dep.is_enabled().await);

    dep.unresolve().await;
    assert!(!dep.is_resolved().await);
    dep.resolve().await;
    assert!(dep.is_resolved().await);

Also, you don't write any message for assert!, it improves debugging

#

and why you grouped task_dependency_test_failures_only.rs and task_dependency_test.rs, imo it should be just task_dependency.rs with the failures, i told you that

formal sedge
formal sedge
#

Just mind that it is just enablibg and disabling methods

fervent lark
#

my bad

formal sedge
#

dw

fervent lark
#

ye i confused you on this

#

i meant to like group things by "action" what each unit test group checks for

#

exactly like you did on the rest

#

dynamic_dependency_test.rs

#

``flag_resolution_test.rs`

#

and so on

formal sedge
#

Fixed all

#

Any other ?

fervent lark
#

don't think so

#

i gtg, il check back

formal sedge
#

Ok

fervent lark
# formal sedge Ok

hey, one thing i noticed is the PR is deleted, any issues / problems that happened?

formal sedge
#

Wth

#

I didnt do anything

formal sedge
fervent lark
#

ok merged

fervent lark
# formal sedge I didnt do anything

just know, since i want to set a bar on quality for contributors to meet, i definitely don't want late night coding, ye like few mistakes there, happens to me as well

#

but like there were kinda much on yours

#

not overly many of them but still

fervent lark
#

can you explain what exactly happened?

#

if you remove this delay

formal sedge
#

Give a sec i reproduce it

fervent lark
#

rn can't really run it

#

so do need some info on it

#

i assume some kind of deadlock is happening

#

thats one hypothesis

#

probably in dashmap somewhere

#

since i've seen notes there detailing this exact issue

formal sedge
# 
#[tokio::test]
async fn test_task_dependency() {
    let should_succeed = Arc::new(AtomicBool::new(true));
    let frame = SimpleTaskFrame {
        should_succeed: should_succeed.clone(),
    };

    let task = Box::leak(Box::new(Task::new(TaskScheduleImmediate, frame)));

    let dep: TaskDependency = TaskDependency::builder(task).build();

    // tokio::time::sleep(std::time::Duration::from_millis(50)).await;

    assert!(
        !dep.is_resolved().await,
        "Task dependency should not be resolved initially"
    );

    let result: Option<Box<dyn TaskError>> = None;
    task.emit_hook_event::<OnTaskEnd>(&result.as_ref().map(|x| x.as_ref()))
        .await;

    assert!(
        dep.is_resolved().await,
        "Task dependency should be resolved after task succeeds"
    );

    dep.disable().await;
    assert!(
        !dep.is_enabled().await,
        "Task dependency should be disabled when calling disable()"
    );

    dep.enable().await;
    assert!(
        dep.is_enabled().await,
        "Task dependency should be disabled when calling enable()"
    );

    dep.unresolve().await;
    assert!(
        !dep.is_resolved().await,
        "Task dependency should be unresolved when calling unresolve()"
    );

    dep.resolve().await;
    assert!(
        dep.is_resolved().await,
        "Task dependency should be resolved when calling resolve()"
    );
}
#

happens on the commented duration

#

wait

#

it works now

fervent lark
#

?

fervent lark
formal sedge
#

running 3 tests
test test_detach_hooks ... ok
test test_get_hook ... ok
test test_attach_and_trigger_hooks ... ok // this test wasnt running

#

I think that it wasnt related to my test

#

Did you committed something else ?

fervent lark
#

hm, from your fork, you were ahead of the commits

#

so you're pretty much up to date

formal sedge
#

strange

#

let me reconnect to my codespace

fervent lark
# 
thread 'dependency_test::task_dependency_test::test_task_dependency' (2641086) panicked at core/tests/dependency_test/task_dependency_test.rs:50:5:

Task dependency should be resolved after task succeeds
#

huh

#

you sure ran those tests?

formal sedge
fervent lark
#

ah ok

formal sedge
#

wait

#

in my codespace thepanic occurs

fervent lark
#

i see

formal sedge
#

but not on my computer

fervent lark
#

ye

#

the panic

#

same here

#

ik what exactly it is

formal sedge
#

im so fckin confused

#

when i uncomment the duration it works

#

Strange

#

thread 'dependency_test::task_dependency_test::test_task_dependency' (6993) panicked at core/tests/dependency_test/task_dependency_test.rs:50:5:
Task dependency should be resolved after task succeeds

fervent lark
#

race condition

formal sedge
#

but the duration is at Ln37

fervent lark
#

its a race condition

#

like 100%

fervent lark
formal sedge
fervent lark
#

when you spawn a tokio task, it doesn't execute immediately

#

so...

formal sedge
#

on the task

fervent lark
#

not really

#

its also a problem on mine

#

i tried changing it to what it was initially, still failed

fervent lark
#

and because the hook isn't attached

#

it doesn't listen to OnTaskEnd when it actually happens

#

and never increments

fervent lark
#

like absolute, from zero*-ish*

formal sedge
fervent lark
#

ye i hate to break it up to you

formal sedge
#

zero idk

fervent lark
#

but

#

its not really a good project especially for this beginner level to dive into

#

not to deter you from contributing

#

but

#

this project especially requires heavy thinking, like mid to senior thinking

#

the project will especially get complex with its SDKs, distributed systems, cloud infra support, integrations... etc.

#

And you will most likely suffer throughout, even if i try to assign you the most basic tasks, its not a welcoming enviorement, its just its nature, i sadly can't tweak it

#

and again no offense

formal sedge
#

understand

fervent lark
#

but i fear and i always monitor the quality of the project, at the end of the day i wanna ship a very good product, i get we all start from somewhere, but i don't think its the right place to be starting. I fear poor experience will lead to wasted time (just like now)

#

or at worst, eat away at quality slowly but surely

#

time is especially critical rn

#

as we got like a deadline we aim towards mid April

#

again sorry if this comes offensive or anything, but i can't word it out in any other way than this

#

do at least appreciate your effort, even if its "amatuer" (again no offenses), you did try

formal sedge
#

ok

fervent lark
# formal sedge ok

i would suggest:

  • Getting a better grasp of the general stuff in CS
  • Learn the details of each thing
  • Stick to one programming language (If you aren't switching like i did and all of us did back then)
  • Learn deeply the features of said programming language
  • Generalize these ideas
  • Make your own projects or contribute to others (matching your experience)
  • Rence and repeat the learning cycle
#

again i do want to state this

#

no offenses, no undermining, i get it, we all start from somewhere, i did, all of us did, its normal

#

it just the project's nature and the demands of it are hostile towards beginners

formal sedge
fervent lark
formal sedge
zenith urchin
#

@fervent lark I have created Draft PR for my documentation part. To be honest, I don't know what to add more in the Prelude section. Could you check out the Draft PR and give feedback? Also, I still cannot add Reviewers for some reason. I am doing PR from my Fork maybe it is because of that do you also have any idea about it?

fervent lark
#

weird for reviewers

#

wait what?

#

@zenith urchin

#

why you extended the API docs guidelines?

#

you seem to be confused with the guidebook and the API docs

#

api docs i refer to

# 
/// This documentation stuff to describe our little code
/// peace, and so on so fourth dummy description insert
fn abc(...) {...}
fervent lark
#

We're gonna have an issue:

impl<T1, T2> From<TaskDependencyConfig<T1, T2>> for TaskDependency
where
    T1: TaskFrame,
    T2: TaskTrigger,
{
    fn from(config: TaskDependencyConfig<T1, T2>) -> Self {
        let tracker = Arc::new(TaskDependencyTracker {
            run_count: Arc::new(AtomicU64::default()),
            minimum_runs: config.minimum_runs,
            resolve_behavior: config.resolve_behavior,
        });

        let cloned_tracker = tracker.clone();

        tokio::spawn(async move {
            config.task.attach_hook::<OnTaskEnd>(cloned_tracker).await;
        });

        Self {
            task_dependency_tracker: tracker.clone(),
            is_enabled: Arc::new(AtomicBool::new(true)),
        }
    }
}
#

this causes a race condition, since we spawn a task, tokio won't execute the task block immediately as it just executes other tasks. The time to execute this task block is unknown so by the time we allocate it, the user then creates the TaskDependency and well the TaskHook isn't attached

#

so if the Task has finished, it won't count causing it to never be resolved and ultimately getting an error in the test cases

#

this is gonna be challenging...

zenith urchin
zenith urchin
fervent lark
#

jess

zenith urchin
# zenith urchin So you allocate memory for TaskFrame, then we want to attach the TaskHook and Ta...

Can’t we do some kind of halting state if TaskTrigger is not attached don’t consume like thing and check it periodically if the Trigger is attached? I think we are not talking about moved data but directly consumed and freed memory space, so if we are not completely consuming it, we can make mut ref it and update the reference then scheduler can check the ref attribute to see if it has changed? I am probably oversimplifying it but just a suggestion

fervent lark
#

-# /j

#

i don't get it

#

what are you trying to say here?

fervent lark
#

btw for async and sync use, ig we might have to try

#

one idea is to abstract the runtime as its own component

#

in the Scheduler

#

since the Scheduler runs stuff, it will manage the runtime used as well

#

actually no lets not get distracted

#

though il do share the idea of how it could be achieved

#

we provide general methods the runtime supports

#

for supporting sync operations, we just have our own runtime that is for sync

#

the power of the runtime being a scheduler component is:

  • Each scheduler can have its own runtime instance
  • Each scheduler composite can be explicit on what runtimes it supports
#

the problem will be how to expose this to the Task side

#

ngl i kinda want to try

#

idk why

#

i am quite tired of working on some stuff, so il try this

fervent lark
#

but

#

we can define a SyncRuntime which "fakes" sync feel, using a thread pool and blocking stuff on async

fervent lark
#

fake sync will hurt performance, yes

#

but

#

the good thing about ChronoGrapher is it doesn't force just one scheduler, you can split to multiple schedulers as such different runtimes per scheduler

#

so you could have an async scheduler for async workload and a sync scheduler for any CPU blocking tasks

fervent lark
#

@jolly frost @placid drum @arctic trench Howz the progression going for API docs? I will have to pressurize a bit you guys, since slowly but surely we are approaching the deadline, we don't have as much time as it may seem

arctic trench
#

I'll only have time on the weekend, unfortunately.

arctic trench
#

Unfortunately, the previous weekend i was busy with work stuff

fervent lark
#

oh ok

fervent lark
#

in parallel, while you write API docs, critique the API doc guidelines

#

since you'l be reading those

arctic trench
#

ok

fervent lark
# arctic trench ok

the goal is to get it in mid April the core, with its SDKs (JS/TS and Python), the features and so on

#

whilist the deadline is set by me partially, i do want to present the project somewhere

placid drum
#

didnt got enough time yet

fervent lark
placid drum
fervent lark
#

ah good

placid drum
#

one question are we creating doc based on files ? like it should be based on features right ?

placid drum
fervent lark
#

you are supposed to document methods, structs... etc.

#

on that file

#

via doc strings

#

like so

# 
/// This documentation stuff to describe our little code
/// peace, and so on so fourth dummy description insert
fn abc(...) {...}
#

and everything i've mentioned for your part

#

it should be mentioned, if a function is private, don't really document it

#

@placid drum i'd reccomend starting with the utils

#

since i do want you to finish some docs, and share those so i can preview

#

even if they are drafts, i do want for feedback

placid drum
#

@fervent lark do you have anything as refrence ?

fervent lark
#

the API doc guidelines which are in docs/ folder

#

i reccomend starting nextjs via pnpm run dev

#

so you can preview in the website

#

then simply go here

placid drum
#

yea read till api documentation guideline

fervent lark
#

its not an example of the API docs being applied, more so a guideline set

#

for examples, well, i had past documentations but these were heavily outdated and didn't use the API guideline (as it wasn't made)

#

wait

#

one good example i think i have comes from @zenith urchin, though they modified the API guidelines chapter and not via doc strings

fervent lark
#

the only thing that is a smell is

#

"We will explain Schedule Types, TaskFrames and other ..."

placid drum
#

?

fervent lark
#

i realized now i forgot macros

#

Can you start the docs on TaskIdentifier

placid drum
#

aah i was confuse here tho

fervent lark
#

in the meantime il add headings for macros

placid drum
fervent lark
#

my bad

placid drum
#

?

fervent lark
#

no like it was my fault on that part

#

for not providing the headers

fervent lark
placid drum
fervent lark
#

ok fewwisSmol

#

published

#

a bit rushed, to not keep you waiting, lmk if there are some issues with the macros headings

placid drum
#

@fervent lark TaskIdentifier is some sort of id provider let say user wants to give x identifier to task which can be integer or anything if its satisfy TaskIdentifier trait user can use it correct ?

fervent lark
#

this could be a UUID, an integer, a string... Anything really which can be unique

placid drum
#

did i was wrong ?

fervent lark
#

?

#

wdym

fervent lark
#

small but do update

placid drum
placid drum
jolly frost
#

Yo @fervent lark

#

=>

fervent lark
#

ok il look into it

fervent lark
#

its unnesscarry

#

start immediately with the summary

#

wait...

#

what?

#

you apply headers like # Exports?

#

which are meant to be for modules, not traits

#

and i see almost no trait related headers

#

You're supposed to use these:

- ``# Required Method(s)`` Lists the various required methods and what they are meant to do, if any. **(REQUIRED)**
- ``# Required Subtrait(s)`` Lists the various subtraits required for this trait to be implemented, if any. **(REQUIRED)**
- ``# Supertrait(s)`` Lists the various supertraits which require this trait and extend on top of, if any. **(REQUIRED)**
- ``# Semantics`` Lists how the trait is meant to be implemented, used and what it does. **(REQUIRED)**
- ``# Implementation(s)`` Lists the various implementations of this trait and briefly describes them (not blanket implementations), if any. **(REQUIRED)**
- ``# Object Safety / Dynamic Dispatching`` Describes if the trait is object safe / dynamic dispatchable, if not why. **(REQUIRED)**
- ``# Blanket Implementation(s)`` Lists the blanket implementations of this trait, if any. **(RECOMMENDED)**
- ``# Optional Method(s)`` Lists the various optional methods, what they are meant to do and their default behavior (ideally follow it after required). **(RECOMMENDED)**
- ``# Generic(s)`` Describes the generics which the trait may have (don't include it if there aren't any). **(RECOMMENDED)**
#

and for the misc headers

# 
- ``# Example(s)`` Lists various examples on how to use this system in practice. **(REQUIRED)**
- ``# FAQ & Troubleshooting`` Common issues when using this thing as well as answering frequently asked questions **(RECOMMENDED)**
- ``# See Also`` Lists any relevant ``struct``, ``enum``, ``trait``, ``methods``, ``type-alias``, ``constants``, in addition to explaining how they relate briefly. That are either mentioned on the documentation or are recommended to be seen, for using this on methods, list as well the parent ``struct``, ``enum`` or ``trait``. **(REQUIRED)**
#

and you were supposed to do it like so

# 
/// Defines how a ``CollectionTaskFrame`` executes its taskframes, controlling
/// order, selection, concurrency, and early termination while preserving the
/// ``TaskFrame`` contract.
///
/// <...>
///
/// # Required Method(s)
/// ...Explain that execute method is required and its supposed to be, its arguments...
///
/// # Required Subtrait(s)
/// List the fact it requires ``Sized``, for ``Send + Sync`` its unnesscarry
///
/// # Supertrait(s)
/// (Nothing to add really there, omit this header)
///
/// # Semantics
/// ...
///
/// # Trait Implementation(s)
/// - [SequentialExecStrategy<P>](...) Executes the TaskFrames sequentially
/// - [ParallelExecStrategy<P>](...) Executes the TaskFrames in parallel
/// - [SelectionExecStrategy<S>](...) Selects one of the TaskFrames to execute
#[async_trait]
pub trait CollectionExecStrategy: Send + Sync + Sized + 'static {...}
#

weirdly it doesn't highlight but whatever

#

also do not use # if its not a header, use something like ## for better grouping in the header section

jolly frost
#

hm

#

okay

fervent lark
jolly frost
#

yeah

fervent lark
#

i'd say rewrite the docs

#

with this in mind

#

also use the # See Also headers, # Example(s) and so on

#

see also is good for anything related to this component (directly or not)

fervent lark
# jolly frost yeah

AI has written this but, this is sort of what i want:

/// Defines how a collection of task frames should be executed.
///
/// The strategy pattern allows different execution behaviors (sequential,
/// parallel, selection) to be plugged into a [`CollectionTaskFrame`] without
/// modifying the container itself.
///
/// # Decorating / Wrapping Behavior
/// A `CollectionExecStrategy` does not wrap child task frames directly. Instead,
/// it orchestrates their execution via the provided [`CollectionTaskFrameHandle`],
/// which gives access to the child frames and the execution context.
///
/// # Events
/// This trait does not fire events directly, but implementations typically
/// fire [`OnChildTaskFrameStart`] and [`OnChildTaskFrameEnd`] for each child
/// they execute.
///
/// # Execution Error(s)
/// - [`CollectionTaskError`] – Returned when a child task fails, capturing
///   which index failed and the underlying error
///
/// # Supported TaskHook(s)
/// - `OnChildTaskFrameStart` – Fired before each child task executes
/// - `OnChildTaskFrameEnd` – Fired after each child task completes
///
/// # Example
/// \`\`\`rust
/// #[async_trait]
/// impl CollectionExecStrategy for MyCustomStrategy {
///     async fn execute(
///         &self,
///         handle: CollectionTaskFrameHandle<'_, Self>,
///     ) -> Result<(), <CollectionTaskFrame<Self> as TaskFrame>::Error> {
///         for i in 0..handle.length() {
///             handle.execute(i).await?;
///         }
///         Ok(())
///     }
/// }
/// \`\`\`
#

(i've escaped the codeblocks via \)

#

this is imperfect, as it misses some headers but it does do a good job

jolly frost
#

hmm

#

okay

#

and one more question

fervent lark
fervent lark
placid drum
#

/// TaskIdentifier trait use for defining unique indentifier, any type that satisfies this trait bound. can serve as TaskId (e.g. [Uuid], integers, strings, etc.) allows user to use any type, user can use any identifier format that suits their case.
///
/// # Semantics
/// Implementors must provide a way to generate unique indentifier for task.
///
/// # Required Subtrait(s)
/// Debug, Clone, Eq, PartialEq, Hash, Send, Sync, 'static
///
/// # Required Method(s)
/// - generate - Produces a new unique identifier.
///
/// # Implementation(s)
/// - [Uuid] - Generates a random UUID v4 via [Uuid::new_v4].
///
/// # Object Safety / Dynamic Dispatching
/// This trait is not object-safe due to the Clone and Sized requirement from its supertraits.
///
/// # Example(s)
///
/// /// use uuid::Uuid; /// use crate::utils::TaskIdentifier; /// /// #[derive(Debug, Clone, PartialEq, Eq, Hash)] /// struct TaskId(Uuid); /// impl TaskIdentifier for TaskId { /// fn generate() -> Self { /// TaskId(Uuid::new_v4()) /// } /// } ///
/// # See Also
/// - [Uuid] - The default implementation, generating random v4 UUIDs.
/// - Task - The primary consumer of task identifiers.

#

?

fervent lark
#

sort of

jolly frost
fervent lark
#

do it

jolly frost
#

or in general maybe use [] instead of `` everywhere

#

not everywhere

fervent lark
#

the See also is super good

#

the Example(s) is amazing

#

the Object Safety / Dynamic Dispatching is also very good

placid drum
#

use crate::utils::TaskIdentifier; ??

#

like it will not gonna work

fervent lark
#

oh wait

#

ye ur right

#

i looked in the docs structure, not the example

fervent lark
#

since thats the name of the library in the Cargo.toml inside core

placid drum
#

make sense

#

finally lol

fervent lark
# placid drum use crate::utils::TaskIdentifier; ??

btw my only real gripe is with # Required Subtrait(s), you just list the modules, which isn't useful, explain why Hash, PartialEq and Eq. I would turn it more like to this:
/// # Required Subtrait(s)
/// TaskIdentifier requires the following subtraits in order to be implemented:
/// - Debug For displaying the ID
/// - Clone For fully cloning the ID
/// - PartialEq For comparing 2 IDs and check if they are equal
/// - Eq For ensuring the comparison applies in both directions
/// - Hash For producing a hash from the ID
///
/// Should also be mentioned the TaskIdentifier requires Send + Sync + 'static

fervent lark
#

like you kinda get what i mean

placid drum
#

ya

fervent lark
#

typically, you'd mainly focus on the trait itself, and not the subtraits, but since its so small, do showcase comparisons, cloning, equality and perhaps hashing

placid drum
#

hmm

fervent lark
#

thats most of my problems with the docs honestly,

placid drum
#

/// use uuid::Uuid;
/// use chronographer::utils::TaskIdentifier;
///
/// #[derive(Debug, Clone, PartialEq, Eq, Hash)]
/// struct TaskId(Uuid);
/// impl TaskIdentifier for TaskId {
/// fn generate() -> Self {
/// TaskId(Uuid::new_v4())
/// }
/// }
///
/// fn create_task<T: TaskIdentifier>() -> T {
/// T::generate()
/// }
///
/// let task_id = TaskId::generate();
///
/// let id: Uuid = create_task();
///
/// assert_ne!(id, task_id.0);

#

@fervent lark

fervent lark
#

that kinda seems unnesscarry?

placid drum
#

/// use uuid::Uuid;
/// use chronographer::utils::TaskIdentifier;
///
/// #[derive(Debug, Clone, PartialEq, Eq, Hash)]
/// struct TaskId(Uuid);
/// impl TaskIdentifier for TaskId {
/// fn generate() -> Self {
/// TaskId(Uuid::new_v4())
/// }
/// }
///
/// let task_id = TaskId::generate();
///
/// let id: Uuid = TaskIdentifier::generate();
///
/// assert_ne!(id, task_id.0);

fervent lark
#

i would suggest adding some comparisons

#

like a new id2

#

/// let id2: Uuid = TaskIdentifier::generate();

#

compare the 2 via eq

#

and hash both

#

and even display them ig

placid drum
#

crazy 🫡

fervent lark
#

look, finish the TaskIdentifier and ig move onto the others if i don't respond

placid drum
#

i am dying

placid drum
#

just 5 min i will finish and sleep lol

placid drum
#

he just wanna know what this trait do that it

#

well anyway i will raise pr you can add comments

fervent lark
fervent lark
#

looks AI-ish

#

nvm

fervent lark
#

/// [TaskIdentifier] trait use for defining unique indentifier for example UUID, integers, strings, and generally any kind of identifier format the user can use which suits their needs.
/// The identifier is used internally in "[Scheduler] Land" to hold references to tasks with a simple representation.
///
/// Specifically its used in the [SchedulerTaskStore] internally. Identifiers can be configured via the [SchedulerConfig] trait, different [Schedulers] may have different [TaskIdentifiers]
/// defined via their configuration.
///
/// # Semantics
/// Implementors must provide a way to generate unique indentifier for task via the generate method as listed in the trait itself.
///
/// # Required Subtrait(s)
/// [TaskIdentifier] requires the following subtraits in order to be implemented:
/// - Debug - For displaying the ID.
/// - Clone - For fully cloning the ID.
/// - PartialEq - For comparing 2 IDs and checking if they are equal.
/// - Eq - For ensuring the comparison applies in both directions.
/// - Hash - For producing a hash from the ID.
///
/// [TaskIdentifier] also requires Send + Sync + 'static.
///
/// # Required Method(s)
/// The [TaskIdentifier] trait requires developers to implement the generate method, which produces a new unique identifier
/// per call.
///
/// # Implementation(s)
/// The main implementor inside the core is [Uuid] which generates a random UUID v4 via using [Uuid::new_v4].
///
/// # Object Safety / Dynamic Dispatching
/// This trait is not object-safe due to the Clone and more specifically the Sized supertrait requirement.
///
/// # Example(s)
/// ```
/// use uuid::Uuid;
/// use chronographer::utils::TaskIdentifier;
///
/// #[derive(Debug, Clone, PartialEq, Eq, Hash)]
/// struct TaskId(Uuid);
/// impl TaskIdentifier for TaskId {
/// fn generate() -> Self {
/// TaskId(Uuid::new_v4())
/// }
/// }
///

#

/// let task_id1 = TaskId::generate();
/// let task_id2 = TaskId::generate();
///
/// // Unequal, as they are unique entries
/// assert_ne!(task_id1, task_id2);
///
/// fn calculate_hash<T: Hash>(t: &T) -> u64 {
/// let mut s = DefaultHasher::new();
//// t.hash(&mut s);
/// s.finish()
/// }
///
/// // They produce different hashes (since they are unique)
/// assert_ne!(calculate_hash(&task_id1), calculate_hash(&task_id2));
/// ```
/// # See Also
/// - [Uuid] - The default implementation, generating random v4 UUIDs.
/// - [SchedulerConfig] - One of configuration parameters over lots of others.
/// - [Scheduler] - The interface around the store using the identifier.
/// - [SchedulerTaskStore] - Manages linking identifiers to tasks.
/// - Task - The primary consumer of task identifiers.

#

here is how i would modify it

#

kinda

#

il modify it slightly more

fervent lark
#

it explains everything needed

#

you should strive to this

placid drum
fervent lark
placid drum
# fervent lark good to hear that

/// fn calculate_hash<T: Hash>(t: &T) -> u64 {
/// let mut s = DefaultHasher::new();
//// t.hash(&mut s); added 4 //// so its not working
/// s.finish()
/// }
///
/// // They produce diff

#

//// t.hash(&mut s); added 4 //// so its not working
@fervent lark

#

see guys bye

fervent lark
fervent lark
#

Perfect

# 
/// [`TaskIdentifier`] trait used for defining unique identifiers. For example UUID, integers, strings,
/// and generally any kind of identifier format the user can use which suits their needs.
///
/// The identifier is used internally in "[`Scheduler`] Land", via a hashmap, it associates an identifier
/// with an owned [Task](crate::task::Task) instance. This identifier is unique, cloneable and comparable.
///
/// Specifically it is used in the [`SchedulerTaskStore`](crate::scheduler::task_store::SchedulerTaskStore) internally.
/// Identifiers can be configured via the [`SchedulerConfig`](crate::scheduler::SchedulerConfig) trait.
/// Different [`Schedulers`](crate::scheduler::Scheduler) may have different [`TaskIdentifiers`](TaskIdentifier)
/// defined via their configuration.
///
/// > **Note:** It should be mentioned, identifiers are held internally in some cases in the "Task Land",
/// but never exposed directly (as to prevent leaking abstractions)
///
/// # Semantics
/// Implementors must provide a way to generate unique identifier for task via the
/// [`generate`](TaskIdentifier::generate) method as listed in the trait itself.
///
/// # Required Subtrait(s)
/// [`TaskIdentifier`] requires the following subtraits in order to be implemented:
/// - ``Debug`` - For displaying the ID.
/// - ``Clone`` - For fully cloning the ID.
/// - ``PartialEq`` - For comparing 2 IDs and checking if they are equal.
/// - ``Eq`` - For ensuring the comparison applies in both directions.
/// - ``Hash`` - For producing a hash from the ID.
///
/// [`TaskIdentifier`] also requires `Send` + `Sync` + `'static`.
///
/// # Required Method(s)
/// The [`TaskIdentifier`] trait requires developers to implement the [`generate`](TaskIdentifier::generate)
/// method, which produces a new unique identifier per call.
///
/// # Implementation(s)
/// The main implementor inside the core is [`Uuid`] which generates a random UUID v4 via using
/// internally [`Uuid::new_v4`].
///
# 
/// # Object Safety / Dynamic Dispatching
/// This trait is **NOT** object-safe due to `Clone` and more specifically the `Sized` supertrait requirement.
///
/// # Example(s)
/// `\``
/// use uuid::Uuid;
/// use chronographer::utils::TaskIdentifier;
/// 
/// #[derive(Debug, Clone, PartialEq, Eq, Hash)]
/// struct TaskId(Uuid);
/// impl TaskIdentifier for TaskId {
///     fn generate() -> Self {
///         TaskId(Uuid::new_v4())
///     }
/// }
/// 
/// let task_id1 = TaskId::generate();
/// let task_id2 = TaskId::generate();
/// 
/// // Unequal, as they are unique entries
/// assert_ne!(task_id1, task_id2);
///
/// fn calculate_hash<T: Hash>(t: &T) -> u64 {
///     let mut s = DefaultHasher::new();
///     t.hash(&mut s);
///     s.finish()
/// }
///
/// // They produce different hashes (since they are unique)
/// assert_ne!(calculate_hash(&task_id1), calculate_hash(&task_id2));
/// `\``
/// In the example, ``TaskId`` is our identifier format (with a couple of traits implemented on top),
/// for demonstration purposes we used ``Uuid`` but as mentioned, any form of data can be used.
///
/// We implement the ``TaskIdentifier`` trait with its ``generate`` method, then we simply generate two
/// instances with that method, more specifically ``task_id1`` and ``task_id2``.
///
/// We compare the two and see they aren't equal (since they are unique), we take the hash of the two
/// and also see they are non-equal (again confirms the fact they are different).
///
/// # See Also
/// - [`Uuid`] - The default implementation, generating random v4 UUIDs.
/// - [SchedulerConfig](crate::scheduler::SchedulerConfig) - One of configuration parameters over lots of others.
/// - [Scheduler](crate::scheduler::Scheduler) - The interface around the store using the identifier.
/// - [SchedulerTaskStore](crate::scheduler::task_store::SchedulerTaskStore) - Manages linking identifiers to tasks.
/// - [`Task`](crate::task::Task) - The object which the task identifier associates.
#

this is what we call a good example

#

^ this example should be followed

light pierBOT
fervent lark
#

I am noticing some issues with the API doc guidelines, for this reason il slightly rework them

#

to explain things better

fervent lark
#

you big fuck...

image.png
#

-# i mean AI

placid drum
fervent lark
#

i will probably need some rest from the project

#

its been a bit exhausting

#

that doesn't mean you guys the contributors won't be still making

#

i will critique, merge and do some stuff as per usual

#

on average i've been pumping out 35.15 commits per week

#

throughout the project's lifetime

#

like i kinda deserve a break lol

placid drum
#

go some beach

fervent lark
#

ye ig lol

fervent lark
#

let me guess

#

no updates from yall

placid drum
#

so i am working on task frame lol

so basically task frame is excution logic, you can wrap mutiple frame

#

so you want to add anything ?

fervent lark
#

yes

#

wait

#

wait wait

#

you're assigned to the TaskFrameBuilder

#

not like the entire TaskFrame stuff, only the builder

placid drum
#

yea just reading task fream to get some idea

fervent lark
#

ah ok

#

ye so in summary

#

TaskFrame is what you want to execute, your "business" (more like execution, though its just to get you familar) logic is a TaskFrame

#

its almost like a typical function

#

but you can stack on top of your execution logic other TaskFrames to modify the behaiviour of how this group acts

#

want to add retries to your execution logic? Use RetriableTaskFrame,
want to add timeouts? Use TimeoutTaskFrame
want to add a fallback? Use FallbackTaskFrame along with another TaskFrame for error handling
want to combine it? You got 4 ways to do this, each producing varied results

placid drum
#

and builder helps to build task frame providing multiple methods/options

fervent lark
#

TimeoutTaskFrame -> FallbackTaskFrame -> RetriableTaskFrame -> LOGIC
FallbackTaskFrame -> TimeoutTaskFrame -> RetriableTaskFrame -> LOGIC
RetriableTaskFrame -> FallbackTaskFrame -> TimeoutTaskFrame -> LOGIC
FallbackTaskFrame -> RetriableTaskFrame -> TimeoutTaskFrame -> LOGIC
...

#

all these are different

#

on the first the timeout acts upon the fallback and the falback acts upon the retriable

#

whereas on the second

#

the fallback acts upon the timeout and the timeout acts upon the retriable

fervent lark
#

its just an ergonomic sugar

placid drum
#

makes sense l'll ask help if need thanks

fervent lark
#

np

placid drum
#

/// the wrapping order matters: methods called later produce the outermost layer. for example, calling .with_retry(...) then .with_timeout(...) produce
/// a TimeoutTaskFrame<RetriableTaskFrame<T>>,

#

@fervent lark

#

correct ?

fervent lark
#

i assume you clarify this in the struct right?

placid drum
#

yes

fervent lark
#

okie

fervent lark
#

and for pipelines, well i would better rename it to workflows

#

Also

# 
/// ... It wraps a frame and provides chainning methods to behavioral wrappers (retry, timeout, fallback, condition, dependency, etc.) around it. Each chaining method consumes the current builder and returns a new [`TaskFrameBuilder`]

I would better do it as:

/// ... It wraps a given [`TaskFrame`] and provides builder-style methods which add on top of this taskframe behavioral wrappers (such as retry, timeout, fallback, condition, dependency, etc.). Each method modifies the TaskFrame and returns the builder to allow for chaining.
#

this is verbose:

/// Each method wraps the current frame inside a new outer frame forming a nested execution pipeline, when the final frame is executed each layer runs its own logic.
#

better remove it as you already clarified it

#

I would use a contrast example on:

/// the wrapping order matters: methods called **later** produce the **outermost** layer. for example, calling `.with_retry(...)` then `.with_timeout(...)` produce 
/// a `TimeoutTaskFrame<RetriableTaskFrame<T>>`,
#

like the reverse

#

then elabrorate the difference a bit

#

avoid bullet lists for just one item

# 
/// - `T: TaskFrame` - The ineer frame type held by the builder at start. with every chaining call encoding the full nesting structure at the type level. for example 
///  after calling `.with_retry(...)`, the builder becomes `TaskFrameBuilder<RetriableTaskFrame<T>>`.
#

and typo ineer should be inner

fervent lark
#

You used chatgpt for this one?

placid drum
#

mrthods

fervent lark
#

but yet again i do see typos

#

which does indicate you actually did work on

#

i would replace all — with - btw

# 
/// - [`with_retry`](TaskFrameBuilder::with_retry) — Wraps with [`RetriableTaskFrame`] using a constant backoff delay between retries.

I would better describe it as:

/// - [`with_retry`](TaskFrameBuilder::with_retry) — Wraps with [`RetriableTaskFrame`] with a constant delay between retries.
placid drum
#

i am saying methods and exampls i use gpt

#

like some is repetiative

fervent lark
#

the rest of the bullet points are fine

fervent lark
#

im completely fine with that

placid drum
fervent lark
#

for structure

fervent lark
#

hmm i kinda take it back

#

the first stuff before the example, excluding the issues i pointed and only looking at the structure are all perfect

#

Now structurally this is where it messes up really bad on the Example

#

and minorly on See Also

#

but most definitely on Example

#

first syntax error:

/// let backup_frame = TaskFrame::
#

second, you can definitely afford to implement my_frame and backup_frame, even if basic, do at least a println! statement

#

then for the builder, i would heavily suggest you add comments, turning it into:

/// const DELAY_PER_RETRY = Duration::from_secs(1);
///
/// let composed = TaskFrameBuilder::new(my_frame)
///     .with_retry(NonZeroU32::new(3).unwrap(), DELAY_PER_RETRY) // Failure? Retry 3 times with 1s delay
///     .with_timeout(Duration::from_secs(30)) // Exceeded 30 seconds, terminate and error out with timeout?
///     .with_fallback(backup_frame) // Received a timeout or another error? Run "backup_frame"
///     .build();
#

this way the user sees how the workflow is being built from the ground up

#

i would suggest removing // `composed` is now a FallbackTaskFrame<TimeoutTaskFrame<RetriableTaskFrame<MyFrame>>, BackupFrame>

#

and instead

#

do it in my next suggestion, which is explain the example

#

in a couple of sentences

#

ye the code is pretty self-explanotory ik, but do explain it still

#

once you finish, then include

With the workflow created, `composed` is now the type:
> ``FallbackTaskFrame<TimeoutTaskFrame<RetriableTaskFrame<MyFrame>>, BackupFrame>`` 

all from this builder, without the complexity of manually creating this type
placid drum
fervent lark
#

for # See Also, again remove — in favor of -. But definitely mention the methods, i would suggest putting them bottom last

fervent lark
placid drum
fervent lark
#

idk if its a good practice to do it

#

ig don't include the methods again

placid drum
#

okay

fervent lark
#

everything i expect is there

placid drum
#

thanks

fervent lark
#

np

placid drum
# fervent lark I would use a contrast example on: ```rust /// the wrapping order matters: metho... 

/// For example:
/// // here my_frame is type taskFrame  
/// TaskFrameBuilder::new(my_frame).with_retry(...).with_timeout(...)
///
/// Produces:
///
///     TimeoutTaskFrame<RetriableTaskFrame<T>>
///
/// Because `with_retry` wraps `T` first, then `with_timeout` wraps the result.
///
/// In contrast:
///
///     TaskFrameBuilder::new(my_frame).with_timeout(...).with_retry(...)
///
/// Produces:
///
///     RetriableTaskFrame<TimeoutTaskFrame<T>>
///
/// Here, `with_timeout` wraps `T` first, and `with_retry` becomes the outer layer.
///
/// Think of it like function composition:
///
///     outer(inner(T))
///
/// The last call is always the outermost wrapper.
#

should i use this ?

fervent lark
#

hm

#

wait

placid drum
#

so shoule i my frame as well ?

#

add*

fervent lark
# 
/// For example ``TaskFrameBuilder::new(my_frame).with_retry(...).with_timeout(...)`` where ``my_frame`` is 
/// your [`TaskFrame`] (lets call its type "MyFrame") produces as a type:
///
/// > ``TimeoutTaskFrame<RetriableTaskFrame<MyFrame>>``
///
/// Because `with_retry` wraps ``MyFrame`` first, then `with_timeout` wraps the result, in contrast using 
/// ``TaskFrameBuilder::new(my_frame).with_timeout(...).with_retry(...)`` produces:
///
/// > ``RetriableTaskFrame<TimeoutTaskFrame<T>>``
///
/// Here, `with_timeout` wraps `T` first, and `with_retry` becomes the outer layer.
/// Think of it like function composition where ``outer(inner(T))``.
///
/// The last call is always the outermost wrapper.
#

this is what i would change it to

placid drum
# 

use std::num::NonZeroU32;
use std::time::Duration;
use async_trait::async_trait;
use chronographer::task::{TaskFrameBuilder, TaskFrame, TaskFrameContext};

struct MyFrame;

#[async_trait]
impl TaskFrame for MyFrame {
    type Error = String;
    async fn execute(&self, _ctx: &TaskFrameContext) -> Result<(), Self::Error> {
        println!("Executing primary task logic!");
        Ok(())
    }
}

struct BackupFrame;

#[async_trait]
impl TaskFrame for BackupFrame {
    type Error = String;
    async fn execute(&self, _ctx: &TaskFrameContext) -> Result<(), Self::Error> {
        println!("Executing backup logic!");
        Ok(())
    }
}

const DELAY_PER_RETRY: Duration = Duration::from_secs(1);

let composed = TaskFrameBuilder::new(MyFrame)
    .with_retry(NonZeroU32::new(3).unwrap(), DELAY_PER_RETRY) // Failure? Retry 3 times with 1s delay
    .with_timeout(Duration::from_secs(30)) // Exceeded 30 seconds, terminate and error out with timeout?
    .with_fallback(BackupFrame) // Received a timeout or another error? Run "backup_frame"
    .build();

// With the workflow created, `composed` is now the type:
// > ``FallbackTaskFrame<TimeoutTaskFrame<RetriableTaskFrame<MyFrame>>, BackupFrame>``

// all from this builder, without the complexity of manually creating this type