#ChronoGrapher - One Unified Scheduler, Unlimited Power
1 messages Β· Page 2 of 1
also for the graph i do know why you don't like it
on darkmode the yellow like turd than like purely yellow xd
I forgot to explain why
But basically, when I first saw that, I couldn't understand what it was showing; it was difficult to get it at first sight.
ah
I'm having some classes about data science and one of the topics is exacly about the graph choices when you want show a data
Graphs is manly desing to express a lot of information with minimum or none explanation
yup
Graph titles is a great help like
Productivity X Time
Subtitles
Etc
Labels, legends
mhm
i think its a bit better right?
Much better
@fervent lark which rust version we are using
While I was verifying the code, some errors appeared.
latest im pretty sure
ah
oopsie
ye i forgot to merge upload chages

they are u[
up*
you can update the project
ig though i would advise you seeing more the docs rn, leave the rust side for a bit
here is something interesting
idk if its me
but isn't the bar for the "apache airlfow" a different color than "celery"
yet for some reason
the apache airflow bar is brighter than the celery one
the other colors trick my eyes into seeing the apache airflow brighter than it actually is
if i extract them and put them somewhere else, the illusion breaks
wait
finished
though it annoys me it ain't centered
nvm found the issue
my dumbass notices issues and proceeds to delete and reupload
and looks slightly better in dark mode
at least its more wood than just shit
btw uh you used AI in the report?
just out of curiousity
Just in one part
On the part that talks about the alt tag missing in the graph image
And of course, I used AI to verify if my reasoning was justified.
Now that I remember, the part where I mention the phrase between parentheses, I questioned with my perspective, and the IA rewrote it to a more robust phrase.
In my opinion, that's the correct use of AI.
In the end, IA is just a very intelligent text autocomplete.
ye slightly noticed it
true
just know i don't want AI do the piloting
ideally you shouldn't let it critique
since it inserts some bias
which you may follow
Did you have problems in the past because some contributors used AI wrong?
Based on the experience of other companies
honestly haven't worked in the industry, a bit of a shocker ik
but im just observing things from afar
and rationalize both on what stuff works and doesn't, while also seeing the results
Just for a test, add a very light text shadow to the colored text, maybe it can improve the text visibility.
sadly i don't have this kind of control
im gonna restructure massively how errors are handled
since rn its a mess
anyhow doesn't work
this includes reworking some chapters (mostly in the workflows section)
rn its painful to do error handling
which is something i hate
il have to define my own error trait as it seems
it would be clever to sort of merge anyhow with thiserror into my own solution
they will be supported
its just that im trying to think for every case
compatibility, ease of use, ergonomical error handling
the idea is a sort of json schema
i got the structure of the error
pub enum ChronoGrapherErrorType {
Unknown,
Custom(String),
TaskFrameFailure,
TaskScheduleFailure,
SchedulerStrategyFailure,
}
pub enum ChronoGrapherErrorSource {
Unknown,
ErrorSTD(Arc<dyn std::error::Error + Send + Sync + 'static>),
ChronoGrapherCore(Box<ChronoGrapherError>)
}
pub struct ChronoGrapherError {
message: String,
metadata: BTreeMap<String, Arc<dyn Any + Send + Sync>>,
backtrace: std::backtrace::Backtrace,
kind: ChronoGrapherErrorType,
source: ChronoGrapherErrorSource
}
impl ChronoGrapherError {
pub fn message(&self) -> &str {
self.message.as_str()
}
pub fn backtrace(&self) -> &std::backtrace::Backtrace {
&self.backtrace
}
pub fn kind(&self) -> &ChronoGrapherErrorType {
&self.kind
}
pub fn get_dyn(&self, key: &str) -> Option<Arc<dyn Any + Send + Sync>> {
Some(
self.metadata.get(key)?
.clone()
)
}
pub fn get<T: Send + Sync + 'static>(&self, key: &str) -> Option<Arc<T>> {
Some(
self.get_dyn(key)?
.downcast::<T>()
.ok()?
)
}
}
impl<T: std::error::Error + Send + Sync + 'static> From<T> for ChronoGrapherError {
fn from(value: T) -> Self {
Self {
message: value.to_string(),
metadata: BTreeMap::new(),
backtrace: std::backtrace::Backtrace::capture(),
kind: ChronoGrapherErrorType::Unknown,
source: ChronoGrapherErrorSource::ErrorSTD(Arc::new(value))
}
}
}
something like so
hmmm...
This is gonna be a hard problem
its still difficult
@arctic trench you have any idea on how to solve this?
if i had a Result<AnyOK, ChronoGrapherError>
where AnyOK is just my value for ok
i would have to do neted match and if statements
error handling is something we have to particularily get right
ye macros exist, but this doesn't seem like a macros problems
so far this error structure helps with gathering info about the error
hummm
@fervent lark Man, I suggest we make something simple at the start.
And try to build, for example, a simple web server using Chronographer and Rocket/Axios, of course not right now.
This way, we can gather some βrealβ scenarios using tasks and a scheduler.
I recommend it because premature abstraction is a common mistake in software engineering; we didn't gather some real examples of usage.
One more thing, You don't think use trait instead of enum for errors is better?
To respect the O of SOLID principle
well, i could use the error trait
though we have some problems per say
the error handling is something we can't just leave out
its tied to real world scenarios, yes ik about premature abstraction, but this is something we can't ignore, i've seen the pain even when writing the guidebook that its not good currently
i do want to try and make real-world stuff with ChronoGrapher, though error handling, the shared data API, the macros have to be finished and stabilized
then we can get to
also urgent, @jolly frost and @arctic trench since the API docs are up, should we rewrite the docs?
is it a good time to?
well not rn, in parallel we will handle the error handling problem
Let's firts finalizy one feature per time before rewrite the docs
true
let's focus then on error hadling
dumb approach, while being assisted in some of the architectural thinking aspect, deepseek reccomended an approach where errors have multiple contexts, this would have to do more closely with TaskFrames more than any other system, when a TaskFrame fails, the error bubbles up right? One idea is for every parent TaskFrame to provide their own context of what happened, forming a context tree
This context tree can be inspected, when displayed it will display the entire tree, which helps a bit on debugging side, but on the error handling side you could downcast errors or ig match contexts to deduce what happened? (Im thinking about it still)
How would this approach solve the issue?
its really a puzzling problem
What is the need of the task tree?
context tree*
well
it contains information
on how the error happened
how it bubbled up
like say FallbackTaskFrame would bubble the error up to RetriableTaskFrame with its own context that both the primary & fallback failed
then the RetriableTaskFrame would continously try (suppose now its the final retry), the error still persists so it throws on top its own context that says "the error still persists even with N retries..."
and so on
i don't think im thinking about the problem clearly tbh
Yeah, good idea, smth similar to my idea with error history
But won't this affect a bit performance?
yeah
tbh ditch the idea
hm
it's actually good
but rn it's kinda big
so we need to remove some stuff
mhm
to make it more lightweight
well
we need to allocate data only if smth returns an error
Good news
Il be completely free around 2 weeks
this means we have to stay focused especially those 2 weeks
which means @arctic trench @jolly frost, il be able to carry more some of the weights you had
hm might join https://itch.io/jam/brackeys-15, its been a while since i've entered a game jam
ok, talking about it, next week during some days i will be free too, just until Tuesday, but i will be free
ok
hmm we're gonna have some issues
the TaskHook events
these are fixable though, by just appending a generic for the error shape
i think...
though not sure how it would feel ergonomically speaking
ig they can execute different side-effects per error
impl<E: Error + Send + Sync + 'static> TaskHook<OnFallbackEvent<E>> for MyTaskHook {
async fn on_event(
&self,
ctx: &TaskHookContext,
payload: &<OnFallbackEvent<E> as TaskHookEvent>::Payload,
) {
// ...
}
}
this is for the case they want to trigger the same code for any error
Ok
actually meh
btw for the critics you managed (not trying to force you, just want to get an idea of progression)
Hmmm, for error handling i do fear things will get complicated
the idea is each TaskFrame appends its own newtype struct, its just a marker wrapping the error (for the most part)
Retriable, Fallback, Threshold and Delay do this wrapping
these newtypes will allow you to get information about the error and how the workflow behaved
now for the error matching side, suppose we have the error type DelayTaskFrameError<FallbackTaskFrameError<RetriableTaskFrameError<T>>> as a result where T is an enum let's just say
users could just do
match res {
Ok(()) => {...}
Err(err) => {
match err.0.0.0 {
T::Variant1 => {...}
T::Variant2 => {...}
T::Variant3 => {...}
// <...>
}
}
}
that .0.0.0 bothers me a lot actually
one idea to minimize could be to define a sort of proceedual macro prune! which does this unwrapping for us
or actually wait
but then
you have the implementation side
#[async_trait]
impl TaskFrame for MyTaskFrame {
type Error: Error + Send + Sync;
async fn execute(&self, ctx: &TaskContext<Self::Error>) -> Result<(), Self::Error> {
// ...
}
}
this is the simplest
its good
but
what if i want to define hook events which have as payload the error
i would have to make them generic
and idk honestly
also for this, nvm
i can implement Deref
match res {
Ok(()) => {...}
Err(err) => {
match &***err {
T::Variant1 => {...}
T::Variant2 => {...}
T::Variant3 => {...}
// <...>
}
}
}
much cleaner
π , ok, just to inform you, unfortunately, I still don't understand all this, but as I say, I'm studying Rust, and even if it's just some minutes a day, I will take those topics that you mentioned to study too.
But one thing: if the error handle implementations go this way, for every task error variant, will I need to create an "impl" to handle each type of error?
i mean if you want to specifically handle and extract specific information out of the error, then ye you need to match per variant
you can leave some if needed by just err => {...} or _ => {...} if you wanna fully ignore the error
humm, ok ok
also i propose a new change, il create an issue
What area has an architectural problem? Please describe. Three areas, specifically ParallelTaskFrame, SequentialTaskFrame and SelectTaskFrame with additionally the GroupExecBehaviour stuff. Why doe...
Hey everyone, does anybody know if the current master working properly or am I missing something?
I have tried to add chronographer = '0.0.1a' to new project. It threw an error about index not being present, so project is not defined under the cargo. Then I used chronographer = { git = "https://github.com/GitBrincie212/ChronoGrapher.git", branch = "master" } to add the project to the Cargo.toml. However, now, I cannot use Chronographer even though I followed the guidelines. I am talking about the Hello ChronoGrapherexample. Also, in the first Workflow example, we generate a random number, it would be nice to add cargo add rand as well I know it is a small thing and they can look at the errors and figure it out, but just for completeness.
So overall, is master branch a complete branch because I cannot run basics. I haven't gone deep into the codebase yet.
oh its still heavily WIP
it hasn't even been published
and
also
the macro syntax so far is conceptual
possible to implement but not ready
there is a lot to work in and let's just say it will be finished around April as it seems
maybe even later
Oh okay I was reading the guidelines and implementing in the meantime, but then I will just go through it and try to implement some Beginner ticket π . To coordinate better, I will inform you which ticket I am planning to take or I can take some ticket that is urgent as well.
ye do mention me
like ping
so i can respond fast
oki
for guidelines, how far you've read?
and in website form right?
not just reading the markdown
not far I have read until Problem space on workflow. Yup, run the pnpm run dev
good
but still could be valuable feedback, how you perceived the guidebook so far?
what you liked vs hated about it
Until now, I really liked the focusing on codesnippets with blur. It would have been nice to have clickable references for codebase. For example, you meantioned TaskFrames I know how to look into the TaskFrames and where it is, but would have been easier to just click it and navigate. I think overall it is a really nice website π
i see
ok good thing @arctic trench perspective seems to hold
for the clickable references for codebases wdym exactly?
btw you don't mention like the structure, how the concepts are presented, i get its minimal overview, but still would like to know for feedback
so i can improve upon
For example, navigating to this link: https://github.com/GitBrincie212/ChronoGrapher/blob/87e0e9422fd226d9e7e9c473a38c5d8173d7dd7f/core/src/task/frames.rs#L294
You can define lines for github pages, so it can be useful to do references for some people that want to have deeper understanding
i see
I can give you better feedback about this when I am done reading Guidelines. I also want to see consistency on the Guidelines etc. So will get back to you
ah sure sure
guidelines are more long on purpose, this part isn't meant to be narrative, like learning more so rules
whereas the chapters on the guidebook are ~1k words in length
okay keep this in mind while reading as well as giving feedback. Thanks
i feel like this is wrong yet the compiler doesn't complain at all
&TaskFrameContext<dyn ErasableTaskFrame>
when
pub struct TaskFrameContext<T: TaskFrame> {...}
#[async_trait]
pub trait TaskFrame: 'static + Send + Sync + Sized {
type Error: Error + Send + Sync;
async fn execute(&self, ctx: &TaskFrameContext<Self>) -> Result<(), Self::Error>;
}
trait ErasableTaskFrame: 'static + Send + Sync {
async fn erased_execute(&self, ctx: &TaskFrameContext<dyn ErasableTaskFrame>) -> Result<(), Box<dyn Error + Send + Sync>>;
}
impl<T: TaskFrame<Error: Into<T::Error>>> ErasableTaskFrame for T {
async fn erased_execute(&self, ctx: &TaskFrameContext<dyn ErasableTaskFrame>) -> Result<(), Box<dyn Error + Send + Sync>> {
self.execute(ctx).await.map_err(Into::into)
}
}
idk why, i just can't prove it
oh god
boi, was i right
im dying inside
slowly and surely i cut it to 70 errors
now 39
man im in pain
man i just wish i avoided Arc and used references
10 more errors to go
and now 6
man this is kinda complex
I feel like making errors typed was the wrong approach
Sure they seem to improve error handling
but Task is being erased when fed to the Scheduler
for the workflow parts it kinda works
a lot of changes will happen
in the scheduler side
in the Tasks
for error handling

and i should be careful about it, as to not leak any Scheduler stuff on to the Task side
@jolly frost
imma need you on this one
same with @arctic trench
both your attentions here



:>
i've made a significant change
pub trait SchedulerConfig: Sized + 'static {
type TaskIdentifier: TaskIdentifier;
type Error: Error + Send + Sync + 'static; // NEW
type SchedulerClock: SchedulerClock<Self>;
type SchedulerTaskStore: SchedulerTaskStore<Self>;
type SchedulerTaskDispatcher: SchedulerTaskDispatcher<Self>;
type SchedulerEngine: SchedulerEngine<Self>;
}
schedulers now can enforce a specific error they have in mind
to handle
this error is for Tasks
the problem is
it leads to cascading effects
what i mean is
i want the EngineNotifier to be aware of the Error type
pub struct EngineNotifier<C: SchedulerConfig> {
id: C::TaskIdentifier,
notify: tokio::sync::mpsc::Sender<(C::TaskIdentifier, Option<C::Error>)>,
}
so it can send it reliabily
previously i erased them under an Any
now bc it has to know
this leaks through SchedulerStrategy
forcing it to know the SchedulerConfig and thus forcing the Task to know the config
like so
#[async_trait]
pub trait ScheduleStrategy<C: SchedulerConfig>: 'static + Send + Sync {
async fn handle(&self, task: Arc<ErasedTask>, notifier: EngineNotifier<C>);
}
we could keep the erasing if we want and check but i feel like its dirty
how would you approach this, make the ScheduleStrategy without generic the config
while still being able to send the errors
we may have to rethink this
the Task architecture specifically
hm
why do you think its dirty
well, it moves the concern to runtime
the ScheduleStrategy can return any error
what do you propose
the problem is also tied to performance a bit, as downcasting will be extremely frequent
its a hot path the SchedulerStrategy
one idea
could be to shift the SchedulerStrategy to the scheduler side
as opposed to living inside the Task
and maybe even remove it entirely, in favor of the SchedulerTaskDispatcher doing this
Maybe better
maybe better what?
Your ideia, maybe beter
I was thinking in define a trait for engine and config
But, well, your idea is better
I will continue tomorrow, this week i was working on a vital fix where i currently work
This next 4 days i will create the critics for all docs
btw fun fact
the codebase was 8-9k LOCs right
with the docs removed
its 4.3k LOCs
kinda crazy to see how much docs there were
and how small it is
btw im removing the default CHRONOGRAPHER_SCHEDULER
the chronographer::main macro will handle this
#[chronographer::main]
async fn main(scheduler: DefaultAnyhowScheduler) {
// CODE
}
the DefaultAnyhowScheduler is just a type alias really
this code expands to:
#[tokio::main]
async fn main() {
scheduler: DefaultAnyhowScheduler = DefaultAnyhowScheduler::default();
// CODE
scheduler.start().await;
loop {} // or better yet the ctrl c signal
}
also new simpler diagram
there's a lot of stuff todo
im working on enhancing TaskHooks, the idea is their payload can now have a lifetime
when needed
this is to prevent DynArcError everywhere as an error
and instead use Box

lol
though how
imma commit all changes btw
@fervent lark I'm reading the documentation in the βInstallation & Theoryβ section.
The setting up of the project and the βHello Worldβ subsection are perfect, but starting at the βWhat Are Tasks?β subsection, we have some dangerous conceptual projects when we talk about the software development (in web development more specifically).
Although I will arrange the ideas and critiques, I believe that certain structures should be significantly altered if the information presented below is what I'm imagining.
mhm
i see
you probably mean the CHRONOGRAPHER_SCHEDULER warning
no
is about that idea:
"Tasks contain the Business Logic and Scheduling Logic, the former ..."
π€£
xd
no need to worry
ye ik lol
fell free to tell
ok once you finish the critique, do elabrorate heavily on this
i am interested to know
ig we need to make windows a bit larger
i don't think the size is the problem
it is actually
I agree, and one thing: look at my screen; the texts of the Bento Grid are too small. For me, who doesn't have visual problems, I can read it nicely, but what about the users? And even the ones who don't have it, will they have the patience to read it?
a bit smaller, could work out though
ye ig upping the size of the bento grid text won't hurt
it's already small
we have so much space
honestly i might add other frameworks for like js/ts
thats why im concerned
and rust
don't forget abt rust
though i do have an idea
since we're adding minimization
what if we just have some frameworks already minimized
and show the most prominent frameworks
and this should be animated
nicee
good idea
like a carousel
going top to bottom and wrapping around
π
yes
ye im like the powerhouse
still kinda bad
eh at least acknowledge the good side
you're helping out
like even if its small
you do help out
as opposed to some others who promised to help but never did anything, AHEM
-# no not you bagu if you are reading this, i haven't assigned you yet concrete task, so its justifiable

neither to you coder lmao
im in pain, like holy sh#t i've been doing so so much stuff
no no, i just that way man, I am very demanding of myself.
hell even some guy that wasn't planning to help, did something, SOMETHING even if tiny thing
like he wasn't planning to be a contributor, just one off fixes
but its SOMETHING, like some respect i have
i hate more than anything false promises of helping out and never doing it (and no not just low-quality but like actual production),
also
@jolly frost
i have some TODO areas
i want you to work on those
specifically the CollectionTaskFrame and inside the DependencyTaskFrame
once those are done and i tweak a bit the API guidelines, @zenith urchin we may need your help in the API documentation
for fully writing it
Okay
i feel like btw ChronoGrapher has slightly improved
in performance
imma test it
nvm seems the opposite happeneed weirdly enough
Later we will need to add code speed test to gh actions
ye
honestly one thing i really want is to get the speed to 1 million empty-ish tasks per second
and even faster
i want ChronoGrapher to be extremely fast
the more incetives i give companies to switch the better:
- Super small package size (34KB vs Megabytes)
- Top dog flexibility / extensibility, litterally unmatched
- Scales extremely well with you from a home laptop to a distributed service
- Integrates with most stuff
- Documented down to the minute detail with superrior quality
- Extremely fast, handling millions of tasks in one machine with lowest communication overhead
- Competitive durability with Temporal
if i give them all these reasons
i mean its irresistible to switch
the only reason you'd ever stick with other solutions is if you are already committed to one and can't afford to change, you're lazy to do so or the ecosystem and maturity is supperior (though ChronoGrapher will reach there at some time)
we need to cook a tool which can migrate all temporal code to chronographer
chronographer is so different than temporal
that a tool can't just migrate it
interesting...
hm
a lot of Arc drops
which is definitely a culprit that slows things down
here is the entire stacktrace
core::ptr::drop_in_place<alloc::sync::Arc<chronographer::scheduler::task_store::ephemeral::EphemeralSchedulerTaskStore<chronographer::scheduler::DefaultSchedulerConfig<alloc::sync::Arc<dyn core::error::Error+core::marker::Sync+core::marker::Send>>>>> [/.../rustlib/src/rust/library/core/src/ptr/mod.rs] <chronographer::scheduler::engine::default::DefaultSchedulerEngine as chronographer::scheduler::engine::SchedulerEngine<C>>::main::{{closure}} [/.../Chronographer/core/src/scheduler/engine/default.rs] <core::pin::Pin<P> as core::future::future::Future>::poll [/.../rustlib/src/rust/library/core/src/future/future.rs] chronographer::scheduler::Scheduler<C>::start::{{closure}}::{{closure}} [/.../Chronographer/core/src/scheduler.rs]
From what i gather
i have a hypothesis
nvm
or wait
i think this is responsible:
async fn retrieve(&self) -> Option<(Arc<ErasedTask<C::Error>>, SystemTime, C::TaskIdentifier)> {
let early_lock: EarlyMutexLock<'_, C> = self.earliest_sorted.lock().await;
let rev_item = early_lock.peek()?;
let task = self.tasks.get(&rev_item.1)?;
Some((task.value().clone(), rev_item.0, rev_item.1.clone()))
}
and more specifically the coordination between the engine and the store
which is
async {
loop {
if let Some((task, time, id)) = store.retrieve().await {
tokio::select! {
_ = clock.idle_to(time) => {
store.pop().await;
if !store.exists(&id).await { continue; }
let sender = EngineNotifier::new(
id,
scheduler_send.clone()
);
dispatcher.dispatch(task, sender).await;
continue;
}
_ = notifier.notified() => {
continue;
}
}
}
}
}
the hypothesis is because these tasks were small
they were executed very fast
causing a lot of Arc<T> instances to be dropped (decremented)
most likely
In the theory section of βInstallation and Theory,
β I found some very dangerous conceptual
ideas.
I love how you say "very dangerous" like its the end of the world
lol
sorry
no no its ok
Hmmm
If we are talking about a web project, using a schedule/task framework like Chronograph and
coupling the business logic with the project dependencies/tools is the worst thing possible
that they can do to their projects maintainability.
Starting with the project tests, unit/integration tests start to become difficult to perform
because of the strong coupling between them.
Furthermore, it is very common to reuse those logics in other places in the project, like, for
example, commands; use it in form classes, or even use another task logic.
I notice that in some of those situations, chronographs have some solutions, but if the base
idea of the following sections/implementations is like this:
explain this more
whats exactly the problem
in simple words
is it a guidebook problem or architectural problem
architectural problem
oh
you mean, why is bay couple?
like, couple the logic with the framework?
i didn't get your question
more like get to the problem
what is you want to explain
this uses AI 100%
as i notice
ye like i get grammar checking and stuff
but
no, kkkk, the project structure and the markdown is
some "impacts" feel arbirtary, no rhyme or reason just list a drawback that is unrealistic just to sound realistic which i do notice AI does occasionally
humm
i haave a extension in my browser
how does gramatical check
i use to avoid those errors, maybe that
i sort of get what you're saying
in summary you say ChronoGrapher forces business logic to be part of scheduling, even when its not supposed to be
right?
exactly
ok for that i disagree completely, and i can prove why
this more feels like a guidebook mistake more than an architectural one
but
ChronoGrapher provides them a way to schedule, now how users organize their code is something up to the user
in a web framework, you can break up the business logic to small chunks
and on the ChronoGrapher do those
like write
yeah
#[task(schedule = interval(2s))]
async fn MyMantainenceTask(ctx: &TaskContext) -> Result<(), MY_ERROR> {
check_db(...).await?;
health_check(..).await?;
cancel_orders(..).await?;
// ...
}
these are functions defined elsewhere
but
they are used in scheduling
its not like you will define the health checks there, or the cancel order code there
@arctic trench am i right? Could be mistaking or misunderstanding something
wdym?
Business logic there
like, the TaskFrames implementation who store the bussines logic?
ig... Hmmm
seems like a guidebook mistake
more specifically the misuse of the word "Business Logic"
when i initially said it, i didn't meant to make it sound like the entire website's code is there but more so the code that executes
so thats the culprit
so in the end, it was just my mistake
eh not really
its kind of mixed
you did word it a bit too cryptic
but
ultimately the critique if its that, its justifiable
imma fix this one, since ye this can be misleading
probably i should rename it to "Execution Logic" right?
ok then
i will redo the critis
you weren't consistent vs the previous critique
if its finished, there is like no score
no breakdown of each part, what gets right vs what wrong
its just one argument
ok
i suggest fetching the new changes btw il make
since i do notice this is the older diagram
ok
Ok
and honestly your critique is the topic i was discussing about "AI Use", you shouldn't follow it blindly even if it may seem correct at a first glance, ask yourself "Am i hearing bullshit or is it a genuine problem", challenge the ideas it makes and so on
ok
@jolly frost actually, im noticing something
perhaps a memory leak
these Arc drops are applied towards the end in bulk
ye as confirmed there are a lot of allocations happening
which seem to be staying
so this hypothesis is aligned
to confirm this
il look if the memory grows
since i've allocated like 200k tasks, the memory should be cconstant-ish
not grow
ok so
the CPU usage is
it seems to spike a lot
and grow slowly
interesting...
memory usage drops?
and towards the end spikes up
this has to be investigated more
Arc on strong count always indicates 2
one thing btw we should change are the arguments and return values
like man
they are fixed
and very specific
which forces a specific design and not something i want
i think i know why the sudden Arc drop happened
its the Tasks no longer be referenced
so they just drop like flies (get the pun)
which is kinda normal
one thing i notice btw is a lot of Vec pushes and pops
@jolly frost honestly for the timewheel
howz goin
we will need probably hierarchical timing wheel
Okay
hey @fervent lark i was going though code base, is docs updated ?
well for guidebooks its sorta up to date
but for API docs currently no, we are considering a heavy rewrite
okay
https://github.com/GitBrincie212/ChronoGrapher/issues/90 is this open ?
ig but someone has kinda already worked on it
plus i don't find it useful in the core
perhaps this one would be better https://github.com/GitBrincie212/ChronoGrapher/issues/93
okay i will take look
though perhaps it might be better a topic we could discuss, on how best to structure it and make it powerful
ya we should discuss but i havnt go thorugh whole code base so dont know much
well the idea of the feature is the Scheduler could inject its own stuff on top of Tasks being scheduled
while you specifically register a Task with the workflow timeout -> fallback -> retry
GSIs could add their own TaskHooks
their own workflows on top
and could fully change the Task itself
make sense
though i was thinking for it to be more powerful
though
actually the more i think about it
the more i should keep it this way
for performance reasons
and rename it to a better name or smth
when you say powerful what actually you want ?
Well it would be able to do more than just modify Tasks
could control some of the behaiviour in the Scheduler
hmm i will think about it and discuss with you right now just check code base, i understand how things going on high i want understand deep before i do anything
you could read the guidebook a bit
though it uses a macro syntax not yet implemented
is it in docs ?
so far it explains a bit about workflows and TaskHooks
thanks
np
just know some areas are a teeny tiny bit oudated (new error handling approach)
plus
it uses the proc-macro syntax not yet made
i reccomend giving them a read and telling your thoughts about it
both it and the contribution guidelines
yes ofc
i've published some changes
reccomend pulling them
also the guidebook guidelines are fully finished
ye, its a lot...
around 2.5k words
also if you think its AI, well its partially only there for polishing like grammar mistakes and incossitencies
generally the draft was fully written by me
@jolly frost what'd you think?
thanks
move checklist to the end
repeatedly states the same purpose of the guidebook in multiple ways
=>
Where is :
^
damn good eye
on like :
The structure is a heading btw
Ik
but then it wouldn't make sense to drop the structure down
Yeah
ah i see
for the checklist, hmmm
idk, we could keep it there or move it at the end
actually move it at the end
rename it to Guidebook Checklist
add the : yourself
?
Structure is the heading
of all 3
if we move structure
we move all the subheadings
oh
also how is like the guidebook guidelines? In terms of what it says
strictness, tips for technical writing and so on
@placid drum i forgot to tell you, anything you might need me for explaination since rn API docs are missing
im here
good actually
okay i will ask you if i am stuck
btw il provide you some important details you should know
-
Never confuse
TaskHooksas just Event Listeners, they are more than that, they can be full on State Managaers (via listening toTaskHook<()>or the aliasNonObserverTaskHook), Post-Error Handlers (similar to error handlers, but they cannot save the error, usually they do external activity to midigate the error, they are a sort of event listener niche), Markers (a niche to state managers, just by their presence, one can deduce information about) -
TaskHooksare even more powerful than just event listening and the above stuff, there are patterns such as Hook-To-Hook communication, registering internal TaskHooks in TaskFrames for tracking, modifying data in TaskHooks, running different code based on one or multiple TaskHooks and so on... -
TaskFramescan be stacked with one and the other, THE ORDERING MATTERS, its the same as decorating where if you switch the ordering, the entire behaiviour of the workflow part changes, for exampleretry -> fallbackβfallback -> retry -
The
Schedulercontains a config generic, this is more than just a fancy way to not write many generics for the individualSchedulercomposites (which areSchedulerTaskDispatcher,SchedulerEngine,SchedulerTaskStoreandSchedulerClock). Each composite has access to this config object and with it, they can do lots of stuff such as:- Running different (perhaps even more optimized code) for specific config shapes (they could detect if a specific composite is present).
- Support only a specific config shape
and another thing to mention, the design philosophy of ChronoGrapher which you should follow is "Minimalism Over Bloat, Emergent Over Predefined, Simplicity Over Complexity (*)", the third principle has an asterisk to it, as usually its best to prioritize flexibility / extensibility (i.e. Power), but not too much, an equilibrium is required
is it helpful?
ok then
also some contribution guidelines are long, they are more made for correctness rather than just accessibility
just don't let it scare you, take things a step at a time
ya i will need time to get comfortable with this
@fervent lark one question: will chronographer suport sync tasks?
in the examples only have async so i was question me if it will have.
use chronographer::prelude::*;
#[task(schedule = interval(2s))]
async fn MyTask(ctx: &TaskContext) -> Result<(), TaskError> {
println!("Hello ChronoGrapher!");
Ok(())
}
#[chronographer::main]
async fn main(scheduler: DefaultAnyhowScheduler) {
let task_inst = MyTask::instance();
let _ = scheduler.schedule(task_inst).await;
}
I almost finalized the critics, but this time I will avoid as much as possible going through the embarrassment that I passed yesterday.

well
sync tasks wwill require some trickery
maybe like using one async instance instead of multiple
honestly is it really important to support it?
If you try run a code that is sync into async context, it will trigger a error in rust?
At least, in python, this happen
But for other languages, could exist a trick to for exemple, run a sync function who came from other language as async
But, what about rust?
the idea would be to just run a block operation on async
or smth
though again, ask yourself, is this something really critical
most applications when it comes to the need of scheduling are async
like lots of I/O operations, lots of networking, lots of concurrent work
this is the perfect enviorement for async to thrive
oh god... @arctic trench the keyboard warrior
Let's suppose that I have a web application that is not async, basically a microservice that trains ML models.
This app will have a simple interface in web app service to manage its models for other microservices.
And I will need a scheduler/task runner to perform the background tasks like gathering data and training the model.
In Python, for example, the scikit-learn and PyTorch libraries are synced.
In the case of Rust, we know the async ecosystem is more mature because it appeared in the language's early stages.
Looking right now for ML libs for Rust, I found the smartcore lib, similar to scikit-learn from Python, and it doesn't have async APIs; it's completely synchronous.
For this example, I would have problems using the Chronographer because I will need to adapt the SmartCore APIs to allow async execution, normally doing something like sync to async.```rust
use tokio::task;
async fn run_sync_code() {
let result = task::spawn_blocking(|| {
expensive_sync_operation()
}).await.unwrap();
}
Searching here about async operations in Rust, I found that if you try to run an expensive CPU-bound operation in an async context, the operation will block the main thread.
This will have performance implications.
Just to have context, you know how the Event Loop works?
In this case, i know for Node.js, but i belive libs like tokio works the same
But is too complex convert the tasks from users into async?
Like, would be no possible verify if the task is sync and if true, run it inside the task::spawn_blocking?
ye it would be a bit of a hassle to convert ChronoGrapher to sync
oh like make ChronoGrapher handle both async and sync tasks
Yeah
Well
but depends
keyword is doable
it will be hard
i don't think its worth it
seems worth it but i highly doubt it is
this is something we will see with the userbase of ChronoGrapher
if they do demand it
then we will try
Well, then, in this case, how our users would avoid the the example i showed?
and not like one to two users but like a couple
we could make the dispatcher hold a thread pool
or smth
and distribute tasks there
What is smth?
smth = something
Humm, i will take a look at the tokio docs, to see how the event loop works
And currently in chronographer, you alocate new threads to execute the tasks?
well right now nope
tokio i think does it
automatically
perhaps
idk
if not, well a worker pool will benifit both async and sync (as it will be easier for compatibility)
humm, men, i don't know, but any way, i will search a little here, after that i pass what i found
my critics
I remember that something was missing, and I was planning to add it later, butβ¦
I forgot what it was 
anyhow and thiserror are optional, and i clearly state We reccomend so its a reccomendation and not enforcement
plus this can be backed up by
this allows for better error handling...
humm, and what about the --features flag?
hold on reading
good point
about the features
might work out better
ye ChronoGrapher would benifit from making them features
But, if I'm not mistaken, Scheduler takes a config as one of its properties, right?
So, how can this config be customized too?
Yes it does, but the config is a trait
we could maybe use a macro to automate the boilerplate if needed
but slightly
not fully
also for the article element, you wondering the TLDR?
like what its goal?
yeah
it doesn't pack much info
its just says what we've done
for the frameworks, yeah definitely
we will need
i do agree the In Hurry section should be added
i do agree with the integrations coming before workflows
thats a good critique
ye
almost...
do not critique the last 2 chapters
Advanced Workflow Primitives and Summary & Practical Patterns
honestly il get to work on those a bit
ok
yup those
do look into the handling failure and basic use of retries
il change Fallback Patterns & Usage a bit, wwork on Advanced Retries & Fallbacks and finish it
the fallback chapter i realize teaches a very bad pattern
#[taskframe]
async fn myfallback(ctx: &TaskContext) -> Result<(), TaskError> {
// Real code is more complex than this demo
println!("Simulating fallback logic, attempting to fix this issue");
let mut rng = rand::thread_rng();
// Fallbacks can fail too, and SHOULD leak through their errors
rng.gen() <= 0.8
}
ye...
It teaches the user to do things randomly and lazily in fallbacks
ok
men, would not be better reformulate the docs only you fully finish this error handle part?
the error handling part is sorta finished
like only some areas to adjust
and its done
humm, ok
also speaking of error handling
anyhow can't really plug in nicely
type Error: Error + Send + Sync;
it requires the Error trait
hmm
@fervent lark , did you read this section?
https://tokio.rs/tokio/tutorial#when-not-to-use-tokio
Tokio is a runtime for writing reliable asynchronous applications with Rust. It provides async I/O, networking, scheduling, timers, and more.
maybe, our aswer for our problems would be here
ye though they are incompatible with each other
sorta
but like i tried before
why?
eh i tried it before
and i had issues
@arctic trench @jolly frost do want a comment
should i remove the error bound?
and just have it Send + Sync?
Man, to be frank, I'm not the most indicated person to suggest it; I'm still learning.
What is this error bound ?
from std
will be quite hard to get it right
though its important
im going mentally insane over this shit
how tf did you do 1k commits??
insanity
and being locked in a white room for 24 hours a day
nah jk
creazy do you have job or somthing ?
mind you, this is the core. THE CORE
im the Unemployment Final Boss
never heard of a job
/j
are you not looking for one ?
no no i have, sort of, though don't want to disclose that detail
private reasons
say so i thought how tfk recruiters can be this blind
lol
what i understand from codebase is we create Task using TaskFrame and scheduler taskfream which where we execute what we schedule and there TaskConfig and Scheduler which use to help us to decide when to and how to excute task
ya task config is struct that have frame and schedule which task trigger
pub struct TaskConfig<T1: TaskFrame, T2: TaskTrigger> {
frame: T1,
schedule: T2,
}
? isnt this correct i read in /src/task.rs
also what is concept of erasedTask ? and like somewhere impl a hook but also you make non emittable or NonObserver i dont recall it properly why ?what is analogy behind it?
well ye but like the name is different
an erased task is just a type alias to Task with dyn TaskFrame (kinda) and dyn TaskTrigger
() is a special event, the reason it exists is so you can implement a TaskHook while never listening to any kind of event
its not emittable
and NonObserverTaskHook is supposed to be an alias to it
oh wait now i get what you mean by TaskConfig
#[derive(TypedBuilder)]
#[builder(build_method(into = Task<T1, T2>))]
pub struct TaskConfig<T1: TaskFrame, T2: TaskTrigger> {
frame: T1,
schedule: T2,
}
this one
i tired but its gives ambiguous ans so just drop the plan
yes
its just a builder thing
honestly though
it might be reworked
for like a better syntax
or actually removed, better, as it doesn't require a builder really
i guess naming can better i guess its little confusing
honestly, it will be removed
as no point using a builder
so should i read whole codebase for like what should i do ?
like you said its gonna have some changes like i can do some small task and learn codebase while doing task.
ig read it to get an idea
then with the tasks
you will get more familiarity
ya i guess i should do some implementation and playaround
ye
one thing you could help btw is error handling
its a pain in the ass
don't think about erasing the error (kinda), fully erasing the error has been done before and lets just say
the error handling side wasn't for sure having fun with downcasting
btw the idea currently is to have an associated type for the error
that implements the Error trait bound
and is Send + Sync
in practice the Error trait bound is problematic
anyhow cannot be supported, it can slightly but it requires you to basically convert it to a box dyn error
its not a smooth process and kinda defeats the purpose of using anyhow imo
another idea was to like use Into<Box<dyn Error + Send + Sync>> for anything convertible to it
but then i ran to conflicting implementations since anyhow "could" implement error in the future
i understand what you are saying but not quite i guess i need more time to adapt to rust give 1-2 weeks
i never build something this complex in rust
mhm
Hi @fervent lark as we talked, here is my feedback for the docs. I tried to give as much as helpful feedback.
- Maybe because of my pc but rendering at the beginning is causing browser to stuck.
- The ordering of retries and fallbacks is pinned ( Shouldn't this be "are" pinned? )
- For the workflow(retry(3, 2s)) whereas task interval was already 2s. We may get the workflow::delay::default as the time of the task. Since people will do the same time most of the time except people who are aware what they are doing, it might help to reduce boilerplate.
- Again this will be about workflow, I liked the timeout macro, but as far as I understood from the docs, as default it is infinite. When you define timeout(5s) then only it timeout. This is nice, but instead of infinite non-defined. Wouldn't it be nice to timeout using the task interval like let's say task interval is T * retry * retry_time as default or only T for timeout, retry, Fallback order. For infinite tasks, we can define timeout(None) or something like that.
- One question, can I define workflow like (timeout(10s), retry(3, 2s) , timeout(1s)) ?
- Advanced Fallbacks felt like it is cut off at the end. I think you could explain it little bit in more detail which can display specific error handling for each fallback. I think we are aiming to implement specific logic for handling errors at least.
- It would be nice to have counting_logic to have Fn(ctx: _ThresholdCtx, err: Error) -> bool , so you can define your own counting logic or even advance to counting logic depending on called retries, timeouts etc.
- I am not sure if I missed it, but I dont think I saw fallback(...) ordering affect is it like first come first serve?
I will continue reading docs later. Introspecting Workflow and Contributing left.
sure thing
- Hmmm, will need more context about it, very weird your rendering causes the browser to be stuck such as hardware, the browser you use and so on. Could be your PC perhaps you had many stuff loaded, have you tried running only the website without any other applications to see if its really the problem
- Yeh probably typo
- One idea would be to have the ability to reference global variables
- We will see about that, not sure tbh, we could, would need a bit more explaination
- Yes you can define that and thats the power of ChronoGrapher versus other workflow orchestrators really,the timeouts act to different sections, the
timeout(1s)acts to every retry whereastimeout(10s)is a global deadline for the entire workflow - Yes its a work in progress, the idea would be to explain the multiple fallbacks patterns where you declare smaller fallbacks each handling their own sets of errors, if it can't handle it, then it errors out (which propegates to the next fallback and the cycle repeats til we went with all fallbacks)
- There is, i do plan on having for
threshold, its just not documented - Hmm, this might be part of the WIP docs of advanced retries and fallbacks
thanks a lot for the feedback, if you can go the extra mile and rate:
- How effectively it taught you stuff per chapter
- How professional it was per chapter
- The cogntive load per chapter
- What you enjoyed about it
- How much friction it posed to you
if you noticed any quality drops and even a total score out of 10, would be heavily appreciated
sometimes i wish Error was more standarized, and everything would be convertible to it
really painful
#[derive(Debug, Error)]
#[error("Something went wrong")]
pub struct ErrorIsWrong;
#[tokio::main]
async fn main() {
let schduler = Scheduler::<DefaultSchedulerConfig<ErrorIsWrong>>::default();
let exec_frame = DynamicTaskFrame::new(|_ctx| async {
println!("Trying task.....");
// Ok(())
Err(Arc::new(std::io::Error::new(ErrorKind::Other, "uh oh")) as ErrorIsWrong)
});
let task = Task::simple(TaskScheduleCron::new("* * * * *".to_string()), exec_frame);
let _ = schduler.schedule(&task).await;
schduler.start().await;
loop {}
}
if i do this it will not let me throw the error like it should do its showing typecasting error but this too bad way to do errors like if i am defining error isnt it suppose get used there something like this ?
@fervent lark
hm wait
it will throw an error?
wdym
the SchedulerEngine will print the error
and shutdown the task
no compiler will throw error it will not let compile code
what error specifically from the compiler
like Non-primitive cast: Arc<std::io::error::Error> as ErrorIsWrongΒ [E0605]
hmm
hm
Err(ErrorDyn)
if use this instead
Trying task.....
Scheduler engine received an error for Task with identifier (4a79773e-9597-4c08-b95c-6dd48419ab9d):
ErrorIsWrong
so what i am saying is that isnt it uneccary to defined
let schduler = Scheduler::<DefaultSchedulerConfig<ErrorIsWrong>>::default();
here ?
its nesscarry
also
so it will throw on schedule error ?
ye hold on let me explain it clearly
pub trait SchedulerConfig: Sized + 'static {
type TaskIdentifier: TaskIdentifier;
type TaskError: TaskError;
type SchedulerClock: SchedulerClock<Self>;
type SchedulerTaskStore: SchedulerTaskStore<Self>;
type SchedulerTaskDispatcher: SchedulerTaskDispatcher<Self>;
type SchedulerEngine: SchedulerEngine<Self>;
}
This is the config
a bit modified in my enviorement
but still same-ish
type TaskError: TaskError;
this is the task error
the reason its something in the configuration is because workflow parts will probably need to know the errors
same with the Scheduler stuff
the idea is TaskError is any kind of error you expect your tasks to throw
defining it, allows you to handle the errors via match
the idea is to contrain the error per Scheduler
one scheduler might be built for ErrorA the other scheduler for ErrorB
when it will be thrown like is it for type only ?
ye do explain it clearly
we are using error per schedule, my question is that
when we use somthing like this
<DefaultSchedulerConfig<ErrorIsWrong>>
is it just used to satisfy the type or will i get thrown as i use this
#[error("Something went wrong")]
wdym is it just used to satisfy the type or will i get thrown as i use this?
can you explain it more clearly
be very specific about stuff generally, since rn its kinda vague
now i get it like we defining in ```
<DefaultSchedulerConfig<ErrorIsWrong>>
like this because we expect task error to be type this. i thought it is different for task and schedule
ok?
ya
DefaultSchedulerConfig btw is for the default scheduler components
it just requires a generic
because different enviorements have different error types
so i can't remove it
yess, i was confuse there, i thought why cant we just remove it then i saw code it like we expect task to throw error same as generic type
is it correct ?
yes
we can't really remove it, for the reason mentioned above, so we keep it as a generic
it is actually great we know before hand that our task is gonna thore same error as we define
yup
fk it i never meant to chat damn!
?
nah just frustrated over spell mistake
its sorta ok, though sometimes they do confuse me, i mean for casual stuff i won't mind
for something you want to tell about ChronoGrapher, then ye i need a bit more specificity
thanks tho
i will try my best to be specific.
np
also what do you mean you need error standardization ?
.
anyhow is a different error type than the typical std error implementations
it doesn't implement the error trait
so you want something uses std error implementations
no
but like i wish errors weren't a pain in the ass to manage
any solution you thought ?
well i had thought of multiple
like providing a conversion trait which can convert to the error types
the above however as turns it out, it can't work
another was enum storing std errors, anyhow errors... etc. But turned to be a bad API design
so the third solution was:
this one
pub trait TaskError: Debug + Display + Send + Sync + 'static {
fn as_any(&self) -> &(dyn Any + Send + Sync);
}
a bit pain for the users managing TaskHooks
but i can't prevent it
i havnt reach that part but if you are saying it might be correct.
a worthwhile part to check
its arguably one areas ChronoGrapher has excelled in
Pushed new changes btw
@jolly frost
slowly ChronoGrapher seems to be finishing
its core
@placid drum if you think the core itself is complex, wait til about persistence and distributed systems
these are super difficult
let see for me all complex are complex
persistence is incredibly hard, like i've tried to deal with it, i've even got a theory for it which could prove much better than other competitors
but
its still hard
i wanted to built sync engine for ws only which will be light weight only and build node freamwork on top-off it
then i saw this project, it same thing i wanted like i wanted somthing in node that schedule task but excute python. well i cant explain any further i am really bad at this thing
so like have one Task in JS, another in Python and another in Java and it would work in harmony?
if you mean that, well it was considered as a feature, like you could listen to events defined in Python from Java
but ultimately its something complex and maybe not really in need
ya like python is good for cpu bound task compare to node and node for io compare to python
otherwise we need queue/db to share task details