But dnt stop, this isn't guarantee
#JShell project for Tj-Bot!
sterile crest
That's okay, I don't think we can figure this out with your knowledge and expertise so there's always the day after
Counting on you
ebon river
And what errors?
timber mirage
@sterile crest I'm here
timber mirage
no
timber mirage
have you pushed your changes to a branch or main?
Yes, on branch improveclose
@ebon river try this branch and you will see
run a one time session
sterile crest
have you pushed the branch?
timber mirage
yes
sterile crest
1 min ago nice
timber mirage
sterile crest
send me the cURL command too you were testing with
timber mirage
send me the cURL command too you were testing with
curl --request POST \
--url http://localhost:8080/jshell/single-eval \
--data 'for(int i = 5; i < 10; i++) {
System.out.println(i);
}'
And what is weird there?
ebon river
I don't get any error either
sterile crest
what version of docker are you using?
ebon river
What version of the image are you using?
sterile crest
I'm on Docker version 26.1.1, build 4cf5afa
timber mirage
v24.0.2
sterile crest
I pulled the repo fresh and had a clean docker with no images or containers
ebon river
REPOSITORY TAG IMAGE ID CREATED SIZE
togetherjava.org:5001/togetherjava/jshellwrapper master 1dddb68cf79d 2 minutes ago 343MB
I just built this from source
Docker version 27.1.1, build 63125853e3
timber mirage
```
REPOSITORY TAG IMAGE ID ...
this is unrelated to the wrapper
it didn't change since a long time ago
ebon river
What is your exception?
sterile crest
once he calls close the container is killed but the underlying docker thread is still open and tries to read from stdin causing it to throw an IOException with some communication link as the error message
timber mirage
wait 30s please
sterile crest
the SDK creates a thread under the hood, at least for:
public InputStream startAndAttachToContainer(String containerId, InputStream stdin)
throws IOException {
PipedInputStream pipeIn = new PipedInputStream();
PipedOutputStream pipeOut = new PipedOutputStream(pipeIn);
client.attachContainerCmd(containerId)
.withLogs(true)
.withFollowStream(true)
.withStdOut(true)
.withStdErr(true)
.withStdIn(stdin)
.exec(new ResultCallback.Adapter<>() {
@Override
public void onNext(Frame object) {
try {
String payloadString =
new String(object.getPayload(), StandardCharsets.UTF_8);
if (object.getStreamType() == StreamType.STDOUT) {
pipeOut.write(object.getPayload());
} else {
LOGGER.warn("Received STDERR from container {}: {}", containerId,
payloadString);
}
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
});
client.startContainerCmd(containerId).exec();
return pipeIn;
}
java.io.IOException: java.io.IOException: Le canal de communication a �t� ferm�
at java.base/java.nio.channels.Channels$1.read(Channels.java:166) ~[na:na]
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149) ~[httpcore5-5.2.4.jar:5.2.4]
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:172) ~[httpcore5-5.2.4.jar:5.2.4]
at org.apache.hc.core5.http.impl.io.IdentityInputStream.read(IdentityInputStream.java:91) ~[httpcore5-5.2.4.jar:5.2.4]
at org.apache.hc.core5.http.io.EofSensorInputStream.read(EofSensorInputStream.java:118) ~[httpcore5-5.2.4.jar:5.2.4]
at com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:30) ~[docker-java-core-3.3.6.jar:na]
at com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:12) ~[docker-java-core-3.3.6.jar:na]
at com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:275) ~[docker-java-core-3.3.6.jar:na]
at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]
Caused by: java.io.IOException: Le canal de communication a �t� ferm�
at java.base/sun.nio.ch.Iocp.translateErrorToIOException(Iocp.java:299) ~[na:na]
at java.base/sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:389) ~[na:na]
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[na:na]
... 1 common frames omitted
@ebon river
sterile crest
update your docker version @timber mirage just to entertain me 😄
timber mirage
@ebon river the thread name is "docker-java-stream--676896705"
ebon river
I mean you could be a good boy and close the stream properly before killing the container
But apart from that I am not sure you can handle this gracefully
timber mirage
I mean you could be a good boy and close the stream properly before killing the ...
Doesn't work
ebon river
How did you try that?
ebon river
I mean, the remote end probably disappears when the container closes
But maybe newer docker daemons are nicer about it
Updating might be worth a shot
Your docker is ~a year old
ebon river
With 24.0.9?
timber mirage
v27.1.1
ebon river
or 27.x?
sterile crest
ala, i'll jump in vc - check that my steps are the same as yours?
timber mirage
which steps ?
sterile crest
my repro steps
timber mirage
which steps ?
sterile crest
bruh
timber mirage
steps are literally
sterile crest
reproduction steps
sterile crest
if this isn't an issue for me and instan and it's not appearing on the vps
it's a you problem
timber mirage
We don't know if it is happening on the vps
sterile crest
We don't know if it is happening on the vps
it's still a you-problem
ebon river
Yea
sterile crest
but in any case, ignore the exception, there's nothing you can do besides thread.join
ebon river
And then the error is swallowed
sterile crest
you probably don't want to thread.join because i'm not sure what's happening under the hood
ebon river
I am not sure there is anything you can do honestly
At some random point your remote end disappears and you have no idea when or why
sterile crest
if it's a new thread per container, then you can otherwise, just swallow the error and leave it
ebon river
If you make a funny handshake close protocol you could store the closable in onStart and invoke it, ending the attachment stream
And only then exit the container
Simplest solution there is probably to make the container wait for a second or two before shutting itself down and stopping the stream in that period
timber mirage
If you make a funny handshake close protocol you could store the closable in `on...
which stream
I have no control on that
ebon river
client.attachContainerCmd
The frame adapter has a onStart method
And this is passed a closable which will close the underlying streams and streamer thread when invoked (using interrupts)
So you can gracefully exit the stdin/stdout/stderr streaming
sterile crest
@timber mirage you could try client.removeContainerCmd(containerId).exec();
sorry
maybe not
ebon river
The problem is that the remote end disappears
sterile crest
what's the inverse for attachContainerCmd?
ebon river
And this is passed a closable which will close the underlying streams and stream...
And the only way to stop that is
ebon river
what's the inverse for `attachContainerCmd`?
Calling the closable you get in onStart when attaching
timber mirage
`client.attachContainerCmd`
but why would this be a problem
are you saying that in the common case where the container would stop for any reason, it will caus this error
this isn't normal
ebon river
You are streaming logs by opening a tcp connection to the docker daemon and you try to read from it
At some point the container says bye
And then the question is where in the protocol your client is stuck currently
Judging by the code in FramedInputStreamConsumer at least
sterile crest
@timber mirage try this:
ebon river
It should signal closure
sterile crest
client.attachContainerCmd(containerId)
.withLogs(true)
.withFollowStream(true)
.withStdOut(true)
.withStdErr(true)
.withStdIn(stdin)
.exec(new ResultCallback.Adapter<>() {
@Override
public void onNext(Frame object) {
try {
String payloadString =
new String(object.getPayload(), StandardCharsets.UTF_8);
if (object.getStreamType() == StreamType.STDOUT) {
pipeOut.write(object.getPayload());
} else {
LOGGER.warn("Received STDERR from container {}: {}", containerId,
payloadString);
}
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
@Override
public void onComplete() {
try {
pipeOut.close();
LOGGER.info("Stream from container {} closed successfully.", containerId);
} catch (IOException e) {
LOGGER.error("IOException while closing stream from container {}: {}", containerId, e.getMessage());
}
}
});
added public void onComplete() { to the block
ebon river
On my machine you eventually receive an EOF in step 1
And then it gracefully shuts down
timber mirage
so what happens in my case ?
why do you not get this error ?
and also
I wish to know
Why I never got this problem before ?
sterile crest
environmental?
did you download a fortnite vbuck generator recently or something?
or maybe a windows update?
timber mirage
no
sterile crest
java version?
ebon river
But it takes a while for the docker container to exit after the evaluation
sterile crest
are you using his branch @ebon river ?
ebon river
enhancement/improveclose
sterile crest
there were some changes made to the close functionality, instead we first mark is as "dead" controlled by a boolean and then there's a scheduled task to remove them
timber mirage
so why is this a problem only for me ?
timber mirage
how am i even supposed to find such answer
ebon river
Add the logging breakpoint there maybe and see what you get
and how it correlates to the containers dying
timber mirage
wdym
ebon river
You could set a logging breakpoint here maybe
.
ebon river
What do you mean?
timber mirage
What do you want me to do ?
sterile crest
at this point, you might have better chances in #1051826284008853505
timber mirage
This is the data of the stream when it crashed
ah
well
it got rid of the null characters
but imagine that there are null chars before the OK and a ton of null chars after the last line
timber mirage
@sterile crest @ebon river
how did you run it ?
I tried to run the api from inside docker
and I don't get the error
sterile crest
try {
writer.close();
} finally {
reader.close();
try {
reader.close();
} finally {
if(!dockerService.isDead(containerName())) {
dockerService.killContainerByName(containerName());
}
}
}
while I get what you're trying to do...
it flipping sucks
timber mirage
Do you have a better idea ?
sterile crest
try {
writer.close();
} catch(Exception e) {
LOGGER.error("Failed to close writer", e);
}
try {
reader.close();
} catch(Exception e) {
LOGGER.error("Failed to close reader", e);
}
if(!dockerService.isDead(containerName())) {
dockerService.killContainerByName(containerName());
}```It's more redondant 🙂
sterile crest
yours is redundant
and worse
here's why
reader.close(); being called twice
that's redundant
timber mirage
wait
sterile crest
if one fails, you don't know which one - less observability
timber mirage
that's a typo
try {
writer.close();
} finally {
try {
reader.close();
} finally {
if(!dockerService.isDead(containerName())) {
dockerService.killContainerByName(containerName());
}
}
}
should have been this
anyway
I'll fix it
sterile crest
you're dong all this unnecessary try-finally nesting
but then again
your code is never readable so idc 😄
jshell is such a toy
timber mirage
public void close() {
try {
writer.close();
} catch (Exception ex) {
LOGGER.error("Unexpected error while closing.", ex);
}
try {
reader.close();
} catch (Exception ex) {
LOGGER.error("Unexpected error while closing.", ex);
}
try {
if(!dockerService.isDead(containerName())) {
dockerService.killContainerByName(containerName());
}
} catch (Exception ex) {
LOGGER.error("Unexpected error while closing.", ex);
}
LOGGER.info("Session {} died.", id);
}
alright
is it more readable tho 🤔
sterile crest
yes but your ordering of logging is wrong
LOGGER.info("Session {} died.", id); needs to go after if(!dockerService.isDead(containerName())) {
timber mirage
it is ?
sterile crest
yes because LOGGER.info("Session {} died.", id); will log even on failures
timber mirage
how did you run it ?
also @sterile crest ?
timber mirage
weird
sterile crest
it working on docker means it's purely environmental when running locally :p
sterile crest
whenever close is called
then change the message
timber mirage
?
sterile crest
imagine seeing Session died in the log message
but all the code to kill the session exception
the session didn't die
timber mirage
then you would see the error just before
sterile crest
not if you're filtering by info messages
timber mirage
who would filter out errors ?
sterile crest
people who work on production systems and actually monitor logs using proper tool
timber mirage
no
sterile crest
it's not common practice to look at log messages in the console or just as a .txt file
sterile crest
you need more profesional experience in software development
if you see:
error
error
error
session died
okay errors idc about now because "session died"
but the session didn't die
but your logging told me the session died
sterile crest
your close method isn't killing a session then
your logging in the wrong area of the code
timber mirage
it is
sterile crest
```
error
error
error
session died
```
give this example
public void close() {
LOGGER.info("Session {} died.", id);
}````this is what your code is doing
if we ignore the errors
sterile crest
yeah 🙂
timber mirage
Alright, pushed
sterile crest
gradle spotlessApply
if(shouldDie()) throw new DockerException("Session %s is already dead.".formatted(id)); don't forget braces
and the error message should be different because the session is not dead explictly
timber mirage
and the error message should be different because the session is not dead explic...
hmm
sterile crest
} catch (Exception ex) {
LOGGER.error("Unexpected error while closing.", ex);
}
try {
reader.close();
} catch (Exception ex) {
LOGGER.error("Unexpected error while closing.", ex);
}
more complicated than that
sterile crest
put "Reader" and "Writer" in the messages for clarity
timber mirage
shouldDie -> session at least half dead
sterile crest
yeah
sterile crest
"Session is no longer available because it's either dead or marked for removal"?
timber mirage
shouldDie -> at least irrecoverable, maybe totally dead
markAsDead -> has been marked for next heartbeat
timber mirage
"Session is no longer available because it's either dead or marked for removal"?
nah
like I said
the problem isn't this message here
since it is true
sterile crest
ok
timber mirage
shouldDie -> at least irrecoverable, maybe totally dead
markAsDead -> has been m...
the problem is here in those names
but I could find anything better
sterile crest
if(service.isMarkedAsDead()) {
try {
jshellSessions.remove(id).close();
LOGGER.info("Session {} died.", id);
} catch (Exception ex) {
LOGGER.error("Unexpected exception for session {}", id, ex);
}
} else {
service.markAsDead();
}
redunant else
session is already marked as dead
sterile crest
ah yeah myb
Maybe this should be renamed: LOGGER.info("Session {} died.", id); to "Session has been removed"
actually
nevermind
its correct
forgot it's doing a close() too
public void deleteSession(String id) {
jshellSessions.get(id).markAsDead();
}
can we remove this and just use jshellSessions.get(id).markAsDead();
since it does the same thing
deleteSession != markAsDead
timber mirage
can we remove this and just use `jshellSessions.get(id).markAsDead();`
since it is called by several places, no
sterile crest
refactor the name?
timber mirage
I will take a pause
sterile crest
ok
timber mirage
Can you move to the pr and do an actual review please
instead of talking here
it will be lost
sterile crest
that's the last comment
compact holly
shouldDie -> at least irrecoverable, maybe totally dead
markAsDead -> has been m...
damn, brutal :(
timber mirage
damn, brutal :(
what ?
sterile crest
He just has an emotional attachment to docker sessions @timber mirage
timber mirage
He just has an emotional attachment to docker sessions <@175710436967055361>
I killed so many of them
compact holly
should is a moral judgement, something that ought to be, compared to somhing that just is
it's like saying container deserves to die :(
you wish death upon it
timber mirage
it's like saying container _deserves_ to die :(
but it is already dead
compact holly
which is arguably worse than just killing it in self-defense
compact holly
but it is already dead
ah, should be dead, not should be killed 👌
sterile crest
No wonder we're having issues with containers dying
And the VPS not being cleaned up
Technology has feelings too
The TogetherJava VPS is supposed to be like heaven for tech, a nice safe place for them to live without the fear of dying
Now it's a graveyard of docker sessions 😦
timber mirage
Now it's a graveyard of docker sessions 😦
Well it's not a graveyard since there is no tombstone to count the history of dead containers, more like a crematorium
sterile crest
We should be kind to our software, let's leave the containers running but they all can host their mini web servers and we can use spring to load balance
Actually, we should have done that
sterile crest
Well it's not a graveyard since there is no tombstone to count the history of de...
Sounds like hell
They must burn!!
timber mirage
anyway #chit-chat
bright lark
yoo
sorry for being late I had a time issue today while checking-out from hotel .. anyway
is it solved ?
timber mirage
no
compact holly
is it solved ?
nope, you got any ideas?
bright lark
alright, based on the discussion above
it looks that alathreon is only one having the issue
without knowing yet how it happens
sterile crest
well spotted
bright lark
that means, ala has something causing the issue
it would be useful if somebody having a no issue version, to assist ala via a tool like rdp
sterile crest
excellent idea, any tools you recommend? maybe you can rdp onto his machine and investigate
bright lark
i mean to reinstall whole app
ye why not
but not today, as i dnt have much time
i m still checking the changes and learning about docker api
gotta sleep early too
bright lark
I keep getting these logs without doing anything yet
2024-08-05T22:20:09.172+01:00 INFO 21452 --- [pool-2-thread-1] o.t.j.service.JShellSessionService : Scheduler heartbeat: started.
2024-08-05T22:20:09.173+01:00 INFO 21452 --- [pool-2-thread-1] o.t.j.service.JShellSessionService : Scheduler heartbeat: sessions ready to die: []
2024-08-05T22:21:09.158+01:00 INFO 21452 --- [pool-2-thread-1] o.t.j.service.JShellSessionService : Scheduler heartbeat: started.
2024-08-05T22:21:09.158+01:00 INFO 21452 --- [pool-2-thread-1] o.t.j.service.JShellSessionService : Scheduler heartbeat: sessions ready to die: []
is this normal ?
compact holly
we all should be ready to die
I think sessions just adopted philosophy of the great Stoics
so looks pretty normal to me
they don't seem depressive or suicidal, just mentally built different, strong 
not scared of the future, calm and ready
compact holly
logs are more detailed, to help you with debugging 
you can pick and choose the log level to your liking
bright lark
the level is info and those logs are infos
it would better to switch them to debug lvl
JShellSessionService looks too complex ngl
alright, it seems that im gonna be stuck with the current code base
as there are no comments, in addition to some ugly stuff
in fact alathreon is not the only one who notice the errors
i had some errors too, and they're mysteriously gone
as this is 1st time i try such project, i think it'd be better for me to exercice a bit
so i'm gonna try a sample with similar objectives, but in minimal shape
will come back with updates
compact holly
if you can reproduce the issues, you can go trough it with a debugger
that way you can see what is happening and you can follow the flow
bright lark
that way you can see what is happening and you can follow the flow
the problem is
i'm unable to understand the current codebase
it's too new to me, or maybe some parts are ugly
compact holly
yeah, it's somewhat complex
bright lark
spaghetti code
compact holly
what do you feel is lacking?
you can ask ala for explanation of parts that are confusing
try to follow the flow of the command with debugger
bright lark
that will not help as i may re-ask anytime
bright lark
i think that's why
not many people came
it lacks clarity
opensource project without documentation is like private
im gonna build a very basic sample that works properly
after that i may come again
timber mirage
```bash
2024-08-05T22:20:09.172+01:00 INFO 21452 --- [pool-2-thread-1] o.t.j.se...
Yes, it's the session killer, each X time, it will check for alive sessions, that's called a heartbeat
timber mirage
why would i get logs why i didn't nothing
A ton of reason, many software have things that run in the background
timber mirage
JShellSessionService looks too complex ngl
Do you have any idea about how to simplify it ?
timber mirage
as there are no comments, in addition to some ugly stuff
Like ?
bright lark
Yes, it's the session killer, each X time, it will check for alive sessions, tha...
alright, i dnt need it
actually it confuses me
timber mirage
as there are no comments, in addition to some ugly stuff
Most of it should be obvious, there are doc for methods whose behavior isn't obvious, but most of the time, it should be fine
timber mirage
alright, i dnt need it
wdym
timber mirage
actually it confuses me
why ?
bright lark
why ?
after i created a session
i expect it to be in the list to die
but the logs keeps saying
sessions ready to die []
timber mirage
wdym
timber mirage
but the logs keeps saying
> sessions ready to die []
yes, so there are no session ready to die
timber mirage
yes
bright lark
and ready to die
timber mirage
nope
sessions have a lifetime of 30min by default
after those 30 minutes, they will die
bright lark
ye but it could die earlier
timber mirage
well no
timber mirage
the idea is to keep the session alive, so the user could use the previous statements and add new ones
or also read their own statements
So you could sent a request which contains a class
bright lark
the idea is to keep the session alive, so the user could use the previous statem...
this is not mentioned
anywher
timber mirage
and then a request to execute code using this class
bright lark
as i said earlier some parts are hard to read (complex)
bright lark
and i dun't know how to simplify them
timber mirage
(it's not up to date but at least the general idea still works)
bright lark
im still learning
timber mirage
as i said earlier some parts are hard to read (complex)
show me those parts
bright lark
https://github.com/Together-Java/JShellPlaygroundBackend/tree/develop/JShellAPI
ahaa !
timber mirage
ahaa !
I mean this https://github.com/Together-Java/JShellPlaygroundBackend/blob/develop/JShellAPI/README.MD
Remember, it's no up to date, but at least the general idea is still here
bright lark
show me those parts
jshellsessionservice
anyway, np
im gonna exercice with a minimal sample
i need to do tht
timber mirage
jshellsessionservice
which part ?
I advise you to just look at the public methods
like
collapse everything
and look at it like you would look at an interface
only see public
and even ignore bodies
compact holly
i advise you to follow the command with debugger, and see how it works, the flow
timber mirage
should help a lot
compact holly
to get the familiarty and see how the systems fits together
timber mirage
i advise you to follow the command with debugger, and see how it works, the flow
You shouldn't even this this, you could follow it with your eyes, since there are nearly no interfaces so no indirection
compact holly
debugger offers a lot of cool tools
bright lark
i know i can do it
compact holly
and you get annotated values
timber mirage
well yea, a debugger would make it even easier
compact holly
so it's easier to follow and play with the codebase
bright lark
i knooow
compact holly
but yeah, whatever you are more comfy with
timber mirage
🤷
sterile crest
Firas struggling to read code?? 
He's one of our top expert members too, so there must be really deep rooted problems
bright lark
He's one of our top expert members too, so there must be really deep rooted prob...
stop making fun of me will u?
sterile crest
i'm just playing, not making fun of you, you're right about some of the observations
bright lark
Today gonna full debug
my 1st lesson in tj-jshell
As for the future it would be nice to create a website for it
A playground with file explorer would be great
timber mirage
A playground with file explorer would be great
We have a discord bot where you can use jshell command 🙂
bright lark
We have a discord bot where you can use jshell command 🙂
I know tht already
But what if we think about extension?
It would be cool, like codesandbox
It's in same context ig
timber mirage
It would be cool, like codesandbox
Well yes, that's one of the reason why I created a REST API at the first place
bright lark
i think about writing a schema.json openapi for the api
without need to the springboot openapi dependency
thus, the controller's code can rest without additional annotations
the schema can be used in swagger ui playground
compact holly
someone already did that iirc, but ala disliked it, check PRs
timber mirage
thus, the controller's code can rest without additional annotations
I want to reduce the number of annotations, it's fine if there are little of them
compact holly
ah, without annotations
bright lark
yes
timber mirage
There might be ways
Like create an interface with swagger annotations
so it's splitted between the code and swagger
bright lark
my approach is to split schema from from java code
but it must be something to validate
so maybe yes
timber mirage
So if you wan't, you can study this idea
timber mirage
I don't think so
bright lark
we need to set that up
compact holly
also sonar and other tooling
sonar will probably raise a lot of issues that would need to be addressed
sterile crest
we don't need an openapi spec right now
the focus should be purely on the functional aspects, like making sure it works
bright lark
the focus should be purely on the functional aspects, like making sure it works
I meant it's not priority
It just came to my mind and wanted to share
timber mirage
do i pick develop branch ?
rebase from develop yes, like tj bot
bright lark
oh boy
so session are kept in memory
private final Map<String, JShellService> jshellSessions = new HashMap<>();
i dnt get this
jShellService is a spring bean
it's the 1st time i see this
ah JShellService refers to session?
the name is ambiguous
is it ok to write some comments and rename some methods to make names have more sense ?
another question: why there are no entities ?!
alright im going to make some changes in a separate branch
and PR
sterile crest
another question: why there are no entities ?!
good question
compact holly
what do you think about architecture and layering?
bright lark
what do you think about architecture and layering?
kinda spaghetti
compact holly
more vertical or horizontal
maybe we should do domain driven development with hexagonal architecture?
how would you do it?
bright lark
oh nice
compact holly
or standard model service repository or whatever
compact holly
right, we can just use json config
bright lark
wdym by that?
sterile crest
everything is in memory
what about an inmemory database?
compact holly
if bot dies, all sessions are lost
bright lark
what about an inmemory database?
possible as enhancement
compact holly
or when we deploy new version
bright lark
the current impl can work without db ig
bright lark
if bot dies, all sessions are lost
if api is down
compact holly
right
bright lark
kinda fragile
if persistence is a requirement
then we should add orm, db and entities
compact holly
maybe containers would infiniteky accumukate, because they wouldn't be kilked
bright lark
however, id add entities package even without db
bright lark
cache tool
ah i heard about it
never used it
id go to firebase
it gives lota options
quartz shell
Redis would be nice, especially if u are just using it for caching.
bright lark
oh it's not free
sterile crest
maybe we can use neo4j
it's a graph based db
it's the best performing graph db and it's also java based
it supports generative AI now
sterile crest
we can replace the HashMap with this
sterile crest
redis is free <a:check:872522683408085032>
that's a problem
sterile crest
free means it's not gonna be good
quartz shell
self hosted 
bright lark
self hosted <:chefskiss:1082421838375817236>
ah as a container
that depends on @compact holly
bright lark
alright, we're getting faraway from objectives
we'll get back to this as soon as we fix the current version of the app
quartz shell
we'll get back to this as soon as we fix the current version of the app
what happened to it? Did it crash or smth
compact holly
that depends on <@238042761490595843>
depends on all the good people of tj that help maintain the infrasturcture
you will se those have the VpsAccess role
compact holly
we'll get back to this as soon as we fix the current version of the app
ye, that's a good thing to focus on
what is the current issue you are having?
timber mirage
is it ok to write some comments and rename some methods to make names have more ...
make an issue
timber mirage
another question: why there are no entities ?!
because there is no db
timber mirage
kinda spaghetti
What do you mean
It's controller -> session service -> jshell service -> docker service
timber mirage
sure , but i dnt see this as priority now
Yes, I don't see a point of persisting something that will disapear within 30min anyway
sterile crest
not if we have firas idea of multiple consumers
timber mirage
yes indeed
sterile crest
anyways that's all free with AWS 
compact holly
not if we have firas idea of multiple consumers
the idea you originally thought was stupid btw
"why we have this API, we don't need it blah blah"
you simply did not see the bigger picture firas did
timber mirage
me*
compact holly
we will have a magnificent website where people could run their code on
and maybe even embed our code runner to their sites
that will promote our community
and create backlinks
sterile crest
the problem with an API is that there's additional latency and overhead
compact holly
ah, the performance
timber mirage
we will have a magnificent website where people could run their code on
Problem is that the server is too weak to handle this
sterile crest
like Kafka
compact holly
we can't spare 3ms of latency 
sterile crest
we can't spare 3ms of latency <:peepo_cry:979524296462467102>
it's more than that
you can't scale the API with the current infra
compact holly
Problem is that the server is too weak to handle this
we will host that on wazei's aws
he claims its free
compact holly
see
timber mirage
we will host that on wazei's aws
...it won't work like that
compact holly
millions of daily users for free
but yes, imagine every technical blog having our code runner exampels embedded
sterile crest
millions of daily users for free
the free tier is in the hope that the business gets thousands of users
compact holly
togetherjava will become intergalactically famous and respected
sterile crest
then they can bill you properly
for TJ and small businesses, it's perfectly fine
i can even make a serverless discord bot
the thing that keeps a discord bot showing as "online" is the heartbeat, you can use AWS EventBridge scheduler for that
and use AWS APIGW Authorizer for serverless web socket events
or use EventBridge to wake up when it receives an event
compact holly
then they can bill you properly
we are broke, it's impossible to bill us
we declare bankrupcy, and whoop, it's gone
sterile crest
our usage is so small anyway
compact holly
that's how dept works in us at least
yeah, but you are not imagining the world firas does
in firas world, we are large and respected organizaton
we have a website where you can run code with our jshell api
and my idea is to make it embeddable in every website, or tool that needs to run code
so every blog uses our tech, stackoverflow answer
to we become internationally famous
sterile crest
well if we used my original "just give the user an entire linux container"
sterile crest
then we have unlimited possibilites
compact holly
firas world is big picture world
compact holly
well if it work in web, it works on phones as well
sterile crest
we can get my guys to create an app using jetpack compose and it'll let people run our jshell service too
compact holly
we can scam people with some electron bs
sterile crest
we can scam people with some electron bs
hello?? jhtmlbridge??
compact holly
right, some people would ask who runs java code on their phone
and our answer should whoever wants
we did it because we could, to prove our technical superiority
sterile crest
We can build our API using https://github.com/surajkumar/wazei
the wazei web framework
compact holly
riight, all wazei stack
sterile crest
exactlyyyy
public class HelloController {
/**
* ---
* $method=get
* $path=/hello
* $content-type=application/text
* ---
*/
public String hello(String nameParam) {
return "Hello, %s!".formatted(nameParam);
}
}
look at that
your comments control logic
compact holly
anyway, once we host brian goetz here on the server
sterile crest
or if you dont like commenting your code
public class HelloController {
public String greet(String name) {
return "Hello, %s!".formatted(nameParam);
}
}```we can help him make project loom and panama happen
compact holly
i have no doubts :D
sterile crest
: D : D
@compact holly we can even use the wazei concurency library: https://github.com/surajkumar/concurrency-library
compact holly
riight
sterile crest
it comes with built in monitoring and metric collection
System.out.println(promise.getMetrics()); // To get metrics of a Promise
// > PromiseMetrics{start=140720275666100, end=140720275677900, success=true, executionTime=11800, memoryUsage=0, errorDetails='', stackTrace=[]}
are these actually real links?
compact holly
you created random projects to spice up your CV?
compact holly
soon you will have more libs than ethan lol
compact holly
ah, more sphisticated
sterile crest
but at least I'm not just repackaging existing libs
u should see my private repo's
like this one
compact holly
rather not
sterile crest
u can test an API using Socks5 proxy
so when we wanna simulate real users
i gotchu
This repository contains 290 open Socks5 servers that are available for use.
/*
* Call our example REST API call 10 times for each proxy.
* Since there are 290 registered proxies, all 290 servers will run our request
* 10 times sequentially.
*/
socks5.run(10, exampleSocks5);
oh, expected something like: https://github.com/Deepshift/DeepCreamPy
in your privte repos
sterile crest
compact holly
was cutting edge AI tech at it's time
compact holly
is that your project as well?
quartz shell
incredible
sterile crest
i also made this "route to mic" feature which lets you pipe the audio from azure voice to in game chat
great for those people shy to talk but still wanna use ingame coms
compact holly
how many stars does that project have?
compact holly
do you have any uwu intellij extensions?
you know, moral support while you write code
someone to cheer you on when our shit finally compiles
sterile crest
i actually replaced all my icons with ur profile pic
sterile crest
ty, i needed an icon change
compact holly
maybe enable partial compilation
sterile crest
i made u my start icon
compact holly
so you can compile justone method
and run it
and not having to build whole ass thing
for quick testing and protyping on larger projects
sterile crest
marko u like anime stuff right?
compact holly
our jshell will make a revolution in the industry
compact holly
uh.. kinda creepy, hopefully not sexual
but evne if it's sexual, at least you are happy
sterile crest
uh.. kinda creepy, hopefully not sexual
if you send a message e.g. -hug- @wazei - it'll send an embed with an anime hugging gif saying "marko hugged wazei"
compact holly
why would I buy if you just sent me the codes, dummy 
never thought hacking would be this easy
compact holly
if you send a message e.g. `-hug- @wazei` - it'll send an embed with an anime hu...
jesus christ 
sterile crest
you can't compile 😄
quartz shell
wanna buy this? https://gist.github.com/surajkumar/099d399a94bcc9a280f1a3955a54c...
JDA no java discord api!/
sterile crest
JDA no java discord api!/
its old code
quartz shell
its old code
smh
compact holly
any code*
sterile crest
i can teach u how to use azure voice once firas does his azure migration
anyways
enough shit posting
compact holly
that would have to be a premium feature
quartz shell
why are we swithing to azure not alibidbaba
sterile crest
why are we swithing to azure not alibidbaba
broo has anybody uses the cloud stuff alibaba has?
compact holly
weebs are easy target to get some monies
sterile crest
quartz shell
broo has anybody uses the cloud stuff alibaba has?
I acually wanna try it, I thought its more popular than gcp
sterile crest
gcp kinda sucks ass
sterile crest
my cloud journey started gcp>azure>aws
sterile crest
I find it's interface hard to navigate
the ui sucks
i never was happy with gcp, back when i was a google fan boy, i did my best to love their products
but gcp was aids
sterile crest
what if a meteor hits ur server?
compact holly
then it would hit me too, so I wouldn't care
sterile crest
okay ill ask firas to put that down as part of our risk mitigation diagram
i think the idea was to run a really long USB cable from your house to his?
sterile crest
actually, it's possible if we become tier 1 ISP
or tier 3 can't remember which one digs up the ground and lays cable s
compact holly
ye, it's easy to be ISP
people done it locally, because monopolies
basically fancy wifi
sterile crest
i looked into doing it before but it wasn't cost effective
compact holly
and you can have 10gbit/s connections
sterile crest
you can't
compact holly
it's not some shitty tech
vivid cliff
and you can have 10gbit/s connections
usb is not similar to ethernet
sterile crest
it wasn't my idea
sterile crest
i think the idea was to run a really long USB cable from your house to his?
"his"
sorry ala
we're off topic
timber mirage
@compact holly @sterile crest please I am tired, stop going offtopic here
sterile crest
so what's the topic on jshell ala?
timber mirage
?
compact holly
so this thin
sterile crest
is there something u wanna talk about around jshel?
compact holly
you can just shoot internet to pepople trough air
and they can have 1 or 10gbit
no issues
it's cheaper to set up, because you don't need to dig up the infra
but more expansive over the long run
sterile crest
but how would u get 10gig
compact holly
these are powerful
sterile crest
connected to what host?
compact holly
not same wifi crap tech
sterile crest
u still need a physical 10gig somewhere
timber mirage
is there something u wanna talk about around jshel?
not really ?
timber mirage
then we're staying off-topic
????
compact holly
i mean yeah
but no digging the whole city
you shoot internet from one place to the user
sterile crest
but i live in the UK
compact holly
well from many places
compact holly
ah, that sucks
sterile crest
so from my house we need a long cable leaving our island?
just dig a hole through the middle of the UK 😄
compact holly
i pay $10 for gigabit
10gig would be maybe $20, dunno
shit's cheap
multiple fiber providers in my building, so strong competition
actually 1 isp called me 10 today
so aggressive, offering me 250mbit and iptv for free
for 2 yeras
in the hope that I will stay with them afterwards or something
I was thinking to maybe have another network with servers with that connection
but can't be bothered with all that bs
it's free tho :/
could be used for good
anyway, this doesn't help us with jshell
@bright lark, what's your progress
update us
compact holly
so from my house we need a long cable leaving our island?
simple solution would be to not live on the island
sterile crest
not that u could afford it
compact holly
having to do cabling through ocean is pain in the ass
compact holly
poor austrailans get triple fucked
they have 300 ping in most games, ozon holes give them cancer
they live upside down, and the wildlife is very hostile
living life on nightmare diffuculty
bright lark
update us
hi again
i don't know why u enjoy talking about offtopic here
ive done some debugging to understand the flow of the app. I ve tried the /jshel/eval/{id}
the docs helped me to get closer
however, i notice some weird patterns
like injecting docker and session services into the model JShellService
this could lead to errors or circular deps
also some method names look ambiguous. some conditions are duplicated unnecessarily
it seems that the project was created quickly without thinking too much about maintainability
JShellService constructor takes 12 args !
as i said previously, the hard part is thinking, code is the easiest part
so i may add some diagrams to show the actors, uc and flows
bright lark
what happened to it? Did it crash or smth
no! it's just tht i dun't want to talk about this rn
bright lark
what is the current issue you are having?
i dn't have any, im just inspecting the current codebase in develop branch
bright lark
make an issue
gotit
timber mirage
this could lead to errors or circular deps
There is a plan to remove session service from here which would solve this problem
timber mirage
also some method names look ambiguous. some conditions are duplicated unnecessar...
like ?
timber mirage
JShellService constructor takes 12 args !
I was fixing this last day, there will be a pr soon
bright lark
There is a plan to remove session service from here which would solve this probl...
is there an issue for it ?
timber mirage
is there an issue for it ?
no, but it will naturally go away
bright lark
duplicated condition
if (!hasSession(id)) { }
wdym
bright lark
a method name must indicate a behavior. session doesn't seem clear
oneTimeSession() too
as for createSession() it looks fine
timber mirage
like `session()`
right, we will rename it to fetchSession
bright lark
right, we will rename it to fetchSession
It's actually addOrUpdate or somthing like tht
bright lark
Idnt think so, it looks clear
The method adds a session or update an existing one
Maybe a more clear name is createOrReuse()
Also session id should be provided by the system
Not by client
So if id is null, tht means NEW, if it's not null, it means reuse/update
timber mirage
Idnt think so, it looks clear
the exact name should be createAndGetOrFindAndGet
That's a no
bright lark
Lmao
What do u suggest then?
Maybe we should split them
Into 2 endpoints
POST/session and PUT/session
timber mirage
`POST/session` and `PUT/session`
no
sessions should automatically be created
And
fetchSession is a correct name
bright lark
fetchSession is a correct name
how would I fetch something that doesn't exist yet ?
the name confuses reader
im typically creating a session, so method must be create() or add()
while when i fetch an existing one, i may use something else like fetch() or get()
🤷♂️
timber mirage
im typically creating a session, so method must be `create()` or `add()`
You can't just make everything more complex because you can't find a name
timber mirage
how would I fetch something that doesn't exist yet ?
You don't need to know that, it's abstracted away
bright lark
You don't need to know that, it's abstracted away
alright, we can keep /eval
but we should change the name of session()
bright lark
createOrFetch
bright lark
how's tht?
timber mirage
And even then, it is included in the word fetch
timber mirage
how's tht?
You can see it as implementation detail
bright lark
imo we need to have 2 methods in JShellSessionService
and call right one based on the args
if(newId){
return create();
}
return fetch();
also, im not certain that providing id from client is ok
timber mirage
```
if(newId){
return create();
}
return fetch();
```
You just reinvented session()
timber mirage
How is it redundant?
timber mirage
and confusing name
Then call it find
timber mirage
the condition is called again
Which condition?
bright lark
i mean this if (hasSession(sessionInfo.id())) {}
timber mirage
Sorry I don't have the code under my eyes
Alternatively you could call it obtainSession
bright lark
public JShellService session(String id, @Nullable StartupScriptId startupScriptId) throws DockerException {
if (!hasSession(id)) {
return createSession(new SessionInfo(id, true, startupScriptId, false, config));
}
return jshellSessions.get(id);
}
Right, and createSession?
bright lark
private synchronized JShellService createSession(SessionInfo sessionInfo) throws DockerException {
// Just in case race condition happens just before createSession
if (hasSession(sessionInfo.id())) { }
}
even the comment is not signficant
bright lark
Does it fail fast?
i dnt recall
i think yetserday when i tried it
for my 1st session, it was false
timber mirage
Wdym you don't recall? Do you not remember the code, do you not have the code under your eyes?
timber mirage
for my 1st session, it was false
?
timber mirage
Please show the code
bright lark
i did
timber mirage
No you didn't
bright lark
there is a collision between session() and createSession()
timber mirage
You cut the part I asked
timber mirage
there is a collision between session() and createSession()
Please show the code
bright lark
private synchronized JShellService createSession(SessionInfo sessionInfo)
throws DockerException {
// Just in case race condition happens just before createSession
if (hasSession(sessionInfo.id())) {
return jshellSessions.get(sessionInfo.id());
}
if (jshellSessions.size() >= config.maxAliveSessions()) {
throw new ResponseStatusException(HttpStatus.TOO_MANY_REQUESTS,
"Too many sessions, try again later :(.");
}
LOGGER.info("Creating session : {}.", sessionInfo);
JShellService service = new JShellService(dockerService, this, sessionInfo.id(),
sessionInfo.sessionTimeout(), sessionInfo.renewable(), sessionInfo.evalTimeout(),
sessionInfo.evalTimeoutValidationLeeway(), sessionInfo.sysOutCharLimit(),
config.dockerMaxRamMegaBytes(), config.dockerCPUsUsage(), config.dockerCPUSetCPUs(),
startupScriptsService.get(sessionInfo.startupScriptId()));
jshellSessions.put(sessionInfo.id(), service);
return service;
}
public JShellService session(String id, @Nullable StartupScriptId startupScriptId)
throws DockerException {
if (!hasSession(id)) {
return createSession(new SessionInfo(id, true, startupScriptId, false, config));
}
return jshellSessions.get(id);
}
public JShellService session(@Nullable StartupScriptId startupScriptId) throws DockerException {
return createSession(new SessionInfo(UUID.randomUUID().toString(), false, startupScriptId,
false, config));
}
session() has 2 versions
timber mirage
Ah I see
Yes it's to avoid a race condition
Alternative would be to extract this if into session and add an additional sync block here
timber mirage
```java
public JShellService session(String id, @Nullable StartupScriptId startu...
Well you could also do that, it's just that it will always reach a sync block
Which while it could backfire if we increase the size of this by a lot, it shouldn't matter right now
bright lark
Yes it's to avoid a race condition
wdym
timber mirage
wdym
There is a chance two threads arrive on the if in session at the same and so try to create a session for same id
bright lark
There is a chance two threads arrive on the if in session at the same and so try...
alright, why we have session() if it's only to call createSession() ?
let's get rid of session() then
timber mirage
alright, why we have session() if it's only to call `createSession()` ?
The problem that I had is that it is syncronized and so it doesn't scale well
bright lark
do you know what syncrhonized do ?
what i know is that it's for thread safety
which means the method cannot have multiple threads on it at a time
so i dnt get what u mean by
it is syncronized and so it doesn't scale well
btw i think that createSession is an async operation as it waits a response from docker
timber mirage
which means the method cannot have multiple threads on it at a time
Yes, so if two requests arrive at the same time, and ession was sync, then the requests would be evaluated one at a time, whicle while it currently works scale vary badly, because the more requests we have, the more problems it will cause
bright lark
how about using @Async
timber mirage
how about using `@Async`
How will this solve anything ?
bright lark
How will this solve anything ?
i dunno 
timber mirage
Ah and because of makro and wazei oftopic, I can't find what they said last time
@sterile crest can your review my pr please ?
bright lark
ive read tht @Async in spring is helpful to assert one thread handles async operation
is tht unrelevant
timber mirage
no
sterile crest
are u able to force merge ur PRs @timber mirage ?
sterile crest
ah okay
timber mirage
I am waiting for a review
timber mirage
do u want me to join ?
join what ?
bright lark
join what ?
in pr
timber mirage
what ?
timber mirage
We were talking about my pr
bright lark
u have more than one
timber mirage
about revewing my pr
bright lark
about revewing my pr
im checking this one https://github.com/Together-Java/JShellPlaygroundBackend/issues/44
bright lark
do the tests
im still debugging (learning)
as i said previously, there are some weird stuff
and i gave examples
tests could be done when there is clean stable codebase
timber mirage
tests could be done when there is clean stable codebase
?