JShell project for Tj-Bot!

[sterile crest]

yeah they pass

i added a new GHA called pre-commit that does gradle build

it'll build + spotlessCheck + run tests

[timber mirage]

@sterile crest you forgot this

[sterile crest]

pushed

[timber mirage]

and did you change the properties + compose in the vps ?

[sterile crest]

nope

i don't wanna clickops in vps - so can u do it 🙂

[timber mirage]

You are giving me more tasks -_-

[sterile crest]

and that's why it's called pair programming

i have to go for a little bit that's why

[timber mirage]

Wait a minute

how do I checkout on your branch ?

Since it's a fork

do I have to clone ?

[sterile crest]

https://github.com/surajkumar/JShellPlaygroundBackend/tree/repo-cleanup

yeah

my account isn't added as a maintainer that's why

so i have to use forks

and I don't wanna setup tj-wazei on intellij because multiple accounts start causing issues

the repo looks so nice though now

[timber mirage]

You didn't fix it

Try to click on it

it will fail

@sterile crest

[sterile crest]

its pushed

Actually

im gonna remove the docker-compose step

since we added the compose to the repo

[timber mirage]

no it's needed

well

[sterile crest]

why

we're created the compose

[timber mirage]

you have to change it

[timber mirage]

you have to change the step into move it to the folder

[sterile crest]

let me to do the README in a different PR

the entire thing needs re-writing tbh

[timber mirage]

Yea, there are things I want to change, better like this

Why did you remove the java plugin for the wrapper ?

[sterile crest]

it's inherited from the parent gradle

[timber mirage]

?

I cant run it

[sterile crest]

subprojects {
    apply plugin: 'java'

[timber mirage]

Intellij doesn't find JShellWrapper:un

look

[sterile crest]

do you mean application plugin?

add the java plugin won't make it runnable

[timber mirage]

ah maybe

I don't remember what I put

but there should be a run task

used only for debugging

[sterile crest]

okay i removed that from the API because Spring gives you a run, it wasn't present in wrapper

but I can add it

sec

[timber mirage]

And add back the comment

saying it's for debug only

[sterile crest]

the comment isn't needed

[timber mirage]

it is needed

this plugin isn't needed

it's only here for debugging

[sterile crest]

it can be documented in the README

which the module doesn't have yet

[timber mirage]

no, in the build.gradle please

so someone who read it know why it is here

and why it should or shouldn't be used

[sterile crest]

okay...

pushedd

[timber mirage]

what did you do so I can't just regular push ?

[sterile crest]

ur commits signed?

[timber mirage]

what are signed comits ?

[sterile crest]

do you know about GPG?

[timber mirage]

no

[sterile crest]

see here:

my commits are verified

ah rip the tests broke

i know the issue

[timber mirage]

I can't even run the wrapper anymore

Exception in thread "main" java.util.NoSuchElementException: No line found
    at java.base/java.util.Scanner.nextLine(Scanner.java:1660)
    at org.togetherjava.jshell.wrapper.JShellWrapper.run(JShellWrapper.java:22)
    at org.togetherjava.jshell.Main.main(Main.java:9)

[sterile crest]

public static Config load() {
        return new Config(loadIntEnv("evalTimeoutSeconds"), loadIntEnv("sysOutCharLimit"));
    }

 static int loadIntEnv(String envName) {
        return Integer.parseInt(System.getenv(envName));
    }

set your environment variables up

[timber mirage]

It's not a problem about the env variables

It's System.in which is closed

see the error

scanner.nextLine is failing

[sterile crest]

pushed a maybe fix

my tests are failing

because of this crappy line-encoding issue

[timber mirage]

I fixed that in the past

by modifying gradle config

Seems like you broke it 🙂

And no, I won't try to fix it again

Me not knowing gradle + trash issue = way too much time lost on it

fix it yourself

[sterile crest]

i didn't ask u to fix it

[timber mirage]

so at least it works

[sterile crest]

yay

[timber mirage]

but now, the tests to fix 🙂

[sterile crest]

oh

it's not gradle

it was spotless i think

ah fuck

spotless

bahaah

or was it...

hmm

i mean they're easy fixes

pusheddd ❤️

perfect checks pass

if you're happy with the PR hit approve

then merge and update VPS

@timber mirage i have added application.yaml and updated the docker-compose.yml

just double check it

[timber mirage]

Found another problem

Seems like you broke something else

@sterile crest

[sterile crest]

I think that's because of the default case

does this come from wrapper or api?

[sterile crest]

maybe not

can u show me what u did

or it could be with verifyStartupEval

[timber mirage]

POST .../jshell/eval/test?startupScriptId=CUSTOM_DEFAULT
2+2

Note that it worked before

[sterile crest]

i kno

ah wait ur doing it using curl?

[timber mirage]

curl --request POST \
  --url 'http://localhost:8080/jshell/eval/test?startupScriptId=CUSTOM_DEFAULT' \
  --header 'Content-Type: text/plain' \
  --header 'User-Agent: insomnia/2023.5.8' \
  --data '2+2'

[sterile crest]

works for me

{
  "snippetsResults": [
    {
      "status": "VALID",
      "type": "ADDITION",
      "id": 23,
      "source": "2+2",
      "result": "4"
    }
  ],
  "abortion": null,
  "stdoutOverflow": false,
  "stdout": ""
}

and the tests on GH are passing too?

i did gradle bootRun and then just made that request

[timber mirage]

tests are pssing

[sterile crest]

is this a new bug?

[timber mirage]

Ah I know

You deleted an else

and there were no tests to test it

So

Moving the files + doing modifications was a very bad idea

Wait you changed a lot of other code

I'm sorry but that's way to much to try to figure out

please roll back all those changes

You introduced at least two bugs here

And within the 60 files, I don't know how many are they

So refactoring code will be in another PR

So roll back the changes for the switches for now

[sterile crest]

No worries

I thought that else block was nothing

I'll fix in an hour

[timber mirage]

What I want you to fix isn't this bug, but rather remove all the refactoring you did and move it in another pr, it's extremely hard to review this when there are so many modifications, that between the code format and the rename of files, it's impossible to tell what was changed into what

so keep this pr as project improvment (docker, gradle, delete useless files) and formatting code

the rest, move it to another pr

[sterile crest]

Skill issue but ok

I'll remove it

But I'm not creating anymore PRs for the day

[timber mirage]

right

not a problem

[compact holly]

ah, boys doing stunt development in discord chat

what a beautiful thing to wake up to

[sterile crest]

@timber mirage i dropped those comits

u can review by commits

[timber mirage]

will do that later, I'm busy right now

[sterile crest]

no rush

[compact holly]

backend is currently chain crashing, for anyone interested

looks like this (but suuper fast moving )

[sterile crest]

?!

ohj

ohh

the current PR will fix it

I didn't realize but as part of using Java 21 the Spring Plugin needed updating

[timber mirage]

Alright

I merged it

@sterile crest

how to I create a pr on master tho

[compact holly]

release it the same way you would on tj-bot

in github UI, on master, you will likely have button "master is behind, click to create PR"

but PR is just there for documentation purposes, and for visibility

[timber mirage]

https://github.com/Together-Java/JShellPlaygroundBackend/compare/master...develop

i can't click the create pr button

[sterile crest]

https://github.com/Together-Java/JShellPlaygroundBackend/pull/20

noob

[timber mirage]

how

alright done

[sterile crest]

all good?

looks good to me

yay it works

[sterile crest]

@timber mirage good for prod?

[timber mirage]

I guess ?

Well

Nobody expect us really tested it but 🤷

[sterile crest]

we're competent enough

[sterile crest]

@timber mirage #chit-chat message

It failed to run?

[timber mirage]

There is a `

[sterile crest]

are these times correct?

@timber mirage

I am also seeing stuff like:

[timber mirage]

30 minutes after last eval, so check their logs and see when was the last eval

[timber mirage]

show more of the logs please

[sterile crest]

JSHELL IS PUBLIC!?

this explains why there was random requests coming in - random bots on the internet do this

[stray wren]

wazei's result

Snippets

Snippet 31, VALID

// Import necessary classes
import java.net.*;```
### Snippet 32, VALID
```java

import java.io.*;```
### Snippet 33, VALID
```java


// Define the URL
URL url = new URL("http://togetherjava.org");```
jshell> `http://togetherjava.org`
### Snippet 34, VALID
```java


// Open the connection
HttpURLConnection con = (HttpURLConnection) url.openConnection();```
jshell> `sun.net.www.protocol.http.HttpURLConnection:http://togetherjava.org`
### Snippet 35, VALID
```java


// Set the request method
con.setRequestMethod("GET");```
### Snippet 36, VALID
```java


// Get the response code
int responseCode = con.getResponseCode();```
## [WARNING] The code couldn't end properly...
Problematic source code:
```java


// Get the response code
int responseCode = con.getResponseCode();```
Cause:
Uncaught exception:
java.net.UnknownHostException:togetherjava.org

Remaining code:
```java

System.out.println("Response Code: " + responseCode);

// Read the response
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuilder content = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
    content.append(inputLine);
}

// Close the connections
in.close();
con.disconnect();

// Print the content
System.out.println(content.toString());```
## System out
[Nothing]

wazei's result

Snippets

Snippet 31, VALID

// Import necessary classes
import java.net.*;```
### Snippet 32, VALID
```java

import java.io.*;```
### Snippet 33, VALID
```java


// Define the URL
URL url = new URL("https://togetherjava.org");```
jshell> `https://togetherjava.org`
### Snippet 34, VALID
```java


// Open the connection
HttpURLConnection con = (HttpURLConnection) url.openConnection();```
jshell> `sun.net.www.protocol.https.DelegateHttpsURLConnection:https://togetherjava.org`
### Snippet 37, VALID
```java


// Set the request method
con.setRequestMethod("GET");```
### Snippet 36, VALID
```java


// Get the response code
int responseCode = con.getResponseCode();```
## [WARNING] The code couldn't end properly...
Problematic source code:
```java


// Get the response code
int responseCode = con.getResponseCode();```
Cause:
Uncaught exception:
java.net.UnknownHostException:togetherjava.org

Remaining code:
```java

System.out.println("Response Code: " + responseCode);

// Read the response
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuilder content = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
    content.append(inputLine);
}

// Close the connections
in.close();
con.disconnect();

// Print the content
System.out.println(content.toString());```
## System out
[Nothing]

wazei's result

Snippets

[WARNING] The code couldn't end properly...

Problematic source code:

// Import necessary classes import java.net.http.*; import java.net.URI; import java.io.IOException; import java.util.concurrent.CompletableFuture;  // Create an HttpClient HttpClient client = HttpClient.newHttpClient();  // Create an HttpRequest HttpRequest request = HttpRequest.newBuilder()     .uri(URI.create("https://togetherjava.org"))     .build();  // Send the request asynchronously and get a CompletableFuture CompletableFuture<HttpResponse<String>> response = client.sendAsync(request, HttpResponse.BodyHandlers.ofString());  // Print the response body when it's available response.thenAccept(res -> {     System.out.println("Response Code: " + res.statusCode());     System.out.println("Response Body: " + res.body()); }).join();```
Cause:
The code doesn't compile, there are syntax errors in this code.

## System out
[Nothing]

wazei's result

Snippets

Snippet 38, VALID

// Import necessary classes
import java.net.http.*;```
### Snippet 39, VALID
```java

import java.net.URI;```
### Snippet 40, VALID
```java

import java.io.IOException;```
### Snippet 41, VALID
```java

import java.util.concurrent.CompletableFuture;```
### Snippet 42, VALID
```java


// Create an HttpClient
HttpClient client = HttpClient.newHttpClient();```
jshell> `jdk.internal.net.http.HttpClientImpl@1f57539(1)`
### Snippet 43, VALID
```java


// Create an HttpRequest
HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create("http://togetherjava.org"))
    .build();```
jshell> `http://togetherjava.org GET`
### Snippet 44, VALID
```java


// Send the request asynchronously and get a CompletableFuture
CompletableFuture<HttpResponse<String>> response = client.sendAsync(request, HttpResponse.BodyHandlers.ofString());```
jshell> `jdk.internal.net.http.common.MinimalFuture@59906517[Not completed] (id=7)`
### Snippet 45, VALID
```java


// Print the response body when it's available
response.thenAccept(res -> {
    System.out.println("Response Code: " + res.statusCode());
    System.out.println("Response Body: " + res.body());
}).join();```
## [WARNING] The code couldn't end properly...
Problematic source code:
```java


// Print the response body when it's available
response.thenAccept(res -> {
    System.out.println("Response Code: " + res.statusCode());
    System.out.println("Response Body: " + res.body());
}).join();```
Cause:
Uncaught exception:
java.util.concurrent.CompletionException:java.net.ConnectException

## System out
[Nothing]

[sterile crest]

@compact holly sorry for ping, maybe you can help

i think it's because it's using "Driver": "bridge" which my wisdom from Virtual Box tells me is that it uses the host network card

There's also "Internal": false,

[compact holly]

somone (possibly maliciously) edited docker compose

that's the only way this can happen

[sterile crest]

docker inspect c0a6cc315cdc

this is on the network

not the container?

[compact holly]

and now our backend service is exposed to the whole internet

that's why it's best to wait for me to double check everything

[sterile crest]

wym u set this up

the network was pre-existing right?

[compact holly]

there are no issues with network

only way this can happen is if someone edited the docker compose

messed with the ports

[sterile crest]

ports:
      - 8080:8080

[compact holly]

or done some weird nginx redirection to make this public

[sterile crest]

this?

[compact holly]

yes, that

[sterile crest]

let me fix that now sec

is this any of u guys?

[compact holly]

open anyway

[sterile crest]

kwl

[compact holly]

you probably don''t even need ports exposed

[sterile crest]

that fixed the issue

[compact holly]

so just remove ports alltogheter

[sterile crest]

i think tj-bot needs it

[compact holly]

try it

[sterile crest]

i don't wanna clickops

:S

but it has no internet now

[compact holly]

you mean it's not publicly exposed?

[sterile crest]

yeah

[compact holly]

since that's a huge security risk

yeah, it's not

the network it is connected to is

[sterile crest]

oh

hm

[compact holly]

wait, let me go over things

[sterile crest]

idk if the IaC pulls this docker-compose

i don't think the docker-compose is pulled from the repo - so this PR might not be needed

fuck

will u do it?

[compact holly]

it's also on the wrong network

with the dev bot

[sterile crest]

so tj-bot needs bridge to access external api's right?

but jshell and tj-bot both need to be on the same networrk to communicate

can you add another network interface, maybe?

[stray wren]

Pong!

marko's result

Snippets

Snippet 23, VALID

2+3```
jshell> `5`
## System out
[Nothing]

[compact holly]

good

[sterile crest]

oh u fixed 😄

[timber mirage]

ah

huh

could have been worse 👀

[compact holly]

well, that's the thing, you had it exposed publicly on the internet, with no checks in place

you don't know how bad it is

[sterile crest]

yeah it's bad especially for many reason, one example you'd understand is having setup spring with no security in mind

[compact holly]

backend is also not properly isolated

so RCE there means RCE on the machine

[sterile crest]

among opening up jshell to the entire planet

any zero day vulnerability can screw us

[sterile crest]

but just to flex, feel free to enjoy ur exploits to your hearts content: https://vz7fbzekqnhroe6jh6zkx3akra0smlov.lambda-url.eu-west-2.on.aws/

omg marko i have a feature idea

can we make a /slash command that can makes API requests? - we can use my lambda

[compact holly]

what kind of requests?

[sterile crest]

any request

[compact holly]

like curl, but in discord?

[sterile crest]

yes

we can accept a cURL request as an input - and run it directly on aws

[compact holly]

maybe

lets see how this goes

[sterile crest]

to make it extra fun, we can censor the auth header and x-api-key header in our response so people are safe to do stuff like that

[compact holly]

because technically we have lambda

[sterile crest]

in my discord project, i can live call the API

[compact holly]

backend has that purpose

you throw some shit, it will manage containers and stuff

and give you back the result

but not sure if we want to continue this route

[sterile crest]

i don't know how comfortable you are using ur vps with egress

because we might as well open up jshell to the internet

then we wouldn't need a dedicated curl command

[compact holly]

mostl likely it would be fine, but we can't really allow it

it's just a bad idea

[sterile crest]

why

[compact holly]

too many bad things you can do with it

[sterile crest]

can we add wazei bot to this server

i'll open source it in the tj gh

[compact holly]

what does wazei bot do?

[sterile crest]

nothing - but all the commands that don't fit tjbot

[compact holly]

that can't be integrated

which commands would that be?

[sterile crest]

jshell with internet and disk, cURL command

or any linux capabilities

have a command that mimics a linux terminal

[compact holly]

that can be integrated into tjbot

[sterile crest]

i was gonna use it as an excuse to try our new discord framework in the wild

[compact holly]

lol

sneaky wazei

[sterile crest]

it doesn't need to be called "wazei bot"

you can use whatver name u want

and keep the bot token

i just wanna eventually implement linux into discord - but i don't think i can with aws lambda - would have to be a vps

but it couldn't be yours because it's already running too many things

[compact holly]

that bad boy can run a lot of things :p

we can expand the vps

[sterile crest]

would you be willing to entertain my idea using a new bot written in our framework?

[compact holly]

hopefully when we decide to implement linux in doscrd, we also have enough money for 4 cores lol

I don't see the purpose

whole point of tjbot was to eliminate other bots

have one bot that does everything

less confusing for newcomers

[sterile crest]

i just wanna cURL so i can show example requests in my project thread when people are discussing fixes/the api

[compact holly]

what's the value add

why would we add it?

[sterile crest]

well having a linux session would mean superior answers in #1051826284008853505

"hey how do i do this"

"run these commands, look i'll show you"

/linux apt update

etc etc

[compact holly]

sure, but we can do that with tjbot

[sterile crest]

yeah we can

[compact holly]

with some limited capabilities

[sterile crest]

u down?

[compact holly]

jshell backend can serve this use

and isolated env spawner

and manager

[sterile crest]

exactly

[compact holly]

you would have to limit tho

[sterile crest]

should i post in #server-suggestions

[compact holly]

we don't want people to donwload illegal material on our network

basically restrict access to only the safe stuff

[timber mirage]

But why was it exposed ?

[sterile crest]

we just need baby containers - 500mb disk, small cpu

that have a lifespan of 5 mins

[compact holly]

because someone edited docker compose

and exposed one of the few ports that are not blocked by firewall

on the host

[timber mirage]

ah 8080 ? that's not me then
pretty sure I put 54321

[sterile crest]

port 54321 was also exposed to the internet

[compact holly]

doens't matter who it is

it wasn't

[sterile crest]

ikt was

it was

[sterile crest]

see

[timber mirage]

wait it was ?

[sterile crest]

check the url

[compact holly]

did someone mess with firewall config?

[timber mirage]

waitwait

[sterile crest]

no

[timber mirage]

why is port 54321 exposed ?

[compact holly]

that's the issue with having too many people vps access

[timber mirage]

nope

[sterile crest]

u can revoke me if u want - i don't mind

[compact holly]

because someone exposed that port

[sterile crest]

i rather have access only when i'm needed

[compact holly]

but that's not the only issue, it should've been blocked by the firewall

[sterile crest]

i reckon the firewall was already misconfigured from the beginning lol

[timber mirage]

it is, unless you can escape docker

[compact holly]

anyway

[sterile crest]

i'm gonna post the suggestion

for linux

[timber mirage]

which shouldn't be possible

[compact holly]

doesn't matter who or how did it

just be extremely careful, and know exactly what you are doing

when you edit live production machine

or wait for me to double check the whole flow

[timber mirage]

can't you add something which alerts when something as important as the firewall is changed ?

[sterile crest]

marko trying to blame us when it was probably his mistake

[timber mirage]

👀

[compact holly]

it's always my mistake

[sterile crest]

😄

[compact holly]

by default, because this is my responsibility

[timber mirage]

We mean, it was you who were playing with the network trying to connect tjbot to backend

[compact holly]

and it shouldn't be possible to expose ports without my approval

etc

[sterile crest]

u can see all the commands that were run anyway - so if you're really curious just check the history

[compact holly]

because I gave you all root access

[timber mirage]

👀

[compact holly]

doesn't matter who did it

but someone exposed 8080 on the host

and network was misconfigured

jshell was on the wrong network

it wasn't prepared for production properly

[timber mirage]

and 54321

[sterile crest]

tbh

i reckon all ports are not being blocked

u can test that by just opening any port

[compact holly]

they should be

[timber mirage]

can you check the firewall and see which ports are blocked ?

[compact holly]

yes

but everything is already fixed

[timber mirage]

because it's not normal that 54321 wasn't blocked
it's a very random port, that only you and me used (and I used it because you told me to use it)
so there is no reason for 54321 to not be blocked

[compact holly]

yes, all prots should be blocked

[timber mirage]

unless all ports are not blocked by default

[compact holly]

and also, container shouldn't expose any ports

if it doesn't need to

[timber mirage]

how do you not expose any port ?

do you just write :8080 ?

[compact holly]

all ports are blocked by default, expect few

it doesn't need exposed ports

they are talking internally

[timber mirage]

instead of 8080:8080 ?

[compact holly]

nothing, no ports exposed

if you do 8080: stuff, you expose your service to the whole world and bots

that's catastrophic failure

[timber mirage]

but can you do :8080 ?

[compact holly]

because 80, 8080, etc are not blocked

you don't have to expose any ports

[timber mirage]

?

don't you need to expose 8080 to tjbot ?

at least internaly ?

[compact holly]

you expose ports to enable port forwarding

like on a router

if you are on internal network

you don't need port forwarding

[timber mirage]

hmm I see

I have a few things to add to jshell

I'll create a few PR soon

[sterile crest]

i posted the suggestion :3

[timber mirage]

ah, I'll check that

[compact holly]

that 54321:8080 port thing I sent you was just for testing

becuase it's likely you are already using 8080 when locally testing backend

it's just random thing, not meant for prod

firewall should be tested too

yeah, I think it's blocking everyhting

you can try 54321 again

[compact holly]

it is, because it has exposed docker socket

so that container can spawn arbitary containers

[timber mirage]

what ?

[compact holly]

backend is not properly contained

you can spawn container with exposed 8080, mining bitcoin, or do whatever

you can spawn container that is not really a container

so you can basically do anything

[compact holly]

yeah, it's possible

[timber mirage]

how ?

I mean

you said that it is possible

so prove it

so I can see that and fix it

[compact holly]

there is no fix, it's designed that way

backend is not properly isolated, so if exploited, it's bad

[timber mirage]

wdym

You can't access docker from jshell

so show me how it is possible to exploit it

Because for me, what you are saying seem impossible

[compact holly]

we have a level 10 CVE almost every day

april has been crazy

RCE is not only possible, but probable on any tech

especially in tech we made

[timber mirage]

mean, this while 2024 year, you seem to forgot all the CVE that appeared in both mac, linux or whatever

[compact holly]

yes

[timber mirage]

then show it please, do it

[compact holly]

i'm not doing security analysis for free

what I'm saying is, if someone finds a way to explot it, it's bad

command injection or

[timber mirage]

except you are talking about our own VPS security

so if I understand properly

what you are saying is that

it isn't that there is a vulnerability

but rather that a vulnerability

but in this case

what you are talking is just air

[compact holly]

...

it's simple

backend is not on the same isolation like jshell containers

not even remotely

it's almost as ran on the host

from the security perspective

[timber mirage]

backend doesn't execute custom code

it's like tj bot

[compact holly]

yes, but if backend has vulns, it's bad

[timber mirage]

same for tj bot

[compact holly]

not the same, because that app is containerized better

[timber mirage]

if you find a way to bypass chatgpt command ratelimits, byebye money

[compact holly]

true, it would have serious consequeces

[timber mirage]

what does better mean ?

[compact holly]

but not RCE on vps

doesn't matter

it should be in back on our minds that backend is not the same as jshell contaienrs, or tjbot containers

[timber mirage]

why ?

why is it not the same as tjbot containers ?

[compact holly]

docker socket is exposed

so it can spawn containers

if you can spawn containers, you can spawn containers with no isolation basically

you can spawn cryptominers, or expose some malicious service publicly on the internet

steal our production database

access filesystem

delete the host OS

[timber mirage]

can you spawn container with less isolation than yourself ?

[timber mirage]

Where ?

[sterile crest]

i'm talking about the #server-suggestions

[compact holly]

yeah, you can spawn any container

it's like on host running docker containers, no difference

so whole vps can be compromised

so design idea was, keep backend as safe and sanitized as possible

[timber mirage]

@compact holly since you are the one who thinks the most about security
Tell me if this issue could help, and if there is anything you want to improve in it https://github.com/Together-Java/TJ-Bot/issues/1119

[vivid cliff]

@timber mirage is the startup script not same for everyone?

[stray wren]

alathreon

Startup script

import java.nio.charset.*;
import java.nio.file.*;
import java.text.*;
import java.time.*;
import java.time.chrono.*;
import java.time.format.*;
import java.time.temporal.*;
import java.time.zone.*;
import java.nio.charset.*;
import java.util.concurrent.atomic.*;
import java.util.concurrent.locks.*;
import java.util.random.*;
import java.math.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.function.*;
import java.util.prefs.*;
import java.util.regex.*;
import java.util.stream.*;

void print(Object o) { System.out.print(o); }
void println(Object o) { System.out.println(o); }
void printf(String s, Object... args) { System.out.printf(s, args); }

Iterable<Integer> range(int startInclusive, int endExclusive) {
    return IntStream.range(startInclusive, endExclusive)::iterator;
}

[timber mirage]

that's the startup script

you can see it in resources

@vivid cliff

[vivid cliff]

then y do u need the id to fetch that

[timber mirage]

In the future, we might have several of them

currently

[timber mirage]

the only option are either use this one or use none

but in the future, we might add other startup script

[timber mirage]

@compact holly @sterile crest can one of you review my PR please ?

[sterile crest]

i don't want to put my name on these tests sorry, they're really badly done and I want to stay away from them

i can help introduce new tests but the current set doesn't adhere to even entry level guidelines - so maybe somebody else could give you the tick

[timber mirage]

I just want to validate this pr because it's for adding a test on something that wasn't tested before
so I really need this test in case a problem happens

[timber mirage]

Improving that will be another PR about redoing it

[timber mirage]

@compact holly ?

[compact holly]

you can just mention me on github ^^

we have done a lot of discussions on discord, that will be lost forever

design decisions, why we done some things certain way..

everything project related should happen on github, ideally

[vivid cliff]

@timber mirage how to setup config

theres no template

[timber mirage]

the default config works

just follow what the readme says

[vivid cliff]

u need to update that

[timber mirage]

what's the problem ?

Have you managed to make it work ?

[vivid cliff]

U don't need to manually compile wrapper noe

[vivid cliff]

It was running I haven't tested it yet

[timber mirage]

wdym

[vivid cliff]

@timber mirage

[timber mirage]

show the exact request please

or curl request

[vivid cliff]

POST /eval?startupScriptId=0 HTTP/1.1
Content-Length: 27
Content-Type: application/json
Host: localhost:8080
User-Agent: HTTPie

System.out.println("good");

the error is 404

ohh its /jshell/eval

[timber mirage]

it's /jshell/eval

[vivid cliff]

yea

[timber mirage]

yea

and

=0 is also wrong

[vivid cliff]

what should i be?

[timber mirage]

this parameter is optional

so if you don't have any starup script

don't use it

and if you want one

use CUSTOM_DEFAULT

[vivid cliff]

@timber mirage

[timber mirage]

what's the error in insomnia ?

[vivid cliff]

{
  "timestamp": "2024-05-22T00:57:30.879+00:00",
  "status": 409,
  "error": "Conflict",
  "message": "Session may have been forcibly destroyed.",
  "path": "/jshell/eval"
}

is the docker not starting or smth?

[timber mirage]

Do you get any other logs ?

[vivid cliff]

no

[timber mirage]

is the container alive ?

for the session

[vivid cliff]

no containers :c

[timber mirage]

what just happened

I never saw that

can you try again ?

@vivid cliff ?

[vivid cliff]

im trying again but same thing

the container is not starting ig

[timber mirage]

windows?

[timber mirage]

If you have such error, then you should have other logs, are you sure you have nothing ?

[vivid cliff]

linux

[vivid cliff]

that is all yes

[timber mirage]

you indeed have another error

[vivid cliff]

wer

[timber mirage]

InternalServerErrorException: Status 500

[timber mirage]

how did you build the image ?

[vivid cliff]

i did not i just did ./gradlew clean bootRun

[timber mirage]

show the images

[vivid cliff]

now it should work

sry for wasting ur time

[timber mirage]

yea

and next time

please follow the readme 🙂

[vivid cliff]

@timber mirage how do u build the image on the server tho?

u sould push the image on docker hub no?

[timber mirage]

automatically built

it works the same as tjbot

the watchtower detects when something is pushed on master branch

and run jib

[timber mirage]

alright

push it

I'll add doc to it later

[vivid cliff]

i'll at least copy paste what u have in readme

[timber mirage]

the readme of jshellapi isn't up to date anymore

[vivid cliff]

@timber mirage y not just autowrie the field?

[timber mirage]

better avoid black magic when i can

tho a better approach would probably be to use the constrcuctor

[vivid cliff]

nah its fine

code looks better

[vivid cliff]

the code is getting cancerous 💀

[vivid cliff]

i'll push now so u can see my abomination

ur spotless made it even worse

[vivid cliff]

@timber mirage i see ur spotless sucks, may i interest u in a new spotless config

[vivid cliff]

@timber mirage whats the difference between eval and single-eval

[timber mirage]

If it's transformming into that, it's not worth it

[vivid cliff]

huh? so u dont want openapi spec?

[timber mirage]

Not me, it was wazei 😦

[timber mirage]

the session is way shorter

[timber mirage]

I want it, but it's not worth it to do it manually if it is such a mess

[vivid cliff]

i mean u gotta write the text manually tho

it generates schema and stuff automatically

            @io.swagger.v3.oas.annotations.parameters.RequestBody(
                            content = {
                                @Content(
                                        mediaType = "text/plain",
                                        examples = {
                                            @ExampleObject(
                                                    name = "Hello world example",
                                                    value =
                                                            "System.out.println(\"Hello,"
                                                                    + " World!\");"),
                                            @ExampleObject(
                                                    name =
                                                            "Hello world example with startup"
                                                                    + " script",
                                                    value = "println(\"Hello, World!\");")
                                        })
                            })

this is kinda excessive we can skip this if u want

also half the problem is bad spotless

[vivid cliff]

https://github.com/Together-Java/JShellPlaygroundBackend/pull/28 @timber mirage

[wispy badger]

https://tenor.com/view/peaking-peak-tyroney-whos-here-gif-14225120

[compact holly]

sakamura, it's been so long I seen you on here

one of the OG devs

[timber mirage]

👀

[timber mirage]

Happy to see JShell ? 🙂
What are you doing here ? Want to contribute ?

[compact holly]

in fact, I think sakamura created our together java github org

his billing email is still there

if we get into some legal trouble, they gonna knock on sakamura's doors

[timber mirage]

👀

[wispy badger]

I've been peeking in and out mostly

It sucks that I have so little time for OS projects

[vivid cliff]

@timber mirage so r u just gonna ghost my prs

[timber mirage]

Please wait a bit

I worked a lot for jshell recently

Now I am trying to work less on it

[vivid cliff]

@timber mirage can u at least merge my spotless pr so i can continnue on other stuff

basically writing openapi docs which u'll later reject

[timber mirage]

Ah

there is a bug with jshell

[vivid cliff]

looks better now

[vivid cliff]

@timber mirage i did it master

pls accept

[vivid cliff]

master?

[timber mirage]

Tomorrow

[vivid cliff]

we can either server this using github pages or just the hosted url (i dont think thats a good idea)

its accessible at <base>/swagger-ui/index.html (changeable)

and the docs should be at /v3/api-docs and /v3/api-docs.yaml (also changeable)

[timber mirage]

@vivid cliff I don't like this

[vivid cliff]

marko wanted same config

[timber mirage]

same for this

[timber mirage]

-_-

[vivid cliff]

my config was better

check the commit with my config

[timber mirage]

Guess I have no choice then

btw

did you run it @vivid cliff ?

in this pr

[vivid cliff]

run what?

[timber mirage]

the backend

also did you change anything else ?

outside of the formatting

[vivid cliff]

no

[vivid cliff]

no

[timber mirage]

wait

it was merged ?

[vivid cliff]

spotless one? yes

[timber mirage]

alright

[vivid cliff]

review the openapi spec

[timber mirage]

mhh

I really don't like the size of the doc in the controller

is there any way to fix it ?

maybe extract an interface ?

so the interface only has method signatures + annotations

[vivid cliff]

• annotations?

is that possible

[timber mirage]

maybe

try to find a way

adding auto openapi generation is not worth this code polution imo

[timber mirage]

added this as a review

I won't accept it unless a solution is found

[vivid cliff]

:c

[timber mirage]

Added a PR with bug fixes

https://github.com/Together-Java/JShellPlaygroundBackend/pull/29

@compact holly any solution https://github.com/Together-Java/JShellPlaygroundBackend/issues/30 ?

[timber mirage]

I also really need to get this merged, because jshell is currently broken right now 😦

[compact holly]

solution to what?

that testing the api requires docker?

[timber mirage]

yes

can the github pipeline handles this ?

[compact holly]

yeah, it's just a ubuntu machine/vm

it should

if it has docker installed

of course, that would be a functional test

[timber mirage]

@compact holly you haven't replied to https://github.com/Together-Java/TJ-Bot/issues/1119

[timber mirage]

@compact holly ↑ ?
Also can you review and merge the PR to master please ?

[compact holly]

history is slightly fucked

someone merged the github PR, and instructions clearly say do not merge the PR

so now histories are not the same

master is not the clone of develop

but it's fixable, slight divergence

[timber mirage]

#1245350506570711050 message

Perfect use case for jshell

[timber mirage]

@compact holly can you do it please https://github.com/Together-Java/JShellPlaygroundBackend/pull/32

[timber mirage]

@compact holly is there a way so jshellbackend logs are reported on a specific channel on discord, like tjbot ?

because it's starting to become annoying, to not know when a problem happens and to have to log in the vps in order to see what is going on

[timber mirage]

↑ @compact holly

[compact holly]

there is, by creating a webhook

and configuring github

to push relevant info

https://dev.to/inezabonte/how-to-get-github-notifications-on-your-discord-server-2j9o

here is a guide I guess

you should have permissions for everything

if not, let me know

[timber mirage]

@vivid cliff any news for swagger?

[vivid cliff]

Umm no i wasn't working on it

I'll try sm stuff after I get back home

[timber mirage]

@sterile crest if you wanted to look at it, it seems that jshell backend is broken again

[sterile crest]

did you grab logs?

[timber mirage]

no

[timber mirage]

I mean

[timber mirage]

Can't you get the logs yourself ?

[sterile crest]

i don't know, do you pipe the logs to a file or anything?

i can't remember much but if you want, we can do an investigation into it later today?

[vivid cliff]

get back to work

[timber mirage]

alright

[sterile crest]

i am

i sent you the updated spec :3

[timber mirage]

Btw any advancement about openapi?

[sterile crest]

Just generate it yourself 😁

My child is busy with my stuff

[compact holly]

wazei doing exploitative child labour in 2024

[vivid cliff]

no

[sterile crest]

i'm available now if you want to investigate?

[timber mirage]

Alright

i'm comming

@sterile crest #810093858690957332

[compact holly]

any updates, does it work properly now?

[timber mirage]

no

we didn't fix anything

[compact holly]

interesting

do we know what is wrong at least?

someone pulled all the logs?

[timber mirage]

no

[timber mirage]

yes, doesn't help

[compact holly]

how is it possible that logs do not help

do we need better logs for the project

[timber mirage]

we added more logs and we hope that it helps

[compact holly]

or it's not a code problem, but environment problem where shit is crashing and burning down?

okay

[timber mirage]

no idea

but we can't reproduce it

and it happens randomly

sometime it happens after 3 days, sometime after 3 weeks

[compact holly]

and there is nothing abnormal caught by logging?

yeah, seemes like logging should be improved a lot

and maybe put it on debug level

so we get a detailed preview for the next week

[sterile crest]

I had an idea but before we blindly implement my theory we added additional logging

[sterile crest]

ala doesn't want verbose logging because it can get repetitive

but me.. i wanted entry/exit logs, all params, null checks etc logged even if the logging is duplicated

easier to trace the logs to code but you know, personal preferences get in the way

but the 2 theories we have are:

1. the scheduler is crashing

[compact holly]

it doesn't have to be duplicated

[sterile crest]

2. the sdk call is hanging - our scheduler is single threaded so a hanging sdk call pauses the entire thing

[compact holly]

just detailed enough on debug/trace level

[sterile crest]

duplicated as in:

[compact holly]

it's justvery weird that shit is crashing, and we have no clue where, why, when..

[sterile crest]

void doSomethng(obj) {
  if(obj == null) logNull();
  anotherThing(obj);
}

void anotherThing(obj) {
 if(obj == null) logNull();
}```

even tho in anotherThing it won't ever be null due to the call chain - i like it to do it 😄

[timber mirage]

The thing is that wazei tried to stress test it without result

wait

did we enable debug logging ?

[sterile crest]

go check

[timber mirage]

I don't think so

[sterile crest]

it could also be a timing with the API - for example, it could be using a persistent connection and after no usage, the connection dies

[timber mirage]

how do we enable it already ?

[sterile crest]

could be anything

[timber mirage]

it doesn't use a db

[compact holly]

it shouldn't be anything :"D

we aboslutely must have at least an idea where the problem might lie

[timber mirage]

It's either the docker api blocking endlessly for no reason or the scheduler crashing or something we have no idea

[sterile crest]

we have debug enabled in spring

is that enough ?

[timber mirage]

I don't think so

[sterile crest]

not with our shitty logs 😄

shitty at a personal standard

[compact holly]

well yeah, that's what i'm trying to say

this is not the first time

logs should point us to the exact call that went wrong, and why it went wrong

[sterile crest]

if I can log the way I wanna log, the logs will show the entire flow of code - statement/block level

[compact holly]

or at least to a method/class

or some smallest unit of the system

in the worst case

[compact holly]

that's a bit aggro, but yeah, that's what trace level should do

[sterile crest]

yeah it's exactly what trace is for

[compact holly]

and not just the flow, but the values

[sterile crest]

^^

[timber mirage]

We currently have enough logs to at least know what the cause is

[compact holly]

at least on the main system

like the main flow

[timber mirage]

once it will happen

[compact holly]

since everything probably goes trough the main flow, there is one way to spawn the container, there is one api endopont, there is one..

[sterile crest]

it's never too late to switch to my AWS solution which is still online and hasn't gone down since I made it :3

[compact holly]

faulty shit has to travel at least a bit trough the main flow

so it will be caught

[sterile crest]

which is like < 200 LoC

consider it as a free public jshell api :3

backed by "Wazei Ltd"

i technically have employees too, so it's maintained :3

and we offer customer support :3

it's unrestricted jshell ❤️

[compact holly]

we could do something like that, just together :p

[quartz shell]

Yeah depending on the usages it could end up being free

[bright lark]

Hello folks

How to get started with this?

[timber mirage]

You could start building it in local

check the repository

Mention me if you have any question

[timber mirage]

@bright lark so did you plan to try it today or ?

[bright lark]

i was busy in something

now im checking

this is it right ? https://github.com/Together-Java/JShellPlaygroundBackend

nevermind

[timber mirage]

Yep

[bright lark]

it takes time to build

https://tenor.com/view/waiting-spongebob-coffee-sad-gif-16525719

[timber mirage]

For the docker image ?

well yes

don't worry, you only need to build it once

[bright lark]

the problem with gradle is that it takes time to build

unlike maven

[timber mirage]

wdym?

for the docker image ?

or something else ?

[bright lark]

nah

the whole repo

[timber mirage]

ah

[bright lark]

i got it building

[timber mirage]

probably because it's the first time ?

ah

perfect

[bright lark]

just after the clone

ye

140 mb

[timber mirage]

as explained in the readme, there are several ways to run it

[timber mirage]

?

[bright lark]

it's downloading gradle

the ide is doing tht probably

alright it's done

[timber mirage]

right ?

how are you running it ?

[bright lark]

im trying to use IDE helpers

on top there is Main

it's for the rest api app

the guide says "Go in Run → Run... → Edit Configurations → create a configuration"

what kind of config ?

[timber mirage]

Java
But, you are trying to run the wrapper itself, why ?

[bright lark]

im reading the readme

[timber mirage]

Yes

[bright lark]

ah

i only need the api ?

[timber mirage]

Yes

[bright lark]

gotit

[timber mirage]

running the wrapper is only needed if you want to work on the wrapper

also

you might notice that there are two ways to run the api

in local, on inside docker

(same as tj bot)

[bright lark]

do i have to use docker ?

[timber mirage]

you have to yes

for the wrapper

I mean

[bright lark]

hmm so i need both

[timber mirage]

look at the first part of the readme

Prerequisites
Java 21
Docker

[bright lark]

ye ok

[bright lark]

i don't know why i dnt have same interface as the one in readme

[timber mirage]

which part of the readme are you following ?

[bright lark]

the 1st

there is a screenshot from intellij

[timber mirage]

ah it's a gradle run

so create a gradle config

my bad

[bright lark]

got it

this should be highlighted in readme

[timber mirage]

So you are trying to run the wrapper itself at the end ?

[bright lark]

i just want to deploy the whole project

that means both apps

[timber mirage]

right

[bright lark]

and see in action how it works

[timber mirage]

then try all three options

tell me if you managed to run the wrapper

[bright lark]

ok

[timber mirage]

So ?

[bright lark]

that means im running the app as container right ?

[timber mirage]

no

you are running the wrapper in local here

press enter in the console

[bright lark]

ah u're right

[timber mirage]

after pressing enter

it should say OK

[bright lark]

ye

[timber mirage]

then enter eval

then press enter

then enter 1

then press enter

[bright lark]

it fails

[timber mirage]

then enter System.out.println("hello");

[timber mirage]

ah I forgot what the order was

you can find examples of execution in the tests

                eval
                1
                System.out.println("Hello world!")""", """
                OK
                0
                OK
                1
                VALID
                ADDITION
                1
                System.out.println("Hello world!")


                false
                Hello world!\\n

like this one

[bright lark]

eval
1
2
OK
1
VALID
ADDITION
1
2
2

false

[timber mirage]

yea

so you first enter a blank line

then eval

then 1

then 2

then 1

[bright lark]

let me try again

Task :JShellWrapper:run
OK
0

Exception in thread "main" java.lang.RuntimeException: No such command ""

this is ugly

it should continue running, this exception should be catched

[timber mirage]

you didn't enter a blank line

seems like it

[bright lark]

i did

[timber mirage]

???

[bright lark]

look i did "enter"

Task :JShellWrapper:run
OK
0

then i pressed "enter" again

[timber mirage]

right

[bright lark]

and got the exception

[timber mirage]

why 0 tho

blank line

eval

1

2

that's the example

[bright lark]

i didn't write 0

[timber mirage]

🤔

ah

yes

it's the length of the startupt script my bad

you enter a blank line

then eval

then 1

[bright lark]

what does that mean?

[timber mirage]

you will understand that later

[bright lark]

eval
1
3
OK
1
VALID
ADDITION
1
3
3

false

this seems wrong though

anyway do i run spring boot locally too ?

cuz i don't like using docker image if it's not hosted

[timber mirage]

blank line
eval
1
System.out.println("hello");

Enter this

[timber mirage]

read the readme

[timber mirage]

wdym

[bright lark]

i mean in a docker repo

[timber mirage]

wdym

[bright lark]

docker images has to be pushed on a repo

like we do for code

[timber mirage]

no they don't need to be pushed to a repo

[bright lark]

ok

if why i must use docker locally ?

while i can run both locally without docker

[timber mirage]

and they won't be if you run it in local, otherwise what's the point of running it in local

[timber mirage]

You need at least docker for the wrapper even if you run the api without docker

as explained in the readme

the api spawns containers where each user can run code

[stray wren]

Alathreon's result

Snippets

Snippet 23, VALID

2+2```
jshell> `4`
## System out
[Nothing]

[timber mirage]

A container was spawned just to run this code

if you try to use this command

it will spawn a container for you

[bright lark]

ah

[timber mirage]

so by design, it needs to spawn containers for each user

[bright lark]

it would be helpful if there is a diagram

[timber mirage]

like ?

[bright lark]

i mean to show architecture and flow

how the whole system works

i get what u mean now

but for somebody else, it would be not ez to get tht alone

[timber mirage]

alright

[bright lark]

let me try it out

[bright lark]

nice one

there is no swagger ui ?

[timber mirage]

nope

@vivid cliff worked to add one but there is no swagger for now

[bright lark]

im gonna add it

[timber mirage]

The problem

Is that I don't want openapi annotations to take too much place on the controller

And I advise you to talk with Taz since he already started @bright lark

[bright lark]

ok, im gonna keep it in local

just to try

[vivid cliff]

just merge it bro

[timber mirage]

Look at this, there are twice as many annotations than code -_-
@bright lark

[bright lark]

we can avoid those