without needing crazy statistics processing
#fast2.v
hushed plank
fresh remnant
Pinning the alpine base to a specific version would also keep things more stable.
Even with that, though... if the underlying docker version that runs the containers changes, there could still be some variations, but at least you would have locked down as much as you could.
shrewd jackal
This is the current setup:
FROM alpine:3.21.0
RUN apk add build-base
RUN apk add --no-cache git make rsync sqlite-dev
RUN git clone https://github.com/vlang/v /data/v --depth 1
RUN cd /data/v && make
RUN /data/v/v /data/v/cmd/tools/oldv.v -o /data/oldv
COPY ./fast.v /data/fast.v
RUN /data/v/v /data/fast.v -prod -o /data/fast
ENTRYPOINT ["/data/fast", "-watch"]
also true; hopefully they are even less significant than the hardware variations
shrewd jackal
alpine is already pinned, but apk will still get newer versions if available
(current latest version is 3.23.0)
shrewd jackal
<https://www.reddit.com/r/docker/comments/gx24rw/how_to_add_the_alpine_linux_bui...
and this is where we go back to caching
fresh remnant
<nod> - you would have to pull in the individual pieces you need, with pinned versions, instead of just build-base... unless that's versioned itself?
hushed plank
yes
with the difference that the first stage, is not a cache
it is a full blown image with its own name and everything as I understand it
so you can back it up etc
and presumably only need to update it say 2 times per year
i.e. just like you have FROM alpine:3.21.0
which does not change
you can have FROM my_fast_measuring_base:v1
which also will not change, until you want to, and not when some cache decides to expire
(or to get purged by something)
naming things is powerful
it may sound stupid, but it is not
(at least it was not for me; others may think it is obvious 🤷🏻♂️ )
docker create could be perhaps used to create that my_fast_measuring_base
from the existing container
hm no; it may be docker commit
docker commit --message "my first image" 883f28bad31b my_fast_measuring_base:v1
this worked for an experimental container of mine
#0 19:23:15 ^ master ~/v>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my_fast_measuring_base v1 9bc3083e5fe6 4 seconds ago 78.1MB
and I think that now I can FROM my_fast_measuring_base:v1 and make a new container from that
and inside have just the commands to clone V etc
without having to run apk etc
and because it is its own image, afaik it has no history and dependencies to previous things/layers
so they can be safely purged without affecting it
yep:
#0 19:23:19 ^ master ~/v>docker container rm 883f28bad31b
883f28bad31b
#0 19:26:28 ^ master ~/v>
#0 19:26:32 ^ master ~/v>docker container rm 883f28bad31b
Error response from daemon: No such container: 883f28bad31b
I've removed the original container, but the new image still remains
imho make the image from the current container, then have a separate Dockerfile.my_fast_measuring_base, to be able to recreate the base image once very few months:
FROM alpine:3.21.0
RUN apk add build-base
RUN apk add --no-cache git make rsync sqlite-dev
then when that is run, make a new image my_fast_measuring_base:v2 from that container
and have:
FROM my_fast_measuring_base:v1
RUN git clone https://github.com/vlang/v /data/v --depth 1
RUN cd /data/v && make
RUN /data/v/v /data/v/cmd/tools/oldv.v -o /data/oldv
COPY ./fast.v /data/fast.v
RUN /data/v/v /data/fast.v -prod -o /data/fast
ENTRYPOINT ["/data/fast", "-watch"]
that runs only the relevant parts (while keeping a stable system with git/gcc as the base image)
hushed plank
then when that is run, make a new image `my_fast_measuring_base:v2` from *that* ...
and when that happens, just bump FROM my_fast_measuring_base:v1 to FROM my_fast_measuring_base:v2
and if everything is ok, docker rmi my_fast_measuring_base:v1 if needed
how big is the image currently?
hopefully less than 1GB?
docker container ls --all can list all the containers (including the stopped ones), and hopefully the one from this morning/yesterday is there too, that does not have SSL issue
so we can cut the base image from it
(if it was not purged yet)
fresh remnant
FROM my_fast_measuring_base:v1
RUN git clone https://github.com/vlang/v /data/v --depth 1 &&
cd /data/v && make &&
/data/v/v /data/v/cmd/tools/oldv.v -o /data/oldv
COPY ./fast.v /data/fast.v
RUN /data/v/v -prod /data/fast.v
ENTRYPOINT ["/data/fast", "-watch"]
will make your final image have 2 fewer layers... basically, every docker "command" (run, copy, etc.) is a new layer.
shrewd jackal
added comptime (hopefully), re-queued the last 5 commits
@hushed plank you'll need to update your charts server if you want to see them on the website once they're done
hushed plank
thanks, will do asap
shrewd jackal
its done with reprocessing
hushed plank
using the latest data.sqlite
seems to work fine
I'll push the changes
done
btw, I downloaded the data.sqlite at ~70KB/s, but it may be some limit on my end, my internet seems spotty today
shrewd jackal
the server has full gigabit. could be your internet or the V webserver renxSHRUG
server is updated
hushed plank
I should probably also add a dark theme 🤔 ... that screenshot looks really bright compared to everything else that I usually use
although no, https://github.com/Max-Github/FireFoxInvertColors is sufficient for me 🤷🏻♂️
shrewd jackal
new stuff should be live now
™
ah loadavg is still a bit high due to the recompile
soon™
hushed plank
thank you
shrewd jackal
you'll also need to update the server
hushed plank
I will asap
hushed plank
tl_stmts etc will not vary at all for any given commit; imho they do not need _stddev columns in the db
i.e. they are similar to csize_mean and vfiles_mean
I'll filter them in the sql query that the server uses
hushed plank
the server is updated now too: https://github.com/spytheman/oldv_charts/commit/dab4f01bcfb937ab74ebdc3d4c6038eb680ed202
shrewd jackal
i.e. they are similar to `csize_mean` and `vfiles_mean`
Those have stddev as well
It’s for catching cases when for some cases it isn’t the same
It’s already reduced from normal stats (which have min and max)
shrewd jackal
server is updated
hushed plank
thank you
@distant garnet the new statement metrics are shown here too: https://fast2.vlang.io/v_self.7.html#tl_stmts
shrewd jackal
are main and non-vlib supposed to be equal
hushed plank
for V itself, yes - the main program with the 16 stmts is in cmd/v/v.v, which is outside of vlib/
distant garnet
Thank you
shrewd jackal
oh the channel died
no the processor died
it's still running, just not doing anything...
ah loadavg says its busy
but why...
aha, fast is in a busy loop somehow?
at least its using 50% cpu
I don't understand, nothing changed
and I have the commit history
I updated to the latest V as well, might be related
I think sqlite performance drastically decresed
manual restart and it seems to be alive again
fun fact, only two commits were tested using a different GCC version:
sqlite> select cc_version, count(*) from commits group by cc_version;
cc (Alpine 13.2.1_git20240309) 13.2.1 20240309|2
cc (Alpine 14.2.0) 14.2.0|21291
24 different fast iterations processed all the commits:
sqlite> select nodename, count(*) from commits group by nodename;
02b3d8a05de8|2
0582ec1f5116|524
0b7475743fed|3
1655317d70ed|396
2bb9eaa9e95b|15
2d303c57c299|131
59cc5e21bcf4|1971
6448f29c024d|6816
6624f4280e8a|7129
85e823e003f6|8
85ee4d386b1e|31
863b1ffc1561|7
8af2e92c43f5|771
8fdf714a20de|4
9413e6189f7f|237
9e6c601d6b10|1
a3229561f60e|4
b3b71c05661d|9
bb231b36e1d4|5
cd8f55987d8a|1249
d9c891f95cb2|2
e97439752b4b|1658
e9aeba0f7ebc|314
ee41e5f85367|1
fdac10513a0e|5
(nodename is using the hostname, which is the docker container id)
hushed plank
is it stable now?
shrewd jackal
it takes ~22s to update the commits database for some reason, but I think it was always this slow
it's working fine now
shrewd jackal
the webserver may be slow to respond currently. Cloudflare is having issues on the way to my server (~50x slower)
shrewd jackal
note: should only affect the first request (which may timeout after 15-30s with a 520/525 error), after that it should be fast
shrewd jackal
going to discontinue fast processing soon (likely march 1st)
I will publish the full database and source code in here, if anyone wants to continue on their own hardware.
it's just ridiculous how resource expensive it is for what it is
300mb of ram for a webserver (and still leaking)
fast container leaking thousands of zombie processes (alpine issue), >10gb of ram usage (and still leaking)
idle furnace
Yep
@leo
I'm confused of why two different database files
❯ which database is expected for the docker setup
● fast.sqlite — that's what's hardcoded in docker-compose.yml:
environment:
FAST_DB: /data/volume/fast.sqlite
All three services (fast, web, regret) point to the same file. The HKuZYml.sqlite is only used locally because of your FAST_DB env var
override.
shrewd jackal
its the same db, the filename was randomized on upload to discord
idle furnace
Thanks
@shrewd jackal what about this aspect
❯ Given that do I need to get a clouldflare account?
● No. The domain is already on Cloudflare and the tunnel is already configured. You just need the TUNNEL_TOKEN that was generated when the
tunnel was created — that goes in env/tunnel.env.
shrewd jackal
I am not giving you access to my domain, no.