#dev-general

1 messages ยท Page 444 of 1

cinder flare
#

i mean you kinda lose the benefit of a treeset if you don't maintain it

#

if you get a stream of similarities and pipe them into a treeset just to immediately take the top out, that's not a good use

#

you should look into sorting algorithms for that

quiet depot
#

the optimization I was looking for was here

#

that method is nlogn

cinder flare
#

that just sorts numbers?

quiet depot
#

yep

cinder flare
#

oh yeah that's the best you're likely gonna get

#

cause quicksort is n log n

quiet depot
#

do you remember the o(n) plan I had though?

cinder flare
#

not at all

quiet depot
#

the number we're sorting by is a double (0., no +1 value)

#

we can convert that safely to a long without losing data

#

the idea from there was to then find a 64 bit array implementation and use the natural ordering of the array to sort the numbers, and their corresponding byte array

#

i.e. the double we convert to a long is used as an index in the array

#

and at that index in the array, is the byte array

#

problem was with this idea though is memory usage

old wyvern
#

True

cinder flare
#

so wait hold on, we're sorting doubles attached to byte arrays?

quiet depot
#

ye

#

the double represents the similarity between the inputted byte array, and the stored byte array

oblique heath
#

that's a big array

quiet depot
#

yep

#

that's why I didn't go that route

oblique heath
#

what's the range of numbers that those longs could end up being

cinder flare
#

i think just quicksorting would be a lot faster than iterating through like 6 million array slots every single time

quiet depot
#

ivan I don't believe it's possible to predict a range like that

#

Long.MAX_VALUE - 0

oblique heath
#

oh so all the way to the max value

quiet depot
#

yep

#

it's really not a feasible solution

oblique heath
#

yeah

#

what exactly do these values represent anyways

quiet depot
#

the doubles or the byte arrays?

oblique heath
#

well both i guess

#

but i'm more curious about the doubles

quiet depot
#

double = similarity between input byte array & stored byte array

old wyvern
quiet depot
#

basically we're got a big list of byte arrays and an input

#

end goal is to sort the list against the input

#

via the double, which is generated by comparing the input and each individual byte array

old wyvern
#

If an o(n) is possible that should be considerably faster

cinder flare
oblique heath
#

i see

quiet depot
oblique heath
#

is the goal to sort the entire list of byte arrays every time

quiet depot
#

I just need to figure out a way to implement it that doesn't require a max size 64 bit array

cinder flare
#

we're talking like all the way to Long.MAX_VALUE here

oblique heath
#

as in, do you need to return the entire collection of byte arrays every time

#

or just the top x

quiet depot
#

yes

cinder flare
#

i mean honestly, it doesn't sound like you need a whole lot of precision here

old wyvern
#

Im confused about what you mean rn star

quiet depot
#

I want to return the entire collection every time

oblique heath
#

hmm

old wyvern
#

Quick sort visits all elements as well

cinder flare
# old wyvern It does

no like, quicksort iterates through the list which would be like a few thousand each time or something. in his idea we'd convert the doubles to long and you'd have to iterate to like the 6 billionth slot to find the first one

oblique heath
#

ideally you'd have some magical way to prepare the collection in advance, before you actually get to sorting it

quiet depot
#

wym prepare?

#

prepare it in what way

cinder flare
#

i mean that's the problem ivan, we need to generate a new list of similarity score per query

oblique heath
#

like leave less work for the sort to perform

quiet depot
#

fyi, not that I think it's relevant, but these byte arrays are ascii strings

cinder flare
#

pig, how close in decimal values are these numbers?

#

like do you have any examples

quiet depot
#

no

cinder flare
#

uym

#

hmm

quiet depot
#

it should vary hugely

oblique heath
#

how big are the byte arrays

#

out of curiosity

quiet depot
#

they're ascii strings

cinder flare
#

okay so nothing within like, 0.001 of eachother?

quiet depot
#

so "average string length"

#

definitely not star

old wyvern
#

Btw piggy docdex is backed by mongo right?

oblique heath
#

what's an average string length

#

xd

quiet depot
#

yes yugi

cinder flare
#

okay so who needs all this precision

quiet depot
#

exactly ivan

oblique heath
#

ah ๐Ÿฅฒ

cinder flare
#

just convert to int and truncate all but like 4 decimal places

#

then do your array thing

oblique heath
#

that's still a huge array

quiet depot
#

no it's not?

oblique heath
#

it's not?

cinder flare
#

yeah but the first entry will be like 300 or something

#

it doesn't matter the size of the array

#

you just go till you find the first item

quiet depot
#

it's 10000 elements if it's truncated to 4 decimals

oblique heath
#

oh wait i thought the elements were gonna be the 4 decimal places

#

dont mind me

old wyvern
#

Is there no way to define the query to mongo itself with the sorting in place?

cinder flare
#

well no because he has to calculate the similarity in the program yugi

#

mongo just returns a list of the possible strings it could be

old wyvern
#

If he can get the score in the query itself he could get mongo to do it I assume

#

Wouldnt that be faster?

cinder flare
#

i mean probably not

oblique heath
#

i like star's idea of truncating the doubles

quiet depot
#

i'm not following yugi

cinder flare
#

mongo probably has to quicksort too

oblique heath
old wyvern
#

You use lev distance for the Algorithm right?

quiet depot
#

no

oblique heath
#

so for the bottom 50(?) percentiles maybe you don't demand 100% accuracy with the sorting

old wyvern
#

Oh you changed it?

quiet depot
#

I use a modified jaro winkler implementation

old wyvern
#

Hmm

cinder flare
#

okay maybe I don't understand what's happening here, but in my understanding whenever someone types a query, it generates a long map of closeness scores to the original paired with the result

quiet depot
#

yes

oblique heath
#

piggy how important is it for the result to be perfectly in order

cinder flare
#

those closeness scores look like 0.1, 0.22, 0.2345, 0.44, 0.567, 0.7889, 0.999

oblique heath
#

without any chance of one result being slightly off

quiet depot
#

ivan it's probably not that important

oblique heath
#

then truncating the doubles sounds like a really good solution

quiet depot
#

I should've mentioned earlier that these "optimizations" are purely my curiosity

#

there's not actually any performance issues in docdex

oblique heath
#

because that's what truncating the doubles would do

quiet depot
#

it performs extremely well for what it does

cinder flare
#

imagine if it performed better!

quiet depot
#

and due to these optimizations being purely my curiosity

#

I want them to be perfect

quiet depot
#

truncating digits, or returning a top k, while in practice, probably wouldn't effect the end results at all

#

it changes the certainty of getting the expected result from 100%, to 99%

oblique heath
#

so it's worth considering ๐Ÿ˜‰

cinder flare
#

i don't think it does

oblique heath
#

it will change the certainty of the result

cinder flare
#

if you assured me that they were always a little bit apart, dropping the part that's irrelevant wouldn't change anything

oblique heath
#

well you can't be certain that it's irrelevant

cinder flare
#

i mean what's the minimum distance?

#

whatever the smallest decimal that double holds is?

quiet depot
#

yes

oblique heath
#

very smol

cinder flare
#

what's the difference between like potato and potatk

oblique heath
#

yes

quiet depot
#

that's gonna take a minute to figure out

oblique heath
#

piggy why exactly do you need the entire collection to be returned

cinder flare
#

so according to this online calculator, it's .93

#

which is not very close

oblique heath
#

this is all in the name of getting the target of the search right

#

why care about the bottom 50% of all possible words it could have been

quiet depot
#

star what online calculator?

cinder flare
#

oh damn ivan has a good point, you can get a sorting algorithm that only returns the top like 5 resulsts

oblique heath
#

yeah

#

if you pick a constant and only get that many results

#

it becomes o(n)

cinder flare
#

yeah that'd make it real fast

#

cause it just has to traverse the list and pick the best ones then sort like, 5 things

old wyvern
#

Piggy maybe building a bst out of it could be faster

oblique heath
#

ye

quiet depot
#

I've considered doing that before, but my main issue is duplicate names

#

there are names that refer to the exact same thing, and this is unavoidable

oblique heath
#

why is that an issue here

cinder flare
#

BST's have to be unique

oblique heath
#

you're still using the double value either way

#

right?

cinder flare
#

nah I think you should listen to Ivan

#

that only sorting the top like 5 is a lot faster

quiet depot
#

bst?

cinder flare
#

binary search tree

oblique heath
#

binary sexy tree

cinder flare
#

what yugi suggested

quiet depot
#

it won't be a top 5, more like a top 100

oblique heath
#

that'

quiet depot
#

to factor in duplicate names

cinder flare
#

still

#

100 out of 10000

oblique heath
#

that's still small in the scheme of things right

quiet depot
#

yes

#

it searches through hundreds of thousands of names

cinder flare
#

i mean that's what google does for pete sake

oblique heath
#

yeah

cinder flare
#

it only shows you the first page of resulsts

oblique heath
#

so you'd maintain a sorted structure of the top 100 results

old wyvern
#

Each node in the bst could be a linked list

#

Like how we prevent hash collisions

#

Hmm

cinder flare
#

well there's no reason to use a BST

#

the only problem we're solving here is how to get the smallest (or biggest) of a list of doubles

#

he already has a fuzzy algorithm

old wyvern
#

The bst is not for a fuzzy search

cinder flare
#

I think it is definitely not worth the overhead to create a full BST for every single search query

#

this is why sorting algorithms exist

old wyvern
#

Ofc yea the space complexity increases

#

This sint inplace like quicksort

cinder flare
#

what are you even talking about then lol

old wyvern
#

Time complexity

#

He wanted to lower that

cinder flare
#

it takes a lot of time to create a full BST

old wyvern
#

Doesnt need to be

#

It can be optimized

#

We dont need a avl

cinder flare
#

by sorting? lmao

old wyvern
#

No star

#

Keeping track of leaf nodrs

#

Altho the issue would be when a collision hits now that I think of it

#

Hmm

cinder flare
#

I think a sorting algorithm that stops early is faster than inserting every item into the BST then getting the sorted list from that

oblique heath
#

yes

old wyvern
#

Thats all you technically need

oblique heath
#

sorting algorithm that stops after a constant # of iterations is o(n), inserting everything into a tree is nlogn

cinder flare
#

well you need the top 100 entries from that BST

#

so you need to insert every element into the BST

#

then get the top 100 entries from it

tawny peak
#

can someone link meh funnys and piggys website..?

old wyvern
#

Forgot cubes

oblique heath
#

you guys know what's probably much more relevant to real world performance

#

caching

cinder flare
oblique heath
#

piggy do you cache results

tawny peak
#

thanks

quiet depot
#

it's really not feasible to cache here due to the nature of the data and the amount of results

cinder flare
#

you'd need a hell of a lot of disk space to cache the results of every query

oblique heath
#

well you don't need to cache every query

#

but what you can do is keep track of which ones are the most popular

tawny peak
#

is there clips website too..?

cinder flare
#

I really don't think DocDex has a most popular list

oblique heath
#

it should

#

tbh

#

๐Ÿ˜‰

cinder flare
#

I think DocDex is quite random

quiet depot
#

there's just too much variation

cinder flare
#

just whatever someone happens to be talking about

#

it's not like google where they can easily predict like 90% of stuff

quiet depot
#

even if docdex was in thousands of servers, I highly doubt there'd be queries that get executed more often than others

oblique heath
#

well docdex has a bunch of stuff that's not standard libs right?

quiet depot
#

yes

oblique heath
#

i imagine those are searched less often on average

#

than standard libs

#

though i guess best way to find out is by actually tracking searches

#

which i want to see the results for

cinder flare
#

it's not feasible to store the entirety of the standard libs

oblique heath
#

๐Ÿ˜

#

well you don't need to store it

#

but imagine this

#

actually nvm

#

i just imagined it

#

and it didnt work out

#

i was thinking you can maintain a second, smaller array of just standard library stuff and match across that

#

but that doesn't really make sense i don't think

cinder flare
#

it really doesn't

oblique heath
#

how about concurrency

quiet depot
#

it's already multithreaded

oblique heath
#

alright

#

how about gpu acceleration

quiet depot
#

lol

oblique heath
#

you're doing the same thing a bunch of times after all

#

it's a legit proposal

quiet depot
#

this is ran on a dedi

cinder flare
#

yeah dude Ivan the only thing that we can improve upon is the sorting algorithm lmao

oblique heath
#

well the algorithm that is run to get the double from the char array can probably be ran on a gpu right

#

if it's an intel dedi it's gotta have some semblance of an integrated card on it

quiet depot
#

ยฏ_(ใƒ„)_/ยฏ

oblique heath
#

idk either

cinder flare
#

ah I see you can use a heap to do a one pass top-k results search of something

oblique heath
#

makes sense, you can fill it up and remove the root every time you insert

frail glade
#

Does it really need to go faster than it already goes?

cinder flare
#

so you put like, the first 100 elements into the heap, then each one after you add it, heapify, then remove the highest (or smallest) item from the top of the heap

quiet depot
#

no waffle

oblique heath
quiet depot
#

here's how I'm starting the issue I'm writing

cinder flare
quiet depot
#

There's not actually anything wrong with the performance of sorting at the moment, but science isn't about why, it's about WHY NOT, and for science I say we research the fuck out of this and find the best way to sort.

oblique heath
#

piggy is the code you use for all this OS

quiet depot
#

yes

oblique heath
#

nice

frail glade
#

Here's my project for the day (not designed for mobile yet)

https://glare.dev/gateways/

I noticed when I work on plugins my clients are all using different forks of Spigot and whatnot so I had to keep going out to all the websites to make sure I was on the latest. I designed a gateway so you can get all the latest versions from 1 page.

quiet depot
#

i don't do closed source unless it's a commission

#

star you're a portal fan

#

did you even notice the reference smh

#

you were meant to get all excited

cinder flare
#

oh yes I see it now, apologies I am extremely tired

quiet depot
#

nice glare

cinder flare
#

also that is one slick website glare

distant sun
quiet depot
#

wat

#

no

#

"science isn't about why, it's about why not"

#

that was the reference

cinder flare
#

the cave johnson quote yeah

distant sun
#

Eh

quiet depot
#

ok

#

Here are the ideas that came up in the discord convo today:

  • Only sort for top k results
  • Truncate doubles to lower number decimals for use in an O(n) array-abusing sort
#

what else did we talk about?

oblique heath
#

gpu acceleration

quiet depot
#

yep

#

that it?

cinder flare
#

lemons

oblique heath
#

that's all i remember

cinder flare
#

also glare's website

quiet depot
#

okie

cinder flare
quiet depot
#

*that I got my engineers to invent

oblique heath
#

i thnk between the two, truncating doubles is more worth it

#

however

quiet depot
oblique heath
#

actually i'll just put what i was about to say in that

old wyvern
#

Piggy

quiet depot
#

hi

cinder flare
#

what if we truncate then only sort for the top k elements

old wyvern
#

Does the score have to be a double? Would floats work?

quiet depot
#

if we truncate then it's an o(n) sort

#

yugi floats would have the same effect as truncating

#

so to answer that I don't know

#

because I don't know if truncating would work either

#

I'll have to test it in the future

#

so star there's no benefit doing both

#

a top k sort is O(n + k) iirc

#

whereas the truncate is just o(n)

cinder flare
#

man how has no one ever thought of the array thing

#

maybe it sucks

quiet depot
#

wym

#

it's not something I just pulled out of my ass

#

it's a real thing

old wyvern
#

Test

cinder flare
#

like it is so smart to just like, multiply by a thousand or something then add to array, then traverse the array until you find a not null thing

old wyvern
#

Back

#

Router decided to restart itself

quiet depot
#

counting sorts rely on this array technique

old wyvern
#

Piggy

quiet depot
#

hi

old wyvern
#

Radix sort

quiet depot
#

yep

#

i know of it

cinder flare
#

oh that is literally what we're doing

quiet depot
#

still requires a 64 bit array without truncating though

#

yeah star

#

they're very similar

old wyvern
#

Radix sort doesnt care about the array size

#

Thats counting sort

quiet depot
#

although both counting & radix sort do extra steps that I wasn't planning on doing

cinder flare
#

wow i am but a virgin to the world of sorting algorithms lmao

quiet depot
#

mr college man

#

i don't even have cs education kid

cinder flare
#

mr freshman lmao

#

i just took a DS&A class and we learned 4 sorting algorithms, barely lmao

old wyvern
#

You can use counting in radix without having a huge array

cinder flare
#

but yeah you just multiply by 100 (or maybe 100 if you wanna get frisky) and truncate any decimal place, then add to array and iterate until you get 100 results

quiet depot
#

what's the iteration at the end for

#

that sounds like an extra useless step

#

yugi i'll look into radix sort

cinder flare
#

that's how you get the values?

old wyvern
#

Alrighty

quiet depot
#

just return the array star

cinder flare
#

it'd just be an array with like a million empty slots

#

you need to traverse that array until you find non-null slots

quiet depot
#

you'd need to traverse the entire array, as the indices aren't guaranteed to be in consistent increments

#

unless the 100 is the top k bit

oblique heath
#

badda bing badda boom

cinder flare
#

wdym

#

you just go until you get 100 non-null entries

quiet depot
#

ah right

#

sorry misunderstood

cinder flare
#

if you want the smallest number, you go like 1->50 (that's 1), then 50->200(that's 2), etc.

#

just go backwards from the top if you want the biggest number

#

so yeah that'd be O(n + kish)

quiet depot
#

and that's where big o annoys me

#

because if instead of just getting the first k non null elements, and we went through the entire array

#

the complexity would just be O(n)

#

deceives you

cinder flare
#

well yeah it'd be 2n

#

but you drop the 2

quiet depot
#

yep

#

that's why it's deceiving

cinder flare
#

yeah that's why big oh is only one part of the picture

#

you also gotta look at what its doing and stuff

old wyvern
#

Big oh only represents variation with input size

frail glade
#

Small update, I just added in flex grid crap so it should look decent on mobile now.

cinder flare
#

dang Glare wish you were in my web dev final project group

#

those guys could not figure out flex grids

frail glade
#

You probably don't want me in your web dev stuff. I used to do web and now I can't do it to save my life.

old wyvern
#

Wait, are you part of the purpur team as well?

frail glade
#

Nah

#

But some of my clients use Purpur.

old wyvern
#

Ahh

quiet depot
#

flex is ez

cinder flare
#

damn imagine being cool enough to have clients

frail glade
#

I used to have Waterfall and Bungee but neither of them have actual icons that I could manage as SVGs.

cinder flare
#

meanwhile my mind is blown by radix sort lmao

old wyvern
#

lol

frail glade
#

When I tried to convert Waterfall's logo to SVG, it did not turn out pretty.

quiet depot
#

oh

#

pro tip for anyone who uses flex in the future

#

bootstrap-grid is your friend

old wyvern
#

๐Ÿ˜ฎ

#

I need to work on bootstrap more

quiet depot
#

bootstrap is modularised and you can use its grid system without using the entirety of bootstrap

old wyvern
#

I suck at designing stuff

quiet depot
#

the grid uses flex

#

it's very intuitive to use

frail glade
#

Yeah that's what I went in and added real quick. I was already using Bootstrap but I just added in rows / cols

quiet depot
#

would highly recommend

cinder flare
#

yeah i used bootstrap grid for my personal portfolio I had to make

quiet depot
#

I use bootstrap on every site I make

frail glade
#

I forgot how sexy that site is

#

It's raw too

quiet depot
#

mine or helpchat?

frail glade
#

Yours

#

Well, both.

old wyvern
#

Elm-Bootstrap ๐Ÿคค

quiet depot
#

yeah I don't use scss or any of that fancy shit

frail glade
#

Meaning you don't use any SSG.

quiet depot
#

just raw css/js/html

frail glade
#

Yup

quiet depot
#

best way to do it

frail glade
#

If I had the patience for it, I would.

cinder flare
#

ssg?

frail glade
#

But I work with GraphQL and some other data too which is kind annoying as raw.

#

Static Site Generators

#

Gatsby, Nuxt, Etc.

oblique heath
#

wait piggy, why are you calculating the entire sorted result array all at once again?

frail glade
#

That gateway page uses Gatsby (react).

cinder flare
#

oh I see

quiet depot
#

ivan that terminology isn't quite correct

frail glade
#

Come on Piggy smh

quiet depot
#

wat

#

what site is that

frail glade
#

Solid on best practices though

#
web.dev

Get the web's modern capabilities on your own sites and apps with useful guidance and analysis from web.dev.

quiet depot
#

ivan there's a stream of byte arrays and an input byte array

#

end result is a sorted stream of byte arrays against the input

frail glade
#

Apparently it doesn't like your image sizes and extra css

oblique heath
#

right

quiet depot
#

bruh

old wyvern
#

๐Ÿ˜ฌ

quiet depot
#

those images are optimized fine

oblique heath
#

so piggy when would you use the second best result from that query

old wyvern
#

Oh wait

#

I have an idea for a portfolio page

quiet depot
#

every time that method is run ivan

old wyvern
#

๐Ÿ‘€

quiet depot
#

that method is only run if there isn't an exact match

frail glade
#

You ever going to add more content to the barn?

quiet depot
#

d;spigot playah

ruby craterBOT
#
public interface Player
extends HumanEntity, PluginMessageRecipient, Conversable, OfflinePlayer```
Player has 18 super interfaces, 89 methods, and  4 extensions.
Description:

Represents a player, connected or not

old wyvern
#

This could go very well, or insanely stupid

quiet depot
#

waffle I wrote a new guide recently

frail glade
#

Ooo?

oblique heath
#

right but i mean if there is no exact match, don't you just want the next best

quiet depot
#

rewrote my hotswapping guide

#

with more info this time

oblique heath
#

why get the entire sorted list

quiet depot
#

because the limit is configurable

#

d;spigot 8 playah

ruby craterBOT
#
public interface Player
extends HumanEntity, PluginMessageRecipient, Conversable, OfflinePlayer```
Player has 18 super interfaces, 89 methods, and  4 extensions.
Description:

Represents a player, connected or not

oblique heath
#

so you're getting the top 8 results

#

or 7 or whatnot

quiet depot
#

8

oblique heath
#

does the ui only display 8 results at a time

#

or how does that work

quiet depot
#

discord can display a max of 10

oblique heath
#

is it possible to get to the 11th

quiet depot
#

there's a hard cap of 10 in the api

distant sun
#

Cant you have 20 reactions on a message?

oblique heath
#

so if you're only getting the 10 best, why find the entire sorted list

quiet depot
#

only 10 number emojis

distant sun
#

Ah, right

distant sun
#

๐Ÿ”Ÿ

quiet depot
#

isn't that why you guys suggested top k?

#

ok 11 then gaby

oblique heath
#

i was under the impression you didn't already truncate results

distant sun
#

Time to create emojis up to 18

quiet depot
#

I don't

oblique heath
#

but now that i know, you definitely want to do top k

#

well you just said you have a limit of 10

quiet depot
#

yeah sorry

#

the sort doesn't truncate them

oblique heath
#

yee

quiet depot
#

the truncate happens when they're returned in the api

#

because I don't want hundreds of mongo calls

oblique heath
#

yeah so that seems like a very clear choice

#

wdym

quiet depot
#

the only thing stored in docdex's memory is names

#

those names are used to query mongo to get the objects

distant sun
#

๐ŸฅญDB

quiet depot
#

that's one red looking mango

oblique heath
#

wait

#

so lets take your sorted, finished array of size 100k

#

would it matter at all if it was size 50 instead of size 100k

#

as far as mongo calls or whatever go

#

actually maybe that was a bad example

quiet depot
#

yeah I don't understand what you're asking

oblique heath
#

anyways you're only calling the sort function one time either way

#

but by truncating it you're making it o(n) instead of nlogn

#

and you're not losing anything since you truncate it later on anyways

quiet depot
#

technically it's still possible to get o(n) rn

#

the truncate just makes it more resource friendly ram wise

oblique heath
#

is this about the arrays again xd

#

or is there another way

quiet depot
#

arrays are essential for an o(n)

#

the truncate is part of the array idea

#

they're not separate

#

the truncate is merely a solution to the memory issue with arrays

oblique heath
#

wait i think you're misunderstanding me

quiet depot
#

@frail glade this is what I usually use for site performance

oblique heath
#

when i said truncate i meant in the context of truncating the result array

quiet depot
#

i'll have a look at that web.dev thing

oblique heath
#

as in, i return the top 50 results instead of all 100k

quiet depot
#

oh

oblique heath
#

i didn't mean truncate as for the indexes

old wyvern
#

Ivan, to get the top 50 results you still need to sort to a point

frail glade
#

web dev uses lighthouse internally

oblique heath
#

you do but you can consider it constant time instead of logn time (times n)

quiet depot
#

then why r the results different

fleet bayBOT
#

:x2: You have to be in a voice channel to use this command.

frail glade
#

ยฏ_(ใƒ„)_/ยฏ

old wyvern
#

Music used /

old wyvern
#

Ok wait

#

Let me see if I can understand what you mean

#

So given an array of 100k elements of random values, how do you select the top 50 values

oblique heath
#

okay so

quiet depot
#

erm

#

waffle

oblique heath
#

first let's make it simpler and say top 1 value

quiet depot
#

I literally get different results to you in web.dev

oblique heath
#

what do you do? iterate through the whole list

#

that's o(n)

quiet depot
frail glade
#

That's really weird. Haha.

oblique heath
#

now what if we want the top 2

old wyvern
#

Top 1 is possible with O(n) but not the same with top k

cinder flare
oblique heath
#

no

quiet depot
#

check your ping to the ip waffle

oblique heath
#

it is possible if the k is small

old wyvern
#

You now have another list to sort

quiet depot
#

maybe you've just got a bad route

old wyvern
#

Hmm

oblique heath
#

you do but the k is small and not determined by n

frail glade
#

Pretty sure web.dev uses their own routes.

oblique heath
#

so it's still n time

#

if the list you were managing was a ratio of n, like say you requested half the total list

#

then you could say insertion into your result array is log(n/2)

#

or whatever

#

but k is a small constant like 50, independent of the number of elements in the actual array

old wyvern
#

Ok so like a replacing insertion sort on a size k array while iterating iver the main array?

quiet depot
#

I gotta go

oblique heath
#

sure

quiet depot
#

but this is all valuable info

#

so pls add it to the issue

oblique heath
#

really any insertion works here

old wyvern
#

That could work

oblique heath
#

because as long as the max size of the accumulator is small it's o(1)

#

even if insertion was technically o(n^3) it'd really be o(1) lol

cinder flare
#

i mean would the heap sort idea be too bad?

#

iterate through the array and keep the best items and drop the newest one

oblique heath
#

star this is your heap sort all over again lol

#

but the heap sort is not the important bit

#

the important bit is that you're only holding on to a small amount of results

#

that's the saving grace

cinder flare
#

yeah that's the whole heap sort idea

oblique heath
#

ye

cinder flare
#

you only keep the 50 best resulsts or whatever

oblique heath
#

exactly

cinder flare
#

exactly

oblique heath
#

e x a c t l y

cinder flare
#

wow we're geniuses

oblique heath
#

i agree

#

we do good work

cinder flare
#

where's that github issues lmao

oblique heath
cinder flare
#

my master thesis is complete

oblique heath
#

very masterful

cinder flare
#

awe nice he actually put lemons in the ideas list

#

that makes me happy

lunar cypress
cinder flare
#

oh hey johnny

#

have you always been that color?

lunar cypress
#

I have not

cinder flare
#

okay it felt new

#

i like it

#

goes well with your pfp

lunar cypress
#

I don't

cinder flare
#

oh

#

i am only 3k xp from being a tier V bois

oblique heath
#

start spamming

cinder flare
#

genius

#

instead im going to go to bed

#

because I'm really tired

oblique heath
#

adios

#

i'll probably do the same

lunar cypress
#

just woke up and preparing for another day of mental torture trying to fix my dual boot

hot hull
#

Tis 11am

quiet depot
#

me an u both johnny

cinder flare
#

better than taking a calc test

quiet depot
#

although i'm not going through dual boot torture

#

more so partition torture

#

still linux tho

#

so it counts

cinder flare
#

what's wrong with your partitiosn?

quiet depot
#

I have to merge a second disk to the root partition

cinder flare
#

sounds like a RAID 0 problem

lunar cypress
quiet depot
#

it's hard to setup star

#

with this environment

cinder flare
#

raid 0? on linux?

#

surely not

quiet depot
#

I have to do it live

#

I can't boot from another drive

cinder flare
#

oh true migrating the data might be a little funky

#

got another compooter?

quiet depot
#

it's remote

cinder flare
#

or use a live boot linux

#

oh

oblique heath
#

you know what you may be able to do

cinder flare
#

you can still live boot a usb remote

oblique heath
#

shrink your current partition that you're using as much as you can

#

and use the empty space on the drive to make a new partition

quiet depot
#

that's the approach I'm taking

oblique heath
#

then raid 0 that with the other drive

quiet depot
#

it's so fucking hard though

oblique heath
#

ah fair enough

#

xd

quiet depot
#

well I wasn't gonna do raid 0 I was gonna use lvm

cinder flare
#

lvm?

quiet depot
#

logical volume manager

oblique heath
#

tbh lvm is probably a better idea as far as data recovery

#

since i doubt it stripes the partitions

#

unlike raid 0 ๐Ÿฅฒ

quiet depot
#

I managed to shrink the partition to 50gb and boot from it

#

but the problem is fdisk doesn't know that the partition is smaller

#

like df -h shows 50gb

#

but fdisk doesn't

#

and it's confusing the fuck out of me

cinder flare
#

oh LVM and RAID 0 are the same

quiet depot
#

no they're not

quiet depot
#

they have completely different purposes

cinder flare
#

though LVM seems better apparently

quiet depot
#

you can achieve the same thing with both but lvm is very different to raid 0

cinder flare
#

both ways to combine drives

quiet depot
#

that's not lvm's focus

oblique heath
#

can lvm use all the space on two partitions of different sizes

quiet depot
#

it's just possible to do with lvm

#

lvm is like an abstract partition manager

#

that's how I think of it

cinder flare
#

see that's when you just use Btrfs

oblique heath
#

binary tree fs

cinder flare
#

i personally like to pronounce btrfs as butter-fs

old wyvern
#

Lol

quiet depot
#

the difficulty comes with changing the root partition while it's live

#

it's by far the hardest thing I've ever had to do

#

literally every guide doesn't work

cinder flare
#

just boot from a logical usb drive piggy

quiet depot
#

well I managed to do it eventually

oblique heath
#

oh actually

quiet depot
#

like the partition size is 50gb now

#

the problem is fdisk doesn't recognise it for some reason

oblique heath
#

can't you use chroot to boot

#

or something

quiet depot
#

tried chrooting

cinder flare
#

yes from a usb drive ivan

quiet depot
#

tried pivoting root

cinder flare
#

the very obvious answer here is usb boot, then you can modify your drives however you want

quiet depot
#

how do you usb boot on a remote server?

cinder flare
#

logical usb?

#

are you not running a hypervisor

quiet depot
#

wat

#

it's a dedi

cinder flare
#

i don't know what that means

#

i know what a VPS is

quiet depot
#

I don't know what you mean

oblique heath
#

he just has a box

#

that he can ssh into

distant sun
#

We can just ask the provider to plug in an USB

cinder flare
#

as in, you have an entire cmoputer just for you?

quiet depot
#

yes

cinder flare
#

yeah just ask them to plugin your usb

quiet depot
#

lol

#

they're useless shits

cinder flare
#

not joking

#

they will

oblique heath
#

actually there might be something about that

distant sun
#

They will not*

cinder flare
#

also why would you want a dedi vs a VPS

oblique heath
#

what if you make a small partition and burn an iso onto it or somethin

cinder flare
#

sounds a lot less flexible

quiet depot
#

what

#

star u got some funky ideas

cinder flare
#

wdym

oblique heath
#

dedis have less abstraction

#

they're better as far as performance goes

cinder flare
#

you can dynamically resize a VPS

#

you're stuck with whatever dedi you get

#

hardly ivan

#

hypervisors are so good nowadays

quiet depot
#

with a vps though you're sharing resources

cinder flare
#

you can do GPU passthrough and everything

#

who cares?

#

you still get your allotted portion

oblique heath
#

not always

#

i know this firsthand

cinder flare
#

i mean I guess if you get a poor host

oblique heath
#

i mean that too but i'm talking purely based on the software used

#

you know how you can limit the resources that docker containers use

cinder flare
#

you wouldn't use a docker container

#

you'd use a hypervisor

#

which is very good at its job

oblique heath
#

well same principles

cinder flare
#

nope

oblique heath
#

as far as sharing resources go?

cinder flare
#

no, hypervisors have total control

#

docker can only work within the operating system

#

a hypervisor can literally only show you 4GB of ram, a limited number of CPU cores, a specific graphics card etc

#

it's not possible for you to use more than that

#

in fact, you often specifically use a hypervisor on your own dedicated machines to ensure that your programs don't resource creep on eachother

oblique heath
#

imagine this on the hardware level

#

you have two programs, one using a single core on the cpu

#

no matter who has control over them or what's going on, they're both left up to the will of the cpu itself as to what happens with the l1, l2, l3 cache

#

so you'll end up with a bunch of stuff overwriting caches and losing you out performance

#

without you being able to tell why

cinder flare
#

that's why server-grade cpus have multiple caches

oblique heath
#

you can't guarantee that each vm will get their own cache

cinder flare
#

i mean you can if you limit the vms

#

you can perfectly split a system if you have the right stuff for it

#

but I mean, I guess its all about what you value more

#

flexibility or a lil bit of performance

quiet depot
#

figured out the fdisk issue

oblique heath
#

you also have to keep in mind that multithreading will always be a thing

quiet depot
#

the script I used only resizes the filesystem

#

the partition doesn't actually change

#

so back to square 1

oblique heath
#

as in, 2 vcores for every physical core

cinder flare
#

okay?

oblique heath
#

so if one vps gets one vcpu and the other gets the other

cinder flare
#

you don't split down vcpus

#

you split down real ones

#

problem solved

oblique heath
#

nobody does that

cinder flare
#

unless you're getting a bottom-tier vps, you get at least 2 vcpus

oblique heath
#

yeah and you're not guaranteed that they're on the same core

cinder flare
#

they could be if you're smart about it

#

hypervisors can do that easy

oblique heath
#

imagine you have 2 cores 4 vcpus

#

sorry 4 cores 8 vcpus

#

wait

cinder flare
#

yeah doesn't matter, it knows which vcpus belong to which core

#

so you just assign in pairs

#

though I guess my experience is more from a company hosting its own servers than a hosting company

#

so maybe vps's are worse than I thought lol

oblique heath
#

discord broken

cinder flare
#

yeah for me too lol

#

okay tired now

jovial warren
#

@quiet depot what sorting method is used atm?

cinder flare
#

default java sort the entire list

jovial warren
#

So timsort

cinder flare
#

sure

#

moral of the story, we sort less things

jovial warren
#

Not sure what's gonna beat timsort

cinder flare
#

not sorting the whole list?

jovial warren
#

Maybe

cinder flare
#

just keeping track of the top k results in a heap as you traverse it is quite efficient

jovial warren
#

Yeah

cinder flare
#

okay glad we caught you up lmao

lunar cypress
#

@old wyvern does this look right to you?

old wyvern
#

Yea, looks about right

quiet depot
#

hold up

#

why separate home & root partition?

old wyvern
#

Hes following that guide from the manjaro forums i followed as well I think

quiet depot
#

@lunar cypress unless you can think of a benefit off the bat for separating them, then I don't think it's worth it

#

from years of using linux on the server & desktop, I've never seen the point or needed to separate them

lunar cypress
#

mhhm

#

maybe I should try the "install alongside windows option" again

#

my current problem is that windows simply isn't recognised anymore

old wyvern
#

Did you have you use rescue earlier or were the partitions not actually deleted btw?

lunar cypress
#

Idk which partition I deleted. But I deleted one on my hdd

#

however I can still mount the other one and that's where all the data is

old wyvern
#

ah ok

lunar cypress
#

man

#

I don't want to reinstall and end up with nothing working at all

#

right now at least I can get into manjaro

#

But I'm really stumped as to why I can't get into Windows anymore

old wyvern
#

Manually adding the menu entry to the grub config might work

lunar cypress
#

well what am I supposed to add

#

Windows isn't in /boot/efi/EFI

old wyvern
#

yea thats a different boot partition

#

Gimme a minute

#

Ill need to look this up

#

oh john, this might be the same issue as what you found earlier on forums

lunar cypress
#

no, this is not the same

old wyvern
#

ah alrighty

lunar cypress
#

I have gpt everywhere

old wyvern
#

Oh btw you also disabled fast boot from windows power settings right?

lunar cypress
#

fast startup you mean? yeah

#

god maybe i did install windows in legacy mode?

#

Idk how to find out

old wyvern
#

uh does the windows boot partition show up as an efi partition in gparted?

jovial warren
#

not necessary though

#

I don't do it because I think it's a bit stupid personally, but to each their own

#

it's pointless having two separate partitions because they're both on the same drive

#

imo

lunar cypress
#

I believe

jovial warren
#

no

#

well, depends

#

for EFI systems, a boot partition is unnecessary, and so it's on the same partition as the main drive

old wyvern
#

UEFI requires it to be in the efi partition afaik

jovial warren
#

or at least, it is for me

old wyvern
#

A Microsoft Reserved Partition (MSR) is a partition of a data storage device, which is created to reserve a portion of disk space for possible subsequent use by a Windows operating system installed on a separate partition. No meaningful data is stored within the MSR; though from the MSR, chunks may be taken for the creation of new partitions, wh...

lunar cypress
#

Well there was no efi partition prior to my manjaro installation... i think

old wyvern
#

๐Ÿ˜ฌ

lunar cypress
#

If there was, I actually deleted it

unkempt tangle
#

Did anyone tried to run a fabric server with Cardboard for Bukkit support?

lunar cypress
#

Ok the question is how tf do I get out of this then

#

If I reinstall windows, will it be smart enough to put itself in the efi partition?

old wyvern
old wyvern
#

startup repair from your windows installation usb* should probably fix it

lunar cypress
#

Well the guide there wants me to format the efi partition

lunar cypress
#

Yes

old wyvern
#

I think it might be safer with startup repair

lunar cypress
#

Whats that

prisma wave
old wyvern
#

The windows startup repair tool, it should show up in the options if you try to boot from the windows installtion drive you made earlier

lunar cypress
#

And btw the dual boot guide used a new partition as well?

old wyvern
#

Yes

lunar cypress
#

As efi partition

old wyvern
#

That was an extra boot partition to isolate manjaro in

#

You should already have had one for windows

lunar cypress
#

Ah

lunar cypress
prisma wave
#

And that didn't setup grub properly? Weird

lunar cypress
#

And I accidentally installed that in legacy mode. And then for some ungodly reason I thought doing it manually might work

unkempt tangle
#

Senpai ;c

prisma wave
#

bonjour

lunar cypress
#

Start up repair doesn't do anything @old wyvern

#

Maybe I can create a second efi partition

#

For windows again

#

find the volume letter which belongs to the installed Windows OS

Do they mean C?

old wyvern
#

Yes

lunar cypress
#

Got re labelled to D for me lol but alright

unkempt tangle
#

Anybody?

lunar cypress
#

failure when attempting to copy boot files
๐Ÿฅฒ

old wyvern
#

๐Ÿฅฒ

unkempt tangle
#

Did anyone tried to run a fabric server with Cardboard for Bukkit support?

tawny peak
#

how do I un-link my github..?

old wyvern
#

From discord?

#

User Settings -> Connections -> Press the "X" on top right of the github card there

spring canyon
#

I don't know about him, but maybe from the Discord bot here

#

I changed my username and now I can't get the reward for starring the Github repo thonking

old wyvern
#

Ah, I have no idea. @pallid gale ?

tawny peak
old wyvern
#

I havent seen anyone unlink github from barry yet

tawny peak
pale shell
#

I hate people on spigot to be honest, Instead of asking for help in a support discord, they put a 1 star rating straight away ๐Ÿ˜

onyx loom
#

๐Ÿฅฒ

pale shell
#

And now I have a 1 star on my plugin fucking great, that's gonna put people off ๐Ÿ˜

onyx loom
#

take tips from frosty, frozenjoin only 5 star reviews!!!

prisma wave
#

Delete the plugin and re-upload

#

All bad reviews : gone

onyx loom
#

lol

pale shell
#

Legit, instead of joining the discord to get help he just puts a bad review straight away ๐Ÿ˜

old wyvern
#

Hire a hitman

onyx loom
#

u can get the review taken off individually by a staff member if its not an actual review

pale shell
#

he put "Plugin is badly coded, Doesn't even function correctly. Still has the same problems as the free version." there was no messaging me asking for support to fix anything first just straight up bad review.

#

Oh looking at his review history he 1 star reviews everyone.

onyx loom
#

well then code properly in the first place and u wont have this situation :kekw:

pale shell
#

bm coded it in the first place

prisma wave
#

classic

hot hull
#

ofc it was trash if BM made it ๐Ÿคก

jovial warren
prisma wave
#

๐Ÿ™„

lunar cypress
#

I want to smash something

prisma wave
#

Understandable

onyx loom
#

me too

lunar cypress
#

Not even my fucking live windows usb stick works anymore

onyx loom
#

๐Ÿ™

prisma wave
#

Huh

onyx loom
#

time to switch to linux!!!

hot hull
#

Please smash my head in

lunar cypress
#

Idk what to do anymore

jovial warren
#

Linux time

lunar cypress
#

Yeah... great

prisma wave
#

Maybe just delete everything and start from scratch if you can

jovial warren
#

also, this is why you don't mess with GParted without first understanding the consequences your actions may have if you fuck up, and without at least somewhat knowing what you're doing

lunar cypress
#

Thank you bardy please lecture me on more things

jovial warren
#

not insulting your intelligence, just saying that maybe messing with partitions wasn't the best idea

onyx loom
#

๐Ÿ˜‚

jovial warren
lunar cypress
#

Yeah thanks for telling me mate, wouldn't have that figured out myself

jovial warren
#

lol

prisma wave
#

So

lunar cypress
#

"I messed up"
"Maybe don't mess up"
Groundbreaking

prisma wave
#

What are we looking at

jovial warren
#

not what I said

#

just said you maybe should've read the big board at the beginning of the tunnel that said "warning! danger ahead!"

lunar cypress
#

Yeah I don't have the patience for this nonsense now

jovial warren
#

anyway

#

let's not dwell on past actions

#

can you boot GParted at least?

lunar cypress
#

Woops

#

Wrong msg

jovial warren
#

also, there's a tool in Arch that can convert MBR partition tables to GPT ones without data loss

#

I did it with my drive

old wyvern
#

His is already gpt

hot hull
#

Just use windows ๐Ÿคก

lunar cypress
#

I don't have any mbr partition tables

jovial warren
#

ah right

prisma wave
#

starting from scratch usually works well. If you've got anything important you could just boot into Linux usb and move it onto another drive

jovial warren
#

so what's your issue? Windows isn't being picked up?

#

is it picked up in the UEFI boot menu?

lunar cypress
#

Nope

old wyvern
#

os prober isnt finding windows boot manager

#

He doesnt have the efi boot partition for windows

jovial warren
#

if it's not in the UEFI boot menu then it's likely corrupted

#

just the boot manager that is

#

Windows has tools to repair the boot manager iirc, you'd just need some way to get into command prompt

#

which is much easier said than done, trust me I've tried it before

lunar cypress
#

Yeah I'm too far in, I'll copy my previous user files to my hard drive and start from scratch

prisma wave
#

Starting from scratch is definitely the easiest way of fixing stuff when you mess things up

#

reinstall windows first

jovial warren
#

bootrec /fixboot I think might do something

#

or bootrec /rebuildbcd, or both

lunar cypress
#

What would you recommend for cleaning up all the partitions? Do I just delete them all?

old wyvern
#

format and delete

prisma wave
#

Probably don't even need to format

jovial warren
#

delete them, move them around, etc.

prisma wave
#

But yea just delete the partitions and let the reinstall overwrite stuff

jovial warren
#

GParted will auto-format when it's deleting partitions

old wyvern
#

Ah ok

jovial warren
#

just give Windows something to work with and it should be happy

quiet depot
#

@pale shell was it legeurnic or something?

#

heโ€™s a troll

lunar cypress
#

Just need to be careful to not install windows in legacy mode this time

hot hull
#

Did someone say legacy ๐Ÿ˜  ๐Ÿ”ซ

lunar cypress
#

Which I still don't really understand

#

Whatever

jovial warren
#

but if you give it a GPT drive, it'll pick GPT

lunar cypress
#

The thing is, it keeps enabling csm in bios

onyx loom
pale shell
hot hull
onyx loom
#

lmao

#

good song ๐Ÿ‘

tough matrix
#

anyone know of any plugins that uh allow you to type like [&aB&bu&ci&dl&ed&fe&1r] on a single line of a sign

#

?

oblique heath
#

i would assume that the character limit that players can type on a sign is a client limitation

#

or do you mean chat color

tough matrix
#

theres some plugin out there where you can right click the sign

#

and then type like /line1 (color coded crap here)

#

i used to see it all over lol

oblique heath
prisma wave
tough matrix
#

ye i was tryna find it myself looking up anything i could and all the plugins i can find do is just allow u to have hex or whatnot

unkempt tangle
#

Whats yarn?

prisma wave
#

alternative to npm afaik

placid rose
#

how u type in a code window?

oblique heath
#

```
code here
```

placid rose
#
2222
prisma wave
placid rose
#

ok thx

oblique heath
#

stop launching your children across the room goddammit

onyx loom
#

๐Ÿ˜ฑ

prisma wave
#

fixed it ๐Ÿ˜Ž

boreal needle
#

can you now launch your children

prisma wave
#

yes

#

thank god

boreal needle
#

i dont want to think about what launching children with bash is

lunar cypress
#

Obscure british sports

placid rose
#

how do i get tiers?

oblique heath
#

obscure? that's like the #1 british sport

boreal needle
#

mostly by talking

#

and =daily

placid rose
#

ok

#

what r u writing in intellij?

onyx loom
boreal needle
#

coursework

#

big sad

placid rose
#

are you recommending buying an ultimate ver?

#

btw tier 1 pog

onyx loom
#

u can get ultimate for free if ur a student

lunar cypress
#

I wouldn't buy ultimate

placid rose
#

really?how

lunar cypress
#

If I weren't a student I would at least make my company pay for it

hot hull
#

Ultimate is only good if you're making projects which use a DB constantly