#ToolKitty

1 messages · Page 3 of 1

dark bay
#

The script editor will need to be integrated into that somehow too.

#

Maybe script editor belongs in the level preview/runner

#

Little more planning I think

#

Cake time 😆

1743112902602.jpg
ocean notch
#

your earned it

dark bay
#

I wonder if anyone has implemented a ui for the solid primitives filesystem AI... if so, I can pinch it and save on making my own file manager.

#

I forgot to reconnect our existing basic file management system to the new file system, it's still connected to the old one.

dark bay
#

Integrating automerge definitely captured some attention:

Screenshot_20250329_081146_YT_Studio.jpg
#

For multiplayer we'll need to use a different technology:
WebRTC + GGRS

#

GGRS stands for good game roll back system. It is similar to automerge, but focuses on rewinding time to apply updates at the correct time, and is online only. No offline deferred sync. But the performance is good.

dark bay
ocean notch
ocean notch
#

web archive project lol

dark bay
#

Hehehe... just in case the page goes offline?
That's a good idea.

#

I can sort of read it still.

#

I just added a nice rustlang based library, we can use it via wasm if we need to.

dark bay
#

I've just transmitted the document url (ID) when a peer connects to another peer to save on sending the document id in the search params before connecting. It should lower some friction when establishing a connection.

#

It seems to be working most the time when I make a connection, but sometimes not at all. There must be a bug lurking in there waiting to be found.

#

its working well at the moment between the mobile and tv, Its gonna be a hard bug to find.

#

better sort out some of the other bugs 1st :P... been on automerge for way too long.

#

there is that non-square texture atlas frame not rendering correctly in the level map bug.

dark bay
#

Hmm... the Rust of GGRS version is better maintained than the JS versions. Might be a good idea just to make a JS wrapper around the Rust version compiled to wasm.

#

Then again. Doesn't look that hard. We can probably implement GGRS ourselves for better integration.

dark bay
#

first update in a long time.... multi-cell blocks added (u choose the number of cells wide and tall in tile's meta data). Handy for mario pipe, so u don't need to break it into 4 pieces.

zenith void
elfin lintel
#

2022 MESSAGES?!?

#

bruh

#

man the amonunt of messages is the amount of years christ was born

dark bay
#

Cheers 🍻.... virtual beers on me 🍺 🍺 🍺 🍺 🍺 🍺 🍺 🍺 🍺 🍺 🍺

dark bay
#

Good thing we are mobile 1st:

image0.jpg
#

A wanna create a proper automerge documents projection through a solid Store which handles the conversion between json object and class on a per field basis (via second level projection through TypeSchema).

But....

What are all the different ways a store can be updated?, so that ppl can just pretend they are using a real solid store.

#

I suppose I could just support a subset of the SetStoreFunction and add to it as I go.

#

That special path syntax will be tricky

#

... unless I just copy paste the Store code from Solidjs source, and tweak it a bit.

#

Or.... multiple level proxy. I can pretend I am feeding a regular object to createStore but it will be a proxy that manipulates something else underneath. And then I can just use solidjs stores as is without modification.

dark bay
#

Firing up solid playground to test the idea 💡

dark bay
#

oh right.. I just need to make a proxy object that fine-grain serializes/de-serializes to/from json sitting on top of an automerge doc, then feed that proxy object to a createStore, then I will have a real proper EcsComponent automerge projection. Without any dirty tricks.

dark bay
#

basically you feed an automerge json doc thing, then it will provide u with a value of type A represented via your type schema

#

when you mutate type A, it will automatically mutate the automerge json doc underneath

#

when the automerge json doc underneath changes it will bubble up and mutate A

#

wrap the returned A in createStore, then you now have a store that can talk to an automerge doc and support types of values that are not usually supported in an automerge doc. (classes, HTMLImageElement, etc.)

#

And this is how you use it:

    createJsonProjectionV2(json: Accessor<any>, setJson: (x: any) => void): Accessor<Result<EcsComponent<S>>> {
        return createMemo(() => {
            let json2 = json();
            let state = createJsonProjectionViaTypeSchema(
                this.typeSchema,
                json2,
                setJson,
            );
            if (state.type == "Err") {
                return state;
            }
            let state2 = state.value;
            let [ state3, setState3, ] = createStore(state2);
            return ok(new EcsComponent({
                type: this,
                state: state3,
                setState: setState3,
            }));
        });
    }

Doesn't get any easier than that. Much cleaner!

dark bay
#

I'll have to debug that. Didn't work 1st go. But u get the idea.

dark bay
#

If anyone knows the trick to do a Proxy of an Array, I can use help... can't seem to figure it out.

ocean notch
#

what does an array proxy do?

dark bay
#

Pretends to be an normal array, but does other stuff under the hood for the get/set of data.

#

Same as proxy for object. Just can not seem to make it work for arrays at the moment.

#

It's connected to the failing Jest test in the wip branch.

ocean notch
#

is the problem with nested arrays ? or just one simple array?

dark bay
#

Single simple arrays. Seems to loose all inbuilt array functions like map.

#

I know solid's Store does it somehow. So should be possible.

#

I probably should write a more focused test for it

ocean notch
#

oh I thoght it enogh to implement the setter getter for array
as the map etc, will use those underlying

dark bay
#

Anyway... gotta work... catch yaz later. And thanks for reviewing.

ocean notch
#

oh I think I get it, the getter trap will catch the map prop so you need to return the Reflect.get and pass arguments for when dealing with prototype methods

#

you can see in my shallow array example

https://playground.solidjs.com/anonymous/abde9452-f0c4-4fd4-b58a-1ee165f5744e

I have tested for this implicitly by asserting mutation only for numeric index, which allow all other property access to go via the native behavior

others prefer to have a lookup table

and others just check if key exists in the prototype ( probably the most expensive )

Solid Playground

Quickly discover what the solid compiler will generate from your JSX template

#

I can probably optimize my test using
prop[0] >= "0" && prop[0] <= "9" check for is numeric

zenith void
#

seems do to the trick

ocean notch
#

good reference

dark bay
#

Thanks guys! I'll try out both ur approaches next time I am on a PC.

zenith void
#

it was around the time i was playing around w automerge too

#

so it might have been when i was attempting different strategies for ephemeral data

#

made around the same time, so pretty sure this were ephemeral automerge data related experiments

zenith void
#

i m constantly rebuilding file-explorer

dark bay
zenith void
#

for sure

#

think i will abstract it into a lil lib

dark bay
#

thats something that would make a good additional as a ui component lib for everyone to use

zenith void
#

i never implemented those lines but i have an idea

Screenshot_2025-03-31_at_11.09.14.png
zenith void
zenith void
dark bay
#

only change I wanna see is the file system in solid primitives to use blobs instead of strings for file data

#

for storing images basically

#

I know we can base64 though

zenith void
dark bay
#

and for text... u just go await blob.text(), so nothing is lost

#

maybe that breaks the non-async interface for the filesystem though

zenith void
#

then you can add additional metadata too

dark bay
#

the current automerge file system is reactive all the way down, even keeps going down in the contents of the files as low as possible.

#

it was about 1000 lines due to extra boilerplate to simulate async signals. When solid 2.0 comes out, it will probably drop to 500 lines 😛

#

will try to copy one of the proxy array solutions you and zulu have provided above real quick before I get kicked off

#

I don't need reactivity in the array proxy, just the ability to intercept its get/sets. While still having all the in-built array functions.

dark bay
#

@zenith void @ocean notch ... you guys rock!! , the array proxy is now working my end thanks to your samples. Thank you!

dark bay
#

Few more bugs with the new cleaner approach. But I am confident I can figure it out.
Now have the automerge doc complaining about Symbol(solid-proxy) being added with value undefined. It's saying undefined is not a valid automerge doc value.

#

Or... I can just stick with the messy but working old solution for now.

zenith void
dark bay
# zenith void Unwrap it mb?

I've work it out I think... just changing the target in Reflect.get/set so it's not the automerge doc seems to do the trick.

#

In termux. Debugging in Kiwi. Slow going, but I think I've got it.

#

createStore (being used on top of the proxy) seems to read all the base fields initially. I'll need to put that in a create memo to prevent the EcsComponent from being continuously recreated on each change.

#

After that I should be bug free... fingers crossed 🤞

#

No problem with Proxies themselves at the moment:

image0.jpg
#

The test suite is starting to make me feel more professional :p

#

I'll need to make one more Proxy to get an Accessor<Store<A>> to Store<A> without reading the accessor to feed to new EcsComponent(...)

#

... and that seems to have worked!... awesome!

dark bay
zenith void
dark bay
dark bay
#

I've put the cleaner EcsComponent on hold for now. I'll just continue with the older working projection. Will go back to features. (Been stuck on it for too long.) I can get away with diff/reconcile to lower network traffic for map updates.

dark bay
#

Then again. I can probably do a reactive deserializer and skip the proxy altogether.

ocean notch
#

yeah maybe better to start with something that works and improve from there

dark bay
#

And also allowing for automerge doc handles 😄

zenith void
#

Exactly!

#

Was thinking to make a filelist view as well, where T would be some metadata

image0.png
dark bay
#

I'd like to see an interface in addition to the concrete implementation of FileSystem. Because you can have a doc handle representing ur entire file system as well.

zenith void
dark bay
#

In my automerge file system. I have an automerge doc for every folder and file. The folder automerge docs contains doc urls for each of the automerge files.
I'd like to be able to implement an interface from ur lib and use it.

#

Ahh... maybe I can still sorry.

export type FileSystem<T> = ReturnType<typeof createFileSystem<T>>
#

It's just a type

#

I think it's fine.

zenith void
#

Ye 😅 it's a lazy type

dark bay
#

Was just looking for the word interface and thought concrete only... sorry.

zenith void
#

But ig for ur example to work you would need to have something more like the async FileSystem

#

Bc you don't know the whole tree beforehand

dark bay
#

Yeah, true.

#

Unless it auto populates a sync one when the data arrives.

#

But will miss spinning circle animation

zenith void
#

True

#

You could also have a doc that handles the tree

#

And it contains all the urls of the actual files

dark bay
#

Even have a in the middle sync fs show files with a spinning circle icon to represent the async fs loading.

dark bay
zenith void
#

i think we mb also would need some type of adapter

#

that you could write to connect it to automerge-documents/...

dark bay
#

Ur directory listing be like Accessor<{type: "Pending"} | {type: "Error", msg: string} | {type: "Success", value: Entries[]}> I think, to allow for both async and remote reactive updates.

#

It will just be Accessor<Entries[]> when solid 2.0 comes out.

zenith void
dark bay
dark bay
#

Is pnpm good?

#

I'm still on npm... don't laugh 😆

zenith void
#

i like it pnpm

#

it's what i always use

#

the caching is quite nice

#

but nothing wrong w npm either tbh

#

i like that pnp copied the api of npm mostly

#

so i use npmjs, copy the command and then write p and then paste

#

it's a simple QOL feature, but i like it

#

w yarn it's like all slightly different for no specific reason

dark bay
#

Ahh.. cool, same command line args

zenith void
#

yes exactly

#

u can do pnpm add ... too, but u don't have to

#

and like --save-dev is all the same

dark bay
#

Was thinking of making kitty a proper mono repo with multiple independent reusable packages.
At the moment it is just 1 big ball of code.

#

Pnpm looks like it will keep down the size of the nodes modules folder, for when it's repeated over and over.

zenith void
#

Yes exactly. It's like npm + cache layer.

dark bay
#

p for pache :p

zenith void
#

So it not only keeps the size of ur monorepo small, but also all yr other projects

dark bay
#

Right... so it's a globally shared cache.

zenith void
zenith void
#

Performant apparently

Screenshot_20250402_121507_Brave.jpg
dark bay
#

Ahhh 😆

zenith void
#

Bit anticlimactic

dark bay
#

Gonna take a while to get used to pnpm when typing npm 100s of times per day.

ocean notch
dark bay
dark bay
#

I'm a sucker for punishment... I wanna fight an EcsComponent projection a little longer. Here is a test case (passing) showing a use case.

    test("TypeSchema json projection v2", () => {
        let json = {
            firstName: "John",
            lastName: "Smith",
            /**
             * This is a non-json object (Vec2) in the surrounding state
             */
            location: {
                x: 10.0,
                y: 15.0,
            },
        };
        type State = {
            firstName: string,
            lastName: string,
            location: Vec2,
        };
        let objTypeSchema: TypeSchema<State> = {
            type: "Object",
            properties: {
                firstName: "String",
                lastName: "String",
                location: vec2TypeSchema,
            },
        };
        let projection = createJsonProjectionViaTypeSchemaV2<State>(objTypeSchema, () => json, (callback) => callback(json));
        expect(projection.type == "Ok");
        if (projection.type != "Ok") {
            return;
        }
        let projection2 = projection.value;
        let [ state, setState ] = createStore(projection2);
        setState("firstName", "Apple");
        setState("location", Vec2.create(1, 2));
        expect(json.firstName).toBe("Apple");
        expect(json.lastName).toBe("Smith");
        expect(json.location.x).toBe(1);
        expect(json.location.y).toBe(2);
    });
#

u can see the location in the automerge json document is proper json, yet the surrounding Store has the location exposed to the surrounding App as a Vec2 class.

#

its converting data types back and forth between the automerge document and what the surrounding App expects to work with.

#

Its incomplete (this 2nd attempt), but I'll aim to get enough of it working for all the existing ecs component types.

#

I just couldn't let go 😛

#

I avoided the Proxy for the Object, but still need to do the Arrays. Arrays I'll probably have no choice but to use a proxy. Been trying to avoid proxies, bcuz they keep giving me dramas 😛

dark bay
#

I seem to be getting an infinite loop in reactivity with this new attempt when connected to the level builder. But no crashes.
Will need a way to get solids reactivity to work properly in vitest so I can debug it more easily.

#

When solidjs is running in vitest. It seems to think it's running server side, and all the reactive stuff is disabled.

#

I wanna avoid adding jsdom bcuz it's large and I won't need it.

#

Just need to tell solid I am still client side when running vitest tests.

ocean notch
dark bay
#

I think it is only required if we are rendering in our tests. (Dom polyfill)

#

Still struggling to convince vitest that I am a client and not a server.

ocean notch
#

for me when i install vitest I see jsdom in the node_modules

dark bay
#

I see. Gotta go sorry back in couple of hours.

ocean notch
# 
import {createSignal, createEffect} from "solid-js/dist/solid.cjs"

might work too

dark bay
# 
Could not find a declaration file for module 'solid-js/dist/solid.cjs'. '/home/clinuxrulz/GitHub/solid-kitty/node_modules/solid-js/dist/solid.cjs' implicitly has an 'any' type.

😿

#

will try out jsdom.... vitest-ing solid works on your end yes?

ocean notch
dark bay
#

Not my day 😅 :

ReferenceError: document is not defined
 ❯ render node_modules/@solidjs/testing-library/dist/index.js:24:5
#

maybe I can try and find a way to not use a proxy on the array part.... proxies seem to be my kryptonite

ocean notch
ocean notch
dark bay
#

was thinking about using createWritable for the array and two way data bind with it to avoid making a proxy.... even though createWritable will be making a proxy under the hood... but Ryan makes better proxies 😛

#

hmm... auto import is not finding createWritable,,, is it a solid primitives thing?

ocean notch
#

probably I don't think it is solid core

#

some talk about it here
#general message

#

createSignal in solid2.x

dark bay
#

I tried to follow ur stackblitz, no idea why I couldn't get it to work. (using createRoot instead of render even). no crash or errors, but reactivity not working for some reason.

#

I will try to debug it without vitest for a bit.

dark bay
ocean notch
dark bay
#

jsdom

#

I tried following a few tutorials online for vitest without success too. Could be a version thing with one of my dependencies.

ocean notch
dark bay
#

I was just copying tutorials

ocean notch
#

it is possible it overrides the enviornment for that test

dark bay
#

U might of saved me from losing a lot of hair 😅

#

A bit of progress.... but not yet:

FAIL  src/index.test.ts > TypeSchema json projection for automerge > TypeSchema json projection v2
TypeError: Cannot read properties of undefined (reading 'registerGraph')
 ❯ createStore node_modules/solid-js/store/dist/dev.js:239:9
#

Definitely thinks it's a now client now, so that is a plus.

#

DEV$1.registerGraph({ from solid. DEV$1 is undefined.

#

I think it's a debugging utility though. Not a required thing.

ocean notch
#

yeah, dev build has it

#

so try removing development from the test config maybe

#

I don't know if that was triggering it
but worth a try

dark bay
#

No lucky unfortunately... gotta go for an hour... timing is not great sorry.

#

I can test in the actual browser though

dark bay
ocean notch
#

oh interesting, i guess no one tests solid

dark bay
dark bay
ocean notch
#

I don't know, seem too messy, all this client server build mix

dark bay
#

I wonder if it's meant to go in vitest.config.ts or vite.config.ts... (test.server.deps.inline: true)

dark bay
ocean notch
#

try anywhere, this is too messy for me

dark bay
#

Once it's working all the pain will be forgotten 😀

#

Lmao... that extra entry in vitest.config.ts made it fail to find the test altogether:

Screenshot_20250405_075316_Termux.jpg
#

Maybe I can use solid-start when testing, and solid-core when running.

dark bay
#

Ohh.... ur stackblitz is working and ur using vite-solid-plugin

dark bay
#

Anyway everything seems to be working perfectly in DebugProjection.tsx, I will have to debug my surrounding usage of it.

dark bay
#

I tested for depth of reactivity as well in DebugProjectiin.tsx:

        createComputed(on(
            () => state.secretCodes,
            () => {
                console.log("A");
            },
            { defer: true, }
        ));
        createComputed(on(
            () => state.secretCodes[2],
            () => {
                console.log("B");
            },
            { defer: true, }
        ));
        createComputed(on(
            () => state.secretCodes[2][1],
            () => {
                console.log("C");
            },
            { defer: true, }
        ));
        setState("secretCodes", 2, 1, 2);

Only "C" gets logged to console which is good. So I'd say it's working, I just miss-used it somewhere.

#

And DebugProjection.tsx is using a real automerge doc, not just a json object.

#

createEntityWithId in EcsWorldAutomergeProjection.ts is most likely the point of miss-use that needs repair.

#

Close to success now... I can smell it :p

#

In createEntityWithId I probably need to use the same projection utility and overwrite the state/setState fields of the EcsComponents passed in by the end-user.

#

It's kinda like pretending the user owns the state, but it is really owned by automerge.

dark bay
#

getting exciting... new cleaner projection starting to work in Texture Atlas tab.... but locking up/infinite loop in Level tab.

#

some bug with projection of arrays in arrays I reckon

dark bay
#

Currently new projection working perfectly on new projection for a single client/peer...multiple client/peers seems buggy still.

ocean notch
#

will it pain you to know that inline solution worked for me too 🤣

ocean notch
#

try with this

test: {
 environment: 'jsdom',
 deps: {
      optimizer: {
        web: {
          enabled: true,
          include: ['solid-js', 'solid-js/web', 'solid-js/store'],
        },
      }
      // inline: [/solid-js/],  // this still works but is deprecated
    },

in the vite config, and remove the vitest config you don't need it

or make sure you have the test config in one place

dark bay
#

Cheers...When I get another chance, I'll give that a go.

#

Putting together another test for the ecs world projection (everything container)

dark bay
dark bay
#

Gonna be stuck on the wip branch for a while until I can merge it back. Eager to sort out last few bugs.

dark bay
#

Just had a thought. If I apply the same patterns/tricks to the ecs world projection as the ecs component projection, then it should neutralise the last few bugs.

#

Bcuz it will be like I am working directly with the automerge docs with just a data transformation pipeline in the middle of the communication.

dark bay
#

[EcsComponent projection went from 2 sources of truth to one. Do the same with EcsWorld projection]

dark bay
#

((Gonna sacrifice some performance for robustness, then reintroduce optimisations gradually))

dark bay
#

Lol... found a bug in my supposedly bug free ecs component projection.. didn't test it enough.

#

It was only reactive in the deep leaves, and not at the branches if something changed right on the branch.

#

May have to rewind some unnecessary changes to apply the correct fix.

#

Thank goodness for git

dire dirge
#

@dark bay just stumbled onto your comments here trying to get vitest working with my solidstart app. If I place my config in the app.config.ts under the test section then it seems vitest doesn't respect the settings. But if I move it out into it's own vitest.config.ts file and add the solid plugin it does... any ideas on how to fix that?

#

could it be because solid start uses app.config.ts and not vite.config.ts?

ocean notch
#

yes, vitest looks either in the vite config or the vitest config

#

it is not aware of solid start custom config

dire dirge
#

hmm, what's the recommended way to config vitest then? if I can't use the app.config.ts does that mean i have to include the vite-solid-plugin etc. in a vitest.config.ts file?

#

what's the best way to avoid duplication of the vite config

ocean notch
#

I am not sure about solid start, there is a channel for that, and support channel you can try there see if any of the experts will know

dire dirge
#

kk ty

dark bay
#

So... close... got new projection working perfectly everywhere... then I switched from broadcast channel to webrtc for automerge syncing.. and it no longer synced between the client.

Could be a timing bug.

#

I was using broadcast channel for a bit bcuz got tired of establishing a web rtc connection for each test run.

dark bay
#

Ohhh... ops... was testing with the published online version connected to the local version by accident, instead of both local.
It appears everything is working perfectly. But will test some more to be sure.

dark bay
#

right... gonna merge the new projection method into main before I loose my mind 😛

#

its not perfect yet, but its better than the old projection in that it does not leak memory, reactivity about to go deep within component (instead of stopping at component level), and has 1 source of truth in the ecs world which should simplify things down the track.

#

gotta commit to something to save on going back and forth 😛

#

will try and include any bugs I find as I go in unit tests to prevent going backwards.

dark bay
#

.... and now it's merged into main branch.... one of the unit tests is failing via vitest, but passing in the browser 😆... can't win.

zenith void
#

LGTM 🚀

dark bay
#

Bit of a detour... but think it was for the best.

dark bay
#

Camping trip this Wednesday... back to Termux :p

zenith void
#

Coding in the wild 🌲

dark bay
#

time to optimize... lets start with a reactive cache over the method calls to the ecs world that wraps the automerge doc...

class ReactiveCache<A> {
    private map = new Map<
        string,
        {
            cache: Accessor<A>,
            refCount: number,
            dispose: () => void,
        }
    >();

    cached(key: string, mkValue: () => A): Accessor<A> {
        if (getListener() == null) {
            return mkValue;
        }
        let result = this.map.get(key);
        if (result == undefined) {
            let { dispose, cache, } = createRoot((dispose) => {
                return {
                    dispose,
                    cache: createMemo(mkValue),
                };
            });
            result = {
                cache,
                refCount: 1,
                dispose,
            };
            this.map.set(key, result);
        } else {
            result.refCount++;
        }
        onCleanup(() => {
            result.refCount--;
            if (result.refCount == 0) {
                result.dispose();
                this.map.delete(key);
            }
        });
        return result.cache;
    }
}
#

the key gets constructed from the method call parameters

#

it allows u to return a shared memo for computations

dark bay
#

.... ahhh ... what a pest, enabling the broadcasting network adapter, breaks the web rtc network adapter. Wanted both enabled for convenience.

#

Probably because the initial doc sync happens when network adapters are available, and the web rtc one comes later after the user passes the invite key over.

#

It's alright... main use case is connecting with other... web rtc only will be fine.

zenith void
#

Mm I thought I brought it up in their discord once but cannot find it anymore

dark bay
#

Not 100% sure... my web rtc adapter is a copy paste of their broadcasting adapter plus some tweaks

zenith void
#

A wait, mb we r talking about different things

#

With broadcasting adapter you mean the BroadcastChannel adapter?

dark bay
zenith void
#

Ok, then we r talking about the same thing 😅 sanity check

dark bay
#

U could be right... repo.find does not seem to resolve when using both broadcasting channel adapter and web rtc adapter together.

#

It could be an error on my half though.

#

I just copy pasted and tweaked their code without fully understanding it.

dark bay
#

Just as an experiment for now in a separate branch.

zenith void
#

not super sure about the api choices i made w the component

#

so things might break

#

if u have suggestions please lmk 🙂

#

mm 🤔 you already have a fully reactive filesystem right

#

AutomergeVirtualFileSystem

#

feels a bit stupid to have to make an additional wrapper over it

#

can't u plug the automerge vfs directly into the component?

dark bay
dark bay
#

I'm using a git submodule to directly use ur project inside my project at the moment.

dark bay
#

@zenith void ... works!

Screenshot_from_2025-04-07_18-13-48.png
#

no change required for your API, fake files called "Loading..." does the trick. (For async to sync)

#

Couple of things that would be nice:

  • multiselect
  • different indent renderer for last file in folder. (L shape indent for last line instead of sideways T).
  • drag and drop to trigger move (rename?)
#

One cool thing is it will only read a subset of the whole file system. (Just the part u can see from expanded folders, when u expand them)

zenith void
dark bay
#

Not sure if the selection state should belong inside or outside the file explorer component.

#

Was thinking of making the file explorer the main part of the app that is always accessible. And the main view automatically switches between different sub-apps based on which file is selected.

#

I currently have the .json extension on all files (except images), and the program knows which sub app it is for based on the folder it is contained in currently.

#

I may keep that pattern, and make some folder undeleable. Then u can click one of those folders and hit the new file button to make a new file for the given sub-apps for that folder.

#

[Master/Slave Pattern]

zenith void
#

looks also a bit weird

Screenshot_2025-04-07_at_12.50.03.png
dark bay
#

Like the icon becomes the fork rather than the letters.

#

It looks correct though

#

Icons can help with visual breaks:

Screenshot_20250407_210559_Chrome.jpg
#

Ascii art icons?

  • [+] Folder Closed
  • [-] Folder Open
  • [■] File
#

https://en.m.wikipedia.org/wiki/XTree#/media/File%3AXtreegold-3.0.png :p

XTree

XTree is a file manager program originally designed for use under DOS. It was published by Executive Systems, Inc. (ESI) and first released on 1 April 1985, and became highly popular. The program uses a character-mode interface, which has many elements typically associated with a graphical user interface.
The program filled a acquired niche in t...

zenith void
#

i think i know what it is

#

you shouldn't cut off the stem

#

then you chop the tree

dark bay
#

To think xtree gold sold 3 million copies at $39.95 USD each...

#

That's a lot of money in the 1980s

#

$120 million :p

zenith void
#

lol crazy

dark bay
#

Not bad for 3 days work (he did it over the weekend starting friday)

#

His office mates told it was impossible on the computers at the time.

#

640Kb ram limit

dark bay
#

Some of the things u can do in text mode (ascii) are amazing:

image0.gif
#

Who needs pixels :p

dark bay
#

new meets old. new tech, old platform.

zenith void
dark bay
#

Works with touch screen. Nice work!

dark bay
#

I honestly did not do any coding the wild... ended up getting too tired from the 6 hour drive.

zenith void
#

pointer-events ftw 🙂

zenith void
dark bay
#

Cheers

#

Looking forward to see what Ryan has to say about alien signals 2mrw. Been having a quite read of their code.

zenith void
#

and ye 👽 gonna be fun

dark bay
#

I'm GMT+10

#

I'll be a sleep while it's live.

zenith void
#

#1200006941586509876 message lesgooo, alpha release of automerge 3.0.0

dark bay
#

Main app ui to connect everything together... very minimal :p

Screenshot_20250413_084227_Chrome.jpg
#

1 network button for optional peer connections. 1 file explorer button for creating/selecting files, which will reveal a different sub app based on the selected file.

#

Gotta show/hide the file explorer rather than keeping it visible to free up screen space on mobile.

dark bay
dark bay
#

Gotta be careful. Can not get at the close popup button:

Screenshot_20250413_122531_Chrome.jpg
#

overflow hidden + scrolls in popup maybe

#

Maybe connections and invites above each over instead of side by side.

#

Only connection manager and file explorer will appear as pop ups, the rest of the sub apps will populate the area below the menu bar based on file selection.

dark bay
#

This is a bit strange for onSelect of FileTree:

Type '((path: string) => void) | undefined' is not assignable to type '(EventHandlerUnion<HTMLDivElement, Event, EventHandler<HTMLDivElement, Event>> & ((path: string) => void)) | undefined'.
#

its like it merged a onSelect event handler for a div with the FileTree

#

ahh... but theres an API change in ur playground (for multi-select), so I probably don't have to worry about it.

dark bay
#

Switched to ur playground version, so I can select folders: :p

image0.jpg
dark bay
#

I do think it is a good idea what u have done with storing the selected state per file/folder. Bcuz that is effectively O(1) selection update for multiselect (or O(m) for m changes of selection).

dark bay
#

there must be some trick we can use to obtain the selection information out of the file tree.

#

Feeding the per file/folder selection state to a reactive map might do the trick for now:

<FileTree
    fs={fs2()}
    style={{ display: "grid", height: "100vh", "align-content": "start" }}
>
    {(dirEnt) => {
        createComputed(() => {
            let path = dirEnt.path;
            let selected = () => dirEnt.selected;
            selectionMap.set(path, selected);
            onCleanup(() => {
                if (selectionMap.get(path) === selected) {
                    selectionMap.delete(path)
                }
            });
        });
#

not a perfect solution, but good enough.

dark bay
elfin lintel
#

the UI reminds me of it

dark bay
elfin lintel
#

mhm, tbh im also using daisyUI for Tungsten

#

to boost productivity

#

even tho i can make a good interface

dark bay
#

Nothing wrong with it. Handles light and dark mode, and it comes with a bunch of themes for free.

elfin lintel
#

yeh

zenith void
zenith void
dark bay
dark bay
zenith void
# dark bay

We probably need a focused state too, for doing things like create a dirEnt inside the focused dir

#

Selection can be multiple, focus can only be 1

zenith void
dark bay
zenith void
dark bay
#

It's not perfect. (Possible diamond issues). But does the trick.

#

A cleaner solution would be a proper projection adding a selected field to the original data before it is feed to FileTree I think. But not 100% sure.

#

Could be createProjection comming to solid 2.0 can handle it properly.

dark bay
#

@zenith void ... not sure if ur a fan of this idea:

createFileTree(props: ...): {
  selection: ReactiveSet<Path>,
  clearSelection: () => void,
  .  .  .
  Render: Component,
}
zenith void
#

mb better then having a bunch of event handlers

dark bay
#

events are sort of like the derivatives of the continuous values. Describing the change in the values.

zenith void
#

Exactly. A signal in an effect basically.

zenith void
dark bay
#

Was also expecting Ctrl+click to select multiple non sequential files. But maybe that is a Windows/Linux thing and the short cut is different on a Mac.

#

Shift+click works well for multiple sequencal.

zenith void
#

will do the navigator trick to check if we are on mac or not

dark bay
#

All good... thought u might be mac :p

zenith void
#

ok, i should probably move development back to vscode and the like

dark bay
#

ohh... u've started on double click for rename too?, u have been busy.

zenith void
#

mb it doesn't make too much sense 🙂 i could make a <FileTree.Input/> that abstracts away that logic

dark bay
#

Nothing wrong with the approach u have there.

dark bay
zenith void
#

Maybe <FileTree.Name editable />

dark bay
#

Just a thought. The file tree itself does not need access to the file contents/data. A type interface could be simplified there.

#

thus supporting string, blob, uint8array, doc id, etc. for file data, just by simply not requiring file read/write in the file system interface.

dark bay
#

Caught myself out. Got the automerge doc ID selection out of the file explorer, but when the file explorer window closed, I lossed my selection :p. I probably should of externalized my selection state, or just hide the file explorer from the dom rather than closing it.

#

Could be a good argument for createFileTree (maintaining selection state while the component is not rendered on the screen). Thus separating model and view. (MVVM / MVC)

#

Or.... just the ability to externalise the selection state will be enough.

#

So just the ability for a <FileTree> to start of with an initial selection will be enough.

#

So when the file explorer is closes and opened, the same file can remain selected... currently I am just disconnecting (from the DOM) but not disposing the file explorer as a work around to keep the selection state.

dark bay
#

Slowly but surely.. starting to connect the loose parts together:

connect-all.gif
#

The pixel editor sub-app does not support automerge yet. But I've had discussions with the automerge crew to form a plan of attack to add automerge support to it.

#

(only texture atlas and level editors support automerge at this stage)

zenith void
#

Aa I see what u mean, the type should only require readdir + rename.

#

I currently do require readdir to have a withFileType option, which is a bit awkward.

zenith void
zenith void
#

what do you think the behavior should be with shift+click when you mount the file-tree again? feels a bit weird to me that it would multi-select from the last selected element.

dark bay
#

Currently I keep it mounted and just add/remove from DOM to keep the expanded state and selection state.

#

Something interesting for auto merge support in the pixel editor. We need to add IDs to each of the pixels. So if from two disconnected peers, one draws some pixels, and the other selects and moves an area of pixel, then u can have the correct expected result after theirs states merge (other a connection).

#

E.g. one peer edits Mario's hat in one of the sprites, and the other peers moves Mario to a different location in the sprite sheet.

# 
type Data = {
  pixels: UUID[][],
  colourTable: Map<UUID,Colour>,
}

I know the memory usage is going to be a lot more.

#

The main reason for the UUIDs is to simulate pointers/references in automerge. So you can do real moves, rather than copy+delete pixels.

zenith void
#

o that's pretty wild

#

would be cool to see it in action, how those states will be merged once u get back connection

zenith void
dark bay
#

Its honestly already good enough for me to use.

zenith void
#

cool 🙂 will drop a release

#

since all the DirEnt are keyed to the path

#

we handle the selection, but the focus is gone atm

#

maybe it could be solved by having a focused selector too and autofocusing on it when it's mount

dark bay
#

I never really understood headless. What does headless mean?

#

I suppose if files / folders each exposed a fixed ID that never changed. Selection and focus can be stored by their IDs. That way if a reactive rename happens somewhere the FileTree is not aware of, then the selection and focus state will remain correct.

#

Ahh right... sorry though the 2nd was a screen, not a video.

#

I suppose ur idea of autofocus on mount will fo the trick.

#

Although.... if it was a <For> of their IDs, then u would not need to refocus. Because the real file will move position, rather than a couple of name changes in fixed positions.

#

Its becuase the path is not really a suitable ID.

#

The api exposed by file system in solid primitives does not seem suitable for headless (no file handles or file IDs)

#

Object references can also be an ID of a kind with some care. There could be a work around.

dark bay
zenith void
zenith void
dark bay
dark bay
zenith void
#

But I also want it to be able to change those compound components with your own stuff.

dark bay
#

I'll research node fs a little bit. It seems strange to not have something like a file handle. But I've never used raw node (just browser stuff)

zenith void
dark bay
zenith void
#

It could be possible there is a way to get a FileHandle from these paths

dark bay
#

I might be incorrect about file handle... it seems u have to open the for reading to obtain it.

#

A file system watcher would be most useful in order to keep it reactive.

zenith void
dark bay
#

Tricky. Maybe IDs can be generated and maintained along side a fs.watch.

#

Need a way to distinguish between a rename and a delete+create done by a remote process.

#

So IDs can be carried over.

#

Another option is to make an ID tracking middleware that is best efforts and gets told able adds/deletes/renames. (To support backends file systems that do not have IDs)

dark bay
#

That way the focus will automatically be correct when u rename a file.

#

(The middleware can be told about fs mutations FileTree makes)

zenith void
#

it even does the strike-through implying that it deletes and recreates

#

i will keep it path based then i think, just to keep the implementation simple

#

yup, definitely path based: when i

  • focus on tsconfig.json
  • rename tsconfig.json to ok.json
  • rename package.json to tsconfig.json
    it keeps the focus on the file named tsconfig.json (fka package.json)
dark bay
dark bay
# zenith void i will keep it path based then i think, just to keep the implementation simple

If ur interested what an ID middleware would look like, then here is a completely untested example typed up in notepad on my mobile:

function idMiddleware<A>(
  x: Accessor<A[]>,
  extractPath: (x: A) => string,
): {
  out: Accessor<{ id: number, value: A }[]>,
  beforeRename: (oldPath: string, newPath: string) => void,
} {
  let nextId = 0;
  let idMap = new Map<string,number>();
  let out = createMemo(mapArray(
    x,
    (x2) => {
      let path = extractPath(x2);
      let id = idMap.get(path);
      if (id == undefined) {
        id = nextId++;
        idMap.set(path, id);
      }
      onCleanup(() => idMap.delete(path));
      return { id, value: x2, };
    }
  ));
  let beforeRename = (oldPath: string, newPath: string) => {
    let id = idMap.get(oldPath);
    if (id != undefined) {
      idMap.set(newPath, id);
    }
  };
  return { out, beforeRename, };
}
zenith void
#

could u show me how it would be used?

dark bay
# zenith void could u show me how it would be used?

I guess its like:

let { out: entriesWithIds, beforeRename, } = idMiddleware(entries, (e) => e.path);
.
.
.
beforeRename("fileA.ts", "fileB.ts");
// then do actual rename.
// now when fileB.ts shows up,
// it will have same ID number
// as fileA.ts had.
#

Its to best efforts attach ID numbers to entries of directory listings.

zenith void
#

aa i see gotcha

dark bay
#

And if u key by that ID, the focused element will actually move on rename. (Move whole DOM element)

zenith void
dark bay
zenith void
#

I will keep it in the back of the head

zenith void
#

Ok, i think i will implement the id idea: when you have selected a path and do fs.readFile(...) and then change the parent there is currently no way to update an opened file's path.

I will need something like <FileTree.DirEnt onRename={(newPath) => { if(selectedPath(dirEnt.path){ setSelectedPath(newPath) } }}/> but I can't because the DirEnt is bound to the path

#

ig i could do <FileTree onRename={(oldPath, newPath) => { }}/> too

#

maybe it actually needs to be on the root-component, because otherwise it requires the parent of <FileTree.DirEnt/> to be opened.

dark bay
#

We can stick with what works now and optimise later if need be.

dark bay
dark bay
zenith void
#

Mm I think we are talking about something different.

I meant like what if the jsx looks like

<FileTree>
{(dirEnt) => <FileTree.DirEnt onRename={(newPath) => ...}/>} 
</FileTree>

And the filetree would first look like:

  • dir
    • file

  1. I click on file which opens it up in an editor

  2. I click on dir, causing it to collapse.

  • dir
  1. Then I rename dir to dir2.
  • dir2

If I would rely on an event handler on the DirEnt, it wouldn't be called, because it's unmount when its parent is collapsed.

#

(Typing on mobile is so hard. So much respect to u for developing on it)

zenith void
dark bay
#

mounted != visible, always

zenith void
dark bay
#

I'll do an example later. (Need computer access)

zenith void
#

a do you mean that <FileTree/> would keep an internal state of the selected file and the file that is opened is kept mount (but not inserted in the dom)?

#

because currently that is not kept inside

#

it's something like

const [openedFile, setOpenedFile] = createSignal<string>()
return <FileTree fs={fs}>
{(dirEnt) => <FileTree.DirEnt onClick={() => { if(dirEnt.type === 'file'){ setOpenedFile(dirEnt.path) } }} />
}
</FileTree>
dark bay
#

Gotta go for a bit sorry. I'll do a fork later and show ya a trick.

zenith void
#

thanks mesh!

dark bay
zenith void
#

O no u mean the opened file in the file editor hahaha naming really starting to become an issue

dark bay
#

Its not finished yet. Need to change selection and focus to use IDs instead of paths.

zenith void
#

nice!

dark bay
#
  • call beforeRename, just before doing a rename.
  • obtainId(path: string): number to obtain ID for path.
  • freezeId(id: number) to prevent an ID number changing for a path. (I.E. freezing selected entries and focused entry). So that the focus/select state remain valid between collapse and expand.
zenith void
#

not too much code!

dark bay
#

In theory if the rendered dir entries are keyed by this ID, the focus state held by the browser will remain valid on rename.

#

Because the actual DOM element will move for the dir entry.

zenith void
#

ye that's awesome

#

please make a pr of it 🙂

dark bay
#

Its incomplete though... my wife kicked me off the computer.

#

She says I have been sitting for too long.

zenith void
#

she takes good care of u!

dark bay
#

Maybe the body. Not sure of the mind.

#

On the TODO:

  • change selection state from paths to IDs
  • change focus state from path to ID.
  • render keyed against IDs instead of paths.
zenith void
#

If u wanna u can make the pr and add the todo in there

#

I can take over, but I want u in the collab list 😁

dark bay
#

Ohh... and:

  • createCompute over the focus ID and selection IDs state to freeze Ids.
dark bay
#

Just incase my theory turns out incorrect down the road.

#

There is a way to change where it goes into after i make a pr

zenith void
#

great!

dark bay
#

Have fun... i have to do some shopping now. Catch ya later.

zenith void
#

see ya mesh!

dark bay
#

Oh... the beforeRename needs adjustment too. So that if a parent folder is renames, then all the child files/folders can keep the same ids.

#

(It needs to trigger the beforeRename for all the decendents of the parent folder that have a monitored ID)

#

There could be a more tidy strategy yet.

dark bay
#

I've been playing around in termux. I had to modify a context a little bit to keep reactivity working with DirEnt when rendered keyed by ID.

const DirEntContext = createContext<{ dirEnt: Acce      ssor<DirEnt>, }>()

Not 100% sure why it needs that, but it makes it work.

zenith void
#

mm strange 🤔

#

u need to add it to an object?

dark bay
#

I have never used createContext before in my own code. I dont fully understand what it does under the hood.

zenith void
#

it attaches to the owner-graph that you can access. it's a pretty simple mechanism

#

this is the code:

export function createContext<T>(defaultValue?: T): Context<T> {
  const id = Symbol("context");
  return { id, Provider: createProvider(id), defaultValue };
}

export function useContext<T>(context: Context<T>): T {
  return Owner && Owner.context && Owner.context[context.id] !== undefined
    ? Owner.context[context.id]
    : context.defaultValue;
}

function createProvider(id: symbol) {
  return function provider(props: { value: unknown; children: any }) {
    return createMemo<JSX.Element>(() => {
      Owner!.context = { ...Owner!.context, [id]: props.value };
      return children(() => props.children) as unknown as JSX.Element;
    });
  };
}
dark bay
#

I see.... so its something to do with where dirEnt is read ig... I'll do a commit so u can see it too.

zenith void
#

great!

dark bay
#

I've commit it to the id-gen branch.

Render is now keyed by id. (Moves physical dom element on rename.)

zenith void
#

nice!

dark bay
#

It might have to change from:

path: dirEnt().path,

To:

get path() {
  return dirEnt().path
}
#

And I can probaby place the ID inside DirEntBase to keep it more tidy.

zenith void
#

That makes a lot of sense, now it's not anymore keyed to the path

dark bay
#

Hacking in termux atm... while someones asleep :p

#

I like the way u have written ur code. Its quite tidy.

dark bay
#

Will have to check it out 2mrw ig.... good night.

zenith void
zenith void
#

ooops

#

i wanted to add some commits to ur pr but i seem to have closed it accidentally

#

cleaned it up a lil: moved all the id generation stuff into createIdGenerator utility

#

it's not possible to work together on a single pr? bit of a pr noob tbh, imma mainpusher 😅

# 
onMount(() => {
  if (dirEnt.focused) {
    element.focus()
  }
})

this will not work anymore: this assumed that the FileTree.DirEnt would remount when the path changed.

#

I guess we could do something like

createEffect(on(() => [dirEnt.focused, dirEnt.name], (focused) => {
  if (focused) {
    element.focus()
  }
})
#

Change selection and focus state to use IDs instead of paths.
For this I guess we will need to have a getPath-method in createIdGenerator for when we want to do the move operation

#

I am not sure I get the function of freezeId tbh

#

it increments the reference count, but why exactly?

dark bay
#

freezeId prevents some paths from obtaining a new ID when the parent folder is collapsed then expanded.

#

(Keeps their ID the same)

#

E.g. select a file, collapse its parent folder, expand its parent folder and the same file is selected.

#

Thats for after selections are a list of IDs instead of paths.

zenith void
#

aaa i see

#

because now it's double incrementing the ref count

createEffect(
  on(selectedDirEnts, selectedDirEnts => {
    for (let path in selectedDirEnts) {
      let id = obtainId(path)
      freezeId(id)
    }
  }),
)
``` but that won't be the case anymore once `selectedDirEnts` will be IDs, i gotcha.
dark bay
#

They also auto decrement reference counts as their reactive scope are lost/reset.

dark bay
#

Their might be a way around that with multiple remotes, but gets messy.

zenith void
#

o damn didn't wanna take over completely lol

dark bay
#

Google is your friend for that one xD

zenith void
#

i chatgpt'd 😅

#

another vibecoding casualty

dark bay
#

Dangerious :p

#

AI can't do software archeture. It will give u parts that typecheck together, but not a design that scales I believe.

#

Bit of a use carefully sort of thing.

zenith void
#

and apparently it's not too knowledgeable about git commands either lol

#

it got all its data from github but it doesn't know how to use it

dark bay
#

Its a temporal sort of thing I think (software archeture)

#

Same reason for inconsistencies in AI videos.

zenith void
#

ye my boss wants me to get copilot but i'm resisting it a bit

#

i think it's nicer for just asking more general questions and then implement it myself

#

also the joy of coding

dark bay
#

So it does save some time if it can respond quicker enough. (Faster than u can type)

#

But if the AI is slow. Sometimes its faster just to type.

zenith void
#

well 🤷‍♀️ everyday i learn

#

if u wanna hack on it further: please be my guest

dark bay
#

thinking about using strings for IDs rather than numbers. That way we can optionally let the end user provide IDs for their file system, if their file system supports IDs, otherwise we just use our ID generator we have currently.

dark bay
#

hmm... not really sure if having the selection state and focus state using IDs rather than paths achieves anything. That is if it is already working with paths, and the interface for selection/focus to the end user is represented by paths.

#

having a go at it, but there is a lot of back and forth conversion to maintain the interface.

#

I'll add a means to reactively convert from an ID number back to a path, just in case we want to swap over to "use IDs for selection" state down the road. And we already have obtainId to convert from a path to an ID.

#

we could give the end-user selection ids, rather than selection paths in their exposed interface (FileTreeContext), and the idToPath function. That way the file name header on their viewer can update when the file gets renamed without reloading the files contents within the viewer.

dark bay
#

I think IDs exposed to the end user through the exposed FileTree interface is the way to go. But I will wait for your thoughts.

dark bay
#

Heres the last utility function we need to get the reactivity more fine grain within the individual directory entry components:

function dirEntAccessorProxy(dirEnt: Accessor<DirEnt>): DirEnt

Basically makes a DirEnt without reading the passed in accessor. (The accessor gets read when the fields of the result get read.)

It will allow Context<Accessor<DirEnt>> to become Context<DirEnt>.

zenith void
#

i also wonder nvm

dark bay
#

We can provide both to end user

#

Both selected paths and selected ids

#

The user can choose what works best for them.

#

Selected ids have the advantage of viewer tab names auto updating without reloading the viewer contents

zenith void
#

or maybe those DirEntProxies?

dark bay
#

Selected paths have ease of use

dark bay
zenith void
#

maybe selectedDirEnts could be those proxies instead of IDs too

dark bay
#

Maybe the signal for selectedDirEnts is IDs, and their are utilities to manipulate it by path (for the end user)

#

We have obtainId for going one way and idToPath for going the other way.

#

idToPath is reactive. For a fixed id, it will auto re-execute of the path for that ID changes.

#

obtainId is non-reactive. But its used kind of like an side effect.

#

I made a start on this one, just slow going in Termux:

Screenshot_20250418_185937_Termux.jpg
#

Could just use Proxy()... but had bad experiences with Proxy() :p

#

Js objects are tricky beasts

zenith void
#

i like getters!

#

they r more performant too

#

although tbh mergeProps kinda negates that: iirc that will return a proxy too

#

but not 100% sure about that

zenith void
#

not sure how handy it is for users to get these internal ids in <FileTree onSelection={}/>

dark bay
zenith void
#

gotcha

dark bay
#

Or.... onSelection: (dirEnts: DirEnt[])

#

Few choices

zenith void
#

ye 🤔 i wonder what would make the most sense. what would be the usecase of exposing the IDs to the user in selection?

dark bay
#

DirEnt has a reactive name they can monitor. (Viewer tab)

#

Then we'd need to make sure selected DirEnt are keeped alive while selected, even if their parent folder is collapsed.

zenith void
dark bay
#

The ID manager can help with that too. It knows the paths to keep alive.

#

Or we'd do a seperate ref counting solution for those too.

zenith void
#

on a separate, tangential note: i was thinking mb the id manager could also handle the rebasing/renaming of the selectedDirEnts, since it holds a reference to all the relevant nodes.

#

then beforeRename could become rename

#

it does make its responsibility a bit more ambigue

dark bay
#

If selectedDirEnts contains just IDs, there is no rebasing/renaming required anymore. U get to delete code.

#

Selected ID 3 is still selected ID 3 after a rename happens

zenith void
dark bay
#

I think not... i will have to hack later and see

#

We have a reactive utility idToPath to get pathnames out of the IDs too.

zenith void
dark bay
#

It uses a ReactiveMap from solid primitives to keep it reactive.

#

idToPath("3") will always reexecute if node with ID 3 changes pathname.

#

And we have freezeId to keep nodes alive when not visible.

#

With a bit of luck we can make all the book keeping disappear like magic.

dark bay
#

Feel free to have a hack too. I might not get time tonight.

dark bay
#

Sorry if I am a bit pushy with the decisions.

zenith void
#

it's fun to work together on it 🙂

dark bay
#

I feel like something can be done at the // Populate dirEntsByDir (lines below comment) to ensure there is only one possible DirEnt object reference per ID.

#

It might be the next place to attack.

#

We have a mapArray on the outside, that may be possible to change to keyArray, and key it by the path's IDs (obtainId()).

#

Also feels a little odd to have an effect set a signal when u can just have a memo. But there is probably good reason for it here.

zenith void
#

you could do something like

const entries = createMemo(mapArray(..., () => [key, value]))
const record = () => Object.fromEntries(entries)

but having to re-create objects the whole time also feels a bit icky

dark bay
zenith void
#

maybe this will be solved with solid 2.0's store-projections?

dark bay
#

Anyway... time for me to sleep. Fresh mind tomorrow.

zenith void
#

sleep well mesh!

dark bay
#

We can keep the effect. Just thinking out loud there.

zenith void
dark bay
dark bay
zenith void
#

it's not that it's updating every frame or something

#

then we don't have to worry about cleanup

dark bay
#

Well even createComputed and mutate a non-reactive state. But need some planning.

#

The important thing is conserving the order the states update.

#

No weither they are immutable or mutable updates.

#

a createMemo with a dummy return that mutates an internal state can be used as a tool to preserve update order.

#

Just thinking out loud again :p

zenith void
dark bay
#

Good night mistqke

dark bay
zenith void
#

mb instead of the mergeProps in the DirEntProxy we could do something like

{
  ...,
  get expand(){
    if(dirEnt().type === 'file') return undefined
    return () => expandDirEnt(dirEnt().id)
  }
}
``` then it's getters all the way down
#

i don't think

<DirEntContext.Provider value={dirEnt}>
  {(() => {
    let dirEnt2 = dirEnt()
    return untrack(() => props.children(dirEnt2, fileTreeContext))
  })()}
</DirEntContext.Provider>
``` this is correct as it will re-render when dirEnt is updated, and i feel that that's kind of the point of the untrack, right?
#

i think for now let's just pass the accessor, it's kind of an established pattern anyway in solid (think <Index/>)

zenith void
dark bay
dark bay
#

I have some magic tricks from another project to work around that i think.

zenith void
#

If the component does not get mount/unmount w the id it's already great

dark bay
#

I think there is a bit more of a battle ahead of us.

zenith void
zenith void
#

If we pass the DirEnt as an accessor it should be fine I think

dark bay
#

This one is problematic

<DirEntContext.Provider value={dirEnt}>
  {(() => {
    let dirEnt2 = dirEnt()
    return untrack(() => props.children(dirEnt2, fileTreeContext))
  })()}
</DirEntContext.Provider>
zenith void
#

I m gonna switch it to an accessor I think

#

Also for useDirEnt

#

It's return dirEnt() rn

#

I m now playing around w selectionDirEnt and the things mentioned above as accessors and getting good progress

#

Found a bug w solid-primitive's <Repeat/> in the meanwhile

dark bay
#

Sounds good i think

dark bay
#

I was surprised Context<number> does not work. Maybe it proxies too.

dark bay
#

@zenith void this line in createProvider: (Context)

Owner!.context = { ...Owner!.context, [id]: props.value };
#

Its doing a destructure

zenith void
#

context is not reactive by default

#

that's a good one to know

dark bay
#

Lol... i really thought it was

#

Its all good. I know now.

zenith void
#

ye it's a bit of an oddball regarding that

#

not too sure about the reason why actually

dark bay
#

So thats why Context<Accessor<DirEnt>> instead of Context<DirEnt> ?

#

I noticed Context<number> does not seem to work. Then though objects only, but Accessor is working.

zenith void
zenith void
#

ye... i m starting to think this id generation is not the way forward

#

adds a lot of complexity

#

i was able to implement the selectedDirEnts quite easily, but now i was trying to do the expandedDirEnts and it's becoming a soup

dark bay
#

We can stick with paths if ur like :p

#

U already have a working solution there

zenith void
#

i think the idea of IDs is maybe not the worst, but maybe trying to have it automatically cleaned up with these onCleanups makes that it sort of difficult to have an overview of what is going on exactly

#

solution of the bug i was facing was to do

// Freeze ID numbers for selected entries
createEffect(() => selectedDirEntIds().forEach(freezeId))
// Freeze ID numbers for expanded dirs
createEffect(() => expandedDirIds().forEach(freezeId))
dark bay
#

And effects are delayed too. Maybe createComputed

#

Delaying incrementing the ref count make let it fall to zero too soon.

#

I'm still interested in pursuing the ID solution. I can make a new repo to keep pushing at it. And have the main solid-fs-components use paths.

#

Like a seperate solid-fs-components-via-ids

#

I think I can add a layer above the actual file system interface that introduces IDs, and then the FileTree can pretent IDs existing all along and not have to simulate them.

#

I.E. a fs.readDirByIds that takes an ID and returns a array of IDs implemented via fs.readDir one layer below.

#

So maybe IDs are at the wrong level.

dark bay
zenith void
#

the whole falsey stuff is really horrendous

dark bay
zenith void
#

BURN IT ALL DOWN

#

i pushed some stuff to my dir-branch

#

expanded/selected/focused are now all using IDs

dark bay
#

Cheers will have a review

zenith void
#

it's working pretty nicely, i can't seem to find any bug immediately

dark bay
#

Something did pop into my mind to simplify things greatly. But might be too late now.

zenith void
#

also using our own project structure for the demo, kinda fun

Screenshot_2025-04-19_at_02.10.37.png
zenith void
dark bay
#

let fileSystemBasedOnIDs = fsIdInjectorWrapper(fileSystem)

#

Then pass fileSystemBasedOnIds to FileTree and have it pretend IDs where supported by the file system all along.

#

Eliminating the need to simulate IDs inside the FileTree

zenith void
#

so it would be like you pass FileSystem<T> and then it returns FileSystem<{ id: string, data: T }> or something?

dark bay
#

But then again, it still needs to know which IDs to freeze.

dark bay
#

Or FileSystemIdBased<T> or something

#

We may face some of the same challenges for FileList and FileGrid.

#

The Tree is the hardest though. Maybe the worst is behind us.

zenith void
#

ye <FileList/> will basically be a<FileTree/> with metadata

#

it could be that it dissolves completely

zenith void
zenith void
dark bay
zenith void
#

or maybe even forcing the FileSystem to have a fs.stats that includes an id or something

#

and like have it dirEnt.id ?? dirEnt.path

#

so if u don't provide id, then it will remount the boys

#

although nvm, thinking out loud

#

we can't combine the both

dark bay
#

Yep... we should do a best efforts IDs if the IDs are missing. A polyfill

zenith void
#

because for the paths we need to do the bookkeeping of the selectedDirEnts

dark bay
#

@zenith void Congratulations on getting her to work!

zenith void
#

thanks!

#

and thank u for all the work too!

#

all from the android lol

dark bay
#

Neo vim is really nice tbh

#

Did we test if the browsers native focus remains when the dom element moves? Thats where the ID battle started.

zenith void
dark bay
#

Good luck seeing React do that :p... solid ftw.

zenith void
#

ok maybe we can merge to main?

#

if we want to improve on the id generation stuff we can do it in another pr

dark bay
#

Here something that might be a weird idea. But see what u think...

onSelection: (paths: Accessor<string>[])

(Updating view tab label on rename)

zenith void
dark bay
zenith void
dark bay
#

So a memo does not get created in the wrong reactive scope

zenith void
#

o i see, not Accessor<string[]> but Accessor<string>[]

#

i m becoming more and more fan of Array<...>

dark bay
#

A taste thing that one :p

#

let selectedPath = () => idToPath(selectedId);

zenith void
#

ye not sure, i think mb it's handier if it's just a bunch of strings?

#

something like ```tsx
createEffect(() => props.onSelection?.(selectedDirEntIds().map(idToPath))

dark bay
zenith void
#

does it matter? what would be the usecase?

#

you mentioned before onSelection(paths, ids) too, but what would be the usecase of the ids?

#

i think it would make sense if the ids would coming from the filesystem itself, but if they are coming from <FileTree/> i am not sure

dark bay
#

I.E. how do u prevent the view of the file from reloading the file when the file is only renamed?

zenith void
#

o i see

#

it's a naming issue i think

#

the onSelection={...} is not necessarily the view of the file

#

it's the highlighted DirEnts in the FileTree, for moving and the like

#

it's why i currently have the view of the file defined in userland

#

but it also means that they have to do some bookkeeping:

onRename={(oldPath, newPath) =>
  setSelectedFile(file => PathUtils.rebase(file, oldPath, newPath))
}
#

it will indeed fs.readFile it again, but since it gets an identical file it will not do anything besides that

#

it would be a lot nicer if you could fs.readFileById irl

dark bay
#

Tricky :p

#

I think we are good at the moment. No further changes required.

zenith void
#

I wonder if

// Update selection from props
createComputed(() => {
  batch(() => {
    if (!props.selection) return
    setSelectedDirEntRanges(
      props.selection.filter(path => props.fs.exists(path)).map(path => [pathToId(id)] as [string]),
    )
  })
})
``` is correct 🤔
#

bc pathToId would throw if it does not have it

dark bay
zenith void
#

but obtainId would cause it to memory leak, because it would only cleanup if the prop changes

dark bay
dark bay
zenith void
#

but it would have an additional reference count right?

dark bay
#

Bcuz ref count is done via reactive scopes.

zenith void
# dark bay Bcuz ref count is done via reactive scopes.

ye that's what i wanna say. all the other paths are either ref counted

keyArray(
  () =>
    props.fs.readdir(idToPath(id), { withFileTypes: true }).map(dirEnt => ({
      id: obtainId(dirEnt.path),
      type: dirEnt.type,
    }))
``` or 
```tsx
// Freeze ID numbers for selected entries
createComputed(() => selectedDirEntIds().forEach(freezeId))
// Freeze ID numbers for expanded dirs
createComputed(() => expandedDirIds().forEach(freezeId))
#

and they get cleaned up when is appropriate

#

but

// Update selection from props
createComputed(() => {
  batch(() => {
    if (!props.selection) return
    setSelectedDirEntRanges(
      props.selection.filter(id => props.fs.exists(id)).map(id => [id] as [string]),
    )
  })
})
``` will only be cleaned up whenever props.selection is re-run
#

it's probably not the worst thing in the world to have a couple of nodes sit in the nodeMap longer then really needed

dark bay
#

Gotta go sorry... maybe test it to see.

#

Its inside a createCompute

zenith void
#

see ya mesh!

dark bay
zenith void
#

thank u too!

#

good fun 🙂

dark bay
#

Reading the code for the FileTree/viewer inside the FileTree/viewer itself from your online demo is pretty trippy 😄

#

Maybe that qualifies as a Quine.

#

Well... kinda cheating at a Quine maybe.

#

@zenith void ohh.... addCleanup does not look quite correct.

#

The path could change before the cleanup is reached.

#

Maybe it should just take the id as a parameter.

zenith void
zenith void
zenith void
dark bay
#

... and beforeRename needs a very minor adjustment to take in account for a asyncronous file system where the rename does not happen immediately... a hand passing temporary ref count bump.

I can do a pr for that later. Or maybe select keeps its ref count over 0, will need to test.

#

We can probably include a global node counter to detect leaks when testing too.

zenith void
#

i fixed the leak in the props.selectedPaths-effect:

  • add second argument to pathToId(path, assert)
    • if true/undefined: will throw if id is undefined
    • if false: will create an idNode with reference count 0
      this way we can do pathToId(path, false)

if the file is in a collapsed parent and another element is selected it will now be removed from the maps.

#

it does bother me a bit that we now both have obtainId and pathToId that are creating idNodes.

dark bay
#

Just thinking :p

zenith void
#

(i made pathToNodeMap back a normal Map, was an oopsie that that got commited in the first place)

#

ok imma crash 🙂

#

see u tomorrow!

dark bay
dark bay
#

I suppose pathToId can be called outside a reactive scope and won't leak is the difference.

zenith void
dark bay
#

Good night

dark bay
#

@zenith void when ur back. I'll let ya know, you have to keep the queueMicrotask for decrementing the reference counters. Otherwise if there is only one reactive scope holding a reference to it, and the reactive scope updates, then the reference count goes 1, 0, and 1 again (cleans up too soon)

dark bay
#

We'll probably need unit tests down the road too :p

dark bay
zenith void
#

lool. that bug doesn't seem so mysterious anymore haha (first delete it from the map and then i get it from the map). i should have gone to sleep a lot earlier 🤣

Screenshot_2025-04-19_at_12.47.38.png
dark bay
#

I did ya another pr for ur pr

zenith void
#

regarding the selectionRanges: i think it's actually the wrong abstraction. you probably don't want to have [start, end][], because then whatever files gets created (or when u move files to a directory and alphabetically there are dirEnts between the) they get selected too.

#

something like [start: string, ...string[]][] is probably the way to go, where we use the first entry as the start, but all the others that fit in the range when you shift-select get added to the last array. so then when you shift-select again, we remove all but the first element and add the new ones to the last array again

dark bay
zenith void
#

i'll have a proper look in a bit 🙂

dark bay
#

It might be a bit of a cheeky solution, and a better one probably exists.

#

I was too tired today to do a pr for that slow async rename scenaro... it can be a later job.

zenith void
#

i'll make an issue for it so we don't forget

dark bay
#

Its probaby a 5 line fix though

#

The issue will have more characters in it :p

#

I'll describe a solution now.

zenith void
zenith void
dark bay
#

Ok... all good xD

# 
type IdNode = {
  refCount: number
  id: string
  idToDecRefCountOnObserve?: string
}
#

Something like anyway

#

Might need tweaks

zenith void
dark bay
#

Yeah... im not 100% sure :p

zenith void
#

Ye tricky one...

dark bay
#

Basically we know something is about to happen, and we need something else to happen once that happens.

#

Which kinda sounds like RPC

#

Or the way we talk to web workers, but not a web worker.

#

We can make a Promise that gets triggered when what we are waiting for happens, and await it.

#

And Promise get garbaged collected in js if the js engines determines they can not resolve.

#

There is always the case when what we are waiting to happen never happens (server side fs error) too i guess.

#

Gotta handle errors as well.

#

Worse case scenario is the file tree sees a rename as a delete and a add instead of a rename. Not very serious.

dark bay
#

A Later job :p

#

I just realized we do not need to do anything with the file system interface to support file systems that have their own IDs. Because their readDir can immediately convert the given path to their ID, and just use the ID.

zenith void
dark bay
zenith void
zenith void
#

Are we ever truly sure about anything?

dark bay
#

I might do a demo later to prove otherwise. There is a trick there.

zenith void
dark bay
#

I.E. readdir being made reactive against the immediate looked up ID that belongs to that filesystem rather than the path itself.

zenith void
#

quite surprised by the behavior around 0:09, wouldn't have expected it to remove the selection of .editorconfig when i click .prettierrc around 0:11

#

not 100% sure if i want to copy that

#

kind of surprising that they don't keep the selection when moving paths. we are already doing better on that front then vscode 🙂

#

imo: shift select should only ever add to the selection

#

maybe if you want to shift+erase a selection you could do shift+ctrl+select or something

zenith void
#

when deselecting it goes through all the ranges and removes it, if a range becomes empty it removes that range. which causes this emergent behavior. not sure if i like or dislike it.

#

(i m gonna check if i can get an overlay where you can see my keyboard shortcuts bc it's a bit hard to follow along)

zenith void
#

pushed it

#

it's also kind of nice that now the complexity of the shift-selection is in shiftSelectDirEntById and not anymore in the memo of selectedDirEnts

dark bay
#

I'll catch up ur messages

zenith void
dark bay
#

My phone told me not to sleep

zenith void
#

loooool

dark bay
#

Very good. I'm glad u took care of that shift+select. Its deeper than I felt like investigating :p

zenith void
#

Also tricky to do on a phone haha

dark bay
#

I didnt wrap my head around Array<Array<string>> even on a PC I would be stuck.

#

Reading the code now, to learn the magic.

#

Ahh... OK... each inner array represents the range, and u can control click to select/deselect element within that range.

zenith void
dark bay
#

I am happy with the solution

zenith void
#

and the first one in the span represents the start. it does cause some a bit odd behavior where when you deselect the next one in the span becomes the start, but i am also not hating it

dark bay
zenith void
#

still more intuitive then how vscode handles it 🙂

dark bay
#

I'm gonna sleep back. Wife asking me to put down phone :p

#

Once again. Good Work!

zenith void
#

sleep well mesh see you tomorrow!

zenith void
#

tweaked the selection further:

before

  • we were setting the selection onPointerDown
  • when we would pointerDown a dirEnt that was already selected we would not reset the selection (otherwise we wouldn't be able to drag multi-selected dirEnts)
  • but besides the dragging it's actually nice to be able to reset the selection

after

  • set the selection onPointerUp
  • onPointerUp is not called when an element is dragged
  • when you drag a dirEnt that is part of the selection it would drag/move the selection
  • when you drag a dirEnt that is not part of the selection it would drag/move that single dirEnt
  • this mimicks more closely the behavior you also see in vscode
dark bay
#

Was thinking we kind of do have a primitive for updating a record without recreating it..... render

#

It's meant for DOM, but same theory.

dark bay
#

Or vs code plugin

dark bay
dark bay
#

... now if only I can get my hands on a PC

dark bay
#

:p.... 1000 posts per month, we're almost 3000.

dark bay
#

The start of an ID backed file system that will not use IDs through the interface:

export function createFileSystem<T = string>() {
  const ROOT_ID = "root";
  const dirEnts = new ReactiveMap</*id*/string, DirEnt<T>>();

  function navigate(path: string): /*id:*/string {
    if (path == "" || path == "/") {
      return ROOT_ID;
    } else {
      let idx = path.lastIndexOf("/");
      let name = path.slice(idx+1);
      let prefix = idx == -1 ? "" : path.slice(0, idx);
      let preId = navigate(prefix);
      let preDirEnt = untrack(() => dirEnts.get(preId));
      if (preDirEnt?.type != "dir") {
        throw new Error(`Expected '${prefix}' to be a dir`);
      }
      let id = untrack(() =>
        preDirEnt.contents.find((id) => dirEnts.get(id)?.name == name)
      );
      if (id == undefined) {
        throw new Error(`path not found '${path}'`);
      }
      return id;
    }
  }
#

navigate non-reactively converts a path immediately to an ID, then we can be reactive based on the ID.

#

its a tree-based file system instead of a flat one is another difference.

#

I maybe no have all the pieces of the puzzle yet..

#

I still need to use that special insideOutUntrack as well

#

so when readdir is executed on a new path that points to the same ID, then it should not update its reactive result. (just sticks to the same reactive result based on the ID)

#

its a tricky puzzle

#

that is where this other part of the puzzle comes in:

  let readdirByIdFromPathMap: Record</*path*/string,{
    result: Accessor<Array<{ type: 'dir' | 'file'; path: string }>>,
    refCount: number,
    dispose: () => void,
  }> = {};
#

(we use createRoots to break free from the reactivity above)

dark bay
dark bay
#

I think it can be refactored into a simpler solution, but I'll get it working first.

dark bay
#

might of been pushing my luck a little bit... the console rename operation lost it's ID number.

id-test.gif
dark bay
#

I think I tricked myself into thinking it was possible. Then worked out that it was impossible the hard way. :p

zenith void
dark bay
#

@zenith void u were correct that it was impossible to keep stable IDs in a ID backed file system through a non-ID based file system interface.

#

I just had to try it first to convince myself.

zenith void
zenith void
dark bay
#

solid-fs-components looks pretty solid. I'd say its ready for release.

#

We can do more stuff in later versions

zenith void
dark bay
#

Maybe optional IDs in the file system interface that override our generated ones is the only way. (To make a remote peer renaming file cause a dom element to move instead of rerender.)

zenith void
#

Does drag and drop work on ur mobile? Can't seem to get it to work on mine, it just moves the page up and down.

#

Seems to be a known issue. Ugh why even have standards if no vendor implements them properly.

#

O it does work, I just have to hold the pointer down long enough for it to select.

dark bay
#

I think its working for me

dark bay
#

pathToId prop param is fine.

zenith void
#

I love that about building these small pieces

#

You can start puzzling them together

dark bay
#

Tempted to do a countdown so u release it :p

zenith void
#

looool

dark bay
#

Whats ur 1st version number?

#

0.1.0-beta?

zenith void
#

tbh never properly figured out what's good version numbers, often just do 0.0.1 🙂

dark bay
#

{non-backwards-compat}.{backwards-compat}.{bug-fix}

#

Maybe start at 1.0.0 makes it look less experimental

#

0 . Is sort of experimental stages

zenith void
#

let's do 0.1.0, we can go 1.0.0 once we have FileList and FileGrid and support for async fs

zenith void
dark bay
#

Not much time left to decide now 😉

zenith void
#

o shit lol haha