#ToolKitty
1 messages ยท Page 4 of 1
If u throw the beta on the end, ppl will be more forgiving if they find bugs :p
0.1.0-beta
Not sure if that works in with sem-version
let's publish ๐
published it under my @zenith void scope but changed my mind, let's go for solid-fs-components instead
I like under ur @zenith void scope though
ok then we can keep it ๐
Another thought that came to mind is... its possible do u a sync file system interface layer over an async file system interface layer via optimisic updates and rollback on error.
https://www.npmjs.com/package/@bigmistqke/solid-fs-components here we go ๐
headless components for visualizing and interacting with reactive filesystem. Latest version: 0.1.0-beta, last published: 3 minutes ago. Start using @bigmistqke/solid-fs-components in your project by running npm i @bigmistqke/solid-fs-components. There are no other projects in the npm registry using @bigmistqke/solid-fs-components.
It solves the timing issues for when we expect immediate rename.
Party Time!!!
woot woot ๐
that's neat, like an asyncToSync layer in between
something we can provide as a util
Definately
mb we should have gone go for solid-fs-primitives ๐ค
then we open ourselves for also exporting utils
Nah... it will get confused with solid-primitives :p
true
ok imma be offline for a bit (on the train)
๐ฅ on the release ๐
also if you wanna add yourself to the contributors in the package.json: please go ahead!
Nah... all good.
I switched from npm to pnpm and hooked up your released version:
A little anti-climatic when u only have 3 files :p
Your download count on npm should now be 1.
@zenith void I wonder if we need something similar to ref for FileTree to provide a controller/context that can be controlled outside the <FileTree>...</FileTree> scope.
To get access to the filetree api?
Yep
Not for me in particular. But for other users.
There is also useFiletree
But you have to call it inside FileTree children
And also useDirEnt
Yeah... could be inconvenient to control from the outside.
Example.... a scroll to selection button (that auto expands parts of the tree) made by the user on the outside.
Good night... sleep well.
Another option, we supply both:
let [ ctx, Render, ] = createFileTree(...)
and
<FileTree ...>
The former is handy for external control of state.
The later is convenient to keep UI code exactly where it sits in the UI.
Internally FileTree can just use createFileTree to avoid code double up.
It is a taste thing I guess. Not everyone will like the style.
All 3 editors united in 1 UI via file tree:
I suppose I can hook up the script editor in the same interface as well, then move onto the engine for running the level.
There is an annoying bug where if the level editor is mounted a second time, it looses its reactivity. (First mount works. But mount, unmount, remount fails.).
There is a chance I could of damaged the state of a in memory automerge doc when wrapping it in a proxy and then wrapping it in a solid store, such that it does not behave the same a second time around.
Will need to investigate further.
Found a small bug in solid-automerge. Sent an issue. It's a very minor easy fix, so all good.
I might do a PR when I get PC access, unless chee beats me to it :p
Hi @scarlet belfry ๐ ,
https://github.com/chee/solid-automerge/pull/5
Ideally for the engine/run code, I'd like to run it in a separate iframe to practice some security (prevent 3rd party code from accessing all the state, incase I have log ins down the track).
I know vite has some tooling for writting code for web workers using the ?worker prefix on import. Just need the same thing for iframes. Will need to do some more research.
Solid playground might know how to do it. I believe it loads user code in a iframe.
Or bigmistqke/repl ๐
From what I can understand repl compiles everything into a single html file that gets converted to a url via URL.createObjectURL(...) which gets loaded into the iframe via it's src attribute.
Any 3rd party libraries I guess I can use a CDN inside that.
I suppose the end user could any libraries they wish, and the levels are just assets for their code.
We can have some default systems for easy of use that they can optionally use.
I'll inspect repl's code closely. It seems to be exactly what I want.
Hmm... multiple URL.createObjectURL calls can make additional urls for the main html page to import/load from within the iframe too. Does not necessarily need to be a single output file.
(fyi have been on the road for the past days, so m a bit low on the communications rn)
Siiiick
All good ๐.... not expecting communication.
But I did worried a bit when I heard u were on a train, then u suddenly dissapeared. Thought there might of been an accident.
Yes, this is not a bad idea and I have been thinking about it too. Definitely possible.
Oo shit. No travel was long but we survived haha.
Barely, but we did!
But was relieved when I saw ur reaction to showcase.... he's alive :p
You can use the toolkit in many ways, but this is how I demo it yes.
The basic idea that the repl toolkit provides is a way to convert a virtual filesystem into objectUrls and utilities to swap the code of imports with these objectUrls.
So as it is written with signals, and you refer the objectUrls while interpolating the import paths, whenever a certain module is re-written, it will update all of the scripts that are dependent on it
So in the demo I have a html that includes scripts/stylesheets, and whenever any of the scripts it will create a new objectUrl that is passed to the iframe
But you could also just get an objectUrl for a function and run this in an iframe
Yes, exactly ๐ฏ
That's clever.
It also means that any file in the filesystem is an executable
There is not a concept of an entry point
They all are
Iirc solid playground bundles with rollup and then creates a data url from that
No, it doesn't bundle anything. There are just scripts that become data-urls that import from other data-urls.
I would love to see it in action somewhere ๐
Plus I got a guide ๐
It's made to be as hackable as possible
I had an earlier version that made a lot more assumptions
But now it's just small utilities
The filesystem api in repl, seems similar to that of FileTree
Ye, it's an fs-like interface over it
There is also createExecutables https://github.com/bigmistqke/repl/blob/main/src/create-executables.ts
If you wanna skip the interface
I think that's perfect.
Imagine if we hot module reload everything (assets and code) while maintaining game state. It would make for an awesome developer experience.
To achieve this, we can have the game and ECS system state outside of the users code.
And the user just provides system code for he ECS in their scripts.
I'll probably use a proper ECS library rather than just one I hacked together in that part.
I guess the trick would be to postMessage a url for the executable into the iframe. So the iframe can keep the state and just swap out some of the code.
@zenith void starting to use your repl ๐
The journey has started on the engine side. I added the ability to add source files in the file explorer as well... just gotta figure out the prelude to get her all working.
3.6Mb, that will be a problem... maybe I can CDN some of it.
Back down to 500KB... CDN of typescript in vite. (To keep github pages happy)
OK... I think if I CDN these two into the game/iframe code:
should be all I need to begin with
unless anyone knows a more popular ECS lib here ๐
actually... maybe this one:
https://www.npmjs.com/package/ecsy
It's more actively maintained.
Now to work out how to import CDN stuff though repl. Here is the compiled code from repl:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
var ecsy = require("ecsy"); //"https://esm.sh/[email protected]";
console.log(ecsy);
document.body.append(document.createTextNode("Hello World!"));
I am getting "exports" is undefined. I am probably doing something wrong.
maybe I need to add/tweak the ts.transpile options
that made it work ๐ :
source: ts.transpile(
source,
{
module: ModuleKind.ES2022,
}
),
Before I can go further. I have to learn some ecsy.
The plan is to get the users system code hot module loaded into ecsy systems via postMessage of the compiled urls to from their code.
That would be really sick!!
Home stretch now I think :p ... last part of the pipeline, then everything can be revisited as needed.
Ye... typescript is a big dependency.
I have wanted to try acorn-ts too. But they don't parse .d.ts files afaik and I am parsing those too for the automatic type imports
Although for .d.ts I could probably use regex instead of a full on parser. It's mostly import('this/type/of.imports') that 100% need a parser
True, curious how the browser support will be then
Although, might not be web suitable
We'll see ๐ค
a little sketchy maybe: (code inside iframe that hot-reloads user code while keeping game state)
let world = new ecsy.World();
let sources = new Map<string,string>();
window.addEventListener("message", (e) => {
let msg = e.data;
if (msg.type == "UpdateSource") {
let path = msg.path;
let url = msg.url;
sources.set(path, url);
(async () => {
let code = await import(url);
code(world);
})();
} else if (msg.type == "DisposeSource") {
let path = msg.path;
sources.delete(path);
}
});
the world holds the game state, and the users code gets imported and passed the world.
the world will keep the game state going as the users code changes
just need to de-register and re-register the users system on code update
i provided some very similar code in an earlier version of the repl-toolkit!
some glue code that you could pass into an iframe and that you then can send those object-urls to
that's really exciting
i should provide a utility for this
how solid playground handles disposal is really funny btw
they regex replace render into window.DISPOSE = render or something alike
and when re-running the script, they call this window.DISPOSE()
which breaks if u have code that relies on a variable/prop called render
For the repl-toolkit would be cool to have a std lib, that could hook into the lifecycles of the module.
Something like
import { lifecycle } from "@bigmistqke/repl/std"
let state = {...}
lifecycle(
// pass it state on cleanup
// you can run other cleanup code to cleanup this module
() => state,
// on load you can do something w the state you passed
// you can also run some other initialization code here too
prev => state = { ...prev, count: prev.count + 1 }
)
And then if you want to you could hide away this code inside the transform function and follow a specific convention
path/to/file.kitty
import.meta.lifecycle(() => state, prev => ...) would also be cool
you could also do it completely from the outside:
function injectScript(element: HTMLIFrameElement, url: string){
const script = element.contentWindow!.document.createElement('script')
script.type = 'module'
script.src = url
element.contentWindow!.document.head.appendChild(script)
return () => element.contentWindow!.document.head.removeChild(script)
}
something like this
Quickly discover what the solid compiler will generate from your JSX template
Quickly discover what the solid compiler will generate from your JSX template
That all can work... I'll have a bit more of a play when I can get PC time.
Here where I got to in Termux before I fell asleep:
let world = new ecsy.World();
let sources = new Map<string,{
url: string,
code: Promise<{
init: (ecsy: Ecsy, world: ecsy.World) => void,
onCleanup: (ecsy: Ecsy, world: ecsy.World) => void,
}>,
}>();
window.addEventListener("message", (e) => {
let msg = e.data;
if (msg.type == "UpdateSource") {
let path = msg.path;
let url = msg.url;
let code = import(url);
sources.set(path, {
url,
code,
});
(async () => {
let code2 = await code;
code2.init(ecsy, world);
})();
} else if (msg.type == "DisposeSource") {
let path = msg.path;
let node = sources.get(path);
if (node != undefined) {
sources.delete(path);
(async () => {
let code2 = await node.code;
code2.onCleanup(ecsy, world);
})();
}
}
});
Then when code is updated, it will need to trigger both "DisposeSource", and "UpdateSource"
I am hoping to have it such that I can have two browser windows open on a PC linked via automerge. One showing the map editor or scripts, the other showing the game. And be able to edit the map or scripts during gameplay without resetting the state of the players or the monsters etc. Good for testing jump distances for placing tiles.
really sick
I will have to do some sort path conversion like that I think. So it knows which source files to send to the iframe vs those that are just utility code for other code.
- conversion -> convension
I keep typing the wrong words :p
Did it again lol... convention
the outside:
createComputed(() => {
let iframeElement2 = iframeElement();
if (iframeElement2 == undefined) {
return;
}
let iframeElement3 = iframeElement2;
let iframeWindow = iframeElement3.contentWindow;
if (iframeWindow == null) {
return;
}
createComputed(mapArray(
sourceFilesAsArray,
(sourceFile) => {
createComputed(() => {
let filePath = "user_code/" + sourceFile.name();
repl.writeFile(filePath, sourceFile.doc.source);
onCleanup(() => {
repl.rm(filePath);
});
createComputed(() => {
let url = repl.getExecutable(filePath);
iframeWindow.postMessage({
type: "UpdateSource",
url,
});
onCleanup(() => {
iframeWindow.postMessage({
type: "DisposeSource",
url,
});
});
});
});
},
));
});
oh... iframe.contentWindow is null for some reason. I need it for passing messages into the iframe.
ahh... gotta wait until iframe is fully loaded.
... or just onMount to wait until the iframe is actually IN the dom :p
The slow speed of ts.transpile won't matter so much if it's executed in a webworker and is throttled.
Interesting... the file explorer is working on the PC, and my phone, but not on my daughters tablet, and she is on the same version of Chrome as my phone.
Bit tricky to grab the error. The tablet is locked down on a kids account.
Could of just been how I used it too. Like a timing issue with the async stuff.
But... it's working on my daughter 2nd tablet that is also running same version of Chrome lol
I won't worry about that for now.
Write once... debug everywhere ๐
On the 1st tablet the css doesn't even look right. So could be a tablet issue. (Tablet too old for the version of Chrome maybe)
I could make uncaught exceptions bring up a UI dialog detailing the crash.
@zenith void The code inside writeFile should be wrapped in untrack because writeFile is a side effect / action.
Same deal for all side effects / actions that we do not react to a value for.
slowly getting there:
console.log not reaching console from iframe which seems strange (security thing?). But alerts get through.
I think iframe's onload events fires too soon. I think I'll just get the iframe to postMessage back when its fully loaded. (DOMContentLoaded event from inside the iframe)
This does the trick: (inside iframe)
let onDOMContentLoaded = () => {
window.postMessage("ready");
document.removeEventListener("DOMContentLoaded", onDOMContentLoaded);
};
document.addEventListener("DOMContentLoaded", onDOMContentLoaded);
console messages from iframe now showing up.
One cool thing, u can untick "Show Game" when trying while-loops so ur browser does not lock up.
Ohh... dam... its only working on FireFox (pc version) at the moment. User scripts failing on my phone in chrome. Will need to investigate more.
Working on Firefox for mobile, but not chrome for mobile ๐ ... will need to find something diffrent for chrome.
Note to self... use chrome when I am on desktop, not FireFox
ah right.... load works for chrome, but not firefox... and the other method works for firefox
I just need to switch depending on the browser.
weird... now the load event method is working for both browsers, firefox and chrome... I'll switch over and leave it alone.
Yes, exactly
Wanna make a pr?
I was having this waitForLoad function that resolves a promise once the load-event is ran
Also handy if you are injecting script/stylesheet-link elements
Let's gooooo
Looool, too real
Yes, it also only needs to transpile the files that are changed.
Currently it also retranspiles the dependent files (for swapping out the import-maps), but that could probably be cached.
All working good so far... next challenge is to get additional types into monaco editor for ecsy and pixi
Or just ecsy... pixi can be hidden under the hood.
Afk for a bit
Back sort of
Ohh lmao.... ecsy is a dead project that does not work on es5+
https://github.com/ecsyjs/ecsy/issues/274
Time to switch ecs libraries.... to think ecsy was written by Mozilla.
This one does look promising:
https://www.npmjs.com/package/ecs-lib
Just the 15 downloads per week, makes it feel less popular.
This might be nice to:
https://github.com/NSSTC/sim-ecs
ecs-lib seems the most user friendly. Not the fastest, but the most friendly to use.
I have a utility in the toolkit to download types from esm.sh: https://github.com/bigmistqke/repl/blob/main/src/download-types.ts
๐๏ธ building blocks to transpile and execute code in-browser. - bigmistqke/repl
And a utility to bind it to monaco too: https://github.com/bigmistqke/repl/blob/eaa0273881029296402afa3e65505a7e81dea605/src/monaco.ts#L9
Very cool... I will use that.
It's a shame ecsy doesn't work on es5+
But I think ecs-lib will be fine.
I tried making a mini game already with ecsy then hit a wall already ๐
Mozilla abandoned it, by the looks of it.
Looks like they tried to dodge the new-operator to lower gc pressure. But it is an error on newer js engines.
Yup seems to be archived
Another cool thing.... if u disable predictive text in android keyboard.. it makes monaco work really well on android.
Kicked off computer now... will play with it on the phone :p.
ecs-lib is in now, except the types in monoco... will get to that later.
Oo good thing to know. Is it a system setting thing?
Nice! Good to know ๐
Here is some user code for a test:
export function init(ecs: any, world: any) {
type Ticks = {
ticks: number;
}
const TicksComponent = ecs.Component.register();
const ticksCmp = new TicksComponent({ ticks: 0, });
const ticksEntity = new ecs.Entity();
ticksEntity.add(ticksCmp);
world.addEntity(ticksEntity);
class TicksSystem extends ecs.System {
constructor() {
super([
TicksComponent.type,
]);
}
update(time, delta, entity) {
let ticks = TicksComponent.oneFrom(entity).data;
ticks.ticks++;
console.log(ticks.ticks)
}
}
world.addSystem(new TicksSystem())
world.update()
world.update()
}
export function onCleanup(ecs: any, world: any) {
}
Now I need to find a way to reuse classes from init, instead of recreating them. Otherwise it will keep making new/additional systems and component types instead of swaping existing.
Temporary work around:
export function init(ecs: any, world: any) {
let last = window["last"];
let TicksComponent;
if (window["last"]) {
TicksComponent = last.TicksComponent;
world.removeSystem(last.ticksSystem);
} else {
TicksComponent = ecs.Component.register();
const ticksCmp = new TicksComponent({ ticks: 0, });
const ticksEntity = new ecs.Entity();
ticksEntity.add(ticksCmp);
world.addEntity(ticksEntity);
window.setInterval(() => world.update(), 500)
}
class TicksSystem extends ecs.System {
constructor() {
super([
TicksComponent.type,
]);
}
update(time, delta, entity) {
let ticks = TicksComponent.oneFrom(entity).data;
ticks.ticks+=100;
document.body.innerText = `${ticks.ticks}`;
}
}
let ticksSystem = new TicksSystem();
window["last"] = {
TicksComponent,
ticksSystem,
};
world.addSystem(ticksSystem)
}
export function onCleanup(ecs: any, world: any) {
}
That allows me to update system code while maintaining game state.
Demo: ๐
https://youtu.be/Tstkya5Gbuw
Cheers ๐ป... the code could use a makeover. But it has a really good start.
Like that init to avoid global window variables.
Maybe I need a state per code that can be kept when the code refreshes.
maybe something like this could be handy for that?
Definitely.
In the same code above. I've got use the same Component classes but swap out the system classes.
Like I need code that only runs once. And code that runs every change to system code.
Separate files maybe.
that might be the cleanest way
Yeah... like component definition files, and system files.
Not as good as Java hot module reload. But not bad. :p
Although... Java is 1 class per file... maybe not so different.
conventions!
Ahh... classes don't really have an identity in a sense in js.
You write:
class A {}
. . .
class A {}
Both those classes, although the same name are really a different class to each other.
Like an annoynimous class gets generated for each class keyword and gets assigned to variable A.
That's what prevents true Java like hot reload.
So separate files is a must.
i think to get closer to "true hot reload" a convention like
export default {
hello: "world",
someMethod(){},
}
// and/or
export default (props) => ({
name: props.name,
someMethod(){},
})
``` mb?
exactly, think filebased
With default... the source pathname is the identity. So that works.
exactly, or u can even get that file id
True
Kinda like a vite.config.ts in style.
random side idea: would be funny to use file-based routing as a scene manager
Would still be performant for an in memory file system.
Even a file system abstraction over an ECS scene would allow <FileTree> to be recycled as a scene graph.
Each entity is like a file. The components are like the file's data.
Entities can have parent/child entities too via special parent/child components that point to other entities.
Maybe an entity is a folder then.
yes exactly, that's where i started in my head too
i don't know if it's necessarily practical, but it is kind of elegant
to just have 1 tree representation
and the filetree could filter out paths that don't end with .kitty or something
it could be imports?
you could specialcase imports too with ?modifiers
cool thing too if everything would be a path is that you would be able to get autocomplete in monaco when writing the import
but it probably does get pretty unwieldly with complicated scenes
and you would want to have other selectors too probably, for more dynamic stuff
Looks like Unity's scene graph just has entities:
https://docs.unity3d.com/Manual/Hierarchy.html
Then they have a separate inspector for the components of the entity.
Unity is ECS based too.
Inspector:
Baby step :p... a type schema on the cmponents can help auto generate the UI for the inspector.
i was playing around w a three editor for doing vtubing a while ago and there i was handling it like this
ig that's where i got this idea from
but your idea is probably better for your usecase
Thats pretty awesome to be honest.
I can visualise a factory that converts components to extensions.
Moew.... what a name :p
In ECS, components are mostly raw data.
Then u make systems which in turn may use factories to make sense of the components attached to entities.
Systems can also provide other views/abstractions over the top of the entities.
Systems can be pretty much be anything really.
i'm reworking worker-proxy so it can proxy deep, so you can do
// worker.ts
import { registerMethods } from "@bigmistqke/worker-proxy"
class Logger {
log(...args){ console.log(...args) }
}
export default registerMethods({ logger: new Logger() })
// main.ts
import { createWorkerProxy } from "@bigmistqke/worker-proxy"
import Worker from "./worker?worker"
import type Methods from "./worker"
const proxy = createWorkerProxy<Methods>(new Worker())
proxy.logger.log("hallo")
I imagine worker proxy can be used with iframes too. Both iframes and workers use postMessage.
One difference (i think, not certain), is iframe uses the same main thread where as a worker spawns a new thread. Because a while (true) {} in an iframe seems to lock up both the iframe window and the main window.
I wonder if vite has some tooling/tricks to generate a src url for an iframe, kinda like it does for workers.
i already have api that you can pass a MessagePort and create a proxy from that
so you could do it like that
i'm almost thinking of redesigning the whole api around MessagePorts, because it's also nice that it keeps all the communication around the proxy contained within a single port
allowing you to still do regular postmessage stuff
I wonder if this is valid.
let preludeUrl = new URL("./prelude.ts", import.meta.url);
.
.
.
<iframe src={preludeUrl}/>
So I can use the same compilation enviroment as the main app as the prelude.
a i see what u mean
i think that might work
or
something like ?url?
Does ?url return a link to the compiled source?
import preludeUrl from "./prelude.ts?url";
I think so, that it makes a chunk
Thats very handy.
// iframe.html
<script src="./iframe.ts"/>
// iframe.ts
import { registerMethods } from "@bigmistqke/worker-proxy"
registerMethods({
ping(timestamp: number) {
console.log('ping from iframe-worker', timestamp)
},
})
// main.html
<iframe src="./iframe.html"/>
<script src="./main.ts"/>
// main.ts
import { createWorkerProxy } from "@bigmistqke/worker-proxy"
document.body.querySelector('iframe').addEventListener('load', () => {
const iframeProxy = createWorkerProxy(iframe.contentWindow)
iframeProxy.ping(1)
})```
Does vite handle this?
<iframe src="./prelude.html"/>
And transforms prelude.html from main app?
Thats easier still.
yup
Vite is more capable than I give it credit for
vite is amazing
I was compiling the prelude with ts.compile in the browser ๐
When it's a static compiled asset.
what's a prelude?
It's like a std
For a runtime environment
Standard runtime library
In my case the prelude provides the ECS library with its shared state and the hot code reload.
gotcha
I should have a chance to have a hack later today. A lot of opportunities open up now that I realise what vite can do :p
I think I might find ur worker proxy useful to for communicating with the game in the iframe.
Like pause/play button for the ECS simulation.
Frame step, slow down, etc.
For frame by frame debugging
would be sick
Might end up depending on 5 bigmistqke projects ๐
Ur resizable split divider is also handy.
you have been using that one too?
i have been working on a fix for that one
Not yet... but feeling lazy to do my own.
didn't work properly on mobile
I see
that's the benefit of coding on mobile: u actually fix mobile bugs
True
corvu has a really nice one too
I'll be using corvu calendar soon. Gotta try to remember that name "corvu", I keep forgetting. (For that monday . com app)
this feature is so cool where you can drag 2 corners at the same time
no idea how they did that
Ohh... neat
i assume: math
Yeah :p
Work out what's under the mouse with math.
Although pointer events bubble. Maybe it is possible with just dom events.
There is pointerover and pointerenter events. One of them triggers on itself as well as its child dom elements.
Can't remember which one.
The dividers might overlap at the junctions.
Easier to not reinvented the wheel though :p
Jasmin has done a wonderful job on corvu.
oh neat..., each corvu ui component is a separate lb install
right... so monaco editor does not honer the size of its parent div when its parent div changes size.
ahh add automaticLayout: true to editor config
Oh dear. ?url worked locally in development mode. But didn't work when compiled and uploaded to gh pages. ๐
It made a url to a non-existing file in release.
And I got kicked off pc :p
Got a tiny bit done. Just the resize split divider hooked up from corvu. And the automerge filesystem syncing itself with the monaco vfs.
(So monaco can import other files without error {without red squiggly line})
Might be able to fix that ?url later in Termux with some luck.
The ts.compile failed to resolve the local imports (between source files). Will need to investigate that later too.
Maybe ?worker&url instead of ?url.
Ohh... looks like vite is thinking a .ts file is a video/mp2 file...
This feels familiar :p... I'm sure I have run into this problem before.
From the Kiwi browser for android:
in the vite plugin of worker-proxy i do ?worker_file&type=module
Thinks I'm loading a video ๐
it still loading the .ts instead of js?
is it at least making the chunk?
I believe so.
I can see two index[hash].js files.
I assume one is from prelude
It inlined the data too data:...
Bizarre
Will give that a go.
This is more hopeful now:
It's doing something, but doesn't like how I am importing Ecs. Yet that's the way the documentation uses it.
maybe try import * as Ecs from "ecs-lib"?
Might be different target
Worked when I get ts.compile to compile the prelude.
And worked in dev mode, but not production :p
It does export default I am pretty sure.
mm strange
Probably the module type when I am doing a worker. Different from module type of main app.
I'll investigate over the weekend I think.
i'm going to rework worker-proxy to rpc-proxy
import { onRequestRPCPort, createRPCPort } from "rpc-proxy"
// worker.ts
const methods = {
hello(){ console.log('hello') }
}
onRequestRPCPort(() => createRPCPort(methods))
type Methods = typeof methods
export type { Methods }
// main.ts
import type { Methods } from "./worker.ts"
import { requestRPCPort } from "rpc-proxy"
import Worker from "./worker.ts?worker"
requestRPCPort<Methods>(new Worker()).then(port => {
const proxy = createRPCProxy(port)
proxy.hallo()
})
That makes sense
for iframe
// iframe.ts
const methods = {
hello(){ console.log('hello') }
}
onRequestRPCPort(() => createRPCPort(methods))
type Methods = typeof methods
export type { Methods }
// main.ts
import type { Methods } from "./iframe.ts"
import { requestRPCPort } from "rpc-proxy"
const iframe = document.querySelector('iframe')
iframe.addEventListener('load', () => {
requestRPCPort<Methods>(iframe.contentWindow).then(port => {
const proxy = createRPCProxy(port)
proxy.hallo()
})
})
Hmm... worker is not bad name though... messageport can sent more than json (Uint8Array)
would be cool to have adapter for https/web-workers too
Definitely.
LOL... most my testing last night was for naught... I pushed to the wrong branch for gh pages, so I was always looking at the old version
(working local but not on server testing)
I went back to ts.compile of prelude for now. Will work out the vite way later.
Heres where I render the users source code my automerge file system to your repl file system:
It could be that "user_code" folder I am throwing it in to avoid conflicting with the prelude files.
Ohh... seems to be working. (Deleted last commit over thing to investigate.)
Just a bit more planning / restructuring and should be all good.
I'm thinking I actually dont need life cycle management in user code.
Bcuz the user can make an init.ts that the other ts files dependable and that holds the global ECS state (if they want)
init.ts does not reload if it is not edited.
system1.ts, system2.ts can be edited and reloaded without loosing game state (init.ts)
Systems can be registered so they can be auto removed/redded to the world when their code changes.
Maybe onCleanup should be kept, but made optional (not required)
I was thinking for another approach to either createFileSystem or createExecutables
const fileUrls = createFileUrls(fs.readFile, { ts: (source) => ... })
- bikeshedding: rename
executablestofileUrls - you pass it a function that gets a path and returns a source and an object of extensions that can run transformations, run side-effects and return transformed code
- this way i don't need to write/maintain another FileSystem implementation for the repl toolkit (could just hook any reactive fs to it)
- there is no need to sync two filesystems
- the file-urls could be initialized lazily (only the file-urls that are getting .get are being created and tracked)
another bikeshedding: i m thinking about eval-utils for rename of @bigmistqke/repl wdyt?
i love that u can build up hmr from within the kit
i think the funny part about the kit is that it always poses the question of where do i draw the line of where the kit's runtime ends and my bundled code begins
like it would be so sick if you could have like a secret developer shortkey that you could click and that it then opens up your site within a repl
Would be interesting for sure.
I'll let u decide on that :p ...
and that you would have a secret shortkey to then open up that repl inside another repl
have been thinking about a name for such a long time already ๐
Developers worry about security when the see the word "eval"
Even though it does not actually use "eval"
that is true
i mean, maybe they should be a bit careful when using the lib though
but not eval-type careful
scope works a bit differently
let var1 = "hallo"
eval("console.log(var1)")
``` will log "hallo"
```tsx
let var1 = "hallo"
createFileUrl("console.log(var1)").then(import)
``` will error something about `Error while accessing undefined variable var1`
so ye good point about eval, back to the drawing board
Both have access to session cookies.
yes, and window globals
i wonder in an iframe too?
I think its safe. Not 100% sure.
Web worker definately safe.
There are properties u can set on an iframe to tweak its security.
An iframe cross origin is safe.
(To prevent advertising from stealing session cookies)
There looks like their is a sandbox attribute we can set on an iframe.
neat
We thought of the same thing at the same time :p
a but is experimental
Yep
Web workers safest. But does spawn another thread.
There is a limit on the number of web workers a page can load.
I think it was a small number too. Like 5.
Sorry.... its 20 for FF. 60 for Chrome, 16 for Opera :p
Knew it was small.. But more than 5.
U can share the one web worker for all ur 3rd party script. But then there is the possibility of cross script attacks.
A 3rd party script accessing the session cookie of another possibly licensed 3rd party script.
No iframe limit though. Just can make many invisible iframes.
Tempted to just let the user use the reactive ECS I have existing in the host app. The "Kitty Demo" link in ToolKitty already uses it to render a tiled map to pixi via a virtual list of virtual lists.
Been a long time. But I can reuse this to render the users map.
damn that's not a lot
Easy to exceed if a few libraries all need web workers to operate.
yeees
this
true :p
Not aiming for self hosting though.
But it is interesting.
An app that can u can update itself from inside itself.
imagine a dev environment where you start with everything is in-browser runtime and then at any point in the development you can build from a certain part of the tree and then from beneath that point it is bundled
And do pull request request to git for the app from inside the app :p
What would be the smallest project containing an editor allowing u to edit its own code?
And possibly hot reload itself
Like a bootstrap of sorts.
nice question
Even if it used evil eval
And was just js
I'll ask AI xD
Our smallest example:
https://g.co/gemini/share/f6e73a712b3f
Not quite right. You loose the ability to edit the page after the first edit.
Ideally we'd want a boostrap starting off with monaco editor and solid :p
And load/save itself to/from indexed db with an import/export.
source maps?
or esm.sh imports
the theoretical smallest example is quite nice too tho
good exercise
Its basically your <FileTree> demo, but with extra features ig.
https://bigmistqke.github.io/repl/ has an editor with solid and the toolkit, using the html tag template literal
Not quite mobile ready ๐
lol
2 character width limit on editing ๐
needs a termux session
loool
I wonder if vite's ?url allows us to send any transformed libraries/utilities from ur host app to an iframe
Sharing the same js between host app and iframe.
I'll ask Termux ๐ (not in English though)
true
in the repl demo i simply build the library and include it into the repl fs
vite-plugin-dts expects a lib target though.
import toolkitDeclaration from '../lib/repl-toolkit.d.ts?raw'
import toolkit from '../lib/repl-toolkit.js?raw'
const typeDownloader = createMonacoTypeDownloader({
target: 2,
esModuleInterop: true,
})
typeDownloader.addDeclaration('@bigmistqke/repl/index.d.ts', toolkitDeclaration, '@bigmistqke/repl')
const transformJs: Transform = ({ path, source, fileUrlRegistry }) => {
return transformModulePaths(source, modulePath => {
if (modulePath === '@bigmistqke/repl') {
return localModules.getFileUrl('repl-toolkit.js')
} else ...
})!
}
// Add file-system for local modules
const localModules = createRoot(() =>
createFileSystem({
js: {
type: 'javascript',
transform: transformJs,
},
}),
)
localModules.writeFile('repl-toolkit.js', toolkit)
Got type gen on non-lib package:
In theory I can use those along with vite's ?url import.
?raw on the .d.ts files of course.
A little tricky, bcuz the build files will depend on the result of the build files. (Has to bootstrap itself a little)
Easier to just copy the .d.ts files back into the source. They should not change often.
i had an idea for a vite-plugin a while back: https://github.com/bigmistqke/vite-plugin-raw-directory
import map from './directory?raw-directory'
// {
// './src/index.ts': 'console.log("hallo")',
// './index.html': '<body>hallo</body>'
// }
console.log(map)
``` but i think it was broken last time i tried it out.
That's pretty cool. Only 50 lines of code. Would be easy to fix if broken.
There is also:
https://isomorphic-git.org/
Might need like a:
- app compiled as bootstrap
- source code for self app bundled with the app.
- indexed db storage for replacement source code and replacement bootstrap (compiled from code in browser)
The original bootstrap can bypass loading itself and instead load the bootstrap from indexed db if one exists.
Or separate initial loader that chooses between original bootstrap and bootstrap in indexed db
Can use that git the browser lib for final storage when the user is happy with their changes.
Tricky part would be bundling all the 3rd dependencies of the app when compiling in the browser. Unless we're full on esm.sh
Is this correct?
{
let model = monaco.editor.createModel(
ecsWorldJsCode,
"javascript",
monaco.Uri.parse("file://EcsWorld.js")
);
onCleanup(() => model.dispose());
}
{
let dispose = monaco.languages.typescript.typescriptDefaults.addExtraLib(
ecsWorldTypeDef,
"EcsWorld.d.ts",
);
onCleanup(() => dispose.dispose());
}
The monaco editor does not seem to be finding EcsWorld when I do an import.
here is how I am getting the transformed code and the type def:
import ecsWorldTypeDef from "../../dist/ecs/EcsWorld.d.ts?raw";
import ecsWorldJsUrl from "../ecs/EcsWorld?url";
let ecsWorldJsCode = await fetch(ecsWorldJsUrl).then((x) => x.text());
Go it, adding this makes it work:
monaco.languages.typescript.typescriptDefaults.setCompilerOptions({
paths: {
"EcsWorld": ["./EcsWorld.js"],
}
});
I should bundle it all together as a library for the users code, so they can do 1 import instead of many imports.
oh tricky.... EcsWorld imports a bunch of other code that must be transferred over too.
might need to do a separate lib for stuff to expose to user
Actually... no... iframes can import all the same files as the host app. So I should be fine.
a dependency of EcsWorld in dev mode:
Uncaught (in promise) TypeError: Error resolving module specifier โ/node_modules/.vite/deps/uuid.js?v=dbf5c761โ.
maybe I need to configure the iframe a little to allow accessing certain paths
It could be that a URL.createObjectURL when loaded in an iframe is not considered same origin maybe.
If that's the case, then I just need to go back to loading the iframe with a url generated by vite.
Or it might be same origin. But I need to have the host name in the module path transformation.
Success:
const currentUrl = window.location.href;
const urlParts = new URL(currentUrl);
const hostnameWithPath = urlParts.protocol + urlParts.host + urlParts.pathname;
function createRepl() {
const transformJs: Transform = ({ path, source, executables }) => {
return transformModulePaths(source, modulePath => {
if (modulePath == "EcsWorld") {
return hostnameWithPath + "src/ecs/EcsWorld.ts";
}
works
actually... that might only work in Dev mode. I need to change it for production mode.
This adjustment should do the trick for production mode:
function createRepl() {
const transformJs: Transform = ({ path, source, executables }) => {
return transformModulePaths(source, modulePath => {
if (modulePath == "EcsWorld") {
let tmp = ecsWorldJsUrl;
if (tmp.startsWith("/")) {
tmp = tmp.slice(1);
}
return hostnameWithPath + tmp;
}
this is really cool. I can call host application code from within the iframe without the need to ts.transpile it.
A lot of possibilities are opened up.
Maybe if I do a bunch of exports from the host app's index.ts. Then just import that into the user's code as a tidy lib.
actually I might be a "export.ts" file in the main host app and use it.
or call it "lib.ts", might be more sensible.
I can't get over just how cool this is ๐
for the type declaration files, I just have the build writing the type declarations back into the source code. And some initial type declaration files in the source code as a bootstrap. Because the source files depend upon the type declaration files as well.
chicken and the egg scenario
ahh... working in dev mode, but not production ๐ ... this battle again.
The problem I am getting now in a production build is an empty lib.js chunk, bcuz the exports in it are unused (in the host app). I need to disable that optimisation from rollup for that one file.
Was hoping to disable tree shaking for the single lib.ts file with a rollup plugin like so:
(lib.ts contains the text export *)
Guess I'll just disable tree shaking on everything for now.
Disabling treeshaking on everything still produced an empty lib.ts chunk ๐... up hill battle.
I think the trick will be to split ToolKitty into a library, and a web app. The web app will pretty much just launch the library. That way I can prevent exports from lib.ts from getting stripped.
Was in the process of making Tool Kitty a library, then consuming it in an index.html to avoid lib.ts from getting stripped of its unused exports. But then got kicked off pc :p
So close haha
To be far I was on the pc for a many hours. :p
Working... but only in dev mode... production getting an empty lib.js chunk.
I am excited, because the program is kinda consuming itself as a library. Giving itself full reflexion. :p
So strange ๐ค maybe good to try it out on a minimal reproduction, away from all the clutter of toolkitty, isolate the issue to vite only.
I think I know the issue. Its the optimisations stripping out what is unused.
I tried faking a use of lib.ts. But then it made two lib.js chunks, one empy, one with the exports. But it ended up using the empty one in the important spot.
I am also very hyped ngl. Cool that the first usage of the toolkit is something that is not a typical playground. Makes me happy about the direction I took w the kit, making the pieces smaller and less opinionated.
Here https://github.com/rollup/rollup/issues/4090#issuecomment-847716887 they also set map: null
2 line app ๐
Lol
Cheers ๐บ ... I'll give that a go when I'm stuck.
Now the chunk is missing ๐.... (ToolKitty consumed as a library approach [in Termux])
I need to manually create the chunk in the app, and feed it back to the library through launch as a parameter.
Kinda feels like a Y combinator.
(The library feed to itself)
maybe you have to import('tool-kitty').then(({ launch }) => launch())?
static imports won't create chunks by default
Currently trying this:
Ah... I see.
I'll try ur manual chunking
Manual chunk looks more promising:
I wonder if there is a reliable way to get the url to that manual chunk. (Looks like it's gonna work)
Ohh.....
const moduleURL = import.meta.url; console.log(moduleURL);
The library can work out its own URL
lol this stuff s so meta
Ohh dear... the exports still got optimised out:
When loading it in iframe
I probably gotta pretend to use them in the launcher app
so stoopid
The optimiser is clever :p
what about: simply watching and building it to a static asset in the /assets folder and importing it from there?
they don't touch that folder iirc
I tried this to ensure EcsWorld is not optimised out:
But still when I imported it from inside tool-kitty it was unable to find EcsWorld
Definitely there in the chunk.
Only thing I can think of now is, maybe it is a conflicting module type.
E.g. ts.transpile module type might be incompatible with the module type used for vite bundling.
Vite is targeting "esnext". I'll check what ts.transpile is targeting.
Ops... it wasn't esnext for the toolkitty lib. Only the host app.
No luck yet :p... will have to sleep now. Good night.
Ohhhh.......its working, but its no longer called EcsWorld due to minification.
All the names have been minified.
Lol that's stupid
Gotta figure out how to disable that for 1 file :p
minify: false globally did not disable those sort names
If I go import { E as EcsWorld } from "prelude", then its working, but not what we want.
Ohh ohh.... multiple entry points should fix this. (In rollup config)
Maybe I didnt need to turn toolkitty into a lib afterall
I think I got it... rewinding kitty back to non-lib, and adding lib.ts to rollup entry points.
Its working!!!!! ๐ ๐ ๐
Works in gh pages too... what a journey.
lesgooooooo
Now I'm too burnt out to continue. :p ... will to take a break and continue later.
I suppose next is some systems in the prelude. That u can optionally import and attached to world. (E.g. the pixi tilemap renderer)
And I should make all systems optional, just incase the end user does not want to use pixi at all. (They may wanna use an svg, a canvas or even three js)
From the users perspective in user code. It is ToolKitty as a library.... that is running in ToolKitty ๐
Now I am dying to use a PC again after I said I would take a break ๐
Well not really.... we're unable to edit all the code for ToolKitty in ToolKitty. Not yet self hosted. :p
And its really TypeScript that is self hosted. Not ToolKitty.
work in progress integration of the pixi render:
This will be interesting...
I need an automerge network adapter to keep the asset file state (the map) in sync between the host app and the iframe.
I suppose I can use broadcast adapter, but there was that bug we ran into. Where the broadcast adapter prevented the web rtc adapter from working.
Ig... what if I add their broadcasting network adapter late, instead of straight away. I believe the bug was a timing bug.
I can just work with the bug ๐ ๐
Next challenge get the Automerge file system into the prelude.
That will be what allows us to edit a map while playing a map. (Live update, that avoids player monster reset)
Which is actually a piece of cake, bcuz ToolKitty uses itself like a library now.
Also I currently need to pnpm build twice to do a build. Once to get the type signatures for prelude, a second time to use them. I wonder if there is a way to do it in one pnpm build instead of two.
Maybe if I can load the generated .d.ts at runtime from the build instead of importing it raw at compile time.
also would be nice to bundle all the type declarations into a single file to pass to monaco
Getting there...
Got the automerge file system, texture atlases and images reaching the user's scripts.
If I wanted a level with destructible blocks (like Terraria), then I'll need to store the diffs between the loaded map and the edits.
One thing I would have to figure out is how solid brakes its libraries into folder like namespaces:
E.g.
u install solidjs, you can access createMemo from solidjs, you access createStore from solidjs/store. It's like it has a store namespace to break it up a bit.
Here is how you currently load a level in user code:
let levelEntity = world.createEntity([
prelude.levelRefComponentType.create({
levelFilename: "l1.json",
})
]);
Seems pretty straight forward.
Then it's upto the systems to render it, do collision detection, etc.. all tucked away so the user does not have to worry about it.
All systems are opt-in, and the user can make their own systems if they wish.
you do it by defining exports in your package.json:
{
"exports": {
'.': { "types": ..., "import": ... },
'./store': { "types": ..., "import": ... },
}
}
``` [solid's exports-property is pretty advanced lol](https://github.com/solidjs/solid/blob/e8e228088785691e634872cd35470f5e43c19050/packages/solid/package.json#L45C3-L49C12)
Can't wait to see all the pieces come together!
Some of it I have to use the exact same library/module of a library comming from lib.ts in user code. E.g. so solid in user code can talk to solid in lib.
Needs to be the same module for same global state if any.
pixi js has global state for loaded assets. Same deal pixi, it needs to tunnel through lib.ts instead of esm.sh
And to figure out a way to get .d.ts files from the bundled libraries.
But will still work with red squiggly lines in monoco. It's just a UX thing.
Pixi only just because there is a pixi render system in lib.ts.
did you have a look at https://github.com/bigmistqke/repl/blob/main/src/download-types.ts#L136 ?
it makes use of X-TypeScript-Types header
is a deno thing that esm.sh supports
and there is https://github.com/bigmistqke/repl/blob/eaa0273881029296402afa3e65505a7e81dea605/src/monaco.ts#L9 to hook that util to monaco by extending tsconfig paths with the results of those fetches
That's a good idea. I'll probably do that. I did see it, but thought there might be an easy way to do it straight of the file system itself. But I guess that would require us to write plugin.
We have typescript lib that can parse .d.ts files to find other referenced .d.ts files. But it's a pain.
As long as we can version lock esm.sh for type declarations should be fine.
That is pretty much what the utility does too
I might have a go at making a plugin for it. It will be a good learning experience. (Using node fs to go local package type hunting)
I've never made a plugin for vite before.
May not need too. Think I saw a plugin.
A now I gotcha, you want to do something like import declaration from "test?dts" or something alike?
Yep. Exactly.
I saw this one here:
https://www.npmjs.com/package/vite-plugin-dts-bundle-generator
It looks like it can produce a single .d.ts file for all exported stuff.
That will be handy for types for prelude
There's also a bundledPackages config for vite-plugin-dts. If it does what I think it does, I can get type declarations out for the host app versions of solidjs and pixi.
i use that one often when building my own declaration files for my libs: p.ex when building the library files for the repl-toolkit demo
importedLibraries
in the monaco-type-downloader utility I don't bundle the declaration-files, but walk up the imports, add all the files to monaco's vfs and alias all the separate declaration-files to the tsconfig path-property
it's not the fastest as it's waterfalls all the way down and can become a lot of requests, but even with a massive library like threejs it's still reasonably fast.
it uses typescript to walk the module-paths (it uses that transformModulePaths-util that is also used when transforming typescript to map the imports)
for declaration-files we could probably also get away with a simpler approach as it does not have dynamic import(...), but then again, you are probably already importing typescript anyway if you want to import declaration-files.
Thanks for your help. I will have a swing at it tomorrow.
Can't wait for solid 2.0. I wanna get rid of all that async signal simulation noise.
yes please
function createRefCounter<T, U>() {
const map = new Map<T, { count: number; value: U; abort(): void }>()
return function ({
key,
constructor,
signal,
}: {
key: T
constructor: () => { value: U; abort(): void }
signal?: AbortSignal
}) {
signal?.addEventListener('abort', () => {
const result = map.get(key)
if (!result) return
const count = result.count - 1
if (count > 0) {
map.set(key, { ...result, count })
return
}
map.delete(key)
result.abort()
})
const result = map.get(key)
if (!result) {
const { value, abort } = constructor()
map.set(key, { count: 1, value, abort })
return value
}
map.set(key, { ...result, count: result.count + 1 })
return result.value
}
}
``` kind of a nice ref-counter.
usage:
```tsx
function createRequester(messenger: Messenger, { signal }: { signal?: AbortSignal } = {}) {
return requesterRefCount({
key: messenger,
signal,
constructor() {
const abortController = new AbortController()
...
messenger.addEventListener(
...,
{ signal: abortController.signal },
)
return { value, abort: abortController.abort }
},
})
}
It's like a bed time story.
i love that
terminal is so cozy too
i really do get the appeal of people only wanting to spend time in there. it's very nostalgic and cozy. makes everything u do like a text based rpg.
Not sure about this one.
What the use case?
feedback?
currently for hooking event listeners up to Messenger, so i was using AbortController there, and it felt natural to extend it to the ref-counter too.
then you don't need refCounter.free(...)
just everything can be in 1 declaration
Ah right. I think it's all good.
Was just thinking about something else when I read it.
This one is not perfect yet. That's what I was thinking of:
https://github.com/clinuxrulz/solid-kitty/blob/main/packages/reactive-cache/src/index.ts
It's for a reactive accessor shared between multiple reactive scope.
It should really return A, not Accessor<A> to make it more ergonomic.
a yes, it's making use of onCleanup
this is for usage outside of solid
Ah OK... understood.
Just unboxing it on return. Makes it nicer to use in my code at work.
Unboxing is not a problem because of the cache.
It's a later job for Tool Kitty :p
Reason is... it makes it look feel like the reactive map in solid primitives.
map.get(key) returns A in reactive map, not Accessor<A>
why not calling the accessor before you return it?
That's what I mean by unboxing ๐
gotcha!
Unboxing is a term that comes from Haskell. I was in Haskell for quite a while. The curse of Haskell tends to follow me around ๐
This makes sense to me now, after a 2nd read.
It's because it is for worker-proxy, which is a separate utility which can be used in non-solid projects as well.
Exactly!
I was wondering how to stop a proxy from working and do the cleanup.
In ComLink they do const proxy = wrap(...); proxy[SomeDisposeSymbol]();, but I landed on const proxy = rpc(..., { signal }) and that you pass an AbortSignal. AbortControllers are cool
I gotta go...
There is also a finalisation API than landed in js not long ago.
U can basically get code to execute on an object when it gets GCed
But there is no garentee when or how often GC happens. (Less control)
However it could be a good last resort, incase the user forget to cause the reference counter to decrement somewhere.
Like a combining the two methods together.
(Widely available since 2021)
Hmm... got a createComputed inside a createComputed inside a.... 14 levels deeps.
Might need an abstraction later to lower indent depth, or just things out into functions with good names.
For rafactoring pass later ig.
That's when u have a monadic bind chain where the next level reactive value is dependent on the previous level.
results of prettier do not look great with large number of indents :P, and a large number of indents makes it difficult to edit in vim.
Little progress. Got the automerge atlas files producing pixi spritesheets (in the iframe).
Was thinking I might need a fluent interface pattern for chaining createComputeds, so each one does not indent the code one level deeper.
Something like (...).then(...).then(...) keeping the same indentation level.
I'll do a proper Monad for it during refactoring pass. Bcuz I also want early bail in the then-chains.
getting close now.... got the pixi render code finished for rendering the map from an automerge level file
Just get a crash on pixi's Texture.load of a blob url for an image. (it's returning undefined)
will need to see if pixi's Texture.load supports blob urls
Closer closer.... looks like it's rendering now with no crashes. But can't see anything on the screen. Could just be my camera position.
(I just had to wait for the Assets.load(...) to finish before Texture.from(...))
Ahh... OK... since blob urls are missing file extentions. Pixi does not know which loader to use:
I followed the blob url to ensure the image was correct and it was.
This is what I need to do:
https://github.com/pixijs/pixijs/issues/9568#issuecomment-2177489078
Strange they wouldn't use the mime type of the object url
More debugging... no warnings this time. But still a blank screen.
Lesgoooo
I think I worked her out... the spritesheet for the tiles is lazy loading, but I only rendered the first frame before loading finished.
Got kicked off PC though... almost there ๐
We'll have it fully working before we hit 4000 ๐
i m hyped ngl
Time to Celebrate!!! ๐ ๐ ๐
little anticlimactic at the moment ๐
Might do a demo video later.
If I add a SpriteRefComponent for players and mobs, and a collision system, then we have enough to make a game.
Before we get too carried away. I need to hook that export/import to/from zip up to the automerge file system. (It's currently connected to an old fs). Because the browser can clear indexed db at any time it likes.
Minimum map loading for my own reference:
import * as prelude from "prelude"
let dispose = prelude.createRoot((dispose) => {
let r = new prelude.PixiRenderSystem({
world: prelude.world,
});
prelude.createComputed(() => {
let pixiApp = r.pixiApp();
if (pixiApp == undefined) {
return;
}
document.body.appendChild(pixiApp.canvas)
})
return dispose;
})
prelude.world.createEntity([
prelude.levelRefComponentType.create({
levelFilename: "l1.json"
})
])
export function init() {
}
export function onCleanup() {
dispose();
}
Creating the levelRefComponent is usually done in a separate source file.
Thank goodness... works on gh pages too.
Just some mirror things to fix. I must of had a fixed tile size when I first did the pixi renderer KittyDemo:
@zenith void that /#/app link is pretty much the final ui design of the app.
Pull requests are welcome.
There is tonnes of stuff to tidy, but only if u feel like it.
One thing I didn't know when I started was mergeProps and splitProps. For layout I would make each of the components 100% width and height, then it get embeded in another div with the correct size/layout. But thats double DIVs :p ... I really like what u did in solid-fs-components to allow the components to have the same props as a div (avoids double divs).
Just making sure you don't feel left out. :p
Fun!
I ll design some screens in a figma first
Be like a proper designer and stuff ๐คฃ
That would be cool.
Main constraint... must be suitable for mobile screen size.
For those poor ppl who keep getting kicked of their PCs
There is also a monster load of github issues too. But I probably wont touch many of those until we've made a game in it. Thus completing the proof of concept.
lesgooooo
SpriteComponent and CollisionSystem next ig
I found a bug where it crashes with multiple tilesets. I'll sort that out when I get home.
Was thinking this might be more user friendly for letting the user decide what systems to hook up:
let dispose1 = usePixiRenderSystem();
let dispose2 = useCollisionSystem();
let dispose3 = useSoundSystem();
. . .
For sound effects. We can have SoundComponent we add to an entity to trigger a sound effect, and the SoundSystem can auto remove that component the moment it starts processing the sound.
still trying to wrap my head around ecs
you are proposing something like hooks?
inside the entity constructor?
A world is a collection of entities.
Entities are like the objects of the scene.
Each entity is made up or 0 or more components.
Each component is basically plain data.
Systems observe, interpret and manipulate the world.
What I do a little differently is have signals in the components. So the systems don't have to diff the them all.
mm curious to see how those pieces fit together
I found a really paper on it years ago. I'll try to find the link.
The pattern was first use in Tony Hawk Pro Skater 3.
This might be the one:
https://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/
The big take away. Is it works around the weakness of inheritance in OOP when defining game objects.
The components achieve the same power of multiple inheritance without the drawbacks.
Some more reading ๐ :p
https://aframe.io/docs/1.7.0/introduction/entity-component-system.html#
I believe aframe was started by Mozilla
I be away for a family get together this weekend. We may not see much change in ToolKitty until next week.
This is connected to my array proxy not being 100% correct for projecting through automerge.... not a fan of proxies :P, but I'll fix it.
Namely [...x, y] operation on proxy array
using produce(...) makes it work, but it is only a work-around until I can fix the Array proxy... proxies are a pain ๐
have fun!
thanks for the literature! i will check it out ๐
ye you need all the little pieces to make it work completely
with array i remember you also need to proxy the length-property and update it with a length-signal and proxy getOwnPropertyDescriptor
That array proxy will be the death of me ๐
Also seems I need a unwrap / rewrap mechanism too for when updating an array.
Some of the elements in the update will already be wrapped and some will new/unwrapped.
Might be easier to two way data bind arrays and avoid the proxy altogether.
Interesting enough... solid's store does not proxy over an array object:
https://github.com/solidjs/solid/blob/e8e228088785691e634872cd35470f5e43c19050/packages/solid/store/src/store.ts#L251
It proxies further up at an object.
Alternatively, I could update ecs components the automerge way.
E.g.
`posComponent.update((pos) => {
pos.x += 10.0;
});
Wouldn't be the end of the world, and I can avoid array proxies. Syntax is still pretty good too.
And it's still compatible with non-automerge via produce(...)
Quickly discover what the solid compiler will generate from your JSX template
kinda cool to see everything going on under the hood
That's pretty neat.
--- Array spreading ---
[numbers] get Symbol.iterator
5
U get to see everything u need to proxy.
exactly
Yep... u gotta do your own iterator with function*()
function* myIterator() {
for (let a = 0; a < n; ++a)
yield z[a];
}
Something along those lines.
This is not a bad syntax though.
yes, this is nice too!
Array spreading ONLY used iterator and nothing else. Interesting.
I won't make a decision straight away, just incase I can still get that array proxy working.
have been mapping out toolkitty for the design pass
some layout tests
it's a lot of different pieces ๐
Wow! You've been busy. I will look at it all.
it has been a lot of fun ๐ trying to imagine the full picture
i think i will try a first attempt to design the sprite editor screen, because it was already pretty worked out with a lot of tools and the like
Can we pin those designs so we can find them easy? It will take some time for us to catch up to them in the code.
Or maybe we can commit those designs files in the code itself in a folder.
Cheers ๐ป
@zenith void you have a lot of good ideas there. That I have not even thought about yet.
thanks appreciate!
next project: automerge based vector editor
Design files saved:
https://github.com/clinuxrulz/solid-kitty/tree/main/design
maybe this could be cool: to combine the entity screen and the scenegraph into 1 single list. that you could build up an entity from scratch and when you like it you can add it to some prefab-list
I'm semi working on a similar project (minus the pixel stuff), like to see that we've thought of similar ideas
sick, always a big fan of ur UIs and editors. got something to show?
It's decently bare since I stripped a lot of stuff out and I'm focusing a lot more on proper implementations but I can make a quick vid
huh
is it cause it's mp4
did u post it?
yeah but it didn't have preview
weird
already looking clean tho ๐
This is trippy... it's automatically creating groups when a selection of objects is transformed together. And it is also creating clones (references?) (like mirror but same hand, not reflected)?
Looks really cool.
I was hitting hotkeys ๐
It was creating groups when I did Ctrl + g and I was making prefabs that move together
2 different concepts, they just look similar in the UI cause I haven't updated it to reflect the difference yet
Ahh... understood... I thought implicit group creation would be odd.
I think that can work. There should be enough screen space.
I'm gonna have some initial components that come with the prelude as well as user component. So the user can hit the ground running with minimal user code.
The user can optionally use the components or the systems that come prebundled in the prelude.
The user should be able to make something like a platformer game with very minimal user code/scripts.
Was also thinking of a UI registry for UI forms for the ecs components to allow us to override the automatic form UIs to provide something nicer for the user.
E.g. components that reference other entities may bring up the entity picker mode when setting the referenced entity.
E.g. a projectile component may have the entity ID of the character that spawned the projectile. So that a player can not be hit by projectiles they spawned.
Heres a sprite component (done in Termux):
Will need a scale, do they don't look like fleas.
For now animations can be defined by a animation name, a list of frame names that make up the animation, and a frame rate or frame delay. That structure will translate easily into pixi.js animations.
This must be the downside to tailwind ig:
This is all built inside toolkitty right? Would love to see a demo of how you would build this from scratch w the current prototype.
In this specific usecase, wouldn't you be able to infer that easily when writing the system for it?
But yes, couldn't agree more.
We need some type of interface for defining and composing Parameter and their ui-widgets. That meow approach comes to mind: { parameter: Parameter, widget?: (props)=> JSX.Element }. If you don't define a widget, we would use the default generated ui from the Parameter-type.
I think would be cool to also have something similar for the tools in the toolkit, that these can become things that are extended inside toolkitty.
In my research I thought it could be cool that tools not only change what the pointer events do, but that they could hide/reveal some of the HUD ui: for example there could be a color tool that also brings up a colorpicker and shows the pallette of colors.
Yes. Could be fun to have a standard demo we continue to make.
Like the Mario platformer demo you have + a death state if you fall and success state if you reach the flag at some platform.
And then we could use this to inform us how to improve the whole development flow + design
Instead of the prelude, we could also move it to the runtime-fs, then they are editable afterwards
continuing with this kind of thinking, maybe we can display the system-list in a similar way.
That would be cool. The user adding systems they want to use to the scene graph.
Yeah, that would be fine.
Yeah, the system could work it out. Just trying to think of an example where a component may reference an entity. Thought mario fire bullets from super flower.
Yeah. Two folders from tool kitty are providing the prelude systems and prelude components at the moment.
https://github.com/clinuxrulz/solid-kitty/tree/main/src/systems
https://github.com/clinuxrulz/solid-kitty/tree/main/src/components
We could expose them in an editable runtime-fs.
In the level editor I should also switch from an svg backend to a pixi backend. Been noticing svg gets slugish with a lot of elements.
For an in-game HUD, I was thinking the same ECS could be used but have our own JSX using solid-universal, that maintains UI entities.
Having a look at aframe, you can see entities instances can be represented quite well by XML.
Its also a quite a lot of code to do without a runtime. It would basically involve the user typing all that code for the prelude code provided by that "systems" and "components" folders into their scripts.
Gotcha ๐
Seems a runtime-fs will allow the user to use any version of solid and pixi without the need for version locking.
Ohh... dont navigate and read ur own code too quickly xD :
For sure. Some type of std-lib makes a lot of sense.
And maybe a revert button for runtime fs files/folders incase the user makes a mistake and wants to change it back.
There is a logic bug I need to fix in automerge fs to get move file working (I've only partially implemented the FileTree fs interface).
I have a fixed parent folder reference for each file/folder. Which is no longer true when u move files/folders:
I may change it from a tree filesystem to a flat filesystem. Because I'd like to open files by ID without knowing/having-access-to the parent folder.
i prefer flat filesystems too, never have to walk stuff
that is a fun idea, i like it
i played around a bit w a proxy version solid-three too a while back
having access to the history of files would be handy for all of the files
and mb doable w automerge?
xml is nice for trees
it's kind of interesting that ecs does not have parent-child like baked into the concept. i guess to implement it you need to do a Parent-component, reference to the id of the parent-entity and then resolve the correct matrices of the entities by walking the tree in a system?
Yep. Thats the common practice. They do it with components. Then they provide a system to help manage child/parent relationships and queries.
Yep automerge can do that.
Children / Parent components are practiced by bevy too.
https://bevy-cheatbook.github.io/fundamentals/hierarchy.html
I think import/export user files to/from zip is the next most important.
I had it going with a non-automerge fs. Just need to hook it up to the automerge one.
Yeah. But also definitely bundle a runtime the user can go back to. (Corrupt or expired indexed db)
And the ability to cull history every now and again. Incase we hit the 500mb limit.
Yep. That too. A system maintains the world transforms from the local transforms.
(ignore this message, using discord to host an image for the solid playground)
export automerge fs to zip done via Termux, but not connected to the ui. import from zip not yet.
Gotta get my hands on a PC :p
I showed my boss ur <FileTree>. He was quite impressed.
We have a assets page in our CAD software for storing textures, profiles and things.
We may end up using your file tree for it.
I also showed him your design files for tool kitty.
He asks what country you are from. He could be considering you for some freelance stuff down the road. (No gareentee though)
very happy to hear that ๐ would be really cool if it was used in ur cad program ๐
i m from the other side of the planet: currently based in berlin for a year, then probably back to brussels.
Ur OK with that?
ofcourse!
Just checking :p... it's sorta like I got u to work for free.
perfect ๐
No garentee though.
He has a few freelances he uses already. But I told him your a good coder.
for sure, no worries ๐ already grateful you are mentioning me
No problem.
My grand dad is full German. I was at his 85th birthday party last weekend.
Makes me ยผ German. I don't know the language though. :p
doesn't discord have a limit to how much a link can be kept up as of recent?
like 24 hours or something
Don't know it either ๐ it's luckily close enough to dutch to understand bits and pieces
Very possible, but just needed something quick and dirty to sketch
Damn 85 year old, that starts to count!
oh ok
Import/export hooked up. But only export working properly at the moment.
Import does work, but not while /#/app is mounted. Gets caught in an infinite loop.
It's probably because the /#/app route recreates any missing folders it depends upon to work properly, and the import deletes all the folders before import. The two are probably fighting each other causing the infinite loop.
Hmm... selecting text in monaco on mobile for copy paste or cut paste is really hard. Might need to consider support multiple editors. Or even a virtual browser based keyboard that offers arrow keys with a shift state to do selection. The later will be tricky.
Microsoft made it quite clear the are not interested providing any mobile support for monaco in their github issues.... so probably back to code mirror as a switchable option.
Hackers keyboard from google play gives us the shift key and arrows that works for selection. I wonder how hard it would be to make a web version of it to add just a few extra buttons to go along with the native mobile virtual keyboard.
Similar to Termux how it adds arrow keys.
Neo vim for the browser would be amazing. But I might be pushing my luck. :p
Someone has done something here:
https://nvim.nry.app/
And this:
https://www.npmjs.com/package/vim-wasm
But needs cross origin isolate.
Oh well... import is now working inside the /#/app route.
I just realized I can not use file IDs inside other files' data for cross referencing files. Because an export followed by an import would cause all the file IDs to change.
I can only cross reference files by file path.
And referencing files by file path is probaby more user friendly in the case of editing text files externally in notepad if needed.
I gotta stop being a noob ๐
There is a perfectly good working demo here using code mirror 6 with a language server:
https://discuss.codemirror.net/t/codemirror-6-and-typescript-lsp/3398/24
Down the road I suppose I can use a toggle button for switching between monaco and code mirror.
Maybe the next focus is the collision system and physics system.
I did see the UI design for adding systems to the scene graph. We will get to that. But for now it's in a user script:
let systemCleanups = [
useSystem("PixiRenderSystem"),
useSystem("CollisionSystem"),
useSystem("PhysicsSystem"),
];
. . .
Probably wouldn't be too far fetched to get some of the UI based structures to simply generate additional script code under the hood that gets included a long side the users scripts.
Also I tried to use the vite-plugin-dts-bundle-generator so I can generate a single bundled type declaration file. But it didn't seem to generate anything. It might be because Tool App is an app and not a library.
Not too keen on refactoring it back into a library again.
mm ๐ค do you also experience it on the ts playground?
such a pity that lsp support is so dreadful on codemirror. tried to give it another shot the other day but didn't got too far
would be sick tho, would fit the whole pixel vibe
lesgoooo
Because an export followed by an import would cause all the file IDs to change. mm this i don't really get
The zip file looses file IDs. I was planning having the map files reference texture files by ID incase they are renamed. But glad I didn't, otherwise it would be corrupt data for an export followed by an import.
Yeah... can not select text on mobile in ts playground too.
mm strange ๐ค an option would be to not bundle it: we could zip the .d.ts and import them accordingly
a nice!
At the moment I am manually referencing all the .d.ts files one by one from node modules into monaco via ?raw. It works. It's just a pain.
looks good!
Having a bit of a play at the moment.
I might go like:
import "prelude/solid-js" to use the same bundled sodium.
Or
import "solid-js" just to use esm.sh.
Sometimes u need the same global module state, sometimes u don't need it.
what would be a usecase where you don't want to same module?
If solid 2.0 is out and kitty is an old build ๐
Might wanna try stuff that does not interact with the runtime directly itself.
Actually I might just focus on the CollisionSystem now. It will be more fun.
Either way I need to make solid-js it's own entry point to avoid the name mangling.
Just few newer features that are not in the bundled version ig
A new shader might come along for pixi for example.
๐ค wouldn't it be handier to not bundle pixi/solid at all in toolkitty, mb have it as import maps?
I believe code mirror demo is using @typescript/vfs which is feed into a TypeScript language server.
They have a method to call to see possible completions at the current cursor position (at line/col)
https://github.com/okikio/codemirror/blob/42803c6b70d0fde48b49d2df1ced1dfd7630d8d3/public/workers/tsserver.ts looks pretty simple
Yeah, it's only 100 lines of code.
Not really sure. I just know if the same module is imported from two different target urls, they are treated as different modules with their own global scopes.
Solidjs has a global state when propergaing updates.
Pixi has global state for loaded assets and caches
ye having an import map where both toolkitty as the user code imports from would be so you would not have 2 different modules with their own scope
Ah I see.
i think the only annoying thing about import maps is that you can only set them at load time
That might be easier than what I'm doing right now.
but it's not that you will set those a lot, mb doing a reload when you wanna change those dependencies is not the worst thing in the world.
File: src/lib/solid-js.ts
export * from "solid-js"
Then that file added as an entry in vite.config.ts.
It creates a separate chunk for solid-js but the file name has a unpredictable hash in it.
For prelude lib.ts I had the trick:
let libUrl = import.meta.url;
That won't work for solid-js, bcuz I can't add that to the code.
Ohh hang on. It still can work.
I can put that in src/lib/solid-js.ts
here is the trick:
export * from "solid-js";
export const solidjsUrl = import.meta.url;
that will give me the url to the solidjs chunk
Got import ... from "prelude/solid-js" working just now. (For accessing the bundled module)
Will do the same for pixi ig
And I'll use your type downloader and esm.sh for the rest of them. I haven't touched that yet.
Oh my goodness...pixi has hundreds of type files ๐
I wonder if there is a trick to grab them all without having to manually code them in 1 by 1.
I believe u did a import.meta.glob thing somewhere to grab a bunch of files as ?raw at once.
Ah yeah... in the solid-fs-components demo
I think I can do the same to grab all the .d.ts files at once.
const pixiTypeDefs = import.meta.glob('pixi.js/**/*.d.ts', { as: 'raw', eager: true })
The glob trick worked.
I now have access to all of bundled pixi under
import ... from "prelude/pixi.js"
In your code.
A lol hahaha
That's where I got the idea :p
Thought ya something u already know ๐