#Core Haxe
1 messages ยท Page 4 of 1
I think I tripped that at some point lol
yeah, sorted now
i think i might have fixed the recusion too... the problem is that the socket would constantly call handleData if there is always data coming through, so if the server was constantly sending data then, effectively, the socket would sit there in a while(true) pumping data to the buffer constantly and calling handleData with no respite
the fix would be to iteratively build up the buffer in the process() call, which seems to work... just need to see if it breaks other things
alright, if you pull latest can you try again at some point, i think it should be fixed...
ive also changed the way it uses the main loop, so by default, should work nicer too (you can activate the threaded behaviour with -D websockets_threaded
hard to tell for sure, but feels faster too
nice, did you do any of the wss changes?
want me to pr those?
or i guess it's probably better to make a change to the existing pr
yeah, if you could, ill take a look, but they need unit tests before incorporate the changes
more interested to see if the initial recursion issue is sorted tbh
will check now
in my app this is the deepest the socket stuff will go now
will try threaded as well
something is wrong with threaded mode
app basically goes to a standstill
i wonder if this is my app or the sockets lib
will go to the smaller test case i made the other day
does it work though in the first image?
without threaded, it works
setting the threaded flag it fails
ie, the first image?
yea
perfect
might have broken something with threaded, not sure, but i think the main loop method (ie, not threaded) is a better way to use it anyway
compiling sleep(0.1) atm to see if that helps
its not relevant anymore in the unthreaded version
i mean for the threaded version
right
so, anyway, the initial recursion issue is sorted ๐
(and that OOM issue too... )
np - defo worth fixing - and fyi - the same issue applies to hxWebSockets
probably hard to tell, since websockets are pretty nippy, but does it feel any faster?
it's hard to tell because my app is a bit sluggish in general on cpp
right
my adventure will probably lead me back to my haxeui backend
I never really used profilers much because they never seemed that useful
my experience was only really with chrome's and maybe it's a skill issue
but here things seem so obviously wrong
but i guess, on js you can get away with a lot of bad
true enough
was threading on or off before?
on
ah okay cool
the default is the main loop now, which i think is better - i kept the threading there though anyway (behind a define) just in case
the hard loop is there also btw (-D hard_loop) but i think that is going away very soon ๐
#if hard_loop
while (true) {
read = _socket.input.readBytes(data, 0, data.length);
if (read <= 0){
break;
}
//Log.debug("Bytes read: " + read, id);
_buffer.writeBytes(data.sub(0, read));
}
#else
read = _socket.input.readBytes(data, 0, data.length);
if (read > 0) {
_buffer.writeBytes(data.sub(0, read));
}
#end
main loop is slower than threading for me but i guess that's expected?
possibly if you are doing other things (which i guess you must be)... you might be able to bump the priority
depends what is more important: comms or render
i doubt thats accurate
my guess is data is more important (ie, comms)
nothing to render without data at the end of the day
anyway, threading should still be a viable option, havent checked why its not working for you, its defo possible i borked something (i wasnt trying to work on this at all this weeked)
I've never heard of priority in this context, how is that a thing?
yeah dw about it, it does work at the end of the day and I'm okay with it in its non perfect state
the most important thing is, everything runs now on the main target I want to use
can we use this for rendering? is that a thing?!
yeah, ill check out threading again at some point, but this currently is a lib i simply dont need - that said, it was nice to get the recursion and and OOM issues out the way for when i do need it
no idea, i presume you are talking about ceramic, so i guess you need to ask Jeremy
i dont know how it uses the MainLoop, might not be via the add(event) at all
i would assume it uses its own event loop (via SDL or something) and simply "ticks" the haxe event loop
if that is the case, then yeah, threads would presumably be much faster
what am i looking at? the 363's?
the stack continues that deep
but it might be my fault
not completely clear
i don't know how these stacks get generated and what constititutes a "new stack"
thats not that deep... it just means it can process more data in that window (timewise)
the other was actually recursive
thats actually a "nice" stack, because it means it didnt have to wait for another "tick" to process more data
even though that looks nicer, actually, thats a "less effecient" frame (from the PoV of websockets)
it had more data, but had to wait to process it
you have "one thing" filling up a chunked buffer of bytes (the socket) and then "another thing" processing those bytes... if they can all happen at the "exact" (not possible) time, that is ideal
โ๏ธ this basically, one will read until it runs out of bytes (and will get out hand fast if there server is constantly pushing bytes), the other will wait (in that case between events on the main loop) before pushing new bytes - haxeui-core/ws will just process the bytes as they come in (until it can build a "data frame" out of it)
yeah, i think hard loop wouldn't work but i can give it a try and see
the market is always active
"hard loop" was the original, the one where you got that huge stack
ahh
exactly, its always pushing data
i just never tested with a server just constantly pushing data... there was always a "request", "response" kinda thing going on
so defo a bug, and pretty good one
that makes sense
i guess it would still benefit your use case as well
my use case just makes it very clear that there's an issue
100%, core-haxe/ws has defo benefitted from your findings, for sure
(not to mention to the alloc(1024) in a hard loop ๐ฎ )
that was a crazy one btw
if you have a client socket (in non-js) and it doesnt get any data, it will sit there create 1kb of data every... god knows... nano second
while(true) {
Bytes.alloc(1024);
}
basically... ... ... crazy
lol
i never investigated those issues
typically i'm recompiling my app before i triggered it so i never thought it was an on going issue
@velvet lodge Am I right to assume that if find() doesn't find anything, it will reject?
nope, it will return a null entity
MyEntity.find(101).then(entity -> {
trace(entity == null);
}, error -> {
trace("error", error);
});
because it may be perfectly valid to not find an entity
yeah, its pretty cool - and kinda mental powerful when you get "into" it
ah yes, uknonwn
sqlite/impl/cpp/SqliteDatabase.hx:149: UKNONWN:,TClass(entities.User)
hm does the sqlite backend not support relations?
yeah
(probably a bad idea to make the package they reside in also called entities)
doubt thats it
what do your entities look like? Its looks like its coming from core-db, not entities
sqlite is defo less tested than mysql, but should be fine
class Session implements IEntity {
@:cascade(deletions) public var user: entities.User;
@:size(36) public var accessToken: String;
// ... static functions
}
@:exposeId
class User implements IEntity {
@:size(254) public var username: String;
@:size(254) public var hash: String;
@:size(24) public var salt: String;
// ... static functions
}
nothing too complex
what if you dont use the fully qualified path?
@:cascade(deletions) public var user:User;
?
nope same
very odd, its a super weird error too
what haxe target, and can you paste some test code using these entities so i can test my end?
I am using hxcpp and yeah give me a bit
by code you mean just a simple main class that connects to a db and adds a users and attaches it to a session?
yeah, basically
just something i can run this end to see if i can repro
class Main {
static function main() {
trace("running");
var db = DatabaseFactory.instance.createDatabase(DatabaseFactory.SQLITE, {
filename: "test.db"
});
EntityManager.instance.database = db;
var session = new Session();
var user = new User();
user.username = "tim";
session.accessToken = "bob";
session.user = user;
session.add().then(_ -> {
}, error -> {
trace("error", error);
});
}
}
class Session implements IEntity {
@:cascade(deletions) public var user: User;
@:size(36) public var accessToken: String;
}
@:exposeId
class User implements IEntity {
@:size(254) public var username: String;
@:size(254) public var hash: String;
@:size(24) public var salt: String;
}
this seemed fine
also, totally unrelated, but do you really want @:cascade(deletions) public var user: User;?
this means when you delete the session, it will delete the user too - might be totally what you want, but feels like you might be accidently nuking your user data ๐ฎ
Oh, I thought it did the opposite lol
nope, it will cascade deletions through
anyways trying your code now
using entities2, right?
correct
latest git version
correct
it works hmm
i even tried splitting it up into multiple files and it works
hm
what does your code look like? Is it paste-able?
yes
cool
hm i think it happens when Session.findByCredentials gets ran
so, i dont know if its the actual error, but there are a bunch of times that you are using .add and .update and arent wating for the promise to return
but i dont actually think thats the issue tbh
oh ill fix that
hang on, i was using entities1 ๐
i tested with entities2 and it worked fine for me
./src/Main.cpp(69): error C2860: 'void' cannot be used as a function parameter except for '(void)'
:/
ok, also worked with entities2
try doing a session find with the user as a query?
OK, i can see what it is ๐
yeah
sqlite/impl/cpp/SqliteDatabase.hx:149: UKNONWN:,TClass(entities.User)
public static function findByCredentials(user: entities.User, accessToken: String) {
return find(Query.query($user == user && $accessToken = accessToken));
}
oh the single equals lmao
the query needs to be the userId, so:
public static function findByCredentials(user: entities.User, accessToken: String) {
return find(Query.query($user == user.userId && $accessToken = accessToken));
}
aaah
let me try now
yup, works. thanks so much!!
i guess i didn't think about that
np - its kinda gotcha because it indicates having knowledge of the table structure (which ideally you never touch in entities), but im not sure there is a good way around it
i mean, is it safe to assume if you mentioned another entity in the function param you always mean the primary key? Im not sure it is
Could I write a custom equals function?
... .. maybe it is ๐ค
wdym?
If entities supported it, maybe in users I could do something like
public function equals(u: User): Bool {
return userId == u?.userId;
}
to override the default functionality
thats not supported no in query-dsl, not sure how it would help tbh either
im kinda thinking thats its safe, for entities to find this function:
public static function findByCredentials(user: entities.User, accessToken: String) {
return find(Query.query($user == user&& $accessToken = accessToken));
}
check the params for entities, and that its used in a query(), and map the primary key in the query (ie, rewrite it at the macro level)... but ill need to think about it because it could be a foot gun
is there a difference between == and =
nope
it would be neat though, because just today i did this:
return @:privateAccess UserData.findById(userToUpdate.userDataId); // TODO: dont like that i had to use the id here
(ie dont use, or want to use, @:exposeId, and i consider it a "fail" when i have to access the primary key directly)
why is the id not exposed by default?
because it goes "against" the idea of entities... the idea that you just dont think about things like that anymore, just your haxe classes, ofc, sometimes you do need the id (which is why that meta exists), but for example, in the WHOLE of glint, thats the first time ive accessed the primary key directly
hm interesting
and by "against" i dont mean "you cant or you shouldnt", just that, the spirit (ideally) is you dont think about "whats the id of this"
and actually, i think i could fix my usage with a "sync" function
for context
the problem here is the userToUpdate could just have and primaryKey, and lets say, username
actually, you know what, im talking shit
i think i can just call "userToUpdate.update()", it will only update things that exist
i thought it would nuke things, but why would it... .. ๐ฆ
yeah, works fine ๐

Out of curiosity, how many people are working on Glint?
It kinda confuses me, I am on the website and there is like 10,000 people in the discord server but only like 70 people online
2 devs, whole marketing team (prolly 5 people), few (new) testers - i think 3, then i woud say, dunno, 3-4 other people (various hats)
we did have some "data engineers", but they are a joke, like total joke
lol
feels like they could move some of the energy from marketing to development, but I guess they have good devs at least
i think most people are after "bounties" and stuff... we paid out a bunch of prizes for creating content on the beta
its getting there (this is just a dashboard i knocked up earlier)
It looks really pretty
yeah, its come along way in a semi short amount of time
the stack is mental nice to work with - so easy to add new bits (client / server / job consumers, its all super modular and super easy to plug bits in)
all haxeui+core right?
yeah, and a bunch of other libs that may (probably will) go into core-haxe / haxeui (as non core things) at some point
๐
i think i got everything ported over from pure db-core -> entities now
outside of the find thing, it was really smooth
hows the new version feel? Simpler? Less "database'y"?
quick fyi, with (alot of) help from the #macros 's guys this now works, you can still use the user.userId, but if you just use the entity, it will auto replace the primary key in the query - i just didnt like that your inital "hunch" was, imo, the right one, and that entities2 forced you down the "you must know about the table structure route"
I really wanna use entities but I'm stuck on this performance/optimisation path atm :/
Feels a lot simpler. With db-core I had this giant class where I handled all of the db stuff. I wasnโt really sure how I was going to split it up. Entities forced me (in a good way) to split things up sanely.
nice - defo the goal - the idea is that you basically just "stop" thinking about the db part - which for the most part we dont care about
It doesnโt feel databasey at all, even compared to other ORMs I have used.
What would the idiomatic way be now? Should I keep it as $user == user.userId or change it back to $user == user.
I am guessing the latter?
personally, i think the latter
exactly, yeah

what is the best way to break out of a promise .then() chain if something goes wrong?
isn't there normally a .catch() you can add to the chain to handle errors ? Not my area but that's what I recall from a a brief bout of js at one point
Oh ok
throw / reject
Is there any minimal entities2 usage sample? I'm a little confused how to sync my entities with the databases, i. e. something like this in sqlalchemy:
Base.metadata.create_all(engine)
You dont sync, thats what entities2 does:
EntitiesManager.instance.db = ...
...
...
var myEntity = new MyEntity();
myEntity.someField = 1;
myEntity.add().then(_ -> {
...
}, error -> {
...
});
ah, ok, missed the EntitiesManager.instance.db = ... part
if you wanted to add a bunch of entities at once you can do something like:
var promises = [];
for (i in 0...100) {
var myEntity = new MyEntity();
myEntity.someField = i;
promises.push(myEntity.add.bind());
}
PromiseUtils.runSequentually(promises).then(_ -> {
...
}, error -> {
...
});
Yep, but I mean, creating a table if it doesn't exist (when I add a new entity class)
table creation is handled automatically (as is table modification), eventually, there will be a system for upgrading the tables (to avoid data loss)
then again, doesn't entities have a mechanism for batch updates? not to perform them individually
is it lazy? I mean, should I do something meaningful with the entity (add a new instance, for example) for it to trigger?
also, table creation is lazy, its only when you use the entity, however, you can init all entities at once with EntitiesManager.instance.initAll (or something like that, dont remember the call)
nice, that's what I was looking for, many thanks
EntityManager.instance.initAllEntities()
extract from a server i use:
public static function initEntities(config:ConfigNode, path:String = "/database/connection"):Promise<Bool> {
return new Promise((resolve, reject) -> {
findEntitiesDatabaseConnection(config, path).then(database -> {
EntityManager.instance.database = database;
return EntityManager.instance.initAllEntities();
}).then(_ -> {
resolve(true);
}, error -> {
reject(error);
});
});
}
(initAllEntities)
yeah, looking to support enums at some point, just not 100% what the table structure would look like... cant just be an index because that could change... its possible i think to use a string or something (ie, the enum item name) but it feels like a hack
... ... would make querying simpler though
if it's of any help, sqlalchemy uses names by default, but it's possible to set a certain flag to use values instead (translating into haxe, that means instance names by default and instance indexes as an opt-in)
that makes much sense, considering index may be accidentally or intentionally changed much more often than name
yeah, ill have a think - probably name is the quickest path - but not something that is a priority for me currently
yeah, thinking about it briefly, name seems like it would be perfectly fine... and think it would greatly simplify any lookups based on that enum
select * from X where enum_field='enum_value'
yep
And, in turn, for enums with parameters I would expect auxiliary tables to be created. Something along the lines of:
enum SomeEnum
{
NoArgs;
OneArg(a:Int);
TwoArgs(b:String, c:Bool);
}
class SomeEntity implements IEntity
{
public var field1:String;
public var field2:SomeEnum;
}
->
CREATE TABLE some_entity (
some_entityId INT NOT NULL AUTOINCREMENT
field1 TEXT NOT NULL,
field2 ENUM('NoArgs', 'OneArg', 'TwoArgs') NOT NULL
)
PRIMARY KEY (some_entityId);
CREATE TABLE some_entity_field2_one_arg (
some_entityId INT NOT NULL AUTOINCREMENT,
a INT NOT NULL
)
PRIMARY KEY (some_entityId)
FOREIGN KEY (some_entityId) REFERENCES some_entity (some_entityId);
CREATE TABLE some_entity_field2_two_args (
some_entityId INT NOT NULL AUTOINCREMENT,
b TEXT NOT NULL,
c BOOL NOT NULL,
)
PRIMARY KEY (some_entityId)
FOREIGN KEY (some_entityId) REFERENCES some_entity (some_entityId);
implies joins when filtering by the enum field value:
...someEntity.field2 == SomeEnum.TwoArgs('abc', true)...
...
FROM some_entity AS t1
INNER JOIN some_entity_field2_two_args AS t2
ON t1.some_entityId == t2.some_entityId
WHERE t1.field2 == 'TwoArgs'
AND t2.b == 'abc'
AND t2.c
...
nope - not gonna happen ๐
why tho 
because its adding alot of complexity for a relatively small feature - and its one that i dont even (personally) need atm
In entities2, can I specify the size for an integer column (tinyint/smallint/int/bigint)? Can I add UNSIGNED restriction?
EntityBuilder seems to only respect the size for String fields, but not Int. I see no mention of unsigned throughout the whole source code either
Of course, that would be backend-dependent, but still, it's a useful feature to have
so yeah, size will (at some point) be used for things other that string - its supposed to (eventually) be sort of generic meta - as for unsigned, thats not a thing, no, it would have to be implemented in db-core first and then exposed to entities
Thanks! What about providing the default values for the columns? Is it possible?
column1 INTEGER DEFAULT 0,
column2 TIMESTAMP DEFAULT CURRENT_TIMESTAMP()
nope, also not possible currently, but it would be very useful
agreed
ill take a look at it at some point, too busy creating incredibly elegant user defined themes for my application
๐
๐
You should write a UI framework
is there anything special I need to run db-core/mysql on linux?
I'm trying to compile and I'm getting
/usr/bin/ld: cannot find -lmysqlclient: No such file or directory
collect2: error: ld returned 1 exit status
Failed to compile C++
but the library definitely exists
Make sure it is the devel version
oh there's something i need for linux?
Oh. Maybe not. Looks like it is vendored
the CI runs it all on linux fine i suspect
i can't imagine they're running on windowu
yeah, it's running on ubuntu
what does this "super hack" mean
the thing in brackets seems interesting
could that be the missing link i require?
i installed mysql-devel
Compile C++
Using libmysqlclient from system
Link: Main-debug
/usr/bin/ld: cannot find -lmysqlclient: No such file or directory
collect2: error: ld returned 1 exit status
Failed to compile C++
now I get that
a Google suggests that Ubuntu and fedora have different names, would that cause an issue with externs? ๐ค
I found it, it should be stored in /usr/lib64/mysql/libmysqlclient.so
should come with mysql-devel
not sure why it isn't being found by libmysqlclient tho
it's defo here
oh
maybe it's a atomic issue
it isn't available in the "normal" mode
but it's available in the container
@toxic stratus not the most elegant, but sudo ln -sf /usr/lib64/mysql/* /usr/lib64/ fixes it
not sure why the linker doesnt pick up stuff in /usr/lib64/mysql
i assume you had the issue as well?
i use sqlite, i was just messing with it now
it finishes but it finishes with an error code somewhere?
umm that seems like a ceramic thing
compiles no error here
but i am not using it in a ceramic project
app runs and i get some traces but it stops responding
might be a me issue here
db doesn't connect though
hard to say without trying it myself
might be a case of my ip has changed so server is blocking the request
it just fails differently on linux
ahah, ipv6 is active
need to find out how to disable that x)
yep, that was exactly it
I'm in. Thanks @uncut phoenix
np
so... ... all good?
Yeah. Just Fedora weirdness I guess.
I'll post a pr for that in the readme at some point
The libmysqlclient issue should be solved by modifying build.xml (to add a -L path) rather than by creating a symlink
It is different on other distributions. Is there a way to add a search path to hxcppโs linker?
I tried messing with ldconfig or whatever but not sure that is applicable
You could allow setting a -D MYSQL_LIBPATH define that is added as a linker path
Also, if it is using the system library, it should also be using the system headers
i cant remember exact the call now, but there was a linux call you could make (similar to wxConfig) that gave you all the compiler settings
by call i mean process
in hxWidgets we make the call in the macro, parse the results and dynamically build the hxcpp xml
mysql_config --cflags --libs
?
๐ฆ[j@fedora ~]$ mysql_config --cflags --libs
-I/usr/include/mysql -m64
-L/usr/lib64/mysql -lmysqlclient -lz -lzstd -lssl -lcrypto -lresolv -lm
That would be the right thing to use yeah
yeah, then use the results of that (in a macro) to dynamically create the build xml - hxwidgets does it, and it seems pretty flawless
the hxwidgets stuff is significantly more complex, but its the same idea: https://github.com/haxeui/hxWidgets/blob/master/src/wx/widgets/EntryMacro.hx
Been trying to do the same thing with linc_dialogs but it wasnโt working for me
Might need to get you to look at it if you ever have time.
look at what? Never used linc_dialogs
The process doesn't seem that bad
it looks like you're just constructing the xml in the macro
and inputting some command responses
exactly that yeah, based on the results of a process call
so find a working configuration, find the appropriate system calls/flags to get the appropriate info
and i think that for the mysql one its even more simple since i dont think you have to take "other" things into account
i doubt you'd even need much macro stuff here
well, it has to happen in the macro to build the xml meta before any compilation starts
it's a pretty neat way to handle it
i think ibilion came up with it (iirc)
Dynamically constructing the XMLs. I think I got the generation portion working, but it fails to compile.
any error? code?
https://github.com/l0go/linc_dialogs/commit/e8055d0239077e55fd0119deaa78d5e5b48a4be6
https://paste.rs/QbASj.bash
a simple, no-frills, command-line driven
pastebin service powered by the Rocket web framework.
the weird thing is if i run the command on line 92 myself and recompile it works fine
is there a way to use http without async?
i know normally thats bad but im trying to make a script to mass delete spam accounts on my gitlab instance (over a thousand of them), and i just want it to do one at a time so it doesnt kill the instance
just time each request
don't spam them all at once
one thing you can do is, don't send another request until you recieve the response from the last one as an example
the requests just never seem to complete 
can you get one to complete?
trace('deleting ${user.username} (${user.id})');
client.delete('https://git.team-prism.rocks/api/v4/users/${user.id}', null, ['hard_delete' => ''], ['PRIVATE-TOKEN' => 'glpat-'])
.then(response -> {
trace('Deleted ${user.username} (${user.id})');
deleteNextUser();
}, error -> {
trace(error);
deleteNextUser();
});
this is what im doing right now, and it traces the "deleting" line, but then just stops. ive let it sit for like 2 minutes at least
i would expect it to at least error out but it just isnt
maybe i dont know how to read gitlab logs but it doesnt appear to show up there at all
it's hard to tell from the screenshot but do you have some form of main loop running?
is the app just exiting after it runs that command?
https://paste.team-prism.rocks/bCeKQA heres the whole thing
but yeah, there is a loop going
awkward to help because i don't have access to the api stuff
if you use https://paste.team-prism.rocks/WeQzLF for users.json, the script will work, just it wont be authorized to actually do the delete
also fwiw i have latest versions of everything, since i downloaded it today
switch the while loop out with this at the end of the main
var timer = new Timer(100);
timer.run = () -> {}
you're blocking the application from running
thats unintuitive but it works
i keep getting Thread 28 killed on uncaught exception Unix.Unix_error(3, "recv", "")
which causes it to freeze again
but i can just kill it and continue where it left off
i actully think this looks fine, its not exactly how i would have done it, but it looks fine - testing
hxcpp / hl i assume?
ok, i think i can see what NB meant, that while () loop isnt a great idea, although you do have a sleep in there, but still, not the best way, two secs
class Main {
static var client = new HttpClient();
static function main() {
trace("started");
deleteAllUsers().then(results -> {
trace("all operations complete", results);
}, error -> {
trace("error", "error");
});
}
static function deleteAllUsers():Promise<Array<Int>> {
return new Promise((resolve, reject) -> {
var url:String = "http://google.com"; // just for illustration purposes
var urls:Array<String> = [];
var results:Array<Int> = [];
for (i in 0...10) {
urls.push(url + "?q=" + i);
}
deleteNextUser(urls, results, resolve, reject);
});
}
static function deleteNextUser(urls:Array<String>, results:Array<Int>, resolve:Array<Int>->Void, reject:Any->Void):Void {
if (urls.length == 0) {
resolve(results);
} else {
var url = urls.shift();
trace("making request", url);
client.get(url).then(_ -> {
results.push(Std.random(0xff));
deleteNextUser(urls, results, resolve, reject);
}, error -> {
trace("error", error);
results.push(-Std.random(0xff));
deleteNextUser(urls, results, resolve, reject);
});
}
}
}
( @hazy mulch )
i was using interp
right, should be the same anyway
yeah, its fine:
src/Main.hx:10: started
src/Main.hx:48: making request,http://google.com?q=0
src/Main.hx:48: making request,http://google.com?q=1
src/Main.hx:48: making request,http://google.com?q=2
src/Main.hx:48: making request,http://google.com?q=3
src/Main.hx:48: making request,http://google.com?q=4
src/Main.hx:48: making request,http://google.com?q=5
src/Main.hx:48: making request,http://google.com?q=6
src/Main.hx:48: making request,http://google.com?q=7
src/Main.hx:48: making request,http://google.com?q=8
src/Main.hx:48: making request,http://google.com?q=9
src/Main.hx:24: all operations complete,[101,129,246,243,45,75,178,214,83,131]
out of interest because it was suggested in haxeui channel
why is websockets far away from being complete?
is it because of the incomplete server?
another question: Is the client usable with javascript target, because I'm targetting mobile and html5 at the moment
I think it's mostly a question of "Ian time" than anything else
pretty much yeah, unlike hxWebSockets, the core-haxe version will all be unit tested, etc... so everything takes more time because as im migrating it across, im writing tests, making changes, etc - so its not quick. Add to that, that im totally swamped recently (and the fact i dont actually need it... ... ... yet)
and this
i think so... @toxic stratus has actually be using the core-haxe version, client only, and i think its working fine for the most part (?)
yeah, it's mostly been fine but I've been on a few side quests the last few weeks
yeah, its very much a "use it at your own risk, i dont have bandwidth to support it currently" kinda thing
its probably not a million miles away to be fair, all the client side unit tests are running for all targets just fine
and they are fairly thorough: https://github.com/core-haxe/websockets/tree/main/tests/cases
filename is wrong, just realised that ๐
actually, not they are alright... i just have unit tests for the test server too just to make sure it worked
What needs to be done for html5/js target? I see that there is a test-js.hxml but I guess it's not yet prepared correctly. Would love to contribute a bit further ๐
i don't think anything
both server(nodejs) and client(js & nodejs) work iirc
okay, then I can experiment a bit with that client to get rid of haxeui hack executing in main loop ๐
sad, that haxelib.json doesn't support dev dependencies. Have to install promises library since it's necessary for the tests ๐
it does I just think there was some issue it caused
Are there any dependencies one requires for mysql on windows, seem to get stuck at "trying to connect". The details work on electron and in heidi. Maybe i'm missing a windows thing?
Getting no errors or anything
bit odd
I have it, app compiles completely. However it just won't connect
what does your connection code look like?
function connect() {
trace('attempting to connect to db');
db.connect().then(function(state) {
if (state.data) {
this.connected = true;
this.connected_time = Date.now().getTime();
trace('Database connected');
} else {
this.connected = false;
this.connected_time -= (Duration.fromString('5hrs') : Float);
trace('Database not connected');
}
}, (err) -> trace(err));
}
works completely fine on nodejs/electron
db = DatabaseFactory.instance.createDatabase(DatabaseFactory.MYSQL, {
database: database,
host: host,
user: user,
pass: pass
});
db.setProperty('autoReconnect', true);
db.setProperty('autoReconnectInterval', 5000);
db.setProperty('replayQueriesOnReconnection', true);
seems fine ๐ค
i'm sure it did work at some point, but for some reason nothing now
this is a new install of windows
but I just get stuck at attempting to connect to db system/DatabaseSystem:63 and nothing happens after that
no errors, no missing dlls no failed to connect messages
i would try and run the unit tests in libmysqlclient, make sure they work
you'll need to add env vars: https://github.com/core-haxe/libmysqlclient/blob/main/tests/TestAll.hx#L12-L14
or just hardcode the details: https://github.com/core-haxe/libmysqlclient/blob/main/tests/cases/util/DBCreator.hx#L11-L14
is the port 3306?
it is, heidi connects to 3306 successfully
whats heidi?
and yeah, you said its fine for nodejs
so yeah, best advice is to to see what the unit tests say from the lowest possible part of the stack (libmysqlclient)
thought I may had to install something on windows like on Linux
but I'll try in a new project
import haxe.Timer;
import db.DatabaseFactory;
class Main {
static function main() {
trace('here');
var timer = new Timer(1000);
timer.run = () -> {
trace('running');
}
var db = DatabaseFactory.instance.createDatabase(DatabaseFactory.MYSQL, {
database: 'test',
host: "127.0.0.1",
user: "root",
pass: "123456"
});
db.connect().then(function(state) {
trace('here');
if (state.data) {
trace('Database connected');
} else {
trace('Database not connected');
}
}, (err) -> trace(err));
}
}
``` just setup a mysql server locally to make sure it isn't some config on the server side
this also seemingly fails to connect
same code connects on node, weird
message : Invalid JSON text: "not a JSON text, may need CAST" at position 0 in value for column 'uisettings.bool'.,
I'm getting this error from mysql/dbcore, with the following data:
_record : {
name : SyncWicks,
bool : true
}
but true/false is valid json, and if I insert directly into the table it seems to work fine with no cast
Not sure where in the chain things are going wrong
could "bool" be reserved?
wouldn't it prevent me from using it as a table name or class name?
named it to flag but have the same issue
this took me quite a bit to hunt down ๐ญ
we can't set records to null values!
ah, there's empty
ahh, but its awkward when working with macros ๐ค
maybe i'll just set it to an empty string instead
Would you be cool if i pr'd setField()?
same function just without the null check
seems like it messes with the query stuff - gets into weird situations like id=null. Just gonna stick with empty(field)
maybe that field thing is ill concieved, should probably setField and getField
its kinda "hidden" behaviour where the function acts a setter depending on what you pass in, which isnt very smart really
it does solve an issue where it won't add empty fields to the list which is convenient
sending id = null, where id is a primary key, auto increment causes a mysql syntax error
Hello, since I am depending on promises library I need to know which license it has. I want to publish my haxe amqp library to haxelib which is planned to be released under MIT and I guess I've to add a license reference to promises
Okay, just saw that it's under MIT, so I've to see how to reference it correctly when publishing it to haxelib ๐
Just put the full mit license in the root of the git repository and haxelib and set the license in the haxelib.json
So for release put both licenses into one file and reference it within haxelib.json?
but in license i cannot reference the file, but since both licenses are MIT as far as I see, MIT would be fine right?
In case that is fine with just concatenating promises LICENSE and my own LICENSE I can release my amqp library to haxelib ๐
No need to concat as long as you keep the original promises license
Okay, the original promises license remains within the subfolder src/promises. Only thing that is going to be excluded from zip command is the .git folder
Sounds good
i know nothing about licences, suffice to say, all the core-haxe stuff is "yours"... i really dont care what you do with it... so if there is a more permissable licence that allows that, just lemme know and ill make that chage (i literally dont care about what you do with the source files, they are "yours" as far as im concerned)
(i thought MIT was the most permissive, maybe not?)
I have haxeui ceramic under this license
seems like the most anti license, license ๐
issue with unlicense is that some large companies don't allow their developers to use code under it
google is the big one
i don't completely understand the reasoning, but MIT is good enough
Google does not allow its employees to contribute to projects under public domain equivalent licenses like the Unlicense (and CC0), while allowing contributions to 0BSD licensed and US government PD projects.[6]
from wikipedia
fedora doesn't allow unlicensed stuff too
oh... indeed to copy a message https://lists.opensource.org/pipermail/license-review_lists.opensource.org/2012-January/001386.html
Paragraph (and sentence) #1 professes to put the covered work into the
public domain. As mentioned, paragraph 2 professes to be a grant of
rights normally reserved by default to a copyright owner, which makes no
sense given that the preceding sentence professed to eradicate the
work's quality of being ownable. _However_ (upon reflection), in itself
that would be harmless if redundant and pointless: One can interpret
paragraph 2 as an elaboration of the consequences of the first
paragraph.
Paragraph 3 is mostly further explanation of the concept of public
domain, and therefore harmless if not useful. Its middle sentence
elaborates that the erstwhile author aims to bind heirs and successors,
too (which is a logical inclusion, irrespective of whether it works).
Paragraph 4, though, is the one that would be amusing if it weren't
tragically broken: It's the warranty disclaimer. People accepting the
covered work are obliged to accept the condition of no warranty,
otherwise there is no licence. Except, oh, wait: Paragraph 1 professed
to put the work in the public domain, so the erstwhile owner has sawed
off and evaporated in paragraph 1 all power to require the condition in
paragraph 4.
interesting, kinda what i was putting together
really? there's no 'I don't care' option is there
0bsd
I'd be surprised if Google wanted to use my haxe projects tho ๐
Copyright (C) [year] by [copyright holder] <[email]>
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
simple enough
they have almost 200,000 employees
rolooking at this problem
issue might be related to my newer haxe/hxcpp versions perhaps
It's haxe version
the test code above connects with haxe 4.3.6
fails to connect with nightly haxe :/
hxcpp git doesn't seem to matter
git hxcpp might actually matter if there's haxe version checks
found a nightly haxe version that lets my app compile, can now do some profiling with tracy 
that's interesting, so core-haxe auto converts Date objects
is the timestamp field type supported? I can't recall if i've tried before
I can't seem to find any date query api's
but it's not a big deal to use the ultra cool raw queries ๐
when you say date query apis, you mean sql data functions?
so i can do queries like
SELECT * FROM your_table
WHERE timestamp_column BETWEEN '2025-04-03 00:00:00' AND '2025-04-03 23:59:59';```
my current timestamp column is a double but i'm quickly finding this is not very great for managing queries but I don't know how much compatibility there is for queries around the date column types
cant you just use date objects directly? Just about to head into a meeting, but if that doesnt work, its an api hole - fairly sure i can use less than and greater than in date queries - but maybe i imagined that
I'm not sure, i thought i tried it in the past but maybe what i did was pass a timestamp float to the query instead
so you would do hx query($dateColumn >= '2025-04-03 00:00:00' && $dateColumn <= '2025-04-03 23:59:59');?
i think they might need to be haxe Date objects
was query sorting added?
... ... i think so, dont remember now, lemme check
cant see anything in the unit tests
nope - still not implemented im afraid
no worries, thought i remembered something about it being implemented but guess i was remembering wrong
have a raw query setup anyway and its easy to switch out whenever its added
thought i remembered something about it being implemented but guess i was remembering wrong
same tbh, but no, nothing there
has the core stack been tested on mac? (ws, http, db)
or is it expected to work/not work
http will probably work, ws may be hit or miss but db is the one i'm not so sure about ๐ค
i would expect them all to work
i dont think they have been tested deeply, although, i have an airbook now, so could (finally)
but i think i would expect them to "just work"
although, now im thinking about it, i wonder about libmysql
hmmm, ok, yeah: https://github.com/core-haxe/libmysqlclient
thats not statically linked from cpp source (because it was a major PITA to extract just the client from mysql)
and yeah, no mention of osx: https://github.com/core-haxe/libmysqlclient/blob/main/Build.xml
unknown mysql field type (daylow:246), detected as 'NEWDECIMAL' [202000.0000000000] mysql/MySqlClientResult:211
it works for nodejs. it's meant to be "the type" to use for currencies
a bit cumbersome to work with on the haxe side tho. seems to be passed around as a string, at least on nodejs hxcpp might be different
so got to parseFloat it all the time
I suspect this is the mysql db that wants the ssl dllsa
do you have a minimal example?
so yeah, i think it is a dep of mysql, and its not compiled from source (mysqlclient) so its not going to be possible to statically link it either
i wonder if you could just turn of ssl (its an option): https://github.com/core-haxe/libmysqlclient/blob/main/lib/include/mysql.h#L271 - feels dodgey though
another option would be to maybe use MariaDB's C connector - which i think (iirc) is decoupled from everything else - yeah, seems to be: https://mariadb.com/docs/server/connect/programming-languages/c/
CREATE TABLE `testtable` (
`id` INT NULL,
`amount` DECIMAL(20,6) NOT NULL
)
COLLATE='utf8mb4_0900_ai_ci';
INSERT INTO `testtable` (`amount`) VALUES (2400.03);
db.table('testtable').then((result) -> {
return result.table.findOne(query($id == 1));
}).then(function(result) {
var rec = result.data;
trace(Type.typeof(rec.field('amount'));
}, (err) -> trace(err));
I think that would do it
trace(Type.typeof(rec.field('amount')); says its a string?
not sure what you mean, but its fine, ill put it in a UT and see whats going on
my guess is there is an auto conversion case its not hitting
unknown mysql field type (daylow:246), detected as 'NEWDECIMAL' [202000.0000000000] mysql/MySqlClientResult:211
Is the output on cpp
OK, that seems to come from mysql though, which seems be saying its not happy with teh DECIMAL.. ?
maybe the field type is really new?
and just not part of the version shipped with dbcore
this was what i googled so i just used decimal
sure, im just saying that the error message on hxcpp seems to be coming from mysql, not the db-core stuff
... ... actually...
mysql/MySqlClientResult:211 is a core file no?
ah lol
oh huh
ahh there's cpp stuff that has to be done here, thought I might have been able to just add the condition x)
can repro
defo does feel dodgy
is it possible to just place the dlls "somewhere" for hxcpp/the app to just detect?
could be put next to the .lib file, but you'd need to copy it to the correct place (via a macro possibly)
would this work cross platform?
nope
or i guess i'd need to find a solution for each os
you would have to have, say, 3 versions and copy appropriately
but not sure if that would be ok for linux
or if the assumtion could be made that there will always be openssl on linux - not sure
could just be a windows thing then
might have to test that at some point, the awkward thing is all my linux setups are dev environments
long term I might be migrating away from mysql in the project anyway and use an api setup instead. But just makes the short term sending to non technical friends to play with a bit awkward
it's not 100% guaranteed that linux would have openssl installed, it's probably safer to have openssl packaged with the binary and to set the executable rpath so it can load it
you mean package as a haxe resource in the app and extract it?
no, just distributed alongside the binary
right ๐
that would be quite the hack ๐
yeah, i was going to say, would that even work?!? It might need the .dll / .so to even run
depends on how it is linked/loaded
yeah
if it was with dlopen then you could do that ๐
would it be a single .so? or a bunch of different ones for different flavours
i think it should be, im not sure of the details of openssl
im actually wondering now if you could statically link openssl into the app and that would be enough, might need to do some shenanigans on the libmysqlclient.h
yeah, that would be a simpler solution
https://github.com/openssl/openssl - openssl is the quite the beast (unsurprisingly)
no unit tests is mysql (https://github.com/core-haxe/mysql), just the bits above (db-mysql) and below (libmysqlclient)... ๐ฆ
oh wow, that's an easy commit - thanks!!
just writing unit tests for mysql so i can see about the parseFloat thing
a side question, i don't know how this stuff works
floats tend to have precision issues
in this case is there any chance that an input of Std.parseFloat(22.03) can output something silly like 22.0300000001
or will it be 1:1
and the precision issues only occur when doing maths
i've never seen that occur when doing parsefloat stuff just want some verification
i think it should be fine... there may be a better candidate than float, though i think float is a double in hxcpp, so i think it will be fine (think)
i'll take it
i've been using double in mysql and doing the parsefloat stuff everywhere internally anyway
issues only seem to occur when doing math ops
and then I'll round to decimal places
rounding in crypto can be a dangerous strategy ๐
i take into account the precision level of the token
yeah, makes sense
and rounding is mostly for surface viewing
i'm sure i've made mistakes here and there
the db abstraction is quite "fun" actually ๐
i tend to just use entities2, so forgot about working with the "guts" of db-core
it's nice, i have my own kind of abstraction on top of it
it fits in a bit better with my general app flow
heathen!
yeah, its super nice, i only realised the other day that in one project, i was using it with sqlite3 and in another mysql... didnt even really occur to me, doesnt even feel like its database'y... only playing with haxe objects... sort of completely forget they are persisted in the db - and that between two projects a totally different db type
sounds nice
the only difference between the two (apart from the actual entities ofc), is:
EntityManager.instance.database = DatabaseFactory.instance.createDatabase(DatabaseFactory.SQLITE, {
filename: filename,
journalMode: journalMode
});
and then its just stuff like:
package das.entities.customers;
class CustomerSiteData implements IEntity {
public var customer:CustomerData;
@:size(255) public var name:String;
@:size(255) public var uniqueId:String;
public var configString:String;
public var routingSlips:Array<RoutingSlipData>;
public var initiators:Array<JobInitiatorData>;
@:cascade public var accessTokens:Array<CustomerSiteAccessTokenData>;
...
public static function findByUniqueId(uniqueId:String):Promise<CustomerSiteData> {
return find(Query.query($uniqueId = uniqueId));
}
public static function findByCustomer(customer:CustomerData):Promise<Array<CustomerSiteData>> {
return findAll(Query.query($customer = customer));
}
}
Db.getLast('balance', 'aid', Globals.aid, 'timestamp', (record) -> {
var bal = DBBalance.fromRecord(record);
trace(bal.tableColumn);
});
Db.get('account', query($aid == Globals.aid && $uid == Globals.user_id), (record) -> {});
Where fromRecord is a macro and these functions just map to haxe enums and get passed to a central database system to get run. A lot of the "fluff" is abstracted away
yours seems to go a step further though, by just putting it all on the objects itself
yeah, and creates linking tables, and query caching, etc... its pretty powerful
query caching sounds cool, i have that on my todo list ๐
that the schema it creates, by itself, you literally just concern yourself with the objects, and almost never dive into queries (although, you use the query abstracting in db-core ofc sometimes)
maybe i'll bump up e2 investigation sooner
ah, maybe not, i don't know. I might move the app to an api model and move db to purely server backend
in one app (glint) the db entites are also used as part of an api... which is nice (using core-haxe/seializers)
how does that work?
so you pass objects around that are "db ready"... but entities2 has the concept of "fieldsets" so you can strip certain data
literally just "implements IHaxeSerializable" and now you have serialize and unserialize functions
so many overlapping projects ๐ฉ
was literally just talking to jeremy about his serialising stuff in tracker lol
the RPC system in glint automatically detects these functions and macros build apis and you just implement the code of the "operation"
ill see if i can find a simple exampe, two secs
@:expose @:keep
@:operation(glint.api.users.UsersApi.findUsers)
class FindUsers implements IRpcServerOperation {
private static var log = new Logger("glint.api.users.server.operations.FindUsers");
public function execute(criteria:FindUsersCriteria, requestor:UserData):Promise<Array<UserData>> {
return new Promise((resolve, reject) -> {
log.info('finding users, criteria: ${criteria} (requestor: ${requestor != null ? requestor.username : null})');
var queryParts:Array<QueryExpr> = [];
if (criteria.name != null) {
queryParts.push(Query.query($username = criteria.name));
}
if (criteria.searchTerm != null) {
queryParts.push(Query.query($username =~ criteria.searchTerm));
}
var query = Query.joinQueryParts(queryParts, QBinop.QOpBoolAnd);
UserData.findAll(query, criteria.pageSize, UserData.NoSensitiveInformation).then(users -> {
resolve(users);
}, error -> {
reject(error);
});
});
}
}
typedef FindUsersCriteria = {
@:optional public var name:String;
@:optional public var searchTerm:String;
@:optional public var pageSize:Null<Int>;
}
interface UsersApi {
public function findUsers(criteria:FindUsersCriteria, requestor:UserData):Promise<Array<UserData>>;
}
then there is an api macro that builds up the client api code (via RPC) so you just go:
Api.instance.users.findUsers(...).then(users -> ...
and it makes the round trip to the server, objects are auto serialized / unserialized, etc... and, from the client persective, you just get an array of users... super super simple (lots happening under the hood though ofc)
oh, and the rpc operations are all discrete js modules, that are loaded at start up and "auto registered" by the server, which means adding new operations is stupidly simple
what i like about this is the centeralised queries
they're all organised in a single place
rather than scattered around the app
defo, the idea of entities2 (and 1 really) is that you just dont use queries, i mean, obviously you use db-core queries like above, but you certainly never need insert, update, delete queries etc... you simply just use the objects
it feels verrrrry natural (although, ofc, im biased)
@toxic stratus - i dont get that parseFloat issue
function testBasicSelect(async:Async) {
var connection:DatabaseConnection = null;
DBCreator.createConnection("persons2").then(c -> {
connection = c;
return connection.query("SELECT * FROM Person");
}).then(result -> {
Assert.equals(4, result.data.length);
Assert.equals(result.data[0].personId, 1);
Assert.equals(result.data[0].firstName, "Ian");
Assert.equals(result.data[0].lastName, "Harrigan");
Assert.equals(result.data[0].iconId, 1);
Assert.equals(result.data[0].amount, 123.456);
Assert.equals(result.data[2].personId, 3);
Assert.equals(result.data[2].firstName, "Tim");
Assert.equals(result.data[2].lastName, "Mallot");
Assert.equals(result.data[2].iconId, 2);
Assert.equals(result.data[2].amount, 222.333);
connection.close();
async.done();
}, error -> {
trace(error);
async.done();
});
}
the .amount stuff is a float, works fine on nodejs
(and hxcpp)
didn't you just start parsing it as a float? ๐
i thought you said you had the issue on nodejs... nodejs doesnt use libmysqlclient
it uses mysql2 npm lib
wait a second, how are you testing it
could be db-core actually i suppose, just though it might be mysql - quite a few layers here i suppose - ill check that now
wdym? I just wrote unit tests
on js "123.456" == 123.456
it won't crash
it will just "act normally"
but the type is actually a string
lol, oh yeah:
var x:Dynamic = "123.456";
Assert.equals(x, 123.456);
that... erm... seems like a hole in utest
it's a haxe quirk if you ask me lol
i defo would have expected utest to fail there
and yeah trace(Type.typeof(result.data[0].amount)); is a string...
Ok, thats good then, unit tests were useful to add... sucks that asset didnt pick it up though
im trying my hardest not to write a unit test lib
lol
haxe has some good support for some unit test libs
no idea how it integrates with them all
utest has some quirks im not a fan of tbh
that Assert not taking the type into consideration might be the "final straw"
it trips me up so many times in haxe in general
i didnt even know about it
like, this is quite laborious to have to do:
switch (Type.typeof(result.data[0].amount)) {
case TFloat:
// pass
case _:
Assert.fail("should be a float");
}
the incoming is dynamic, it can't really know anything about the type other than what you tell it to be
and then its just down to the target differences
yeah, makes sense i guess
so really, its js being insane
... so it comes back from mysql2 as a string... nice
I think it is stored as a string on mysql's side
that might be intentional behaviour
to preserve precision
right, makes sense
its just you can still treat it as a number in mysql land
it'll get indexed as a number, you can do query maths on it etc
thankfully i already have code that converts js buffers into haxe bytes objects, so just need to expand on that
alright, should be fixed now @toxic stratus
nice - thanks!
it could also just be an explicit dependency which the user has to install
oh that's an old message
unknown mysql field type (json:245), detected as 'null'
What type should I map it to if it says "detected as null"
seems to be tripping up on the json column type now (cpp only - nodejs works fine)
public static var JSON:Int = untyped __cpp__("FIELD_TYPE_JSON");
doesn't seem to get picked up
./src/mysql/MySqlFieldType.cpp(394): error C2065: 'FIELD_TYPE_JSON': undeclared identifier
my guess is the libmysqlclient being used doesnt work with JSON field types
not sure when it was introduced... ... cant find a version in the mysql.h used in the haxe externs either :/
... actually, i wonder if ive used the wrong fields:
json fields were added in 5.7+ though
ive been using these: https://github.com/core-haxe/libmysqlclient/blob/main/lib/include/mysql_com.h#L961-L988
but there is also this enum: https://github.com/core-haxe/libmysqlclient/blob/main/lib/include/field_types.h#L54-L89, which does contain the JSON
and in fact, the #def version is just using that enum version
oh, i might have it
yeah mapping to field type works, but not sure about the type of the rawdata
{"lev": 1, "qty": 0.05, "name": "defaults", "tpsl": true, "symbol": "BTCUSDT", "stop_roi": 10, "take_roi": 20, "stop_limit": false, "take_limit": false, "trade_limit":
true}
TBool mysql/MySqlClientResult:193
It mapped to bool? ๐ญ
what did you map it to though? And how... really, the lib should be using the enum directly and not the #defs
@:headerCode('
#include "mysql.h"
#include "mysql_com.h"
#include "field_types.h"
')
class MySqlFieldType {
public static var JSON:Int = untyped __cpp__("MYSQL_TYPE_JSON");
public static var DECIMAL:Int = untyped __cpp__("MYSQL_TYPE_DECIMAL");
public static var NEWDECIMAL:Int = untyped __cpp__("MYSQL_TYPE_NEWDECIMAL");
public static var TINY:Int = untyped __cpp__("MYSQL_TYPE_TINY");
public static var SHORT:Int = untyped __cpp__("MYSQL_TYPE_SHORT");
public static var LONG:Int = untyped __cpp__("MYSQL_TYPE_LONG");
public static var FLOAT:Int = untyped __cpp__("MYSQL_TYPE_FLOAT");
public static var DOUBLE:Int = untyped __cpp__("MYSQL_TYPE_DOUBLE");
public static var NULL:Int = untyped __cpp__("MYSQL_TYPE_NULL");
public static var TIMESTAMP:Int = untyped __cpp__("MYSQL_TYPE_TIMESTAMP");
public static var LONGLONG:Int = untyped __cpp__("MYSQL_TYPE_LONGLONG");
public static var INT24:Int = untyped __cpp__("MYSQL_TYPE_INT24");
public static var DATE:Int = untyped __cpp__("MYSQL_TYPE_DATE");
public static var TIME:Int = untyped __cpp__("MYSQL_TYPE_TIME");
public static var DATETIME:Int = untyped __cpp__("MYSQL_TYPE_DATETIME");
public static var YEAR:Int = untyped __cpp__("MYSQL_TYPE_YEAR");
public static var NEWDATE:Int = untyped __cpp__("MYSQL_TYPE_NEWDATE");
public static var ENUM:Int = untyped __cpp__("MYSQL_TYPE_ENUM");
public static var SET:Int = untyped __cpp__("MYSQL_TYPE_SET");
public static var TINY_BLOB:Int = untyped __cpp__("MYSQL_TYPE_TINY_BLOB");
public static var MEDIUM_BLOB:Int = untyped __cpp__("MYSQL_TYPE_MEDIUM_BLOB");
public static var LONG_BLOB:Int = untyped __cpp__("MYSQL_TYPE_LONG_BLOB");
public static var BLOB:Int = untyped __cpp__("MYSQL_TYPE_BLOB");
public static var VAR_STRING:Int = untyped __cpp__("MYSQL_TYPE_VAR_STRING");
public static var STRING:Int = untyped __cpp__("MYSQL_TYPE_STRING");
public static var CHAR:Int = untyped __cpp__("FIELD_TYPE_CHAR");
public static var INTERVAL:Int = untyped __cpp__("FIELD_TYPE_INTERVAL");
public static var GEOMETRY:Int = untyped __cpp__("MYSQL_TYPE_GEOMETRY");
public static var BIT:Int = untyped __cpp__("MYSQL_TYPE_BIT");
}
added the #include fieldtypes.h
hmmm, i suppose that could work
i'm a complete cpp/cpp extern noob was just seeing if i could fall into a lucky solution x)
... refactoring, 2 secs
alright try latest
got null errors in my code. trying to figure out source
differences between js and cpp ๐
so its working?
i'm not getting unmapped type error anymore
just haven't confirmed with actual data input
im parsing the field value too for good measure - but not sure thats wise... i think it is
i think it would be fine as well
that's how it works on nodejs
and it works okay
cool
issue is a completely different part of my code that i'm using to load the json fields doesn't seem to work at all on cpp
db.table(table).then((result) -> {
trace('here');
trace(Query.queryExprToSql(query));
return result.table.find(query);
}).then(function(result) {
trace('here');
trace(Query.queryExprToSql(query));
trace(Json.stringify(result.data));
if (result != null) {
callback(Records(result.data));
} else {
callback(Error('No data', result.data));
}
}, (err) -> trace(err));
this is utter madness ๐
One single, simple query, never receives the result from .find only on cpp
uid = '0c82d31e-79c1-11ef-96e7-0201461afda4' system/DatabaseSystem:258
how the heck am i supposed to repro that ๐
how do I print out the full query again
the issue only occurs on cpp so might be some quirk with the query syntax somewhere
SELECT * FROM
uisettingsmysql/MySqlClientConnection:59
WHERE (uisettings.uid= "0c82d31e-79c1-11ef-96e7-0201461afda4")
Nope, query is completely valid
yeah i'm stumped, seems like a similar issue to my haxe 4 vs haxe 5 thing, except this is also happening on haxe 4
it doesnt get to the second "here"?
what is the table def? is that ui settings json?
Yeah
CREATE TABLE
uisettings(
idINT NOT NULL AUTO_INCREMENT,
uidVARCHAR(50) NOT NULL COLLATE 'utf8mb4_0900_ai_ci',
titleVARCHAR(50) NOT NULL DEFAULT 'default' COLLATE 'utf8mb4_0900_ai_ci',
categoryVARCHAR(50) NOT NULL DEFAULT '0' COLLATE 'utf8mb4_0900_ai_ci',
jsonJSON NULL DEFAULT NULL,
isdefaultINT NOT NULL DEFAULT '0',
numberINT NULL DEFAULT NULL,
flagJSON NULL DEFAULT NULL,
stringTEXT NULL DEFAULT NULL COLLATE 'utf8mb4_0900_ai_ci',
PRIMARY KEY (id) USING BTREE
)
COLLATE='utf8mb4_0900_ai_ci'
ENGINE=InnoDB
AUTO_INCREMENT=28
;
ill see if i can repro in a UT
oh huh, just noticed i can make isdefault a json and get an actual bool type in the db without the getter ๐
do you delete the db at any point?
"drop db ... "
i could be totally wrong, and might be something completely seperate, but i seems that in the UT, when i drop the db first, the _handle pointer is no longer valid... ... ... which i think makes sense.. ... ?
never do
oh... drop the db
read that as drop the table
i don't think so, at least not intentionally. It happens on app init right after connecting
It happens on app init right after connecting
so you do drop the db? Weird it works for you, maybe thats not the issue im seeing and im assuming
can you specify what you mean by dropping the db? I don't disconnect intentionally
i meant the problem occurs right after connecting to the database
and other queries go through fine
before and after that particular one
DROP DATABASE IF EXISTS persons2;
yeah I defo don't do that anywhere myself
if i put that at the end of my UT, its all fine, if i put that at the start, before creating the db (which should also be fine), then i get errors and it seems the _handle is invalid
its probably unlrelated to your issue, but seems like it should be fixed (or at least understood)
nvm... im dumb as fuck... the delete code in the unit test also closes the connection... ๐คฆโโ๏ธ
lol just a consequence of not working with code for a while ๐
i wrote this code like a week ago... ๐คซ
i get an error with this:
connection.query("SELECT * FROM Person WHERE (settings.intSetting= 22)");
Unknown column 'settings.intSetting' in 'where clause'
the table doesn't have a column intSetting
mine does
ahhh
promises.push(query.bind(connection, "CREATE TABLE IF NOT EXISTS Person (
personId int AUTO_INCREMENT,
lastName varchar(50),
firstName varchar(50),
iconId int,
contractDocument blob,
amount DECIMAL(20,6),
settings json,
PRIMARY KEY (personId)
);"));
am i missing it
missing what?
cgpt says this is the correct what to query: SELECT * FROM Person WHERE (settings->'$.intSetting' = 22) and indeed that works
i get errors in both hxcpp and nodejs for the "dot" method
ohhh, settings.intSetting is in the json i gotchu
yessir
oh, lol, i read this totally wrong then
i thought uisettings was a json object... oh well, well, some nice json column unit tests i guess ๐
lmao
i cant repro it at all
have you tried using backticks in your column names? Its reaching, but maybe its it
in the CREATE TABLE
so:
`uid` VARCHAR(50) NOT NULL COLLATE 'utf8mb4_0900_ai_ci'
and in the query, you can try a few things:
remove the table name, so:
WHERE (uid = '0c82d31e-79c1-11ef-96e7-0201461afda4')"
and try and using quote backticks for the column name there too
rendered sql does have the ticks
discord just parsed them so it didn't show
yeah, technically, you should defo always backtick any identifiers
WHERE (`table`.`column` = 'value')
kinda stuff
doesn't db core do that tho
can you paste the full raw query...
yeah, should do
ok, so yeah, ive testing this on mysql, maybe its a db-core issue ๐ค
SELECT * FROM `uisettings`
WHERE (`uisettings`.`uid` = "0c82d31e-79c1-11ef-96e7-0201461afda4");
the query works in heidi
I went to MySqlClientConnection.query and traced the sql param
yeah, so im following now, its db core that built the query, but it seems fine
yeah sorry, i just went digging to try and find where things were going wrong
maybe try a "raw" query there and see if you can whittle it down
oh cool, get to make a cool new Raw(sql) event for my db system ๐
even I tried to put off doing that lol
well, tbh, it would be more to understand the issue, if you find a raw query that works, id be interested to know why its different to the generated one
i dont personally think there will be a difference... the query looks perfectly valid, works on nodejs for you, and works if you just paste it in
so i think something else is happening
one thing that feels a little off for me in the generated query is that its using double quotes, i suppose they are interchangable, but it certainly initially look "weird" to me
do you happen know ANSI_QUOTES is enabled in your db?
no idea but can find out
so i can't seem to get raw to work with this as well
Database connected system/DatabaseSystem:68
SELECT * FROM uisettings WHERE (`uid` = "0c82d31e-79c1-11ef-96e7-0201461afda4"); system/DatabaseSystem:356
Invalid char 0 at position 0 system/DatabaseSystem:357
db.table(table).then((result) -> {
return result.table.raw(sql);
}).then((result) -> {
if (result != null) {
callback(Records(result.data));
} else {
callback(Error('No data', result.data));
}
}, (err) -> {
trace(sql);
trace(err);
});
Note, that I whittled down the sql to that to try to identify what char 0 is
i tried with this sql first
both work fine in heidi
| sql_mode | ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO _DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION |
defo disabled
found another command I could run
what about:
SELECT * FROM `uisettings` WHERE (`uid` = '0c82d31e-79c1-11ef-96e7-0201461afda4')
same deal
Invalid char 0 at position 0 system/DatabaseSystem:357
var query = "SELECT * FROM `uisettings` WHERE (`uid` = '0c82d31e-79c1-11ef-96e7-0201461afda4');";
Db.raw('uisettings', query, (records) -> {});
whats the first param to "raw"?
table
yea, might not be worth much more effort for now, all other queries seem to work fine
it's a strange one
looks similar to the no connection on nightly haxe
same kind of weirdness
yeah ...
SELECT * FROM
uisettingssystem/DatabaseSystem:356
Invalid char 0 at position 0 system/DatabaseSystem:357
issue occurs on linux as well
it has to be something with the db engine you are using... i mean, some setting or something
not saying you need to change it... just find out what it is
does your remote db have any logging or anything that might give info?
mmm not sure, i'm just using the mysql server that's installed on ubuntu
i don't particularly mind changing it if you have a suggestion
i defo dont... just its the only thing remaining i can think of
what do you use in the unit tests?
you know....
that error looks like a json error rather than a mysql error
very interesting....
I deleted all the entries in the table
and all of a sudden the query goes through
You know what it might be
what is a mysql NULL value represented as? Could it be that that is invalid json?
i wonder if i set the column default to an empty string if it will all just magically start working
json field can't have a default value ๐คฆโโ๏ธ
ok, that might make sense though, maybe the json parse i added is trying to parse bad json
although, hang on, this is coming from the sql server
is it?
but the server accepts the value ๐ค
could be a bug in the mysql externed tho
system/DatabaseSystem:357? "Invalid Char at pos 0,0"... thats from the server, no?
its not even getting to the "haxe.Json.parse" part
it's the promise response from the query, i'm not sure where it is in the backend
but i'm just assuming here, it looks more like a json error than a mysql error
well, its not in the response processing, im fairly convinced of that (happy to be wrong ofc)
maybe i do have invalid json... lets validate the entries first
all entries are valid
will now try deleting all the null ones and see if i get a response
you can try and remove the parse here: https://github.com/core-haxe/libmysqlclient/blob/main/src/mysql/MySqlClientResult.hx#L134
(if you can repro)
but i suspect it will make no difference, i dont think its even getting that far, i think its failing on making the query, not handling the result
mysqlclient externs for hxcpp. Contribute to core-haxe/libmysqlclient development by creating an account on GitHub.
I got rid of all null json values in the db and it works
i got my json settings and they seem to be parsed fine
will see if it reaches that point with a null entry
Yeah, you're right, it doesn't even get to that stage
repro is easy
just add a row of data to your persons2 table with the settings field be set to null
root@ubuntu:~# mysql --version
mysql Ver 8.0.42-0ubuntu0.22.04.1 for Linux on x86_64 ((Ubuntu))
so odd
oh it has to be an expression! my bad!
Just curious, but why ever deal with the database directly? Do you not like ORMS?
Is that what NB is using?
he is using db-core directly
โ๏ธ
I'm using db-core, not using it directly
from what it sounds like its around 10 levels below entities2 from an abstraction pov
any reason why entities2 isn't just entities lol
but, as an fyi, entities2 is built ontop of db-core
because there is entities1... really it needs to be deprecated now and entities2 should just be entities
https://github.com/core-haxe/entities
(oh, i archived it, i dont remember that ๐ )
mostly don't wanna intro it whilst i'm still in the building phase
get a better feel for things at this lower level
you are missing out :p
entities fan? ๐
i used it for a small project and it was great
the least awful ORM I have used in any language
(looking at you gorm)
damn that's a good review
yeah, ive used a bunch, and its defo my fav - i like also that it just works nicely with serialization etc... so i send entities back and forth between client and server (via RPC) and it all "just works"...
and no leaking of data because of "fieldsets" (that was one of the issues with entities1)
Might use it for my own stuff then too
it's okay, its on the list for when i start looking at backend things
How does it compare to record-macros?
i think the ethos is quite different... entities is more "just dont think about the db anymore"
you just deal with objects... entities will take care of all the data, including 1->1 and 1->many relationships
A list you say
what else is on this list? And when will you start doing your exploration?
Oh that's cool
hopefully soon, priority #1 is get the frontend usable for me
backend stuff is generally simple and quick for me to get done
but frontend is always a ball ache
yeah, i played with record macros, and its nice, but its very "manual"... i just generally dont want to think about the db layer anymore - ive written that code 100 times before, and i dont need to write it again ๐
I wonder if you can take that same approach for networking?
another thing i may explore at some point is making an api service
ooooh sounds fun
i already kind of feel it tbh
it kind of just gets in the way
i should do a weblink+entities2 project someday
i have some verrrrrry cool libs to do that all nicely too ๐
unreleased currently, but they work crazy great
lmao
all using entities, rpc, http, etc
โ๏ธ
from around there
yeah, i want to be able to "secure" some things a bit better and having a live db connection in app as far as i understand it isn't a great thing x)
one less thing to rely on for the client sounds nice in general
why? You going to connect / reconnect each query?
db-core already uses connection pooling btw
I think if I hadn't started working on go2hx before this set of libraries came out, I likely would've been a contributor to it, I like the selection
security reasons, if i'm going to be handling peoples api keys with access to money, which i almost certainly will be doing for at the minimum my frends and myself, i want it to be a bit proper
yeah, thats an endorsment that makes me happy ๐
tbh, these sets of libs are ones ive using for years in production apps, but thought i should probably share them because they have been insanely useful to me... also, i stops them being a "copy it here for this project" and makes them more "real"
i dont see how having a live db connection has anything to do with that tbh
That's a great way to build nice high quality libraries, I am inspired by using them in such tandem with your production work
Theoritical is great and all, but production is king.
100%... writing a lib for other people never works out imo... writing a lib for yourself that other people might find useful is the way to go
even if no one uses it / them... its perfectly fine
fair, just wasn't thinking x)
bunch of thoughts scattered around
haven't gotten to that point yet
yeah, fair do's ๐
haha
then no one gets hurt
Yeah I agree with that completly, I had to change my mentality to that because I was getting burned out for a while thinking what others want and not doing what I really wanted at the start, it's a common trap I see now with others too.
im good at those too ๐
took me a while to shift but i got there in the end
10000%... its very easy to spot once you stop doing it yourself tbh
when you're building something for yourself the priority is the "product/thing"
That's great to here, yeah we talked about this a little before if I recall correctly
and when you're building it for someone else the priority is "returns"
maybe not an all encompassing statement to make, but, its certainly an "off" value proposition
i think, for me at least, its about not being able to "fully test"... you are inventing scenarios, non existant usually, its literally creating problems for solutions... with the "its for me" its all about actually making it work for a specific issue... if you can make it a lib and share it across products, even better... and if someone else can glean some use out of it... then even better better
defo, and real use leads to real progress
yeah that is a great statement
entities1 was mostly created as a "im going to have to do some db stuff in a bit, and i dont want to, so ill make a lib"... and it worked, which was great... but when i actually started using it in said product(s), its clear there was design issues (hence entities2)
When you have an exact use in mind and you need it, you can prioritize correctly, when you build for others, it's really hard to map out the priorities because a group of user's wants and needs are constantly fluctuating and it's hard to give the same level of direction
I tend to rewrite what I use often 3 times before I am happy with the base design.
When I was thinking of just users and not in my own interests, I wasted so much time going down needless rabbit holes and trying to search for problems.
yeah, 100%, i also think you are just wrong / dumb / dim sometimes... you are over thinking, under thinking... with the actual problem in front of you, its much easier to build the solution... i dont "regret" writing entities1, it proved it was viable, but it was "wrong" to write it before i really knew the problem scope...
... though, also, tbf, some of the issues in entities1 were just fundemental, like leaking db fields, its just about being naive and coding way to optimistically
I am so much more content working on a problem I know it exists because it's my own and if it indirectly leads to other problems getting fixed that would be great, but letting go of needing/wanting to solve every use case under the sun has been such a relief.
yeah, Instead of going through some massive debate. I just compile my app to release, use it for a bit and just ask myself "what's missing?" "what don't I like?" "what problems are there?" then I just write down about around 5-10 of them on a piece of paper, tackle the list and repeat
That sounds golden
the kinds of things that are going on the list change over time, there's less bulky things, less backend stuff, "layer 2" details, oh... those are done, what about "layer 3" ones etc
Often time you don't know the problem scope until you run into the problems with your head
defo
yeah makes sense, if the list didn't change over time, it would signal that you're not growing very fast as a dev.
i don't worry too much about that the time contexts
i can just be slow sometimes so i take it into account x)
i'll just make a new list and move items over if i get a bit stuck
Yeah it's not a race
dang
awkward thing with my little "json" flag trick
query($isdefault == true) doesn't bring up any results
I'm guessing it likely needs some kind of special syntax
query($isdefault == 1)? (total guess)
this might have something to do with it, let me retry "true"
yes, $isdefault == "true" works fine when there's actually a true value
pretty cool
and Type.typeof(settings.isdefault) is bool!
this is a neat little setup ๐
i wonder if this is a bug in the query expr though ๐ค
i can't see how the query expression is going to be able to map that to a json field
like, maybe its taking "true" as a string literal and not a bool
dunno, ill have to unit test to see, but i dont like "type is bool" and "query where value is string"
mysql "true" is the equiv of json true