@amber raven finally sitting down to start looking at the merge & fix compile errors. I'll start with mpy-cross.
#v1.20 / v1.21 merge
eternal grail
amber raven
thanks! - i'm waylaid by a nina-fw cert issue at the moment
eternal grail
got it, OK
eternal grail
I have a pile of changes. mpy-cross reaches the link stage but has compile errors apparently because the translated messages are not being linked in. ```/usr/bin/ld: build/py/runtime.o: in function m_malloc_fail': runtime.c:(.text.unlikely.m_malloc_fail+0x17): undefined reference to translation169'
/usr/bin/ld: runtime.c:(.text.unlikely.m_malloc_fail+0x25): undefined reference to translation168' /usr/bin/ld: build/py/runtime.o: in function mp_unary_op':
runtime.c:(.text.mp_unary_op+0x19c): undefined reference to `translation92'
error: failed to push some refs to 'github.com:dhalbert/circuitpython.git'
``` yeah I cannot push ATM.I thought you had given me access the last time but maybe not
amber raven
ok, sent you a github invite
amber raven
before our work yesterday, these were already in runtime.h:
#include "supervisor/linker.h"
#include "supervisor/shared/translate/translate.h"
We somehow missed moving those over. I can put them back. That was kind of the whole point: linker.h and translate.h didn't need to be included if runtime.h was included, which was nearly all the time.
amber raven
do you have any pending pushes? I will fix the linker.h thing to reduce the changes.
eternal grail
no I do not have pending pushes. I will probably work on other things tomorrow.
eternal grail
I'm making progress. Figuring out how to define named tuples in the core was a challenge. Still not linking. Probably working on other ports should wait until Unix is "compiles but fails tests" so we don't duplicate effort
Last problem before I stopped for lunch was having no ability to call "flush" on a file from print()
amber raven
I'm making progress. Figuring out how to define named tuples in the core was a c...
i can wait, I have some admin tasks to do
yes, that namedtuple thing looked like it was going to be a problem. Also I'm not sure if the old buffer_p.get_buffer slot is equivalent to the new plain buffer slot, but I hope so, and I did assume that when rewriting the type definitions
eternal grail
842 tests passed
53 tests skipped: …
35 tests failed: …
@amber raven what I have is pushed, and I'm calling it a day. unix builds and passes quite a few tests!
amber raven
great! thanks!
eternal grail
have a great week-end! next week I think you'll be set to start getting boards to build if that's where your time is to be spent
amber raven
that is what I plan
eternal grail
looks like most of today will be meetings & enhancements related to dot clock LCDs but I'll try to answer any questions that come up for you
amber raven
I'm working on the Trinket M0 build, making progress; still on compile issues. I got a late start today.
amber raven
I've gotta change over to MP_REGISTER_ROOT_POINTER(). that will a bit of time
eternal grail
@amber raven for namedtuples see e48984872b4c41964d63eeee71132083cae2d558
const mp_obj_namedtuple_type_t qrio_qrinfo_type_obj = {
- .base = {
- .base = {
- .type = &mp_type_type
- },
- .flags = MP_TYPE_FLAG_EXTENDED,
- .name = MP_QSTR_QRInfo,
- .print = namedtuple_print,
- .parent = &mp_type_tuple,
- .make_new = namedtuple_make_new,
- .attr = namedtuple_attr,
- MP_TYPE_EXTENDED_FIELDS(
- .unary_op = mp_obj_tuple_unary_op,
- .binary_op = mp_obj_tuple_binary_op,
- .subscr = mp_obj_tuple_subscr,
- .getiter = mp_obj_tuple_getiter,
- ),
- },
+ NAMEDTUPLE_TYPE_BASE_AND_SLOTS(MP_QSTR_QRInfo),
.n_fields = 2,
.fields = {
MP_QSTR_payload,
MP_QSTR_data_type,
},```Trinket comes up, I am scared to try gc:
Adafruit CircuitPython 9.0.0-alpha.1-96-g171aa42168-dirty on 2023-09-28; Adafruit Trinket M0 with samd21e18
>>> 1+2
3
i have to do more root pointer stuff - didn't finish it, but did enough for Trinket
also I found some q's to myself in py/gc.c :/
i'll commit this as a fixed point
eternal grail
16 tests failed: …```making progress! over half of failed tests fixed since my last check-inI checked in some stuff over the weekend. I got Feather M0 to build, though it is too large. Was planning to investigate that next
amber raven
@wary stratus I sm using this thread as a log of what I’m up to on the merge
Currently debugging a ble exception definition compilation problem. Getting Metro M4 to build
wary stratus
👍
I think I'm going to need 1.21 for the autogrowing heap stuff
so I should probably help with the merge at some point too
amber raven
Metro M4 now builds and I pushed those updates. Did not smoke test. Working on RP2040 next
amber raven
Main things I’m fixing are errors in the type definitions and errors in the root pointers. All simple
wary stratus
I got the branch pulled down and will start on tests tomorrow
wary stratus
I'm fixing the extmod/vfs_fat_* tests
wary stratus
I pushed to the merge branch
amber raven
I just pushed a bunch of changes that allow raspberry_pi_pico to build. I had to do a merge with your changes, so pull now maybe to save a merge
wary stratus
👍
wary stratus
my branch has more changes. trying to get the CI going
will rebase the CI fixes before pushing to your branch tomorrow
wary stratus
down to one more error with asyncio stream trying to do an allocation when the heap is locked
wary stratus
CI is going. many builds still failing: https://github.com/tannewt/circuitpython/actions/runs/6436984731/job/17481461547
amber raven
I pushed a few more changes to get stm amd mimx to build. started working on silabs and spresense. silabs has a linking issue. spresense has a problem with the assert() it supplies, which is not an expression but a statement.
frozen modules manifest.py is not being set up right.
The broken builds you point to are weird, looks like mpy-cross is in the wrong place? But I only spent a minute or two on that.
amber raven
my current task is to work on the frozen modules issue
wary stratus
Ya I think the build directory changed. Will be back at my desk tomorrow
wary stratus
my plan is to rebase my changes onto yours in the next couple of hours
wary stratus
I think I have a fix for frozen stuff
wary stratus
looking at the docs build failure while waiting for boards to build
amber raven
I fixed the frozen stuff and pushed it. I did not see your message above.
wary stratus
what are you working on next?
wary stratus
should we merge asyncio #47 before my PR?
amber raven
I just came back to the house so I can work more uninterrupted. I was going to see why the Feather M0 basic build is too large.
did you finish the if 5 merge? Think it is worth me now making a draft PR? Happy to video.
wary stratus
@amber raven I'm at my desk and can video if you like. just back from lunch
had one test I set to skip
amber raven
@wary stratus going into Amelia
amber raven
successfully merged your branch --ff-only to mine
wary stratus
build is going here: https://github.com/tannewt/circuitpython/actions/runs/6488066317
one nrf failure is CI issue
wary stratus
everything else is green so far
wary stratus
just pushed wip cyw43 changes
amber raven
👍 eating dinner
amber raven
mp_execute_bytecode() is hugely larger by about 4500 bytes in the v1.20-merge branches. Will investigate that. Also, Metro M4 build is way smaller by like 90kB (!) -- maybe it is missing something significant
wary stratus
whoa! weird
amber raven
the metro m4 build being smaller is due to ulab being missing. Sounds like something missing in a Makefile
amber raven
The Feather M0 Express build may have to do with some "SUPEROPT_" stuff that was removed but probably needs to be put back.
amber raven
i fixed ulab so it's now included and pushed that change to my branch.
amber raven
I restored SUPEROPT_VM and SUPEROPT_GC, and pushed that. Feather M0 Express size is fixed. It's about 1k smaller than before this merge. Trinket M0 is about 500 bytes smaller, so looking good.
amber raven
@wary stratus I just merged the two-months-old PR in the asyncio library that we talked about yesterday and made it 1.0.0.
it was still 0.5.x for no good reason
wary stratus
kk, should I undraft my PR?
I was going to try and get my branch going with your changes
I didn't finish getting pico w building yesterday. at my parents' house now so I'm not sure I'll get back to that
amber raven
think you should rebase your PR just to make sure?
wary stratus
ah, ya. I can
amber raven
trying to think about what to do about the "assert is a statement, not an expression" thing in spresense. Maybe I should just change the upstream code, which uses assert() in a comma expression. Changing spresense is hard because the assert definition comes from nuttx submodule. But maybe that's just wrong... Maybe assert() should be expressionable in general
or maybe it's not a submodule, I'll look
amber raven
i was vaguely thinking of that too, good idea. I'll research a bit more and present something to him.
wary stratus
@amber raven the presence problem is the same assert I fixed for memorymap. It’s a terrible error message for micropython to say that the mp_obj_is_type that tests for something it shouldn’t
I switched it to mp_obj_is_exact_type()
wary stratus
build looks quite good! https://github.com/tannewt/circuitpython/actions/runs/6502012019/job/17660359047 pico w and a few esp32
wary stratus
pushed pico w build fix
amber raven
<@329766224093249548> the presence problem is the same assert I fixed for memory...
so if I pull your changes (or you push to my branch), it should be fixed? That would be great. Not sure what you mean about the terrible error message. thanks
wary stratus
so if I pull your changes (or you push to my branch), it should be fixed? That w...
I can fix it next time I have a moment. Wanted to check with you first
amber raven
I have nothing to push at the moment. Sounds like we could make a draft PR soon. I can look at the silabs build issues: there's some linking problem.
wary stratus
You may know the change that’s needed for the esp failures
The new dot clock type is using the old style
amber raven
The new dot clock type is using the old style
i can fix that quick
amber raven
The new dot clock type is using the old style
Fixed and pushed that to my branch. adafruit_qualia_s3_rgb666 now compiles
amber raven
The silabs build problem I had I think is a red herring. I forgot to intall git-lfs on my laptop.
amber raven
successfully built a silabs board locally
eternal grail
a minimal change could be to use the statement-expression extension of gcc: ( ({ assert(x) }), y) or so. (hi yall)
wary stratus
I fixed it by turning off the assertions
amber raven
i have nothing to push. I could test broadcom but will have to look up how to load firmware
wary stratus
don't bother
I bet its fine
(loading is copying a flash image to an sdcard)
I got silabs loading locally and seemed to load
but nothing on the serial connection...
wary stratus
I haven't used the board before so it should be empty
amber raven
does CIRCUITPY appear?
we could pass this back to the silabs folks if there's nothing immediately obvious
are there two USB ports on this board?
I don't have any to test, sorry
wary stratus
they don't have native usb
amber raven
hoo boy, i forgot
wary stratus
it shouldn't block this merge anyway
but I feel like I should do this little diligence
8.2.6 does work
amber raven
i have a meeting at 8pm until 9:15 or 9:30. But I made the PR, so you can push changes and you have to approve it anyway to merge.
wary stratus
e232b13 off s3 doesn't work
so this doesn't change it
do you have the changes I suggested? you should push those and then I'll approve
🚀
amber raven
i can do all those quick, sure
wary stratus
I'll make a separate PR for these silabs changes
totally ok to wait until tomorrow if you are out of time
amber raven
@wary stratus mpy-cross upload failed; i think the paths changed (you dealt with that elsewhere): https://github.com/adafruit/circuitpython/actions/runs/6540604996
amber raven
v1.20 / v1.21 merge
I renamed this thread so we can keep using it.
I submitted a PR to fix the mpy-cross uploads.
There might be some post v1.21.0 PR's in MicroPython we will want to pick up. There is a thread fix for ESP which is not relevant to us, but maybe there are others.
wary stratus
merged the mpy-cross PR
amber raven
want me to do a 9.0 alpha release?
Sure. Or I can do that a bit later if you have other things to do. There are a lot of PR's to summarize.
Also we can turn on building 9.x bundles.
wary stratus
ya, I've got some silabs changes to PR
probably good to get that in actually
I was thinking we'd do 9.x bundle once we have an alpha release
we can have the build tools use the alpha tag
wary stratus
done with the silabs stuff. want to pair on the 1.21 stuff or should I dust off the gcc update?
amber raven
done with the silabs stuff. want to pair on the 1.21 stuff or should I dust off ...
start on gcc; I am just getting started on 1.21 but I have to go out for a while
amber raven
making progress on the merge; usual tedious stuff. Also a few crazy things, like ridiculous renames:
renamed: ports/raspberrypi/common-hal/rtc/RTC.h -> extmod/modtime.h
and the auto-merge part deleted a large chunk of ports/raspberrypi
amber raven
finished all the merge conflicts; tomw I will go through the automerges and fix the crazy ones. Made a few notes to discuss with you later
wary stratus
👍 I'll have time tomorrow to chat. I'm in the middle of splitting displayio
wary stratus
I'm at my desk now if you want to chat
amber raven
I'm at my desk now if you want to chat
about to go to try to give blood; nothing urgent right now; I am going through the non-conflict merges and doing some fixups. Also annotating all the CPy changes in py/ and similar places more clearly. I will finish that first. May free up around 3:30.
many fewer "what the heck is this change" q's this time around
wary stratus
want to check in before the meeting?
amber raven
want to check in before the meeting?
Sorry - I missed that! I just finished the initial merge and pushed it; I didn't try to compile anything at all.
amber raven
@wary stratus I have like 10-15 minutes before we are going to a concert. Do you want to do a brief checkin or wait until tomorrow?
wary stratus
now works for me
amber raven
sure, in amelia
amber raven
I merged from main and pushed back to my branch. So now will switch to using gcc12
amber raven
dont' bother to try yet: I am fixing some basic compile problems
amber raven
Trinket M0 does 1+2 in REPL
amber raven
@wary stratus representative boards from every port builds except for silabs. Not sure what is going on with silabs; maybe it's gcc12 or something I am doing wrong.
on atmel-samd, I am getting this new warning:
lto-wrapper: warning: jobserver is not available: '--jobserver-auth=' is not present in 'MAKEFLAGS'
lto-wrapper: note: see the '-flto' option documentation for more information
but the build still finishes.
I haven't tried to understand this well yet, but it has to do with the jobserver preventing runaway stuff or something.
pushed all my changes to my branch
The // CIRCUITPY annotations are now // CIRCUITPY-CHANGE to make them easier to find.
wary stratus
@amber raven I can look at silabs tomorrow. I’m almost out of the displayio weeds
amber raven
I'm fixing some ERRNO issues...
wary stratus
my split displayio PR is failing the pdf build...
amber raven
silabs builds now
was not so weird after all; i broke the mpy-cross build but didn't realize that
wary stratus
👍
amber raven
i am starting to try tests locally
amber raven
getting ports/unix to build before getting the tests to build, but going for a walk now
wary stratus
@amber raven I’ll probably have time to poke the merge on my stream
So let me know what you are looking at already (and please push it all)
amber raven
<@329766224093249548> I’ll probably have time to poke the merge on my stream
when does your stream start?
amber raven
i am on a roll, but might stop by then. I'll push everything before the hour and let you know the status.
right now resolving differences between the various time modules: ours, extmod, ports/unix
wary stratus
k, no rush
amber raven
@wary stratus so right now the stumbling block is that our time is pretty different. it has monotonic_ns(), etc. that MPy doesn't have. Also, ports/unix/modtime.c is a bunch of addons to extmod/modtime.c.
wary stratus
🙄
amber raven
so this is boring, but have to figure out whether to change the tests or copy a bunch of impl's
not sure it's great stream material. Did you solve the displayio .root thing?
wary stratus
ya, I have a fix for root_group
I can look at the neopixel issue though
brb, restarting
amber raven
the rp2040 one? that soudns good
wary stratus
amber raven
I was thinking of https://github.com/adafruit/circuitpython/issues/8489
wary stratus
@amber raven I pushed some test fixes to your branch
5 still failing and 4 are asyncio
amber raven
<@329766224093249548> I pushed some test fixes to your branch
I don't see these pushes. I pulled and nothing came down
amber raven
got em now!
wary stratus
@amber raven I'm gonna take a look at the 1.21 tests. Have you done anything on them?
amber raven
no, go ahead, I am about to eat
spent too much time trying to do video editing
and then took a walk
amber raven
<@329766224093249548> I'm gonna take a look at the 1.21 tests. Have you done any...
https://github.com/adafruit/circuitpython/issues/8500 may be relevant
wary stratus
ya, seems related but the tests passed with the bug
wary stratus
@amber raven pushed fixes for the remaining test failures 🤞
maybe time to PR to run the CI
amber raven
<@329766224093249548> pushed fixes for the remaining test failures 🤞
Great! will do
amber raven
working on various CI build issues
wary stratus
kk, got a slow start after a btrfs-progs upgrade killed my booting....
amber raven
a bunch of tests are still failing but I think I fixed the other build problems
did you start looking at tests yet?
i had a long phone call before just now
wary stratus
just about back to 1.21 but getting hungry for lunch
amber raven
i will start running tests locally to see if I can duplicate the CI failures
wary stratus
ah, I didn't do the native version
looks similar to the failures I was seeing yesterday re:asyncio
and how we actually have __await__
amber raven
and how we actually have `__await__`
they must have added a bunch more tests because we already had that in the v1.20 merge tests
wary stratus
I had to add it back yesterday to fix the regular tests
I may have missed the native way of doing it
amber raven
i will look -- i added it back to a bunch of tests manually (they were not merge conflicts, but I audited the diffs, but I think at some point I got tired)
wary stratus
I added it back to py/compile.c
amber raven
so this is in the native emitter... I know very little about that. Maybe I'll do another diff first
wary stratus
amber raven
EMIT_ARG(load_method, MP_QSTR___await__, false);
EMIT_ARG(call_method, 0, 0, 0);
EMIT_ARG(load_const_tok, MP_TOKEN_KW_NONE);
EMIT_ARG(yield, MP_EMIT_YIELD_FROM);
in particular. But I wonder about the previous tests...
wary stratus
I'm gonna eat now and will check in with you when I'm back
amber raven
fixed, I think: one missing line in py/compile.c for native await() labels.
out for a walk
tests all run, so board builds now running
wary stratus
I'm looking into the mp_frozen_names issue
pushed fix to that and the docs build (hopefully)
amber raven
I'm back now
wary stratus
I just got through looking everything over
only thing I saw was enabling built in subpackage support
pushing that to your branch and also my own to run the CI
two things todo before the alpha are fixing the _transfer issue and adding the show impl that raises a friendly error
wary stratus
I'll looking at the docs build failure
fixing the DEBUG_PRINT build issue
will let the CI finish before I push fixes
amber raven
yay!