#circuitpython-dev
wraith crow
@slender iron Did you miss the PR I made to your pathlib fork or am I missing something about the proposed fixes?
slender iron
<@252717193496756235> Did you miss the PR I made to your pathlib fork or am I mi...
Should be in my branch now. I force pushed it
wraith crow
Looks like you got one of the commits, but I don't see the second one
slender iron
Looks like you got one of the commits, but I don't see the second one
I just did it by eye. Thanks for commenting on the PR
@lone axle the clicks are garbage collections
lone axle
The "loud chirp" or more "scratchy" sound? or both are the same root cause?
mortal kernel
<@849815081251766292> you around?
Right here.
slender iron
got time to chat?
mortal kernel
You bet.
lone axle
Nice! the audio sounds clean to my ear.Tthe letters disapearing and white static visual artifacts seem gone as well. The fly-on and sprite frame animations feel a bit slower than before. Was the color sweep causing issues?
slender iron
Nice! the audio sounds clean to my ear.Tthe letters disapearing and white static...
I think the color sweep was stuttering the audio a bit
The letters disappearing was caused by DMAing the audio out of PSRAM
working on a more robust fix for it now
blissful pollen
The letters disappearing was caused by DMAing the audio out of PSRAM
Just curious how did you stop the gc from interfering with the display/audio?
slender iron
I removed the use of enumerate
it generates tuples that then get unpacked immediately
but still need to be collected
blissful pollen
ah cool, thanks for the info. Good to know.
jaunty juniper
oh that's sneaky
slender iron
yup, I kinda want to look into how it is implemented
in case I can pass the iterator a "I'm going to throw away the tuple" flag so it can reuse one
blissful pollen
I guess the other option is to disable the GC during the animation (or similar high processing times) but not sure if that has any side effects
jaunty juniper
I wonder what the Micropython people think about that
slender iron
it is pretty large in this case because the bitmaps are loaded into it
the collect was taking 120ms
mortal kernel
@danh A big thank you for the flashing tips! Just got back around to it about 10 minutes ago and that's all it took to build and flash the uf2 bootloader. The J-Link .deb package installed without a hitch and JFlashLiteExe worked on my first try.
UF2 Bootloader v3.16.0-19-g4365018 SFHWRO
Model: SAME54 Xplained
Board-ID: SAME54P20A-Xplained-v0
onyx hinge
Miss y'all. Here are some snaky snakes.
stuck elbow
Miss y'all. Here are some snaky snakes.
Enjoy Barcelona
onyx hinge
ooh good guess! It's actually https://en.wikipedia.org/wiki/Tarot_Garden but the artist Niki de Saint Phalle was inspired by Gaudi.
mortal kernel
For anyone struggling with OpenOCD, I've found pyocd to be a great alternative.
hazy nebula
Hi @tulip sleet this is ross from mchp
tulip sleet
mchip curiosity
manic glacierBOT
manic glacierBOT
manic glacierBOT
tulip sleet
SST flash debugging
zinc walrus
@jaunty juniper, I know you've got a lot on your plate at the moment, so this is not at all pressing, but I've raised an issue on your CircuitPython_hsvtorgb_rainbow repo.
I'm busy too, so I can't troubleshoot in detail today, but I'll probably revisit it this weekend. Any suggestions you might have on how to debug would be welcome.
slender iron
@wraith crow have time for a voice/video chat about the display limit changes?
wraith crow
<@899160869777342464> have time for a voice/video chat about the display limit c...
Sure, probably voice only, my video setup is not in great shape
wraith crow
sure, pythonisita is fine
manic glacierBOT
nimble dawn
Any idea when As7343 will be released?
wraith crow
@slender iron supervisor_background_tick iterates the displays (via displayio_background) to perform refreshes do we still want that functionality?
slender iron
<@252717193496756235> supervisor_background_tick iterates the displays (via disp...
Ah, yes we will.
wraith crow
I can (at least temporarily) maintain a list of displays for refresh purposes. I suspect there also may be a way to have the displays register a refresh routine when it's allocated but if that's possible I'd need to do some research to figure it out.
slender iron
ya, I think you'll want a list somewhere that isn't the VM heap. if it is in a VM object, then it'll keep all of the displays around
wraith crow
okay, I'll work on that approach
slender iron
thank you!
manic glacierBOT
orchid basinBOT
I just installed Firefox on iOS, it’s never loaded the site, and see a similar issue:
gusty siren
For those familiar with CAN bus and canio, I made a (very limited) CircuitPython-friendly database "file" generator / parser that takes in a kcd file and spits out a CircuitPython file. I don't love working in CAN and don't want to really be on the hook for maintaining the thing, but is this something that Adafruit would like to see and inspect if it is worth supporting? If so, I can send the source code <150 lines to Adafruit with an MIT license/whatever or figure out how to use the cookie cutter thing and publish some open source package.
tulip sleet
@lone axle @short tendon https://forums.adafruit.com/viewtopic.php?t=217785 some typos in the secrets.py cleanup. Not sure who did the PR for this.
lone axle
<@382939733107408897> <@1152454697614069811> <https://forums.adafruit.com/viewto...
Thank you, that is updated now. It was in a code block directly on the page, not in the repo code.
slender iron
@tiny peak I want to see if we can make vblank longer to get our refresh rate lower
@tulip sleet I had a really good idea (hopefully) for improving gc collect times..... leaf allocations!
ones that promise to not include pointers
tulip sleet
<@329766224093249548> I had a really good idea (hopefully) for improving gc coll...
!
only pointer is the type field?
is there a bit available to indicate that?
tulip sleet
would framebuf fall into that category?
slender iron
I kinda think we could redo the vm gc to mimic tlsf it has metadata per-allocation instead of separate tables
the framebuf object wouldn't but the buffer itself would
basically the specific types would know they are leaf allocations
tulip sleet
If there are also objects with nothing but data except for the type field (e.g.g array.array), could also save time
slender iron
the actual bytearray for example
exactly
bitmaps too
I was hacking this by allocating the bitmap via port_malloc
adding leaf allocations to the vm though means it still gets marked and swept
tulip sleet
oh, I didn't realize there was a separate allocation of the holder object vs the data block
this is an excellent idea
slender iron
I think there often is
and we can change it if there is
here is array and bytearray: https://github.com/micropython/micropython/blob/db854270719bfa4dda003d38893c1b8f39bf58de/py/objarray.c#L110
just doing bitmaps yesterday dropped gc time from 200ms to 40ms
(for a bitmap heavy fruit jam program)
I think the psram is pretty slow
tulip sleet
slender iron
interesting that 1161 says all allocated chunks have a type word
(from 2015)
I could just mimic FTB
for now
ionic elk
could somebody remind me what the best call is for sending runtime string information to the REPL? I just need to print an integer error code
I know there are a few options but I've forgotten which is safest.
is it mp_printf?
tulip sleet
could somebody remind me what the best call is for sending runtime string inform...
mp_printf(&mp_plat_print, ...)
manic glacierBOT
turbid radish
I've been making some additions to Awesome CircuitPython
ionic elk
Is it possible to use the esp32-s3's built-in jtag while running circuitpython, over the USB or UART port? Using the devkitC, which has both. Circuitpython docs have a TODO section for the S3. Want to check before I go all in on using my jlink.
mortal kernel
Is it possible to use the esp32-s3's built-in jtag while running circuitpython, ...
I've had my best results connecting a J-Link to the debug connector on a Metro ESP32-S3 board. Getting it to work requires blowing a configuration fuse on the ESP32-S3 and externally strapping a GPIO pin. Here's an incomplete gist that explains it up through blowing the fuse and external strapping: https://gist.github.com/eightycc/b27d0da6d2864b758d59eeb083552344
ionic elk
idk if I can swing that haha
mortal kernel
There's also an issue with CP resetting the JTAG pins that needs to be dealt with. Let me know if you do want to try and I'll fill you in on more details.
ionic elk
I guess I'm seeing now that using the jtag pins in general requires an irreversable fuse burn
mortal kernel
Yup. I did try to use the CMSIS-DAP that gets exposed on the USB port, but could never get it to work.
ionic elk
Disabling Jtag from ever working over USB again. I'd rather just use USB, if it's an option
is there a debug flag that will disable CPY drive access and allow JTAG to work over the USB line?
mortal kernel
Actually the fuse I documented leaves it configurable by external strapping.
ionic elk
oh I see yes, STRAP_JTAG_SEL
That probably does make the most sense and should also work on my DevkitC
is the 33 in espfuse.py --chip esp32s33 -p /dev/ttyACM0 burn_efuse STRAP_JTAG_SEL a typo?
guess it must be, not in the help option
ionic elk
it's also espefuse.py, not espfuse.py
thanks for the link! I've just burned that and I'll see if it gets me up and running.
mortal kernel
Good eyes and lousy typing on my part🙏
You'll need to patch the ESP Makefile and add a flag to the build command to stop CP from resetting the JTAG lines. Let me look it up.
ionic elk
this is some pain, probably worth an addition to the readme
mortal kernel
It's a WIP. I've moved on to some other issues for now but do plan to do a more complete write-up as time allows.
ionic elk
oh no not yours sorry! I meant the general espressif readme for circuitpython
your readme is great and this info would be great to share more widely
thank you for sharing it
mortal kernel
You'll need to add these lines to ports/espressif/Makefile:
ifeq ($(ENABLE_JTAG), 1)
CFLAGS += -DENABLE_JTAG=1
endif
sorry I didn't mean to sound like a jerk judging your gist right after you offered help 😱
mortal kernel
No worries, you didn't.
When you build, add ENABLE_JTAG=1 to your make command line.
ionic elk
I wonder why that's not in there by default
mortal kernel
🤷♂️
ionic elk
any advice on the actual openocd command to run? The ones from my old notes are apparently obsolete
can't use openocd -f interface/jlink.cfg -f boards/esp32s2-saola-1.cfg etc
also, weird that the espefuse tool says 'STRAP_JTAG_SEL' (Set this bit to enable selection between usb_to_jtag and pad_to_jtag through strapping gpio10 when the docs clearly say gpio3
mortal kernel
Yes, I noticed the inconsistencies in the ESP doc re: the pin number, too. GPIO3 is the one that works!
For OpenOCD you want to pick up is the one provided by ESP-IDF. The easiest way to do that is to run the openocd command from inside the ESP-IDF venv.
openocd -f interface/jlink.cfg -f target/esp32s3.cfg -c "adapter speed 2000"
Ok, that's pretty much what I've been running so far
we'll see if the new build with ENABLE_JTAG puts it over
mortal kernel
Don't forget the strap!
BTW, DEBUG=1 is the way to go on the build. Otherwise, you don't get symbols and compiled code is too optimized to make sense of.
ionic elk
yep got that in already
trying to get some info out of a precompiled library so I've run into the limits of what mp_printf can get me
mortal kernel
Took me a ton of time to figure this all out, so I'm delighted to share it.
From my perspective, it's an indispensable tool to have in the kit.
ionic elk
hmmm, still no chip info
Open On-Chip Debugger v0.12.0-esp32-20240318 (2024-03-18-18:28)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 1000 kHz
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : J-Link V11 compiled Apr 27 2021 16:36:21
Info : Hardware version: 11.00
Info : VTarget = 0.000 V
Info : clock speed 1000 kHz
Error: JTAG scan chain interrogation failed: all ones
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: esp32s3.cpu0: IR capture error; saw 0x1f not 0x01
Warn : Bypassing JTAG setup events due to errors
Warn : [esp32s3.cpu0] Unexpected OCD_ID = ffffffff
Error: [esp32s3.cpu0] Examination failed
Warn : target esp32s3.cpu0 examination failed
Warn : [esp32s3.cpu1] Unexpected OCD_ID = ffffffff
Error: [esp32s3.cpu1] Examination failed
Warn : target esp32s3.cpu1 examination failed
Info : starting gdb server for esp32s3.cpu0 on 3333
Info : Listening on port 3333 for gdb connections
mortal kernel
Not to worry, I've seen that. First, check your strapping. The ESP32 stores it internally on a full POR, so just a reset won't change the JTAG selection.
If that's not the answer, double tap the board into the bootloader and give it another try.
I'm going AWK for an hour or so, I'll check back then.
ionic elk
I messed up some JTAG pins. I think I'm good now!
Thanks again for your help @mortal kernel !
I may try and get this stuff compiled in some public espressif port documentation
jaunty juniper
oh I'm getting a EADDRINUSE on bind() after an auto reload, I'll make a note and re-explore it later see if I can repro, some socket/status is not cleaned up at the end of code or there's some async in it, it seemed to resolve itself after a moment
crimson ferry
espressif or raspberrypi?
jaunty juniper
espressif, 9.2.7
I was actually using your tcp_server_CircuitPython_NATIVE.py to do some tests
crimson ferry
would be interesting to repro, thought that was fixed a year ago
jaunty juniper
would be interesting to repro, thought that was fixed a year ago
do you have an issue reference ?
on latest, this code cause this output (with a telnet to the IP, and a ctrl-D in the middle)
jaunty juniper
it then gets better but it took around 2 minutes of regular ctrl-D for listen() to not raise EADDRINUSE
crimson ferry
there's the usual web of issues and PRs, but this is the key one I think implementing setsockopt and SO_REUSEADDR https://github.com/adafruit/circuitpython/pull/8954
does it fix to add setsockopt to work across reloads? https://github.com/adafruit/Adafruit_CircuitPython_HTTPServer/blob/09e5431071d9e484726df87841a5b9bbe33b6d76/adafruit_httpserver/server.py#L271
jaunty juniper
does it fix to add setsockopt to work across reloads? <https://github.com/adafru...
yeah it does
crimson ferry
cool, I should update the server examples to be more robust
jaunty juniper
cool, I should update the server examples to be more robust
I'll make a note to open an issue, a reload (and therefore close/deinit of the listening socket) should still not keep an "addr in use" I don't think ?
crimson ferry
I'll make a note to open an issue, a reload (and therefore close/deinit of the l...
I seem to recall a bit of discussion about this, but can't find it. IIRC, without setsockopt, the server could re-bind after some timeout. I think the EADDRINUSE arises if the code exits abnormally and the socket is left in TIME_WAIT state... looks like that may be happening at a low enough level to survive reload? although there could be something else going on here
ionic elk
This is a dumb question I'm sure but how is Circuitpython handling the espressif configuration steps normally handled by menuconfig? I've gotta figure out how to port some settings from a kconfig file to a circuitpython-appropriate format
slender iron
CP has a number of sdkconfig files that get merged together depending on the mk settings
boards can also supply one
manic glacierBOT
manic glacierBOT
lone sandalBOT
mortal kernel
@tulip sleet The cascade mechanism implemented for nvm.toml does not allow overrides once a key has been set in one of the upper sheets. Maybe we want to change that?
tulip sleet
got it; let's see if @slender iron has any thoughts about that
mortal kernel
got it; let's see if <@252717193496756235> has any thoughts about that
@slender iron Just got pinched by this, again. I think it's worth making nvm.toml truly cascade similar to CSS.
slender iron
<@252717193496756235> Just got pinched by this, again. I think it's worth making...
I don't want to keep using nvm.toml in the long run anyway. With the move to Zephyr, we'll just do whatever they do
lone axle
How does git decide what is a file deletion and what is a change to a new file? I've realized in this PR: https://github.com/adafruit/Adafruit_Learning_System_Guides/pull/3014/files#diff-fd7f25eb86acad20ec568992e5a8b5c883efb0d83da8d8ec3662374ca9767885 the file PyPortal/PyPortal_NeoPixel_Color_Picker/secrets.py was deleted as far as I can tell. But git / github seems to report that this file was actually moved to create a settings.toml file in a different subdirectory.
The result of this behavior, which is why it caught my attention, is that the deletion of of that secrets.py file is not really listed in the list of changes on the PR. Searching "NeoPixel" in the 'filter changed files' reports no results either, where-as a search for "User_Interface" does return a result for a different directory which also had files from inside of it deleted (which git counted as a real deltion instead of a move).
jaunty juniper
How does git decide what is a file deletion and what is a change to a new file? ...
I guess it does some heuristics to decide that a deletion and addition was a move, but my understanding is that there is actually no notion of "move" or "rename" inside git, it's just how it's presented when reading the changes. Some tools can extract more information by analyzing the history, but for example renaming a file or directory breaks that file's history in git/github
tulip sleet
we've had this problem a lot when merging MicroPython. It tries too hard to mark things as move with edits
lone axle
Interesting. It's innocuous in this case, but slightly disconcerting how well it "hides" the deletion from the reviewers front end in github.
tulip sleet
this is about the other way round, but there may be some clues about how to prevent it. One way is to split it across commits
tulip sleet
<@&356864093652516868> weekly meeting is in less than hour. Add your notes here: https://docs.google.com/document/d/1q0T9l-qybjcu0ID3zVOcn2vIgiXJgV_teC48ew8GNrI/edit?tab=t.0
manic glacierBOT
lone axle
It does work with circup (when it's build suceeds at least)
lone axle
A library could be in two bundles...
I think typically this would not happen. Is there an instance of one library in two different bundles that you know of?
I think there is nothing technologically preventing it. But I also think it wouldn't break anything if it were to happen off the top of my head at least.
turbid radish
Thanks all!
lone axle
Thanks for hosting Scott, have a great week everyone!
jaunty juniper
if a library is in 2 bundles, circup would use the first it finds in order of bundles I assume, unless the last one overwrites the first one ?
mortal kernel
Have a fine week, everyone!
thorny jay
Could the repetitive part be pre-recorded?
Do we know how much time each section takes?
The juicy parts are "hug report", "status update" and "in the weed".
I could live without the number of pull request or updated libraries.
lone axle
That is true, It could be ambiguous which bundle it would pull from or go by whatever order happens to be hardcoded in circup. Theoretically the actual library files should both the same though as they should both be built from the same git release automatically in each bundle.
thorny jay
So circup will not be confused if a library move from one bundle to another?
lone axle
not confused in a way that makes a functional difference as far as I understand it.
slender iron
Could the repetitive part be pre-recorded?
Do we know how much time each section...
timecodes in the doc will tell you how long each section is. repetitive stuff at the beginning is three minutes or so. The tradeoff would be editing time to add it
slender iron
Here is the notes document for next Monday’s CircuitPython Weekly Meeting. It is at the normal time of 11am Pacific / 2pm US Eastern here on Discord. Add your hug reports and status updates to the document before the meeting. If you are unable to attend but would still like to contribute, feel free to add your notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868> https://docs.google.com/document/d/1kpHthhyD6M3mV8NQJG-J0e0iFCaIiq1rwGCaE1qnFzM/edit?usp=sharing
mortal kernel
I'm back. Here's where I left off: I can change PLL_SYS output to any multiple of 48 MHz. Initially, I'm setting it to 144 MHz. At a small performance hit, that frees up PLL_USB to drive the HSTX clock domain exclusively.
With this config, CLK_SYS, CLK_PERI, CLK_USB, and CLK_ADC are all on PLL_SYS or CLK_SYS.
slender iron
Any idea if we could adjust VBLANK instead of changing the pixel clock?
mortal kernel
I'm guessing that wouldn't work because it's likely that the pixel clock is the issue with the capture card, not the vertical frequency.
slender iron
it is weird to me that the pixel clock matters given that it is within the range of what the capture card can do (higher reses have faster clocks)
mortal kernel
Without knowing how it's logic work, I couldn't say. My model for how I think it works imagines it setting up timing for each mode.
The DMT spec gives just a +/- 0.5% on the pixel clock.
slender iron
I think it'd be fine to do that. I'm curious about vblank though too
I don't know a ton about it though
@tiny peak related pixel clock discussion ☝️
@mortal kernel where did you leave off on the psram optimization?
is it mostly blocked on the stage2 stuff?
mortal kernel
No. That's an issue for overclocking and flash. On the PSRAM, I've got an outstanding pull pico-mac that implements PSRAM up to 133 MHz. I've tested it successfully on a Metro RP2350.
mortal kernel
75 MHz. The clock divider for qspi only supports integers, so we're stuck with that at 150 MHz SYS_CLK.
slender iron
same with the hstx divider
mortal kernel
Yeah. Thing is that for ones that do support fractions, like PIO, you get jitter with non-integer dividers.
Here are some notes I made on pixel clock:
// Note: By default, the HSTX clock domain AUXSRC mux selects CLK_SYS as its
// source. Also by default, the HSTX clock divider is set to 1, so the HSTX
// clock domain runs at 150 MHz. The selected HSTX_CTRL CLKDIV is 5, so the
// effective pixel clock is 30 MHz. At 640x480, this results in a slightly
// out of spec vertical frequency of 75 Hz. To be in spec for 640x480 at 75 Hz
// the pixel clock would need to be 31.5 MHz +/- 0.5%.
// For 720x400 at 85 Hz, the required pixel clock is 35.5 MHz +/- 0.5%. At
// 30 MHz, this one is far off spec.
// See "VESA Display Monitor Timing Standard", Version 1.0, Rev. 13, 2013.
// PLL and clock divider settings(*) to achieve selected resolutions/pixel clocks:
// 640x480@60: 25.175 5 125.875 126.0 1 126 1512 6 2
// 640x480@75: 31.5 5 157.5 157.5 1 105 1260 4 2
// 720x400@85: 35.5 4 142.0 142.0 2 213 1278 3 3
// ^---PD2
// ^-----PD1
// ^---------VCO
// ^-------------FBDIV
// ^---------------REFDIV
// ^---------------------Actual HSTX_CLK
// ^----------------------------Desired HSTX_CLK
// ^-------------------------------CLKDIV
// ^--------------------------------------Desired Pixel Clock
// (*)PLL settings determined by vocalc.py
I wonder if it'd be worth running the core at 133 so that we can 1:1 flash and psram
mortal kernel
I've pre-calculated the parms for the PLL, and in these three cases I can hit the desired clock within the spec.
We'd lose the ability to free the USB_PLL at 133 unless we're willing to risk overclocking part of the clock tree.
slender iron
right. that'd help everyone though
the lower voltage psram part is rated up to 144mhz
could also just try it 🙂
mortal kernel
As long as we made it a setting we could change in the .toml, I'm up for risking it.
slender iron
This says the USB_PLL can only generate up to 48mhz
the HSTX would need more than that
mortal kernel
That's why I thought driving a higher frequency out of FOUTPOSTDIV might be a problem. Weird thing is, it seems to work.
slender iron
There is an earlier comment suggesting to use USB_PLL for HSTX
I don't see any other documentation about it
mortal kernel
Sometimes the doc isn't correct. You can see from my table that I'm counting on an HSTX clock of up to 157.5 MHz off the USB_PLL.
That's so the HSTX logic has enough headroom to keep DVI fed.
slender iron
doesn't the hstx clock need to be pixel clock * 10?
8 bits in, 10 tmds bits out?
or pixel clock * 5
since it is ddr
mortal kernel
We're running it x5 now.
slender iron
so the USB_PLL would need to be less than 150
for pixel clocks between 25 and 27mhz
mortal kernel
I think we could risk going over 150 by a smidge. In practice, the RP2 parts have a crazy amount of overclocking headroom.
slender iron
what refresh rate and resolution are you targeting?
mortal kernel
640x480@60 and 75, 720x400@85.
I was thinking about reading the monitor's EDID pages and going completely adaptive. But, that's just me thinking.
slender iron
I added a little of that to CP
mortal kernel
I saw that. It's what got me thinking along those lines.
slender iron
I wouldn't do anything more than 60 fps
unless it is 720x400 which bottoms out at 70
I referenced https://en.wikipedia.org/wiki/Extended_Display_Identification_Data#Structure,_version_1.4 bytes 35-37 for the most supported modes
mortal kernel
Yeah, we're kind of stuck down in that range. Here's a better reference: https://glenwing.github.io/docs/VESA-DMT-1.13.pdf
I'll keep poking at it on my own time. I like the idea of going to 133 MHz to see if we can get better overall performance with fully-clocked PSRAM.
slender iron
maybe 640x350 and 640x400 are worth trying since they are both 31.500
you can try the 133mhz psram thing on a couple hours of adafruit time
the hstx clock thing doesn't seem like a huge blocker to me now
I only know of the one capture card that is unhappy
mortal kernel
Sounds good. I'll give 133 a try. I did have a thought re: gc collect scan of psram allocations. Maybe use uncached accesses?
slender iron
ya, could but that'd be more rp2350 specific
the selective collect has the nice property of benefitting everything that enables it
mortal kernel
Good point, there I go thinking the RP2 parts are the whole world...
slender iron
🙂 all of the esp parts with psram will benefit too
anything with a bunch of ram you want big buffers in
mortal kernel
OK, I think we've beaten this one up enough. Appreciate the chat.
slender iron
thanks!
wanton dirge
Hi folks! Coming back around to this, I was just finishing up my BUSIO support getting ready for the PR, but then I noticed the new Zephyr port (which is super exciting!). Running it on my trusty AD-APARD32690-SL board, it works great, but doesn't appear to instantiate any BUSIO support.
Modules
>>> help("modules")
__main__ collections micropython struct
array digitalio os sys
board gc random time
builtins microcontroller storage zephyr_serial
Plus any modules on the filesystem
We have quite a few boards well supported in Zephyr, so thought I'd try it. I also noticed some strangeness with the board module and the boot configuration. Sorry for the lateness, feel free to get to this tomorrow. I have a couple questions about the Zephyr port:
- Is it possible to get BUSIO support for our boards with some simple tweaks?
- I'm assuming the port pulls info from the board's DT files? Why do I see so many strange repeated entries below? Is there any way of setting up specific configuration for CircuitPython vs. our Zephyr configuration?
- Anything I should do to resolve the boot log message at the end here? Looks like it fails to retrieve a UID.
Board (apard32690//m4)
>>> import board
>>> help(board)
object <module 'board'> is of type module
__name__ -- board
board_id -- apard32690
D5 -- board.D5
D4 -- board.D4
LED3 -- board.LED3
LED2 -- board.LED3
LED2 -- board.LED2
LED1 -- board.LED2
D10 -- board.D10
D6 -- board.D6
D7 -- board.D7
D13 -- board.D13
S2 -- board.S2
D12 -- board.D12
D11 -- board.D11
D9 -- board.D9
D8 -- board.D8
LED1 -- board.LED1
LED -- board.LED1
LED0 -- board.LED1
D2 -- board.D2
D0 -- board.D0
D3 -- board.D3
D1 -- board.D1
D14 -- board.D14
D15 -- board.D15
A0 -- board.A0
A1 -- board.A1
A2 -- board.A2
A3 -- board.A3
A4 -- board.A4
A5 -- board.A5
Boot Log
*** Booting Zephyr OS build 8f4225038bcf ***
Init heap at 0x20007b48 - 0x20020000 with size 99512
Init heap at 0x200d0000 - 0x200e0000 with size 65536
Init heap at 0x20020000 - 0x20040000 with size 131072
Init heap at 0x20040000 - 0x20060000 with size 131072
Init heap at 0x20060000 - 0x20080000 with size 131072
Init heap at 0x20080000 - 0x200a0000 with size 131072
Init heap at 0x200a0000 - 0x200c0000 with size 131072
Init heap at 0x200c0000 - 0x200d0000 with size 65536
hello from flash init
flash 0x10025bb8 flash_controller@40029400
16 pages
page 0: 0 4000
page 15: 3c000 4000
16 uniform pages
setup flash
erase page size 16384
write row size 512
blocks per page 32
Adafruit CircuitPython 10.0.0-alpha.2-13-g4a16ed1856 on 2025-04-14; AD-APARD32690-SL with max32690
Board ID:apard32690
UID retrieval failed: -88
UID shorter 0 than defined length 32
UID:0000000000000000000000000000000000000000000000000000000000000000
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
Code done running.
Press any key to enter the REPL. Use CTRL-D to reload.
Adafruit CircuitPython 10.0.0-alpha.2-13-g4a16ed1856 on 2025-04-14; AD-APARD32690-SL with max32690
>>>
@slender iron In the IDF mods there's a patch to rtc_io_ll.h that checks for other pins mux'ed to the RTC before turning off the mux enable. The code has changed substantially w/5.4, rtcio_ll_iomux_func_sel() no longer touches iomux_clk_gate_en. I suspect the underlying issue is fixed, but I'd like to test it to be sure. Can you point me to the original issue?
tulip sleet
<@252717193496756235> In the IDF mods there's a patch to `rtc_io_ll.h` that chec...
if you find the call that uses the routine in rtc_io_ll.h on github, you can use "Blame" to find out when that code was last changed, which will lead you to the PR.
blame is also available locally, but I always use it on the website
mortal kernel
I did that, and the changes are too large to pin down what they were addressing. I was hoping to get back to the CP issue that prompted the patch.
ESP did a huge amount of refactoring, presumably to simplify code for their growing family of parts.
tulip sleet
I did that, and the changes are too large to pin down what they were addressing....
which rtc_io_ll.h are you looking at?
mortal kernel
components/hal/esp32s[2,3]/include/hal/rtc_io_ll.h
tulip sleet
I am not sure there is an issue for this; it may have just been found when we were upgrading esp-idf
see https://github.com/adafruit/circuitpython/pull/7486
I checked our changes to release/v4.4 against this. No conflicts. @slender iron's adafruit/esp-idf#10 fix has already been incorporated upstream. There are some additional fixes in that file for other things.
"already incorporated upstream" says fixed upstream, I guess. Are you looking at this branch https://github.com/adafruit/esp-idf/tree/circuitpython-v5.3.2 or the default branch https://github.com/adafruit/esp-idf/tree/release/v4.4-circuitpython (which shouldn't be the default branch anymore)
mortal kernel
I'm looking at circuitpython-v5.3.2.
I'll do a bit more sleuthing in the ESP-IDF code to see if there's still a similar bug in the refactored code. I haven't been able to find anything in the old CP issues that looks like it, so it likely was found by inspection.
tulip sleet
I don't know why I said it was incorporated upstream. It is not in esp-idf master in that particular file
but maybe it was fixed elsewhere
mortal kernel
Ah, a true mystery. Maybe @slender iron can shed some light...
mortal kernel
You are amazing. That looks like it's it.
tulip sleet
that fix is way after my comment, though
slender iron
@mortal kernel to tickle that bug in circuitpython I think you'd need to do pin wake on multiple
or it could be ULP related
slender iron
Hi folks! Coming back around to this, I was just finishing up my BUSIO support g...
My intention is to use the port-specific busio implementation when we add support for it. We'll do zephyr_* implementations for the fixed busses.
The build does autogenerate the friendly pin names based on the Zephyr DTS. The simplest way to change those is by modifying the DTS.
we haven't updated to zephyr 4.1 yet. it definitely feels like the future of CP but we won't likely get back to it until next year when I'm back full time after paternity leave
wanton dirge
Awesome. Thanks for the info. I really like both platforms a lot, and it could save a lot of work porting things over. But I'll probably be poking at CP one way or another for a while anyway. One more question -- is there any preference for whether or not the BUSIO implementation uses interrupts or not? I currently have UART using interrupts and other things blocking. I want to make that consistent before I PR but wanted to see if there's a preference.
slender iron
Awesome. Thanks for the info. I really like both platforms a lot, and it could s...
Yup, UART buffers incoming data. Others block
wanton dirge
Oh...sweet!
Thanks for the input as always 🙂 I will work on my branch some more and hopefully PR soon.
slender iron
great! I'm out in three weeks but dan and eightycc will be around
copper crescent
There doesn't currently seem to be a way to get the rp2pio txfifo level, pio_sm_get_rx_fifo_level is exposed via rp2pio.in_waiting but from what I can see there isn't an "rp2pio.out_waiting" or similar for pio_sm_get_tx_fifo_level?
lone axle
Anyone have an idea of what could cause the core to raise a RuntimeError when it is trying to raise a ValueError? I'm noticing that when it attempts to raise the exception from here: https://github.com/adafruit/circuitpython/blob/main/shared-bindings/bitmaptools/__init__.c#L369
Instead it results in:
Traceback (most recent call last):
File "code.py", line 101, in <module>
RuntimeError:
In a local build I added print statements before and after the value error raise. I see the before print and then this generic RuntimeError with no details about what the problem is.
stuck elbow
Anyone have an idea of what could cause the core to raise a RuntimeError when it...
I know that circuitpython will "degrade" exceptions when it fails to allocate the memeory for them -- for example, will remove the message, etc. but I never saw it falling back to a superclass of the exception
do you know what is the memory situation at that point?
lone axle
gc.mem_free() reports free: 193504 just before the call to alphablend() which ends up raising the RuntimeError
lone axle
It seems to be somehow tied to what string is given to the error. If I change it to lowercase first letter
mp_raise_ValueError(MP_ERROR_TEXT("bitmap size and bits per value must match"));
It works and does raise the ValueError with the message.
However it can also output the capital 'B' as long as there aren't "too many" other characters with it seemingly. This works:
mp_raise_ValueError(MP_ERROR_TEXT("Bitma"));
and this does not:
mp_raise_ValueError(MP_ERROR_TEXT("Bitmap"));
perhaps there is an error during the error string interning, maybe a hash collision?
lone axle
Does that process differ by port? I've just noticed that on ESP32-S3 with the same code the ValueError does get raised successfully with it's full original message.
I'm not familiar with string interning. If something is hashing the strings per word though that could definitely be where the culprit is. If I use
mp_raise_ValueError(MP_ERROR_TEXT("Bitmaq size and bits per value must match"));
it works and raises the ValueError and full message. So it seems like the specific string "Bitmap" is the trigger for it in this instance.
stuck elbow
I think the hash values could be different on different ports
but it could also be some error in how the strings are handled
tulip sleet
@lone axle it does seem like a translation string processing error. Could you open an issue with "at this exact commit (or this release) this doesn't work, but if I change this one thing, then it does", then we can look at the generated translation data to see what's up.
slender iron
@tulip sleet got time to talk memory allocation/gc changes?
tulip sleet
ya sure
slender iron
@tulip sleet we could merge the outer heap and the VM heap in CP 11 and use tlsf for it all
then we don't need VM heap splitting or growing at all
and keep object alive outside the vm
slender iron
it'd make the primary display stuff simpler
tulip sleet
still want separate SRAM vs PSRAM pools?
mortal kernel
vs explicit free
@slender iron I like this approach, too.
copper crescent
There doesn't currently seem to be a way to get the rp2pio txfifo level, pio_sm_...
Would a patch to add "rp2pio.out_waiting" (or whatever you'd prefer it to be called) exposing pio_sm_get_tx_fifo_level be welcomed as it would be helpful to know how much can be written to the FIFO before it's going to block?
tulip sleet
Would a patch to add "rp2pio.out_waiting" (or whatever you'd prefer it to be cal...
if you think there's a use case, sure
are there things in micropython or pico-sdk support that we don't have?
slender iron
tlsf on used blocks is 4 bytes for allocation size. It uses the lower two bits for metadata. We could likely pack into that too
copper crescent
if you think there's a use case, sure
It's from pico-sdk, I couldn't find anything in micropython to get the txfifo level either.
mortal kernel
@slender iron ESP-IDF does more strict checking for valid use of CONFIG_ flags. It now checks that all used flags are defined somewhere in the chain of Kconfig files for the target processor. This applies not just to flags that are defined, but also to flags that are commented out. Although CP does not use Kconfig, these checks are still performed. CP config files all pass, except when building for the ESP32H2. CP configures this for a 48 MHz flash spi clock, but according to Kconfig this isn't a valid setting. The valid setting are 16, 32, and 64. Is this a typo in CP or a setting we want?
mortal kernel
@slender iron I'm wondering whether we need to coordinate an ESP-IDF update to v5.4.1 with TinyUSB?
slender iron
<@252717193496756235> ESP-IDF does more strict checking for valid use of CONFIG_...
I think this is usually done on a per-board basis. We should have different kconfig fragments for the different speeds.
slender iron
<@252717193496756235> I'm wondering whether we need to coordinate an ESP-IDF upd...
We don't usually need to. I think we do all USB separate from IDF. The only thing that may be overlapping is the init code
mortal kernel
I think this is usually done on a per-board basis. We should have different kcon...
It is. I've added support for 48 MHz to the ESP-IDF fragment for ESP32H2 that meshes with CP's Kconfig fragments. All good with that. Now I'm on to a problem that's breaking CI which is fixed on the ESP-IDF 5.4 branch but is not included in the v5.4.1 tag.
slender iron
Bundle updates are getting killed due to rate limiting: https://github.com/adafruit/circuitpython-org/actions/runs/14512434132/job/40713999002
you can go past the tag if you want. I've just had mixed luck with doing so
mortal kernel
I'm wondering if I want to cherry pick the patch on our v5.4.1 branch or temporarily work around the problem in CMake?
slender iron
I haven't looked at the error. Either is fine with me. You can also rebase on a commit from their 5.4 branch instead of the tagged one
tulip sleet
Bundle updates are getting killed due to rate limiting: https://github.com/adafr...
@lone axle maybe time to work on this again
lone axle
fwiw I think that action is for the contributing pages not necessarily the bundles page. Currently https://circuitpython.org/libraries shows adafruit bundle 20250415 and community bundle 20240417, both of which are the most update to date bundle released for each one as far as I can tell.
slender iron
@tulip sleet any object to me carving out double conversion support for samd21s
tulip sleet
not sure what you mean
slender iron
looks like samd21 boards have f2d and d2f that take at least 280 bytes
I want to disable those
tulip sleet
if they are not used, sure
slender iron
I think it'd impact printf and arrays with type 'd'
tulip sleet
i have an issue opened with some big savings for samd21, let me find it
mortal kernel
Any advice for shrinking dram0 usage on ESP32S2? I'm overflowing by 376 bytes. ☹️
Also out of flash on ESP32S2 for some national languages: ru, pt.BR, ja, fr, de_DE.
I've also got some missing symbols to chase down, so I'm not blocked.
tulip sleet
Any advice for shrinking dram0 usage on ESP32S2? I'm overflowing by 376 bytes. ☹...
which size flash on ESP32-S2? We are about to double the image size on 4MB boards: https://learn.adafruit.com/adafruit-feather-m4-express-atsamd51/update-the-uf2-bootloader. I was originally doing this only on S3 boards, but this PR is for all 4MB Espressif.
Also if OPTIMIZATION_FLAGS = -Os is not set (-O2 instead) that will help a lot
@slender iron think we should drop the OTA partition for all 4MB boards, including ESP32 and ESP32-S2? We didn't ask Thach to do that, just the S3 boards.
mortal kernel
Flash overflow is on ESP32-S2. It's the espressif_esp32c6_devkitm_1_n4, 4MB flash.
tulip sleet
that's what my PR does, but TinyUF2 0.30.0 was only certain S3 boards.
tulip sleet
Flash overflow is on ESP32-S2. It's the espressif_esp32c6_devkitm_1_n4, 4MB flas...
i'm confused, S2 or C6?
mortal kernel
My bad, C6.
gcc is a bit overzealous with inlining.
i doubt the performance gain is worth it in many cases
tulip sleet
-Os should help, but I thought we were already doing that for the RISCV chips
C<n> chips are all RISC
tulip sleet
you can do make V=commands BOARD=... to see the command lines
maybe turn off PYCAMERA for now
or ULAB, just temporarily, until https://github.com/adafruit/circuitpython/pull/10265 is merged
the RAM issue may require some sdkconfig tuning
mortal kernel
That should get me by on the flash, I'll need to look at the elf.map to see what to try trimming for the dram0 overflow. Thanks!
slender iron
That should get me by on the flash, I'll need to look at the elf.map to see what...
ESP has a lot of code loaded into iram that may squeeze DRAM.
I don't see any code size mentions here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/migration-guides/release-5.x/5.4/index.html
mortal kernel
Is the USB IP on the ESP32-S2 an Analog MAX3421? The reason I'm asking is that CIRCUITPY_MAX3421E is getting set by default in espressif/mpconfigport.mk. That's causing a missing __atomic_test_and_set in tinyusb's hcd_mac3421.c:hcd_edpt_xfer().
mortal kernel
Is the USB IP on the ESP32-S2 an Analog MAX3421? The reason I'm asking is that C...
Yup. I see that:
.dram0_reserved_for_iram
0x3ffb8000 0x14d64
0x3ffccd64 . = ((ORIGIN (dram0_0_seg) + _iram_end) - _iram_start)
*fill* 0x3ffb8000 0x14d64
.dram0.data 0x3ffccd70 0x3d30
0x3ffccd70 _data_start = ABSOLUTE (.)
it's support for a separate chip.
mortal kernel
I see. Now to find atomic functions in ESP-IDF...
tulip sleet
I see. Now to find atomic functions in ESP-IDF...
probably fine, but make sure lib/tinyusb's submodule commit is up to date
mortal kernel
probably fine, but make sure lib/tinyusb's submodule commit is up to date
TinyUSB looks fine, last commit is 4/3/2025. The wanted atomic is present in esp-idf's newlib component, but it looks like something has changed in the macro defines that get from__atomic_test_and_set in the <stdatomic.h> to SYNC_LOCK_TEST_AND_SET in newlib.
Is updating ESP-IDF always so complicated?
tulip sleet
No, but there are always API changes. Might be worth looking at the esp-idf/examples, if any of them use the atomic op
slender iron
fyi I have an audio fix for rp2 in the works too
switching the dma setup so readaddr always gets reset
mortal kernel
No, but there are always API changes. Might be worth looking at the `esp-idf/exa...
The missing atomic is a known ESP-IDF regression that appeared with v5.4: https://github.com/espressif/esp-idf/issues/15167. According to the issue, the fix was supposed to go into v5.4.1, but it did not make it. I'll patch our branch.
tulip sleet
The missing atomic is a known ESP-IDF regression that appeared with v5.4: https:...
maybe worth using the tip of the release/v5.4 branch instead of the 5.4.1 tag?
mortal kernel
That feels risky to me. I've resolved all of the build issues except the dram0 overflow on ESP32-S2 parts.
tulip sleet
You could advance that as they move toward 5.4.2
got it; testing with the tip could show you if the DRAM overflow is inherent or not, but it probably is
mortal kernel
It's grown very slightly, but just enough to overflow. I can work around it by eliminating PYCAMERA as that uses quite a bit. I don't understand (yet) why iram reserves such a large chunk of dram.
.dram0_reserved_for_iram
0x3ffb8000 0x14d88
0x3ffccd88 . = ((ORIGIN (dram0_0_seg) + _iram_end) - _iram_start)
*fill* 0x3ffb8000 0x14d88
.dram0.data 0x3ffccd90 0x3d30
0x3ffccd90 _data_start = ABSOLUTE (.)
For comparison, from an ESP32-S2 build with v5.3.2:
.dram0_reserved_for_iram
0x3ffb8000 0x14c04
0x3ffccc04 . = ((ORIGIN (dram0_0_seg) + _iram_end) - _iram_start)
*fill* 0x3ffb8000 0x14c04
.dram0.data 0x3ffccc10 0x3c74
0x3ffccc10 _data_start = ABSOLUTE (.)
@tulip sleet My understanding is that all of the text (compiled code) in iram is copied into dram, presumably to work around the access alignment limitations of iram. If I'm understanding this correctly, it should be possible to shrink the text in iram, and thereby claw back some dram, by making more nuanced adjustments to gcc's optimizations controls, specifically inlining.
tulip sleet
<@329766224093249548> My understanding is that all of the text (compiled code) i...
I was not aware of this, I would have though it would be a linker segment thing, not needing to be copied, but it sounds like that's not true. @slender iron may know more
mortal kernel
I'm not clear re: when the text gets copied. It could be linker or crt0.
I've scanned through dram0.data, and I don't see anything obvious that can be tuned out by adjusting ESP-IDF confg parameters.
tulip sleet
you read this already, I assume: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/memory-types.html
i'm out for a couple of hours momentarily
mortal kernel
@tulip sleet No luck adjusting compile flags. I did learn that the optimization flag settings in CP's Makefile are overridden for ESP-IDF modules. For ESP-IDF, optimization flags are set indirectly by the CONFIG_COMPILER_OPTIMIZATION_* parameter. CP already configures this as CONFIG_COMPILER_OPTIMIZATION_SIZE=y.
(this is effectively -Os)
tardy current
Hi I'm using the Adafruit FT232H breakout (https://www.adafruit.com/product/2264) and I want to send output to one of the GPIO pins. I'm running a simple example with Adafruit_Blinka and it seems to connect fine, but the board doesn't send any signal outside of when it turns on. Another weird thing is no power is coming out of either the 3V or 5V pin - could be a defective board? Here's the script I'm running. Any help appreciated. ```import time
import board
import digitalio
PIN = board.D5
led = digitalio.DigitalInOut(PIN)
led.direction = digitalio.Direction.OUTPUT
while True:
led.value = True
time.sleep(0.5)
led.value = False
time.sleep(0.5)
print("turned led on/off")```
austere acorn
Hi I'm using the Adafruit FT232H breakout (https://www.adafruit.com/product/2264...
This channel is for CircuitPython core development discussions, please use https://discord.com/channels/327254708534116352/537365702651150357 or https://discord.com/channels/327254708534116352/330406777009209346
mortal kernel
@tulip sleet I've removed ESP_CAMERA from the ESP32-S2, and it no longer overflows DRAM. This may have to be a long-term solution unless there is something else that we'd prefer to remove.
The last issue with ESP-IDF 5.4.1 completely passing CI is flash overflow on the ESP32-C6 for certain translations. ESP_CAMERA and ESP_ULP are already removed. Any suggestions?
For jp, we need to claw back 3312 bytes.
tulip sleet
The last issue with ESP-IDF 5.4.1 completely passing CI is flash overflow on the...
The flash overflow will be solved by https://github.com/adafruit/circuitpython/pull/10265 if these are 4MB boards.
so turn, off, say ULAB, for now, mark it as temporary, and we'll turn it back
after that merges
mortal kernel
What do think about removing ESP_CAMERA from ESP32-S2?
tulip sleet
well, I think there may be some ESP32-S2 boards that have cameras on board (not sure about that), but other things could be turned off for those. I can try to mess with it later. It would be nice to turn it back on. I turned it back on for many boards in that PR, without RAM overflow
mortal kernel
You won't hit the DRAM overflow until IDF 5.4.1 is integrated. I wonder about the utility of a camera on the ESP32-S2 because it has so little DRAM.
tulip sleet
You won't hit the DRAM overflow until IDF 5.4.1 is integrated. I wonder about th...
all the boards with "cam* in their name are either ESP32-S3 or plain ESP32
have you done some wifi or BLE testing on the 5.4.1 builds?
if not I would make that part of my review
after it's merged, I'll rebase my partition-change PR and see how the build goes
mortal kernel
Good point. I was going to bring up testing. I've done nothing more than smoke testing on ESP32-S3 at this point.
tulip sleet
The "Internet test" that's in all the guides is plenty
tulip sleet
For BLE I install adafruit_ble and use https://github.com/adafruit/Adafruit_CircuitPython_BLE/blob/main/examples/ble_uart_echo_test.py, which acts as a BLE "UART" peripheral. Then I try connecting to it with the UART page from Adafruit's Bluefruit Connect app on my phone. If you can type at it, hit enter, and it echoes back what you typed, it's working.
full plume
Is there a reasonably accurate and maintained roadmap for MicroPython/CircuitPython development? There are a few specific features I'd really like to see which don't appear be actively pursued (based searching issues, etc. on GitHub), such as
- weakref
- compatible semantics for using a weakref.ref instance, i.e. someRef() returns the original object or None and merely holding an instance does not introduce a circular reference blocking gc
- extra steps for creating the initial weakref.ref instance might be OK , and potentially avoid overhead for all instances
- extract valid keyword argument names from a function/method/closure ( i.e. inspect.signature(...) )
Is this the right place for questions like these?
tulip sleet
- weakref
- compatible semantics for _using_ a **weakref.ref** instance, i...
This would be a micropython core thing. Note that MicroPython/CircuitPython don't use reference counting for gc: they use mark and sweep
turbid radish
The annual CircuitPython 202x posts each January solicits input from the community. That and how the industry is shaping up through the year. GitHub issues and PRs help.
Yes, aas CircuitPython is a fork of MicroPython, core functionality is typically added to MP and then rolled into CP
tulip sleet
so weakref is not as important if the circular references are unconnected to another known object
the circularity will get gc'd
full plume
This would be a micropython core thing. Note that MicroPython/CircuitPython don'...
I did notice that while digging through things, hence the possibility of an extra step to create the weakref instance. Assuming MICROPY_ENABLE_FINALISER is enabled in the build, it might be feasible without having to mess with the base MicroPython object model (mp_obj_t ).
full plume
the circularity will get gc'd
hmm, then maybe all I need is a wrapper that emulates weakref semantics and implements them by conditionally importing weakref or replacing it with a pass-through which lets the gc deal with it if import weakref throws?
tulip sleet
This was briefly discussed in:
https://github.com/micropython/micropython/issues/646
https://github.com/micropython/micropython/issues/1878
tulip sleet
hmm, then maybe all I need is a wrapper that emulates weakref semantics and impl...
yes, you could just implement a no-op weakref
full plume
My main issue with weakref is that I'm using some techniques which definitely require weakref to break circular references on a reference counting/CPython implementation, and I'd prefer to have the same codebase working on standard CPython and CircuitPython.
So a no-op weakref sounds like a winner for now
tulip sleet
Found this: https://pypi.org/project/micropython-weakref/
full plume
This was briefly discussed in:
<https://github.com/micropython/micropython/issue...
Yeah, I had seen those (I did dig a little before posting...), but they were a bit old
tulip sleet
full plume
Actually I think I'd prefer an explicit replacement (i.e. a module that supports weakref semantics and uses the weakref implementation if available but is not named weakref) for my purposes, as it would add near zero overhead but leaves some hint in the source that you're using a weakref_-ish_ thing which may or may not behave similar to a CPython weakref (specifically with del / scoping and finalization timing ). And I can do that in pure python without any changes to the CircuitPython firmware. (Already did something similar for typing)
tulip sleet
You may have also already seen https://docs.micropython.org/en/latest/genrst/index.html#micropython-differences-from-cpython
we try not to change the MicroPython interpreter core very much. We do have a few differences involving traceback and maybe certain aspects of property support: I forget much about the latter.
jaunty juniper
note that for typing we just try/except the import, since CP ignores type hints anyway, so there's no need for dummies
full plume
The other issue - extracting valid keyword argument names - is potentially more significant. Use case is allowing callbacks to be registered (often through decorators), and wanting to provide optional keyword arg values when invoking them only if the callback expects them (or takes **kwargs).
tulip sleet
there is no accessible saved data structure to get that info
that info is compiled away
full plume
that info is compiled away
yeah, given the dearth useful bits on a function instance, and the inability to set an attribute on a function / closure, I figured it might not be low hanging fruit
But there's logic that throws an exception if you pass in an unexpected keyword parameter, so it seems plausible that you ought to at least be able to ask "will this Callable accept these keyword parameter names" (although it might require some compiled extension code to get there).
tulip sleet
hmm, maybe. The name checking is part of the arg parsing logic. Sometimes there is a table, but sometimes not. For example, in busio_i2c_make_new() in shared-bindings/busio/I2C.c, there's this table:
static const mp_arg_t allowed_args[] = {
{ MP_QSTR_scl, MP_ARG_REQUIRED | MP_ARG_OBJ },
{ MP_QSTR_sda, MP_ARG_REQUIRED | MP_ARG_OBJ },
{ MP_QSTR_frequency, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 100000} },
{ MP_QSTR_timeout, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 255} },
};
but there's no way to get at those tables, and sometimes the arg validation is more ad hoc.
and that's just for native modules
so hmm, yes, maybe there is some dict somewhere
for python code
full plume
note that for typing we just try/except the import, since CP ignores type hints ...
Hadn't even considered this for builtin/native calls - that could get sticky. For my purposes, it would always be Python code (i.e. a method/function/lambda/closure)
(oops, ignore the reply bit...)
TL;DR I want to make it easy to create callbacks to do do the basics without having to write (or understand) def myCallback( **optionalStuffIDontNeed ): ... but that can use optional extras if they're asked for
tulip sleet
I see no thoughts about inspect.signature for MPy anywhere
full plume
I'm considering creating a callback "wrapper" that traps "unexpected keyword argument" and tracks which arguments it will accept. Kludgy and adds overhead, but plausible
tulip sleet
can you do this checking at compile time with mypy or something?
full plume
I see no thoughts about `inspect.signature` for MPy anywhere
Yeah, the inspect module in general doesn't get much love. I was a bit surprised about that.
full plume
can you do this checking at compile time with mypy or something?
Technically yes, effectively no. I don't need it for my code - I'm building a framework with a specific goal of making things easier for non-programmers, so this has to work all the way back to code.py. The majority of my "target audience" wouldn't have a clue what mpy modules are....
(and I realize this is also a goal for CircuitPython - I'm just adding some extra layers)
tulip sleet
I was thinking of the "mypy" checker as a pre-processor here
full plume
I was thinking of the "mypy" checker as a pre-processor here
The concept of a preprocessor is great - I'm a huge fan of DSLs, code generation, etc. and I've done some absolutely bizarre stuped Python self-modifying code hacks.
mortal kernel
@tulip sleet Dang, something's broken in TLS:
code.py output:
ESP32-S2 WebClient Test
My MAC address: ['0x34', '0x85', '0x18', '0x9b', '0x54', '0x4c']
Available WiFi networks:
ATTiPdMEuI RSSI: -48 Channel: 6
Bevs wifi RSSI: -89 Channel: 1
TMOBILE-E0B2 RSSI: -86 Channel: 5
Connecting to ATTiPdMEuI
Connected to ATTiPdMEuI
My IP address: 192.168.1.213
Pinging 'google.com' took: 24.0 ms
Fetching text from http://wifitest.adafruit.com/testwifi/index.html
----------------------------------------
This is a test of Adafruit WiFi!
If you can read this, its working :)
----------------------------------------
Fetching json from https://www.adafruit.com/api/quotes.php
Traceback (most recent call last):
File "code.py", line 55, in <module>
File "adafruit_requests.py", line 711, in get
File "adafruit_requests.py", line 639, in request
File "adafruit_connection_manager.py", line 347, in get_socket
File "adafruit_connection_manager.py", line 249, in _get_connected_socket
OSError: -30336
Code done running.
but part of the KISS requirement is that I don't want to have any specific IDE required - it needs to "just work" whether you're using https://code.circuitpython.org/, Thonny, VSCode, ....
tulip sleet
<@329766224093249548> Dang, something's broken in TLS:
```
code.py output:
ESP32...
-30336 is look-up-able; doesn't look like we got that before, though (searches don't turn up any matches0
mortal kernel
It's this MBEDTLS error: "0x7680 SSL - No CA Chain is set, but required to operate"
tulip sleet
but part of the KISS requirement is that I don't want to have any specific IDE r...
You could ask about whether inspect.signature() is possible in, say, https://github.com/orgs/micropython/discussions. But a Signature object is a kind of complex thing, and that's a lot of extra code in the firmware, with little demand right now.
tulip sleet
It's this MBEDTLS error: "0x7680 SSL - No CA Chain is set, but required to op...
aha, ok, that should be defaulted to include the standard root cert bundle. There's a config option for that which should be turned on already. I wonder if they changed the API
full plume
You could ask about whether `inspect.signature()` is possible in, say, <https://...
yeah, a full blown Signature would probably be a lot of work and is way more than what I need
tulip sleet
It's this MBEDTLS error: "0x7680 SSL - No CA Chain is set, but required to op...
or it might be a build thing that's not running: it re-creates the bundle as needed.
by fetching from mozilla or something
i just spent too much time on this a few weeks ago
discovering that the bundle was there already
wait, wait, but we build our own bundle, we don't use the default, so maybe that's not working
mortal kernel
aha, ok, that should be defaulted to include the standard root cert bundle. Ther...
My good luck that you looked at it.
tulip sleet
sorry, it's taking a bit to come back to me (I was using arduino-esp32)
we just stuff all the .pem's together and put it somewhere in the firmware. lib/certificates is the submodule with roots.pem
mortal kernel
I can re-test with a non-5.4.1 CP 10, if that helps.
tulip sleet
I was thinking of switching to the ESP-IDF default. It's in a more compact format
well, I think I did test alpha.2 and later stuff. I just used the wifi test program with the partition-changing PR, and it worked fine there
so this seems like a 5.4.x thing
mortal kernel
Probably is 5.4.1. At least I've got a thread to start pulling...
tulip sleet
Probably is 5.4.1. At least I've got a thread to start pulling...
esp-idf-config/sdkconfig.defaults
101:CONFIG_MBEDTLS_CUSTOM_CERTIFICATE_BUNDLE_PATH="../../lib/certificates/data/roots.pem"
maybe this is an mbedtls problem
full plume
I'm getting a "Hard fault: memory access or instruction error." and switched to safe mode after attempting some optimizations based on class RGB(namedtuple('RGBBase', ['r', 'g','b'])):...
I suspect it's related to also having get/set properties defined for r/g/b (retrofitting an existing class, slots doesn't seem to help)
tulip sleet
It's fine to open an issue, but if you can also reproduce it in MicroPython, it would be good to file an issue there.
mortal kernel
maybe this is an mbedtls problem
The MBEDTLS version in ESP-IDF is a an Espressif fork at 3.6.2. Unchanged from v5.3.2.
There are large changes in crypto with v5.4.1.
full plume
It's fine to open an issue, but if you can also reproduce it in MicroPython, it ...
I'll see if I can come up with a simple way to reproduce the problem
(I have it working, just need to figure out a repeatable way to crash it)
tulip sleet
There are large changes in crypto with v5.4.1.
I can take a look at this if you have other fish to fry
mortal kernel
This is the big fish in my pan right now.
Which reminds me, it's time for late lunch.
If you do want to have a peek, ESP-IDF 5.4.1 is here: https://github.com/eightycc/esp-idf/tree/circuitpython-v5.4.1, the corresponding CP pull is: https://github.com/adafruit/circuitpython/pull/10267. Any help is accepted with gratitude.
mortal kernel
@tulip sleet Found in the v5.4 release notes breaking changes section:
"Security / ESP Cert Bundle: Due to changes in the certificate bundle format to reduce RAM usage, the use of pre-generated certificate bundles using the older format would now be incompatible with the newer application binaries. You now need to either use the default ESP-IDF build system method for attaching the certificate bundle while building applications or update your pre-generated certificate bundle binaries by generating them using the latest gen_crt_bundle.py script."
https://github.com/espressif/esp-idf/releases/tag/v5.4
tulip sleet
<@329766224093249548> Found in the v5.4 release notes breaking changes section: ...
thanks, that is extremely helpful. I didn't start on this yet
i may not get to this until monday or even tuesday
mortal kernel
No problem, I'm done for the week, as well.
tulip sleet
but it should be straightforward. I've been the one to deal with cert issues for a while now
mortal kernel
Feel free to clue me in on what's involved.
tulip sleet
either I will munch the roots.pem into their format, or else just use their default list (which may or may not be too big for us). https://github.com/adafruit/certificates/issues is quite pruned
but partly is was pruned because the .pem's are large
I was working on nina-fw with Thach (https://github.com/adafruit/nina-fw/pull/70). That's the firmware used on AirLift co-processors. for certs, in particular: https://github.com/adafruit/nina-fw/pull/70/commits/74339a4809d21f4f39ebac911ecffbbc7d668d4c.
the arduino-esp32 Board Support Package for Arduino IDE uses the defaults cert bundle
mortal kernel
@tulip sleet I read "ESP32-S2 Technical Reference Manual" section 3 "System and Memory" to gain a better understanding of how this part addresses its various memory address spaces. My understanding of how iram/dram works was incorrect. Both are mapped onto the same internal SRAM. The contents are not copied, rather they are shadowed from the individual iram/dram address spaces onto the same SRAM section.
Reconsidering iram use led me here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/performance/ram-usage.html. In the section "Optimizing IRAM Usage" there are several recommendations for reducing iram usage. Implementing the first recommendation, moving FreeRTOS functions to flash, resolved the dram overflow without sacrificing the ESP_CAMERA component.
tulip sleet
<@329766224093249548> I read "ESP32-S2 Technical Reference Manual" section 3 "Sy...
Great! I have a recollection of Scott doing this on some builds or other, but I might be wrong.
mortal kernel
Scott gave me a hint in this direction, but I lacked the context to fully understand. Now I do.
jaunty juniper
it's kind of funny to see on the display of a C6 board "Autoreload is on. Simply save files over USB". Like sure you could over USB serial, but I wonder if we should rephrase it altogether or remove the USB mention on non-USB boards. Or if it doesn't matter at all.
jaunty juniper
Sparkfun has a USB VID, how are boards added ? Have they added them themselves ? Are PID requested or documented somewhere ?
lapis sapphire
Adafruit_CircuitPython_asyncio repository description in GitHub has an uppercase I after C, so CIrcuitPython instead of CircuitPython. I could issue, but should I?
lone sandalBOT
tulip sleet
Adafruit_CircuitPython_asyncio repository description in GitHub has an uppercase...
I fixed that -- it was just a settings change. Thanks!
manic glacierBOT
crimson ferry
@danh for PR Wifi power management #10271, the latest commit seems to have removed the artifacts for espressif and raspberrypi. Is there somewhere else to find them?
oh n/m, I figured how to get at them
n/m the n/m, the artifacts aren't there, only the build output
tulip sleet
n/m the n/m, the artifacts aren't there, only the build output
Sorry! I pushed another commit to fix the zephyr builds, and it only rebuilt the zephyr builds. The older artifacts are here: https://github.com/adafruit/circuitpython/actions/runs/14554565710. Scroll way down past the errors to find the artifacts list. (I wish there was a better link to the artifacts.)
crimson ferry
I'm getting safemodes ( memory access or instruction error) consistently on Pico W and ESP32-S2 QT Py, should I put more info here, or in the PR?
(haven't tried custom power modes yet)
tulip sleet
I'm getting safemodes ( memory access or instruction error) consistently on Pico...
I am not seeing this on PIco W (only one tried yet), but I am using a local build. I'll try the artifact.
OK, reproduced with artifact
@crimson ferry I'm seeing these crashes on the tip of main, so they are totally unrelated to #10271. The tip of main is broken in some way.
mortal kernel
@tulip sleet I'm working through my collection of Espressif boards testing the ESP-IDF upgrade. I've run into a problem with an older Feather Huzzah ESP32. I've followed the instructions in the Primary Guide, erasing and flashing CP with the WebSerial ESPTool. On reset, this message repeats on the usbserial:
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
invalid header: 0x61646100
invalid header: 0x61646100
invalid header: 0x61646100
invalid header: 0x61646100
invalid header: 0x61646100
invalid header: 0x61646100
invalid header: 0x61646100
ets Jul 29 2019 12:21:46
I’m out but will try this, this afternoon.
mortal kernel
@tulip sleet NP, To be clear, this is happening with ESP-IDF 5.4.1. Earlier builds work correctly. I'm seeing a similar problem with an ESP32-C6 feather.
tulip sleet
Does esptool.py also fail?
mortal kernel
Does esptool.py also fail?
No! Flashed the ESP32-C6 with esptool.py and it's fine with ESP-IDF 5.4.1.
full plume
<@329766224093249548> I'm working through my collection of Espressif boards test...
If you can give me a link/version for grabbing the ESP-IDF upgrade and a branch/tag for CircuitPython, I'd be willing to try getting it set up and tested on some ESP32s in my collection too. I haven't done much directly yet with ESP-IDF, but I've got plenty of experience with complex build systems. I was already planning to try migrating some of my C++ stuff to ESP-IDF (currently using PlatformIO) before I started playing with CircuitPython.
blissful pollen
Does the code ever really use int64_t and if I had to is that "slow"? This is for a synth fix where I think we are overflowing. I could scale the 32bit value signal down first but then if it is a quiet sound we risk loosing the original all-together.
The alternative (and maybe "right" way) is to go through to all the places we do calculation and ensure we scale into an int16_t value and don't let a in32_t value live too long.
mortal kernel
If you can give me a link/version for grabbing the ESP-IDF upgrade and a branch/...
There are a couple of show stoppers we're working through, so the updates aren't merged yet. If you'd like to give it a try in its current state, the build artifacts are in this PR: https://github.com/adafruit/circuitpython/pull/10267. TLS is not working due to a certificate format change, and the older ESP32 parts that don't support the uf2 bootloader may also have problems. If you do find other regressions, please document them in https://github.com/adafruit/circuitpython/issues/10191.
@tulip sleet The Feather Hussah ESP-32 fails differently when flashed using esptool.py. Here's what I see on the serial:
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0040,len:400
load:0x40078000,len:14048
load:0x40080400,len:4
load:0x40080404,len:3108
entry 0x40080554
After that, nothing.
slender iron
Sparkfun has a USB VID, how are boards added ? Have they added them themselves ?...
I think they give them out through the forum. I've also gotten some by emailing them
slender iron
<@329766224093249548> I read "ESP32-S2 Technical Reference Manual" section 3 "Sy...
I wouldn't move freertos stuff to flash because I imagine we may use them when flash is disabled to write it.
mortal kernel
I wouldn't move freertos stuff to flash because I imagine we may use them when f...
With FreeRTOS functions in flash, I'm able to write successfully to CIRCUITPY, so perhaps that's not an issue.
full plume
uf2 bootloader support cuts out quite a few, but I still have a variety of S2 and S3 boards (primarily Wemos/Lolin, Lilygo, and Freenove). I'm not directly using TLS, most of my experimentation has been with uf2/CIRCUITPY and USB console/REPL.
tulip sleet
@mortal kernel just to be clear, you are trying to flash the ESP32 to 0x0000, and you're flashing firmware.bin?
mortal kernel
<@849815081251766292> just to be clear, you are trying to flash the ESP32 to 0x0...
Yes.
full plume
There are a couple of show stoppers we're working through, so the updates aren't...
Didn't take me too long to figure out pip install setuptools==71.0 cryptography=42, although it certainly was longer than it would have been if I had read https://github.com/adafruit/circuitpython/issues/10191 first...
(FWIW I had to do that with ESP-IDF 5.3 - want a known good build working before trying ESP-IDF 5.4.1 and https://github.com/adafruit/circuitpython/pull/10267)
lone axle
<@&356864093652516868> The weekly meeting will occur here on discord at the usual time today 2pm US eastern / 11am US pacific. That is in about 1 hour and 15 minutes from now. You can add your notes and hug reports to the shared document: https://docs.google.com/document/d/1kpHthhyD6M3mV8NQJG-J0e0iFCaIiq1rwGCaE1qnFzM/edit?usp=sharing We look forward to catching up with everyone.
mortal kernel
@tulip sleetThe ESP32 is double faulting early in startup. I'll need to hook up JTAG. Fortunately, the pins are exposed on the Feather Huzzah ESP32.
full plume
I have the current main branch / ESP-IDF 5.3 built, loaded, and running successfully in a Lolin/Wemos S2 Mini. I'll setup another build with ESP-IDF 5.4.1 and https://github.com/adafruit/circuitpython/pull/10267 later today. I have a Lolin/Wemos S3 Mini, Lilygo T7-S3 / T-Energy S3 / T-Display S3, FreeNove ESP32-S3 WROOM, SeeedStudio Xiao ESP32Sense, and a couple of generic S3 dev-kit / micro S3 / ESP32-Cam boards available. Not all of those have their own port yet, but should be easy enough to get working. My current focus is the Wemos D1 Mini compatible boards (Lolin S2/S3 Mini, Lilygo T7-S3), but I could test a few others. Any preferences on priority?
tulip sleet
I have the current main branch / ESP-IDF 5.3 built, loaded, and running successf...
eightycc is off for a bit starting today. The most interesting testing would be with boards with/without PSRAM, 4MB vs 8MB (or more) flash, etc. Similar boards with similar modules would be expected to work -- we don't test against all boards by any means.
I have some debugging and fixing to do on v5.4.1 for TLS and plain ESP32 builds
other testing to do would be I2C, especially on S3, wifi testing, and BLE testing
turbid radish
Please subscribe
slender iron
@blissful pollen you are unmuted
lone axle
full plume
eightycc is off for a bit starting today. The most interesting testing would be ...
I don't think I have any S2/S3 boards without PSRAM and I haven't done much with BLE (and that only with Arduino libraries in C++). But I intend to test the LilyGo T7-S3 configured to use the full 16MB flash (and 8MB PSRAM), and I could test the FreeNove S3 with 8MB flash and 8MB PSRAM. Unfortunately I don't think I have any Adafruit ESP32-S2/S3 boards, but I do have a couple dozen Stemma QT modules (plus a handful of Sparkfun Qwiic and a bunch of general I2C breakout boards) so I can give I2C a workout. Any particular I2C / Stemma QT bits you'd specifically like to see tested on an S3?
thorny jay
Thanks
lone axle
Thanks everyone, have a great week 👋
blissful pollen
<@228322543507865601> you are unmuted
Sorry guys switching from my car to my AirPods somehow decided to unmute me
tulip sleet
I don't think I have any S2/S3 boards without PSRAM and I haven't done much with...
Thanks! If you have BNO055/BNO085 you could try those on ESP32-S3. They don't currently work on Espressif: see https://learn.adafruit.com/i2c-addresses/troublesome-chips. I am not expecting this to be fixed but it would be interesting if it were.
lone axle
I've just realized that my recording stopped after Dan's status update :(. I was trying to take a timestamp for Scott and accidentally had focus on OBS instead of the doc. I didn't realize what had focus at the time, or that the space bar in the timestamp caused OBS to stop recording.
Unforunately this means Scott's status report and the wrap up will not be included in the video / audio recordings this week unless anyone happens to have been running a backup recording.
full plume
Thanks! If you have BNO055/BNO085 you could try those on ESP32-S3. They don't cu...
Don't have the Stemma QT BNO055, but I did find a https://www.adafruit.com/product/2472 in my parts box. If it's been fixed in the new ESP-IDF release, should the existing adafruit_bno055 module work with it just fine?
tulip sleet
yes, but I doubt it will 🙂
slender iron
I've just realized that my recording stopped after Dan's status update :(. I was...
I didn't do a backup recording. I could read it again if you really want to. I think it is fine without anyway
full plume
true, but it should be easy to try once I've got the EDP-IDF 5.4.1 CP firmware working on an S3
lone axle
I'm adding an extra note at the top of the YT description mentioning to see the notes doc for full details of the last few minutes.
Here is the notes document for next Monday’s CircuitPython Weekly Meeting. It is at the normal time of 11am Pacific / 2pm US Eastern here on Discord. Add your hug reports and status updates to the document before the meeting. If you are unable to attend but would still like to contribute, feel free to add your notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868> https://docs.google.com/document/d/1kT1f2pPZgEUXu0J9QgaDrGnV59SNDoVBhh-sEESoSoQ/edit?usp=sharing
tulip sleet
@slender iron you're using ARM gcc 14 for a while now? Shall we update main to that?
tiny peak
@slender iron does the current Fruit Jam board build have support for the IR rx, ESP32-C6 airlift, etc. yet?
tulip sleet
<@252717193496756235> does the current Fruit Jam board build have support for th...
not for C6 yet, but main thing there is finishing the NINA-FW software for it. It will use the ESP32SPI library, it will not be import wifi
we have existence proof of C6 working in a breadboarded setup
tiny peak
ok. I think I'll have my hands on a board pretty soon. Could potentially test firmware builds out if there are things you'd like some extra eyes on. Planning to write some Playground guides.
tulip sleet
I should be getting one soon as well, so far I've been mocking it up
tiny peak
do you know if the schematics for the latest board rev are up someplace publicly accessible yet?
slender iron
<@252717193496756235> you're using ARM gcc 14 for a while now? Shall we update `...
I think so. I don't actually remember when it updated
slender iron
do you know if the schematics for the latest board rev are up someplace publicly...
I don't think so but I have access
so I can answer any questions you have
tiny peak
IR rx pin?
tiny peak
or, generally, are the pins from the schematic already defined in the circuitpython port? In that case, I can just dig around until I find the right defines
tulip sleet
or, generally, are the pins from the schematic already defined in the circuitpyt...
you can always use microcontroller.pin.GPIO29, etc.
or maybe those are GPnn or IOnn I don't remember
they are is identical to the board pins
tiny peak
I'm mainly wondering about changes that may have happened across board revisions and whether those are reflected in the port defines yet
slender iron
they are. I'm working off rev b
slender iron
@tulip sleet samd shrink pr is ready for a final look https://github.com/adafruit/circuitpython/pull/10269
full plume
I'm working on a framework with support for a variety of Wemos D1 Mini-compatible expansion boards. All the C++/Arduino docs and examples are primarily based on using the original ESP8266 Wemos D1 Mini controllers, so I've "normalized" pin assignments such that my driver code will "just work" using the original pin names. This is done using a mapping from the actual pins for the ESP32-* D1 Mini "compatible" to the original pins. For example :
full plume
I'm using board.board_id as the key to find the right mapping based on the actual running controller.
tulip sleet
I'm working on a framework with support for a variety of Wemos D1 Mini-compatibl...
instead of using strings, could you use the microcontroller.pin names
the board and microcontroller.pin objects are the same internally
full plume
One thing that seemed odd is that I've had my S2 Minis show up as both "lolin_s2_mini" and "WemosS2Mini" in board.board_id
tulip sleet
One thing that seemed odd is that I've had my S2 Minis show up as both "lolin_s2...
about 7 months ago, we changed the hostname from LS2Mini to lolin_s2_mini. I cannot find WemosS2Mini, though
full plume
instead of using strings, could you use the `microcontroller.pin` names
I don't think that will work - the idea is to provide a single source set of configurations that works with any of the mapped boards. So if, for example, the original D1 "A0" pin mapped to microcontroller.GPIO40 on a given controller, but another controller doesn't have microcontroller.GPIO40 available, I can't have the main mappings in a single source file.
tulip sleet
ok, I didn't know whether they were all S3 or whatever
full plume
I used strings to get around this. Ultimately, "GPIO21" will end up being filled in as getattr( microcontroller.pin, "GPIO21")
full plume
ok, I didn't know whether they were all S3 or whatever
This is only for D1 Mini (pin) compatible controllers, but there are quite a few ESP32 flavors out there compatible with the standardized physical pin locations (5V/3v3/GND, plus usually default SDA/SCL) but widely varied internal pin names / assignments.
So I think I saw mention that CircuitPython 10 is going to drop the OTA partition? Which would presumably free up more space on the CIRCUITPY drive (for controllers with uf2 support)?
jaunty juniper
One thing that seemed odd is that I've had my S2 Minis show up as both "lolin_s2...
board_id is CIRCUITPY_BOARD_ID, which is name of the board directory, can you check where and in what version you get WemosS2Mini ?
jaunty juniper
So I think I saw mention that CircuitPython 10 is going to drop the OTA partitio...
for all 4MB ESP32 boards (making space for BLE support on S3 and C3 for example)
stuck elbow
for i2c usually you can just use board.I2C() and it will use the default pins
tulip sleet
CIRCUITPY is going to remain the same size. The "ota0" partition (the "app" partition) for firmware will double the size
jaunty juniper
right, it frees up space for Circuitpython not for the drive
full plume
board_id is CIRCUITPY_BOARD_ID, which is name of the board directory, can you ch...
I'm fairly certain I've always used https://circuitpython.org/board/lolin_s2_mini/ for my S2 Mini boards, and all my CircuitPython installs have been in the last two or three months.
stuck elbow
I'm fairly certain I've always used https://circuitpython.org/board/lolin_s2_min...
you can check boot_out.txt for the exact version
slender iron
@tulip sleet did you mean to approve the samd pr?
tulip sleet
I'm doing a couple of smoke tests on a trinket
full plume
CIRCUITPY is going to remain the same size. The "ota0" partition (the "app" part...
Oh well, I was hoping for a little more CIRCUITPY space. OTOH, sounds like there have been some audio fixes that might actually be better - my main problem is that I need to use audiomixer to play multiple samples at a time (and to have volume control) . My attempts to do so with audiomp3 sounded terrible. So I switched to WAV files - which sounded much better, but they chew up way too much of the CIRCUITPY drive.
tulip sleet
Oh well, I was hoping for a little more CIRCUITPY space. OTOH, sounds like th...
you could flash a different partition .bin and achieve that. For audio, it's going to be a tight squeeze in any case, without an SD card. Maybe the extra 1.4MB would help in certain applications
@slender iron interestingly a trinket m0 with gcc14 isntead of gcc13 is 100 bytes larger
full plume
you could flash a different partition .bin and achieve that. For audio, it's goi...
Yeah, that was part of the motivation for getting set up to build CircuitPython locally. But I'd really prefer to avoid any dependencies on a custom build out of my framework framework bits. I definitely intend to add some native code (ideally dynamic native modules, but not sure if those are ready yet...) but they'd mostly be optimizations, so the "user" code works (abliet potentially slower) whether or not they're using a custom firmware build. Although as far as the CIRCUITPY drive size goes, moving up to an 8 or 16MB flash controller should help far more than dropping the OTA partition if someone wants more audio.
slender iron
<@252717193496756235> interestingly a trinket m0 with gcc14 isntead of gcc13 is ...
I'm not surprised. I doubt they test that per-release
jaunty juniper
on 4MB boards, the no-ota-no-uf2 partition has a ~2MB drive, with a ~2MB ota0, while the no-ota version has ~1MB drive with a 2.8MB ota0 (since it just fuses ota0 and ota1 rather than change the drive size, which avoids corrupting the drive)
tulip sleet
I'm not surprised. I doubt they test that per-release
but metro m4 is 130 bytes smaller on gcc14
🤷♂️
after the next alpha release I'll bump the version, I think, maybe too much churn for now
full plume
Gotta go find that S3 mini, firmware build just finished. I'm used to some rather large builds - 16 cores, 96GB ram, and fast SSDs usually make them manageable - but this one had me wanting more coffee...
full plume
lolin_s2_mini build (w/ ESP-IDF 4.51, ...) finished and runs just fine.
The lolin_s3_mini build does not - after copying in firmware.uf2 it stops booting. Can't even git back to the UF2 bootloader.
I can still go back and reinstall 9.2.7 from the beginning and it works fine (CIRCUITPY drive appears normally, USB REPL works)
But if I restart in UF2 bootloader mode (which is press-release RST/RESET followed quickly by press-release 0/BOOTLOADER - instructions are wrong on CP download page for Lolin S2/S3 mini) and then copy in the new fimware.uf2 (exact same process I used with lolin_s2_mini) it drops/reconnects USB but CIRCUITPY drive does not appear nor can Thonny find the USB REPL/console.
But this time I can still get back to the UF2 bootloader on the S3 mini.
So since the S2 Mini build worked fine, I think it's time to make clean BOARD=lolin_s3_mini and try again later...
I don't see anything obvious in the files changed (https://github.com/adafruit/circuitpython/pull/10267/files) related to either of those boards which would explain it. Although CONFIG_FREERTOS_PLACE_FUNCTIONS_INTO_FLASH=y gets added to sdkconfig-esp32s2.defaults, but sdkconfig-esp32s3.defaults remains unchanged (and does not set CONFIG_FREERTOS_PLACE_FUNCTIONS_INTO_FLASH). Guess I'll try that if the clean build still fails.
slender iron
@tiny peak did you try the extended back porch? my tv seems fine with a longer back porch
tiny peak
<@1220648290941669377> did you try the extended back porch? my tv seems fine wit...
No. wasn't aware that was a thing that could be adjusted. How did you make the change? Doesn't appear to be an argument for picodvi.Framebuffer().
tiny peak
I've just been planning to use 320x240 or 640x480 with my capture card and call it good. If that gives me any trouble, I might end up getting an Extron video scaler off ebay.
I looked to see if I could find any XR1 Lite capture cards for sale. Seems that model must have been discontinued or something. Didn't find much except for a few ebay listings at higher prices than what you mentioned.
tulip sleet
@mortal kernel It's fishy that on ESP32 I got the double fault with DEBUG=1 on the tip of main, but without that it ran normally. So the DEBUG=1 crash may be different than the failure of #10267. Or it might be something like a storage overflow somewhere, which is provoked on #10267 and on the increased size of things due to DEBUG=1. I think it might be worth doing your tracing on both DEBUG and non-DEBUG builds to see if they are related.
I do see a hang or crash on non-DEBUG on #10267, as did you.
mortal kernel
<@849815081251766292> It's fishy that on ESP32 I got the double fault with `DEBU...
It's fishy. I'm thinking the same thing you are, something's moving in memory that's tickling the bug. Based on your testing, I also think that it's only related to IDF 5.4.1 due to where things fall in memory. Hopefully JTAG will show something more definitive.
tulip sleet
I will stop working on this and move on the root certs change
I did look to see if there were any interesting changes in sdkconfig settings by doing a menuconfig to get the defaults, but I didn't see anything immediately interesting. The number of differences between the default sdkconfig and the one our builds generate is small. But it could be just one thing I missed. @slender iron what was your procedure for finding necessary sdkconfig changes/additions? I tried running some of your scripts in tools/ but didn't get that far
candid crag
hello, I have been trying to compile the latest circuitpython firmware for the rp pico with my main.py code within a uf2. I have been following this guide and have been able to make uf2s but none have my code baked in. any help would be greatly appreciated. flashing 200 boards drag and drop is tedious https://learn.adafruit.com/building-circuitpython/build-circuitpython
tulip sleet
hello, I have been trying to compile the latest circuitpython firmware for the r...
you can set up a single Pi Pico with what you want, and then use picotool to produce a .uf2 that is a dump of everything in the flash, including what's in CIRCUITPY
https://github.com/raspberrypi/picotool#save
https://embeddedcomputing.com/technology/open-source/create-custom-uf2-file-for-easy-circuitpython-flashing
a custom build is not needed with this technique
the uf2 will be big and will take a long time to flash
but it's a one-step thign
candid crag
thank you @tulip sleet i will give this a try
slender iron
I did look to see if there were any interesting changes in sdkconfig settings by...
I think it is mostly the migration guide. The scripts can help update more broadly if things are removed