Alright, I can do that myself then. Thanks again!
#circuitpython-dev
serene token
onyx hinge
oh not sure if the guide mentions it but you have to make BOARD=... clean after changing .mk files, the changes are not automatically detected (or may be partially applied)
serene token
Oh, that makes sense. I'll remember that!
Ok, question:
A handful of the settings were removed in this commit. This includes the CIRCUITPY_ROTARYIO = 1. I've added it back and set it to 0, but that makes me think there are some kind of defaults not included in the file that might make it hard for me to know what I can remove. I don't know if you might know anything about this, but it's a point of confusion/concern for me.
slender iron
Ok, question:
https://github.com/adafruit/circuitpython/commit/c37f021791ec3d9a...
I tend to add V=2 to the make command because all of the CIRCUITPY_ directives end up on the command line
that way you can see what they are all set to
serene token
Oh cool! That definitely helps. Thanks!
stuck elbow
I'm trying to compile one of my own boards with some frozen py files, and I'm getting:
MKMANIFEST ../../frozen/circuitpython-stage/pewpew_pi
(cd build-ugame22/frozen_mpy && find * -name \*.py -exec printf 'freeze_as_mpy("frozen_mpy", "%s")\n' {} \; )> build-ugame22/manifest.py.tmp && mv -f build-ugame22/manifest.py.tmp build-ugame22/manifest.py
MPY pew.py
MPY stage.py
MPY ugame.py
GEN build-ugame22/frozen_content.c
build-ugame22/stage2.c:128:16: error: 'read_flash_sreg' defined but not used [-Werror=unused-function]
128 | static uint8_t read_flash_sreg(uint8_t status_command) {
|
anybody has any idea what changed?
I never defined or used such a function in my code
onyx hinge
I'm trying to compile one of my own boards with some frozen py files, and I'm ge...
stage2.c is a file generated during the build for rp2040 boards. If something went wrong when generating stage2 (what? not sure) maybe that error could result. does it happen after a make clean?
stuck elbow
that is after make clean
onyx hinge
what flash chip?
stuck elbow
W25Q16JVxM
onyx hinge
I bet there's a bug in stage2.c.jinja. the function read_flash_sreg shouldn't be generated if not quad_ok
stuck elbow
it is a qspi chip
onyx hinge
OK, just guessing here
onyx hinge
I use git submodule update --recursive but I'm not sure if that's the official advice
can you post the full content of the generated stage2.c in the build directory?
stuck elbow
ah, fetch-port-submodules
stuck elbow
can you post the full content of the generated `stage2.c` in the build directory...
make: *** No rule to make target 'fetch-port-submodules'. Stop.
hmmm
onyx hinge
OK I do get that problem if I change the flash chip to W25Q16JVxM
stuck elbow
serves me right for not submitting that board definition
I thought this has something to do with my library called stage
onyx hinge
I think the template is concluding that there is no register to set to enable quad spi access, so it never uses that register read function, which causes the fatal error
so it would make sense to fix that, by NOT defining the functions if they are not used. but from what you said, it's not correct that this chip should be treated as not quad SPI
so there may be a bug in nvm.toml or cascadetoml?
stuck elbow
but the mini_sam_m4 and datalore_ip_m4 boards also use the same chip?
onyx hinge
this stage2.c.jinja thing is only for rp2040
it prepares the 256 bytes at the start of the flash chip that function as the 2nd stage bootloader. it configures the flash chip and XIP before starting the main application (circuitpython)
stuck elbow
The W25Q16JV supports Quad SPI operation when using instructions such as “Fast Read Quad Output
(6Bh)”, and “Fast Read Quad I/O (EBh). These instructions allow data to be transferred to or from the device
four to six times the rate of ordinary Serial Flash. When using Quad SPI instructions, the DI and DO pins
become bidirectional IO0 and IO1, with the additional I/O pins: IO2, IO3.
onyx hinge
diff --git a/ports/raspberrypi/stage2.c.jinja b/ports/raspberrypi/stage2.c.jinja
index 4c001525dc..ee6f3bac21 100644
--- a/ports/raspberrypi/stage2.c.jinja
+++ b/ports/raspberrypi/stage2.c.jinja
@@ -36,7 +36,10 @@
#define SREG_DATA 0x02
+// If !quad_ok, these functions are not used.
+__attribute__((unused))
static uint32_t wait_and_read(uint8_t);
+__attribute__((unused))
static uint8_t read_flash_sreg(uint8_t status_command);
// This function is use by the bootloader to enable the XIP flash. It is also
```Adding these tells the compiler it's OK if the functions are not used and lets compilation continuenothing about any registers, though
onyx hinge
@slender iron ^^^ do you have some wisdom about this? I know you had all this info in your head at one time
stuck elbow
I'm worried that they should be used, but the chip definition lacks the register number or something like that
onyx hinge
quad_enable_bit_mask = all_match(flashes["nvm"], "quad_enable_bit_mask")
```there has to be a `quad_enable_status_byte` and `_bit_mask`.The Quad Enable (QE) bit is a non-volatile read/write bit in the status register (S9) that allows Quad SPI
operation. When the QE bit is set to a 0 state (factory default for part numbers with ordering options “IM”),
the /WP pin and /HOLD are enabled. When the QE bit is set to a 1(factory default for Quad Enabled part
numbers with ordering option “IQ”), the Quad IO2 and IO3 pins are enabled, and /WP and /HOLD functions
are disabled.
but I'm not sure what is the register number?
2?
onyx hinge
quad_enable_bit_mask = 0x02
```nvm.toml indeed lacks the register number.but it's there:
# Settings for the Winbond W25Q128JV-PM 16MiB SPI flash. Note that JV-IM has a different .memory_type (0x70)
# Datasheet: https://www.winbond.com/resource-files/w25q128jv%20revf%2003272018%20plus.pdf
total_size = 0x1000000 # 16 MiB
start_up_time_us = 5000
memory_type = 0x70
capacity = 0x18
max_clock_speed_mhz = 133
quad_enable_status_byte = 2
quad_enable_bit_mask = 0x02
write_status_register_split = true
01_continuous_status_write = true
6b_quad_read = true
32_qspi_write = true
ah, that's the 128. sorry
ok, adding quad_enable_status_byte = 2 makes it compile and work on my device
onyx hinge
at least that's good information that the correct value is probably 2.
onyx hinge
but it's there:
```
# Settings for the Winbond W25Q128JV-PM 16MiB SPI flash. Not...
you probably know the drill but you'll need to submit a PR to nvm.toml and once that's merged submit a 2nd PR to update nvm.toml in circuitpython
that 2nd PR could just be the one for adding the board, if you choose to do that.
stuck elbow
you probably know the drill but you'll need to submit a PR to nvm.toml and once ...
yeah, on it
shaves that yak
@onyx hinge thanks for help, I would have never found it myself
onyx hinge
you're quite welcome of course
devout jolt
I have a socket question. In CPython, I can create a Multicast UDP sender or receiver by doing socket.setsockopt() , e.g. for a multicast receiver:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
ttl = 2
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, ttl)
while True:
sock.sendto(udp_message, (udp_host,udp_port) )
time.sleep(1)
and then if that's invoked with a udp_host,udp_port=224.0.0.1,5000 I am sending on the LAN multicast net to multicast group 5000. (I put some simple working examples in CPython here: https://gist.github.com/todbot/375e539740cfb790e8c6296ca423ebfe)
This UDP receiver code is exactly the same as unicast UDP (which does work in CircuitPython), except for setsockopt.
So how do I do setsockopt equivalent in CircuitPython? I'm mostly concerned about ESP32-S{2,3} cores if that helps
onyx hinge
we do have setsockopt according to the docs: shared-bindings/socketpool/Socket.c://| def setsockopt(self, level: int, optname: int, value: int) -> None:
but the related constants are almost certainly not available.
it looks like from doing full text search that IPPROTO_IP is 0 (matches linux python) and IP_MULTICAST_TTL is 5 (linux python is 33)
tulip sleet
it does appear that Espressif will support multicast
manic glacierBOT
devout jolt
it looks like from doing full text search that IPPROTO_IP is 0 (matches linux py...
ahh! how did I miss that. trying it now
onyx hinge
adding more constants is not tough, so we can do that if they're useful
tulip sleet
devout jolt
it looks like from doing full text search that IPPROTO_IP is 0 (matches linux py...
omg that actually worked. I now have an ESP32-S3 QTPy receiving multicast UDP while my Linux box and Mac laptop are also receiving them. gist updated: https://gist.github.com/todbot/375e539740cfb790e8c6296ca423ebfe
onyx hinge
ok cool!
serene token
Ok, I've got things setup for compiling a custom CP for rotary trinkey, and I used the advice of @slender iron to get a list of the CIRCUITPY parameters. I have some questions about some of them. To start with:
I see that busio.SPI and busio.I2C are part of the default modules. The rotary trinkey doesn't have enough exposed pins to use either of those procols (well, I guess you could solder to the touchpad for the forth...). Is there some other reason to keep those, if I'm not going to use them?
devout jolt
interesting: <https://stackoverflow.com/questions/51150014/esp32-multicast-udp-h...
Just tried sending multicast UDP at 100 Hz for a minute or so and the number of missed messages by the QTPy ESP32-S3 was about the same as my Linux box. So perhaps that stackoverflow issue is not that big of a deal now
tulip sleet
the writeup sounds more like any interference is an issue for multicast, not necessarily bad for ESP in paritcular
glad it works in general!
devout jolt
glad it works in general!
yeah I am pretty stoked. thanks to you and @onyx hinge
slender iron
Ok, I've got things setup for compiling a custom CP for rotary trinkey, and I us...
you'll need SPI if there is onboard flash. I don't remember the details of that board
serene token
Ah, that's the kind of thing I was looking for. I don't know if there is onboard flash, but I figured there might be something I hadn't thought of, that it needed for its own operations.
serene token
Well, it worked! My temperature sensor is definitely a thermistor, so now I have to build the voltage splitter and figure out how to do it on a tiny board. Turns out this board is perfect though, since I'll need the ground pad on the three rotary encoder pins for the voltage splitter. (I just set the other pin to high as a voltage source, but I'll have to measure the internal resistance to work out the numbers for calculating the temperature. That's the fun part though!...well somewhat fun.)
Anyhow, thanks! I now have a Rotary Trinkey build with analogio.
manic glacierBOT
buoyant lagoon
@patent peak when i run anything in the REPL on my zero, it just hangs and prints \n infinitely
i went into shared/runtime/pyexec.c and added prints until i could narrow down the cause:
#if CIRCUITPY_ATEXIT
mp_hal_stdout_tx_str("g\r\n");
if (exec_flags & EXEC_FLAG_SOURCE_IS_ATEXIT) {
atexit_callback_t *callback = (atexit_callback_t *)source;
mp_call_function_n_kw(callback->func, callback->n_pos, callback->n_kw, callback->args);
} else
#endif
{
mp_hal_stdout_tx_str("e\r\n");
mp_call_function_0(module_fun);
mp_hal_stdout_tx_str("c\r\n");
}
mp_hal_stdout_tx_str("f\r\n");
this just prints g&e, c&f never show up
ah, and on a second look, i now see a MICROPY_DEBUG_VERBOSE i could turn on....
/circuitpython/ports/broadcom/../../py/gc.c:124: undefined reference to `DEBUG_printf'
but that just breaks the build
forced that on, i now have debug!
gc_alloc(4 bytes -> 1 blocks)
gc_alloc(0x21f9240)
gc_alloc(2 bytes -> 1 blocks)
gc_alloc(0x21f9230)
0;�🐍REPL | 8.2.0-45-g75ba17e84-dirtygc_alloc(32 bytes -> 2 blocks)
gc_alloc(0x2c5900)
Adafruit CircuitPython 8.2.0-45-g75ba17e84-dirty on 2023-08-19; Raspberry Pi Zero with bcm2835
Type "help()" for more information.
>>>
>>>
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(0x2c5ba0)
gc_alloc(33 bytes -> 3 blocks)
gc_alloc(0x2c5bb0)
gc_alloc(24 bytes -> 2 blocks)
gc_alloc(0x2c5be0)
gc_alloc(12 bytes -> 1 blocks)
gc_alloc(0x2c5c00)
it gets into a loop doing this, and doesnt seem to return
e
calling function 0x2c5940(n_args=0, n_kw=0, args=0x0)
call == 0x2a9c4
0002a9c4 <fun_bc_call>:
2a9c4: e92d4ff8 push {r3, r4, r5, r6, r7, r8, r9, sl, fp, lr}
2a9c8: e1a05001 mov r5, r1
fun_bc_call() doesnt seem to be printing its own debug properly
manic glacierBOT
Prompted by a user on discord who asked why canio cannot be imported on the Adafruit RP2040 CAN Bus Feather
tulip sleet
@buoyant lagoon Since I think the printing stuff used to work, doing a bisect or similar would narrow down what changed caused the problem. You could narrow down the version range manually with the builds archive: https://adafruit-circuit-python.s3.amazonaws.com/index.html?prefix=bin, and then do an actual bisect. Doing bisects across major version changes can be tricky.
buoyant lagoon
i did also find another issue
i think its calling fun_bc_call()
because in the mp_call_function_n_kw() area, that is what was in the call pointer
but the logs in fun_bc_call() never run
is the MMU in use at all?
tulip sleet
the cause may be some second-order bug. Not sure about the MMU, but there have been plenty of cache issues.
buoyant lagoon
fun_bc_call() also uses a different print function for its debug
buoyant lagoon
the tinyusb side appears to be working fully
when it crashes, its spamming \n's over the cdc uart!
so its still able to drive the usb device fully
and its spamming them to the framebuffer, and hw uart at the same time
so all of the IO seems to be working
tulip sleet
but that could be caused by some error in USB interrupt handling or something: repeatedly sending same buffer or someting
so I think bisecting will be your friend here
buoyant lagoon
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(0x3c62a0)
gc_alloc(33 bytes -> 3 blocks)
gc_alloc(0x3c62b0)
gc_alloc(24 bytes -> 2 blocks)
gc_alloc(0x3c62e0)
gc_alloc(12 bytes -> 1 blocks)
gc_alloc(0x3c6300)
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(0x3c6310)
gc_alloc(33 bytes -> 3 blocks)
gc_alloc(0x3c6320)
gc_alloc(24 bytes -> 2 blocks)
gc_alloc(0x3c6350)
gc_alloc(12 bytes -> 1 blocks)
gc_alloc(0x3c6370)
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(0x3c6380)
gc_alloc(33 bytes -> 3 blocks)
gc_alloc(0x3c6390)
gc_alloc(24 bytes -> 2 blocks)
gc_alloc(0x3c63c0)
gc_alloc(12 bytes -> 1 blocks)
gc_alloc(0x3c63e0)
that doesnt explain it looping like this
tulip sleet
I almost always start with a bisect if it used to work, unless I have some confidence that a certain PR is involved
buoyant lagoon
tinyusb avoids malloc at all costs
let me plug the zero back in, and try to narrow it down some more...
tulip sleet
but the things that call tinyusb malloc, and if they are getting different or wrong info back from tinyusb...
buoyant lagoon
i think first, i should try to backtrace from gc_alloc
i'm expecting trouble from git bisect, because i modified the makefile some
DEBUG_printf("gc_alloc(" UINT_FMT " bytes -> " UINT_FMT " blocks) caller=%p\n", n_bytes, n_blocks, __builtin_return_address(0));
gc_alloc(16 bytes -> 1 blocks) caller=0x297b0
gc_alloc(0x3c5ba0)
gc_alloc(33 bytes -> 3 blocks) caller=0x297c8
gc_alloc(0x3c5bb0)
gc_alloc(24 bytes -> 2 blocks) caller=0x29864
gc_alloc(0x3c5be0)
gc_alloc(12 bytes -> 1 blocks) caller=0x29888
gc_alloc(0x3c5c00)
it now prints this while looping
tulip sleet
I'm saying try 7.3.3, then 8.0.0 final, etc., to see if you can get it narrowed to a version range. Then you can start with a bisect with a narrow range that doesn't span branch changes in a clean clone
buoyant lagoon
00029790 <mp_obj_new_exception_msg_vlist>:
297ac: ebff7aa6 bl 824c <m_malloc_maybe>
297b0: e1a04000 mov r4, r0
...
that seems promising, if its in an infiniteloop, trying to build an exception
and then has an exception while building an exception!
tulip sleet
going afk for a while
buoyant lagoon
fmt: 0xc9b8c, caller: 0x29948
0002992c <mp_obj_new_exception_msg_varg>:
0001f840 <mp_raise_msg>:
msg: 0xc9ba4, caller: 0x1fa28
0001fa18 <mp_raise_RuntimeError>:
1fa18: e1a01000 mov r1, r0
1fa1c: e92d4010 push {r4, lr}
1fa20: e59f0000 ldr r0, [pc] ; 1fa28 <mp_raise_RuntimeError+0x10>
1fa24: ebffff85 bl 1f840 <mp_raise_msg>
1fa28: 000c60e0 .word 0x000c60e0
msg: 0xc9ba4, caller: 0x21e4c
00021e40 <mp_raise_recursion_depth>:
21e40: e92d4010 push {r4, lr}
21e44: e59f0000 ldr r0, [pc] ; 21e4c <mp_raise_recursion_depth+0xc>
21e48: ebfff6f2 bl 1fa18 <mp_raise_RuntimeError>
21e4c: 000c9ba4 .word 0x000c9ba4
aha, recursion! while creating a exception about recursion! lol
#if MICROPY_STACK_CHECK || MICROPY_ENABLE_PYSTACK
NORETURN MP_COLD void mp_raise_recursion_depth(void) {
mp_raise_RuntimeError(MP_ERROR_TEXT("maximum recursion depth exceeded"));
ah yep, that explains it perfectly
buoyant lagoon
but the logs in `fun_bc_call()` never run
this does a stack-check before it prints
fun_bc_call() before stack check
msg: 0xc9bec, caller: 0x21e4c
msg: 0xc9bec, caller: 0x1fa3c
fmt: 0xc9bec, caller: 0x1f878
fmt: 0xc9bec, caller: 0x299a4
yep, bingo
usage: -176121704, limit: 65712
@tulip sleet aha, i'm using negative 167mb of stack! lol
top: 0x2309734, current: 0xcaffe98
top: 0x2309734, current: 0xcaffe9c
usage: -176121704, limit: 65712
so, the top of the stack is at 35mb
but the sp is near 202mb
Press any key to enter the REPL. Use CTRL-D to reload.
mp_stack_set_top(0x2309734)
Initializing GC heap: 0x30a4f0..0x22f9270 = 33484160 bytes
GC layout:
so, i think we have 2 bugs
1: the top of the stack is wrong
2: it checks the stack limit, while complaining about a stack limit, and infinitely recurses
port has dynamic stack
stack top 0xcb00000
allocated stack at 0x305344, size = 66736
limit 0x22f9284, length 0x104b0
mp_stack_set_top(0x2309734)
mp_stack_set_top(0x2309734)
so first, it gets the size of the current stack
the it malloc()'s about 65kb, an extra EXCEPTION_STACK_SIZE bytes so it can report failure (which failing)
then it changes the stack top, but doesnt actually change the stack
#define MICROPY_STACK_CHECK 0 is also non-functional
i patched mp_stack_check() to just not check, and now it works
buoyant lagoon
ok, lets see, where do i begin with displayio....
stuck elbow
was there any bug in 8.2.3 that would explain it not working on the PewPew M4? I compiled the latest source, and it works, so it must have been fixed in the mean time?
buoyant lagoon
[root@amd-nixos:/etc]# hexdump -C /dev/sdc
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000001c0 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 00 |................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.|
00000200
[root@amd-nixos:/etc]# fdisk -l /dev/sdc
Disk /dev/sdc: 512 B, 512 bytes, 1 sectors
Disk model: Zero
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000
Device Boot Start End Sectors Size Id Type
/dev/sdc1 1 1 0 0B 1 FAT12
ah, thats why thats borked
its only a 512 byte disk drive!
supervisor_flash_get_block_count() claims its 0 blocks
ah, the partition loop code is rather dumb
if it encounters a 3rd or 4th partition, it uses none
tulip sleet
was there any bug in 8.2.3 that would explain it not working on the PewPew M4? I...
not that I know of; do you mean it doesn't come up at all?
buoyant lagoon
@tulip sleet any tips on where i would start with implementing sprites with displayio? i know how the hw works, but not how circuitpython expects it to be exposed
stuck elbow
not that I know of; do you mean it doesn't come up at all?
yeah, no disk, no serial
no screen
I didn't try jlink with it
tulip sleet
yeah, no disk, no serial
is the pewpew similar to any board I might have?
stuck elbow
I think it's the only samd51 board without external flash out there
tulip sleet
<@329766224093249548> any tips on where i would start with implementing sprites ...
TileGrids are sprites, basically, e.g. https://learn.adafruit.com/circuitpython-display-support-using-displayio/sprite-sheet
buoyant lagoon
the issue, is that i believe the pi-zero currently lacks sprite/displayio support
and i wish to add it
tulip sleet
the issue, is that i believe the pi-zero currently lacks sprite/displayio suppor...
there is displayio support for a framebuffer thing that works with the HDMI output. I thought it was turned on
buoyant lagoon
yes, but that is not hardware sprites
so it takes a ton of cpu usage to do anything
even just scrolling the text console is cpu intensive
tulip sleet
does the broadcom chip have hw sprites?
buoyant lagoon
yes
this demo uses <1% cpu
the 2d pipeline on the SoC is sprite ONLY
the framebuffer CP is using, it just a static sprite
tulip sleet
you should talk to Scott about it, since he was directly inspired by sprites for TileGrid. Maybe he had possible hw in mind. But it was insspired by much less powerful machines without a lot of specialized display hw, e.g. old game consoles
buoyant lagoon
at the most basic level, you give the hardware a list of sprites
for each sprite, you define:
- the addr in ram
- the source width/height/stride
- the x/y on the screen
- the dest width/height
by using a width of 32 pixels, but a stride of 320 pixels, you can easily extract a tile from a tileset and draw it
tulip sleet
yeah, no disk, no serial
I didn't see anything in the release notes. Do you know at what version it stopped working (and then started again?)
stuck elbow
the last version I tested is 8.0.0-beta.1
but since it works in the current version, I don't think it's important
tulip sleet
it's weird it broke and then started working again
stuck elbow
I was just asking out of curiosity
tulip sleet
I could probably try the pewpew build on a random SAMD51 board, since I'm just trying to see if it loads
stuck elbow
hmm, let me try with the "abosulte lates" build from the download, maybe it only works when I compile it myself
tulip sleet
and/or build 8.2.3 yourself and see
buoyant lagoon
you should talk to Scott about it, since he was directly inspired by sprites for...
the only real differences vs older consoles, is that the w/h are plain ints, not an enum
the overall res is higher
and it can handle a lot more sprites
tulip sleet
so is there something missing from the current TileGrid API?
buoyant lagoon
so is there something missing from the current TileGrid API?
i think its just missing an implementation of TileGrid for the bcm283{5,6,7}
stuck elbow
hmmm, adafruit-circuitpython-pewpew_m4-en_US-20230818-babd2de.uf2 also crashes
tulip sleet
TileGrid is is in shared-module: there are not port-specific impls for that kind of class in displayio
buoyant lagoon
where is the port specific code, for configuring hardware sprites?
or is it always cpu rendering?
buoyant lagoon
ahh, thats a much bigger task then
tulip sleet
because on boards up to now, the CPU had to drive the display anyway
you could just provide an alternate impl for TileGrid, but I'm not sure it's at the TIleGrid level -- it's in the display update code, maybe?
buoyant lagoon
@slender iron you got any thoughts?
stuck elbow
hmm, if I press reset to get into the safe mode, it doesn't crash
tulip sleet
i have seen that too, when loading a new UF2. I go into safe mode but a reset cures it
I am usually connected via serial so it's not surprising
I have been seeing this for a long time
stuck elbow
if I press the rst just right, I get this, but no disk and no serial, otherwise it's just unitialized screen
lsusb doesn't show it either
which compiler is used for the builds published on circuitpython.org?
I will connect a jlink to it tomorrow and see what it crashes on
tulip sleet
which compiler is used for the builds published on circuitpython.org?
arm-none-eabi-gcc (GNU Arm Embedded Toolchain 10-2020-q4-major) 10.2.1 20201103 (release)
See https://adafruit-circuit-python.s3.amazonaws.com/index.html?prefix= for the tar files we use
tulip sleet
how about the bootloader version you are using?
I don't see why that should matter, but maybe..
tulip sleet
i tried loading PewPew M4 on a Metro M4: it loads but nothing happens
it does not go back to the bootloader
stuck elbow
no, it just hangs
if you press reset to get into safe mode, it will go into safe mode, but without usb
(I guess you won't see it wihout a screen)
tulip sleet
oh it is a 51g, I should try an itsy. I have to put some food in the oven, back in a bit
stuck elbow
I compiled 8.2.3 myself and it also crashes, I'm going to try earlier versions
onyx hinge
fwiw I was using up to date versions on a pygamer just this week, so it's not that all samd51 are broken.
stuck elbow
yeah, someone would have noticed I'm sure
why did I save those few cents on the flash chip
8.0.5 works, 8.1.0 crashes, I'm going to check the betas
tulip sleet
see if any changes to the samd-peripherals submodule
stuck elbow
I'm not updating the submodules, so it must be a change in the main code
tulip sleet
i was thinking maybe someone submitted a PR to samd-peripherals which we incorporated
stuck elbow
but 8.0.5 with samd-peripherals from 8.2.3 works
tulip sleet
and you're not updating tinyusb either?
tulip sleet
itsy m4 8.2.3 on itsy works fine
I wonder if some part of flash is overflowing but it's not being checked properly. Could turn off some more modules and see.
stuck elbow
some of the crashing uf2s are actually smaller than the working ones
tulip sleet
so i think a bisect is in order. it's a pretty narrow range, that's good
tulip sleet
it stops working on commit 66edcf5d034432519aac18690806adad2cacf2ac
maybe the stack checking in https://github.com/adafruit/circuitpython/commit/66edcf5d034432519aac18690806adad2cacf2ac#diff-a0cb465674c1b01a07d361f25a0ef2b0214b7dfe9412b7777f89add956da10ec. There are also some shared changes in displayio. If you turn off displayio does it work?
stuck elbow
also a change in usb.c
tulip sleet
jsut initializing the identification, but you could turn off the usb identification too
@clever was poking around that stack code above
buoyant lagoon
i have found circuitpython to be fragile in terms of config
there are a number of compile-time config options, that just break it
verbose debug logging fails to link
stuck elbow
it works with displayio enabled, but display init disabled in board.c
buoyant lagoon
removing the stack bounds checking fails to even compile
stuck elbow
yup, it's the ^&#$#&* logo, removing #define CIRCUITPY_REPL_LOGO 0 makes it work again
I think it's because of this?
changing it to
#if CIRCUITPY_REPL_LOGO
scroll_area->height_in_tiles -= 1;
#else
scroll_area->height_in_tiles -= 1;
#endif
makes it work again, but why
ah, because the status bar is still there
buoyant lagoon
yup, it's the ^&#$#&* logo, removing `#define CIRCUITPY_REPL_LOGO 0` makes it wo...
just more config being fragile, like i mentioned above
stuck elbow
just more config being fragile, like i mentioned above
that's not helpful
buoyant lagoon
more a comment on the general codebase, those config flags need better testing
buoyant lagoon
looks
i can confirm
changing CIRCUITPY_REPL_LOGO to 0 breaks the rpi-zero port too
so its not a samd problem
it enumerated over usb, then stopped responding
buoyant lagoon
checking the source...
if the logo is on, blinka_bitmap_data and blinka_bitmap are declared as extern vars
stuck elbow
this fixes it for me
buoyant lagoon
ah, so the height was probably undefined
and then it went out of bounds on the framebuffer
stuck elbow
it was defined, but was too big together with the status bar
because CIRCUITPY_REPL_LOGO disables the logo, but the status bar is currently hardcoded
buoyant lagoon
and then because the height wasnt decreased, it went 1 tile past the end of the framebuffer?
stuck elbow
possibly
stuck elbow
no it shouldn't, it slow enough as it is
buoyant lagoon
a debug only check then?
so when it does crash, you can turn that on, and instantly get an obvious error
stuck elbow
nobody ever enabled debug
tulip sleet
glad this is figured out
stuck elbow
@tulip sleet I'm still suspicious about what happens when the logo is bigger than the status bar
shouldn't then it do -2 or even more on the scroll area?
tulip sleet
maybe so -- I have a vague memory of using an emoji that just fits in the status bar, instead of the standalone logo?
stuck elbow
but the code clearly allows the logo to be bigger than the font
buoyant lagoon
no it shouldn't, it slow enough as it is
it could also be a one-time check at startup, that terminal top + height <= framebuffer height
stuck elbow
it could also be a one-time check at startup, that terminal top + height <= fram...
and then you have to go fishing for more room for japanese translation
it's a compromise
buoyant lagoon
no matter what font your using, the terminal should be contained within the bounds of the display
stuck elbow
that's what happens when the code is correct
when it's not correct, run-time checks won't help with it
(though they may help pin-pointing the problem)
let's face it, the whole micropython is a huge hack, with a lot of very clever but fragile tricks
stuck elbow
written in a language from the '70s
buoyant lagoon
one thing i could see more work being put into, is automated testing on real hardware
tulip sleet
we have worked on this on and off; I would like to. Scott originally did it with an RPi, and it was not reliable enough. A more capable test computer would be better. I have spare mini PC's.
buoyant lagoon
https://gist.github.com/cleverca22/f7ec8335d719875c805fb775a00b14d7
with a minor change like this, and
sudo ./rpiboot -l -d ../circuitpython/ports/broadcom/build-raspberrypi_zero/rpiboot
you can boot circuitpython on a pi-zero or pi-zero2, without having to re-flash the SD card
then all you need is a gpio pin to hard-reset the device when its crashed
buoyant lagoon
the pi1/pi2/pi3/pi4 can network boot, so if you serve the above directory over tftp, you can repeat the same thing
(pi1/pi2 need bootcode.bin on an SD card to netboot)
that covers everything under ports/broadcom i think
ports/raspberrypi, either use 2 gpio, for bootsel+run, or 2 gpio for full SWD
if bootsel is pulled during boot, you can UF2 flash it over usb
if you use SWD, you can program it with an elf and openocd
tulip sleet
@stuck elbow weird build failure in the scheduler job in your PR. I am re-running just in case it is transient.
stuck elbow
the yaks just keep on coming ;-)
tulip sleet
:🐃 💈
buoyant lagoon
@tulip sleet after thinking about it some, i can picture some ways you could take the broadcom SoC design, and apply it to things like the pico-dvi, to change up how sprites are handled, and potentially improve performance, and maybe even reduce ram usage
i assume that pico-dvi currently just has a standard framebuffer, for whatever the full resolution is, in ram at all times?
tulip sleet
yes
buoyant lagoon
and the dvi code is just converting it constantly
the sprite code then draws into that
tulip sleet
it is PIO
buoyant lagoon
yep
internally, the HVS on the rpi, will scan the sprite list, composite sprites on the fly, and fill a FIFO with pixels
and the HDMI block then converts that into HDMI/DVI
in theory, you could recreate that HVS on the pico, and just delete the framebuffer
tulip sleet
is the background a big "sprite"?
buoyant lagoon
is the background a big "sprite"?
exactly
what i'm thinking, is that you have a FIFO between the new displayIO, and pico-dvi, that holds pixels
stuck elbow
plus there is stuff like the shapes
buoyant lagoon
and any time that FIFO has room, you scan the sprite list, fetch pixels from ram, and write to the FIFO
and as long as you can generate pixels at the pixel clock, and the FIFO eats any lag spikes, it works
stuck elbow
that's how displayio works on spi displays, roughly
buoyant lagoon
ah, so you already have that code
buoyant lagoon
the rpi hardware is doing exactly the same thing on every frame
stuck elbow
it was inspired with how gameboy did it
buoyant lagoon
yeah, there is a trade-off, of time per pixel, constantly
vs time per draw operation
stuck elbow
it gets slower if you have many overlapping layers
so at some point you can no longer keep up
buoyant lagoon
in the broadcom implementation, the FIFO holds a couple scanlines
and rather then being a true FIFO, its just a block of ram, so you have a set of pointers defining where the new scanline is, and it draws the sprites into that region
buoyant lagoon
it gets slower if you have many overlapping layers
the hardware in the rpi has the exact same issue
the glitching you see here, is when too many sprites overlap, and i underclocked it hard
i was mostly under-clocking it, to measure where the limit was
stuck elbow
wouldn't it make more sense to use the gpu?
buoyant lagoon
this is the gpu
stuck elbow
I mean the 3d one
buoyant lagoon
the rpi has a 2d only gpu, and a 3d gpu
page-flipping and double-buffering can only be done via sprites on the 2d core
buoyant lagoon
I mean the 3d one
hmmm...
with this example screenshot...
i have a 9x9 grid of sprites, 81 sprites in total
i can repaint the entire map, by just doing 81 x 32bit writes to the display list
while with the 3d core...
i would need to update 4 vertices per sprite, each with a 32bit floating point X/Y
2592 bytes
then i have to run the 3d job, which will write the entire frame to ram
not a cpu task, but still eats up dram bandwidth
then i have to change out the image behind a 2d sprite, to make the 3d render visible
so, for a 2d job, the 2d core is far more light weight
stuck elbow
I know there are some python libs for rpi that use the 3d gpu for 2d sprites and they are pretty fast
faster than sdl anyways
buoyant lagoon
throw enough cpu at the problem, and you can mask the overhead from something taking over 8x the resources
tulip sleet
@stuck elbow because the build is only building your board, it appears SRC_SUPERVSIOR is not being set (I'm not sure why), but I think the fix would be to change if file in settings["SRC_SUPERVISOR"]: in tools/ci_set_matrix.py to if file in settings.get["SRC_SUPERVISOR", ""]. There are some similar examples in that script.
stuck elbow
get() not get[], I will add that
tulip sleet
it is working ok on 8.2.x?!
yes, sorry, copy/pasta
but WHY is a mystery, since you do need SRC_SUPERVISOR files
stuck elbow
I tested on that 8.0.1-beta-2, but there were no other changes in that code since
tulip sleet
yeah, I looked at blame
buoyant lagoon
I know there are some python libs for rpi that use the 3d gpu for 2d sprites and...
a lot of the demo animations ive been doing, use <1% cpu time, which means more cpu time for real work
stuck elbow
a lot of the demo animations ive been doing, use <1% cpu time, which means more ...
it's easier when you can just write it for your own needs, gets harder when you have to abide by existing api, like displayio
buoyant lagoon
yeah
stuck elbow
but of course if you are willing to do the work, that's great
buoyant lagoon
what does displayio have in terms of api?
buoyant lagoon
displayio.Colorspace, the rpi hardware allows each sprite to exist in its own colorspace, and supports a palette
are shapes rects or polygons?
stuck elbow
yeah, each bitmap or ondiskbitmap has its own palette
buoyant lagoon
rotation or just mirroring?
stuck elbow
they are are more complex, lines, circles, polygons, rectangles, rounded rectangles, ellipses, etc.
buoyant lagoon
ah, so you would need to use per-pixel alpha then
stuck elbow
rotation by 90° and mirroring
buoyant lagoon
rotation by 90° is the only part i havent figured out yet on the rpi
its an extra step
mirroring is just a 1bit flag, for each axis
stuck elbow
the way the shapes work, they calculate their pixels on the fly
buoyant lagoon
/* 8bpp */
HVS_PIXEL_FORMAT_RGB332 = 0,
/* 16bpp */
HVS_PIXEL_FORMAT_RGBA4444 = 1,
HVS_PIXEL_FORMAT_RGB555 = 2,
HVS_PIXEL_FORMAT_RGBA5551 = 3,
HVS_PIXEL_FORMAT_RGB565 = 4,
/* 24bpp */
HVS_PIXEL_FORMAT_RGB888 = 5,
HVS_PIXEL_FORMAT_RGBA6666 = 6,
/* 32bpp */
HVS_PIXEL_FORMAT_RGBA8888 = 7,
HVS_PIXEL_FORMAT_PALETTE = 13,
these are the pixel formats that sprites can be in, which look useful
stuck elbow
spi displays just use the RGB565
buoyant lagoon
1/3/6/7/13 all support per-pixel alpha
3 only has 1bit of alpha per pixel, so its only good for masking a shape off
the rest support varing amounts, from 4bit to 8bits of alpha, so they can blend
palette formats support 1/2/4/8 bits per pixel
and the palette is just an array of RGBA8888, so you have the full 8bits of range, for every color, and alpha
stuck elbow
perhaps you should start by writing your own graphics library first, and once you are familiar with it, come back to displayio
buoyant lagoon
i already have
buoyant lagoon
stuck elbow
(which is not hard)
buoyant lagoon
lines 33-49 will generate a sprite that is 32x34, and at the right coords on-screen
lines 51-55 will change the tile within the sprite
and this is the end result of that code
one option, would be to just port this to circuitpython, as its own api, and just ignore displayio temporarily
stuck elbow
that's what I meant
buoyant lagoon
yeah, i realized that after saying it
i could also slightly hijack display-io, and turn it into another sprite, so you can still see the old ui
another thing sprites can massively help with, is scrolling
this would be the standard terminal in little-kernel, it just uses memcpy() to scroll
stuck elbow
yeah, scrolling is something displayio struggles with
buoyant lagoon
and it scrolls slow enough that you can almost read it as it scrolls
this treats the framebuffer as a ringbuffer, and the write-ptr just wraps, and it never scrolls
stuck elbow
<@351136009922019331> because the build is only building your board, it appears ...
looks like that helped
buoyant lagoon
it then uses sprites to draw the 2 halves of the buffer
for debug, there is a seam in the framebuffer, which you can see on the final frame of the video
but i can just align things properly, and then its seamless
and it actually scrolls too fast, it updates multiple times within 1 vsync period, and overflows some hw buffers, lol
stuck elbow
I'm heading to bed, goodnight
buoyant lagoon
nn
serene token
Fetching submodules:
Entering 'ports/espressif/esp-idf'
BUG: config.c:129: kvi should not be set while parsing a config source
error: path='ports/espressif/esp-idf'; git fetch --tags --depth 1 died of signal 6
fatal: run_command returned non-zero status for ports/espressif/esp-idf
Any ideas?
midnight ember
Well my bug hunting day is done. Found a big juicy one that affects any S2 or S3 module that is using adafruit_requests starting in 8.2.2
manic glacierBOT
onyx hinge
well that's a new wrinkle. If long ints aren't supported, only 14-bit(!) integer values can be frozen.
midnight ember
Showing 8.2.2 was released on July 31st but adafruit_requests last real commit was in June. That makes no sense to me. Is it possible the bug was introduced in 8.2.3 and then backported to 8.2.2? I have no clue how the backporting process works. Can't find a reason why things just magically broke in 8.2.2 on July 31st but everything works fine on 8.2.1 from July 25th.
July 20th was the TLS cert update for AdafruitIO. The timing is pretty close but the S2/S3 module shouldn't need that cert with Github for example.
devout jolt
huffman module problems. again. Task: I want to build mpy-cross to make mpys with minimal fuss from a fresh circuitpython clone. What do I do? My options appear to be make fetch-all-submodules or make fetch-port-submodules.
So is the most expedient way to build mpy-cross doing this?
git clone https://github.com/adafruit/circuitpython/
cd circuitpython
(cd ports/atmel-samd && make fetch-port-submodules)
make -C mpy-cross
It looks like the CI build uses a whole different thing, python3 tools/ci_fetch_deps.py mpy-cross.
you can also grab bins of mpy-cross from s3 for several architectures https://adafruit-circuit-python.s3.amazonaws.com/index.html?prefix=bin/mpy-cross/
tulip sleet
`huffman` module problems. again. Task: I want to build `mpy-cross` to make mpys...
we could also add a Makefile target for submodules to the mpy-cross Makefile
devout jolt
It looks like the CI build uses a whole different thing, `python3 tools/ci_fetch...
Interesting I didn’t know there were modern prebuilds of it. That might be the route to go then for most folk
devout jolt
we could also add a Makefile target for submodules to the mpy-cross Makefile
Seems like I’m the only who keeps having this issue so I can just download the prebuilds 🙂
midnight ember
Normally I would think # is commented out. Does this mean it is or isn't included as a frozen library? https://github.com/adafruit/circuitpython/blob/main/ports/espressif/boards/unexpectedmaker_feathers2/mpconfigboard.mk
blissful pollen
Normally I would think # is commented out. Does this mean it is or isn't includ...
It is commented out so not included
midnight ember
So it's not frozen in and the library bundle should be used.
blissful pollen
I was having some issue with the TinyS2 and dotstar too but didn't have time to investigate. I thought I had the latest but maybe there is something else out of date
midnight ember
It's been a while since I've done a fresh UM install but I believe there might be a dotstar function in the feathers2.py helper file included? I'm not sure if that file is included with a new Circuit Python install... I know it ships with it at least. Possible the helper file might be overriding something? Might be a difference in what ships with a UM Feather that you get in the mail.
blissful pollen
I'll try to load mine up quickly (i'm half asleep my results may vary ha)
blissful pollen
It's been a while since I've done a fresh UM install but I believe there might b...
So there looks like an issue.
Adafruit CircuitPython 8.2.3 on 2023-08-11; FeatherS2 with ESP32S2
>>>
>>> import board
>>> import adafruit_dotstar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "adafruit_dotstar.py", line 44, in <module>
ImportError: no module named 'adafruit_pypixelbuf'```
I do not see any bundle library `adafruit_pypixelbuf`. It looks like it was archived so maybe then it does not get included in the bundle?Which 100% does not seem right looking at the dotstar module
Okay told you I was tired, I had an old dotstar version installed (oops) so on 8.2.3 with the latest library bundle download it works for me.
midnight ember
@blissful pollen So you pretty much did exactly what they did step for step. That's a happy accident. Everything working as intended now on 8.2.3 with the dotstar?
blissful pollen
<@228322543507865601> So you pretty much did exactly what they did step for step...
Yup worked fine for me
midnight ember
Awesome
I've done it myself, it's easier to make that mistake with the UM boards for some reason.
blissful pollen
I'm trying to remember why I even had adafruit_dotstar.py on my drive (not even the mpy!)
midnight ember
I think it might actually ship that way, I could be wrong.
Same reason why there's a feathers2.py or feathers3.py helper file included in the root dir.
blissful pollen
Could be, been a while since I received this board
midnight ember
same and I don't think I have any brand new ones laying around to whip out.
Thanks for buying a FeatherS3!
Your board has been tested and here are the results for your reference.
Test Date & Time: 05/29/2022, 07:21:18 <- Australia
MAC Address: obfuscated
PSRAM: 8189552 bytes
FLASH: 12224512 KB
VBAT Sense Check: PASS
VBUS Sense Check: Pass
GPIO Test: PASS
WiFi Test: PASS
WiFi Connection Time: 3.998 secs
This board is good to ship! I hope you enjoy using it :)
Cheers,
Seon
Unexpected Maker
http://unexpectedmaker.com
``` this is the results.txt included on the board... the timestamp shown would make a lot of sense if he included a dotstar.py in the lib folderI don't have it on this particular board because I already started customizing it for a project. My guess is a new board would likely have it.
It's in the root dir on my UM Feather S2... and the date.
boot_out.txt must be a message to myself from the future. Will definitely read in private.
Shipped with 8.0.5 pre-installed
Adafruit CircuitPython 8.0.5 on 2023-03-31; FeatherS2 with ESP32S2
Board ID:unexpectedmaker_feathers2
Figured out you did nothing wrong. Unexpected Maker ships the board pre-installed with Circuit Python and an adafruit_dotstar.py in the root directory which will override any adafruit_dotstar.py or .mpy in the /lib folder. That's the issue.
Delete that file and upload a newer version from the 8.x bundle to /lib and should be good to go. Was unaware of this issue.. I'm sure we'll...
midnight ember
We might need to contact Unexpected Maker and see about using a different method as it's causing us some support issues. Might be time for him to use a frozen lib and let Circuit Python update the frozen lib?
"""
`adafruit_dotstar` - DotStar strip driver (for CircuitPython 5.0+ with _pixelbuf)
=================================================================================
* Author(s): Damien P. George, Limor Fried, Scott Shawcroft & Roy Hooper
"""
# pylint: disable=ungrouped-imports
import sys
import busio
import digitalio
if sys.implementation.version[0] < 5:
import adafruit_pypixelbuf as _pixelbuf
else:
try:
import _pixelbuf
except ImportError:
import adafruit_pypixelbuf as _pixelbuf
__version__ = "0.0.0-auto.0"
__repo__ = "https://github.com/adafruit/Adafruit_CircuitPython_DotStar.git"
``` yup, the included dotstar.py is from early 2022.That's just a snippet. I can post the whole contents but I'm sure it's unnecessary for the sake of solving this issue.
tulip sleet
@atomic summit See https://github.com/adafruit/circuitpython/issues/8302 and ^^
atomic summit
We might need to contact Unexpected Maker and see about using a different method...
I was advised not to freeze any of my shipping code or libraries in my board firmware, after I did that way back when I first submitted my FeatherS2 CP board files. So that's why the files are separate.
And is why it was there and then commented out 🙂
atomic summit
I was having some issue with the TinyS2 and dotstar too but didn't have time to ...
TinyS2 doesn't use a Dotstar - It has a 1515 Neopixel.
blissful pollen
TinyS2 doesn't use a Dotstar - It has a 1515 Neopixel.
Thanks I did notice that after I tested against the FeatherS2. I knew I was too tired to be properly debugging 🙂
atomic summit
I do freeze the Neopixel library in my newer boards, so I guess it's ok to do that now, so I'm happy to go back and freeze dotstar now as well, and stop adding it during my testjig process.
manic glacierBOT
midnight ember
I do freeze the Neopixel library in my newer boards, so I guess it's ok to do th...
That helps thank you. Was going to bring it up in the meeting tomorrow but you’re right on it. Thank you.
atomic summit
That helps thank you. Was going to bring it up in the meeting tomorrow but you’r...
Nps. Sorry this popped up again - I remember it happened when the dotstar lib originally changed, but for some reason I didn't think to just freeze it. I hope to have a PR ready for tomorrow 🙂
midnight ember
You really wouldn’t even need to necessarily freeze it. Could just add a /lib folder with the .mpy version. That would allow for easy updates overwriting it with a newer version and the most minimal work on your end.
Just snag the .mpy out of the latest 8.x bundle, throw it in /lib and your all set to ship boards like that. Easiest way for you and us, to my knowledge.
Unless you’re doing something custom in the dotstar library? Seems the easiest way to me.
thorny jay
No tagging because this is the night in the US and Scott is in vacations...
I tried to replicate the MIDI Host Synth from tannewt, but on a Feather RP2040 USB Host (with I2S output): https://github.com/adafruit/Adafruit_CircuitPython_USB_Host_MIDI but I get very bad result in receiving Midi message over USB (they come a lot of seconds late). I know it works for Scott on that huge imx board but maybe there are unfixed things in TinyUSB, fix from TinyUSB not included in CP yet?
Of course it could by my keyboard... but it works well with PC and is detected by CP.
No issue yet as I should build a version of CP with TinyUSB debug if I want to know more... or have some special hardware to sniff the USB. I will just try again later.
onyx hinge
@tulip sleet you may have noticed but the merge PR is now green. I worked a bunch on it over the week-end.
tulip sleet
<@329766224093249548> you may have noticed but the merge PR is now green. I work...
I very much did notice -- thanks!! Do you want to do a proper review now, or merge?
onyx hinge
I really don't know how to effectively review it 😕 I mean, we can't possibly review every change upstream made.
tulip sleet
I agree -- I mean would you take the time to go through each file to catch anything obvious, or it's not worth it.
I sort of did review the upstream changes, in the sense that I had to understand them and merge appropriately.
onyx hinge
do you have a scheme for making an adafruit_circuitpython_asyncio that can run on CP 8.x or 9.x?
tulip sleet
do you have a scheme for making an adafruit_circuitpython_asyncio that can run o...
I think the updated asyncio might still be compatible for normal use. The exception-handling edge cases might be different, but that may not matter. If the issue is _asyncio differences, that is optional, and we can turn it off on one or the other.
onyx hinge
@tulip sleet thanks for moving/centralizing ```make
Print out the value of a make variable.
https://stackoverflow.com/questions/16467718/how-to-print-out-a-variable-in-makefile
print-%:
@echo $* = $($*)
We can have the 9.0.0 builds point to a branch for now, as it does in the current PR
onyx hinge
yes though that doesn't help folks who use the bundle
tulip sleet
we don't have a 9.0.0 bundle anyway, or we can special-case something in the bundle code. I'm not even sure we should build 9.0.0 bundles until the mpy version is not going to change.
But I think it's worth testing the changed asyncio against 8.x.x.
tulip sleet
the changes were really all internal: the queue stuff was redone, but it's internally consistent. There aren't user API changes that I recall
onyx hinge
+ File "extmod/uasyncio_gather.py", line 55, in <module>
+ File "/home/jepler/src/cp-pristine/frozen/Adafruit_CircuitPython_asyncio/asyncio/core.py", line 303, in run
+ File "/home/jepler/src/cp-pristine/frozen/Adafruit_CircuitPython_asyncio/asyncio/core.py", line 214, in create_task
+AttributeError: 'TaskQueue' object has no attribute 'push'
```doesn't seem to work, sadlyOK, this is due to _asyncio.TaskQueue native module differences. we could add compatibility code to check for . push() replaced push_head() and push_sorted() -- not sure why there were two.
tulip sleet
I don't think we need to solve this problem yet. Let's get v1.20 merged, and use the branched asyncio for now, and include a warning to use the new branch version in the release notes.
This issue of needing incompatible library versions does keep coming up -- I have an issue in the bundle repo about it.
When we get closer to a 9.0.0 release, we can think about adding compatibility code.
onyx hinge
OK
tulip sleet
if we just turned off using _asyncio if the version was 8 or lower, that would work, later
onyx hinge
it looks like adding hasattr() based check would work too
tulip sleet
yes
but I don't think we need to mess with the main-line asyncio librarynow, it's too early to do that
pop_head() mapped to pop() is fine. I just don't know why push has sorted and head options.
manic glacierBOT
@dhalbert I got my RGB matrices all from https://www.digikey.com.au/en/products/detail/adafruit-industries-llc/2278/7035036
For the RGB Matrix in the videos. I don't have the original box.
I do have another two spares I haven't tested yet.
lone axle
<@&356864093652516868> Reminder, the weekly meeting will be occuring here on Discord about 90 from now at 2pm Eastern / 11am pacific. The The notes Doc is here: https://docs.google.com/document/d/1DMsb91O4e6KFiV7sn0TroXW4g6Dr9yukYOOC_vaCMto/edit if you'd like to add your hug reports and / or status updates. We look forward to hearing from everyone!
idle owl
@tulip sleet How would I go about checking to see if a file exists on storage? in CP. Same directory as code.py.
I assume os something or other, but I'm not sure what.
tulip sleet
<@329766224093249548> How would I go about checking to see if a file exists on s...
os.stat()
idle owl
Cheers!
tulip sleet
Cheers!
You need to catch FileNotFoundError. You could also just try open()ing it and catch the failure in your program
you can also do os.listdir() and check for it in the list of files
idle owl
You need to catch `FileNotFoundError`. You could also just try `open()`ing it an...
stat worked.
Oh ok
Thank you!
lone axle
@onyx hinge are you available to read the core section of the state of CircuitPython for the meeting?
thorny jay
Sorry, missing the meeting and poor quality report... but I was asked to start a BBQ (and now I am eating the result of it).
onyx hinge
@lone axle yes I will read it!
lone axle
Thank you.
random junco
midnight ember
Nice Todbot and Tyler!
onyx hinge
amazing @devout jolt seems to have written MicroOSC just yesterday, after mentioning something about it on CircuitPython day.
midnight ember
Watching FoamyGuy use the tileset program was really interesting.
buoyant lagoon
yeah, tiled saves so much time with tile based games
idle owl
stuck elbow
but xml :-(
devout jolt
oh yeah if other circuitpythonistas would like a gratis board from my Tindie, DM me your address and what you want 🙂
onyx hinge
I also meant to specifically thank those who had good questions when we moved into the Q&A segments of the streams.
idle owl
😂
midnight ember
You have to check every single silkscreen for the revision? oof.
idle owl
You have to check every single silkscreen for the revision? oof.
There's a pic of the board on the product page that has the revision in it. And you have to be on the product page to get the rest of the info.
So, oof, yeah, but not as bad as it sounds.
midnight ember
Do they always update the images with new revs 100% though?
idle owl
95% of the time.
And the OSHWA submissions can be updated if needed. So it's not a huge deal if it's not quite right.
Most of what foamyguy is going through are the new products though, so there won't be new revs yet.
devout jolt
good sleuthing, @midnight ember on that adafruit_dotstar.py issue! I was following that and omg
midnight ember
I'd be really interested in learning the # of boards that bug will end up affecting.
Only tested on 3 boards with S2/S3 and it failed on 8.2.2 and 8.2.3 on all of them with the requests library.
silver tapir
bodge life ❤️
onyx hinge
@devout jolt ```py
import adafruit_ticks
from asyncio.core import SingletonGenerator
def deadline_ms(t, sgen=SingletonGenerator()):
"""Sleep until not ticks_less(t, ticks_ms())"""
assert sgen.state is None, "Check for a missing await in your code"
sgen.state = t
return sgen
import random
import time
import asyncio
async def periodic():
when = adafruit_ticks.ticks_ms()
i = 0
while True:
i += 1
await deadline_ms(when := adafruit_ticks.ticks_add(when, 500))
print(i, adafruit_ticks.ticks_ms())
await asyncio.core.sleep_ms(random.randint(10, 50))
asyncio.run(periodic())
On it.
midnight ember
yeah shoutout to JP for a really long day waking up early for everything.
onyx hinge
even though it waits a random amount from 10 to 50ms each time, it still starts at about the intended time, and it's not drifting compared to the ticks_ms clock.
(the random wait is to simulate doing a variable amount of work)
midnight ember
The pretty pins diagrams are so nice just to look at. Appropriately named.
The 3D parts for Melissa's message board were also noteworthy. Great job on the enclosure design.
onyx hinge
idle owl
The pretty pins diagrams are so nice just to look at. Appropriately named.
Sneak peek.
midnight ember
So pretty, well done.
idle owl
Also a sneak peek of the silk. The images in the shop are of the prototype. 😉
midnight ember
I do love the N16R8 variants. It's just the right amount for any requests projects I can throw at it.
devout jolt
For the MilkV boards, is there Arduino core yet? That could really help
silver tapir
Makes sense, I'll try to run blinka on top and report back. Might be tight, and I also think that they are using linux as an example of what it could run. I am trying to compare it with an m7, but using linux directly might be easier.
And sure, let me know anyone who might be interested in playing with such a hardware. Thanks for the notes jeppler, todbod and dan.
stuck elbow
iirc the issue was that the dac wasn't supported in the idf yet
midnight ember
Ohh DAC's would become available on most ESP boards possibly?
devout jolt
iirc the issue was that the dac wasn't supported in the idf yet
right. no DMA to DAC
silver tapir
For the MilkV boards, is there Arduino core yet? That could really help
I'll research that.
thorny jay
Do the library owner need to do something, like make a new release? Who/what does compile the new mpy version?
tulip sleet
Do the library owner need to do something, like make a new release? Who/what doe...
it's automatic when the bundles are rebuilt
midnight ember
Any kind of timeline left for 9? I still have a lot of requests libraries to update with settings.toml but currently can't do it with 8.2.3 until that bug gets fixed.
tulip sleet
Any kind of timeline left for 9? I still have a lot of requests libraries to up...
which bug?
midnight ember
Wait for 8.2.4?
tidal kiln
thanks! yep, please keep the lib bundle version -to- CP version number mapping nice and simple (even if redundant, like 7 and 8 are same etc)
idle owl
thanks! yep, please keep the lib bundle version -to- CP version number mapping n...
This will happen.
midnight ember
Thank you for hosting @lone axle with the numbers in the name.
thorny jay
Thanks
midnight ember
which bug?
It's started off as a matrix portal bug then turned into an adafruit requests bug.
idle owl
@lone axle I'm good whenever. I assume you want to get the post-meeting bits done?
tulip sleet
https://github.com/adafruit/circuitpython/issues/8294
I don't think you meant this one?
thorny jay
I have a question... I added Wii Drums support to https://github.com/jfurcean/CircuitPython_WiiChuck , it is accepted in main. But to be present in it's new version in the bundle, do I need to ask @gloomy shuttle to make a new release of the library?
lone axle
<@382939733107408897> I'm good whenever. I assume you want to get the post-meeti...
Yep, I'm going to do the wrap up things and grab a snack. I'll let you know once I'm done with that.
midnight ember
yeah fixed wrong one
midnight ember
yes ugh posted the wrong one again. it's 8299.
ohh you changed the name that's why i didn't recognize it. thank you for that. i was thinking of changing the name after it shifted to a broader issue.
tulip sleet
I think #8299 is some cert problem. Brent is looking at it. I have some testing in mind but may not have a chance to do it right away
I want to build a fresh set of certs and use those instead of the ones from NINA-FW 1.7.5.
midnight ember
it's a bug that potentially affects all S2/S3 boards running adafruit_requests on 8.2.2 and 8.2.3
buoyant lagoon
who might i talk to, for more information on how mp_stack_check() and allocate_stack() are meant to work?
midnight ember
i'm not sure it's a NINA issue unless NINA is pre-programmed on the non-coprocessor modules on feathers.
tulip sleet
it's a bug that potentially affects all S2/S3 boards running adafruit_requests o...
it means you can't connetc to certain sites, but I don't think it has anything to do with basic wifi functionality, so it shouldn't hold up settings.toml changes. 8.2.1 would work
tulip sleet
i'm not sure it's a NINA issue unless NINA is pre-programmed on the non-coproces...
we use the cert list from the NINA-FW build. We don't use the firmware. We use the roots.pem file that's in that repo
midnight ember
haven't tried with a board using a co-processor yet
tulip sleet
who might i talk to, for more information on how `mp_stack_check()` and `allocat...
Scott is best for that, but he is afk until Wednesday
buoyant lagoon
ah
midnight ember
sorry that's above my head. i'm just good at breaking things.
buoyant lagoon
i havent had a change to bisect yet, i justspammed printfs until i found the root cause
let me see what git history says for those 2 files...
midnight ember
yes bisected it to it beginning in 8.2.2
tulip sleet
8.2.2 is JUST cert changes
midnight ember
8.2.1 works fine but that's not going to help anyone updating to the latest version from circuitpython.org/downloads
tulip sleet
@buoyant lagoon check the stack-checking changes in this commit: https://github.com/adafruit/circuitpython/commit/66edcf5d034432519aac18690806adad2cacf2ac
deshipu narrowed down some other issues to this commit
too many @clever usernames
buoyant lagoon
dang, ~4 of em, no wonder i have 3 _'s on my name, lol
buoyant lagoon
<@204112578425782272> check the stack-checking changes in this commit: <https:/...
thats just a minor change, to query where the top&bottom are and set the globals
tulip sleet
discord hasn't offered you a unique username yet, I guess 🙂
buoyant lagoon
but the problem i ran into, is that allocate_stack() allocates a whole new stack, sets the new bounds, then doesnt use it
and because the stack isnt within those bounds, i'm using -167mb of the stack
underflow!
tulip sleet
but the problem i ran into, is that `allocate_stack()` allocates a whole new sta...
that could be a long-standing bug then
buoyant lagoon
i was also unable to disable stack protections
if i change MICROPY_STACK_CHECK in the config file, it fails to compile
something about a .x not existing
tulip sleet
@slender iron is the right person for that - the rest of us have little experience with the broadcom port
buoyant lagoon
the weird thing, is that the faulting code, is mostly shared, let me find the URL...
that looks like something that would run on many ports
gated only by port_has_fixed_stack()
atmel-samd, litex, nrf, rp2040, stm, all appear to have dynamic stacks
so i'm confused as to why only the broadcom port is breaking
still zephyr
I have a question... I added Wii Drums support to https://github.com/jfurcean/Ci...
Yes, Adabot will verify the tag generated in the new release and include it in the bundle automatically.
buoyant lagoon
is there better documentation on how the stack is meant to be handled in general, like what allocate_stack() should be doing, and how it switches stacks?
less in the port specific stuff, and more how to make/maintain a port
tulip sleet
no real doc, no, just the examples of each and every port
is port_has_fixed_stack true or false?
buoyant lagoon
https://github.com/adafruit/circuitpython/blob/main/ports/broadcom/supervisor/port.c#L126-L132
bool port_has_fixed_stack(void) {
#ifdef __aarch64__
return true;
#else
return false;
#endif
}
false on 32bit, true on 64bit
pi-zero is 32bit
changing it to always true breaks things differently
tulip sleet
maybe cpu_get_regs_and_sp() is returning an incorrect sp for some reason? I would just plow through the code, sorry.
buoyant lagoon
ports/broadcom/mphalport.c:mp_uint_t cpu_get_regs_and_sp(mp_uint_t *regs) {
oh, interesting
the broadcom port has its own implementation of that...
lone axle
Here is the notes document for next Monday’s CircuitPython Weekly meeting. It is at the normal time of 11am Pacific / 2pm Eastern here on Discord. Everyone is encouraged to attend! Please add your hug reports and status updates even if you’ll be attending the meeting - it’s super helpful! If you are unable to attend but would still like to include updates, feel free to include them in the notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868> https://docs.google.com/document/d/1j5Pv4_oJHyVMV9M9QQvvpCHj0E4yjOM3AaDDe58oDsU/edit?usp=sharing
buoyant lagoon
__asm__ ("mov %[out], sp" : [out] "=r" (sp));
buoyant lagoon
its just inline asm, fairly standard stuff, should just work
but this is only reading sp, it cant change the active stack
so i dont see how allocate_stack() could ever switch to it
tulip sleet
i don't think it is allocating a new stack to switch to, it is reserving a chunk of memory for the current stack, so that further allocations can be done below the "top" of the stack (going down)
no big switch to a new stack is going on. Stack just starts at a place in RAM and grows down, and we need other space too, so figure out how big the stack should be, set a limit on its size, and then use the rest of RAM below that for heap and other thigns
buoyant lagoon
ah, that would imply that in the "fixed" stack case, its just at a hard-coded location
but in the "dynamic" stack case, something in boot.S picked a value dynamically, and you have to figure out what, and mark it as used so the heap doesnt step on it
tulip sleet
yes, I think so
buoyant lagoon
so i was mis-understanding the code
tulip sleet
it is really "set limits on stack size"
buoyant lagoon
i'll take another stab at it, from that angle
lone axle
@idle owl okay, I am good to go any time.
buoyant lagoon
that implies that its just mis-detecting where the stack is
oh, but there is also a second kinda more major bug
it reserves some extra space for the exception stack
then recursively faults while throwing the exception
so it can never actually say it ran out of stack space
idle owl
<@330227457296957440> okay, I am good to go any time.
Ok, let's do this. Want to do it in the public channel? We can vid and screenshare from there. Jeff and Dan do it regularly for the CP core issue sweeps that way. No worries if you'd rather do DM. Figured I'd offer both.
lone axle
Ok, let's do this. Want to do it in the public channel? We can vid and screensha...
Yep, sounds good to me. I'll hop in Amelia.
buoyant lagoon
@tulip sleet oh, i just found comments, in exactly the place i was going to add some!
https://github.com/adafruit/circuitpython/blob/main/supervisor/port.h
// Get stack limit address
uint32_t *port_stack_get_limit(void);
// Get stack top address
uint32_t *port_stack_get_top(void);
// True if stack is not located inside heap (at the top)
bool port_has_fixed_stack(void);
// Get heap bottom address
uint32_t *port_heap_get_bottom(void);
// Get heap top address
uint32_t *port_heap_get_top(void);
Python docstrings and readthedocs question: where do I find documentation on how to do more complicated docstrings for Python methods/classes? I'd like to describe some function args in more detail and I'd like more structure than just paragraphs of text. When I try to do things described here: https://dev.to/zenulabidin/sphinx-docstring-best-practices-2fca, sphinx errors out
tulip sleet
Python docstrings and readthedocs question: where do I find documentation on how...
I usually just go straight to the sphinx documentation. sphinx is a slowly-moving target, and I find its syntax leaves something to be desired.
buoyant lagoon
and reading things closer, i can now see why aarch64 broadcom claims a "fixed" stack
the firmware loads the kernel8.img to 0x80000 (512kb) (i think, might be mis-remembering)
and rather then waste all the space below the kernel, circuitpython puts the stack in that hole!
but 32bit broadcom loads the kernel lower by default, so you instead put the stack at the top of ram
however, that is ignoring config.txt, and the fact that you can just load the kernel anywhere
with just 1 line in config.txt, i could have both 32bit and 64bit loading to 1mb (0x100000), and then just have a fixed stack below that, in every case
idle owl
Python docstrings and readthedocs question: where do I find documentation on how...
Ooooooooof. Welcome to Sphinx. 😕 If you're still super stuck after whatever you find, ping me. I might be able to help? Sphinx and I have fought many a time. I don't win every time. 🙄
devout jolt
I usually just go straight to the sphinx documentation. sphinx is a slowly-movin...
Thanks yeah I've been there too but have not had much luck. I'm just trying to add an unordered list to a function docstring
devout jolt
Here's an example:
def do_something(good_arg, better_arg):
"""Do something amazing with args
Args should be:
* pretty good
* or better
* but no bad args or excellent args
:param string good_arg: the gooder arg, may not be bad
:param string better_arg: the better arg, may not be excellent
"""
pass
(the "errors" would be if I didn't use the exact right amount of whitespace between lines or at the beginning of lines, so not important, just frustrating). In this example, I would like the bullet list to render as a visible bullet list but instead it renders as paragraphs. The HTML appears to be appropriately <ul><li><ul> but no bullets. If I try to add bullets, sphinx just removes them
tulip sleet
Here's an example:
```py
def do_something(good_arg, better_arg):
"""Do somet...
There are unordered lists (with -) in shared-bindings/digitalio/DigitalInOut.c and shared-bindings/_bleio/UUID.c. Maybe how they do indentation would help
devout jolt
weird so why does that render with bullet points but mine does not
but thank you, I'll investigate
idle owl
Whitespace.
devout jolt
ag;lgehg=afh3hgawheghag
idle owl
@devout jolt Sphinx is the worst about whitespace.
tulip sleet
or could be CSS when you're inside a Parameters list
idle owl
One space here, and suddenly it formats everything as code, or fails completely.
tulip sleet
in general parameters are not listed in an unordered list
idle owl
I spend more time dealing with sphinx than writing the code lol
I feel seen.
devout jolt
oh weird, for some reason the cookiecutter setup I got doesn't render bullets at all in the param lists: https://circuitpython-microosc.readthedocs.io/en/latest/api.html
tulip sleet
@onyx hinge I can do smoke tests on representative boards starting now, unless you have already done some
on all ports
buoyant lagoon
ah, found setup_mmu_flat_map(); in the port.c, that answers that question
but where is that defined?
ports/broadcom/peripherals/broadcom/mmu.c:STRICT_ALIGN void setup_mmu_flat_map(void) {
ah, git grep ignoring submodules as usual
and it works!
onyx hinge
@tulip sleet no I didn't get out any HW. I was working on asyncio. do you have a sec to consider https://gist.github.com/jepler/de50fe9c89fab489c9fe9a53a997c68b ?
I will push it to that PR branch if this looks like what we agreed on. I tested with main & with the 1.19 branch
and then I'll update the big PR too
tulip sleet
it looks good, exactly what we were thinking about. I just tested 4 boards, have 9 to go. It's very quick
onyx hinge
OK done
onyx hinge
I spend more time dealing with sphinx than writing the code lol
I saw your commit messages and I feel your pain
devout jolt
Ooooooooof. Welcome to Sphinx. 😕 If you're still super stuck after whatever you...
have you ever had readthedocs.org give a "no theme named 'sphinx_rtd_theme' found" error? I had to add "sphinx_rtd_theme" to my docs/requirements.txt in my latest library, which I've never done before and haven't seen anyone else do
onyx hinge
they have been making changes to their build process and requiring a lot more things to be explicit than before. so not that specifically but I guess I'm not surprised.
tulip sleet
did you pip3 install -r requirements-doc.txt
idle owl
have you ever had readthedocs.org give a "no theme named 'sphinx_rtd_theme' foun...
We use it. So, I always have it installed. 😕
onyx hinge
@devout jolt what were the socket constants that were needed for the UDP thing?
devout jolt
We use it. So, I always have it installed. 😕
sorry I mean when building the docs on readthedocs.org
idle owl
sorry I mean when building the docs on readthedocs.org
Oh hmm.
No, I haven't run into that.
devout jolt
<@352910176736772096> what were the socket constants that were needed for the UD...
from the code I'm currently dinking with
if impl == "circuitpython":
# these defines are not yet in CirPy socket, known to work for ESP32 native WiFI
IPPROTO_IP = 0 # super secret from @jepler
IP_MULTICAST_TTL = 5 # super secret from @jepler
else:
import socket
IPPROTO_IP = socket.IPPROTO_IP
IP_MULTICAST_TTL = socket.IP_MULTICAST_TTL
@devout jolt and you were using which micro?
buoyant lagoon
@tulip sleet and there is my initial fix, but i can see how its changing some files that people may not want to change...
tulip sleet
I will assign it to Scott to review
spice gate
hi, I've found a little compilation issue than arises when using the FAT fs with a block size other than 512 bytes: https://github.com/adafruit/circuitpython/pull/8304
buoyant lagoon
ive also only tested the build on a pi-zero, but i made changes that impact the pi3/pi4
tulip sleet
ive also only tested the build on a pi-zero, but i made changes that impact the ...
do you either or both to test?
buoyant lagoon
i do have one of everything in the entire model range
tulip sleet
hi, I've found a little compilation issue than arises when using the FAT fs with...
Thanks! - we will review
tulip sleet
i do have one of everything in the entire model range
we would certainly want to test on other affected hw, so go ahead now if you have time
buoyant lagoon
youve also given me an idea on a further change to the makefiles
devout jolt
<@352910176736772096> and you were using which micro?
ESP32-S2 and ESP32-S3
buoyant lagoon
its possible to make a single sd card image, that works on every pi model
shal i throw that into the same pr, or keep it for another pr?
tulip sleet
shal i throw that into the same pr, or keep it for another pr?
keep that separate, since it could hold up review or approval
buoyant lagoon
now i need to find a better uSD for testing...
buoyant lagoon
root@d235fe3c2fdb:/circuitpython/ports/broadcom# for b in raspberrypi_pi4b raspberrypi_zero2w raspberrypi_zero_w; do make BOARD=$b ; done
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
QSTR not updated
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
QSTR not updated
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
QSTR not updated
root@d235fe3c2fdb:/circuitpython/ports/broadcom# ls */*.zip -lh
-rw-r--r-- 1 root root 13M Aug 21 21:07 build-raspberrypi_pi4b/firmware.disk.img.zip
-rw-r--r-- 1 root root 13M Aug 21 21:08 build-raspberrypi_zero2w/firmware.disk.img.zip
-rw-r--r-- 1 root root 13M Aug 21 21:08 build-raspberrypi_zero_w/firmware.disk.img.zip
these are the 3 main ones i can test
tulip sleet
@onyx hinge one odd build thing I noticed while testing is that some of the artifacts have all languages and some only have english. I know microdev set it to build only en_us, but it doesn't seem to be consistent. Not sure if that is new or not. It might be uf2 vs not, i haven't investigated yet
buoyant lagoon
we would certainly want to test on other affected hw, so go ahead now if you hav...
i can confirm, the raspberrypi_zero_w works on the zero-w, usb-device, help() in the repl, and hdmi out
that image also boots on the pi-1b, and generates the same hdmi
but usb-device isnt possible (board is wired for host only), so thats as far as i can test that model
ive written a dwc2 HCD in another kernel, so pi1b usb-host could be a future possibility
then you just plug in a usb keyb and go to work!
raspberrypi_zero2w is next, thats aarch64, so its covering what i didnt test, but is largely the same
tulip sleet
we are doing work in other ports on usb host, as you have probably seen
buoyant lagoon
yeah
i briefly tested the pio-usb on a pico
still need to investigate the double-setup issue tannet had
onyx hinge
<@426200547024961539> one odd build thing I noticed while testing is that some o...
among the github artifacts? I don't see anything that would immediately explain that.
tulip sleet
yes, and it is not uf2 vs bin
maybe it is initial push vs subsequent, but not sure that makes sense
onyx hinge
if you look at the log of an affected board does it build the translations?
tulip sleet
haven't looked yet; doing the last board and then I'll look
all boards test fine except Spresense, but there is some version skew with the bootloader, so I'm not sure it has anything to do with this.
buoyant lagoon
Adafruit CircuitPython 8.2.0-46-g01595c900-dirty on 2023-08-21; Raspberry Pi Zero 2W with bcm2837
>>>
>>> help()
Welcome to Adafruit CircuitPython 8.2.0-46-g01595c900-dirty!
Visit circuitpython.org for more information.
To list built-in modules type `help("modules")`.
buoyant lagoon
@tulip sleet zero2w also works, so i expect pi3 to also work...
tulip sleet
if you look at the log of an affected board does it build the translations?
cp0_sapling_m0_revb built all langs, cp_sapling_m0_spiflash said "Skipping languages" in the Build board step, and built only English
buoyant lagoon
sorta, hdmi is blank
tulip sleet
as an example
onyx hinge
@tulip sleet there's some logic that leads to this: tools/build_release_files.py: print("Skipping languages")
try:
with open(
f"../ports/{board_info['port']}/{build_dir}/firmware.size.json", "r"
) as f:
firmware = json.load(f)
if firmware["used_flash"] + LANGUAGE_THRESHOLD < firmware["firmware_region"]:
print("Skipping languages")
break
I tried the other two panels I have and have the same issue.
@phantomsixthplayer Could you show a picture of your wiring from the Grand Central to the panel, and a bird's-eye view of the back so we can see all the inter-panel jumpers?
I have labelled the components that are connected when I recorded the video.
tulip sleet
oh, it skips big ones
onyx hinge
"if the build has plenty of room don't bother building other languages"
tulip sleet
ok never mind
buoyant lagoon
>>> ��Serial console setup
allocate_stack()
fixed stack
setting canary
allocate_stack() done
stack_init() done
Adafruit CircuitP
hangs mid-boot, but its not a supported model, so i can skip it for now
tulip sleet
i forgot about the conditionalizing about the builds, to catch overflows
buoyant lagoon
that just leaves the 4
tulip sleet
@onyx hinge Anyway, all boards are fine except spresense, so I'd say let's merge after any further updates
we would want the updated asyncio library submodule, that's it, I think
onyx hinge
OK! thanks so much for testing. sitting down and iterating through a bunch of boards is a discipline I just can't find on any given day.
buoyant lagoon
pi4 is failing
@tulip sleet pi4 boots and has a working repl on hw uart
but usb-device isnt working, dont think i did that
hdmi is also not working
tulip sleet
@onyx hinge - should we merge, or wait until some number of people try the artifacts?
idle owl
<@426200547024961539> - should we merge, or wait until some number of people try...
Meeeeeeerrrrrrrrrrrrge.
(I'm not the one you asked.)
tulip sleet
i understand your sentiments 🙂
idle owl
I mean, when it comes down to it, we're breaking things, and so on. We might as well get it out there and find out what else we broke.
Folks are far more likely to test main than a PR.
And it's already not even alpha. So we're not making any guarantees.
So, while I'm being silly about it, I do have reasons for the sentiment. 🙂
tulip sleet
@onyx hinge I'm happy to merge now - I agree a build in S3 is worth many in artifacts
onyx hinge
Merge!
wraith crow
The DV Base uf2 you posted does boot up with DVI output for me. My build machine is fighting me so I haven't yet checked out the failing builds.
tulip sleet
I don't see any video on my 1920x1200 monitor with DVI, but I've never tried this before
oh, now it's working, some old code.py was making it not work
thorny jay
That was fast, now I have sound and image. I plan to play more synthio with that setup as my DIY solution is a bit fragile (and loud) with my speaker attached.
Time to sleep.
manic glacierBOT
onyx hinge
my remark about other people testing meant I wanted it merged.
wraith crow
Did #7922 merge into a branch other than main and recently get moved? I could have sworn that I ran the DV base with a current build in late July, but #7922 was back in April. Just hoping for explanation other than losing my mind :/
Maybe I only attempted a Pico W build (which I didn't really expect to work anyway). I should probably try that built again now that this fix is known 😄
tulip sleet
Did #7922 merge into a branch other than main and recently get moved? I could ha...
#7922 got incorporated into 8.2.0, which was built from main. So I don't think you are losing your mind. 🙂
maybe you were testing with manually created displays?
wraith crow
Not sure, maybe I had some stale code, but probably just never built the standard Pico version since the question came up about the W. Good news though, with your resolution fix, the Pimoroni base does seem to build for the Pico W now. I'm going to do some more testing and hopefully submit the PR 😄
Now I just have to remember how to create a CP branch in my MicroPython fork.... :/
onyx hinge
@tulip sleet I'll look into it.
@tulip sleet it's because of this line I added, which makes it try to "call into" esp-idf build routines when parsing Makefile ... PR builds didn't catch it because a change to a platform Makefile doesn't cause docs to be built.
Makefile: $(BUILD)/esp-idf/config/sdkconfig.h
tulip sleet
<@329766224093249548> it's because of this line I added, which makes it try to "...
aha - yes I looked at olde builds, and never saw docs being built
ye olde builds ? 🙂
missing "r" there
onyx hinge
but of course this won't TEST doc building during the CI 😕 @tulip sleet if you tested doc building locally did you get the error?
tulip sleet
let me try again, I think it was fine
it was fine before i merged from upstream, I think I didn't have your change
fails in the same way make[1]: *** [Makefile:358: build-adafruit_feather_huzzah32/esp-idf/config/sdkconfig.h] Error 1 plus a lot of previous stuff
I think this is because esp-idf/export.sh was not run?
onyx hinge
it shouldn't have to be
tulip sleet
I need to clean this up a bit, brb
this multiple times:
raceback (most recent call last):
File "/home/halbert/.local/lib/python3.10/site-packages/sphinx/config.py", line 354, in eval_config_file
exec(code, namespace) # NoQA: S102
File "/home/halbert/repos/circuitpython/conf.py", line 54, in <module>
modules_support_matrix = shared_bindings_matrix.support_matrix_by_board()
File "/home/halbert/repos/circuitpython/docs/shared_bindings_matrix.py", line 378, in support_matrix_by_board
[board for matrix in mapped_exec for board in matrix], key=lambda x: x[0]
File "/home/halbert/repos/circuitpython/docs/shared_bindings_matrix.py", line 378, in <listcomp>
[board for matrix in mapped_exec for board in matrix], key=lambda x: x[0]
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
yield _result_or_cancel(fs.pop())
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
return fut.result(timeout)
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/halbert/repos/circuitpython/docs/shared_bindings_matrix.py", line 306, in support_matrix
settings = get_settings_from_makefile(str(port_dir), entry.name)
File "/home/halbert/repos/circuitpython/docs/shared_bindings_matrix.py", line 192, in get_settings_from_makefile
raise RuntimeError(error_msg)
RuntimeError: Invoking 'make -C /home/halbert/repos/circuitpython/ports/espressif -f Makefile BOARD=adafruit_feather_huzzah32 print-CFLAGS print-CIRCUITPY_BUILD_EXTENSIONS print-FROZEN_MPY_DIRS print-SRC_PATTERNS' exited with 2: make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
CMake Error at esp-idf/tools/cmake/project.cmake:296 (__project):
The CMAKE_C_COMPILER:
xtensa-esp32-elf-gcc
is not a full path and was not found in the PATH.
Tell CMake where to find the compiler by setting either the environment
variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
CMakeLists.txt:19 (project)
or it may depend on whether the esp-idf submodule is populated
tulip sleet
it is populated in my tree
but it shouldn't have to be, to just build the docs
now I did esp-idf/export.sh, and it is running a zillion cmakes
well, 12
now make
it did a whole espressif build in the middle of building the docs
onyx hinge
It does invoke make a bunch of times, to get make's final idea of what the makefile variables are
but it shouldn't "do" anything like generate targtets; my hasty change cuased it to do that
tulip sleet
it took a long time to do that
onyx hinge
take a look at the PR I opened
yeah it's not fast even when it's not messed up. unfortunately.
tulip sleet
why do we need that?
or more precisely, why did we not need it before?
why is it descending into ports/espressif/Makefile at all?
onyx hinge
we did need it before. make is our configuration language for boards, so the only ultimately accurate way to get information about which modules are enabled is to invoke make and inspect its output (things like make -C ports/... BOARD=... print-CFLAGS
tulip sleet
so did it used to do this before, but more quickly?
onyx hinge
what I broke is that with my change in ports/espressif/Makefile, make print-CFLAGS caused a lot more work to be done, including work that required esp-idf environment to be set, ran cmake, etc.
tulip sleet
ah, ok, so it should be back to how it wsa before?
onyx hinge
this e.g., time make -C ports/atmel-samd/ BOARD=trinket_m0 print-CFLAGS is about 150ms, but when you multiply that by all the boards it is a substantial amount of work
tulip sleet
i have an old repo of adafruit/circuitpython from before this merge, I just did a time make html. Will do the same in your PR barnch.
onyx hinge
OK
tulip sleet
no problem: just about a minute for either old or the PR. So I'll approve the PR
onyx hinge
also should monitor https://readthedocs.org/projects/circuitpython/builds/21678598/
which just failed 😦 😦 ```
[2Kreading sources... [ 1%] BUILDING
[2Kreading sources... [ 1%] CODE_OF_CONDUCT
[2Kreading sources... [ 2%] CONTRIBUTING
[2Kreading sources... [ 3%] README
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/circuitpython/envs/latest/lib/python3.11/site-packages/sphinx/events.py", line 96, in emit
results.append(listener.handler(self.app, *args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/circuitpython/checkouts/latest/docs/rstjinja.py", line 13, in rstjinja
if "shared-bindings/support_matrix" not in docname:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
The above exception was the direct cause of the following exception:
bleh
onyx hinge
OK I get that when upgrading to current sphinx. I previously had 6.2.1, current is 7.2.2.
tulip sleet
was it returning "" before? I guess it's an easy fix.
onyx hinge
I'm not 100% sure
will want to fix this in 8.2.x as well, maybe don't merge that? but I'm done for the night.
in fact let's not use that fix
tulip sleet
I was just thinking that
we have a pile of stuff to merge up from 8.2.x to main anyway
onyx hinge
there are some conflicts including in this file.
tulip sleet
tomw I think I will do a little more asyncio testing with 8.2.x and the updated library
onyx hinge
in 8.2.x I added the ability to tag a doc file content to say "I want jinja templating to run on this file". The "check for substring of path" can be removed once we add that mark to shared_bindings_matrix.rst or whatever it's called.
origin/8.2.x:shared-bindings/busio/__init__.c://| .. jinja
wraith crow
It looks to me like the micropython 1.19.1 merge may have applied some of the leading 'u' to module names. I don't seem to be able to import json anymore but ujson works.
manic glacierBOT
onyx hinge
@tulip sleet I notice that uncrustify.cfg changed, presmuably due to the 1.19 merge? ```diff
-sp_after_comma = ignore # ignore/add/remove/force
+sp_after_comma = add # ignore/add/remove/force
must be - I didn't change that. That would explain all those changes. I prefer the 'add'
onyx hinge
I do too
tulip sleet
I am doing a lot of stuff to fix #8314, removing u a bunch of places, including in ports/unix tests and stuff. Also removing unused tests like usocket, etc. to reduce the number of skipped tests
onyx hinge
OK, that sounds good.
tulip sleet
it's jumping the gun a bit on what will be in v1.21. I am not renaming the files to drop the u. The merge can do that.
onyx hinge
there's supposed to be code in the import path that allows import json to pick up ujson, but it must not be working in cp; but we also want help() and import <tab> to work
that works in micropython-coverage but I guess not in circuitpython per the user report?
True
I'm not sure where the extra module defs were, but just making (almost) everything not-u is working. There are testfile changes, but those will start matching micropython by v1.21
so coverage is not going to be special here either. We still need moduos for certain things
onyx hinge
what I'm saying is, in the testing environment import json works
so it's weird that it didn't work on devices
ah it's subject to #if MICROPY_MODULE_WEAK_LINKS
tulip sleet
aha, ok, well, I am removing import ujson as possible now, and changing the tests to match
onyx hinge
that's fine by me