sd cards are your best bet for gigabyte scale storage
#circuitpython-dev
slender iron
not sure exactly but I know it include bad block tracking
definitely not worth your time
slender iron
sd cards include microcontrollers that manage it all for you
you can find chips that act like sd cards but I don't think they are worth the cost over an sd card
slender iron
ya, but then they aren't removable
ornate breach
I’m not bothered by not being removable
I am making some gifts that will be used by my kids so removable isn’t super desirable
spare jacinth
Whatever the hardware underneath is, you could (no idea how) display a "fake" filesystem (like uf2 bootloaders do), such that writing actually writes to the real filesystem under the hood and "removing" does nothing. Maybe with some button on boot to disable this (cant remove) behaviour.
I would just use a SD card and edit its content by plugging on the computer tho
ornate breach
I’m actually wondering if given the right NAND flash that SDIO would work.. I kind of want to try now..
Lol.. I’ll let you know the results
slender iron
ya, if you can find an IC that acts like an SD card then it should work
no promises
ornate breach
My guess is the biggest thing would be needing continuous read support and being able load the entire contents of the flash.
I am sucking at English today, wow.
stuck elbow
ya, if you can find an IC that acts like an SD card then it should work
adafruit sells one, doesn't it?
ornate breach
Yeah, it’s one that’s not exactly readily available outside Adafruit’s module
thorny jay
The SD card does not have to be a physical SD card. https://www.adafruit.com/product/4899
ornate breach
I see the raw chip on lcsc or versions of it anyway
manic glacierBOT
slender iron
@candid sun want to video chat to go over HUSB review comments? I had a lot...
candid sun
@slender iron sure yeah
slender iron
I'll hop in the amelia voice/video chat channel
slender iron
@candid sun I just realized I sent you the wrong talk. Will find it after lunch
candid sun
<@347203168121126912> I just realized I sent you the wrong talk. Will find it af...
oh okay, thanks!
slender iron
manic glacierBOT
fringe trench
Is here more the right place to ask than #help-with-circuitpython ?
There's a core library - usb_midi - that isn't available on the Winterbloom Big Honking Button board. What would I have to do to make that library available on that board in a future CP release?
manic glacierBOT
tulip sleet
There's a core library - `usb_midi` - that isn't available on the Winterbloom Bi...
https://learn.adafruit.com/building-circuitpython/customizing-included-modulesT he Winterbloom manufacturer chose to turn it off. We would go by what they want. You could ask Thea about it.
manic glacierBOT
fringe trench
Thanks! I'll ping her on her discord when she's back online.
slender iron
@lone axle have you had issues with the .root_group changes? I'm seeing it ignore when assigning to root_group on a subclass of BusDisplay
lone axle
<@382939733107408897> have you had issues with the .root_group changes? I'm seei...
I haven't seen anything like that
slender iron
hrm
lone axle
What is an example of a BusDisplay subclass? I don't recall seeing that name before I don't think
slender iron
its the new name for Display
I think the MP merge may have broken it
(which is good)
needs to add a test
tests 8.2.6
yup, that works
(I was worried we'd have to add show() back)
wraith crow
I know there's going to be an effort to update the libraries replacing .show with .root_group once the 7.x libraries are frozen but if I run into individual cases while working with pre-release 9.x (main) should I just modify a local copy of the library or would it be helpful to also submit a PR?
tulip sleet
I know there's going to be an effort to update the libraries replacing .show wit...
feel free to submit a PR and we can hold off merging it until we're ready. We can do a survey of all the libraries by grepping over the Bundle repo, but all the changes will need PR's anyway. Thank you.
slender iron
I think we can remove show() from libraries now because the 7.x bundle is turned off
orchid basinBOT
main furnace
I'm using framebufferio.FramebufferDisplay. display.rotation=270 works, display.rotation=-90 does not. Seems like either should work.
stuck elbow
I'm using `framebufferio.FramebufferDisplay`. display.rotation=270 works, displa...
documentation only lists 270: https://docs.circuitpython.org/en/latest/shared-bindings/framebufferio/index.html#framebufferio.FramebufferDisplay
tulip sleet
I'm using `framebufferio.FramebufferDisplay`. display.rotation=270 works, displa...
rotation (int) – The rotation of the display in degrees clockwise. Must be in 90 degree increments (0, 90, 180, 270)
These actual values are checked for
@grand hamlet your latest push still is changing the submodule commits. You need to do make fetch-port-submodules in the ports/espressif directory
remerging does not automatically fix the commits
grand hamlet
Trying to sort things out a step at a time, got rid of the mpy-cross/mpy-cross
Just now.
I have done done make fetch-port-submodules before, I guess I need to do it again to get things properly up to date?
tulip sleet
right, or you probably could jsut do git submodule update extmod/ulab and the same for the other one, in the right directory
grand hamlet
You mentioned my ulab probobably being out of date, does that get sorted out with that? I was just looking into running updates through apt but not even sure what to update 😛
tulip sleet
ulab is just another submodule
grand hamlet
This has been such a disaster haha. Not helped by 2 random power outages and the VM randomly crashing. I'm very much out of my depth here.
Submodules are updating at least
What do I need to run after that because I'm very much lost
slender iron
git add the changed folder and commit them
grand hamlet
axeia@axeia-VirtualBox:~/repos/circuitpython-ESP32-S3-Zero/ports/espressif$ make fetch-port-submodules
wall of things completing
axeia@axeia-VirtualBox:~/repos/circuitpython-ESP32-S3-Zero/ports/espressif$ git status
On branch main
Your branch is up-to-date with 'origin/main'.
nothing to commit, working tree clean```
Going by git, there's nothing to commitTried running make BOARD=waveshare_esp32_s3_zero
But that's running into a problem.
LINK build/mpy-cross
text data bss dec hex filename
330068 14200 904 345172 54454 build/mpy-cross
make[1]: Leaving directory '/home/axeia/repos/circuitpython-ESP32-S3-Zero/mpy-cross'
In file included from ../../py/mpstate.h:35,
from ../../py/runtime.h:31,
from ../../extmod/ulab/code/ulab.c:17:
../../extmod/ulab/code/ulab.c:140:11: error: 'ndarray_properties_attr' undeclared here (not in a function)
140 | attr, ndarray_properties_attr,
| ^~~~~~~~~~~~~~~~~~~~~~~
../../py/obj.h:801:44: note: in definition of macro 'MP_DEFINE_CONST_OBJ_TYPE_EXPAND'
801 | #define MP_DEFINE_CONST_OBJ_TYPE_EXPAND(x) x
| ^
../../py/obj.h:805:179: note: in expansion of macro 'MP_DEFINE_CONST_OBJ_TYPE_NARGS_9'
805 | , _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24, _25, _26, _27, _28, N, ...) MP_DEFINE_CONST_OBJ_TYPE_NARGS_##N
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../extmod/ulab/code/ulab.c:129:1: note: in expansion of macro 'MP_DEFINE_CONST_OBJ_TYPE'
129 | MP_DEFINE_CONST_OBJ_TYPE(
| ^~~~~~~~~~~~~~~~~~~~~~~~
make: *** [../../py/mkrules.mk:84: build-waveshare_esp32_s3_zero/extmod/ulab/code/ulab.o] Error 1
You could save the changes you need out of the repo. Then maybe just delete that fork. Re-fork, get the latest main, git fetch-port-submodules and check out a branch for the changes. Check in yoru changes, and push a new PR. Close the old PR. That may be less work than trying to clean up your fork, since your main out of sync with ours anyway.
I was looking for your circuitpython repo but it's not a fork of adafruit/circuitpython. That was confusing 🙂
you may want to rename it to something else so you can then fork adafruit/circuitpython and keep the circuitpython name
main furnace
Where is the setter code for display.rotation? It should raise an exception for an invalid value.
grand hamlet
You could save the changes you need out of the repo. Then maybe just delete that...
Probably a better plan, thought it might be easier as well myself to just set the actual things that need changing aside and start afresh.
I'll give that a go tomorrow. Definitely turned out to be a lot more involved process than I thought it was.
tulip sleet
it's often easier to bake a new cake than cut the burned parts off the original one
grand hamlet
Not related but I saw in the documentation a little while ago that ESP32's should have enough memory to have 'dual screen' to be enabled by default.
Is that still something that's being considered?
I plan on making a keyboard with 2 screens and it would be nice if I could stick to the normal builds rather than having to roll my own and having to maintain the entire environment to be able to do so.
tulip sleet
Scott is working on something that will allow dynamic allocation of displays, if I understand correctly
devout jolt
Question for all the CircuitPython devs who know RP2040 and neopixel & adafruit_pixelbuf: Is there any reason why for neopixel.NeoPixel(some_pin, n=8) that "n=8" would be a "special" value that could cause weird issues? Over in help-with-circuitpython, user Muck is having this weird issue where on the PropMaker RP2040 when having i2s audio going, doing things like for i in range(8): pixels[i] = (255,0,0) goes all non-linear for them. But changing it to 12 or 16 and it works. Link to video of the issue as replicated by me. But it's really flaky when it happens: #help-with-circuitpython message
blissful pollen
Question for all the CircuitPython devs who know RP2040 and `neopixel` & `adafru...
The RP2040 neopixels are driven by the PIO and the problem seems to be a timing issue. Which something with I2S could be triggering. I have found sometimes adding "extra" non-existent pixels fixs a problem as it evens out timing sending data nothing interprets.
The green instead of red light is also a good indications of a timing issue. Not sure any more off the top of my head.
manic glacierBOT
tulip sleet
Interesting CircuitPython build result. I have a 6-core/12-thread Intel CPU, i7-8700. I have been running make -j12, assuming that the threading would buy me a little speed. In fact it's the opposite: -j6 is faster, by up to 10% or so (wall-clock time)
devout jolt
Interesting CircuitPython build result. I have a 6-core/12-thread Intel CPU, i7-...
Same. Not directly comparable, but I found a similar result a few months where on my M1 Mac getconf _NPROCESSORS_ONLN returns 10 but about half that was anecdotally faster at CirPy builds. Weird because I was taught to "+1" the nproc value because of I/O contention
tulip sleet
i was also using that getconf, and to get the actual number of cores I found this compact search on stack overflow: lscpu | awk -F ":" '/Core/ { c=$2; }; /Socket/ { print c*$2 }. There is not a comparable getconf or similar that I know of.
mortal kernel
Interesting CircuitPython build result. I have a 6-core/12-thread Intel CPU, i7-...
Building on an i7-10700 (8C/16T) running Ubuntu 22.04, scaling tracks up to 4T, then additional threads give diminished improvement.
tulip sleet
i would divide by 2 but it varies across CPU's
tulip sleet
Building on an i7-10700 (8C/16T) running Ubuntu 22.04, scaling tracks up to 4T, ...
10700 does not have "efficiency cores", is that right? So you are using -j12 or -j4? I could try -j7 and up
mortal kernel
10700 does not have "efficiency cores", is that right? So you are using -j12 or ...
Right. All cores are the same. Using -jxx to set number of threads. Terminology is confusing, in this case threads are Intel hyperthreads.
mortal kernel
What's your OS?
mortal kernel
Weird. You may be having heating issues that are causing your cores to slew down their clock dividers.
tulip sleet
fan is quiet. it's a Dell Optiplex small form factor
mortal kernel
I've got a heatsink the size of a brick on mine. And my fans spool up running the build. Maybe the stock Dell fan speed profile is skewed to quiet?
tulip sleet
-j6 33.3ecs, -j7 36.7 secs, -j5 32.8 secs, -j4 33.7 secs, -j3 43.0 secs
i can check the bios settings
i hear it spool up sometimes
metro m4 build, but it's not whining
the builds above are pico, not metro m4, so no LTO
mortal kernel
Your -j5 is just one second more than my best at -j8. This is a Pico W build here.
tulip sleet
it's an old machine, maybe I should redo the thermal paste (Dell Optiplex 7060 SFF)
I should rummage around for a temp monitor
in Linux
mortal kernel
Also consider disabling hyperthreading in your BIOS. You'll get more bang per core on this sort of workload.
tulip sleet
I am not sure how much is in parallel beyond the basic compilation of all the sources. The qstr, etc. stuff at the begininng is probably single thread, and so is the linking, I assume
mortal kernel
True, that's what I've observed. That's where no hyperthreading will double the single thread bits.
tulip sleet
using watch sensors, highest CPU temps I am seeing are 77C, typical is low seventies or lower
mortal kernel
You can also monitor clock slewing with watch -n1 "lscpu | grep MHz | awk '{print $1}'";
onyx hinge
There are some long sequential steps. The final link, even if it uses LTO, doesn't use a while lot of parallelism but can be the very longest running step...
Generating compressed error messages is also lengthy and not parallel
mortal kernel
New thermal paste time! Last time I re-pasted my rig the old paste had dried to a powder.
tulip sleet
I turned off hyperthreading; seems to make no difference. SpeedShift, SpeedStep, TurboBoost are all on. I will check the paste -- thanks!
blissful pollen
Interesting -j3 to -j6 (i5-9600) goes from 37 to 33 seconds. (-j1 is 1:45 or so). I see all CPU cores showing about 80% usage at 3 but max at 6
tulip sleet
Interesting -j3 to -j6 (i5-9600) goes from 37 to 33 seconds. (-j1 is 1:45 or so...
do you see improvements above -j6?
blissful pollen
Just tried and seems consistent. Going to try -j5 too
blissful pollen
-j4 on and higher seems to be consistent in build time, not up or down.
mortal kernel
I'm seeing the same as @Mark. I wonder if the top-level make is again invoking make with a smaller -jxx somewhere in the build?
tulip sleet
The jobserver stuff may not be set up right. As I mentioned in the merge thread, with gcc12 I am now seeing lto-wrapper: warning: jobserver is not available: '--jobserver-auth=' is not present in 'MAKEFLAGS' lto-wrapper: note: see the '-flto' option documentation for more information on the Metro M4 builds
I think just trying gcc12 on main would have the same effect. Merge build is branch v1.21-merge in dhalbert/circuitpython right now
mortal kernel
For example, see this rule inports/raspberrypi/Makefile:
$(Q)cmake -S pioasm -B $(BUILD)/pioasm
$(Q)$(MAKE) -C $(BUILD)/pioasm PioasmBuild
i was looking at this last night and saw mention of adding + prefix to the $(MAKE) lines.
i am unfamiliar with that and haven't had time to research it yet.
blissful pollen
I did notice all my cores would max out for the first 10-15%, then dip for a few seconds, then shoot back to 100% till the end when again a few more seconds of 50%. Just the two dips from maxing it all out though.
mortal kernel
On second take, make accounts for nested use of make automatically. Breaking it down, the bulk of the build time is spent constructing qstrdefs.generated.h.
FWIW, I'm thrilled having ~30 second full build times. Seems like just yesterday I was watching the dots while a small Pascal program compiled on my Apple II.
slender iron
I'd love to see us optimize across board builds to reduce our CI time
we should merge 8.2.x into main now
mortal kernel
we should merge 8.2.x into main now
Confirming what @onyx hinge said earlier, compressing error strings is a large part of the single-threaded portion of the build. Re-coding maketranslationdata.py and huffman in C could help. Huffman encoding is bit-string based, which is not one of Python's strengths.
onyx hinge
Confirming what <@426200547024961539> said earlier, compressing error strings is...
I'd rather preserve the flexibility, we keep refining just what maketranslationdata does. Huffman hasn't changed in ages though so a pip installable c version wouldn't hurt, even if it changes API
slender iron
I'd rather compress them all once and use them for all boards 🙂
mortal kernel
I'd rather preserve the flexibility, we keep refining just what maketranslationd...
Good point. It should be possible to pull huffman plus the generic encode/decode bits from maketranslationdata.py into a C library.
slender iron
compression depends on qstrs now and they change per-board for now
onyx hinge
and it depends on the subset of all compressed messages in a build as well
what we could do is have a "go faster" knob where all the fancy tricks aren't used for boards with plenty of flash space
like almost all rp2040 & espressif boards
slender iron
ya, it'd be good to factor all of that out
mortal kernel
I've got an experiment in mind that factors out the huffman library, decompress, and compress from maketranslationdata.py to a new C library. I'll report back and open a PR if it pans out.
slender iron
are you hung up on the pico w ble stuff?
mortal kernel
are you hung up on the pico w ble stuff?
Just need a pallet cleanser. You were right, it is complicated.
onyx hinge
I know the feeling, I fled the country after my last big project. 🌍
and this laptop has never compiled circuitpython before, wild
tranquil ether
I'd like some help with UART control of the Audio FX Sound Board (Product ID: 2217). I got it working with discrete pin triggering, but I would like to change to serial control over UART. I have found some example code, but for some reason it only does receive(https://learn.adafruit.com/circuitpython-essentials/circuitpython-uart-serial). I think a simple example of CircuitPython doing both Rx AND Tx would get me going. Can anyone think of some example code I can learn from?
mortal kernel
@onyx hinge , @slender iron Looks like the problem is not bit-string manipulation but rather the search for an optimal set of n-grams. Factoring out huffman encoding/decoding to C won't speed thing up as all of the overhead is in the while len(words) < max_words: loop.
slender iron
I'd like some help with UART control of the Audio FX Sound Board (Product ID: 22...
This is a question for #help-with-circuitpython
tulip sleet
@slender iron ok I got ports/unix coverage variant to compile, pushed all changes, didn't try any tests yet; will go eat
slender iron
👍 I'll try the tests
atomic summit
Hey folks - just finished work on my CP board support for my new BLING! and TinyWATCHS3 boards and they are not like my usual "development boards" - but instead have fixed functionality and no user exposed IO.
What I'm wanting to know is what the consensus is on including frozen libraries?
For instance, my TinyWATCHS3 uses an ST7789 screen, has a bunch of peripherals like RTC, Fuel Gauge, IMU, Magnetometer etc... these are fixed hardware. Should I provide support for these in a frozen form?
It's not like users would choose to "use or not use" the screen 😉
I know I can ship a bunch of libraries in \lib but that means if someone stomps their CP install, they no longer have them and then have to hunt them down themselves and add them back in etc.
tulip sleet
we originally started using frozen modules to save RAM space on chips with limited RAM. There are pluses and minuses: it's very convenient, but if there's a library change, then we need to release a new CircuitPython or else have people override the frozen module. The libraries had more churn early on; now the base libraries don't change all that much: new releases often have nothing to do with code: they are just build infrastructure changes.
So I'm saying it's really up to you. It's a use case question, not really a policyt question. There are manufacturers who like things frozen to make it easy to use out of the box, and there are others who are tweaking their custom libraries often and might want a new release.
you could always make custom releases: 8.2.7+c-um-tinywatchs3 or whatever
atomic summit
Right-o. I notice that not all libraries I'd want to include are available in the frozen folder anyway.
tulip sleet
we're happy to add more there; it's just more submodules
atomic summit
Yah, I don't want to have to maintain a seperate build or list of libraries/bundles etc and then have to worry about keeping them current etc.
tulip sleet
it's support one way or the other: either you maintain it in our repo, and tell people how to override if there are bugs, or you supply builds
atomic summit
Yeah, just what I need - extra tasks 🙂 hahaha
Ok, I might just include essential libs frozen - Display, Neopixels etc and then any "can work without ones" I'll have as seperate things I include and users would need to add back in.
tulip sleet
it's convenient for the onboard sensors too, if you think those libraries are stable
atomic summit
There are bunch of peripherals on these boards I've never used in CP befoe, so not even sure the libs are compatible or how well they work etc - or even if they are supported!
Maybe my best bet is to get the board definitions PR'd and then over time update them and freeze "known good libs".
I was trying to get these ready for 8.2.7 - woops! haha. I'll still add them for 8.x today and then start work on adding them to main.
tulip sleet
8.2.7 is almost certainly not the last 8.2.x release 🙂
atomic summit
8.2.7 is almost certainly not the last 8.2.x release 🙂
Yeah I figured it might not be, and I've done the work anyway, so might as well raise a PR as they are ready now.
tulip sleet
@atomic summit scott is doing a deep dive right now and looking to update your 8.2.x boards
to merge 8.2.x to main
atomic summit
Oh....
tulip sleet
i am not paying that close attention
grand hamlet
fingers crossed
I see there was some discussion on using more threads above. Coincidentally I tried using -j6 (AMD Ryzen 5 1600 AF under virtualbox) - I did see all 6 cores being pushed hard at some points
Annoyingly virtualbox refuses to install on my main rig, I'd be curious to see how much faster it goes on my Ryzen 7 5800X.
lone sandalBOT
grand hamlet
Does that mean it will get a page under https://circuitpython.org/board/waveshare_esp32_s3_zero/ whenever the next circuitpython version is released?
wraith crow
Does that mean it will get a page under https://circuitpython.org/board/waveshar...
If you haven't already, take a look at https://learn.adafruit.com/how-to-add-a-new-board-to-the-circuitpython-org-website
grand hamlet
Ah thanks, I'll check that out. Thought it would create some place holder by default which can be edited from there.
manic glacierBOT
grand hamlet
It doesn't really show which image it's not happy about. I guess it's unhappy about the 'original' being only 800x600?
or.. I completely overlooked one being in a completely wrong directory.
How the heck did it up end there
wraith crow
It looks to me like the Makerfabs TFT7 board has shared the I2S audio pins with the USB data pins (D-, D+). If I attempt to initialize audiobusio I get an I2S_BIT_CLOCK pin in use message and when I run the Arduino audio example, the USB serial connection repeatedly disconnects/reconnects while the audio is playing. Is there anyway to get Circuit Python to release the USB data pins?
manic glacierBOT
manic glacierBOT
atomic summit
Has anyone played with the latest macOS 14.1 beta to see if the USB sync issue is still there? I'm not signed up for the betas, and don't really have a "spare Mac" I'm able to risk betas on right now, but I'm curious to know if by some fluke the issue has been addressed?
Also, @slender iron I released this morning I accidentally PR'd my updated BlizzardS3 board files in main with my other 2 boards. I know you did the work already on it moving it from 8.2x to 9.x sdkconfig setup - sorry.
slender iron
Also, <@252717193496756235> I released this morning I accidentally PR'd my updat...
No worries! I think someone did try the Sonoma release candidate and still had the problem
atomic summit
No worries! I think someone did try the Sonoma release candidate and still had t...
bummer 😦
humble iris
Anybody available to verify a potential bug with me? It's in the RP2040 version of the PulseIn component on the PulseIO module.
humble iris
I can't download the build tools I need to build circuitpy locally (hotel wifi, 44 hour estimate). I need to change 2 files to fix this bug; can somebody help build a test branch for me?
wraith crow
I can't download the build tools I need to build circuitpy locally (hotel wifi, ...
Let me know what branch you're building from and where I can find the two files and I'll git it a shot
humble iris
I'm modifying the main branch, I can toss the two files up here if that's OK, or DM them to you.
ports/raspberrypi/common-hal/pulseio/ ( the files I'll give you ) for position.
humble iris
Let me know what branch you're building from and where I can find the two files ...
I suck at replying. >.<
wraith crow
ok, be a few minuts to get things going
humble iris
(Want to build against the latest version, not against the alpha)
thank you!
SO much!
wraith crow
what board are use using?
wraith crow
(Want to build against the latest version, not against the alpha)
:laugh: reply....
humble iris
:laugh: reply....
Oh, sorry - just the default raspberry pi pico (non-w)
humble iris
... I didn't change that. :/ I might have pulled files from the alpha. Let me make sure I have the latest 8.2.7 files and put my modifications onto that.
humble iris
Thank you so much for trying; I think it's ultimately going to need more work. I don't know if I can debug my problem well enough while I'm in a hotel... so... can I tip you or buy you a coffee for your help?
wraith crow
No worries, I was working on CP anyway so wasn't any bother at all. Good luck tracking the bugs down when you have real internet 😄
humble iris
No worries, I was working on CP anyway so wasn't any bother at all. Good luck tr...
Thanks. I think ultimately I'm going to have to make a custom version of the pulseio that won't get mainstreamed. It's not great.
(My solution is not great, that is)
humble iris
No worries, I was working on CP anyway so wasn't any bother at all. Good luck tr...
I found better internet. 😄 Managed to compile. For real, though, I want to thank you for your willingness to help. Anything I can do?
wraith crow
All good, thanks though, and congrats on finding the Internet 😄
lone axle
I am trying to test out the cleveland art project to see if I can replicate any of the issues that a user on the forums and github is having or see if I can find a way to help. In reading into the issue I saw that this message from someone whose seemingly associated with Cleveland art: https://github.com/ClevelandMuseumArt/openaccess/issues/8#issuecomment-1743348001 They are asking for the removal of the CMA logo from the learn guide: https://learn.adafruit.com/cleveland-museum-of-art-pyportal-frame. My assumption is they mean the one depicted on the pyportal display in the main photo on the overview page.
I'm not sure whether they realize that photo is used in the project that runs o n the pyportal also (the photo is of it running on a device). The CMA persom mentions removing it from the web page, but perhaps it is best to remove it from the project as well?
tulip sleet
I am trying to test out the cleveland art project to see if I can replicate any ...
There was an additional in-line image of the logo in the guide, which I removed a few days ago. Not sure if we need to remove it from the photo as well, but I could just substitute one of the other photos
I think I will move up the landscapce and remove it from lower down on the first page
lone axle
do you think it's okay to leave the logo image file in the project itself? It uses it basically as a sort of loading splash screen.
tulip sleet
maybe we should just change it to text
lone axle
Okay, yeah I like that idea. I was thinking to just credit them by name as well even though not explicitly required by cc0
tulip sleet
i removed the photo but if the code is going to change anyway that can be done at the same time.
humble iris
@tulip sleet Got your suggested changes, I’ll make those when I get a chance later today. Do you know the cycle penalty for disabling and reenabling interrupts?
tulip sleet
no, but I think it's pretty trivial. I'll look
lone axle
There are some other small improvements I think can be made in this project as well. The random image selection uses a hardcoded upper bound. But it appears there are more images in the collection now so we could increase it. Or perhaps read the value out of the JSON responses so it's "self updating"
humble iris
no, but I think it's pretty trivial. I'll look
Thanks. It definitely reads better and doesn’t need a comment. I DID test my solution though and it worked. I’m assuming this glitch might be present in other platforms, too.
tulip sleet
it boils down to a couple of single instructions; I think it's nothing to worry about.
if you could re-test that would be great, thanks
humble iris
Oh, nice. I can retest, yeah.
It’s probably better than the occasional branch penalty I was getting in my solution.
tulip sleet
other ones typically use PWM hardware, atmel-samd would be a typical one to look at
humble iris
Thanks for the help. My first real contribution upstream!
tulip sleet
we appreciate it!
tulip sleet
Thanks. It definitely reads better and doesn’t need a comment. I DID test my so...
atmel-samd guards already:
uint16_t common_hal_pulseio_pulsein_popleft(pulseio_pulsein_obj_t *self) {
if (self->len == 0) {
mp_raise_IndexError_varg(translate("pop from empty %q"), MP_QSTR_PulseIn);
}
if (self->errored_too_fast) {
self->errored_too_fast = 0;
mp_raise_RuntimeError(translate("Input taking too long"));
}
common_hal_mcu_disable_interrupts();
uint16_t value = self->buffer[self->start];
self->start = (self->start + 1) % self->maxlen;
self->len--;
common_hal_mcu_enable_interrupts();
return value;
}
I was just looking but I’m on my phone atm.
That’s where I was going to put my interrupt guards, at least, so that’s a nice sanity check
tulip sleet
<@&356864093652516868> Regular Monday meeting is in less than two hours, at the normal time of 11am Pacific / 2pm Eastern here on Discord. Add your hug reports and status updates if you haven't already.
https://docs.google.com/document/d/13dm8Dw4AJq_dwmmmlnnBj79qUaau48uXwYyJTO2_YMs/edit?usp=sharing
slender iron
I was just looking but I’m on my phone atm.
That’s where I was going to put my i...
You can use github actions to build for you too in a pinch. Just enable actions on your fork and push your changes
mortal kernel
@slender iron I've improved maketranslatedata.py so that build times are improved from 31.5 seconds to 20.2 seconds. I'm having a small loss of compression efficiency from 12873 to 12652 bytes saved (I'm 221 bytes worse). The loss is due to qstr compression, where some cases trade-off on the razor's edge.
slender iron
<@252717193496756235> I've improved `maketranslatedata.py` so that build times a...
what port's build time?
mortal kernel
The iterative dictionary word search is the heart of the problem. The regular expression with qstrs included becomes quickly becomes unwieldy.
slender iron
using a bit more space is fine as long as all of the boards still fit
mortal kernel
The port is raspberrypi
lone axle
@tulip sleet I think I recall recently you mentioning working on something related to SSL certs issued by Let's Encrypt. I'm starting to wonder if there is something similar in play for ESP32SPI or nina-fw? I was able to replicate reliably one of the errors that this user reported and simplify the reproducer down to a slightly modified wifi test script rather than the full art museam project.
I'm noticing that requests to https://openaccess-api.clevelandart.org/api/artworks?cc0=1&has_image=1&indent=2&limit=1&skip=9786 are always failing with TimeoutError: ESP32 not responding The requests to adafruit test file, and coindesk API both suceed as does an https version of the coindesk URL so I know it's not generally an https issue. Looking at the cert for that cleveland art API it does show that it's from Let's Encrypt.
mortal kernel
Also, considering all possible n-grams for scoring is very expensive. I've found a heuristic that trims the set under consideration.
lone axle
even if unrelated to the recent Let's Encrypt cert stuff it does seem that CircuitPython 8.2.7 with nina-fw 1.7.5 on the PyPortal Titano does reliably always fail the request to that art API.
slender iron
@lone axle my display split PR will break a library of yours I think because it removes displayio.Shape. https://github.com/adafruit/Adafruit_CircuitPython_DisplayIO_Layout/blob/main/adafruit_displayio_layout/layouts/grid_layout.py#L267
tulip sleet
even if unrelated to the recent Let's Encrypt cert stuff it does seem that Circu...
If it's Let's Encrypt with a particular cert chain, then it appears to be unsolvable on the AirLift (ESP32) NINA-FW
Could you give me a URL?
lone axle
Could you give me a URL?
tulip sleet
lone axle
going for a walk be back in a bit.
cunning sundial
I've come up with a new version of a board that I had working earlier (background: https://discord.com/channels/327254708534116352/1143997801274015774) The board is mostly the same, but attempting to use a W25Q32JVSSIQ memory chip rather than the W25Q128JVSIQ I was using before.
RPi shows up, allows me to flash it with firmware, then doesn't show up again unless I reconnect with the boot button depressed. So it SEEMS to be storing something. I changed the mpconfigboard.mk around as shown below and recompiled.
Is there something I am doing wrong here, and/or is there something off about how this particular memory module compiles. Or might something else not be correct?
What is interesting is that when I flash an older board with my new(?) firmware, it still says it has ~16MB of memory even though I put in the 32mbit ~ 4MB. Does this mean I'm not compiling it correctly?
USB_PID = 0x80F4
USB_PRODUCT = "Pico"
USB_MANUFACTURER = "Raspberry Pi"
CHIP_VARIANT = RP2040
CHIP_FAMILY = rp2
EXTERNAL_FLASH_DEVICES = "W25Q32JVxQ"
CIRCUITPY__EVE = 1
CIRCUITPY_PICODVI = 1What is interesting is that when I flash an older board with my new(?) firmware, it still says it has ~16MB of memory even though I put in the 32mbit ~ 4MB. Does this mean I'm not compiling it correctly?
You specified the new chip, so that's what it thinks is on the board.
RPi shows up, allows me to flash it with firmware, then doesn't show up again unless I do the reset button. So it SEEMS to be storing something. I changed the mpconfigboard.mk around as shown below and recompiled.
if, when you press reset again, RPI-RP2 shows up, that means the flash was unsuccessful.
Can you do anything else with the board, such as loading an Arduino Blink UF2 that you built with Arduino, or something with pico-sdk? I would be inclined to think there might be a hw problem with the board if that is unsuccessful. Since the bootloader is in ROM on the RP2040, that will show up even when the flash chip is messed up in some way
cunning sundial
> RPi shows up, allows me to flash it with firmware, then doesn't show up again ...
*Meant boot button (original edited)
slender iron
changing the flash device will also change the stage2 flash setup code run after the bootloader
cunning sundial
changing the flash device will also change the stage2 flash setup code run after...
So do you mean that I have done what I needed to here?
slender iron
that should be what you need to do for it to work
tulip sleet
any board that you flash with your W25Q32JVxQ version will think it has that flash chip on the board, even if it has something else
cunning sundial
any board that you flash with your W25Q32JVxQ version will think it has that fla...
OK, so if I flash a board with what I think is the W25Q32JVxQ and it says ~16MB of memory in Mac/Finder, then is it fair to say I haven't actually compiled the UF2 correctly?
For that matter, if I put a UF2 that specifies 16MB on a Flash chip with 4MB of memory, would Finder even recognize it?
tulip sleet
@slender iron is it using the generic ??
we are jsut going into a meeting, may not be that responsive
slender iron
I thought the rp2040 read the flash size from the chip
tulip sleet
that's what i meant, so maybe all I said is just wrong
slender iron
the stage2 will try to optimize for the specific chip though
cunning sundial
the stage2 will try to optimize for the specific chip though
So I can spec a generic flash chip instead of Wxxx? How would I state that?
slender iron
So I can spec a generic flash chip instead of Wxxx? How would I state that?
It'll do the most basic of multiple flash chips
most basic settings
I usually use a logic analyzer to debug flash chip issues
cunning sundial
Hmm, what I mean is if I want the RP2040 to take care of deciding on the flash chip, what would I put here instead of what I have: @slender iron
USB_PID = 0x80F4
USB_PRODUCT = "Pico"
USB_MANUFACTURER = "Raspberry Pi"
CHIP_VARIANT = RP2040
CHIP_FAMILY = rp2
EXTERNAL_FLASH_DEVICES = "W25Q32JVxQ"
CIRCUITPY__EVE = 1
CIRCUITPY_PICODVI = 1Do I need to change anything?
(If that is what you mean)
slender iron
you can comma separate the two
lone axle
Yep, it's the E1 cert, which uses ECDSA 😦
When I first started testing today I was getting successful responses from that API. My Nina-fw was on a rather old version though 1.2.2 I'm not sure if I ever updated it on this device. If I start working backwards in nina-fw versions and find the most recent one that allows it to succeed would it make sense to add alertbox or something to the learn guide stating which firmware version it's known to work with?
onyx hinge
py/obj.c: mp_warning(MP_WARN_CAT(BytesWarning), "Comparison between bytes and str");
apparently there is anmp_warningnow @slender iron
manic glacierBOT
onyx hinge
> py/obj.c: mp_warning(MP_WARN_CAT(BytesWarning), "Comparison between...
you could probably call it in the module's __init__ to warn on the import of an entirely deprecated module
slender iron
you could probably call it in the module's `__init__` to warn on the import of a...
Right now I'm mapping displayio.Display -> busdisplay.BusDisplay
manic glacierBOT
slender iron
just in the local dict
midnight ember
i will not be talking today had dental surgery, mouth full of gauze.
onyx hinge
yipe, feel better soon @midnight ember
midnight ember
oh i got some feel good meds, feeling really good 🙂
lone axle
gilded cradle
I believe they are still memory mapped, but the device name changed
was previously /dev/gpiomem and is now /dev/gpiomem4
onyx hinge
10.0 milestone I assume?
slender iron
yup, did I say label?
onyx hinge
I think you said "issue" but I don't remember now
I'm dropping off now so do read my status update 🙂
midnight ember
have a great vacation @onyx hinge
errant grail
midnight ember
CGrovers automatic sensor calibration code is really great.
Been seeing a lot of Sonoma related issues in discord, figured that was going to be a big issue.
FoamyGuys work on the template engine and stuff is great. Almost like a local host of AdafruitIO which has a lot of advantages. Very exciting to see that progress.
lone axle
@errant grail I think that everything in requirements.txt needs to be formatted as pypi library names. I'm not sure how circup translates those over to the format it needs for installing, but I think something similar would need to occur for the community bundle libraries in order to have them be counted as dependencies that get installed automatically.
midnight ember
Playground note on that project Liz? Sounds great.
candid sun
Playground note on that project Liz? Sounds great.
i plan to 😺
errant grail
<@339922377746022402> I think that everything in `requirements.txt` needs to be ...
Ah, that would make sense. Ironic that the second Community Bundle dependency in this case isn't on pypi.
midnight ember
With Pi 5 I imagine Melissa is going to be quite busy with Blinka. Good luck!
gilded cradle
Yeah, I have a pi 5 and was working on adding Blinka support over the weekend
midnight ember
A lot of people use Display_Shapes is that a different library?
lone axle
A lot of people use Display_Shapes is that a different library?
yeah that is different
displayio.Shape (the one that was removed) isn't used hardly anywhere. The one place mentioned so far was a place I chose to use it mostly because it wasn't used anywhere else 😅
wraith crow
I just found and used shape, I sure I can find another approach:)
lone axle
vectorio should work for the most basic use-cases.
wraith crow
Thx 😊
candid sun
+1 for vectorio
errant grail
Thank you!
candid sun
thanks for hosting Dan!
midnight ember
Thank you for hosting Dan. Lots of neat changes on the horizon.
gilded cradle
thanks
blissful pollen
Ha my last meeting ending and I joined just in time to hear "... join next week"
wraith crow
Trying to load as few libs as I can, I think I'm already displaying a bitmap so that should work
midnight ember
Can do a monitor capture instead of window capture to get around all of that.
lone axle
Thanks for hosting Dan, have a good week everyone!
midnight ember
I use hitclips, free video editing program. It's for windows though.
lone axle
I might go back to having discord open in a browser instance instead of the app. I believe my OBS still see's it that way without any issue.
slender iron
I do that for my streams but not the meeting
lone axle
Same, I used to do it for the meeting, but I started suspecting that having the browser instance and the app open was making the audio call get broken sometimes. If I can't figure out how to have it find the discord app I might switch over to the browser entirely for running the meeting.
lone axle
Ah, that would make sense. Ironic that the second Community Bundle dependency in...
It might be a good topic for a future "weeds section", it would be nice if there was something seperate from requirements.txt and pypi that circup could look for at least optionally. IIRC circup's usage of requirements.txt and pypi names was the thing behind our descision to start releasing all libraries on pypi. Before there was a time when libraries that aren't meant to work with Blinka weren't released on pypi. Maybe there could be an optional circup-requirements.txt or something that can hold the list of requirements that can be found specifically with circup without necessarily being released on pypi.
errant grail
It might be a good topic for a future "weeds section", it would be nice if there...
Historically, Community Bundle libraries weren't and currently aren't required to simultaneously exist on pypi -- and none of mine are there. In my case I'm probably being recalcitrant about having to open up another account just for publishing Community Bundle libraries.
lone axle
Historically, Community Bundle libraries weren't and currently aren't required t...
agree'd I think we should try to find a solution that allows it to remain optional for community bundle. Whatever that solution is though I think it will need to be seperate from requirements.txt because we don't want to break usage of pip install -r requirements.txt so we can't list things in there that can't be found on pypi.
slender iron
maybe something in pyproject.toml?
lone axle
I thought that file was also associated with pypi, though I'm not that familiar with it beyond having seen it in our libraries and knowing a few of the things that have popped up needing to be changed in there.
lone axle
it does look like our cookie-cutter creates it even if the user selects no for pypi so we could use it as a place to keep the circup-requirements.
slender iron
I thought that file was also associated with pypi, though I'm not that familiar ...
I think its meant to be a generic metadata file
lone axle
neat. I'm guessing it would be best to use some specific namespace for a block within the toml like [circup.requirements] or something to try to ensure that this value won't be recognized by any tool except circup. Does that sound like a reasonable way to add it? Id be willing to tinker with circup some to try to get it to read an alternative list of requirements sometime.
slender iron
https://peps.python.org/pep-0621/ that's my understanding of it
lone axle
Is there any place to find older releases of nina-fw than the ones listed on this page? https://github.com/adafruit/nina-fw/releases the earliest one here is 1.4.0 and even it has the same "ESP32 timed out" issue with these ECDSA style certs. My titano had 1.2.2 on it before I upgraded to day and I was getting successfull responses with that one.
tulip sleet
Is there any place to find older releases of nina-fw than the ones listed on thi...
I was getting successfull responses with that one.
Recently, or in the past? There is no regression that I know of that would cause
lone axle
>I was getting successfull responses with that one.
Recently, or in the past? T...
recently, I got successful responses today before upgrading the nina-fw.
I was at first unable to replicate the issue the user reporeted but eventually realized my nina-fw was a different version and updated it, now I do always see that timed out error.
tulip sleet
you can look at the 1.2.2 tag and see the sources. Building it is a bit of a pain. I am looking for a download. I don't think it's a root cert issue, but it might be an sdkconfig settings issue. I tried the ECDSA cert chain with the latest Arduino NINA-FW and it also didn't work, so whatever stopped working was a long time ago.
maybe you could try the arduino nina-fw 1.2.1 or 1.2.2, but I'm not sure that's going to work for other reasons. I don't remember why we forked it originally.
@lone axle ^^
lone axle
<@382939733107408897> ^^
Thank you ill poke around and try a few more things. Perhaps weird question, but is it possible to pull a .bin off of a co-processor, I have a few others that I think it's likely I never upgraded, maybe if I could grab it from one of them I could load it back to the titano esp32
tulip sleet
btw I have tried to insert a clip into the meeting video with both pitivi and flowblade, unsuccessfully
tulip sleet
Thank you ill poke around and try a few more things. Perhaps weird question, but...
yes, you should be able to load the passthrough uf2 and use esptool.py to save the firmware
i am going to stop wasting time on these video editors for now, since the video is not that important. I am missing about 11:30 of screenshots in the beginner
I can insert a static image as a clip, but in pitivi, it did not render for long enough, and in flowblade, it did not render the audio. I can fix this but it seems like a waste of time.
openshot was worse
lone axle
Sounds good to me, the video component for one meeting isn't worth too much extra time imo.
lone axle
you can look at the 1.2.2 tag and see the sources. Building it is a bit of a pai...
Strangely there are no 1.2.2 tags on either the adafruit or arduino repositories. Adafruit goes from 1.1.0 to 1.4.0 and arduino goes from 1.1.0 to 1.3.0. I am certain that my device was originally reporting Firmware vers. bytearray(b'1.2.2\x00') I still have the history in my terminal from this morning somehow.
tulip sleet
Strangely there are no 1.2.2 tags on either the adafruit or arduino repositories...
looks like it was not tagged, but there are version changes: https://github.com/adafruit/nina-fw/pull/1
lone axle
Nice, thank you! I'm going to work on something different for a bit. After a break from this i'll come back to it and try to get it building locally. I don't necessarily think pointing folks toward installing the very old ones is the best option either though.
It might be good to add a warning in learn to projects that are known to be affected by it.
In time I imagine esp32-s2 and newer will eventually find their way into more of the form factors and the projects could go back to working on the newer hardware. But on the original hardware it's looking very tricky to get it working.
tulip sleet
i will do a diff of that old version and the current one and see whether there's something obvious. Not at the moment, but soon.
tulip sleet
Here is the notes document for next Monday’s CircuitPython Weekly meeting. It is at the normal time of 11am Pacific / 2pm Eastern here on Discord. Everyone is encouraged to attend! Please add your hug reports and status updates even if you’ll be attending the meeting - it’s super helpful! If you are unable to attend but would still like to include updates, feel free to include them in the notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868>
https://docs.google.com/document/d/1Pz4-FdEH1Du5bWghjJ7dUheGoq61ijuFpowd5qFlIFI/edit?usp=sharing
lone sandalBOT
manic glacierBOT
mortal kernel
What is our most flash constrained port? Thinking it's atmel-samd.
devout jolt
What is our most flash constrained port? Thinking it's atmel-samd.
Samd21 yeah. And it’s not just flash constrained but also severely RAM constrained too
tulip sleet
What is our most flash constrained port? Thinking it's atmel-samd.
trinket m0, gemma m0, feather m0 basic: 256kB flash, no external flash chip ("express"), so 192kB for firmware + 64kB for CIRCUITPY
no longints, other missing features
feather m0 basic has more pins(and so pin names) than trinket or gemma so it's even tighter
manic glacierBOT
lone axle
i will do a diff of that old version and the current one and see whether there's...
I made it back to nina-fw 1.2.2 on my titano but I am stumped because now the requests to the art API are failing the same way as the newer nina-fw's did (ESP32 Timeout). I still have the logs from the successful requests yesterday morning and confirmed in them that my device was reporting 1.2.2 during the time it was succeeding.
I found that there may have been multiple versions technically that were all 1.2.2. This PR that you pointed me to: https://github.com/adafruit/nina-fw/pull/1 actually had accidentally included a bin file in the repo. I tested with that bin file and it does report 1.2.2 however it was failing the requests just as all the newer ones were.
The next PR after that removes the bin file from the repo. The following one https://github.com/adafruit/nina-fw/pull/3 adds wpa2 enterprise support but does not increase the version number. I think I was able to build successfully from the commit where #3 was merged and it does also report version 1.2.2 but does also fail the requests to those URLs
The next PR after that one is when the version is changed to 1.3.0, I haven't tried that one yet. I also haven't yet tried building from the commit where #1 was merged, perhaps the .bin file that was checked in was different somehow. in https://github.com/adafruit/nina-fw/pull/2 Jerry mentions that the accidentally added bin file was "only built by using make firmware" implying perhaps that the "real" builds are made some different way?
One thing I noticed when I built from the commit merging #3 was that after running make firmware there was a long list of prompts that wanted me to input answers for various bits of configuration. I tried to stick to the defaults, but I didnot understand much of anything that was asked 😅. Do you know if there might be specific choices needed for use with circuitpython or anything like that?
It'll be much later tonight or tomorrow before I can get back to it again. I still intend to round up my other airlift style devices to see if I can luck out on finding another one that succeeds the requests to those URLs.
tulip sleet
I made it back to nina-fw `1.2.2` on my titano but I am stumped because now the...
I wonder if the museum might have multiple servers, with different cert chains
lone axle
That could be a possibility but the timing of consistent failures for me starting after I changed firmware and having had no other successes since then would be terribly unlucky.
tulip sleet
I will see if I have an old one with 1.2.2, but I think I upgraded them all
lone axle
Unfortunately I was also not in a completely known state on my device at the time. I had been using it before this for testing the changes to ESP32SPI library. I am unsure whether at the time of my successes I was using the frozen libraries, or if I had them overwritten with ones I put in. And if they were overwritten I have no idea what specific versions I would have had at the time.
tulip sleet
That could be a possibility but the timing of consistent failures for me startin...
I have a PyPortal Titano that identifies as 1.2.2 !
mortal kernel
@slender iron Want to discuss 8514 or does the explanation I added make sense?
slender iron
<@252717193496756235> Want to discuss 8514 or does the explanation I added make ...
All good. Will merge now. Thanks!
manic glacierBOT
slender iron
any feedback on ```
Adafruit CircuitPython 9.0.0-alpha.1-99-gecaf9e6b14-dirty on 2023-10-24; Adafruit Feather ESP32-S3 TFT with ESP32S3
import board
board.DISPLAY.show(None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: .show(x) removed. Use .root_group = x
any rewording?
slender iron
@tulip sleet I think the _transmit fix will be blocked by the displayio split PR
because I can use some test stuff there to verify that _transmit works
tulip sleet
you mean wait on merging, or merge split first?
manic glacierBOT
slender iron
I have a fix without a new test
could PR that separately and add a test with the displayio PR
tulip sleet
i am starting to review the split pr
slender iron
👍
tulip sleet
@slender iron MP_TYPE_FLAG_HAS_SPECIAL_ACCESSORS is set if there are class properties?? (They didn't document the flag.)
tulip sleet
okee, i didn't add it anywhere, glad you got those. no examples of using that flag in MPy
slender iron
they don't use it on native class afaict
since they don't use properties in their native apis
@tulip sleet I've got the _transfer fix ready too
lone axle
any feedback on ```
Adafruit CircuitPython 9.0.0-alpha.1-99-gecaf9e6b14-dirty on...
This wording looks good to me.
tulip sleet
@slender iron
Also MP_TYPE_FLAG_HAS_SPECIAL_ACCESSORS is not documented in obj.h with the other flags. Could you add a comment line there (with CIRCUITPY-CHANGE?). Thanks.
could you add one comment line to obj.h to document that flag? Not vital.
also is the .show fix going to be outdated by this merge? Maybe you said that already above.
Never mind, I will add a comment and approve and merge.
Never mind even more. The set of comments there needs a lot more work.
slender iron
Thanks @tulip sleet I’ll do it tomorrow. Had to run off to get the kiddo
manic glacierBOT
stuck elbow
any rewording?
maybe "obsolete" instead of "removed"?
tulip sleet
maybe "obsolete" instead of "removed"?
it is now non-functional, so it's not just obsolete, it doesn't work. The error message is an attempt to reduce support questions when it doesn't work.
stuck elbow
right, but then you get questions of why it is removed and if there is a way to put it back and so on, while I think that at least in the computer jargon the word "obsolete" conveys the reason and the fact that it won't be coming back
tulip sleet
I was thinking of "obsolete" as "deprecated" (still working but going away)
digital ibex
Hey, still working on this idea: #circuitpython-dev message
I think it's best if the Minecraft scripting functionality is implemented in Blinka. However, in order to communicate between Blinka and Minecraft, we'd probably need another library (websockets and grpc are my top two choices) as a dependency of Blinka. Would the Blinka folks be opposed to adding to the package requirements? Is there a better solution? Thanks!
stuck elbow
websockets are really complex for a microcontroller
digital ibex
It won't be running on a microcontroller. It'll be blinka running on a PC
stuck elbow
then why do you need to add it to Blinka?
spare jacinth
im not quite sure what's the API/functionality Blinka adds, but as i already said back in the day, i feel like it wouldn't add any value
digital ibex
The hope is that you'll be able to use NeoPixels, pin io, etc in Minecraft via blinka
spare jacinth
just add some external program which takes python code (or perhaps some GUI in the game within the mod) + then interacts with the game
stuck elbow
or a library outside of blinka for talking to minecraft
spare jacinth
note: im guessing your goal is that some python code causes signals (redstone) to appear on the game, not like a lever activating a LED on your MCU and whatnot
digital ibex
or a library outside of blinka for talking to minecraft
That exists, but Blinka's api is very user friendly, and I think there's potential for its existing APIs to have virtual counterparts in Minecraft, such as temp sensors, LEDs, basic pin io, etc
spare jacinth
and afaik blinka adds support for hardware-specific things (eg I2C) to be run somewhere other than a MCU/dev-board...
which would probably be best(and easier) to handle on the mod itself .... ie Java, which doesnt quite fit Blinka (?)
you probably won't be using stuff like SPI or I2C in the game, to begin with, so may not even need a bunch of those things being provided by blinka
digital ibex
I'm not sure I understand what you mean. Here's an example of how it would work, just so we're all on the same page:
import board
import digitalio
import time
pin = digitalio.DigitalInOut(board.D0)
pin.direction = digitalio.Direction.OUTPUT
while True:
pin.value = 1
time.sleep(0.5)
pin.value = 0
time.sleep(0.5)
In-game, that would be a redstone lamp connected to a modded-in block, which would blink every half second.
Again, this might not be a plausible/useful idea, but I think it is because it provides a way to get used to CircuitPython syntax in a familiar way (minecraft) without needing actual hardware.
I think this could be great for young kids who already enjoy Minecraft and want to learn how to code
I2C could maybe be implemented. An example of this is an in-game temperature sensor block, and if you put it close to lava, the value goes up.
spare jacinth
May be worth looking into Jython or whatever it was named (Py interpreter written in Java), instead of using CPy/CircuitPy and having to come up with a way of communicating that code (C) with the game (Java)
digital ibex
I think I would just write a Minecraft mod in Java, and it would use gRPC or Websockets to talk to Blinka. That goes back to my original question if it would be OK to add a dependency. The user wouldn't need to interact with Java code.
spare jacinth
So then your java mod would be just a RPC used to place/destroy blocks, listening to an external program (written in Py? C?) which sends those events?
slender iron
@digital ibex I don't think you need to use blinka. instead you could create your own version of digitalio that has a DigitalInOut that does the calls into minecraft
spare jacinth
So then your java mod would be just a RPC used to place/destroy blocks, listenin...
How/where would the the user input their code? Custom block and mod talks to such program, which then causes some RPCs to trigger logic in the game?
Switching windows back and forth between game and some editor?
Also, all of this would mean the mod relies on another program to be running, which doesnt sound too good for me
I'd rather make Java handle everything without an external dependency
digital ibex
<@525432105224765494> I don't think you need to use blinka. instead you could cr...
My concern is that most of the work is already done with Blinka, and adding a backend that interacts with Minecraft is somewhat simple. I also want this to have support for complex I/O devices, such as NeoPixels, simple sensors, etc. I'd really hate to make what would essentially be a copy of Blinka with hardware functionality replaced with Minecraft functionality. Most of the code is already there.
digital ibex
How/where would the the user input their code? Custom block and mod talks to suc...
The user would just use a code editor (Mu, VSCode) to write code that controls Minecraft.
spare jacinth
I dont like the interface, i would aim for all to happen in-game
digital ibex
The problem with that is it would be harder to transition from an in-game editor to a real hardware device, when the purpose of this project has been served and it's been used to teach CircuitPython to the user. Also, writing an entire code editor in Minecraft from scratch would be pretty tough. I can imagine the process I have in mind being a little clunky, but I think it's worth not having to use an in-game code editor with limited functionality.
spare jacinth
I meant just a text input where mostly you can:
- See the code running
- Paste existing code to start running it
I wouldnt plan on making a full editor lol 😬
My question was more on how to feed the code to the game, not on how to edit it.
Your approach would be something like minecraft_py code.py as im understanding it. While i would opt for having a (basic) method of writing/reading code within the game
slender iron
My question was more on how to feed the code to the game, not on how to edit it....
why are you suggesting micha do a different project? it isn't helpful
spare jacinth
Im not saying "do this", im giving some ideas/opinions
slender iron
@digital ibex are you thinking the user code would run on a CP device or from with python on the desktop?
digital ibex
Desktop python
slender iron
and what does neopixel support mean in a minecraft context?
digital ibex
pixels.fill((255, 0, 0)) would make a red block appear in-game
There's some tricks you can use to get blocks of any RGB color
slender iron
I think you really do just want to reimplement things then
make a neopixel class that looks like it from the outside but does custom minecraft stuff under the hood
digital ibex
So this would be completely outside of Blinka, but with the exact same API syntax? I could see that
spare jacinth
regardless of what the "interface" ends up being, there has to be some basic building blocks like reading/writing digital/analog pins(redstone lines). i would start with that code as soon as posible as it may lead to some meaningful insight of what limitations you will face along the way
then, however that code is called shouldn't be a problem, that compatibility layer is what will mostly cause headaches
slender iron
So this would be completely outside of Blinka, but with the exact same API synta...
right. that's the beauty of python. code that uses DigitalInOut shouldn't care about the implementation
digital ibex
Ok, thanks! I'll try that.
slender iron
@tulip sleet I think ☝️ is the last thing for an alpha
tulip sleet
<@329766224093249548> I think ☝️ is the last thing for an alpha
I think I found some leftover debugging prints in the code. I marked that in the review.
mortal kernel
Wondering if there's a discount code for devs purchasing from Adafruit?
I need a couple of nrf and espressif boards for Bluetooth LE comparision testing
lone axle
I'm not sure I understand what you mean. Here's an example of how it would work,...
One extra complexity is that you'll somehow need to associate an x,y,z location in the game with your pin You might end up needing some property or init argument for that which differs from the Blinka / CircuitPython API since that concept of location doesn't exist in the same way for the physical pin in the real world.
Temperature sensor in the game might be hard, I don't know if the game tracks a heat value beyond the most basic "touch lava or fire -> take damage" . The game does track light level I believe, and there is already a light sensor in the game. I think it'd be fairly straight forward to rig up similar python code that treats it as an AnologIn that is connected to a photoresistor.
For DigitalInOut you can get away without needing a custom block created in your mod I think as well if you want to go that route. You can have it just spawn or not spawn a redstone torch at the specificed location based on the value of the pin.
digital shoreBOT
slender iron
I'm looking into the build failure now
qrio passed CI before 1.21 was merged and wasn't run again
manic glacierBOT
manic glacierBOT
manic glacierBOT
Well, turns out it does in fact seem to fix the issue. I probably should have tested it myself before posting. Sorry for that.
tulip sleet
@slender iron the latest rtd builds have been failing for several days, e.g. https://readthedocs.org/projects/circuitpython/builds/22353971/
i am working on 9.0.0-alpha.2 release notes
slender iron
Ari is home today so I’ll be off and on
tulip sleet
@slender iron I finished the 9.0.0-alpha.2 release notes. I think the RTD build should be fixed, but other than that, maybe ready to go. Did you look at RTD or should I do that?
slender iron
I haven’t had a chance yet. Going to the pharmacy. Will have time once he’s down for nap
tulip sleet
Error
Build output directory for format "pdf" contains multiple files and it is not currently supported. Please, remove all the files but the "pdf" you want to upload.
Maybe we should just drop the Support Matrix PDF
slender iron
> _Error
> Build output directory for format "pdf" contains multiple files and i...
ah bummer! ya, I'd drop the support matrix
want me to do it? ari just went down
tulip sleet
sure, you are familiar with the CI steps you just changed
anything else you want to get in for alpha.2?
slender iron
not really. the warning stuff can go in the next release
the docs build isn't really needed for the release either
(though I know the change to make)
tulip sleet
the docs build isn't really needed for the release either
I'd like to have to doc build work so people can see the displayio changes; I looked myself to write up the release notes, and that's when I noticed it wasn't building
but otherwise I think we have a good snapshot
onyx hinge
Thanks Dan for fixing that. I must have forgotten to set up pre-commit
slender iron
PR to disable the second pdf is in the queue
orchid basinBOT
manic glacierBOT
manic glacierBOT
digital shoreBOT
tulip sleet
@slender iron https://github.com/adafruit/circuitpython/pull/8519 was green before merging, but had a bunch of errors in the merge build. I'll take a look.
slender iron
I can look too. Just got to my desk
tulip sleet
ok sure, go ahead
slender iron
@tulip sleet my guess is that the PR was green based on a merge before my warnings PR went in
this is what the build queue is supposed to solve
tulip sleet
yah, I think so, so it should be easy to fix
does turning on the build queue change our workflow?
slender iron
I don't think so: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue
looks a bit complicated to setup
tulip sleet
just sent a PR to build 9.x bundles too
want me to fix the bundles thing (see email), or do you want to do it
slender iron
I can look
pallid creek
I might be able to help with the merge queue issues. My company's tools specifically help with high PR velocity repos. I got permission to use it for free on open source projects.
slender iron
what tool is it? I was thinking we'd enable the github merge queue to start
pallid creek
Yes, that's a good start.
I mainly work on the Check tools, which manage all of the linters and code prettiers, but I just learned about Merge from another engineer this week, and it sounds exactly like what would help your PR collisions.
manic glacierBOT
slender iron
build tools PR is green
tulip sleet
merged - i changed to 8.2.0 instead of 8.0.0 to get the paths right
tulip sleet
@mortal kernel - ugh, this is sort of worse, since now there's something hidden going on
mortal kernel
<@849815081251766292> - ugh, this is sort of worse, since now there's something ...
Do I know it. The manual remount works around the problem, so I think that's the best we can do until Apple finds and fixes the underlying problem. We just don't know why it works around it. File your bug reports early and often!
Any luck with 14.2 beta?
tulip sleet
Any luck with 14.2 beta?
I have not installed 14.2 beta yet, but @romkey said they were going to.
mortal kernel
@danh I'm checking into the >8M filesystem reported by @romkey on 8449 just now. If I can confirm that, I have an idea for a workaround. If we report a >8M filesystem, but mark the excess blocks as in use in the FAT without a corresponding directory entry, the OS will think it has a larger device but won't be able to access the non-existent blocks. If the OS does try to read those blocks as part of a caching scheme, return them as zeroes. Could this work?
tulip sleet
@danh I'm checking into the >8M filesystem reported by @romkey on 8449 just now....
See https://github.com/adafruit/circuitpython/issues/2751, including second post.
mortal kernel
See <https://github.com/adafruit/circuitpython/issues/2751>, including second po...
Exactly. Marking the excess clusters bad looks like it could work. Your comment that it was "not so simple" worries me. Are there complexities beyond filesystem formatting?
tulip sleet
Exactly. Marking the excess clusters `bad` looks like it could work. Your commen...
I started to work on it several years ago and I think I found I would have to do some restructuring in oofatfs. Maybe it was not so clear where I could initialize the FAT; I don't remember that well. Or maybe it was that normally FAT16 and up have two FAT tables for redundancy and it was not clear how to enable that.
i don't remember if oofatfs has the ability to have two FAT's. Also it would take up extra space in the 64kB CIRCUITPY's.
mortal kernel
I started to work on it several years ago and I think I found I would have to do...
FAT12 will handle up to 32M, so we can skip going to FAT16 with the redundant FAT's.
tulip sleet
FAT12 will handle up to 32M, so we can skip going to FAT16 with the redundant FA...
the original goal was to get to FAT16, because Android didn't/doesn't have support for FAT12. Also Windows had a bug of long write times for FAT12 (floppy legacy), which supposedly was fixed but I haven't checked that again. See https://github.com/adafruit/circuitpython/issues/111, which was a really old problem we had with Windows early in the life of CircuitPython.
oofatfs does support FAT16 and up, but maybe just with one FAT. So maybe the two FAT's are not necessary, but it seemed like some OS's wanted two FAT's. We'd need some experimentation and studying of oofatfs
this Sonoma problem is all too familiar, in some ways
pallid creek
this Sonoma problem is all too familiar, in some ways
Could Sonoma have gotten rid of some older filesystems in their tools? That seems very Apple.
tulip sleet
They broke FAT12 support in the past, and then fixed it again. It's not missing, but it doesn't work the same way.
mortal kernel
the original goal was to get to FAT16, because Android didn't/doesn't have suppo...
So, seems like a maybe. How about I study oofatfs for a bit and get back to you later?
tulip sleet
sounds great!
If you could make it fake FAT16, that might solve two problems. But also see if it reports a false size with the bad blocks. That will be a support issue.
mortal kernel
If you could make it fake FAT16, that might solve two problems. But also see if ...
Yes, it probably will report a larger size. Is that show-stopper for support?
tulip sleet
Previous Apple issues:
https://blog.adafruit.com/2022/10/31/uploading-uf2-files-with-macos-13-0-ventura-apple-microbit_edu-raspberry_pi-circuitpython/
https://blog.adafruit.com/2019/06/25/ios-13-beta-1-erases-files-off-drive-resets-ipad-fat12-issue-it-seems-apple/
https://blog.adafruit.com/2019/07/30/its-now-possible-to-open-edit-and-save-python-code-on-ios-for-embedded-electronics-plugged-in-circuitpython-usb-devices-apple-ios-circuitpython/
tulip sleet
Yes, it probably will report a larger size. Is that show-stopper for support?
if we could just get Apple to fix the problem, then we can live with the curren FAT12 support. ... Of course, what we really want is for Apple to support MTP...
tulip sleet
Yes, it probably will report a larger size. Is that show-stopper for support?
i think if say 75% of the FAT was bad blocks, would it show the remaining capacity properly, or does it subtract off the bad blocks from the capacity. It may depend on the OS.
i thinik I have seen capacity go down on hard drives that had a lot of bad blocks
mortal kernel
i thinik I have seen capacity go down on hard drives that had a lot of bad block...
I'll look for that when I get some oofatfs mods to test. What I think it will do is show the fake size as the total fs size but show available size with the "bad" clusters subtracted out.
crimson ferry
just copied the entire .py library bundle to a 16MB FAT16 CIRCUITPY (UnexpectedMaker FeatherS2) w/o mishap (macOS 14.1)
crimson ferry
Ditto 32MB FAT16 (espressif N32R8). 300+ library files copied to 8MB FAT16 (espressif N8R8) also look intact. I didn't reboot between or do any kind of manual mounting, just plugged in new devices.
mortal kernel
Ditto 32MB FAT16 (espressif N32R8). 300+ library files copied to 8MB FAT16 (esp...
Thank you for testing that. The cluster size will grow from 4K to just 8K on devices with small flash for a faked 8M filesystem.
crimson ferry
statvfs:
N32R8 (2048, 2048, 15176, 10309, 10309, 0, 0, 0, 0, 255)
UM16 (2048, 2048, 5969, 511, 511, 0, 0, 0, 0, 255)
N8R8 (1024, 1024, 5796, 728, 728, 0, 0, 0, 0, 255)
wasn't the 8MB expected to fail on the copy though? (or is that just on UF2)
I can't get an error at all now on macOS 14.1 ... 4MB FAT12 (QT Py S3 N4R2): copied (and edited) files appear OK ((1024, 1024, 2348, 2050, 2050, 0, 0, 0, 0, 255))
(I had copy issues on 14.0)
not at all convinced all is good, just can't get an obvious failure
mortal kernel
wasn't the 8MB expected to fail on the copy though? (or is that just on UF2)
Re-read @romkey 's entry in the issue and it looks like anything greater than 8M should be OK. Maybe/maybe not coincidentally that's the line where cluster size jumps from 8K to 16K. 16K happens to be the native page size for Apple silicon (M1/M2). My pet theory du jour is that the underlying Sonoma bug is in xnu's mmap.
crimson ferry
I thought the 8MB FAT16 and the 4MB FAT12 would fail, but not so far
mortal kernel
I thought the 8MB FAT16 and the 4MB FAT12 would fail, but not so far
That's what I think, too.
crimson ferry
oh, btw, did this on an Intel-based macOS14.1
(not ready to do other Macs yet, need them issue-free)
orchid basinBOT
mortal kernel
oh, btw, did this on an _Intel_-based macOS14.1
That's an interesting data point. I'm on an Apple Silicon machine at 14.1, but I don't have anything handy with a flash bigger than 4M. This bug gets uglier and uglier.
keen eagle
That's an interesting data point. I'm on an Apple Silicon machine at 14.1, but I...
We don't have any Intel machines to test on anymore or I'd be happy to
gusty plover
where do I start helping figure this out? https://github.com/adafruit/Adafruit_CircuitPython_BLE/issues/186
tulip sleet
where do I start helping figure this out? https://github.com/adafruit/Adafruit_C...
Looking at that issue ... have you upgraded the firmware to 1.7.4, and are you sure it's upgraded? Use the Internet test program in the guide to check the firmware version.
gusty plover
I am flashing NINA as we speak
gusty plover
1.7.5 still happening
mortal kernel
@danh Disappointed to report that faking out the filesystem will eat up a relatively large amount of flash on smaller devices. For example, on a ATSAMD21E18 64K is allocated for the filesystem. Ballooning that to 8M+64K increases the cluster size from 512 bytes to 2048 bytes. It also increases the FAT table size from 512 bytes to 6656 bytes decreasing the amount of available filesystem space to about 53K. Each file stored will cost ~896 bytes more.
onyx hinge
https://fosstodon.org/@brettcannon/111309638874068503 something to potentially investigate for circuitpython libs
mortal kernel
@danh If you saw my message from last night, I think I've got a workaround that doesn't eat up flash on flash-constrained device (ex., M0 trinket). Changing to FAT16 increases the FAT table entry size so that cluster size remains 512 bytes. This gives us 128 clusters on a 64K device. Only the first 512-byte block of the expanded FAT is meaningful, the remainder is filled with bad block codes, so it can be faked without consuming flash.
tulip sleet
@danh If you saw my message from last night, I think I've got a workaround that ...
Thank you for working on this!
gusty plover
BLE on Titano
mortal kernel
Thank you for working on this!
NP, I'm enjoying this. oofatfs gets a bit sloppy setting cluster sizes, i.e., it sets FAT16 cluster size using a FAT12 table that in turn sets the cluster size too large for FAT12. Also, it always allocates 512 root directory entries, eating up 16K on a device that can only hold a few fewer than 128 files.
Looks like a similar, but not identical oofatfs is in tinyusb that has the same issues. Any suggestions on how/if I should proceed? File pulls for tinyusb?
stuck elbow
@tulip sleet was there a change in keypad.Keys in 9.x? I'm getting AttributeError: 'Keys' object has no attribute 'events'...
>>> import board
>>> import keypad
>>> keys = keypad.Keys((board.X, board.DOWN, board.LEFT, board.RIGHT,
... board.UP, board.O), value_when_pressed=False, interval=0.01)
>>> keys.
deinit reset
>>> keys.events
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Keys' object has no attribute 'events'
>>> dir(keys)
['__enter__', '__exit__', 'deinit', 'reset']
hmm, I see I need CIRCUITPY_KEYPAD_KEYS = 1 now
but even that doesn't make it work
scratches his head
I can't get it to work, even if I put CIRCUITPY_KEYPAD_KEYS define in mpconfigboard.h
@onyx hinge I think it's your change at https://github.com/adafruit/circuitpython/commit/05cf5213e20568f269f4fab678b307b06bf97fe3 – any hints on how to re-enable keypad.Keys?
onyx hinge
<@426200547024961539> I think it's your change at https://github.com/adafruit/ci...
my money's on it being something weird with the latest merge, not the ability to disable individual classes within keypad, but I don't have any HW to test with rn
tulip sleet
<@329766224093249548> was there a change in `keypad.Keys` in 9.x? I'm getting `...
No deliberate change. Seems like a bug.
Ok I see previous dialogue. Will look at this but I’m in issue review mode at the moment
stuck elbow
sorry, no hurry, I was just surprised
stuck elbow
hmm, it looks like the attribute is there on the class, it just it doesn't work as a property on the instance
adding MP_TYPE_FLAG_HAS_SPECIAL_ACCESSORS to the class definition fixes it
random junco
Please don't crosspost - this is already in #help-with-circuitpython which is the right place. This channel is for development of CircuitPython itself
gusty plover
Please don't crosspost - this is already in <#537365702651150357> which is the r...
my apologies
