now RTD builds are doing this ...
#circuitpython-dev
onyx hinge
tulip sleet
by the time you return, we will be working on CircuitPython 9 🙃
@jaunty juniper RTD ^^
jaunty juniper
hmffff that's a super useful error message
Read the Docs build information
Build id: 16910203
Project: circuitpython
Version: latest
Commit: None
Date: 2022-05-14T12:49:43.558024Z
State: finished
Success: False
versus
Read the Docs build information
Build id: 16909970
Project: circuitpython
Version: latest
Commit: 9fb4582cc881d6b45e3eebfee64308549b64df84
Date: 2022-05-14T11:39:27.431669Z
State: finished
Success: False
I don't understand, no error message, it just stops at git clean -d -f -f
that's the example:
submodules:
include:
- one
- two
recursive: true
that's what we have:
submodules:
include:
- extmod/ulab
- frozen
so it might be because "frozen" is not actually a submodule
so that means we would have to list them all manually ? >_>
tulip sleet
so that means we would have to list them all manually ? >_>
Looks like you can use submodules.exclude to exclude the ones we don't want, like the broadcom firmware, but that is still tedious. can't use include and exclude at the same time
jaunty juniper
we can call a custom command too, looking at that
tulip sleet
can you use the cross-RTD links and not fetch the submodules? Are the links that will be created here in this RTD, or are they across to the libraries RTD?
jaunty juniper
RTD calls the matrix, to list submodules the matrix needs to know what is inside the frozen modules directories. For a directory called Adafruit_CircuitPython_CircuitPlayground, the RTD is called circuitplayground (how do we know that ? is it always the same format ?), but we want adafruit_circuitplayground in the list. I don't want to do string transformations (like replace("_circuitpython_","_")) and some are not on RTD
tulip sleet
I think the matrix script looks either for a directory (that is not examples/, etc.), or it looks for the top level .py file?
no string transforms are necessary. It determines the import name from the submodule code
jaunty juniper
it takes every .py and every directory that contains py files that is not in the exclude list
but that requires checking out the frozen directory
tulip sleet
same algo is used for freezing
jaunty juniper
yep
tulip sleet
that is the original motivation
so if you could do a custom command that just fetches submodules under a particular dir? or could get the info from .gitmodules
jaunty juniper
build:
os: ubuntu-20.04
tools:
python: "3"
jobs:
post_install:
- git submodule update --init --force frozen
that is simple
jaunty juniper
that's also what I expected the submodules rule to do, but it doesn't do that, nor accept globs
tulip sleet
there's probably an issue for that 🙂
jaunty juniper
now I have a new error that seems unrelated and might just be a difference in configuration between my test repo and the core https://readthedocs.org/projects/circuitpython-neradoc/builds/16910376/
tulip sleet
trying to parse the version name?
jaunty juniper
oh right !
tulip sleet
i had that issue before in some way
jaunty juniper
it passes with a tag 👍
shall I test test that ? (it takes the github ref, but it's for a fetch)
jobs:
post_install:
- python tools/ci_fetch_deps.py docs main
i'm not sure what you mean
jaunty juniper
reuse the ci script that fetches the submodules, the same one that is used for the docs test action
instead of having to sync the rtd conf with it
tulip sleet
sure, that seems straightforward, and would pick up any special casing it needs to do in the future
will be afk for a while
tulip sleet
@jaunty juniper When I merge #6400, it should build the docs, I think. It skipped the doc build in the PR build
I am not sure there is a reason to bother to let the current built go to completion, but I will, just in case.
tulip sleet
@jaunty juniper
jaunty juniper
that's a relief
lone axle
Where is the code that is responsible for the tab completion prompts in REPL? Would it be possible to add 1) the files in the root of CIRCUITPY and 2) the files / directories inside of CIRCUITPY/lib/ That way it could auto-complete import [beginning of filename][Tab]
Looks like here maybe: https://github.com/adafruit/circuitpython/blob/main/py/repl.c
lone axle
Oh, nice. Thank you
jaunty juniper
didn't find it first, found the MP PR to backport the feature (which I remembered happened), the poster's name, then searched for them in the CP repo 🔎 🕵️
tulip sleet
proven garnet
@idle owl, ran the .pre-commit-config.yaml patch on a few of the libraries and it looks excellent so far so I'm going to let it run
idle owl
<@330227457296957440>, ran the `.pre-commit-config.yaml` patch on a few of the l...
Sounds good!
proven garnet
Is there something speical about the floppy library that it's .pre-commit-config.yaml is different?
Or is it just based on an older? That's the only I don't think the patch managed to get
Otherwise, the patch is complete! The few I've checked don't seem to have any CI issues, but happy to fix any I botched 
tulip sleet
floppy was added just recently, so it's probably not up to date
proven garnet
I can patch it today then! I'll also PR the cookiecutter as well so future repos will be up to date as well.
proven garnet
Cookiecutter is patched with the same changes!
lone sandalBOT
manic glacierBOT
proven garnet
One thing I noticed during the patch is that the floppy library isn't available to CircuitPython librarians, should it be or is it still WIP? I don't think I had write access.
tulip sleet
One thing I noticed during the patch is that the floppy library isn't available ...
Fixed- CircuitPython Librarians only had read access.
proven garnet
Thanks!
proven garnet
@idle owl I reverted part of the cookiecutter patch but tagged you in it.
tidal kiln
@tulip sleet replacement did same thing?
https://forums.adafruit.com/viewtopic.php?f=60&t=190755
tulip sleet
<@329766224093249548> replacement did same thing?
<https://forums.adafruit.com/v...
Do you have one to try 7.2.5 on?
tidal kiln
a m4sk? yes.
UF2 Bootloader v1.23.1-adafruit.1-210-g795abd2 SFHWRO
Model: Hallowing Mask M4
Board-ID: SAMD51G19A-Mask-v0
current bootloader. leave that? or update?
tulip sleet
```
UF2 Bootloader v1.23.1-adafruit.1-210-g795abd2 SFHWRO
Model: Hallowing Mask ...
i am somewhat baffled. I would ask for one back, but maybe they're in Europe?
Try it with the old bootloader, then try it again after updating the bootloader
tidal kiln
Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit Monster M4SK with samd51j19
>>> import os
>>> os.listdir()
['.fseventsd', '.metadata_never_index', '.Trashes', 'lib', 'boot_out.txt', 'System Volume Information', 'config.eye', 'hazel', 'splash.bmp']
>>>
want me to erase the FS somehow? it's got stuff from whatever eye thing was last being tested on it.
tulip sleet
```python
Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit Monster M4SK with...
yah sure, to get initial conditions
tidal kiln
use the REPL erase_filesystem() method?
tulip sleet
that's fine
tidal kiln
Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit Monster M4SK with samd51j19
>>> import os
>>> os.listdir()
['.fseventsd', '.metadata_never_index', '.Trashes', 'code.py', 'lib', 'boot_out.txt']
>>>
seems fine? that's with older bootloader.
next step is try bootloader update?
tulip sleet
yes, as the poster did, same version
tidal kiln
UF2 Bootloader v3.14.0 SFHWRO
Model: Hallowing Mask M4
Board-ID: SAMD51G19A-Mask-v0
Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit Monster M4SK with samd51j19
>>> import os
>>> os.listdir()
['.fseventsd', '.metadata_never_index', '.Trashes', 'code.py', 'lib', 'boot_out.txt']
>>>
works
in general, it seems like something they are doing. very doubtful a hardware failure would happen on two copies, and also after initially working fine upon arrival.
but no idea what they might be doing. their description all seems fine.
any chance it's some as-yet-unknown OS or 3rd party software thing?
tulip sleet
it seems unlikely. The "You are in safe mode because:
CIRCUITPY drive could not be found or created." messages really has nothing to do with USB.
tidal kiln
marginal power?
tulip sleet
That is an indicator the startup code could not mount or reformat the CIRCUITPY drive. ... maybe ... I have a couple of Macs I could try this on after the meetings
if they could try both boards on some other computer... I'm not sure if they said they had access to non-Macs or not
we have an internal CPy meeting in a few minutes, then the regular weekly meeting
lone axle
<@&356864093652516868> The weekly meeting will occur here on Discord in about 1 hour. You can fill in notes ahead of time in the shared document: https://docs.google.com/document/d/1fc8UvMSBTq0x2Sh94_R6pSRm1v81sZCWA4L7TwsFzfg/edit?usp=sharing Looking forward to seeing you there.
tulip sleet
marginal power?
also that thread contains a post from lm2425 who says they have the same trouble, also purchased from Pi Hut, so I am wondering about a bad batch
worth following up with them as well
maybe we can get an Arduino program to identify the flash chip that's on the board, see if it's the right one (and if it responds in Arduino)
manic glacierBOT
tidal kiln
lm245's thread is here:
https://forums.adafruit.com/viewtopic.php?f=60&t=190755
claiming same thing happens on mac and win...but win8 😦
jaunty juniper
I don't have issues with my monster m4sk on my intel mac mini
updated the bootloader with the UF2, installed 7.2.5
tulip sleet
i'm still leaning toward hw issue
tidal kiln
i'm still leaning toward hw issue
ok. yah. guess still possible. i think worth raising issue. ill do that and cc you.
@tulip sleet the test UF2 does show some SPI flash info
tulip sleet
i'll check that id # later
do you see that same ID wit hthe test program? That's the ID from the user photo
slender iron
is there a picture of the chip?
tidal kiln
is there a picture of the chip?
nope. just front of board.
tidal kiln
do you see that same ID wit hthe test program? That's the ID from the user photo
i get 0xC84017
proven garnet
@lone axle I'm adding the EMC2101 thing to In The Weeds under my name but feel free to edit/add to it.
lone axle
Thank you.
idle owl
<@330227457296957440> I reverted part of the cookiecutter patch but tagged you i...
Ok, got it. Good catch.
lone axle
@tulip sleet, @slender iron Is one of you up for reading the core status section during the meeting today?
slender iron
I'm around and happy to
proven garnet
Before I go down the rabbit hole - decorators also work for classes in CPY, right?
I've seen them for functions and methods but can't recall for classes
tulip sleet
@lone axle cater JEDEC ID is two nibbles mfr id, two nibbles chip id, one nibble type, and one nibble capacity, so the last 7 says it's the right capacity
jaunty juniper
it looks like they have a different chip than the one CP is configured for
lone axle
Before I go down the rabbit hole - decorators also work for classes in CPY, righ...
Like @ at sign decorator? I think I've only ever seen them used with functions. Mostly @property is the one I've seen.
tulip sleet
it looks like they have a different chip than the one CP is configured for
matrixportal include support for all these:
EXTERNAL_FLASH_DEVICES = "S25FL116K, S25FL216K, GD25Q16C"
Yeah exactly, I guess I can test it out real fast here
jaunty juniper
matrixportal include support for all these:
```
EXTERNAL_FLASH_DEVICES = "S25FL1...
Monster Mask only has GD25Q64C, EF4017 is W25Q64FVSIG
tulip sleet
oops i was looking at matrixportal -- I'm mixing up issues
EXTERNAL_FLASH_DEVICES = GD25Q64C
I can post a build with EXTERNAL_FLASH_DEVICES = "GD25Q64C,W25Q64FV"
tulip sleet
ok that's not good. @tidal kiln I will make a build supporting the other chip as well, and we can have them try it
we crossed, sure, go ahead, post it in that thread
jaunty juniper
I think that's the right value
tulip sleet
should be derivable from https://github.com/adafruit/nvm.toml
proven garnet
Like @ at sign decorator? I think I've only ever seen them used with functions. ...
It does work with classes, cool!
def add_wacky(cls):
def wrap_add_wacky(cls):
cls.wacky = lambda x: print("Wacky!")
return cls
return wrap_add_wacky(cls)
@add_wacky
class TestClass:
def __init__(self, a):
self.a = a
inst = TestClass("haha")
inst.wacky()
A totally useful example of course
tulip sleet
@jaunty juniper it might be W25Q64JVxQ instead, that's much more common with our boards
idle owl
@lone axle We'll be right there. Finishing up a meeting.
jaunty juniper
<@199811382793863168> it might be `W25Q64JVxQ` instead, that's much more common ...
I'll put a few, and ask for a picture of the chip too
random junco
PyCon US 2022 Highlights: https://ehmatthes.com/blog/pycon_2022_highlights/
Micro:bit Educational Foundation feedback: https://form.jotform.com/221313264750346
Pico Dual Circle Displays: https://twitter.com/todbot/status/1524482667919552513
Smart Cooler monitor using CPX and CircuitPython: https://blog.adafruit.com/2022/05/16/smart-cooler-monitor-using-circuit-playground-express/
@timid lintel Very cool, nice work!
blissful pollen
You can read it, sorry only can half listen at the moment
turbid radish
Yay Liz & Paul!
errant grail
Good luck pursuing the slightly revised new chapter @solar whale. Have always enjoyed your expertise, insights, and humor. Great laugh, BTW.
tidal kiln
@solar whale bummer. you're a great help and contributor here. will be missed. enjoy the change of focus. thanks for all the fish!
solar whale
Good luck pursuing the slightly revised new chapter <@343570430042308609>. Have ...
Thanks so much!
solar whale
<@343570430042308609> bummer. you're a great help and contributor here. will be ...
Thanks!
slender iron
oh, and if you know of a remote file editing protocol that is used in phone apps then let me know. maybe we want circuitpython to do the same
idle owl
This came up during the Sprints as well.
proven garnet
I think the only thing @tulip sleet had mentioned in one of my PRs was grouping doc things in the core if possible to save CI runs.
I know use of Protocol is also pinned to 3.7
proven garnet
Oh I didn't realize, cool!
tulip sleet
i think!
marble hornet
example of the 30 files https://github.com/adafruit/circuitpython/compare/main...TG-Techie:maybe_precommit_error
proven garnet
Nope, you're right Dan!
idle owl
pre-commit also works to run it manually. We stumbled onto that during the Sprints.
lone axle
Some protocols live in here: https://github.com/adafruit/Adafruit_CircuitPython_Typing
proven garnet
It's the weird reason you can have failed import and still not fail on type annotations for CircuitPython, that took me a minute to wrap my head around haha
lone axle
This page ends up being functional pretty similar to the "category" page it seems: https://learn.adafruit.com/products/4116/guides
timid lintel
imho it's important it shows up in all existing PyPortal learn guides
tulip sleet
@marble hornet hmm, it might be true that MicroPython does not support annotations with square brackets. Would need to test. We do use Protocols, but as subclasses, e.g. in adafruit_requests:
lass CommonCircuitPythonSocketType(CommonSocketType, Protocol):
thanks @idle owl that would be great
tulip sleet
@marble hornet: this works:
>>> def f(x: Protocol):
... pass
...
...
...
>>> def f(x: Protocol[True]):
... pass
in CircuitPython
even if it didn't, you could always use string annotations, since that defers evaluation of what is inside the tstring
proven garnet
I think that's it!
tulip sleet
the from __future__ import annotations I added does not actually do anything at all; it is a no-op. I simply made it possible to add that statement without it causing an error
marble hornet
I don't think string annotations work in class base lists?
class Foo(Protocol[T]):pass # doesn't work on circuitpython
# or
class Bar("Protocol[T]"):pass # Errors on cpython and CP
# etc
It's more of a warning if anything as well, for some of them
tulip sleet
I don't think string annotations work in class base lists?
```python
class Foo(P...
i see, that was not a case I thought of
proven garnet
Thanks y'all!
marble hornet
If we want to support that syntax, I think there are two-ish (and a half) possible ways to do it?
proven garnet
I don't think string annotations work in class base lists?
```python
class Foo(P...
Is there a reason you would need to use strings?
marble hornet
i see, that was not a case I thought of
Enough for a PR?
digital shoreBOT
proven garnet
Is it something other than something not being defined yet?
jaunty juniper
what does it do to inherit from a protocol in python ??
marble hornet
Is it something other than something not being defined yet?
I think so, b/c the circuitpython_typing library is only used on cpython right?
proven garnet
what does it do to inherit from a protocol in python ??
Its how you normally use, it's what makes a usable typing Protocol
turbid radish
@lone axle please email when the newsletter PR is in. Thanks!
lone axle
<@382939733107408897> please email when the newsletter PR is in. Thanks!
Will do.
proven garnet
You can just define all the methods and attributes in that definition
jaunty juniper
but you don't need to inherit, a Protocol declares in a type annotation what attributes you expect
proven garnet
I think so, b/c the circuitpython_typing library is only used on cpython right?
Basically - since CPY will discard type annotations, only Blinka platforms get angry if type annotations aren't real and defined
tulip sleet
@marble hornet can you point to an example of using generic Protocol inheritance
marble hornet
lone axle
I'm going to drop and start getting the video and notes uploaded. Thanks all.
jaunty juniper
ah generic types
marble hornet
<https://peps.python.org/pep-0484/#instantiating-generic-classes-and-type-erasur...
thank you
tulip sleet
proven garnet
@lone axle did you say you were gonna respond on the EMC2101 PR?
jaunty juniper
we solve that by putting the protocols in a module that is guarded by try: import typing
lone axle
<@382939733107408897> did you say you were gonna respond on the EMC2101 PR?
Yep, I will here in a bit. Need to wrap up the post meeting tasks and will be taking a short walk soon. It's top on my list when I get back into it after that.
jaunty juniper
so your issue is that you want to put implementation in the class Something(Protocol) to inherit from ?
tulip sleet
@jaunty juniper we are discussing this in the audio chat
proven garnet
I will say, what would be nice about getting away from try/except, is that my type checker will always say that something either what it is, or Unbound because it recognizes that the import could fail
tidal kiln
@tulip sleet @jaunty juniper cool. looks like general source of issue has been found:
https://forums.adafruit.com/viewtopic.php?f=60&t=190755&start=30#p925222
just need to formalize a solution based on getting info on what chips got used
jaunty juniper
ok, sorry I shouldn't get into it, typing in python is not my battle
marble hornet
I will say, what would be nice about getting away from try/except, is that my ty...
right! I think we're coming from the same place.
I'll make a PR to discuss it
And/or if you have input and wanna chat I'm around a lot this week
proven garnet
Yeah definitely, though my input is gonna probably more about things like that since I'm core-ignorant haha
Honestly just having the TYPE_CHECKING or something that will get VS Code to stop being so mad and (fairly) calling everything Union[X, Unbound] would be neato hahaha
tulip sleet
@jaunty juniper your MONSTER M4SK fix works! If you do a PR I could do a quick release. Which chip did you add that worked?
but I also want to investigate the MatrixPortal memory issue, so i would probably wait for that
to make sure there isn't a fix needed for that as well
jaunty juniper
<@199811382793863168> your MONSTER M4SK fix works! If you do a PR I could do a q...
I blindly put all the chips that match 64 Mb winbond, they should know what they put on the boards at the fab though !
EXTERNAL_FLASH_DEVICES = "GD25Q64C,W25Q64JVxQ,W25Q64JVxM,W25Q64FV"
tulip sleet
i have an inquiry in for that, and I think I can look at the factory fab stuff too to find out
tulip sleet
I blindly put all the chips that match 64 Mb winbond, they should know what they...
according to the work orders, it is either Gigadevice GD25Q64CSIGR; or Winbond W25Q64JVSSIQ
that makes sense to me, that is a completely typical substitution, and those are the two parts we often use for this
jaunty juniper
what is the cost of adding flash types ?
tulip sleet
uses up internal flash space
jaunty juniper
could there be other boards that have had/could have that substitution that we could anticipate ?
tulip sleet
look for any other boards that have only GD25Q64C or GD25Q16C
and not the W25Q
we have a lot with GD25Q16C only, I don't think we are substituting those
ports/atmel-samd/boards/grandcentral_m4_express/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = "W25Q64JVxQ, GD25Q64C"
ports/atmel-samd/boards/hallowing_m0_express/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = "W25Q64JVxQ, GD25Q64C"
ports/atmel-samd/boards/hallowing_m4_express/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = GD25Q64C
ports/atmel-samd/boards/monster_m4sk/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = GD25Q64C
ports/atmel-samd/boards/pygamer/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = GD25Q64C
ports/atmel-samd/boards/pyportal/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = "W25Q64JVxQ, GD25Q64C"
ports/atmel-samd/boards/pyportal_titano/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = "W25Q64JVxQ, GD25Q64C"
ports/atmel-samd/boards/trellis_m4_express/mpconfigboard.mk
10:EXTERNAL_FLASH_DEVICES = "W25Q64JVxQ, GD25Q64C"
ports/raspberrypi/boards/adafruit_feather_rp2040/mpconfigboard.mk
9:EXTERNAL_FLASH_DEVICES = "GD25Q64C,W25Q64JVxQ"
removed non-adafruit boards, so add to hallowing_m4, monster_m4sk, and pygamer
lone axle
Here is the notes document for next Monday’s CircuitPython Weekly meeting. It is at the normal time of 11am Pacific / 2pm Eastern here on Discord. Everyone is encouraged to attend! Please add your hug reports and status updates even if you’ll be attending the meeting - it’s super helpful! If you are unable to attend but would still like to include updates, feel free to include them in the notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868> https://docs.google.com/document/d/1koPkyRaRg548N7dqxa-ymx8nybmd6kUSXxO-g8ZQDaw/edit?usp=sharing
slender iron
@idle owl it looks like the pre-commit upgrade of reuse broke some CIs
due to an unused license
idle owl
Ok. I'll look into it. I've had to add license types before.
manic glacierBOT
idle owl
@proven garnet Hiya! So we broke some CI. Apparently you can no longer have unused licenses in a repository, or that's what it looks like from the one linked above. I deleted the unused license from the LICENSES directory, and it passed. Is there a reasonable way to sort out what's not passing, and see if it's Reuse-update related?
@proven garnet In related news, the Library Infrastructure checks are failing on "incorrect pre-commit version/reuse version", meaning we need to update the version Adabot is checking for.
Ooh looks like Eva did it.
N. m. hehe.
orchid basinBOT
marble hornet
Honestly just having the TYPE_CHECKING or something that will get VS Code to sto...
😅, your not alone in having that same issue.
manic glacierBOT
tidal kiln
@tulip sleet @jaunty juniper worth opening a new issue for the qspi thing? for sake of tracking (not seeing one)
actually. maybe the PR is good enough. just want to give forums a link to keep an eye on.
tulip sleet
actually. maybe the PR is good enough. just want to give forums a link to keep a...
PR is fine, I think.
idle owl
@tulip sleet Ok, so this guide page.... Is meant to explain the difference between .mpy, .py and frozen libraries. I have a snippet of one of the conversations that came out of me asking about it here, and it gets into why .py files take up more memory than .mpy files, e.g. explaining exactly what memory is being used (compiling, and then the resulting bytecode which must be stored in memory during and after the compilation).
Is this getting too much in the weeds for this guide page? Or do you think it's worth including.
It would allow me to differentiate between the mpy and py memory usage, but maybe it's too much.
I'm trying to make sure this also answers a lot of questions we get. Because we do get a lot of questions about this topic.
tulip sleet
<@329766224093249548> Ok, so this guide page.... Is meant to explain the differe...
People ask all the time for the explanation, so I don't think it's too much. import of an .mpy loads the compiled code, import of a .py must compile the code. In both cases the compiled code lives in RAM, and the resulting compiled code is basically the same size in either case. But the compilation takes extra RAM temporarily during the compilation, and if there's not enough RAM available, it will fail. Compilation may also cause some fragmentation, but we are mostly able to avoid that.
I am probably just repeating what you said in the guide
idle owl
Not yet, I didn't. That's a great wording.
Thank you!
I'm definitely going to pass this by you before putting it in moderation.
tulip sleet
you can bring a pre-made cake to someone's house, or you can make it at their house. Either case, the final cake is the same size. But if you make it their house, you need space and resources temporarily to make it.
not sure that's a great analogy
proven garnet
<@135415127879974912> Hiya! So we broke some CI. Apparently you can no longer ha...
Well that's frustrating, but hooray! We found the botch! 
Let me check out gh CLI...
proven garnet
Okay I thiiiiiiiink I can get a list together of the libraries that the patch failed for using gh
@idle owl
idle owl
<a:this:407945287534116864> <@330227457296957440>
Thanks!
proven garnet
@idle owl @slender iron Wrote a script to find the latest CI triggered by me in the main branch across all the libraries (should be the patch). The only one that was flagged was the AzureIoT, but I am pretty confident it worked because it found another failure, though it's cause wasn't that, which I submitted a PR for: https://github.com/adafruit/Adafruit_CircuitPython_ESP_ATcontrol/pull/57
So I think that was the only one. Happy to share the script if anyone wants to check
idle owl
Thank you, @proven garnet!
proven garnet
No worries! Glad it wasn't any larger of a problem!
manic glacierBOT
I made a small 2x2 test setup of four https://www.adafruit.com/product/4978, NeoKey breakout. On these boards, the A connection is the anode, and the C connection is the cathode. Here is the schematic. I only used the SWITCHA and SWITCHC pins.
I wired the SWITCHAs together as thecolumns, and the SWITCHCs together as the rows. I believe this corresponds exactly to y...
rough folio
is there any guidance on building cirPy on windows? or is this a nix/mac only ballgame unless I want to do some digging
jaunty juniper
is there any guidance on building cirPy on windows? or is this a nix/mac only ba...
the Buiding Circuitpython guide uses WSL on windows
rough folio
yeah fair enough. WSL breaks my existing VM stack which is kind of fun so I was hoping I wouldn't have to deal with that whole affair
rough folio
oh well nbd
to that end- are there docs on making board configurations? Or just infer from existing
rough folio
<https://learn.adafruit.com/building-circuitpython/windows-subsystem-for-linux>
ah, this is useful in it's own way- I was on https://docs.circuitpython.org/
jaunty juniper
there is, but I think it needs slight updating ? Just copy from an existing one that's similar enough, there's not much more to it, we'll help you if there's something missing
https://learn.adafruit.com/how-to-add-a-new-board-to-circuitpython
rough folio
I've got a little sparkfun esp32s2 which is pretty similar to the adafruit feather so it shouldn't be much work
jaunty juniper
which one ?
rough folio
esp32-s2 thing plus
jaunty juniper
oh yeah that should be rather simple
tulip sleet
yeah fair enough. WSL breaks my existing VM stack which is kind of fun so I was ...
If you have a working VM setup, could you just create another VM instance and load Ubuntu on it? No need for WSL in that case.
rough folio
If you have a working VM setup, could you just create another VM instance and lo...
Yep, that's what's going on lol
proven garnet
Should fontio.FontProtocol be moved to circuitpython_typing? It needs the Glyph class defined, and I don't know if it makes sense to effectively have to mirror that in the other library in case it changes.
It just means right now one Protocol needs the stubs while type annotating
idle owl
@proven garnet Grabbing a couple of documents, but otherwise I'm ready when you are.
proven garnet
Sweet! Be right there
manic glacierBOT
idle owl
@tulip sleet I decided to make your cake analogy work because I felt like you mostly had it on point, but it needed a little more content to really work. Or, I wrote something terrible that should be deleted. 😆 You can bring a ready-to-eat cake to your friend's house, or you can show up with the plan to bake the same cake at your friend's house. Both completed cakes will take up the same amount of space. However, waiting to bake the cake means you'll need more space and resources before the cake is ready to eat, to combine the ingredients, bake it, and frost it. If it turns out your friend doesn't have the space available, or is missing a crucial ingredient, you wouldn't be able to complete the cake. Basically, if you're unsure whether you'll have everything needed to bake a cake at your friend's house, it's better to show up with a completed cake to ensure there is cake to be had!
manic glacierBOT
idle owl
Other than import order sometimes mattering, and dealing with memory allocation, what other "issues" can one run into with CircuitPython libraries? This is something that would be generalisable to all libraries, not something involving a specific library.
stuck elbow
memory fragmentation?
stuck elbow
loading the wrong library, because you have a leftover .py file lying around? (they take precedence)
idle owl
Oooooooh yes. Definitely.
stuck elbow
naming your test code the same as the library you want to test (for example "neopixel") and wondering why when you import it, it doesn't have all the things it was supposed to have, only your code
idle owl
Excellent
stuck elbow
shadowing the built-in libraries in general
especially when some 3rd party library uses it, and not you yourself
idle owl
shadowing?
stuck elbow
if you have neopixel.py it will get loaded instead of the built-in neopixel
idle owl
ahh right right ok
stuck elbow
not sure if this is relevant here, there is one common mistake with the library API, when people read what parameters a function takes, and then pass them as positional parameters, but don't realize some of the are keyword-only parameters, and then they get a confusing error saying that they provided a wrong number of parameters
jaunty juniper
or a directory, which is seen as a package module
like don't put all your neopixel scripts and files in a neopixel/ folder
idle owl
not sure if this is relevant here, there is one common mistake with the library ...
Oh like foo(10, 12, True) when it's supposed to be foo(x=10, y=12, position=True)?
stuck elbow
@idle owl yup
idle owl
like don't put all your neopixel scripts and files in a `neopixel/` folder
This is a good one too.
idle owl
not sure if this is relevant here, there is one common mistake with the library ...
Yah that might be a little too in the weeds for this page.
idle owl
like don't put all your neopixel scripts and files in a `neopixel/` folder
Got it on the list
stuck elbow
maybe it would be good to mention that modules are only ever imported once, even if you import them in multiple files or many places, the subsequent imports simply do nothing and give you the already imported library
idle owl
maybe it would be good to mention that modules are only ever imported once, even...
Oh, fair enough. I always thought it caused issues to do that. Didn't realise it was a non-issue.
stuck elbow
some people might expect it would re-load the library
idle owl
Agreed
stuck elbow
for instance, in the repl, you can import your code to test it, but a second import won't run it anymore
stuck elbow
you can also exploit it to set a global on an imported library in one file, and read it in another :P
jaunty juniper
which is how you can get the correct keyboard layout in macropad hotkeys
(using a module global variable in the macros scripts to share the Keycode instance)
tulip sleet
<@329766224093249548> I decided to make your cake analogy work because I felt li...
seems like fun!
idle owl
So my "Memory Fragmentation" section contains the following. Is there more to be added / a better way to explain it? A memory allocation failure does not necessarily mean you are entirely out of memory. It specifically means that there is not a single large enough chunk of memory to import a library of the given size. CircuitPython memory can become fragmented the more you do, and have smaller chunks of memory available across the entire heap, but not a single chunk big enough to complete a particular task.
Comes right after the "Memory Allocation" section. So that's not an oblique reference it starts with.
tulip sleet
So my "Memory Fragmentation" section contains the following. Is there more to be...
If this is in the context of import, yes. Fragmentation can block allocation of any chunk of memory that needs to be contiguous, such as a large buffer.
"It specifically means" in the case of import
idle owl
If this is in the context of `import`, yes. Fragmentation can block allocation o...
Ah, ok. I guess, I want it to be in the context of everything? I'm not sure how to explain it beyond what I posted.
Or at least in the full context of libraries
Does that limit the scope to import at that point? Or are there further bits involving the library files etc.
tulip sleet
"It specifically means that there is not a single large enough chunk of memory" to allocate the chunk requested. In the case of importing a .mpy file, the library is imported into a single chunk`. In the case of importing a .py file, you could run out of memory because more RAM than you have available is needed for compilation. In that case, the chunk needed is not necessarily large.
stuck elbow
so there is that optimization thing, that once a module is finished importing, it's moved to the end of available memory, because it's unlikely to be ever freed — and this decreases fragmentation
but I'm not sure we want to got that deep about this
tulip sleet
so in the kitchen example, you have run out of counterspace, or whatever. It's not that you necessarily need a huge chunk of counterspace all at once, but you keep using up sections of counterspace until there's none left. "I can't find anywhere to put the sugar down!"
idle owl
Nice ok
tulip sleet
i'm not sure the kitchen analogy is going to be that helpful in detail, maybe, maybe not
"there is no way I can make a cake in this closet"
idle owl
I feel like maybe not. I probably won't extend the reference beyond earlier usage.
stuck elbow
once you finish with one part of the cake, you clean up all the utensils you used to make it, and move the finished part to the end of the counter, to make room for the next step
idle owl
OK here's what I have now: ```A memory allocation failure does not necessarily mean you are entirely out of memory. CircuitPython memory can become fragmented the more you do, and have smaller chunks of memory available across the entire heap that add up to the amount needed, but not a single chunk big enough to complete a particular task. Fragmentation can block allocation of any chunk of memory that needs to be contiguous, such as a large buffer. In the context of importing a library file, it specifically means that there is not a single large enough chunk of memory to allocate the memory requested.
In the case of importing a .mpy file, the compiled bytecode is imported in a single chunk. In the case of importing a .py file, the size of the chunk needed for compilation may exceed the size of the chunks available, in which case, you would run out of memory before there is even any bytecode to import.
CircuitPython does do some optimisation to avoid memory fragmentation. Once a library is imported successfully, it is moved to the end of the available memory, which decreases fragmentation. Some fragmentation is unavoidable over time, but the import optimisation definitely helps with avoiding as much initially.```
tulip sleet
OK here's what I have now: ```A memory allocation failure does not necessarily m...
you don't need a single large chunk for compilation necessarily. You might need a lot of smaller chunks, which get used, and then discarded. So the "large chunk" is more for .mpy than .py.
I'm actually not even sure that .mpy needs a single large chunk, though I thought so . @slender iron could verify
idle owl
you don't need a single large chunk for compilation necessarily. You might need ...
Ok thanks for the clarification
slender iron
I don't think mpy needs a large chunk either. strings and functions will take linear space
stuck elbow
what about the AST representation and so on? I don't even know if micropython has an intermediate representation...
slender iron
I don't think it does... but I'm not sure
so, the dotenv library handles errors weirdly by logging and changing part of the return value: https://github.com/theskumar/python-dotenv/blob/master/src/dotenv/main.py#L208=
should I throw an exception, return the same weird tuple or just silently fail?
tulip sleet
so, the dotenv library handles errors weirdly by logging and changing part of th...
is this so it won't fail catastrophically if your .env is messed up?
slender iron
I'm not sure why they do it that way
tulip sleet
so if for instance the password key does not exist, but you need it, it will return "SORRY, NO PASSWORD" or equivalent for the value?
are there issues about it?
slender iron
the most common is probably trying to write a read-only file
slender iron
printing isn't really a solution
tulip sleet
no, not at all. It doesn't seem to be a high priority to fail more obviously in a non-interactive environment
slender iron
ya
tulip sleet
@slender iron If you search for "strict" in the closed issues, there is some idea it might be more stritc, but stuff like this got closed without being added: https://github.com/theskumar/python-dotenv/pull/263
tulip sleet
so they return a tuple on error, but a string otherwise?
tulip sleet
if not os.path.exists(dotenv_path):
logger.warning("Can't delete from %s - it doesn't exist.", dotenv_path)
return None, key_to_unset
removed = False
with rewrite(dotenv_path, encoding=encoding) as (source, dest):
for mapping in with_warn_for_invalid_lines(parse_stream(source)):
if mapping.key == key_to_unset:
removed = True
else:
dest.write(mapping.original.string)
if not removed:
logger.warning("Key %s not removed from %s - key doesn't exist.", key_to_unset, dotenv_path)
return None, key_to_unset
return removed, key_to_unset
It is never going to return False, key_to_unset because the if not removed: will catch both None and False
slender iron
ya...
tulip sleet
🤷
slender iron
I'll just return None
tulip sleet
i have something else to talk to you about: more fragmentation after 1.18 merge, I think
slender iron
np
manic glacierBOT
rough folio
so I built circuitpython for the sparkfun ESP32S2 thing board
except it's spamming invalid header 0xfffffff out the whahoozie with this bit of text every second or so
steps taken:
clone the esp32s2 feather board configuration and modify the board name in mpconfigboard.h/k
make board=sparkfun_thing_plus_esp32s2 (the new folder)
sudo esptool.py --port /dev/ttyUSB0 erase_flash
sudo python3 esptool.py --port /dev/ttyUSB0 --chip esp32s2 write-flash -z 0x1000 ../circuitpython/ports/espressif/build-sparkfun_esp32s2_thing_plus/firmware.bin
which are the steps listed on the micropython site which I perhaps mistakenly assumed would translate here
aaand I've just noticed that the feather is not the s2-wroom which is probably what's causing this :bleh:
onyx hinge
heya folks! It's not really CircuitPython related (except that I'm CircuitPython related!) but I'll be sharing a few photos during this trip. https://photos.app.goo.gl/u2c52WapAATLQ65u6 including the smallest elevator I've ever been in. I am having a great time but I miss y'all.
rough folio
aaand I've just noticed that the feather is not the s2-wroom which is probably w...
doesn't appear to have been it, used the saola_1_wroom configuration to the exact same issue
rough folio
so it was something to do with the offset... writing to 0 instead of 0x1000 fixed that part
I suppose reading that in the output of the build would have helped because it says the start addr right there
but now the UART port outputs naught but ESP-ROM:esp32s2-rc4-20191025 Build:Oct 25 2019 rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT) Saved PC:0x4002c15e SPIWP:0xee mode:DIO, clock div:2 load:0x3ffe6100,len:0x1368 load:0x4004c000,len:0xa4c load:0x40050000,len:0x2d60 entry 0x4004c1f8
which I'm pretty sure isn't what it should be outputting
jaunty juniper
here's what I get on the debug output on the FeatherS2, followed by a few more lines in green as ESP-IDF boots I assume
ESP-ROM:esp32s2-rc4-20191025
Build:Oct 25 2019
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:2
load:0x3ffe6100,len:0x1368
load:0x4004c000,len:0xa4c
load:0x40050000,len:0x2d60
entry 0x4004c1f8
~~that's what comes after:
I (79) spi_flash: detected chip: winbond
I (79) spi_flash: flash io: qio
I (80) cpu_start: Starting scheduler on PRO CPU.
I (82) gpio: GPIO[21]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
```~~
[wait no, I had not erased first, the green lines come from tinyuf2]it looks like it's just bootlooping I think
because it's just dumping that text above in a loop
jaunty juniper
ah
rough folio
RTC_SW_CPU_RST
hm
maybe it thinks the RST pin is held down?
or something?
that should just be... directly wired though right?
and even then...
jaunty juniper
I hadn't 😄 😇
rough folio
ok yeah so it's definitely bootlooping
observationally the feather appears to just interrupt the power for reset if I read the schematic right which I'm probably not?
I think I'm probably onto something now because flashing the espressif_saola_1_wroom board boots to SP-ROM:esp32s2-rc4-20191025 Build:Oct 25 2019 rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT) SPIWP:0xee mode:DIO, clock div:2 load:0x3ffe6100,len:0x1368 load:0x4004c000,len:0xa4c load:0x40050000,len:0x2d60 entry 0x4004c1f8
and doesn't bootloop
it doesn't give a console or anything like I think it's supposed to but
jaunty juniper
it is my understanding that the reset pin on the ESP is actually the enable pin
so there is no hardware "reset", only power on
rough folio
makes sense
but with the saola board firmware we're finally somewhere
whoops I already posted that
jaunty juniper
which I believe is the reasons it needs an external circuit to allow double-click reset to be detected (as the memory is wiped on poweron)
rough folio
interesting
so should I expect a cirpy console on uart or do I need to get the native usb pins out (which luckily is just a trace cut+jumper to solder)
jaunty juniper
you would need the USB, unless setup for repl on debug uart
rough folio
gotya
what does one need to do to setup repl on debug uart
jaunty juniper
lemme check
I believe you would need these in mpconfigboard.h:
#define DEBUG_UART_RX (&pin_GPIO16)
#define DEBUG_UART_TX (&pin_GPIO5)
with the right pins
rough folio
👍 thanks
jaunty juniper
or maybe that's just the debug output ?
rough folio
unthank
rough folio
is that just a pin definition as above or
jaunty juniper
yes
rough folio
rethank
alright well that was a fun little adventure but it looks like I'm pretty much set
tulip sleet
@slender iron the fragmentation problem looks like it's straightforward. The usual automatic gc_collect() after an import was removed in the v1.18 merge, so restoring that should fix things. py/builtinimport.c was very much reworked in v1.18
tulip sleet
np, bugs are to fix
warm flame
Should I program all the Lilygo boards ?
onyx hinge
just dropping by 👋
warm flame
yes no no yes yes yes no no yes no ?
slender iron
Nice find @tulip sleet ! Interesting it makes that much impact
onyx hinge
@warm flame we're happy to support more boards (we treat it as a metric or at least a cause for celebration, since we just passed 300), so if you have the boards for testing why not!
warm flame
<@385537177838288898> we're happy to support more boards (we treat it as a metri...
not yet, but I am looking to buy some of them. I like the ones with battery packs on them
onyx hinge
It's probably wise to only add support for a board you have in hand
jaunty juniper
note that we don't do suffix-less ESP32 (yet) which I think is most of their boards
rough folio
@jaunty juniper I'm back poking at this situation- do you know what common_hal_never_reset_pin is about? I've found it adds pins to a mapping used somewhere, but I've really got no idea what this accomplishes
it's one of the handful of differences between the adafruit feather def and the espressif devkit
the only real others are the pins.c and reset_board turns on i2c power
actually
what calls reset_board, board_deinit etc?
slender iron
the only real others are the pins.c and reset_board turns on i2c power
actually
...
I believe main.c calls reset and deinit
rough folio
hm ok
orchid basinBOT
idle owl
@tulip sleet When you're done with the release, and you have a moment, I'd appreciate some feedback on this guide page. I still need to add a couple of screenshots so it's not so wall-of-text-y, but I think the text content is ready. https://learn.adafruit.com/welcome-to-circuitpython/library-file-types?preview_token=WsdVgHQVBTr9iHLerIfyWA
@stuck elbow or @jaunty juniper if you're up for providing feedback and able to view the link above (I don't know how public the previews are), I would also appreciate it.
tulip sleet
<@329766224093249548> When you're done with the release, and you have a moment, ...
I am just waiting for builds; i'll take a look now!
idle owl
I am just waiting for builds; i'll take a look now!
Even better! Thanks, epic CI time!
@tulip sleet If you end up wanting to make edits, let me know so I can back out of any editing I'm doing.
Oh I need to mention somewhere that the Project Bundle will download frozen modules, but it's ok, they don't run, they simply take up disk space. And how to delete them if you run out of disk space.
I feel like the Project Bundler should be mentioned on the existing Libraries page, and then linked into the Frozen Modules section with the info about how they work together in the File Types guide page. We don't mention it anywhere in this guide, afaik.
jaunty juniper
what do you want to say about it ?
the "download bundle" buttons and how the code is organized inside the zip would be useful maybe
idle owl
That it exists, how to use it, and why it makes things easier. I don't think all that needs to be on the File Types page. Put that on the CircuitPython Libraries page, and then link that section in the File Types page when I explain the situation with frozen modules and the bundler.
idle owl
the "download bundle" buttons and how the code is organized inside the zip would...
Good point.
tulip sleet
@idle owl
- do you want to say how using mpy-cross makes it possible to import some libraries that couldn't even be imported in .py form on small boards, because they take too much RAM to compile on the board?
- "there is nowhere to download frozen libraries, because they are built into CircuitPython" -> ... because a frozen library is built in to in the CircuitPython build for particular boards [or equiv]. Just to emphasize they are not built in to CPy in general.
- "If there are not frozen modules" -> "If there are no frozen modules"
- bold
CIRCUITPY/on second mention - "memory" usually means RAM. Sometimes people are confused and think that because there's still room on CIRCUITPY, they should be able to import. Maybe define "memory", or use RAM (and explain the difference?)
- Title of page could be "Library File Types and Frozen Libraries", since that's a major part of the page.
idle owl
<@330227457296957440>
1. do you want to say how using mpy-cross makes it possibl...
Yes! Thank you. And I was unhappy with the title, but wasn't thinking about how to update it.
tulip sleet
I really like this page; it is a really thorough and readable explanation of something we have to explain constantly
idle owl
That's what I was going for 🙂
rough folio
the mpy files seem pretty surface level similar to the cpy cache files
if I read it right
idle owl
@tulip sleet Re: RAM vs memory. Hmm. Maybe include a note at the beginning that for the purposes... wait no. Because I'm about to add something about disk space. Which means the entire page cannot be said to be using "memory" and "RAM" interchangeably. Ok, I'll update it to RAM.
tulip sleet
the mpy files seem pretty surface level similar to the cpy cache files
yes, they are like .pyc: the result of compiling into bytecodes
idle owl
Only 44 instances of memory to update. 😖
tulip sleet
so maybe have to define RAM vs diskspace
no global search/replace in guide pages, right?
idle owl
Correct, I have to do it manually. Search in page, and change them up.
rough folio
is the largest distinction that an mpy can have C code embedded?
jaunty juniper
have we enabled that in CP ? I don't think anyone has tried
rough folio
I was just looking over the micropy docs and it mentioned that
tulip sleet
is the largest distinction that an mpy can have C code embedded?
.pyc get generated and saved in __pycache__. .mpy are generated manually, not automatically. Also the bytecode formats between CPython and MicroPython/CircuitPython are different
rough folio
yeah I assumed the bytecode was different was just curious if that was kind of the end of it or if there was something more
tulip sleet
upcoming version of MicroPython will allow executing bytecodes directly from .mpy files, so they don't have to be loaded into RAM, but they have to be contiguous in the filesystem, which we can't guarantee with the FAT filesystem
rough folio
that's a pretty tidy improvement though
I've been playing around with nuitka at work and started wondering about it and hardware
probably generates too much boilerplate to work
idle owl
Ok, yeah, I definitely need to explain the difference because I can't change "Memory Allocation" to "RAM Allocation" because the actual error is "memory allocation failed". I'll ping you in a minute here, Dan, for feedback on this. It feels a bit stilted.
idle owl
CircuitPython is unaware of disk space, right? Like it won't error when you're out of it. Only your computer will? Or editor will fail to save or whatever..?
tulip sleet
<@329766224093249548> https://learn.adafruit.com/welcome-to-circuitpython/librar...
I think this can be quite bit shorter. You can just say that "memory" in this guide means RAM, random access memory, which is the read/write memory in the microcontroller. So sometimes you would just say "memory", and sometimes you would say "CIRCUITPY filesystem space". I'm not sure about "disk space", there is no disk, but it is the vernacular.
idle owl
Ok thank you
tulip sleet
CircuitPython is unaware of disk space, right? Like it won't error when you're o...
CircuitPython will tell you if CIRCUITPY is full if you try to write to it from Python
idle owl
CircuitPython will? In the serial console or something?
I've only ever seen popups on my machine.
tulip sleet
The host notices if the disk is file if the host is writing to CIRCUITPY. But CIRCUITPY will throw an OSError with the ENOSPC code ("no space left on device") if you try to write to a full disk from Python
idle owl
Ah ok. Huh.
tulip sleet
so that only happens if you mount CIRCUITPY r/w, etc.
idle owl
I think this can be quite bit shorter. You can just say that "memory" in this g...
Can you take another look?
tulip sleet
There is one section that references CIRCUITPY filesystem space, which can also be considered memory. Filesystem space is not RAM; it is where files are stored
idle owl
That is better.
tulip sleet
You could use "RAM memory" once here to make it clearer: The result is a file that is smaller and uses less RAM memory.
idle owl
Yeah ok. That seems like a solid idea.
tulip sleet
"This page refers regularly to library file memory usage, as well as to RAM usage."
these are the same thing, 'This page talks about library file memory (RAM) usage". "as well as" kinda makes one think they might be idfferent things
idle owl
Exactly. It made way less sense as an intro to this section once it was shortened.
Fixed. Thank you!
I think I'm going to do some future proofing and refer to the 7.3.0 version of mpy-cross. Even though we're not stable yet.
slender iron
do you want to talk about volatile vs non-volatile
aka, whether you lose data on power loss
idle owl
I don't think so. This is more about libraries, the explanation of RAM is simply because I used "memory" and "RAM" interchangeably on the page.
slender iron
kk
slender iron
if user code does wifi.connect() should stopping the VM also stop wifi?
Or we can leave it connected and only reconnect if connect() is given a different ssid
@jaunty juniper thoughts on wifi connection lifetime?
I'm thinking I'll track if a socket is opened by the VM and close those that were
idle owl
@jaunty juniper Were you able to view the guide preview I linked earlier? I wrote up the Project Bundle section on a separate page so it wasn't ever in an incomplete state on the live page. You could take a look if you like, and let me know if there's anything obvious missing or better said. Ignore the page title, it's a throwaway page. https://learn.adafruit.com/welcome-to-circuitpython/project-bundle-section-for-cp-libs-page?preview_token=WsdVgHQVBTr9iHLerIfyWA
Time for dins here. Be around later!
manic glacierBOT
rough folio
how should I reference pin TXD0 (pin 37) in pins.c?
I also see { MP_ROM_QSTR(MP_QSTR_I2C), MP_ROM_PTR(&board_i2c_obj) }, for a number of boards- how does this pull pins?
jaunty juniper
that would be IO43 ?
jaunty juniper
I also see `{ MP_ROM_QSTR(MP_QSTR_I2C), MP_ROM_PTR(&board_i2c_obj) },` for a num...
that's the bus pins defined in mpconfigboard.h
rough folio
that would be IO43 ?
oh, just based off sequence
jaunty juniper
which should be the same as SDA and SCL
jaunty juniper
oh, just based off sequence
it should be explicit somewhere in the datasheet, some table I guess
rough folio
you'd think so wouldn't you
but if it is I can't find it https://cdn.sparkfun.com/assets/b/4/8/0/c/esp32-s2-wroom_module_datasheet.pdf
jaunty juniper
rough folio
yeah but this doesn't strictly list IO43 which is what I was getting caught on
jaunty juniper
it does
tidal kiln
@tulip sleet is that weird ESP32S2 TFT feather issue CP only? boards that have that behavior work fine with factory reset UF2?
tidal kiln
TFT is working when "double click" switch to get loader
and after a ROM reload. Once any CircuitPython other
than 7,1.1 is loaded, the TFT is off by default.
seems CP only
tulip sleet
it should be fixed now
tidal kiln
just checking behavior here:
https://forums.adafruit.com/viewtopic.php?f=57&t=191289
sounds like even the bootloader screen works in those situations
forgot the bootloader did that
tulip sleet
It sounds like it might have been fixed in 7.3.x but not in 7.2.x, so have them try the 7.3.0-rc.1
tidal kiln
but if bootloader is dark also, that'd be something else?
tulip sleet
they didn't say they double-clicked. They could try the factory reset. I'd start with that and see if it doesn't show.
tidal kiln
it didn't
tulip sleet
wait I don't remember that the ESP32-S2 TFT has a boot screen
tidal kiln
yah. it shows UF2 splash (not sure what else to call it) when in bootloader mode.
tulip sleet
that's the UF2 bootloader, but not the raw bootloader. If they loaded with the raw bootloader, it wouldn't show
I would have them erase the whole thing and reinstall the factory reset with UF2 bootloader. Also, the double-click is a bit tricky, have to wait for purple
tidal kiln
the factory reset is a UF2 (well one of them), they say they tried that. that would have required getting into UF2 bootloader mode.
tulip sleet
the original load is loading a .bin
ok, I see, two boards, one doesn't work, yes fishy
tidal kiln
learn page has two "factory reset" options:
https://learn.adafruit.com/adafruit-esp32-s2-tft-feather/factory-reset
it is a bit unclear what they actually did
tulip sleet
yes, I would pursue having them do exactly the same thing on both boards
I have had to full-erase at least one Feather ESP32-S2 to get it to work right
tidal kiln
@jaunty juniper just fyi. see above. relates to a forum thread.
jaunty juniper
oh hey another library that crashes because of typing annotations !
idle owl
Uff.
stuck elbow
They have played us for absolute fools, Python was never supposed to have strict typing.
jaunty juniper
put me on record as being against breaking libraries, bloating code, ram and the core to jump through hoops to add typing to python of all languages.
stuck elbow
amen
jaunty juniper
opened an issue in adafruit_rfm9x
stuck elbow
unit tests, on the other hand...
you can add the type annotations in a separate file, then it doesn't add any bloat or break anything
I think that would make a lot of sense in CP
proven garnet
opened an issue in `adafruit_rfm9x`
TIL typing.Literal only got added in 3.8, whoops. I can submit a hotfix PR now and for others using Literal, I don't think it's ever been guarded so far like Protocol has
I also submitted issues for the other libraries with the same issue, less than a handful.
Using stubs would be cool, I never really thought about the effect on RAM until it came up in this week's meeting
proven garnet
I also submitted issues for the other libraries with the same issue, less than a...
Meant to say I can take care of this either today or tomorrow
manic glacierBOT
proven garnet
Never mind, @lone axle is faster than me 
Thanks!
lone axle
Never mind, <@382939733107408897> is faster than me <a:sonic:932489292817793044...
Thanks for finding them and making issues and doing the first one. These were top on the stack when I sat down to work a bit ago and figured I could knock them out real fast after I saw the first one.
slender iron
@tulip sleet my pr ran out of flash so I was looking into graphing section sizes
tulip sleet
<@329766224093249548> my pr ran out of flash so I was looking into graphing sect...
my PR gains back some flash, would it help?
slender iron
Graph as in dependencies
tulip sleet
are you trying to get it to run on M0?
manic glacierBOT
slender iron
C3 space worries me
tulip sleet
how much RAM on C3?
slender iron
Don’t remember, on my phone
slender iron
Np
jaunty juniper
is there anything blocking https://github.com/adafruit/Adafruit_CircuitPython_CircuitPlayground/pull/117 ?
manic glacierBOT
onyx hinge
Wow danh
proven garnet
🤯 look at all that room!
@idle owl I had forgotten my personal laptop that I do dev stuff on at the office, so I just got it back. I'll get it set up for Adabot and try to submit at least one patch for review this weekend. I'll scream let you know if I have trouble getting it set up!
Also, I think I found the way to crunch multiple commits into a single patch; might try to test it with this patch if it makes sense!
warm flame
I just installed the build for the Lilygo T-OI plus... looks like it is not operational
I f-ed up somewhere
Can someone helps me compile the build on Debian ?
stuck elbow
you are following the instructions?
warm flame
What compiler should I use
stuck elbow
it says in the instructions
tulip sleet
it's described in the Building CircuitPython guide
warm flame
thanks
warm flame
I messed up the board.c I think
oh dear. I'll have to test stuff all over the weekend
lol
installing all tools to be able to build circuitpython for that specific board is long
jaunty juniper
the ESP IDF environment is quite something
warm flame
and I should have installed it in circuitpython directory but didn't
so I end up with 2 copies of it
idle owl
<@330227457296957440> I had forgotten my personal laptop that I do dev stuff on ...
OK, sounds good! No worries about timing.
warm flame
my make BOARD=lilygo_ttgo_t-oi_plus returns me : find: ‘../../extmod/ulab/code’: No such file or directory
make: *** No rule to make target 'lib/protomatter/src/core.c', needed by 'build-lilygo_ttgo_t-oi_plus/genhdr/qstr.split'. Stop.
jaunty juniper
did you run make fetch-submodules ?
misty garnet
any advice on how robust the rs485_dir timing is on the RP2040? are we better off doing our own software implementation where we can tweak the timing? Is it more or less robust at higher baudrates?
misty garnet
https://github.com/adafruit/circuitpython/blob/main/ports/raspberrypi/common-hal...
thx, should be more responsive than doing it at the python level but might be to fast for some transceivers, looks like it's time to experiment
lone axle
Is there a known issue around make fetch-submodules and the pico SDK?
From https://github.com/raspberrypi/pico-sdk
* branch 2062372d203b372849d573f252cf7c6dc2800c0a -> FETCH_HEAD
! [rejected] 1.3.0 -> 1.3.0 (would clobber existing tag)
fatal: run_command returned non-zero status for ports/raspberrypi/sdk
.
make: *** [Makefile:332: fetch-submodules] Error 128
no, I haven't seen that
seems like that means they changed the 1.3 tag
(which is bad form imo)
1.3.1 is free
tulip sleet
seems like that means they changed the 1.3 tag
@lone axle @slender iron https://github.com/raspberrypi/pico-sdk/issues/773
there is a 1.3.1 now
slender iron
tulip sleet
is that entire diagram encoded in the URL? 🙂
lone axle
How does the 'Download Project Bundle' button on learn guides "choose" which files to include in the bundle download? I found an instance on this page: https://learn.adafruit.com/circuitpython-made-easy-on-circuit-playground-express/play-file where the examples make use of some .wav files but the bundles come with .mp3 files instead of the .wav ones. I clicked through the github link and can see that the library repo this is embedded from includes both mp3 and wav files. But somehow the bundle button is omitting the .wav and including the mp3 even though this particular example is using the wav.
I also notice github seems to be changing the name of raw file downloads. when I click these buttons:
I end up with files named examples_dip.wav and examples_rise.wav instead of just dip.wav and rise.wav as is used in the code.
Even downloading from the github page directly with this button results in the file named examples_dip.wav
I assume maybe github changed this behavior at some point? I'm not sure why they wouldn't serve it using the original filename which they definitely know.
tulip sleet
I also notice github seems to be changing the name of raw file downloads. when I...
what is the URL?
I would think is Learn Guide behavior,
tulip sleet
that is totally weird, this seems like a misfeature
lone axle
totally agree, cannot fathom why they would think people want the filename to be altered.
maybe it's accidental and there is some bug with a slash somewhere in their system.
tulip sleet
trying this on the circuitpython repo: https://github.com/adafruit/circuitpython/raw/main/py/argcheck.c, I get a redirect and it just downloads in the browser
redirects to https://raw.githubusercontent.com/adafruit/circuitpython/main/py/argcheck.c, so maybe it is a repo setting??
hmm, that is a human-readable file, but the .wav files are not...
lone axle
interesting, or maybe examples/ is a special case directory?
tulip sleet
interesting, or maybe `examples/` is a special case directory?
I think this might be a browser thing! https://github.com/adafruit/Adafruit_CircuitPython_CircuitPlayground/blob/main/examples/dip.wav does different things in Firefox and Chrome
sorry, tried wrong link
lone axle
Strange, that would be even weirder for a browser to decide to change it's name. I get the same results in both Chrome and FF here.
tulip sleet
i was using the blob instead of the raw link
$ curl -I https://github.com/adafruit/Adafruit_CircuitPython_CircuitPlayground/raw/main/examples/dip.wav
HTTP/2 302
...
location: https://raw.githubusercontent.com/adafruit/Adafruit_CircuitPython_CircuitPlayground/main/examples/dip.wav
...
$ curl -I https://raw.githubusercontent.com/adafruit/Adafruit_CircuitPython_CircuitPlayground/main/examples/dip.wav
HTTP/2 200
cache-control: max-age=300
content-disposition: attachment; filename=examples/dip.wav
Note the filename=examples/dip.wav. The browser can't do that so it substitutes _ for /
so the original link does a temp redirect to raw.githubusercontent.com
tulip sleet
tulip sleet
@slender iron turning on f-strings everywhere in 7.3.0 looks like it is going to work with just one board shrink; I am a little psyched about that; it will remove consideration of whether one has to use format() or not in library and example code. Made it ready to review. Going for a walk.
slender iron
that's something worth having everywhere
jaunty juniper
speaking of f-strings, I completely missed we had that:
Adafruit CircuitPython 7.2.4 on 2022-03-31; Adafruit CLUE nRF52840 Express with nRF52840
>>> value = 1
>>> padding = 5
>>> f"{value:0{padding}d}"
'00001'
onyx hinge
so much good stuff happening! 👋
lone axle
is mpy version different for 8? Will we need some update inside of circup? I'm noticing I can't install libraries with it on a build started from current main:
❯ circup install adafruit_display_text
Found device at /media/timc/CIRCUITPY, running CircuitPython 8.0.0-alpha.0-dirty.
Searching for dependencies for: ['adafruit_display_text']
Ready to install: ['adafruit_bitmap_font', 'adafruit_display_text']
Traceback (most recent call last):
File "/home/timc/.local/bin/circup", line 8, in <module>
sys.exit(main())
File "/home/timc/.local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/home/timc/.local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/timc/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/timc/.local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/timc/.local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/timc/.local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/timc/.local/lib/python3.8/site-packages/circup/__init__.py", line 1316, in install
install_module(
File "/home/timc/.local/lib/python3.8/site-packages/circup/__init__.py", line 1016, in install_module
bundle_path = os.path.join(bundle.lib_dir(bundle_platform), module_name)
File "/home/timc/.local/lib/python3.8/site-packages/circup/__init__.py", line 123, in lib_dir
self.basename.format(platform=PLATFORMS[platform], tag=tag),
KeyError: '8mpy'
8 is not going to be different. We will do that in 9, when we merge MicroPython 1.19. circup needs an update to know about 8.
8 may be superseded by 9 relatively quickly. We thought 1.19 was quite imminent, but it is a big change, and there are still some PR's in progress in MicroPython. And 1.19 will be a big change for MicroPython, so we might wait a bit for it to shake out.
jaunty juniper
so we won't need to make CP8 versions of the bundles since the mpy format doesn't change (and we don't do versioning based on code compatibility). It is a potential support issue, what with the bundle having "7.x" in the name. We could change the name but that means changing conventions and therefore tools at many levels.
(meanwhile circup can just be told to treat 8 as 7)
tulip sleet
I think we will anyway, we usually do, even if they are the same, so it's not a question. We'll have both 7.x and 8.x bundles, same as what we did for 6.x and 7.x
we haven't published the 8.x.x release yet; the alpha.0 tag is just a stake in the sand
we won't publish an 8.x.x release until 7.3.0 final.
since there is only one "unstable" listing on circuitpython.org downloads pages
jaunty juniper
ok so, when that happens, the bundles add the 8.x and we can just add it to circup
tulip sleet
yes, we have to update circup, update the bundle builds, and update circuitpython.org/libraries
also RTD has to be updated
orchid basinBOT
tulip sleet
same as rc.1, except f-strings enabled in all builds (so now in small M0 builds, for instance)
tulip sleet
@fathom sand Besides https://github.com/adafruit/circuitpython/issues/6254, there is also https://github.com/adafruit/circuitpython/pull/6349
manic glacierBOT
manic glacierBOT
manic glacierBOT
manic glacierBOT
orchid basinBOT
fathom sand
Hi, is there anyone having success to talk to a DFR0627 board using CircuitPython?
I can see the board showing up in a i2c.scan()
I also can write to and read from the board.
But “nothing” seems to work.
Even writing a value to a register and reading the register back always only gives 0 as the value read.
Any hints are very welcomed!
https://wiki.dfrobot.com/Gravity%3A IIC to Dual UART Module SKU%3A DFR0627
I am experiancing problems while using busio.UART on a Feather ESP32-S3 No PSRAM (and other ESP32-S3 boards)
Assuming TX and RX are connected!
''''
import sys
import time
import board
import busio
import supervisor
uart = busio.UART(board.TX, board.RX, timeout=0.01)
again = 0
count = 0
while True:
if supervisor.runtime.serial_bytes_available:
k = sys.stdin.read(1)
if ord(k) == 27:
break
again += 1
if again > 100:
count += 1
s = f"hello uart! {count}".encode()
uart.write(s)
print("sending:", s)
again = 0
if False:
r = uart.read(uart.in_waiting)
time.sleep(0.01)
else:
r = uart.read()
if r:
print("receiving:", r)
print("done")
''''
The code above "behaves" very different if used on a M4 board compared to a ESP32-S3 board using the same CP version.
(Tested today with CircuitPython 7.3.0-rc.2 on a ItsyBitsy M4 vs. a Feather ESP32-S3 No PSRAM)
And chainging "if False:" to "If True:" does not change the results.
It even seems that the "offset" of the buffer / queue in the ESP32-S3 is increasing with every restart of the code.py app.
tulip sleet
@fathom sand please add to that issue we discussed, or maybe open a new issue -- I don't remember whether it was an aside in the original issue or not.
tulip sleet
Hi, is there anyone having success to talk to a DFR0627 board using CircuitPytho...
If you are testing the DFR6027 on an ESP32-S3, note there are multiple reported bugs in the ESP-IDF SDK for S3. In particular we found it doesn't do clock stretching well. Could you try it on another board, like M4?
jaunty juniper
so, int.from_bytes is weirdly defined as taking 2-3 arguments, but ignores the 3rd argument, and C python does not take a 3rd argument, so I'm tempted to do that (or use MP_DEFINE_CONST_FUN_OBJ_3 if I get that correctly), or am I missing something ?
- STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(int_from_bytes_fun_obj, 3, 4, int_from_bytes);
+ STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(int_from_bytes_fun_obj, 3, 3, int_from_bytes);
CPython takes signed as the 3rd arg, I think: https://docs.python.org/3/library/stdtypes.html?highlight=from_bytes#int.from_bytes
jaunty juniper
it's keyword only though ?
jaunty juniper
so, it should error on int.from_bytes(bytes([0x10,0x00]),"big",444444)
tulip sleet
it should; there is minimal validation on some of these keyword-only functions, inherited from micropython
jaunty juniper
and it doesn't take keyword arguments (in CP)
tulip sleet
so yes, maybe the right thing is to not permit the third arg, which is not implemented and cannot be a keyword
I ran into this in some other case; I'll look for it
I guess it was this: https://github.com/adafruit/circuitpython/issues/5407
manic glacierBOT
fathom sand
If you are testing the DFR6027 on an ESP32-S3, note there are multiple reported ...
I did my tests with a ItsyBitsy M4.
manic glacierBOT
proven garnet
@idle owl, got patches for the .pylintrc files, I can PR those to adabot repo tonight. I should also draft PR those changes to the BNO055 repo right? Anything else I need to do?
proven garnet
@tulip sleet trying to understand from last meeting, if CircuitPython had a typing module with an attribute TYPE_CHECKING that was always false, would it be able to be optimized? And if so, would that be for the core or mpy?
Or is that not correct?
tulip sleet
<@329766224093249548> trying to understand from last meeting, if CircuitPython h...
Jeff showed a way to optimize it, using the trick of const(an integer)
but I don't think we necessarily want to make typing a required library everywhere. What problem are we trying to solve?
Maybe we could make it an almost no-op builtin
proven garnet
More of just wondering if moving away from using the try/except block to something that could be optimized
tulip sleet
The try/except block is cheap, but it does add a little extra code in the .mpy
proven garnet
_Maybe_ we could make it an almost no-op builtin
The only issue I can think of is that you it would make CPython code compatibility more difficult because you probably don't actually want to wrap it in something that won't run at runtime
tulip sleet
we could make a typing that just had the TYPE_CHECKING = false in it, but you still need to avoid from typing import TYPE_CHECKING, Optional etc.
What is an example of using it vs not, and are there CPython precedents
proven garnet
we could make a `typing` that just had the `TYPE_CHECKING = false` in it, but y...
I typically don't think there's a use case actually in CPython the way we might use it, since there isnt much of a penalty to just importing it I imagine
Plus the annotations do actually need to be imported
onyx hinge
Plus the annotations do actually need to be imported
No, use from __future__ import annotations so from typing import Union etc can be conditional. At least, I think it's preferable
onyx hinge
<@329766224093249548> trying to understand from last meeting, if CircuitPython h...
Without doing something clever in the compiler from typing import TYPE_CHECKING doesn't seem to allow dead code elimination
onyx hinge
Jeff showed a way to optimize it, using the trick of `const(an integer)`
I thought that was your idea
tulip sleet
no, you presented that 🙂
maybe I said something that inspired you
I think I didn't say const, but proposed defining it. You came up with the const() int idea
onyx hinge
Hmmm we could add a bit of code to make it as though every file had TYPE_CHECKING = const(0) though. Too bad if someone wants to use that identifier for anything else, ever
proven garnet
No, use `from __future__ import annotations` so `from typing import Union` etc c...
Oh okay, that's neat
onyx hinge
importing a const doesn't make it a const in the module where that imported into. It'd be in the compiler, at the beginning of every module, the constant table is populated with an entry for TYPE_CHECKING with a value of 0.
onyx hinge
==> typing.py <==
TYPE_CHECKING = 7
==> typetrick.py <==
TYPE_CHECKING = const(0) # If this line was implicitly at the top of every module...
from typing import TYPE_CHECKING # and we had a mini-typing module...
import typing
print(TYPE_CHECKING, typing.TYPE_CHECKING)
0 7
This doesn't show that the unused code optimization takes place for sure, but I think it's a good sign.
tulip sleet
i will be afk for a while
manic glacierBOT
manic glacierBOT
idle owl
<@&356864093652516868> The Weekly Community meeting is in < 40 minutes. Everyone is welcome! If you plan to participate, please leave your hug reports and status update in the notes document. Hope to see you all there! https://docs.google.com/document/d/1koPkyRaRg548N7dqxa-ymx8nybmd6kUSXxO-g8ZQDaw/edit?usp=sharing
idle owl
We're finishing up an internal meeting. We'll be there as soon as we're done.
@inland tusk Can you mute please?
thorny jay
@proven garnet without spoiling what you are going to say, I am super interested in your "image transfer setup for the BluefruitConnect library". I think there was only Arduino implementation available so far. I have a direct use for this (if I find some dithering code).
manic glacierBOT
lone axle
I think Jeff may have added some dithering functionality to bitmaptools somewhat recently.
slender iron
I muted @inland tusk for now
gilded cradle
@idle owl you muted him for you, but @slender iron muted him for everyone
thorny jay
I think Jeff may have added some dithering functionality to bitmaptools somewhat...
Oh, I need to map to a fixed palette of 100+ colours... I guess I will have to work a little bit to make something that work for me.
turbid radish
😊
thorny jay
slender iron
does anyone remember the visualizer of people working on an open source code base?
its an animation
turbid radish
Yes, just a sec Scott
thorny jay
Should it be circup detecting frozen module and not installing the matching mpy?
turbid radish
@slender iron https://github.com/acaudwell/Gource
slender iron
thank you!
turbid radish
Make the directory read only
proven garnet
I've definitely had issues with symbolic links in windows
gilded cradle
hehe nice
proven garnet
<@135415127879974912> without spoiling what you are going to say, I am super int...
Haha sorry for being text only
inland tusk
Need to unlock my voice rights
proven garnet
I basically tried to get the library to work with the image transfer but still WIP
I can share my branch with you
thorny jay
I might have wrote an issue long time ago to say it was missing. 🙂
thorny jay
I can share my branch with you
Yes please.
proven garnet
It so far looks like the app isn't sending it properly