ok, three answers at once 🙂
#circuitpython-dev
tulip sleet
prime flower
Perfect that's exactly where to look. Thank you
lone axle
it's making the build tool look for fake_bme280.py instead of fake_bme280/
prime flower
actually thanks all 🙂
midnight ember
is there a better example that doesn't assume we're working in blinka?
# imports
import math
import time
from micropython import const
from adafruit_bus_device.i2c_device import I2CDevice
import busio
try:
from typing import TYPE_CHECKING, List, Tuple, Optional, Iterable
from PIL import Image
if TYPE_CHECKING:
from circuitpython_typing import ReadableBuffer, WriteableBuffer
except ImportError as e:
pass
I'm hesitant to remove TYPE_CHECKING because I don't know if that's needed by Blinka. I'd rather not hack a script designed for Blinka to pieces and break their compatibility for microcontrollers sake.
spare jacinth
I'm hesitant to remove TYPE_CHECKING because I don't know if that's needed by Bl...
afaik it is always set to False at runtime, and is only used for static code analizers and whatnot
lone axle
I don't think the TYPE_CHECKING if statement is needed for blinka
spare jacinth
afaik it is always set to `False` at runtime, and is only used for static code a...
no idea how or why it works (or is needed/useful), but i think it "works" that way
prime flower
I'm not sure that's required for Blinka, I've been pretty deep in the blinka codebase this past week and haven't seen it utilized.
midnight ember
then why is it there? actually i'm probably the one that put it there to begin with by unknowingly copying it from a script designed for blinka.
spare jacinth
then why is it there? actually i'm probably the one that put it there to begin ...
as i just said... for some stuff that static analyzers (eg mypy) can use but the actual code would not
i guess it is useful for scenarios where file.pyi with type annotations exist, but there is not an equivalent file.py or file/__init__.py that the runtime could use
and there are probably some other cases where it is useful (?) but im not much of a type-checking guy, i just write my type hints and hear PyLint about obvious issues lmao
midnight ember
i removed it and it passed my local pylint.
tulip sleet
it's there so we can run mypy and do type checking
spare jacinth
(please ping if you do, i will be deep into VSCode for a while :P)
lone axle
(please ping if you do, i will be deep into VSCode for a while :P)
I will try it, but it'll be quite a while before I can build it. If you can share a magtag UF2 I'll be able to test it sooner.
midnight ember
Nope, still failed NameError: name 'WriteableBuffer' is not defined
spare jacinth
```py
# imports
import math
import time
from micropython import const
from adaf...
is that code something you are writing from scratch or editing?
midnight ember
trying to add typing to the library, so both?
spare jacinth
i guess a possible issue is that, since you have a huge try/except for everything, maybe some other import is failing (eg micro-controller specific stuff, that is not available on the computer), and is preventing the circuitpython_typing import to get evaluated
looking at the code it seems like Image is required for the code to work, so i would def. not have that import on a try/except block
and you should probably have 2 different try/except for the typing and circuitpython_typing imports, and maybe a print("Could import ...) comes handy on the except branches
midnight ember
It passes on my local pylint
C:\Users\Devon\Documents\GitHub\Adafruit_CircuitPython_IS31FL3731\adafruit_is31fl3731>black __init__.py
reformatted __init__.py
All done! ✨ 🍰 ✨
1 file reformatted.
C:\Users\Devon\Documents\GitHub\Adafruit_CircuitPython_IS31FL3731\adafruit_is31fl3731>pylint __init__.py
-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 2.41/10, +7.59)
but fails in adafruit actions pylint
I keep submitting it because on my end it says everything is good. I am trying different changes and syntax, going to be a hundred commits before I get it right.
tulip sleet
@midnight ember did you try to use pre-commit for all the changes you were testing
If it's working locally but not on CI, then we need to fix the inconsistency.
i mean pre-commit, not using pylint manually
midnight ember
i get a million CRLF errors
tulip sleet
then that is a git misconfig
spare jacinth
i get a million CRLF errors
you ran pre-commit install on that repo in your machine, right? CI is most likely doing the exact same checks defined in there
midnight ember
C:\Users\Devon\Documents\GitHub\Adafruit_CircuitPython_IS31FL3731\adafruit_is31fl3731>pre-commit run --files __init__.py
black....................................................................Passed
reuse....................................................................Passed
check yaml...........................................(no files to check)Skipped
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
pylint (library code)....................................................Passed
pylint (example code)................................(no files to check)Skipped
pylint (test code)...................................(no files to check)Skipped
do pre-commit run --all at first, and then just run it without the --files arg and it will test the files that have changed
onyx hinge
import typing
import this_import_fails
import typing.Union
except ImportError:
pass
def f(x: Union[list, str]) -> int:
return len(x)
```this also gets a pylint error about `Union` being undefined, because the ImportError fires for `import this_import_fails` and the third import never occurs. This may be happening for the from PIL import line.pre-commit run --files __init__.py didn't do what you wanted, because pre-commit files lists are relative to the top of the repository, not the current directory.
so the problem could be that in the pre-commit pylint environment, PIL or circuitpython_typing, either one, is not actually available to import.
but I am really entering the realm of speculation at this point
spare jacinth
but I am really entering the realm of speculation at this point
same theory i have, so there's that at least 😄
midnight ember
This is why I don't like pre-commit
spare jacinth
WSL is your friend hehe (so is Git For Windows' setting about LF/CRLF)
tulip sleet
This is why I don't like pre-commit
this is because you checked out the file as CRLF, or your editor is doing that. It's a git issue.
midnight ember
the one file i'm working on i've converted to LF. it's only when running all files i get that
onyx hinge
We strongly STRONGLY recommend changing your git configuration so that all files are checked out with pure LF line endings.
many of our tools only work when your environment is configured in this way
midnight ember
ohh i didn't do that for this library. i did that for adafruit_requests.
onyx hinge
do you need help finding documentation on how to do that?
tulip sleet
really, yes, using native Windows as a dev environment causes problems. If you can use WSL or a VM or a Linux machine, you will have an easier time.
It should be a global git setting
spare jacinth
the one file i'm working on i've converted to LF. it's only when running all fi...
git can apply LF<->CRLF changes when you commit/pull code, and it wont matter what the actual file is on your drive, it matters how it looks on git's "database"
tulip sleet
but pylint should be aware then that CRLF is OK
onyx hinge
@spare jacinth except it DOES strongly matter, because tools like pylint check line endings in the working file
tulip sleet
if the file is mixed, that's a problem
midnight ember
😅 i like point and click environment. yes i've done that for my requests repo should be able to manage for this one
peer pressure, mom i'm being peer pressured into using Linux.
tulip sleet
spare jacinth
<@168269895988019200> except it DOES strongly matter, because tools like pylint ...
i meant "does not matter" as in "you should not take that info as granted, since git tree != your drive"
of course it does matter as the tooling is complaining 😛
tulip sleet
does our .pylintrc have expected-line-ending-format=LF??
onyx hinge
@tulip sleet yes that's what gets cookie-cut: expected-line-ending-format=LF
tulip sleet
ok, then people should NOT set core.autocrlf = true in their git config
midnight ember
it's in the .pylintrc in my repo
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
# expected-line-ending-format=
expected-line-ending-format=LF
it explains the options. It does not suggest what to do with our .pylintrc
@midnight ember what editor are you using?
have you installed both Git for Windows and GitHub DesktoP/
I will see what the default option settings are
midnight ember
Mu and Notepad++
I use the device with Mu and then transfer working code to github directories via copy/paste with Notepad++. I can see the line endings in Notepad++ so that I know they're LF.
yeah gotta be Github Desktop doing it when I clone a repo. I swear I thought I fixed that using a global config. 😦
tulip sleet
what is in C:\Users\YOUR_USER_NAME\.gitconfig
https://github.com/desktop/desktop/issues/16522 is of interest. The .gitconfig in your home dir is "global"
as opposed to "system"
lone axle
I seem to be getting this error consistently when I try to run make fetch-all-submodules in a fresh clone of the main repo. Is there some change that I need to make on my end somewhere to get it to be able to complete the fetch?
❯ make fetch-all-submodules
python3 tools/ci_fetch_deps.py all
::group::Sync submodule URLs
git submodule sync --quiet (in /home/timc/repos/circuitpython/circuitpython_copy/circuitpython)
::endgroup::
Duration: 0.6285505659998307
Target: all
Submodules: . frozen
::group::Init submodules (using depth)
git submodule update --init --depth 1 . frozen (in /home/timc/repos/circuitpython/circuitpython_copy/circuitpython)
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'extmod/ulab', but it did not contain eacb0c9af47f85f5d4864b721c3b28661364e8e3. Direct fetching of that commit failed.
fatal:
::endgroup::
Duration: 0.47614098000121885
Traceback (most recent call last):
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 237, in <module>
main(target)
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 219, in main
fetch(submodules)
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 133, in fetch
run(
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 109, in run
subprocess.run(shlex.split(command), stderr=subprocess.STDOUT, check=True, cwd=cwd)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'submodule', 'update', '--init', '--depth', '1', '.', 'frozen']' returned non-zero exit status 128.
make: *** [Makefile:341: fetch-all-submodules] Error 1
maybe the submodules are not fetched by branch/commit but by tags, and you need to fetch-tags?
i must say, i have no idea how submodules work in general, even less knowledge on how CircuitPy handles them
midnight ember
what is in `C:\Users\YOUR_USER_NAME\.gitconfig`
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = true
[core]
autocrlf = input
``` plus my user credsthat's what i "thought" fixed it last time but apparently not. the library i'm working with is a recent checkout after i configured that too.
tulip sleet
hmm, so somehow the LF input file got converted to CRLF's. If Mu didn't do that, not sure what did. I think Notepad++ has a conversion function
midnight ember
yes i've been using notepad++ conversion but if i do it to every file i'm afraid it's going to commit every file.
tulip sleet
git status will tell you if it thinks a file has been changed
and it will be obvious in the PR
midnight ember
actually when i only change the line endings git says there's no change. so might be able to get away with it. that's a lot of manual work though, depending on how many files are in a library.
tulip sleet
In the Git for Windows shell there is a command dos2unix which will convert many files at once
assuming you install the msys2 shell
midnight ember
after manually converting every library file and example file it works as intended
C:\Users\Devon\Documents\GitHub\Adafruit_CircuitPython_IS31FL3731\adafruit_is31fl3731>pre-commit run --all
black....................................................................Passed
reuse....................................................................Passed
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
pylint (library code)....................................................Passed
pylint (example code)....................................................Passed
pylint (test code)...................................(no files to check)Skipped
``` yes i was just thinking i need to find a way to automate thisIn linux you can assume there exists some command-line command to do something like this, and then you find out what it is
because you are not the first person with this (or some other) problem and someone wrote a command for it
midnight ember
i wonder if notepad++ can be integrated with a batch script to do it.
midnight ember
judging from the outcry of CRLF issues filed with Github Desktop and them ignoring it year after year yeah.
i don't know 😅
i don't think so, only thing in apps is github desktop.
tulip sleet
did you install Git for WindowS? https://gitforwindows.org/
midnight ember
i thought running 2 instances is a bad idea? github desktop and git?
slender iron
It is implied by CIRCUITPY_USB in mpconfigport.mk. You'll want to set CIRCUITPY_CONSOLE_UART_TX and CIRCUITPY_CONSOLE_UART_RX in mpconfigboard.h to connect serial to the CP input.
A debugger link a jlink or stlink can be used to see what the CPU is running. You can also add code to turn on an LED from CP to see if it starts up.
fringe yarrow
A debugger link a jlink or stlink can be used to see what the CPU is running. Yo...
I think it's an issue with the spi flash not being one of the supported devices.. trying to cobble together a toml for it from the datasheet now
slender iron
CP will start up without spi slash usually. The drive just won't work. Is the CP code stored on the flash too?
fringe yarrow
Seems like if you don't enable SPI Flash with STM devices it won't even compile.
tulip sleet
i thought running 2 instances is a bad idea? github desktop and git?
it is not two isntances. It is complementary software. "Git for Windows" provides a shell environment that does git.
fringe yarrow
CP will start up without spi slash usually. The drive just won't work. Is the CP...
oh, wait.. i see you have to have ONE of spi or internal_flash_filesystem enabled..
i mistakenly assumed that setting spi_flash_filesystem to 0 would just default to internal flash
slender iron
a debugger will really help you at this point
fringe yarrow
Yeah, I agree.. there's not realyl anywhere to PUT it unless I want to tack it to the smd pins on the mcu..
its a silly project anyway.. i'll probably be done with it if i get no results before lunch 😄
I've added some blink code to the board.c and it's not running it.. I have flashed basic sketches from the arduino ide and gotten expected results, so I know it works like a normal dev board at least in some instances.
midnight ember
Ive been up all night working on typing 2 files. My brain hurts. Will resume this tonight.
proven relic
Don't know if this needed reporting, but I was running res = network.fetch("http://worldtimeapi.org/api/timezone/"+tz) in a loop 6 iterations long. This rarely (one in such loops or so) caused MemoryError: memory allocation failed, allocating 472 bytes. gc.collect() before each iteration solved the problem.
Before gc.collect(). The number after the "Retrieving data..." is gc.mem_free()
Retrieving data...4768
Retrieving data...2272
Retrieving data...1360
Retrieving data...10624
Retrieving data...1008
Retrieving data...Traceback (most recent call last):
File "code.py", line 246, in <module>
MemoryError: memory allocation failed, allocating 472 bytes
After gc.collect() before network.fetch:
Retrieving data...12512
Retrieving data...12512
Retrieving data...12160
Retrieving data...11968
Retrieving data...11888
CircuitPython 8.2.10 on 2024-02-14; Adafruit Matrix Portal M4 with samd51j19
tulip sleet
Don't know if this needed reporting, but I was running `res = network.fetch("htt...
RAM is tight on the Matrix Portal, so this is a good idea.
... The github link is a 404 for me. Is it a private repo?
proven relic
Sorry, should be public now
slender iron
multitasking is so complicated
onyx hinge
seems like maybe safari doesn't support mdns names, but who knows why. I found various references to this, like this unanswered stackexchange .. https://apple.stackexchange.com/questions/396073/why-cant-safari-connect-to-server-local-when-firefox-and-chrome-can-is-mdns and this unhelpful document https://support.apple.com/en-us/101903
slender iron
seems like maybe safari doesn't support mdns names, but who knows why. I found v...
doesn't it work with cp-MAC.local but not circuitpython.local?
crimson ferry
doesn't it work with cp-MAC.local but not circuitpython.local?
yes
and it works with espressif
timeouts and times-to-load vary between curl and safari, but results essentially similar
manic glacierBOT
lone axle
When was the most recent mpy format change? circup currently reports this:
The following modules are out of date or probably need an update.
Major Updates may include breaking changes. Review before updating.
MPY Format changes from Circuitpython 6 to 7 require an update.
I'm in there doing some refactoring, I could change the versions listed here to be more relevant. Was 8 to 9 the last one that required an update?
tiny peak
@slender iron OrangeCrab port update: Board came yesterday. Today I mounted it to a universal plate for easier handling, did a smoke test, and started a github repo to document progress on my ECP5 RISC-V SoC quest. Repo is at https://github.com/samblenny/ocfpga
midnight ember
:arm-twist: 🙂
Yes I definitely got my arm twisted into installing git bash. OK OK it's installed.
I also came up with a powershell script that replaces \r\n with \n in every .py file in a directory (and child directories). All you have to do is name the file AutoLF.ps1 ```
Option 1: Run in the same directory as this file (drag and drop)
$folderPath = $PSScriptRoot
Option 2: Set the folder path manually
$username = $env:USERNAME
$folderPath = "C:\Users$username\Downloads\AutoLF_Test"
Get a list of .py files in the folder and its subdirectories
$pyFiles = Get-ChildItem -Path $folderPath -Filter *.py -Recurse
Write-Host "Converts line endings from CRLF to LF so pre-commit will not throw thousands of errors"
Write-Host "-------------------------------------------------------------------------------------"
Loop through each .py file
foreach ($file in $pyFiles) {
# Read the content of the file
$content = Get-Content -Path $file.FullName -Raw
# Replace CRLF line endings with LF
$content = $content -replace "`r`n", "`n"
# Write the modified content back to the file
Set-Content -Path $file.FullName -Value $content
Write-Host "Line endings converted to LF for $($file.FullName)"
}
Keep the PowerShell window open (remove the entire section below to automatically close the window instead)
Write-Host "Press any key to exit..."
$null = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
Nice!
Will look more tomorrow
midnight ember
The bad thing about this method is it will replace all instances of \r\n which I think the RFM library and others have hard coded into their examples for newlines in a send/receive bytearray. :/
spare jacinth
The bad thing about this method is it will replace all instances of \r\n which I...
Having a string with "\r\n" is quite different than having an actual CRLF
In memory they would end up being
ord("\\"), ord("r")is something like33 45(2 bytes)ord("\r")would be a single one, lets say9
Or course i made up the numbers instead of looking at the ASCII table 👍
manic glacierBOT
manic glacierBOT
midnight ember
Having a string with `"\r\n"` is quite different than having an actual CRLF
Using ordinal values now, thank you.
spare jacinth
Using ordinal values now, thank you.
Phone + video moving fast = no clue what you did
I didnt mean you should use ord, but that you didnt need to change anything
Code looking for "\r\n" wont get reformatted by LF vs CRLF utilities
midnight ember
One can hope. Yeah on a mobile device it's probably not going to show up as well. It's a powershell script that installs pre-commit to a local github repo (will not install if already installed), then formats every .py file in the directory and child directories (such as \examples), then runs black on every file, the runs pre-commit --all-files on the main directory. This is a workaround for Github Desktop converting all files to CRLF on checkout. Run once on a directory and I'm all set to go.
It prevents this situation in Windows where Github Desktop checks out every file with CRLF line endings which fails horribly with pre-commit. I won't have to deal with this situation again.
Here's the final AutoLF.ps1 powershell script
spare jacinth
Yeah, but my understanding was that you were worried about code using "\r\n" to check for new lines in UART messages or the like
And i explained why that wouldnt get changed
midnight ember
yes, i only thought about that after I'd run it a couple time and then remembered that it might be hardcoded into some scripts.
i misunderstood you. :/ welp it's using ASCII.GetBytes now.
spare jacinth
Re read my prev message and try to understand it, i guess
midnight ember
it would not work with the pure ordinal values, powershell did not like that format. ended up having to go with different formatting.
i tried a lot of different variations and while \r\n was definitely easiest the way you phrased it I thought it might not be checking for actual CRLF. It's my understanding that CRLF is just \r\n anyway?
spare jacinth
My whole point is that a CRLF in a file (two invisible chars) is different than a string with \r\n (4 visible chars)
And that, if you had code like
while "\r\n" not in read_data()
It wouldnt get reformatted
midnight ember
I'm not even sure how it would know the difference though. I'm not seeing any logic that is checking for the last characters in a line. It seems just to check for any instance of \r\n.
Yeah I honestly don't know how it can see that they're the EOL characters but it does work, as far as I can tell. I haven't done extensive testing on it yet.
spare jacinth
The actual CRLF, in disk, would be two bytes (what i showed with ord, you can run that in python to see the actual values), so something like 8, 9
The string looking for them would be 4 bytes (slash, r, slash, n), so something like 20, 45, 20, 10
The utility looking for CRLF (8,9) would not change the (20, 45, 20, 10) strings
Again, i made the numbers up
midnight ember
I tried that formatting and powershell didn't like it. I can try again later. Will look into it.
yeah i get what you're saying.
spare jacinth
No idea what you mean with format/powershell
But i hope you have a better understanding now 🤓
midnight ember
I appreciate your advice!
The way the script looks for .py files recursively is done through windows itself using powershell, not python. so the way powershell interprets line endings might be different.
the script kind of weaves in and out of powershell and python scripting for different things.
spare jacinth
The bad thing about this method is it will replace all instances of \r\n which I...
This is what i was replying to
spare jacinth
And that, if you had code like
`while "\r\n" not in read_data()`
It wouldnt get ...
Those scripts have something similar to this, dont they?
lone axle
It's only the absolute basics at this point, but it does seem to successfully execute the test script on a device and report a Pass / Fail based on whether the output matches the expected results. https://github.com/FoamyGuy/CircuitPython_OnDevice_Testing
spare jacinth
It's only the absolute basics at this point, but it does seem to successfully ex...
looks cool, some feedback if you dont mind:
- could make
expectfileoptional and default it tof"{testfile}.exp" - dont mix the use of different logging utils (
printandclick.echo), always use the same so you dont get the chance to output those into different streams and cause yourself some confusion- i would probably use
loggingto emit messages of different levels and filter them depending on how bad something has gone (...or some other utility that also provides such capability)
- i would probably use
and you probably thought about that too, but a yaml/json/whatever listing board_name: [tests_to_run_on_it] + parsing thru it sounds like a good idea moving forward 😄
lone axle
PR's welcome 🙂 Defaulting sounds nice just a QoL feature I didn't make at first and didn't get to. Print vs. echo there are still some strewn about inconsistently because I was having a hard time figuring out why the test was failing when the output looked identical visible, it turns out echo does some additional processing and print allowed me to see "real" values that helped me fix it, I just didn't go back and change everything yet.
logging I'm not opposed to, I just don't tend to gravitate towards it by default when I write.
spare jacinth
given this is a test suite, it probably makes sense to print everything unconditionally, but it could become difficult parsing through the log to see exactly which test failed and/or where
lone axle
I definitely think some kind of summarized output would be necessary if it evolves to test multiple things.
spare jacinth
or... could make the scriot stop as soon as a test fails, but i'd rather run everything and have full info
lone axle
I've already noticed that printing the actual and expected values makes it scroll kind of far to where you may not still see the pass fail indicator line without scrolling back. I'll probably still manipulate the way it outputs quite a bit this is definitely just first draft that I got working last night and this morning.
spare jacinth
f"{board}_{test}.err" sounds like a good idea for failing tests
spare jacinth
PR's welcome 🙂 Defaulting sounds nice just a QoL feature I didn't make at first...
will try and do a couple things soon, dont start commiting changes 😛
lone axle
I won't be touching anything on it until at least later tonight 8+ hrs likely.
midnight ember
It's only the absolute basics at this point, but it does seem to successfully ex...
run_test looks pretty wild. I don't recognize any of that. Is that Python? Ahh requires discotool. Neat.
lone axle
run_test looks pretty wild. I don't recognize any of that. Is that Python? Ahh ...
yes everything except click is "vanilla" python i.e. modules built-in to CPython. click is a command line interface helper library that does a bunch of nifty helpful stuff if you're making a CLI.
and yep, discotool was the quickest / easiest way I could think of to find the attached devices file location and serial port. Possibly it could be integrated at the python level instead of using subproccess to run it as a command but this was definitely quick and easy 😄
midnight ember
"easy"
easy is relative. what takes you an hour would probably take me months.
Submitted a new API PR. Good excuse to try out my new formatting tool in a real world use. It worked great! Too great. Pre-commit ran on the entire repo which I wasn't expecting. It didn't do just child directories it went recursively up parent directories too. That would have been really really bad if it was a larger library. Still threw thousands of CRLF errors for files I didn't want it to check. :/
I guess that's how pre-commit on an entire repo is supposed to work? Even though I ran it in /examples/wifi/expanded.
Might be because I'm used to doing pre-commit run --files filename instead of pre-commit run --all-files
lone axle
That is true, perhaps 'lazy' would have been a better word than easy. I just meant I didn't want to write the code that figures out the differences between Windows, Linux, and Mac to find the CIRCUITPY drive location and serial port. Discotool has already done that hard work so I opted for the lazy approach of just use it from there.
"Easy" is interesting and relative like that though, which I find fascinating. I've been programming as a full time job for a decade (😬) and I've definitely spent many months in that time grinding away at stuff. Even with that there are plenty of folks both in and outside of the community that make plenty of other things look easy that would no doubt take me months or longer. There are so many things out there and so little time to learn about them.
midnight ember
You are anything but lazy. 😉
lone axle
I'm not sure about pre-commit on specific files, I've always done pre-commit run -a or --all-files I do think it's primarily intended to operate on the whole repo, but there may be more percise controls available
midnight ember
I started running it only on the file I was working with in an effort to surpress a lot of the EOL errors. Probably where that stemmed from. Now I can focus on maybe doing everything the right way. Github Desktop does kind of throw a unique wrench into the workflow compared to any other GIT tool out there. Just gotta roll with the punches.
I did install Git for Windows (GIT SCM) so maybe I can start to transition to a different GIT tool and slowly learn Git CLI stuff.
tulip sleet
I'm not sure about pre-commit on specific files, I've always done `pre-commit ru...
I usually just let it run automatically when I do a commit. It will check the changed files only.
lone axle
I tried like that with pre-commit install before but something about the way it integrated into my commit workflow bothered me and I ended up just building the habbit to run it manually. I should give it another try though, I don't even recall the specifics about it.
tulip sleet
@slender iron https://github.com/adafruit/circuitpython/issues/8886 is on the 9.0.x. This is Have to Unplug / Re-Plug USB Host Keyboard. Are you awaiting something from TinyUSB? I'm not sure based on the comments. I could make a 9.0.3 - only fix is to nrf AnalogIn.
lone axle
It's not directly related, but I wish there was a way we could make reuse enforced on files that are actually checked in to the repo. I find myself with other files in my local copy that aren't checked in but cause pre-commit to fail from reuse. It then succeeds in the github actions since those files aren't checked in.
onyx hinge
It's not directly related, but I wish there was a way we could make reuse enforc...
yes this is an irritation for me as well. Sometimes, if I get the same kinds of files in many repos, I'll add them to my personal, global gitignore files (editor backup files, for instance).
lone axle
Ahh, I forget about that global gitignore. Maybe I can commit to using a naming prefix convention for my non-added files that is ignored by that global file.
spare jacinth
my idea of doing a PR pretty much ended in a rewrite 🙃
just run python ondevice_testing from root of the folder (note, havent tested it)
https://github.com/elpekenin/CircuitPython_OnDevice_Testing
TODO?: accept a single argument to point the code at another YAML file
tulip sleet
A somewhat different testing thing I was working on but didn't take very far: https://github.com/dhalbert/Adafruit_CircuitPython_Port_Test
lone axle
Is there something extra needed to fetch submodules now? I'm testing in a fresh clone fully updated and this error prevents make fetch-all-submodules from completing successfully:
git submodule update --init --depth 1 . frozen (in /home/timc/repos/circuitpython/circuitpython_copy/circuitpython)
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'extmod/ulab', but it did not contain eacb0c9af47f85f5d4864b721c3b28661364e8e3. Direct fetching of that commit failed.
fatal:
::endgroup::
Duration: 0.34272488499846077
It seems like something to do with a specific commit from ulab failing to be fetched.
tulip sleet
Is there something extra needed to fetch submodules now? I'm testing in a fresh ...
that looks like some temporary problem
I would recommend that you add the git PPA if you are using ubuntu, as that will bring in more modern versions of git, including "partial clones", which are easier to deal with.
lone axle
I've had it consistently since yesterday. I've tried pulling to make sure everything is up to date and remove-all-submodules + re fetch again a few times as well and always same result so far.
lone axle
Ah, I do have git version 2.34.1 I'll try adding that ppa and update
tulip sleet
lone axle
updated to 2.43.2 removed all again and re-fetching now 🤞. I wonder if something could have changed with the repose or submodules that could cause this newer version to be a requirement instead of a recomendation. I know I was able to fetch them successfully even fairly recently but all the sudden it started getting stuck there.
On this version it got a similar looking error, but this time referencing a different one of the submodules not ulab:
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'ports/broadcom/firmware', but it did not contain bf96d0eda5952595d717fedb797aeb168483e9fa. Direct fetching of that commit failed.
::endgroup::
Duration: 96.18597572200088
Traceback (most recent call last):
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 237, in <module>
main(target)
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 219, in main
fetch(submodules)
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 127, in fetch
run(
File "/home/timc/repos/circuitpython/circuitpython_copy/circuitpython/tools/ci_fetch_deps.py", line 109, in run
subprocess.run(shlex.split(command), stderr=subprocess.STDOUT, check=True, cwd=cwd)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'submodule', 'update', '--init', '--filter=blob:none', '.', 'frozen']' returned non-zero exit status 128.
make: *** [Makefile:341: fetch-all-submodules] Error 1
I'll try a completely fresh clone with this new version of git. I'm still on the one that I cloned yesterday with the old version.
tulip sleet
On this version it got a similar looking error, but this time referencing a diff...
I just made an absolutely fresh clone of my repo and did make fetch-all-submodules, and it worked. hmm
lone axle
completely fresh clone for me fails with another similar error but on another different one of the submodules:
Submodule path 'ports/atmel-samd/peripherals': checked out '82e514b6e0d1a2b09dc73be9973663b6b837a817'
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'ports/broadcom/firmware', but it did not contain bf96d0eda5952595d717fedb797aeb168483e9fa. Direct fetching of that commit failed.
::endgroup::
Duration: 89.00782027999958
Well thats no good I wonder what is broken inside of my environment 😟
tulip sleet
completely fresh clone for me fails with another similar error but on another di...
are you using ssh or https for your fetches in general?
lone axle
https I believe I clone the repo with a command like:
git clone https://github.com/adafruit/circuitpython.git
is there anything unusual in your ~/.gitconfig? Anything about protocol.file.allow?
https://www.githubstatus.com/ looks fine
lone axle
I started trying to run the command that subprocess is reporting as failing directly instead of via make. I found that adding a --remote to the command so it's:
git submodule update --init --filter=blob:none --remote . frozen
seems to allow it to at least git farther than before, it's still running now. I found this here: https://stackoverflow.com/a/56621295/507810 but don't fully understand it honestly.
lone axle
is there anything unusual in your `~/.gitconfig`? Anything about `protocol.file....
nothing about protocol.file.allow that I can tell:
❯ cat ~/.gitconfig
[credential "https://github.com"]
helper =
helper = !/usr/bin/gh auth git-credential
[credential "https://gist.github.com"]
helper =
helper = !/usr/bin/gh auth git-credential
[user]
email = foamyguy@gmail.com
name = foamyguy
[core]
autocrlf = input
maybe the gh utility messed it up somehow. I have that installed and use it for PR reviews. Not really with fetching submodules or anything but maybe it's interfering or something.
slender iron
<@252717193496756235> <https://github.com/adafruit/circuitpython/issues/8886> is...
I don't think hathach could reproduce it. I definitely still see it happening though
tulip sleet
I don't think hathach could reproduce it. I definitely still see it happening th...
so maybe move to 9.x.x?
slender iron
yup, that's fine
tulip sleet
nothing about `protocol.file.allow` that I can tell:
```
❯ cat ~/.gitconfig
[cr...
mine is pretty much a superset of that
lone axle
after adding the --remote it did make it much farther and now has a different error:
❯ git submodule update --init --filter=blob:none --remote . frozen
fatal: Unable to find refs/remotes/origin/v4.2.1 revision in submodule path 'ports/silabs/gecko_sdk'
I don't have autocrlf = input, but that seems kinda irrelevant
is git-lfs installed?
lone axle
I don't think I've ever used silabs port, I'm not sure which hardware that is. If that is truely the only thing I'm missing now though I probably don't need it.
lone axle
is git-lfs installed?
yep,
$ git-lfs --version
git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
go ahead and install it, it doesn't hurt
i have the same versions as you
how old is your gh?
i have 2.46.0
lone axle
Got to head out for now though. I'll attempt a build with this one later.
$ gh --version
gh version 2.25.1 (2023-03-21)
https://github.com/cli/cli/releases/tag/v2.25.1
oof looks like I'm behind a decent ways, I'll update that too.
tulip sleet
I have a ppa for that too
slender iron
we're definitely seeing a slight bug influx as folks move to 9
tulip sleet
we're definitely seeing a slight bug influx as folks move to 9
I agree, though I am seeing about half of them either be not an actual bug or not introduced in 9.0.x
midnight ember
we're definitely seeing a slight bug influx as folks move to 9
I think that's a good thing. 🙂
slender iron
Ya, I think so too. I'm just thinking about my prioritization of work
midnight ember
I agree, though I am seeing about half of them either be not an actual bug or no...
Yeah a lot of user error reported issues out there 😅
bright robin
So, I've been playing around with the Cardputer keyboard (M5Stack Card Computer).
#help-with-circuitpython message
I was thinking of trying to get the keyboard working in the built in REPL
I'd be perfectly happy to write the code but I'm not sure whether it's practically possible or within the scope of the project (I haven't heard of any similar boards.).
Is this a feature that could be realistically implemented?
slender iron
So, I've been playing around with the Cardputer keyboard (M5Stack Card Computer)...
It isn't too bad to connect into CP internally. It should be doable.
It uses keypad demux to read the keyboard right?
bright robin
Yeah that's it. I built a little table of all the possible keycodes.
slender iron
You could make board_ variants of these weak functions: https://github.com/adafruit/circuitpython/blob/main/supervisor/shared/serial.c#L110-L131
bright robin
Well that raises a lot of questions. How would I go about getting keyboard scanning to happening in the background? How do keystrokes get passed to CircuitPython's internals?
slender iron
and then implement them in board.c
bright robin
Oh I see what you mean.
slender iron
keypad already works in the background. so you basically initialize it in board.c the same way you would from python
bright robin
Um, what's the difference between port_serial_early_init and port_serial_init ?
slender iron
I think early is before usb starts
bright robin
So would I want to initialise it in the early version?
bright robin
I'm a little worried if it's too early half the system will be in some kind of non-functional state.
slender iron
serial_workflow_start() calls serial_init()
ya, I'd wait in case you want to add a configuration knob into boot.py
walk time. back in a bit
bright robin
No problem. I'll look through these shared bindings.
forest badge
Is there possibly a preliminary sketch to look at regarding the INA228 library or learn guide?
slender iron
@onyx hinge did you enable the panic handler in IRAM for protomatter?
that helped me debug protomatter
onyx hinge
@slender iron this is about Arduino so, probably no
I found documentation on setting debug level and that's it. I think it's just the IDF debug level but not sure.
slender iron
kk
I don't know what IDF settings they use for arduino
I had issues with the gptimer being when IRAM is set
but I suppose you aren't writing to flash anyway
bright robin
So if I'm getting this right I can just define board_serial_read in the board code and that will be called automatically?
And am I correct in assuming that all the background work like creating queues will be automatically done if I call common_hal_keypad_demux_demuxkeymatrix_construct and I can just check the keypad queues whenever a read is requested?
https://github.com/adafruit/circuitpython/blob/436f9829f85e19019b7a51a3bfe849fabf1b29a7/shared-bindings/keypad_demux/DemuxKeyMatrix.h#L35C6-L35C54
Or is there something else that needs doing?
tulip sleet
So if I'm getting this right I can just define `board_serial_read` in the board ...
yes, the constructor will construct the queue, as it would when invoked in CircuitPytyhon
slender iron
Make weak versions in the file I linked to and a non-weak version in board.c
The board.c one will take precedence
bright robin
So you're saying I just need to define a non-weak version with the same name in the relevant board.c file?
So the name should actually be port_serial_read then and not board_serial_read?
slender iron
So you're saying I just need to define a non-weak version with the same name in ...
Correct. Add a board_ version just like there is a port_ version.
port_ is already used in esp for the internal usb to serial converter
bright robin
Yeah, no problem then.
So will the internal usb to serial converter still work if I define board_serial_read ?
Will my code need to handle that somehow?
slender iron
ya, you'll need to add it to serial_read (iirc) just like the port version
bright robin
Now possible a silly question, but what what the correct way to write the new firmware to the device again?
Would this work for the ESP32S3?
(assume the port is correct)
esptool.py -p /dev/ttyACM0 write_flash 0x1000 build-m5stack_cardputer/firmware.bin
Ah, nevermind. Works now I just have the offset wrong.
bright robin
Would this new version of 'supervisor/shared/serial.c' be okay? I've basically added board_ versions of all the port_ prototypes and duplicated all the calls to port_ functions (header changes are also made). I haven't actually modified any internals before and I don't want to cause any issues.
orchid basinBOT
slender iron
Now possible a silly question, but what what the correct way to write the new fi...
There is a flash make target you can use too
slender iron
Would this new version of 'supervisor/shared/serial.c' be okay? I've basically a...
Make a PR. It is way easier to review
bright robin
Make a PR. It is way easier to review
Fair enough. I'll probably submit the whole thing together. I'm getting any events yet though so still a WIP.
slender iron
sounds good. if you have regular usb you can do mp_printf(&mp_plat_print, <format string>, <format arguments>) to print out stuff
bright robin
Ah that might help.
slender iron
you can also do a DEBUG=1 build and use a usb-to-serial converter on the output
that'll give you IDF logging
bright robin
Is there any useful info in there?
slender iron
only if you are interacting with the IDF or see other ESP_LOG statements you want to see
bright robin
So I made this constructor for keypad_demux:
common_hal_keypad_demux_demuxkeymatrix_construct(
&keyboard, // self
3, // num_row_addr_pins
row_addr_pins, // row_addr_pins
7, // num_column_pins
column_pins, // column_pins
0.01, // interval
20, // max_events
1 // debounce_threshold
);
And later check if with:
common_hal_keypad_eventqueue_get_into(keyboard.events, &event)```
But there's no events at all (returns false), when it works perfectly fine with CircuitPython code:
```python
kbd = keypad_demux.DemuxKeyMatrix(
row_addr_pins = (board.KB_A_0, board.KB_A_1, board.KB_A_2),
column_pins = (board.KB_COL_0, board.KB_COL_1, board.KB_COL_2, board.KB_COL_3, board.KB_COL_4, board.KB_COL_5, board.KB_COL_6)
Is there something I need to do to... tell it I want events, beyond just constructing an object?
slender iron
So I made this constructor for keypad_demux:
```C
common_hal_keypad_demux_de...
You may need to prevent the pins from being reset
either from calling a never_reset function or adding a board reset function that doesn't change the specific pins
you can comment out the pin reset in port_reset in supervisor/port.c to see if that's the issue
bright robin
I'll give it a go.
I'm not sure when the reset happens. Is it after board/port_serial_init?
slender iron
it does between every vm run
we're moving away from the bulk resets but not done yet
bright robin
Doesn't seem to make much difference. I'll keep debugging I suppose.
bright robin
Okay, I took another look and it seem that the device stops working as soon as the default program ends. So I'm guessing the supervisor is doing a deinit on it to clean up after running the program? I'm not sure how to prevent that from happening.
bright robin
Ah, here it is:
https://github.com/adafruit/circuitpython/blob/50ad62d205c9f024ee037de48cb7110c47fe6260/main.c#L386
All keypads get deregistered at the end of a program. I'll have to add some kind of never_reset function.
Just having it on the pins doesn't help because the scanner is deregistered.
bright robin
Alright, the Cardputer's built in keyboard is now working fine after some modifications to the board and matrix code. Control keys, shift, alt, arrow keys, etc. Nice little portable terminal now.
orchid basinBOT
tulip sleet
https://blog.adafruit.com/2024/04/04/circuitpython-9-0-3-released/
The only changes to 9.0.3 from 9.0.2 are a fix for AnalogIn on Nordic nRF boards, and the addition of the Waveshare RP2040-GEEK board.
slender iron
Ah, here it is:
<https://github.com/adafruit/circuitpython/blob/50ad62d205c9f024...
Instead of adding never_reset, the bulk reset should be removed and all of the keypads should have a finalizer to turn them off when the heap is deinit. It'll skip yours because it isn't on the heap.
thorny jay
Project idea: Build a Copilot key in CircuitPython (F23 seems supported... https://docs.circuitpython.org/projects/hid/en/5.1.1/api.html#adafruit_hid.keycode.Keycode.F23): https://arstechnica.com/gadgets/2024/04/shoehorned-windows-copilot-key-is-just-a-reprogrammable-macro-journalist-shows/
slender iron
@tulip sleet trevor found a bug between uf2 0.8.2+ and cp
I think it impacts 8.2.x as well
midnight ember
Project idea: Build a Copilot key in CircuitPython (F23 seems supported... https...
That's what a modern help button should do.
thorny jay
That's what a modern help button should do.
The project idea is for anybody to take, implement and document. I guess I know how to do it, but I have no clue how I would test that as I have no plan to use Copilote anytime soon. 😉
tulip sleet
<@329766224093249548> trevor found a bug between uf2 0.8.2+ and cp
did you open an issue in the nrf5t2 bootloader or is that not necessary?
slender iron
did you open an issue in the nrf5t2 bootloader or is that not necessary?
I didn't. I just pinged thach. For a moment I thought it'd only be a CP fix
tulip sleet
@slender iron I'm choosing what to work on next in the 9.x.x issues list. Are you planning to do the reset/finalizers stuff, or do you want to pass that off? Also this socket timeout issue maybe should be addressed: https://github.com/adafruit/circuitpython/issues/8266
I am less confident about all the storage/sd-card issues.
slender iron
I'm not planning on doing reset/finalizers
I'm in the max3421 weeds now. it uses spi from an interrupt 😕
tulip sleet
I'll start on the finalizers, not necessarily first thing. If I have questions I'll of course talk to you.
slender iron
the mqtt while web workflow issue seems weird
bright robin
Hopefully that covers everything!
I might want to do a bit more work later on getting the ^C key working. At the moment it is working in REPL, but it won't actually stop a running program unless it is waiting on user input.
Anyone know how the serial port input does this? I'm still figuring it out.
On another note it might be good to have an option to run a callback function from the keypad scanner when new events become available.
At the moment I'm just calling my function to process key events from board_serial_read or board_serial_bytes_available when they are called.
bright robin
Ah crap... the _serial_bytes_available prototype just changed didn't it?
tulip sleet
Ah crap... the `_serial_bytes_available` prototype just changed didn't it?
It is going to change in this PR: https://github.com/adafruit/circuitpython/pull/9141
bright robin
Ah, that must of what I was thinking of.
tulip sleet
On another note it might be good to have an option to run a callback function fr...
checking for the interrupt char on, say, an ESP32, is done by scanning the UART interrupt fifo. See ports/espressif/supervisor/usb_serial_jtag.c. If it finds a char == mp_interrupt_char, it calls mp_sched_keyboard_interrupt().
maybe you could check for mp_interrupt_char in the routine that queues the keyboard presses for serial input?
bright robin
maybe you could check for `mp_interrupt_char` in the routine that queues the key...
The problem is I don't get any kind of callback when an event is put in the event queue.
I'm just processing the events to generate keystrokes when the system requests input through board_serial_read.
So it's pointless to look for mp_interrupt_char if the character isn't read until input is requested.
What I was thinking of was to have the keypad scanner call a previously given processing function after it detects changes.
Maybe putting something here:
https://github.com/adafruit/circuitpython/blob/50ad62d205c9f024ee037de48cb7110c47fe6260/shared-module/keypad_demux/DemuxKeyMatrix.c#L151
tulip sleet
The problem is I don't get any kind of callback when an event is put in the even...
you could conditionally compile something there for boards that are taking REPL input from a keypad. Or something could scan the event queue later, as the serial_jtag code I mentioned above does. It's not expensive to scan the keypad queue: it's not that long
could be scanned by a background task. I haven't been following this that closely. I have to sign off to sleep. I'm too tired to think hard about this. If you put your code in a repo, we can look later
bright robin
I'd probably still want it to have a callback link back to the board file as there would still need to be some kind of keypad specific processing done to generate the keystrokes - what event sequence actually corresponds to mp_interrupt_char depends of the keypad. Sorry to keep you up so late.
tulip sleet
I'd probably still want it to have a callback link back to the board file as the...
🙂 I wasn't complaining about the sleep, just explaining. I see what you mean about a callback now. I thought you meant a callback to Python, which is not a good idea in this context.
I didn't realize you had a PR open already. Thanks.
lone axle
Should Blinka_Displayio get backwards compatibility for displayio.FourWire added back to it? I think right now we are in a state where displayio.FourWire does work on CircuitPython core (without the warning message as well now). But that does not work on Blinka_Displayio and it looks like fourwire.FourWire must be used there.
tulip sleet
Should Blinka_Displayio get backwards compatibility for `displayio.FourWire` add...
We discussed that briefly in the PR: https://github.com/adafruit/Adafruit_Blinka_Displayio/pull/131. If the Learn Guides describe the new API, then I think an abrupt change is OK. I am not sure in general whether the major versions in Blinka should follow the "old - old&new - new" pattern of introducing an incompatible API that we use in CircuitPython core. It is not so hard to backtrack to a previous version with pip. @gilded cradle do you have an opinion?
lone axle
Okay, I did find this page: https://learn.adafruit.com/running-pyportal-code-on-blinka-with-displayio/running-examples that has displayio.FourWire in a code snippet. I did not think to seek out Blinka_DisplayIO guides seperately from core displayio ones when I went through learn guide code and pages recently so unless anyone else went through to update pages in guides for Blinka_DisplayIO there may be others that still have the old way showing.
misty garnet
I have a W550-EVB-PICO publishing a MQTT message to an rasppi5 broker once every second. It plugs away happily for hours/days at a time. However, if I ssh to the rasppi5, open minicom so I can see the pub messages that are printed to the REPL, and then close minicom or just terminate the ssh session, the PICO stops publishing after a few seconds. If I open up minicom again shortly after, it will resume MQTT publishing. If I wait a bit longer (~minute-ish) it won't resume publishing, and if I interrupt with a Ctrl-C, it shows it stuck here: File "/lib/adafruit_wiznet5k/adafruit_wiznet5k.py", line 532, in socket_available. Is there something about connecting/disconnecting from the serial that interferes with socket operations?
fringe yarrow
I promise this is my last Sunton ESP32 device for a while 😄
tulip sleet
I have a W550-EVB-PICO publishing a MQTT message to an rasppi5 broker once every...
Please open an issue about this. It does sound like a bug.
misty garnet
Please open an issue about this. It does sound like a bug.
sure, no problem. what's the proper repository to raise the issue?
tulip sleet
https://github.com/adafruit/circuitpython because even though it fails in wiznet, I don't thin the library is the issue.
gilded cradle
We discussed that briefly in the PR: <https://github.com/adafruit/Adafruit_Blink...
As you mentioned, since displayio is a separate library with Blinka, they can just go back to a previous version. It's more integrated with CP, so that's a bit trickier.
@lone axle @tulip sleet, so, I think leaving the Blinka_Displayio lib as is for now is the correct move.
wraith crow
I promise this is my last Sunton ESP32 device for a while 😄
I may have to try building the makerfabs board with the I2S audio to see if it works now 🤷
bright robin
🙂 I wasn't complaining about the sleep, just explaining. I see what you mean ab...
Do you have a specific way of doing callbacks, or can I just pass a function pointer?
manic glacierBOT
bright robin
Instead of adding never_reset, the bulk reset should be removed and all of the k...
I don't really understand how finalisers work, is there any example I could use?
tulip sleet
Do you have a specific way of doing callbacks, or can I just pass a function poi...
No special callback techniques. Just define a type for the callback and then declare the routine with it.
bright robin
No special callback techniques. Just define a type for the callback and then dec...
Thanks! The callback seems to be working now, Ctrl-C on the board's keypad can now interrupt a running program.
I've also replaced my custom circular buffer with ringbuf as you wanted, and modified my board_serial_bytes_available to return uint32_t in line with the new API changes.
All in the pull request. https://github.com/adafruit/circuitpython/pull/9145
Though it still seems to have a conflict with serial.c that I'm not sure how to resolve.
midnight ember
There seems to be circuit python support for the C3 and C6 now. Time to update the product listings? For example, as of the time of this writing, there is no CircuitPython support - only Arduino and ESP IDF! https://www.adafruit.com/product/5337
manic glacierBOT
orchid basinBOT
fringe yarrow
oh jeez.. now i've made work..
orchid basinBOT
manic glacierBOT
manic glacierBOT
@RetiredWizard It is working now ( at least the size ) , color are still off but text and pics fits now , I used your commands with only width=240,height=320 swapped
display = adafruit_ili9341.ILI9341(bus,width=240,height=320,backlight_pin=board.LCD_BCKL)
I will continue to experiment to fix the colors mismatch which i assume is okay with your board , appreciate th...
manic glacierBOT
I could not add it here but it is just a bmp ( 24 bit bitmap ) file i made with paint to calibrate the display , the same way worked with TTGO display
If you just load a picture with green , red and blue then that would do the same
manic glacierBOT
Adding invert=true give me TypeError: unexpected keyword argument 'invert'
tulip sleet
@proven garnet hi, could you add adabot as a maintainer to https://readthedocs.org/projects/circuitpython-nau7802/. Otherwise I can't add a readthedocs hook. Thanks. Not urgent.
proven garnet
<@135415127879974912> hi, could you add adabot as a maintainer to <https://readt...
Invitation sent!
tulip sleet
Invitation sent!
thanks!
tulip sleet
@lone axle @proven garnet @gilded cradle I just made a manual pass through all the webhooks for the CircuitPython libraries. I had discovered several that had not been updated with "secrets" for ReadTheDocs. I added secrets as necessary (maybe about 10% needed it). A few had multiple RTD webhooks, maybe from failed attempts. A few had the wrong repo (someone's personal repo). I had to re-create the subjprojects and added " x" to their name, because the name couldn't be the same.
A few projects have the wrong formatting style. I should have written those down, but I didn't.@gilded cradle that was true of a few blinka libraries, and in some, the "Download" links were pointing to nonexistent repo URLs, like Adafruit_CircuitPython_Blinka_something. So maybe worth looking at the RTD set up files for the blinka repos.
also there were some dead travis webhooks, etc. I deleted
bright robin
Is there anything that needs to happen to my keyboard pull request (9145) before it can be merged? I'm genuinely not sure. Or do I just need to show a little patience? 😆
bright robin
Hmm, I'm getting some error when I try to build a new ESP32 board frozen modules:
-- Configuring done
-- Generating done
-- Build files have been written to: /home/joshua/src/circuitpython-watchy/ports/espressif/build-sqfmi_watchy/esp-idf
QSTR not updated
Module registrations not updated
Root pointer registrations not updated
In file included from ../../py/reader.h:29,
from ../../py/lexer.h:33,
from ../../py/parsenum.h:30,
from ../../py/runtime.c:34:
../../py/runtime.c: In function 'mp_init':
../../py/runtime.c:153:53: error: 'MP_QSTR__dot_frozen' undeclared (first use in this function); did you mean 'MP_QSTR_root_group'?
153 | mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__dot_frozen));
| ^~~~~~~~~~~~~~~~~~~
../../py/obj.h:221:56: note: in definition of macro 'MP_OBJ_NEW_QSTR'
221 | #define MP_OBJ_NEW_QSTR(qst) ((mp_obj_t)((((mp_uint_t)(qst)) << 4) | 0x00000006))
| ^~~
../../py/runtime.c:153:53: note: each undeclared identifier is reported only once for each function it appears in
153 | mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__dot_frozen));
| ^~~~~~~~~~~~~~~~~~~
../../py/obj.h:221:56: note: in definition of macro 'MP_OBJ_NEW_QSTR'
221 | #define MP_OBJ_NEW_QSTR(qst) ((mp_obj_t)((((mp_uint_t)(qst)) << 4) | 0x00000006))
| ^~~
make: *** [../../py/mkrules.mk:85: build-sqfmi_watchy/py/runtime.o] Error 1
My build it working fine otherwise.
I added the modules to the frozen modules directory with Git and to my FROZEN_MPY_DIRS variable in mpconfigboard.mk.
Was there something else I was suppose to do?
bright robin
Ah, actually I have another problem.
It seems that for the new board I need a 'CIRCUITPY_CREATOR_ID' and 'CIRCUITPY_CREATION_ID' to be defined.
The board will not build without one, I put in the USB ID but it seems that's not enough anymore.
In the test builds for the Watchy I originally just put in a random number to get rid of the error, but I don't think I can actually put that into production.
So... do I have to define one? And if so how can I obtain an ID? It's not my board of course. It's from SQFMI.
manic glacierBOT
midnight ember
Ah, actually I have another problem.
It seems that for the new board I need a '...
https://pid.codes/howto/ free to open source developers created and maintained by one of our circuit python developers. the application process is pretty straight forward with a PR. Here's an article by Hackaday that covers the gist of the pid.codes project. https://hackaday.com/2015/04/03/usb-pids-for-all/ So far there are 472 contributors to the repo and a VID has 65,000 PID's available.. a long way to go before this method might not be viable. 😉
bright robin
So does it have to be your project? Or can I request a VID and PID for this board I find myself with?
midnight ember
That's a great question. Most people have developed their own boards that apply. If it's not your board then I'm going to guess it's an unbranded aliexpress board that you have little to no hope of finding the real PID for (if it has one)?
I don't know the answer to that question but there are people here that can chime in later to give you a better answer.
bright robin
No, it is a branded board with a website.
midnight ember
OK try to contact the manufacturer and ask for their VID & PID? That might be a speedbump in your project but it's better to use the real PID if possible.
bright robin
The USB VID and PID? Well I already have that if I plug it in.
But obvious that's non-unique. It's just the ID of the USB interface chip used.
midnight ember
ohh I'm sorry I misunderstood.
bright robin
It's an ESP32 with no extra digits, so no "native USB".
midnight ember
This is the first time I've heard of creator_id and creation_id... I haven't submitted a board in a while though. Yeah someone will be along to answer your question better sorry about that.
bright robin
Yeah, it wouldn't let me build an ESP32 project without it... the other one I'm working on uses an ESP32S3 so that's no problem.
Well I might have to see how other people solved this problem. There must be lots of them.
midnight ember
Is it documented in the "how to submit a new board" learn guide?
bright robin
Well kinda, it is a little vague. It just says you can use USB IDs if you have native USB, otherwise you'll need a creator ID.
midnight ember
For boards that don't have native USB, there is still a way to identify your board uniquely. You will see mentions of CIRCUITPY_CREATOR_ID and CIRCUITPY_CREATION_ID in their mpconfigboard.mk files. You can register new Creator and Creation IDs at https://github.com/creationid/creators.```Seems as if they're board porting ids not necessarily submitted by the manufacturer.
Yeah and they're mostly for non-native USB chips like the original ESP32.
bright robin
Yeah, so I will need to submit some kind of request for IDs to that repo.
midnight ember
Yep, should be straight forward.
bright robin
With a few legacy exceptions, each different hardware product that runs CircuitPython needs a different unique identifier.
Oh... right. That's how the old boards don't need them. They have some kind of hard coded exception.
midnight ember
Neat, didn't know that. My guess is these projects were born from a necessity to make the whole porting process easier for other developers.
bright robin
So as far as I can tell I just have to add a pull request to that repo. Looks easy enough.
midnight ember
If it's anything like pid.codes then yes. I have one registered and it was just a simple PR using other submissions as a template. Swap in the correct info, get the code, and plug it into the make.
I think my PR took a couple of weeks to get approved so it might not be a quick process.
bright robin
Oh well, submitted a pull request. So if any maintainer asks where my ID numbers come from I'll just direct them there.
manic glacierBOT
tulip sleet
Oh well, submitted a pull request. So if any maintainer asks where my ID numbers...
The creator/creation ID's are our own invention to distinguish boards that can't have unique USB VID/PIDs. If the mfr has their own USB VID, you can use that in the creator id. the creation id might be the part # or some other mnemonic number
onyx hinge
hm I discovered something interesting while working on the POSIX port: some stuff, like entering safe mode, can happen before starting the VM for the first time. This will print a compressed message. Compressed messages can refer to qstrs, but the root qstr pool is not initialized this early. So, you can get a crash this way. I wonder why we didn't see this before, or maybe we didn't understand it.
Program received signal SIGSEGV, Segmentation fault.
0x0000555555579fe5 in find_qstr (q=q@entry=0x7fffffffd628) at ../../py/qstr.c:142
142 while (*q < pool->total_prev_len) {
(gdb) where
#0 0x0000555555579fe5 in find_qstr (q=q@entry=0x7fffffffd628) at ../../py/qstr.c:142
#1 0x000055555557a2e0 in qstr_str (q=<optimized out>) at ../../py/qstr.c:290
#2 0x00005555555b9c9f in decompress_vstr (
compressed=compressed@entry=0x555555617c9d <translation59>,
decompressed=decompressed@entry=0x7fffffffd690)
at ../../supervisor/shared/translate/translate.c:138
#3 0x00005555555b9d15 in decompress (
compressed=compressed@entry=0x555555617c9d <translation59>,
decompressed=decompressed@entry=0x7fffffffd6d0 "CircuitPython core ")
at ../../supervisor/shared/translate/translate.c:149
(gdb) p MP_STATE_VM(last_pool)
$1 = (qstr_pool_t *) 0x0
maybe something's different about how I set things up with the "commandline workflow"
fringe yarrow
So here's something interesting.. I wrote two simple programs for the same ESP32S3, one in Circuit Python, one in Arduino C, to calculate Pi to a large number of decimal points (more than is actually stored, I suspect) and return the time it took to do so.. Both used the Bailey–Borwein–Plouffe formula in exactly the same way.. Against all expectations, the Arduino code was 25% slower than the Circuit Python code.. I would have expected the interpreted nature and overhead of Circuit Python would cause it to be at least somewhat slower to execute the same task..
spare jacinth
did they have the same precision tho?
perhaps circuitpy was faster as it was working on less "resolution" and could gain some extra speed from optimizations or some tradeoff on math operations
if values are somewhat close, im also quite surprised of the speed 🤯
fringe yarrow
They were both going through the motions of calculating to 600,000 places, though I don't know if the float type in arduino is the same precision as a float in circuit Python.
I only put it to that many places to make it a significant enough amount of time to get a good comparison.
I should write it again in espidf for a comparison.
spare jacinth
They were both going through the motions of calculating to 600,000 places, thou...
what i mean is that on float maths there is plenty of space for size vs precision vs speed tradeoffs, depending on how the compilation is setup (optimization flags) or some other "environment" things like eg CircuitPy using lets say 25 instead of 32 bits of precision for floats (and doings some clever tricks based on that), maybe you indeed get faster speed (trading off so much speed that you overcome Py's overheads) but at the cost of numbers being off
tulip sleet
what i mean is that on float maths there is plenty of space for size vs precisio...
CircuitPython uses the float hardware at 32 bits. It just gets truncated when it gets stored back. Is it clear the Arduino program was using the floating-point hardware?
and CPy has all the interpretation overhead.
or were you using double in Arduino?
fringe yarrow
I tried float and double to see if it made a difference in execution time. It did not.
tulip sleet
is your code available somewhere?
fringe yarrow
float calculate_pi(unsigned long n) {
float pi = 0.0;
int k;
for (k = 0; k <= n; ++k) {
pi += (1.0 / pow(16, k)) * ( 4.0 / (8 * k + 1) - 2.0 / (8 * k + 4) - 1.0 / (8 * k + 5) - 1.0 / (8 * k + 6));
}
return pi;
}
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
int start_time = millis();
float pi = calculate_pi(600000);
int end_time = millis();
Serial.print("Pi: ");
Serial.println(pi);
Serial.print("Execution time: ");
Serial.print(end_time - start_time);
Serial.println("ms");
}
void loop() {
// Nothing to do here
}
def calculate_pi(n):
pi = 0
k = 0
for k in range(n):
pi += (1 / 16.**k) * ((4 / (8 * k + 1)) - (2 / (8 * k + 4)) - (1 / (8 * k + 5)) - (1 / (8 * k + 6)))
k += 1
return pi
start_time = ticks_ms()
pi = calculate_pi(600000)
end_time = ticks_ms()
print("Pi: ",pi)
print("Execution time:", (end_time - start_time), "ms")```The k += 1 is not needed. It's not messing up the loop, but it's not doing anything useful.
I would check to see if Arduino is compiling to hw floating point or not. There may be some flags for that in the dropdowns.
I looked up about when conversions to floats happen. In C I believe the inside 8 * k + <n> operations will still be done as ints (unlike in Fortran). Same in Python.
manic glacierBOT
onyx hinge
speaking of "generating pi", the programs from this page tend to run on Python: https://www.gavalas.dev/blog/spigot-algorithms-for-pi-in-python/ but I tried the "gospers_pi" variant and it got through 2 80x25 screens (about 4000 digits) of pi and started to markedly slow down. This program is carefully written to work in python integers, which can contain a number of digits limited only by usable memory.
As the precision of a Python float is about 6 to 7 digits, I'm not sure doing a summation of 600,000 items from the BBF formula is usable (3 + 16**-6 == 3 is True in circuitpython arithmetic, i.e., they are exactly the same number, so I'm not sure anything past the first 5 hex digits is actually contributing to the final sum!)
def gospers_pi():
q,r,t,n,i = 1,0,1,8,1
while True:
if n == (q*(675*i-216)+125*r)//(125*t):
yield n
q,r = 10*q,10*r-10*n*t
else:
q,r,t,i = i*(2*i-1)*q,3*(3*i+1)*(3*i+2)*((5*i-2)*q+r),3*(3*i+1)*(3*i+2)*t,i+1
n = (q*(27*i-12)+5*r) // (5*t)
print(end="3.")
def rest(seq):
next(seq)
yield from seq
for digit in rest(gospers_pi()): print(digit, end="")
fringe yarrow
I doubt it is contributing anything, least of all to 600,000 digits. It was purely an exercise to take up time to gauge how fast different environments would do it.
Also with that += 1 in the for loop in python, it would be incrementing k twice each loop, so its actually only doing 300,000 iterations, leading to a lower total time, i bet.
wait, no it's not.. k isn't a counter, it's a temporary variable set to the next item in range(n) regardless of what it's value is by the end of the loop
i'll try that other one that gradually calcualtes it out..
thorny jay
The title is suspicious. I don't remember what it usually look like, but it is the first time I notice this title: "Copy of TEMPLATE ONLY -- CircuitPython Weekly Meeting for [DATE (TEMPLATE ONLY)]".
onyx hinge
I corrected the title, thanks.
tulip sleet
wait, no it's not.. k isn't a counter, it's a temporary variable set to the next...
Yes, I wasn't sure about that either, but I tested and k gets reset by the for.
lone axle
Would there be any interest in, or resistence to changing circuitpython-build-tools to generate library subfolders within the examples/ that get put in the bundles that are built? Right now we end up with 1 folder containing all examples (prefixed with library name, which is helpful). But subdirectories that happen to exist in any repo's examples dir also get put in there, and they don't always have specific names. So some are like: bmps, camera, cave, cellular etc.. it's unclear at first glance what library they would belong to.
If we had library level subdirectories within examples/ then each library could have it's own directory for examples and it will be very clear which one they belong to.
Ahh, I see this was discussed here: https://github.com/adafruit/circuitpython-build-tools/issues/28
midnight ember
yes that's very confusing. i rarely look through the examples bundle... and perhaps that's precisely why. all of the directories in /examples should be in library named folders. cave and cellular aren't libraries that i know of and even if they were i probably would end up looking them up on github instead of the examples folder. yes i'm in favor of a complete restructuring with actual library names in /examples bundle.
it's a hot mess in there
tulip sleet
Would there be any interest in, or resistence to changing circuitpython-build-to...
that is a great idea
tiny peak
@slender iron I got the orangecrab-examples for riscv/blink and riscv/button to build and load on my 85F using the Embecosm Ubuntu RISCV Embedded Stable (gcc 13) toolchain. Needed Makefile modifications to get things working with the proper USB vendor:product IDs and a gcc13 compiler flag change. Nothing major. All the notes are in my ocfpga repo. This gets me pretty close to the point where I can try the litex + circuitpython example.
manic glacierBOT
candid sun
over the weekend i noticed that bitmapfilter isn't available on the matrixportal s3. is that because it would make the build too big?
tulip sleet
over the weekend i noticed that bitmapfilter isn't available on the matrixportal...
it might or might not. I think it was originally targeted for boards with cameras or that you might want to use with cameras. I will see if it was there and then removed.
... yes, it's added automatically for boards with cameras, and not otherwise. but you have a use for doing bitmapfilter on the matrix image?
candid sun
@tulip sleet i'd been looking at todbot's generative graphics experiments (https://gist.github.com/todbot/5b649b0ff314f2e723b6dd68d36c05b5) and wanted to try them out on a matrix 😺
lone axle
I haven't yet, but will eventually use it on devices without cameras as well. For processing bitmaps to show with displayio.
tulip sleet
I can give you a test build. If we were to turn it on more generally, it could be for any board with displayio, but that might be a squeeze on some boards
I don't know how much space it takes, but it wouldn't be hard to find out
lone axle
maybe any that have bitmaptools enabled is a good deleniation?
candid sun
the matrixportal did have bitmaptools
tulip sleet
i'm doing space-check builds now
tulip sleet
maybe any that have `bitmaptools` enabled is a good deleniation?
that sounds like a good dependency
6.3k bytes
jpegio depends on bitmaptools
fringe yarrow
Is there a benefit to a supported board with built-in display NOT having the display initalized, defined and presented to the user as board.DISPLAY?
tulip sleet
Is there a benefit to a supported board with built-in display NOT having the dis...
we might have had one mfr not want that, I can't remember why, but we would strongly advise against it, since the display is very useful to show errors, etc.
fringe yarrow
I just have noticed, in my efforts to add support for the devices I had on hand, that there's a fair number like that..
tulip sleet
these are third-party boards?
stuck elbow
we might have had one mfr not want that, I can't remember why, but we would stro...
I think it was @empty salmon because he didn't want his e-ink display to wear off by showing errors
fringe yarrow
yes, third party.. e-ink makes sense..
tulip sleet
thanks for remembering that
fringe yarrow
I'm looking at things like the Waveshare rp2040 Touch LCD and the Makerfab 7".. those have pins defined for LCD functionality, but nothing in board.c to make it work without the user initializing the bus and display.
tulip sleet
@lone axle @candid sun bitmaptools is about 12k of code, bitmapfilter is about 6k of code
so I will propose a PR to link them, as you suggested
jepler is afk today, I would have him comment
lone axle
Is there any way I can see the output of print() statements that result from a test run inside of an action? I'm running into a problem in circup where I'm able to run the tests locally and they all pass, but 2 of them are failing when they run in the actions container. I'm trying to troubleshoot why they are failing but have thus far not found any way to see any kind of output so I am limited to just making a guess, changing something, pushing, then waiting for the pass / fail, but can't see any kind of debugging information about the value of variables during the test.
tulip sleet
Is there any way I can see the output of `print()` statements that result from a...
are they not in the CI log?
you should be able to run the tests locally. That would save a lot of pushing to github
lone axle
are they not in the CI log?
I don't see them in there anywher. I added this commit with a print statement: https://github.com/adafruit/circup/pull/208/commits/30e80f34867a659ced2c0ea2477b51027fd04873 but I do not see any of it's outputs in this log: https://github.com/adafruit/circup/actions/runs/8602027315/job/23570604472 I even searched "checking dir" in the search logs input near the top right and it shows no results.
when I run them locally I have to add -rP or -rx for it to print the output of passed and failed tests respectively. So I guess I could try modifying the actions with that flag
candid sun
so I will propose a PR to link them, as you suggested
thanks!
lone axle
No luck unfortunately. I even tried adding a new command to the actions build.yml file that runs a command to execute a single test with the -rx flag since it's failing in actions. when I run this command locally (with the flag swapped to -rP since it passes locally for me) I see the print outs. But I still don't find them in the log. It seems like actions is vacuuming up the stdout :/
tulip sleet
No luck unfortunately. I even tried adding a new command to the actions `build.y...
does it make any sense to print to stderr?
lone axle
I am not sure, but I'm willing to try anything at this point. Making my best guess changes in the dark is getting me no where so I really need to be able to see something in order to solve this I think.
maybe this can help https://docs.pytest.org/en/stable/how-to/capture-warnings.html I just noticed it's linked in the output showing in the actions log.
spare jacinth
error reads (as far as i can tell) that you expected the mocker logger to have called logger.error once, but that didn't happen (on test_ensure_latest_bundle_bad_bundle_data)
why the logger.error didnt happen? beyond my knowledge :p
and welp, circup.command_utils.get_bundle didnt get executed either (on test_ensure_latest_bundle_no_update)
lone axle
test_ensure_latest_bundle_no_update is the one I'm focused on ATM.
and my understanding is that the test is attempting to ensure that get_bundle does not get called. The fact that it is being called is what causes the test to fail.
empty salmon
@tulip sleet / @stuck elbow - I cannot find the context for the tag but the consensus from y'all was that errors on the board.DISPLAY - regardless of the type - was intentional for CircuitPython.
As such, I accept that original decision and documented it for users.
spare jacinth
and my understanding is that the test is attempting to ensure that get_bundle *d...
yeah, read that backwards. oops
lone axle
I believe that something about the way os.path.isdir is being mock patched is the root cause. but I have very little experience with this (prior work in circup is my only experience using these things 😅 )
tulip sleet
`test_ensure_latest_bundle_no_update` is the one I'm focused on ATM.
Note that the tests are being run with Python 3.7. If you are not using that locally there may be an environment mismatch
lone axle
oof, that is definitely the case then. I am on py 3.10 locally.
tulip sleet
some testing dependency is too old, etc.
lone axle
I'm not sure if I am ready for the deep dive into it, but it would be nice to document a process that could be used with docker or whatever to allow running the actions tests locally in as close to an identical environment as we can get.
tulip sleet
3.7 was chosen to help support old Macs or something. It reached end of life in june 2023
spare jacinth
I'm not sure if I am ready for the deep dive into it, but it would be nice to do...
there is a tool i have barely used that lets you "run"(quoted as in: probably not 100% equivalent) github actions locally, its called act
cool it just does this for me 🙃 ```
...
[Build CI/test] ❗ ::error::Version 3.7 with arch x64 not found%0AAvailable versions:%0A%0A
[Build CI/test] ❌ Failure - Main Set up Python 3.7
😟 I am at a loss on the outputing. warnings.warn in the code when run locally does output into the ==warnings summary== section. But the same thing inside actions just does not output the messages into that section.
spare jacinth
anyway, get_bundle is only called from a single place: ensure_latest_bundle
and that call is behind a if do_update: check, so seems like you have somehow made that variable change from False to True in the test case
lone axle
Right, I think I understood that far...
and I'm trying to figure out why it's the opposite value by adding print statements to check the value of variables internally.
spare jacinth
doesn't seem likje you've change lib_dir either, which is pretty much the only thing happening there 🤔
tulip sleet
instead of printing you could raise exceptions or whatever the current test is doing and put some stuff in the exception messages
spare jacinth
doesn't seem likje you've change `lib_dir` either, which is pretty much the only...
...maybe the self.dir string "template" is now looking at some other folder (due to some environment changes), which happens to exist
or maybe you created some folder that matches the outputs that you are getting from that template, also making the check return a True now
lone axle
raising an exception instead of printing causes pylint to fail with "unreachable code" 🫠
wraith crow
I'm looking at things like the Waveshare rp2040 Touch LCD and the Makerfab 7".. ...
I believe the Makerfabs 7" (and the ESP32-S3 Hacktablet) were test cases along with the Qualia board for "Dot Clock" displays. While the Qualia board doesn't have an attached display, I suspect that it was easier to test the other two from Python and no one ever got around to moving the display init to board.c when development finished.
fringe yarrow
There's no time like the present, if you have the hardware 😉
wraith crow
Neither has much of a user base, but I suppose they could serve as examples.
lone axle
okay, very strange. I added the exception raising: https://github.com/adafruit/circup/pull/208/commits/53541551136f5ac3743889724ddf4be438c54b38#diff-d3f87283746c1f8ed3d8fa8a6b567932fb878810935c95cadb28e6ca6142454dR116 but the test still fails with the same assertion error and doesn't mention anything about the RuntimeError that was raised. https://github.com/adafruit/circup/actions/runs/8602788106/job/23573140295#step:12:42 So maybe there is some other logical error or incorrect assumption on my part and it could actually print, but in this case something is causing it not to get to that branch of code.
I must take a break from this for a bit. But I have a new direction to look (validating any assumption I've made about control flow), and a new thing to try: putting prints or other outputs in higher level code that runs without conditions to see whether it does come through to the actions log.
spare jacinth
perhaps you are running directly to the else: update = True?
aka: tag != bundle.current_tag
i would print self.url to see if it is checking releases against your fork (origin of PR) or adafruit repo (target of PR), as a starting point
lone axle
perhaps you are running directly to the `else: update = True`?
yep, I'm thinking this is the case. I'm not sure why the difference in mine locally and the container. But I'm guessing something about one of the other mock.patch's used isn't having the same effect.
I will probably also look into upgrading to a newer python inside of the actions container. Using something newer would be nice for consistency, and especially if 3.7 is past EoL.
spare jacinth
perhaps it runs against your fork on local, but against adafruit repo on the PR, which would explain why tag names (or whatever else that depends on github's account/repo) are changing and thus going thru other parts of the code flow
lone axle
I'm pretty sure it's trying to spoof the tag anyhow with this line https://github.com/FoamyGuy/circup/blob/refactoring_and_issue_fixes/tests/test_circup.py#L997 and 1008 a bit below here. So it should be using the fake tag which doesn't match any on the real repo.
spare jacinth
oh yeah, i didn't look at the test code... because! 🙃
are you sure you fixed the patch's paths correctly when moving code?
eg: maybe you patched circup.bundle.Bundle and it had to be circup.Bundle or the like
that kind of stuff can be much of a PITA
nvm, that would also fail locally, wouldnt it? :p
lone axle
I'm not really sure of anything related to the mock patch's tbh. It's pretty likely I've done it wrong somehow but just don't recognize how. And something about the difference in my environment and the actions container is causing it to fail in the container, but suceed for me locally.
Ideally I'd like to make another pass through the tests (after getting them passing and setting this down to work on other stuff for a bit). With an eye towards trying to remove as much of the mock patching as possible. In many cases I think these are used in spots where we could also just have hardcoded files / values in the mock_device that exists for testing and read "real" values out of those files instead of mock.patching things like json.loads and file opens etc.
we could similarly check in a mock_bundle that has whatever necessary files it needs to look real enough for testing.
wraith crow
There's no time like the present, if you have the hardware 😉
The Makerfabs_tft7 firmware actually supports two display options and probably isn't worth two versions of the firmware. I'll check it out on the hacktablet though 😁
lone axle
It did indeed turn out that I just made a false assumption about the conditions which led to the print statements not executing... When they are moved into a branch of the logic that does execute they do in-fact get printed into the github actions log https://github.com/adafruit/circup/actions/runs/8603223248/job/23574622787#step:12:43 🤦 At least now I know.
slender iron
<@&356864093652516868> Excited to see you all in an hour for our community meeting. Notes doc is here: https://docs.google.com/document/d/1_cpbdNujG6Gb6YB18n2v94Jqyz09h4yPsyiRRcy9tgk/edit?usp=sharing
lone axle
Alright, I've finally got all tests and other checks passing ✅ . The root of the remaining issues were due to the way json was being mock patched. That seems to behave differently for me locally (parhaps result of py3.10 vs. py3.7). I was able to change the tests to mock something other the json module while still validating intended thing within the test.
solar whale
Have a good meeting — I’ll be out for the eclipse. See you next week.
midnight ember
P4 no wifi but will likely be coupled to C6 for wifi. C6 airlift coming for the P4?
I saw their new video today too. The marketing isn't clear enough that the P4 does not have radio and needs to be used with a C3, C6, (or H6) i think.
silver tapir
Costa Rica as well, just partial here.
midnight ember
It's always a plus when you can see what you're soldering
thorny jay
Maybe the guide can explain the difference?
I assumed there is "the guide" on the topic. 😉
Especially with the connection manager and other new changes in that topic... so a guide would be welcome so that I can catchup or re-learn.
silver tapir
circuit jupyter would be so cool.
midnight ember
There is a playground note on connection manager from Justin.
thorny jay
Does watching NASA live stream count as an excuse?
midnight ember
always
is that a thing? link?
https://www.youtube.com/watch?v=2MJY_ptQW1o It's in total eclipse right now and there's a screaming crowd lol.
in dallas
slender iron
Here is the notes document for next Monday’s CircuitPython Weekly Meeting. It is at the normal time of 11am Pacific / 2pm US Eastern here on Discord. Add your hug reports and status updates to the document before the meeting. If you are unable to attend but would still like to contribute, feel free to add your notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868> https://docs.google.com/document/d/1qribXknsk4RKNPSjjIPpvTiuhSzboGrLf4m8cRLw9JI/edit
lone axle
Making some progress on getting examples into their own subdirs in the bundle build tool 🎉 (I've modified my local bundle to only include 4 libraries during testing)
short tendon
@tulip sleet missed the meeting today (spring break), but would be happy to put together a playground note for networking if that would be helpful
tulip sleet
<@329766224093249548> missed the meeting today (spring break), but would be happ...
We are thinking about doing this internally. Will let you know.
manic glacierBOT
manic glacierBOT
manic glacierBOT
midnight ember
Thank you @slender iron for giving me homework on context managers. There's a lot more you can do with with than I realized. Learned a lot today. Hope you feel better today. 🤗
slender iron
@tulip sleet the nrf bootloader only sets aside 32k for the SD
tulip sleet
so it will use the RAM right after that for its own purposes, hunh? OK, then, I guess it's NO_INIT>
slender iron
noinit is at the top of that 32k
tulip sleet
ack, well, if you smash our SD RAM in the middle, does it matter, becuase you're going to the bootloader immediately?
slender iron
that's what I'm thinking
tulip sleet
is this to communicate to the bootloader, or the other way?
slender iron
it's to communicate to ourselves on startup
tulip sleet
or it's just for us to stash a state?
ok got it
I did look for some place to put this info, but couldn't find something like a persistent reg. You said there were a couple of bytes?
if you check it before the sd is set up at all, maybe it will be ok
slender iron
ya, I think it'll be fine
the two gregrets are 8 bits a piece
the first is already used for resetting into the bootloader
tulip sleet
do we do a hard reset to go into the bootloader?
i was thinking about some state in some peripheral, but that would be reset. Anyway, I think you have the solution
slender iron
testing it now
midnight ember
are PR states only open, closed, and merged? are there other possible states, draft perhaps?
midnight ember
Had the chance to push a new PR and try out my new powershell script for pre-commit. It worked perfectly. Updated PIP and then ran pre-commit without a single CRLF error. Super happy. 🙌
onyx hinge
achievement unlocked
bright robin
Good stuff, I've been meaning to get an extension for VSCode that highlights all your blank lines and whatever else. I remember an extension existed but I can't quite remember what it was called. This automated whitespace nitpicking is starting to annoy me.
spare jacinth
The option to show whitespace works good for me
And git diff showing everything red helps too
lone axle
The newest bundle release 20240411 is the first one built with the examples separated out into sub-directories for each library. 🎉
tulip sleet
Thank you @lone axle !
slender iron
@tulip sleet you should factor out that caching logic while you are in there 😉
tulip sleet
<@329766224093249548> you should factor out that caching logic while you are in ...
maybe 🙂 I just got it working with a minimal set of changes
my changes are in supervisor/shared right now anyway
slender iron
👍
slender iron
no, I don't think so
midnight ember
@crimson ferry I appreciate you and @short tendon advice on the PR. The examples might be tucked away now but they might become more used in the future. Anything to make them easier to use for people of all skill levels I'm all for.
bright robin
Ah so it looks like my PR for SQFMI Watchy was merged.
When does it appear on the website? I haven't actually added a board from scratch before.
tulip sleet
When does it appear on the website? I haven't actually added a board from scratc...
did you make a PR to the circuitpython-org repo?
bright robin
Ah, no problem. Will do.
tulip sleet
Ah, no problem. Will do.
There is a guide for that: https://learn.adafruit.com/how-to-add-a-new-board-to-the-circuitpython-org-website
atomic summit
Hey folks - Question re the "Serial JTAG" speed of ESP32-C3 and C6 in CP. Is this set somewhere in code?
tulip sleet
Hey folks - Question re the "Serial JTAG" speed of ESP32-C3 and C6 in CP. Is thi...
it's USB, so there's no "speed", right?. It delivers bytes on the internal side. It doesn't expose a UART-like thing on the internal side with speed considerations.
esp-idf/components/hal/esp32c3/include/hal/usb_serial_jtag_ll.h
atomic summit
it's USB, so there's no "speed", right?. It delivers bytes on the internal side....
Ok, so it's just running at max USB full speed? 12 Mbit ?
tulip sleet
Note that while the USB Serial/JTAG device is a USB 2.0 device, it only supports Full-speed (12 Mbps) and not the High-speed (480 Mbps) mode the USB2.0 standard introduced.
atomic summit
Because I feel like my transfer speeds are much slower than that.
Even just doing an esptool erase_flash (nothing to do with CP) seems to take WAY longer than the same thing on an S3 with the same amount of flash
atomic summit
> Note that while the USB Serial/JTAG device is a USB 2.0 device, it only suppor...
Yeah, I found that in the datasheet, but I assume "download mode" on an S3 would be also using the serial jtag peripheral, and that can erase_flash WAY faster.
The erase_flash thing rules out any CP settings.. but it's just weird - as I'm also seeing slow sending of data/files to the C6, and have that same RAW repl paste issue others have seen in that githb issue.
tulip sleet
the CDC-ACM part of the peripheral delivers 64-byte chunks. I think the slowness may be due to the ROM bootloader or what the stub is doing. If you use no_stub, does that make any difference?
atomic summit
I'll give that a try. I've never really grok'd the no_stub thing tbh.
tulip sleet
erase_flash doesn't need any data, so I'm at a loss to explain why that's slower
The stub might be quite different, especially since this is a RISC CPU, not xtensa
atomic summit
27 seconds to erase_flash my TinyC6 8MB of Flash with CP on it. 8 seconds to erase_flash my TinyS3 with CP on it, also 8MB.
tulip sleet
i would have thought erase_flash was just a small command to the stub do do the erase.
maybe there is no stub! Does it say it's loading a stub?
atomic summit
I'll have to pull out a test jig and have a look - I'll do that today and report back!
Thanks for the info and place to look @tulip sleet 🙂
This has all come about because a TinyC6 takes 85-90 seconds to go through a testjig cycle (which tests and loads CP + shipping files) - and all of my S3 boards, including ones with 16MB of Flash take under 45 seconds, and I have narrowed it all down to flashing and file copying times.
A TinyPICO which I still ship with MP, takes 27 seconds.
But the TP doesn't need to wait for the board to "mount" before I can start copying files to it.
Which on the rPI4 can take up to 7 seconds, and up to 17 seconds on a rPI 5 !
tulip sleet
esptool.py --port /dev/ttyACM0 erase_flash
esptool.py v4.7.0
Serial port /dev/ttyACM0
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.3)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: 10:91:a8:32:27:d8
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 16.3s
Hard resetting via RTS pin...
ok, there is a stub, and erase took 16.3 seconds with a QT PY ESP32-C3
tulip sleet
Nope! A fatal error occurred: ESP32-C3 ROM does not support function erase_flash.
tulip sleet
So I'll just have to live with the test times 🙂
I tried an Espressif C3 dev board, which has a USB-serial chip (! I guess for completeness so you can test the USB peripheral seprately). It appears to have 4MB flash, and took 22.5 secs to erase
so this seems inherent to the C3/C6
atomic summit
But the onboard Serial2UART will be baudrate specific - and will default to 115200
You should be able to push that to 2000000, or at least 1200000
Though I guess that wont change the "erase speed" - just the comms back and forward.
tulip sleet
I tried 921600 and the erase speed is just about the same
atomic summit
Yup, ok, so the speed is what it is! Thanks for testing this Dan, it's much appreciated.
tulip sleet
loading circuitpython on that dev board at 921600 takes 24 seconds
empty salmon
In my board.c, I understand how to setup a gpio pin for digital output and set it.
gpio_init(pin);
gpio_set_dir(pin, GPIO_OUT);
gpio_put(pin, 1);
I can not seem to find an example of an anlog pin ... and specifically, reading the pin value.
Is there something akin to gpio_get_analog() for reading a raw value?
tulip sleet
In my board.c, I understand how to setup a gpio pin for digital output and set i...
no, you have to set up the ADC, etc. If the SDK (is this pico?) provides it, you could call a driver it provides. Or you could instantiate an AnalogIn object, fetch the value, and then deinit it.
empty salmon
Yes, RP2040. OK
tulip sleet
Yes, RP2040. OK
simple ADC read in pico: https://github.com/raspberrypi/pico-examples/blob/master/adc/hello_adc/hello_adc.c
empty salmon
@tulip sleet - thanks! that was the pointer I hwas hoping for. Now I have found examples to work from.
tulip sleet
pico-examples is a great repo to get started on things
fringe yarrow
If I have a new board (ESP32s3), with a USB UART schematic that looks like this, should I be able to get it working with the CIRCUITPY drive?
onyx hinge
@fringe yarrow no, for the CIRCUITPY drive (& USB HID, etc) the USB signal pins would need to connect to the esp32s3's USB pins. Based on that schematic the only USB functionality will be USB serial via the CH343P chip.
fringe yarrow
thats what I thought.. thanks for the sanity check
manic glacierBOT
empty salmon
@tulip sleet , @stuck elbow - circling back regarding the board detection solution.
- The voltage divider trick is working well to compute a value (0 .. 9)
- The code sample for adding a function to the board (eg board.FOO()) is working
- The user can get the value if they should want it. For most cases, the user doesn't care.
- Internally, the computed value is used to determine which unit sequence is needed for the display.
So, a round of applause for you. Please take a bow!
tiny peak
@slender iron I made you a gist to demonstrate the idea I mentioned this afternoon during your deep dive stream. It's all python, and the config data is mostly in dictionary literals at the top of the file. The main exception is some stuff for conditionally modifying the ARM mcpu stuff once all the dictionary lookups have completed. Main possible benefit of this approach is that it could cut down on a lot of boilerplate. Perhaps there will be some useful ideas, but if not, no worries. This is the gist: https://gist.github.com/samblenny/a0128b49033ed6a70903243a9bf7e5c0
lone sandalBOT
neon cape
To who it might consern 🙂 SSL not working when importing Adafruit-azureiot on a rpPico W, I get this error when I try to reach a HTTPS-site. When I comment out "from adafruit_azureiot import IoTCentralDevice" it is working. It is also working if I get HTTP-site. This is the errormessage:
Fetching text from https://minip.no/
x509-crt-bundle:PK verify failed with error FFFFBD70
x509-crt-bundle:Failed to verify certificate
Traceback (most recent call last):
File "adafruit_connection_manager.py", line 271, in get_socket
OSError: (-12288, 'MBEDTLS_ERR_X509_FATAL_ERROR')The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "code.py", line 80, in <module>
File "adafruit_requests.py", line 591, in get
File "adafruit_requests.py", line 525, in request
File "adafruit_connection_manager.py", line 282, in get_socket
RuntimeError: Error connecting socket: (-12288, 'MBEDTLS_ERR_X509_FATAL_ERROR')
Not a big problem for me since I can use HTTP, but might be a problem someone wants to look at, when I googled it I found very little. 🙂
empty salmon
Is it possible to include a FROZEN library (e.g. adafruit_bitmap_font) in firmware which is not currently in the $(TOP)/frozen/ folder?
onyx hinge
@empty salmon no, any change to freeze in a new library would have to add the related submodule in frozen.
empty salmon
I found some instructions ( https://learn.adafruit.com/building-circuitpython/adding-frozen-modules\ ) which imply it is possible to add a new module. But, after following the steps, I still get an error when building my board that there is no rule to make target. (One step in the instructions does not work: ``make update-frozen-modules)
What I am a bit apprehensive about is how to commit this change.
midnight ember
To who it might consern 🙂 SSL not working when importing Adafruit-azureiot on a...
Please submit bug reports here: https://github.com/adafruit/circuitpython/issues/new/choose Bug reports are definitely appreciated!
manic glacierBOT
willow totem
anyone feeling generous, point me in the right direction... I was trying to get a debug build of main branch, for the adafruit_feather_esp32s2, but it's over max dram size, looks like it's included camera support, so can I turn that off and get past the error?
https://github.com/adafruit/circuitpython/actions/runs/8674507461/job/23786891435#step:10:1076
onyx hinge
anyone feeling generous, point me in the right direction... I was trying to get ...
yes, turning off some features you don't need is usually the way to get a successful debug build. I'd write it in the board/<boardname>/mpconfig.mk, then make clean, then make. ulab is another one to turn off that uses quite a bit of flash.
willow totem
yes, turning off some features you don't need is usually the way to get a succes...
can I do that with a build flag, I'm using the custom board build action in GH, so is it something like CIRCUITPY_CAMERA=0
onyx hinge
yeah I'd try putting in that Flags: line
manic glacierBOT
