#circuitpython-dev

1 messages Β· Page 33 of 1

proven garnet
#

Awesome, that's a huge help, thank you!

jaunty juniper
#

though we could use rp2040js to verify that it is

devout jolt
#

Hi! I have a (quick?) CircuitPython library aesthetics vs efficiency question. This is not critical, but I am curious: does const() really help? How do I test this? I've not found any real data on the benefits of const.

In the adafruit_seesaw library there are two approaches for defining a configuration class:
https://github.com/adafruit/Adafruit_CircuitPython_seesaw/blob/main/adafruit_seesaw/attiny8x7.py
https://github.com/adafruit/Adafruit_CircuitPython_seesaw/blob/main/adafruit_seesaw/samd09.py
The "attiny8x7" approach is obvious & readable, while the "samd09" approach uses const for most everything, The result is rather verbose for the amount of information conveyed. Is it worth it? Also, is const() useful when inside of tuples like analog_pins? For Seesaw, every bit of speed can really help, so I'd like to regularize these config classes if so. Thoughts?

slender iron
#

const() only helps in very specific cases

#

In this case, I don't think the names in the samd09 file provide any extra info

devout jolt
#

too bad we can't have const int tuples

slender iron
#

I'm not sure what you mean by that

devout jolt
#

like analog_pins = (1,2,3) vs analog_pins = (const(1), const(2), const(3)) I assume the latter does nothing useful

slender iron
#

ya, the latter will just leave in extra function calls during the tuple construction

#

b"\x01\x02\x03" is the most compact way of expressing it

devout jolt
#

(this is the crux of my question: the samd09 version uses const() and then puts those in tuple)

jaunty juniper
#

the samd09 version pretty much compiles to the same as the attiny8x7

slender iron
#

and by "compiles" that includes mpy-ification

jaunty juniper
#
_PWM_0_PIN = const(0x04)
_PWM_1_PIN = const(0x05)
_PWM_2_PIN = const(0x06)
_PWM_3_PIN = const(0x07)
pwm_pins = (_PWM_0_PIN, _PWM_1_PIN, _PWM_2_PIN, _PWM_3_PIN)

becomes:

pwm_pins = (4, 5, 6, 7)
#

both in memory when loaded and in mpy

slender iron
#

its a way to give a value a name without the name being included in the compiled version

sage elm
#

hi, I am trying to mount my circuitpython storage on an android device via termux. Does anyone know what drivers are needed on a linux device to mount it? I am hoping if I can find an equivalent for termux/android I can get it to work.

slender iron
#

the mass storage drive?

#

it's a standard usb driver

sage elm
#

yea the mass storage. I am able to see the device in lsblk and in /sys/bus but nothing from fdisk -l. When connected to an x86 linux laptop it shows as fat12 and I am wondering if somehow it can't read that fs type

devout jolt
slender iron
#

I don't know what you mean by const tuple

#

const isn't really a python thing

jaunty juniper
#

note that the micropython import itself does add some bytes to the mpy though, so that's not quite free πŸ€”

#

(but it beats keeping the variable names)

#

(but it doesn't beat not using names)

#

(except in readability)

sage elm
slender iron
#

@lone axle is the guide author

devout jolt
# slender iron I don't know what you mean by const tuple

in the same way that const(int) essentially does a preprocessor fill-in of that int to where it's used (instead of making a python number), it would be cool there was a similar fill-in from tuple(ints) to an array of ints, and get comparable speed benefits from doing if num in my_tuple

slender iron
#

in will be a linear search in either case I think

#

tuple, list, and arrays will be

devout jolt
#

correct, but I assume a linear search of int_t would be faster than a linear search of mp_obj_t (or whatever the appropriate type is), especially for small lists like analog_pins

slender iron
#

ya, I'm not sure how it does the comparison

#

arrays are great for that

jaunty juniper
#

no but it's still python, there is no int_t, a tuple remains a tuple, you want an array if you want optimized for type

#

or a bytearray

devout jolt
slender iron
#

agreed. samd09 would be fine like the attiny8x7 version

devout jolt
#

because that pinmap is accessed for every adafruit_seesaw analog call, so making it as fast as possible is worth it. but looks like, not much I can do

slender iron
#

because the names don't add aty value

jaunty juniper
lone axle
#

@sage elm I do not know how to mount the device from terminal, so can't offer much help on that front. I do think the issue was that Android didn't have support for FAT12 or whatever was needed to read from CIRCUITPY drive. I've noticed that some newer devices do have support for that though. On my newer phone if I plug in a circuitpython device with an OTG cable the OS notices when it's connected and basically mounts it automatically just like it would for a thumb drive or external hard drive. I maybe had to choose the "transfer files" or similar option from a thing in the notifications, but I'm not 100% certain, it's been a little while since I messed with it from Android.

The guide hasn't been updated to reflect it, but at least some devices can now connect it like storage and use it basically the same way that a PC does rather than requiring any extra mechanism to transfer files.

jaunty juniper
lone axle
jaunty juniper
tulip sleet
sage elm
#

@lone axle thanks for the pointers!

tulip sleet
#

there were some third-party programs that did support it, maybe "manually"

sage elm
#

@tulip sleet @lone axle that should be a software thing right? I am running a custom firmware with root access so I am wondering if I might be able to get around it if it is not a hardware issue

tulip sleet
#

Some quick websearching does not turn up "now it's supported". The latest Android device I have is a Samsung tablet that's Android 11

lone axle
#

one more gotcha I found, if you are using an OTG cable with either micro, or USB C on the Android side... When I was messing with it, it was pretty particular about which way the stuff was plugged in (i.e. OTG on the android side, vs. OTG on the microcontroller side. iirc it liked on the android side better, but I'm not 100% certain any more.)

tulip sleet
devout jolt
lone axle
#

There is support in some newer devices. I assume support was added to Android itself, because I think it's unlikely the manufacturer of my specific device would have done the work for it, but I also don't know that for sure, or when it may have been added. Just know my current phone can work with them successfully whereas my older ones could not.

sage elm
#

fair, but I guess if it is a driver that I might be able to install from a repo then it could work? termux has access to linux repos (I think debian) so was hoping they might have a package for a driver

tulip sleet
lone axle
#

Current phone says Android 9.

sage elm
#

mine is running lineage os version of android 11

#

I might mess with changing cables or something but serial port works fine from the terminal so I think that should mean that data connection is working

#

I guess if it is a FAT12 issue maybe I can bug the lineage OS folks to see if anyone has info on that.

tulip sleet
#

Thonny uses the REPL to do file I/O, so in principle you can avoid needed to access directly. There are other tools (ampy, outdated, and various tools for MicroPython) that do similar things. Whether they will run on Android is a q

lone axle
#

I just verified on my device with a USB C OTG cable when I plug it in like Phone> <OTG> <Micro USB Cable> <PyGamer Device. It shows up automatically in my notification drawer with 2 entries. One about being connected as a keyboard (I assume because CP presents as HID also), and another that says "Adafruit USB Drive, For transfering photos and media". If I click on that latter one it opens up a file browser and I can see the contents of CIRCUITPY.

sage elm
#

oh interesting. thanks! so somehow it is possible. what type of phone is this?

jaunty juniper
#

note that the official command line tool for Micropython is now mpremote, it's not fully supported with CP, but maybe we could look into some of it (not support mount but check that it can run and upload files)

lone axle
#

Mine is a CAT S61, not exactly widely known or used.

tulip sleet
#

what does mount do that we can't do?

sage elm
#

oh ok nice I will check out ampremote. I mostly just want to be able to use my local ide and then scp the files over to update the code

tulip sleet
#

local IDE on Android?

sage elm
#

local ide on pc

lone axle
#

interestingly it seems microcontroller dependent too though. When I plug in a Feather ESP32S2 TFT with the same cabling, the microcontroller turns on and Android has a notification entry that says "Android system charging connected device via USB". If I click on it, it takes me to a menu of different ways to connect, but all of them are grey'd out. I can't actually select anything in there. And I do not have the the entry about file transfer and it doesn't seem like Android can see any of the files for this device as opposed to the PyGamer.

tulip sleet
#

i'm confused, why not just plug the board into the PC?

#

I wonder if the PyGamer is big enough to have a FAT16 filesystem

sage elm
#

I have an led matrix panel connected to my wall which has the usb c connected to the android device. I wanted to use the android device as a server to be able to scp the files to. I don't want to take the device down everytime I want to test something

#

the circuitpython device is a Adafruit matrix portal which has the M4 chip

jaunty juniper
lone axle
#

does it have it's own seperate power source? Another potential concern would be if the Android USB port will supply enough power to drive the matrix.

jaunty juniper
#

mpremote uses uos.ilistdir() I noticed, and maybe ubinascii, but i got it to work by creating those as .py files

tulip sleet
sage elm
#

it seems to be giving it enough power so far. I can also connect to the serial console so data seems to possibly be working over the same cable

lone axle
#

I will try a Matrix Portal with the same setup so you can have at least one other data point with that hardware.

sage elm
#

thanks @lone axle !

lone axle
#

The matrix portal behaves the same as the ESP32S2 Feather TFT, so does not connect and allow editing files :(. Dan may be right about the PyGamer using a different FAT because of it's larger storage.

tulip sleet
#

PyGamer is only 8MB, so I wouldn't expect it to be FAT16, hmm

sage elm
#

yea from my linux laptop. fdisk -l shows it is formatted as FAT 12. is pygamer a version of circuitpython?

lone axle
#

The Adafruit PyGamer is an entry-level gaming handheld for DIY gaming and maybe a little retro-emulation, all in one compact dev board.The PyGamer is powered by the ATSAMD51 microcontroller with 512KB of flash and 192KB of RAM. There is an additional 8 MB of QSPI flash for file storage, handy for...

sage elm
#

oh ok, sorry, just looked back at your message, this is the device that worked

lone axle
#

Yep, the PyGamer does allow my phone to see its files.

tulip sleet
#

I plugged in my PyGamer. fdisk -l says it is FAT12

#

I am baffled as to why you would see a difference between the two

lone axle
#

I have one other Android device I'll try. It's Android 11 or 12, I'm not sure which. It is much less standard than typical smartphones though, so I'm not sure how much knowledge of it's behavior will carry over, I am curious now though.

#

This one was a bit of a mix of the others. It has the 2 entries in notifications, one for keyboard, and one that I assume tried for storage. But the mssage is different and it says "this device isn't in a supported format" or something similar, and if I click on it, it's going to a wizard which looks like it's going to try to format it, which I definitely don't want it to do. So perhaps slightly closer to working than on my phone, but still not quite working all the way. This was with the Matrix Portal. And this device has regular USB ports so there is no OTG cable in the mix.

sage elm
#

thanks for checking! I am gonna look into mpremote and check with the lineage OS folks. it seems like it could be possible since you have had mixed results

tulip sleet
#

@lone axle @sage elm On my Samsung Tablet, Android 11, I can use a regular USB stick with the OTG cabling I have. A FAT12 CIRCUITPY does not show up, but the fake UF2 BOOT drive does show up, and it's also FAT12. So it might be because it's presenting as a composite device

#

I could turn off everything but CIRCUITPY in boot.py, but it would still be a composite device with just one device, the MSC device

#

haven't tried that yet

lone axle
#

Interesting, another different behavior. I do like Android, but it's consistency or lack thereof is maddening sometimes πŸ˜…

#

I have Samsung phone with Android 9 as well and it doesn't work with the PyGamer the same way the Cat S61 does. It's similar with 2 notification entries, but the storage one says "drive corrupted click here to fix" but I'm too scared to click it.

#

I thought I had success with more devices in the past, but don't remember any specifics about which combinations worked.

tulip sleet
#

I get "Unsupported Adafruit USB Drive" on any of these. It offers to reformat, but letting it reformat brings up the same message again.

slender iron
tulip sleet
#

The PyGamer is slightly unusual in that it has its own power source. I don't think this has anything to do with composite devices

proven garnet
#

@devout jolt thanks for asking about the const usage, I caught up (thanks everyone who chimed in). Maybe it's worth making the others like your file then as the improvement πŸ™‚

lone axle
tulip sleet
#

There is something about the filesystem that Android doesn't like. I'd like to see the system log, not sure if that's possible

lone axle
#

System log from the Android side? I can collect those at some point.

tulip sleet
#

yes, I will see if I can do it on the tablet

#

plug it in and see what syslog says and why it doesn't like the filesystem

lone axle
#

I verified just now on my phone that I can edit code.py and save it and everything works as expected. I had done that in the past but not recently. Works out perfectly on that phone with the PyGamer.

#

One, tricky bit for getting the logs I guess is by default those go though USB and communicate with a tool called ADB on the computer. Since the microcontroller needs to be plugged in though it occupies the typically sole USB port.

tulip sleet
#

so i think the problem is not FAT12 support (any more), but something else. But why that PyGamer is special I have no idea. It's some accident of the filesystem. I'll bet if you storage.erase_filesystem()d it, it would stop working

unique python
idle owl
lone axle
still zephyr
# devout jolt Hi! I have a (quick?) CircuitPython library aesthetics vs efficiency question. T...

Late to the conversation, but according to my experience, const would only help when you are in the edge for a library to work in memory constraint board. the Definition of some_variable = const( ) as previously mentioned will add some memory usage too. However, we have used the names across the libraries as it is more readable. AKA easier to understand the name, than a Register Hex Address. I do like the way you define the pins and it could help to define some things in some of the libraries.

manic glacierBOT
#

I tried this on a production (non-green) board, and could not get it to fail after about 10 minutes of waiting. I made the loop as fast as possible:

import board
from adafruit_neokey.neokey1x4 import NeoKey1x4
keys = NeoKey1x4(board.STEMMA_I2C())

while True:
    keys.digital_read_bulk(0xff)

I am going to close this for now, but please reopen if you see it again on a production board.

sage elm
#

@tulip sleet @lone axle I tried using logcat on the android device to view logs when plugging/unplugging the m4 I see it recognizing the device but no (obvious) errors

#

this was from termux as root. I can possibly try from adb because my device has two usb ports

unique python
#

Thanks Kattni, Foamyguy!

tulip sleet
sage elm
# tulip sleet thanks - i am surprised there are no errors reported in the log, since there are...

I was seeing the device show up when I opened my file explorer app and it asks if I want to mount it. it fails and I got this in the logs for that app:
β€Œ53 D/SYSFAC> Checking FAT
β€Œ54 D/SYSFAC> FAT FAILED suspicious sectors per cluster count 0
β€Œ55 D/SYSFAC> Checking ExFAT
β€Œ56 D/SYSFAC> ExFAT FAILED invalid FAT count
β€Œ57 D/SYSFAC> Checking NTFS
β€Œ58 D/SYSFAC> NTFS FAILED suspicious sectors per cluster count 0
β€Œ59 E/TABLE> FileSystem Error: libs.va1
β€Œ60 D/PartitionTable> MasterBootRecord
β€Œ61 D/UsbMSDevice> Initializing 1 partition...
β€Œ62 D/UsbMSDevice> Partition 1, Logical Block Pos: 1
β€Œ63 D/UsbMSDevice> Create partition 1
β€Œ64 D/SYSFAC> Checking FAT
β€Œ65 D/SYSFAC> FAT Type:0, FAILED length=131072; index=131072
β€Œ66 W/Partition> Unsupported fs on partition libs.va1
β€Œ67 D/UsbMSDevice> Done!
β€Œ68 D/UsbHelper> No partitions!

brazen hatch
slender iron
#

πŸ™‚

tulip sleet
#

@slender iron which HDMI display are you using for testing? I think maybe I should get one.

slender iron
#

@tulip sleet adafruit has some. I'd use one of those. not exactly sure where I got this one

tulip sleet
#

ok, thought it was an adafruit one. There is one in a case: the rest are bare with driver boards. I could make a case

#

what res is yours

slender iron
jaunty juniper
#

say I don't think we support "RGB" TFT display interface in Circuitpython ?

slender iron
#

not yet. though it's on our radar

jaunty juniper
#

or parallel 16 bits ? (only 8)

#

(I was browsing larger screen ESP32 boards on aliexpress)

slender iron
#

we do 8 bit parallel

#

rgb displays typically don't have their own ram

#

but 8 bit parallel still assumes the display does

worldly escarp
#

Hello! I've written some cybersecurity labs in circuitpython but I'd like some help making more for an upcoming kids cyber camp. Where would be the best place to find circuitpython devs for paid educational projects?

stuck elbow
#

there was a job board on adafruit website

worldly escarp
#

I just looked it up, thank you!

jaunty juniper
#

thanks for the quick release @tulip sleet

tulip sleet
#

you're welcome; no problem

manic glacierBOT
#

Thanks @silabs-ChatNguyen, sorry for my delayed response. This week is a bit hectic with some travel thrown in the mix. It doesn't look like I'll be back home again to pick this up until early next week.

Could you pull code and try again with ubuntu or rebuild without option -j ?
make BOARD=explorerkit_xg24_brd2703a slc-clean (clean code)
make BOARD=explorerkit_xg24_brd2703a V=2 (without -j )

I did pull the code again this evening and tried without the -j option. I think the bui...

jade rivet
#

Hello. I am having issues installing Circuit Python. it seems cargo is too old on debian distros.

#

I have some log files. Is this where I post log files?

#

Sorry. I thought I had log files.

#

Let me retry and show the output if that would be okay...

jaunty juniper
#

yeah

jade rivet
#

minify-html keeps hanging the system (armv7l) and then gnarly output presides.

#

I installed rustc and some other dependencies. I will post the output of the build errors if anyone will get to it. Anyway, thank you in advance.

jaunty juniper
jade rivet
#

Yeppers. minify-html is the issue for me so far...

jaunty juniper
#

(doesn't help, but shows that it's known to be an annoying dependency)

jade rivet
#

Would you like my output?

jaunty juniper
#

I won't know what to do with it, but go on, others might (if not today, tomorrow)

jade rivet
#

Anyway, I will post in case others may find it useful:

#

Okay.

#

Is there a way to just not use minify-html?

#

For instance, say I do no have a need for use in html pages. Maybe I can erase or not install that specific package?

#

or would I need to install an updated version of cargo via the rust compiler?

#

Anyway. It is probably too late now. Off into slumber!

mental nexus
# jaunty juniper or parallel 16 bits ? (only 8)

I got 16 bit parallel working with FrameBuffer on the ESP32-S3 it was hackety-hacky but worked for the HACKTablet. Here’s the issue: https://github.com/adafruit/circuitpython/issues/6049

GitHub

The ESP32-S3 has a built-in LCD controller, including the ability to drive parallel (8080) and dot-clock driven displays. If CircuitPython can directly drive RGB displays, it may allow the use of l...

manic glacierBOT
#

Lol.. What is completed? Nothing.. Excellent work sir!

On Tue, Apr 11, 2023 at 9:52β€―AM Dan Halbert @.***>
wrote:

Closed #3763 https://github.com/adafruit/circuitpython/issues/3763 as
completed.

β€”
Reply to this email directly, view it on GitHub
https://github.com/adafruit/circuitpython/issues/3763#event-8975658511,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRY7JLGAZ2BCMYV5IA5JQTXAV43PANCNFSM4UCPIZMA
.
You are receiving this because you ...

lone axle
manic glacierBOT
#

@kmatch98 hi, i've been working recently on esp32s3-wroom1-n16r8 and rgb lcd, i just read all the messages here and i would like to ask you if you can help me understand here what's happening with my code.
I have two different TFTs, one is 7 inch (800 x 480) the other is 10 inch (1024 x 600) the only way i found to make the screens go decent without errors or so is using double buffers, but to maintain synchronisation with the two buffers, i need to set the "full refresh" to true.

I past...

manic glacierBOT
#

This looks like great work. As for the frame per second. Based on your dot clock and all the timings, if you calculate the max possible frame rate, does it match to what LVGL is saying. From what I remember (it’s been a while since I worked on this), I think getting 30 fps was really at the high end. I suspect you can’t achieve this for the larger display but you can calculate your best case based on your settings and the number of pixels.

If you redraw the full framebuffer each time, I t...

manic glacierBOT
#

The Onewire reset function returns false when it finds one or more devices on the bus by detecting a presence pulse following the reset low state. Unfortunately, if the bus faults to a stuck low condition for an number of reasons (i.e. 2 terminal device attached backwards, no pullup, hardware failure, etc) the reset will also return false indicating the presence of devices.

A stuck low bus will report a serial number of all zeros, which again unfortunately resolves to a valid crc8 of 0, in...

manic glacierBOT
#

Thank you for your reply.

After some tests i can say that:

  • if i have to redraw little portions of screen but i use the double frame buffer, nothing changes (as expected)

  • if i have to redraw little portions of screen but i use singla frame buffer + bounce buffer, i also can set the lcd clk at higher speed so i can get 50 fps (that would be the case of some static interfaces with little widgets and no animations).

  • i am going to 80Mhz with octal PSRam in DDR and the DMA is automati...

slender iron
manic glacierBOT
#

CircuitPython version

Adafruit CircuitPython 8.0.5 on 2023-03-31; Adafruit Feather M4 Express with samd51j19

and also tested on ItsyBitsy 8.0.5 with same results.

Tested with the Feather M4 on versions as far back as 6.3.0 with same results.

Code/REPL

Adafruit CircuitPython 8.0.5 on 2023-03-31; Adafruit Feather M4 Express with samd51j19
>>> import analogio
>>> import board
>>> a = analogio.AnalogOut(board.A0)  # A0 slowly climbs to 3.3v and holds
...
manic glacierBOT
#

I overrode some of those functions because an earlier revision of the esp-idf didn’t support some of the functions that I needed. I think the latest version has added all of those. So At first glance I don’t suspect that is affecting your performance. It may be primarily limited by the PSRAM access speeds.

As for β€œarbiter” I’m haven’t heard that term so unfortunately can’t provide any new insight. Keep digging, sorry I can’t be more helpful.

manic glacierBOT
#

Currently on esp32-s3, jtag support is broken.
On none of the board definitions is #define CIRCUITPY_ESP_USB_SERIAL_JTAG (1) enabled like it is on C3.
Enabling it creates linking errors.

Those linking errors can be fixed by appending:

ifeq ($(IDF_TARGET),esp32s3)
SRC_C += \
	supervisor/usb_serial_jtag.c
endif

after line 242.

But still, after compiling a debug build with those fixes and #define CIRCUITPY_ESP_USB_SERIAL_JTAG (1) in the board definition, we can see:
`...

brazen hatch
#

tl;dr usb resets & disables jtag

#

ran through source
couldn't find anything

#

searched all refs of jtag

#

searchef on micropy as well

tulip sleet
manic glacierBOT
jaunty juniper
#

@crimson ferry I have a PR for allowing HEAD requests with files on httpserver (as in curl -I someurl) which currently return a 400 error, but I want to make it return the header (with file size or 404 for example).
I can look into other behaviors to fix with it.

  • getting http://192.168.1.28/existingdir is an empty page
  • getting http://192.168.1.28/existingdir/ is a 404
    Since we don't have directory listing, a directory URL could send a 403 like apache does instead.
jaunty juniper
#

I was also looking into adding some syntax like @errohandler(404)

crimson ferry
#

@jaunty juniper Ignore my comments-thrashing on that PR... combination of some existing quirks in the library, test code amplifying those quirks, and bad test hygiene. HEAD could be handy. I'm no expert on web servers, I just know how I use them. Standard behaviors are good. Some .htaccess features (like directory listings and default index file name) could be simulated with an appropriate route (and server code.py). We may be starting to get complex enough with this library where un-obviously-malformed code runs but gives unexpected results.

tulip sleet
#

I'm not sure it's true now, but it would also be good if the file access stuff disallowed /../ etc.

crimson ferry
#

I tried that, doesn't sem to work, fortunately

#
# PATH = "/www"
% curl -i http://192.168.6.161:8080/../settings.toml
HTTP/1.1 404 Not Found
Content-Length: 0
Content-Type: text/plain
Connection: close
jaunty juniper
#

it looks like curl might fix the URL, I'll see if a raw request makes a difference

crimson ferry
#

oh, yeah, -v shows that

jaunty juniper
#
b'HTTP/1.1 200 OK\r\nContent-Length: 143\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\nAdafruit CircuitPython 8.1.0-beta.1-23-g0aea45516 on 2023-04-12; Adafruit FunHouse with ESP32S2\r\nBoard ID:adafruit_funhouse\r\nUID:C7FD1A71736D\r\n'
#

that's in response to b'GET ../boot_out.txt HTTP/1.1 \r\n'

crimson ferry
#

oops

jaunty juniper
#

I can fix

plucky tulip
#

Currently it just adds root path and filename together as strings, it could be prevented by checking if file path contains ".."

crimson ferry
#

this would be within the root path http://192.168.1.50/subdir1/../index,html but is there really a use case for it?

#

do we need a CP library channel?

plucky tulip
#

If you specify subdir1 as root, this would prevent accessing file outside subdir1

jaunty juniper
#

curl, python requests, and browsers translate that kind of url

tulip sleet
#

this is a very elementary security provision

jaunty juniper
#

you don't want someone to access ../settings.toml
also I was thinking that root_path shouldn't default to "/" but to not serving files because of that

crimson ferry
#

is there a way currently to wildcard a route (so that any file in a directory can be accessed)?

stuck elbow
#

you can set the root directory iirc

plucky tulip
#

There are wildcards, haven't tested it with files though

jaunty juniper
#

server.start takes a root_path that defaults to "/"

stuck elbow
#

so set it to something like /static

idle owl
#

Wait, are you all talking about hosting a tiny site on a microcontroller? Sounds like yes, and if yes, sounds like you're saying it would actually be quite simple for folks to get to the settings.toml file if it's public? (Maybe I'm way off here on what you're even talking about.)

idle owl
#

Oooooooooooooff.

#

I was going to be doing that next week.

tulip sleet
#

a PR by somebody would be welcome

plucky tulip
#

I can work something out

crimson ferry
#

@plucky tulip can you say more about wildcards? is there a syntax?

idle owl
#

An explanation of how to use the update would also be appreciated.

jaunty juniper
#

the simpletest example exposes settings.toml:

unique python
#

Pretty interesting

idle owl
crimson ferry
#

but URL params aren't paths

plucky tulip
#

It should be possible to make a simple handler that takes "/<file_name>" and returns HTTPResposne.send_file with that file_name

jaunty juniper
#

I can make it default to "/static" or I like "/www"

crimson ferry
#

it seems like the combination of "/" root and raw requests are a problem, any other security holes?

plucky tulip
crimson ferry
#

malicious users :p

#

secrets in files

stuck elbow
#

if you open it up to the internet, there is no hope

crimson ferry
#

this is true

#

but even LANs have hazards

stuck elbow
#

sure, so define what situations you want to handle

#

and which you don't

#

otherwise talking about security is pointless

crimson ferry
#

we want to plug known. holes that make it easy to exfiltrate secrets

jaunty juniper
stuck elbow
#

easy for whom

#

it shows as a usb drive when you connect it, there is very little that could be easier

stuck elbow
plucky tulip
#

Can be accessed not only via USB

crimson ferry
#

you could have it physically secured, but it's on the network

stuck elbow
#

"the network"

#

which one

plucky tulip
#

May be local, may be forwarded

#

Do don't know how people may use it

stuck elbow
#

if you forward a port from your microcontroller device, you are as good as hacked

#

dos-ed at the least

plucky tulip
#

Maybe we want it configurable with default to false, something like, "directory_ascending"

stuck elbow
#

and the local network is already protected with a password

tulip sleet
#

i think we just shouldn't allow it; it has very little real-life use

#

DOS is certainly an issue, but it doesn't leak secrets. Let's fix the obvious holes.

plucky tulip
#

Preventing DDOS on microcontroller would be hard in my opinion

tulip sleet
#

o

plucky tulip
#

without other tools

stuck elbow
#

it doesn't even have to be distributed

#

you can just send it a request that doesn't fit in memory

tulip sleet
#

that is not a secret-leaking threat, it's a denial-of-service threat. I'm willing to have the latter but not the former

stuck elbow
#

it would be nice to have it written down, so we can refer to it later

plucky tulip
#

Ok, so to sum it up:

  • no accessing files in parent directories
  • no default "/" as root path

Anything else?

stuck elbow
#

so that we don't get scope creep fixing things that fall outside our threat model

tulip sleet
#

@plucky tulip If you could add a sentence to two to the README re the threat model as part of your PR, that would be great

plucky tulip
stuck elbow
plucky tulip
#

It will introduce breaking changes though

stuck elbow
#

because the filesystem isn't

crimson ferry
#

some are

plucky tulip
#

Well, URLs in general are case sensitive

#

headers for example are not

stuck elbow
#

the fat12 on the circuitpython devices isn't

tulip sleet
#

open("CoDe.Py") works, so for file paths, it won't be case-sensitive

stuck elbow
#

not sure about other fat types on sd cards

tulip sleet
#

ok by me if routes are case sensitive but file accesses are not

stuck elbow
#

so I suggest the file paths are normalized to lower case before being used, to avoid a situation when you can sidestep a blacklist by using different case

plucky tulip
stuck elbow
#

also, maybe unicode normalization?

#

what about things like slashes that are url-encoded in the path?

#

would that let me escape from the web root directory?

tulip sleet
#

i think we don't handle escapes at all now

stuck elbow
#

should we?

plucky tulip
#

Can you give an example of what you mean?

stuck elbow
#

http://example.com/%2fetc%2fshadow

tulip sleet
#

% escapes are not currently parsed

jaunty juniper
#

yeah url decoding is out of scope as far as I'm concerned

tulip sleet
#

if/when they are added, then we'll need to up the security

stuck elbow
#

so if your file name has a space, you can't access it?

tulip sleet
#

this is a SIMPLE HTTP server

plucky tulip
#

I believe you can just use space in url

stuck elbow
#

the browser encodes it behind your back for you

#

I'm not saying we need to add this, I just want the decision to be documented

#

mapping urls to filesystem paths is a huge can of worms

lone axle
# idle owl Wait, are you all talking about hosting a tiny site on a microcontroller? Sounds...

fwiw with the implementation here: https://github.com/FoamyGuy/CircuitPython_PyBadger_Snake/blob/main/code_badge_with_wsgi.py the server does not expose any directories. The only file on the device that it will the contents to is static/index.html and even that is for convenience of updating. It could be a hardcoded string in the .py file I'm pretty sure and then no other files would ever be read or returned by the code that handles the web requests.

plucky tulip
jaunty juniper
#

url decoding is too big, I wouldn't do it

idle owl
jaunty juniper
tulip sleet
#

the original motivation for HTTPServer was to serve Wordle-ish sites. It has grown, of course

lone axle
jaunty juniper
#

ah ok

#

yeah the wsgiserver from ESP32SPI doesn't serve files by default

stuck elbow
#

there is also the problem of guessing the mime type

tulip sleet
#

right, so when I was asked to make a Wordle server, the existing WSGI servers weren't that helpful. So I took a tiny file-serving server and adapted that.

stuck elbow
#

do we trust the content of the files? that is, no files uploaded by external users?

plucky tulip
plucky tulip
stuck elbow
# plucky tulip Can you give an example?

well, you have to serve the file with the right mime type for that file, html files with text/html, javascript files with application/javascript, images with image/jpeg, image/png, image/gif, image/webp, image/tiff, image/bmp and so on...

plucky tulip
#

yes, but where is the problem

stuck elbow
#

you have to know the type of the file to serve it

plucky tulip
#

it is now determined by its name

stuck elbow
#

how

stuck elbow
#

and if the extension is not on the list, or there is no extension?

plucky tulip
#

then "text/plain"

stuck elbow
#

so if I put, say, a .cbr file in there, it will be served as text?

#

wouldn't application/octet-stream make more sense?

#

sorry, I got carried away, please ignore me

#

I need to go to bed, goodnight

plucky tulip
#

It could be considered, it was "text/plain" in the first version of the lib and I left it that way

stuck elbow
#

by the way, I'm using that library in fluffbug, and it made things a lot easier for me, thanks

crimson ferry
#

it's a really nice library, I've loved it from the beginning

plucky tulip
#

So, let's maybe start with the ".." in path and root path issue, and then expand on that if necessary, @tulip sleet ?

stuck elbow
#

I usually do this by converting the path to absolute with abspath, and then checking that it starts with the directory I need, in big python at least, that's much more proof than searching for forbidden character combinations

tulip sleet
#

I think that's fine for now.

#

yeah, we don't even have abspath

stuck elbow
#

there has to be something like it internally, no?

#

in the filesystem code, if nowhere else

tulip sleet
#

it's pretty crude, I think

stuck elbow
#

if you want to do it by searching for ../ then you also need to check for a leading /

plucky tulip
#

Not only for ../ but for .. in general

stuck elbow
#

like http://example.com//etc/shadow

stuck elbow
plucky tulip
#

I remember an issue with nginx static files and paths that starts with ..

stuck elbow
#

it's a tricky thing

plucky tulip
stuck elbow
#

in fact you could have .. at the end of a directory name too, like /foo../bar

#

and that's a perfectly valid path

plucky tulip
#

These are very edge cases I believe, as it a simple http server on microcontroller, I believe we can just force some rules on users

plucky tulip
jaunty juniper
#

I would filter for "../", I don't want to filter every path with a complex RE or something

stuck elbow
#

and you probably want to disallow backslashes alltogether

plucky tulip
#

I will try to come up with something

slender iron
#

the web workflow also hosts files from the fs

#

and it only protects using http auth

plucky tulip
stuck elbow
#

but it allows access to the whole filesystem, so there is no problem of escaping from a directory

plucky tulip
#

Can we do a milestone for v3 of adafruit_httpserver, we could bundle all incoming changes and then release it as a new major version, I am also working on authentication system and some unifications/name changes so I could also add that? @tulip sleet

manic glacierBOT
plucky tulip
crimson ferry
#

you can do it with a raw TCP socket

jaunty juniper
#

C python

plucky tulip
#

Thanks

manic glacierBOT
#
adsr = np.concatenate(
    (
        np.linspace(3277, 32767, num=40, dtype=np.int16, endpoint=False),
        np.linspace(32767, 0, num=20, dtype=np.int16, endpoint=True),
    )
)
#...
synth = synthio.Synthesizer(sample_rate=48000, waveform=waveform, envelope=adsr, envelope_hold_index=42)

A basic linear volume ramp, then slight decay, followed by a sustain at about 80% of full scale. At 48kHz, it takes about 0.2s to ramp to full volume.

The envelope can in principle ...

deep acorn
#

good evening, I was pointed here for the individuals that could answer a technical question about adding libraries before compile

#

I could be wrong but Ive only ever gotten my code in python [https://github.com/FriendlyNGeeks/PiVAC] to work with the _thread lib, running a http/json server on one thread and a seprate fetch loop on another thread. If a dev were to accidently point me to the make files I could copy over from micropython it would save me days in trying to write custom driver support for the feathers3 reverse tft and 8pwm feather servo

manic glacierBOT
manic glacierBOT
manic glacierBOT
#

Thanks @silabs-ChatNguyen, sorry for my delayed response. This week is a bit hectic with some travel thrown in the mix. It doesn't look like I'll be back home again to pick this up until early next week.

Could you pull code and try again with ubuntu or rebuild without option -j ?
make BOARD=explorerkit_xg24_brd2703a slc-clean (clean code)
make BOARD=explorerkit_xg24_brd2703a V=2 (without -j )

I did pull the code again this evening and tried without the -j opt...

manic glacierBOT
manic glacierBOT
#

@jedgarpark @todbot in case this is of interest, I'm working on adding ADSR-style envelope to synthio. I think it's ready for somebody to try. However, I'm on vacation :grinning: so I won't necessarily act on feedback in a timely fashion. The changes also give another bump in loudness, because I improved (I hope) how the overall envelope of all playing notes is bounded.

manic glacierBOT
tulip sleet
#

@slender iron do you want to try again for a triage meeting? I have to stop kind early today, like around 1pm your time

slender iron
#

I'm gonna go get my package right now but then can in 5 min or so

tulip sleet
#

sounds good

manic glacierBOT
slender iron
#

@tulip sleet I'm in the Amelia voice channel

manic glacierBOT
#

@jepler this seems to be an ASR envelope, and is working very nicely (or maybe I haven't found the decay time control?) Here's my guess at what the adjustable values are:

attack_time =  40 
sustain_level =  32767
release_time =  140
asr = np.concatenate(
    (
        np.linspace(3277, sustain_level, num=attack_time, dtype=np.int16, endpoint=False),
        np.linspace(sustain_level, 0, num=release_time, dtype=np.int16, endpoint=True),
    )
)
synth = synthio.Synthesizer(
  ...
blissful pollen
#

Just curious of the DVI changes were merged or if there is a PR for them? Saw the update on AAE but only in passing. Just got the feather for it

manic glacierBOT
blissful pollen
#

Thanks

crimson ferry
#

lots of issues need to be re-tested, I guess I know what I'm doing this weekend :p

manic glacierBOT
manic glacierBOT
#

I don't have any devices with me to test, so please feel free to remove the workaround if your testing indicates it's not needed. I don't have any specific recollection, so I can only rely on what was in the related commit message:

After this change, it's possible to connect multiple times to wifi in different runs of code.py or the repl after soft rebooting.

so it sounds like @DavePutz 's testing shows this is no longer needed.

I can't do any testing before May so I recommend going ...

manic glacierBOT
#

Adafruit CircuitPython 8.1.0-beta.1-23-g0aea45516 on 2023-04-12; Raspberry Pi Pico W with rp2040
Reset between testing each type of socket.

Web Workflow not enabled, same as before:
5 SOCK_STREAM, then RuntimeError: Out of sockets
3 SOCK_DGRAM, then RuntimeError: Out of sockets
4 SOCK_RAW, then RuntimeError: Out of sockets

Web Workflow enabled, same as before:
5 stream
2 dgram
4 raw

<details>

<summary>(test code...)</summary>

import wifi
import sock...
manic glacierBOT
manic glacierBOT
#

I think this should be a fully configurable ADSR (instead of jepler's original where the decay rate is the same as the release rate), but perceptually the sound seems to "bounce" down-then-up, after the decay. Weird.


attack_time =  100
decay_time = 50
release_time =  300
attack_level = 32767
sustain_level =  5767

adsr = np.concatenate( (
    np.linspace(2, attack_level, num=attack_time, dtype=np.int16, endpoint=False),
    np.linspace(attack_level, sustain_level, num=d...
jaunty juniper
#

@lone axle you mentioned trying the adafruit_httpserver library with ESP32SPI, but ESP32SPI doesn't have socket.bind() and other server related methods, were you talking about another library ?

lone axle
#

in which case, I also notice now that I could probably lower the chunk size to help the RAM trouble.

jaunty juniper
#

I see

jaunty juniper
#

I took the UniversalSocket class I made for websockets, added some server functions and was able to run httpserver on a PyPortal, at least right now I have a page that sends a text to the pyportal's screen without changing the code inside adafruit_httpserver or adafruit_esp32spi, which is a testament to duck typing

lone axle
#

Nice!

crimson ferry
#

how did you get os.getenv to work in that environment?

jaunty juniper
#

os.getenv is just one way to get the ssid and password

crimson ferry
#

that's where I saw the os.getenv

jaunty juniper
#

are you under the impression that it's not implemented on all ports ?

crimson ferry
#

oh I guess it is πŸ€¦β€β™€οΈ

jaunty juniper
#

I use the same names as the web workflow because I just copy my settings.toml, even though the web workflow is not on the pyportal

crimson ferry
#

gotcha, i don't mess with esp32spi much anymore and I use settings.toml mainly for native-wifi-specific stuff, so I just didn't associate it

jaunty juniper
#

I'll upload my full test code, a web page with a text, size and color fields, that sets what is shown on the screen of the board, tested on FunHouse and PyPortal

crimson ferry
#

I guess there's still CIRCUITPY_BLE_NAME and CIRCUITPY_PYSTACK_SIZE

#

I think the ESP-equivalent of listen is client socket connected and available

#

not so clean though

#

no, it just doesn't seem to map well

manic glacierBOT
#

I haven't revisited my weather station project because of that bug, it completely derailed my project. The bug was in 7.x and continued into 8.x beta. We're now past the 8.x stable release date. It's been permanently off for months. Got sidetracked with other projects. I'll make it a point to load this one back up tonight and test with the current stable version of 8.0.5. I am still hesitant to close this issue as it was a doozy of a bug. Will check and report back.

manic glacierBOT
#

I switched to the Adafruit ESP32-S3 which was exhibiting the same symptoms at the time during 7.x and 8.0 beta. Gave up on it at that point.

Adafruit CircuitPython 8.0.5 on 2023-03-31; Adafruit Feather ESP32S3 4MB Flash 2MB PSRAM with ESP32S3
Board ID:adafruit_feather_esp32s3_4mbflash_2mbpsram

Been running like a dream for an hour. Everything is much much faster too. I don't want to swap the UM FeatherS3 back in because it's deep inside an enclosure and would take about 20 m...

midnight ember
#

Closed 6791. Runs great on 8.0.5 so far. No more hard faults. πŸ‘

#

S3 has stability now with wifi projects. Looking good.

#

It's been off since January. Nice to have it back on my desk working. Thank you to all the developers who work on circuit python every day. ❀️

#

I still have no idea what the original culprit was or what fixed it. As long as it's working it doesn't really matter.

manic glacierBOT
midnight ember
#

Running 6 hours, no hard faults, definitely fixed.

manic glacierBOT
#
  • If the first element of ADSR is 0, it's just like starting the sound 1 time-unit later, so start with nonzero. 3277 was left over from when the attack portion was 10 units long, so it's about 32768/10.
  • it is ADSR, not ASR, because you can set the "envelope hold index" to set a sample within the envelope array which is used during the sustain portion. So for instance if you have the very simple ramp [10000, 20000, 30000, 20000, 10000] and set the hold index to 2 (indicating the element ...
manic glacierBOT
manic glacierBOT
deep acorn
#

@crimson ferry are you able to use adafruit http and continue to do other task while server is waiting for socket connections?

manic glacierBOT
lone sandalBOT
manic glacierBOT
#

The Magtag Google Calendar from the learning guide uses some Arial PCF fonts that apparently contain bitmap elements with a width and height of 0. The DisplayIO argument validations introduced in https://github.com/adafruit/circuitpython/pull/7548 cause the example code to fail.

Traceback (most recent call last):
  File "code.py", line 233, in 
  File "adafruit_magtag/magtag.py", line 170, in set_text
  File "adafruit_portalbase/__init__.py", line 291, in set_text
  File "adafrui...
crimson ferry
manic glacierBOT
manic glacierBOT
#

I'm realizing the only reason I hesitated adding the test for the 0 width/height object for the border wrap issue is that I don't know how to call a micropython max() function. Is there a way to call something like mp_MAX(0, source->width - 1) rather than adding an if statement and two assignment statements?

i.e.

int16_t x = mp_arg_validate_int_range(args[ARG_x].u_int, 0, mp_MAX(0, source->width - 1), MP_QSTR_x);

rather than:

if (source->width == 0) {
    int16_t x = m...
blissful pollen
#

@slender iron Hope you see this after the weekend I did try the DVI support for a while with some success. If you need any testing let me know.

Couple notes:

  1. Disk access would go at times, more so if I created my own display. Many times this would corrupt the FS.
  2. Something in ColorConverter is causing an issue when colors change. If I just hard coded a the conversion code from 565->8bits color it worked. I can show an example later if needed.

16 bit didn't seem to go smoothly and I didn't look into it any further.

#

Was fun to play with overall though!

manic glacierBOT
#

Hope you find ...

manic glacierBOT
manic glacierBOT
#

Thanks @jepler, that makes sense. This is a lot of fun to play with and already so very useful.

Some comments from trying it out:

  • On key release, the envelope currently plays from envelope_sustain_index which causes a jump in loudness if the note is still ramping up from. The more expected behavior would be to play from equivalent loudness.

  • Synths often use MIDI velocity to change the amplitude envelope: both peak/sustain levels and attack & release rates. e.g. play softly t...

analog bridge
lone sandalBOT
manic glacierBOT
manic glacierBOT
onyx hinge
tulip sleet
manic glacierBOT
#

Still occurring with Adafruit CircuitPython 8.1.0-beta.1-25-g4d7e54310 on 2023-04-16; Adafruit PyPortal with samd51j20.

Even simpler to demo is to control-C during run, the KeyboardInterrupt pops into the title bar. Hard reset, and the KeyboardInterrupt is still in the title bar after the tio reconnects and code.py is runing and printing to the serial console.

The title bar otherwise stays current: for example, control-C then control-D will refresh away the KeyboardInterrupt...

lone axle
#

I found a driver library that runs successfully on microcontrollers (ESP32 S2 only one tested). But seems to raise an exception consistently when running on RasPi w/ Blinka.

With a bit of experimentation, I've discovered that a small time.sleep(0.1) inserted during the initialization of the driver after an existing RESET command, and before it starts doing anything else seems to resolve the issue and allow the Pi to use the driver and hardware.

Since it does seem to work as-is on microcontrollers, I assume the Pi can just get through the execution faster and happens to be "too fast" so the sensor isn't fully back up and running after the reset yet.

I can PR the library to add that sleep, but I hesitate since it does work on microcontrollers as-is. Should it use an optional argument or something to signify to sleep so that devices that don't need it don't have to spend any time on it? Or if the sleep time is small enough is it okay to just include it and have it wait for everyone?

slender iron
blissful pollen
slender iron
#

hrm. ok. I suspect the second core is executing some code from flash during a flash operation then.

#

I'll check it all

#

I'm rapidly approaching the moment I hack on lld to make it do what I need

#

"move this function and everything it needs to ram"

blissful pollen
#

It seemed to only occur if I created my own framebuffer. I don't think I saw it occur when I was using the one created automatically

slender iron
#

hrm ok. I didn't get the framebuffer creation working for myself

#

were you trying on a board without a built in display?

#

I was finding that it didn't start a second time on the feather dvi

blissful pollen
#

Tried it on the feather DVI board only.

#

I would release displays. Create the new framebuffer and framebufferio and then saves wouldn't work. But I didn't dig too deep

slender iron
#

did the display work?

#

or did everything stop?

blissful pollen
#

It did

slender iron
#

hrm. I was seeing it hang in start up

blissful pollen
#

I can try it out some more too. I know its extra preliminary so didn't dig too deep yet

slender iron
#

nah, I'm still poking at that bit

#

hoping my fresh eyes today will see something

#

the picodvi lib doesn't have a deinit so I've gotta do it myself

#

I think folks will want to be able to switch modes though

blissful pollen
#

Yeah if you want to specify your own bit depth or resolution not sure how else you can do it

slender iron
#

ya. I know retro computers can switch between monochrome text mode and lower res color

#

so I want us to be able to do that too

turbid radish
#

The Python on Microcontrollers Newsletter is ready to crib from for the 2PM meeting, in GitHub

digital shoreBOT
manic glacierBOT
#

Yes, right now releasing just jumps from whatever spot in the envelope to the first index after the sustain index. Given the underlying table-based approach I chose, what else would be sensible to do? Or should I abandon the table-based approach? (it's not too late for that)

I'd like to directly support velocity (as relative note volume), but I don't know what the API would look like. Notes would either become objects with multiple properties, or all the 'parts' would be packed together in...

#

Tried various combinations of printing as fast as possible before serial is connected, and calls to supervisor.runtime.usb_connected and supervisor.runtime.serial_connected.

Not able to reproduce with:
Adafruit CircuitPython 8.1.0-beta.1-25-g4d7e54310 on 2023-04-16; Raspberry Pi Pico W with rp2040
Though I don't think we ever saw a reproducible procedure. I also haven't seen this in Discord since the two previous occurrences mentioned above.

I think this can be closed.

idle owl
#

@lone axle I have some guide feedback on the PyCharm and CircuitPython page in the Welcome guide. They're using a newer version, and evidently the UI has changed significantly. I didn't confirm this. Can you take a look at the page and update it if indeed an update is needed? Thanks!

lone axle
idle owl
#

Excellent. Thank you!

manic glacierBOT
lone axle
#

<@&356864093652516868> The weekly meeting will take place here on Discord at the normal time of 2pm Eastern, about 90 minutes from now. We look forward to hearing from you! You can fill in your hug reports and status updates and review the document ahead of time here: https://docs.google.com/document/d/1qghGAtx8DGO8UV_0hIy9R8n3UWM9ICSSSMR0Wkgs0u4/edit

manic glacierBOT
lone axle
#

@slender iron are you available to read the core section when the time comes?

still zephyr
# lone axle I found a driver library that runs successfully on microcontrollers (ESP32 S2 on...

I think the time is the case. The sensor does have a required conversion time that is not present in the library. however this time is max=10.8 ms (table 2 datasheet) for a temp 14 bit resolution. We do have, sleep time in some sensor libraries to allow the sensor to get back to normal, and as you mentioned in the issue the library does this when using the function _data(self). I do have seen problems with this sensor when you try to read the temp or humidity and the sensor is already doing a conversion, that is why the library uses the start_measurement function

midnight ember
#

I'm still dealing with internet dropping occasionally for 2-6 minutes just a heads up.

random junco
idle owl
#

Jeff and I!

#

Tekktrik will be with us as well!

midnight ember
#

Represent!

random junco
ember iris
#

I'll be at pycon as well!

idle owl
#

Excellent!

random junco
midnight ember
#

That's a lot of new contributors. That's awesome.

#

Scott got his 1 page finally? πŸŽ‰

crimson ferry
#

what generates the stats?

lone axle
#

There are scripts in adabot that compile them.

midnight ember
#

Nice JP!

slender iron
tulip sleet
midnight ember
#

I was really impressed with tabler

idle owl
#

😊 You're welcome!

#

FeatherWeather

turbid radish
#

Yay PyLadies!

midnight ember
#

debounce isn't easy without keypad library you're not alone.

idle owl
#

It was all we used to have. I have some pseudocode that works, but it's a bit extraneous. Neradoc's suggestions were slimmer, which is good because this demo is already pretty beefy.

#

πŸŽ‰

midnight ember
#

The django tabular dashboard is gorgeous. Worth checking out his stream to see that one.

#

Went deep on the DAC testing. Very nice CGrover!

silver tapir
#

Of course wait until the meeting is over to watch the video.
https://www.youtube.com/watch?v=_IGj-wyZRRc

This pedal uses demucs to split any song into 4 tracks: vocal, drums, bass and others (which includes guitar, keyboards, winds, etc), and then you can use the pedal itself to control playback and mixing of those channels, with a rotary encoder assigned to each track.

It's all open hardware and open software, using a case from a JΓ­cara instead o...

β–Ά Play video
#

There is a comic in there that explains what is that weird green wood thingy. Working on an english translation of this comic.

midnight ember
ember iris
#

πŸ‘‹ 😁

modern wing
#

Hope y'all have a most excellent time!

turbid radish
#

Thank you, Melissa!

midnight ember
#

Hope your move and your projects go well Melissa.

gilded cradle
#

Thanks

midnight ember
#

ohh danh on the circuit python show? ❀️

random junco
#

yup!

slender iron
#

yup, and I'm hosting

modern wing
#

Thanks for another excellent meeting!

gilded cradle
#

Thanks

ember iris
#

Thanks all, have a wonderful week!

midnight ember
#

@crimson ferry Did you notice the whizz-bang I named after your suggestion? πŸ™‚

crimson ferry
#

i remember you mentioned a plan, but didn't see the #thing

midnight ember
#

it's the FPV tank.

#

I was like "ohhh that's perfect"

crimson ferry
#

the sewer scout?

midnight ember
#

yes, perfect play on words

#

sewer>whizz

#

tank>bang

#

it's perfect

manic glacierBOT
lone axle
#

Here is the notes document for next Monday’s CircuitPython Weekly meeting. It is at the normal time of 11am Pacific / 2pm Eastern here on Discord. Everyone is encouraged to attend! Please add your hug reports and status updates even if you’ll be attending the meeting - it’s super helpful! If you are unable to attend but would still like to include updates, feel free to include them in the notes and we’ll read them off during the meeting. Hope to see you there! <@&356864093652516868>
https://docs.google.com/document/d/1O1R88xjW7HiJjJ1zOHpSwMhXNh-jX7-BOy8P8jLgLyA/edit?usp=sharing

manic glacierBOT
manic glacierBOT
ember iris
#

@lone axle I'll let you merge the pull request if that's alright, as I'm really rusty with the process πŸ˜…

manic glacierBOT
#

Given the underlying table-based approach I chose, what else would be sensible to do? Or should I abandon the table-based approach? (it's not too late for that)

Oh, if the API is still fluid, I have opinions! πŸ˜€

The current waveform approach is interesting and allows for complex envelopes, so it'd be great for LFOs, but I don't see how to make it work easily as an ADSR envelope, where the envelope can constantly change based on user playing.

Most of the synth APIs I have come acros...

tulip sleet
#

@slender iron I did not realize that the silabs PR has 150MB of .zip files (I thought they were much smaller). This is much larger than anything in any of other port directories. I think we really don't want this: we want that stuff to be a submodule. And we don't want it in the history either. I think I need to revert the PR and clean out the history of those large files.

#

@slender iron do you think I should do a force push, or do something trickier to get rid of that history?

slender iron
#

we don't want the files in the history because we then need to download them?

#

force push is ok with me

tulip sleet
#

I will do that immediately

#

@slender iron I have to undo the branch protection rule temporarily

slender iron
#

kk. I'll be up from my desk soon when ari wakes up

tulip sleet
#

@slender iron talk to you in Amelia for a bit?

onyx hinge
tulip sleet
#

so if you just pulled within the past hour, you will want to pull again to get rid of that merge commit.

onyx hinge
#

No, I just got back from the trails

#

Suspect the problem is missing apt-get update though

tulip sleet
#

but why should we have to do apt-get update? That is their Ubuntu image problem

onyx hinge
#

I mean, I agree but in the past they've closed it as "meh"

#

(the Ubuntu archive now only has a package newer than the one referred to in cached data in the Ubuntu azure image)

manic glacierBOT
#

@silabs-BelaV @silabs-ChatNguyen I merged this PR, and then realized that the ports/silabs/tools/slc_cli_linux/* files were 150MB. That is much larger than anything else in this repo. We would really like these files to be somewhere else. Do you have another repo you can submodule them from, or just fetch them as necessary in the Makefile, say?

I undid the merge commit by force-pushing to get rid of the history with these files.

tulip sleet
onyx hinge
tulip sleet
#

I deleted all the caches that might be causing this

onyx hinge
#

Oops phone use problems

#

(having the indexes be there at all is an attractive nuisance)

tulip sleet
#

:throws up hands: oh well, it will slow down things, and notice in that 6486 above it says that it didn't work even after that

#

This error occurs even after adding sudo apt-get update in the command:

onyx hinge
#

They should remove the index from the runner images so it just always fails when not following the instruction without which it works most of the time

tulip sleet
#

@onyx hinge so in build.yml:

      run: |
        sudo apt-get install -y latexmk librsvg2-bin texlive-fonts-recommended texlive-latex-recommended texlive-latex-extra
        pip install -r requirements-doc.txt
#

add sudo apt-get update as the line before? Simple as that?

onyx hinge
#

I think so!

tulip sleet
#

there are few other apt-get install lines in other places

#

but I think we can just do it once there, since the other things run after that

onyx hinge
#

Not necessarily, each one gets a fresh vm right?

#

Each job that does an apt get needs it afaik

tulip sleet
#
workflows/build.yml
177:        sudo apt-get install -y latexmk librsvg2-bin texlive-fonts-recommended texlive-latex-recommended texlive-latex-extra

workflows/pre-commit.yml
33:      run: sudo apt-get install -y gettext uncrustify

actions/deps/ports/broadcom/action.yml
10:        sudo apt-get install -y mtools

actions/deps/external/action.yml
33:      run: sudo apt-get install -y ninja-build
#

build.yml and pre-commit, yes, but the actions.deps/... I think those are used inside the workflows

#

hmm, maybe not, because they are used in individual board builds

#

from custom-board-build.yml

manic glacierBOT
#

I don't have a logic analyzer, but AnalogIn measurements indicate the same issue with:
Adafruit CircuitPython 7.3.3 on 2022-08-29; Adafruit Feather ESP32S2 with ESP32S2
It's amazing this hasn't been noticed until now.

Mini module:

2 IO18 on the module is pulled up to VDD33 through a 10 kΞ© resistor. For details, please refer to Figure
5 and Figure 6."
(https://www.espressif.com/sites/default/files/documentation/esp32-s2-mini-1_esp32-s2-mini-1u_datasheet_en.pdf p.11 via <https:/...

manic glacierBOT
#

I don't have a logic analyzer, but AnalogIn measurements indicate the same issue with: Adafruit CircuitPython 7.3.3 on 2022-08-29; Adafruit Feather ESP32S2 with ESP32S2 It's amazing this hasn't been noticed until now.

Mini module:

2 IO18 on the module is pulled up to VDD33 through a 10 kΞ© resistor. For details, please refer to Figure
5 and Figure 6."

(https://www.espressif.com/sites/default/files/documentation/esp32-s2-mini-1_esp32-s2-mini-1u_datasheet_en.pdf...

manic glacierBOT
#

GPIO18 is apparently used for serial bootloading as a TX line for UART1. I guess that's in the ROM. It is a strong design suggestion that this pin be tied to 3.3V to prevent it floating.
From https://www.espressif.com/sites/default/files/documentation/esp32-s2_hardware_design_guidelines_en.pdf:

GPIO18 works as U1RXD and is in an uncertain state when the chip is powered on, which may affect the chip’s entry into download boot mode. To solve this issue, add an external pull-up resistor....

manic glacierBOT
#

I'm experiencing the same bug on Adafruit CircuitPython 8.1.0-beta.1-27-g3871501b5 with QT PY ESP32-S2.

It would be very helpful to get this fixed especially when working with Adafruit IO.

Here's my code:

from microcontroller import watchdog
from watchdog import WatchDogMode
from time import sleep

wdt = watchdog
wdt.timeout = 10
wdt.mode = WatchDogMode.RESET

counter = 0
while True:
    counter += 1
    if counter >= 5:
        raise Exception("Simulated Error.")
 ...
manic glacierBOT
#

The decision about blitting zero width or height bitmaps is above my pay grade :grin: but in case it helps I poked around the libraries for awhile.

I believe the MagTag issue stems from this code fragment in the adafruit_bitmap_font.pcf libarary:

                width = metrics.right_side_bearing - metrics.left_side_bearing
                height = metrics.character_ascent + metrics.character_descent
                bitmap = bitmaps[i] = self.bitmap_class(width, height, 2)
       ...
manic glacierBOT
manic glacierBOT
#

Ran it with the try/except for 24 hours successfully. The device rebooted when I unplugged my multi-card reader. Windows must have gone around and touched every other USB device because the S3 weather station rebooted. That reset the clock back to 0 for me in looking into this issue. 24 hours without a single error is definitely an improvement as before it was about 10-12 hours reliably failing two days in a row. Granted I've only had the feather weather station up and running for 2 days s...

manic glacierBOT
manic glacierBOT
#

Thank you for your patience with this, and for doing the submoduling. This PR does still have at least one commit (50d82ef30e9392186dd43a449429d48c5277664d) in it that adds the original .zip files (and then one that takes them away). So after a regular merge, the intermediate commit will still be in the history, and the .zip files will be part of the history, and will get fetched when the rjepo is cloned.

I think we could do a squash merge (these are currently disabled on this repo), and g...

slender iron
#

@tulip sleet what's the state of everything?

tulip sleet
#

silabs resubmitted the PR. They made a repo for the .zip files and submoduled it, which is great. I plan to merge that PR as a squash merge, so the intermediate commits with the .zip file history disappear.

It may or may not be true that the .zip files are still accessible via the reflog even after I did git reset --hard HEAD~1; git push -f. If so I can clean that up later. Or, I think that the reflog will get gc'd after 90 days or something (I may be confused about that, but I know some things have a limited lifetime.)

slender iron
#

why did they leave the files zipped?

#

you could squash the branch before doing a merge

tulip sleet
#

it's like 150MB

#

I think the squash merge PR will be fine, and GitHub will figure out the squash, instead of my having to do it by hand and possibly make a mistake. I just have to temporarily enable squash merges and then disable it again.

slender iron
#

ya, I think that's ok. theres a lot of noise in the commits anyway

tulip sleet
#

yeah, thanks, I'll proceed.

slender iron
#

looks like the repo they submodule doesn't have zips

#

we should probably add a CI check for large files

#

or turn off lfs some how if that's our limit

tulip sleet
slender iron
#

didn't retiredwizard need to enable lfs for them?

tulip sleet
tulip sleet
#

regular file size limit (before LFS) is 50MB, so these were just under (or at least some of them were).

slender iron
#

ah, from the other sdk

tulip sleet
#

actually I see different sizes, maybe 100MB is the limit

tulip sleet
slender iron
#

we should have a CI test for file size

vague geyser
#

I'm glad to write some documentation if I can get an understanding of it all.

manic glacierBOT
jaunty juniper
#

like hex files downloaded from make code

#

unless it changed in a new version of v2 ?

#

I guess the site also has a feature to directly send the code to it

#

as far as I know it doesn't need to be put in bootloader mode though, the drive is always there

tulip sleet
#

it appears to be the same. They seem to have tried hard for upward compatibility, even to the extent of NOT taking advantage of the USB on the nRF52833

manic glacierBOT
jaunty juniper
#

yeah the USB pins are not connected

tulip sleet
#

the original design seemed to try to get as many manufacturers involved as possible πŸ™‚

jaunty juniper
#

(it's an LED matrix, requiring to actively "scan" through rows and columns)

manic glacierBOT
#

I did a test build and got this error. Only extra thing I did beforehand was sudo apt install default-jre, on Ubuntu 22.04.

...
arm-none-eabi-gcc -mcpu=cortex-m33 -mthumb -mfpu=fpv5-sp-d16 -mfloat-abi=hard -T"build-devkit_xg24_brd2601b/autogen/linkerfile.ld" --specs=nano.specs -Xlinker -Map=build-devkit_xg24_brd2601b/circuitpython_efr32.map -Wl,--gc-sections @build-devkit_xg24_brd2601b/linker_objs -Wl,--start-group -lgcc -lc -lm -lnosys /home/halbert/repos/adafruit/scircuitpython/por...
#

I did a test build and got this error. Only extra thing I did beforehand was sudo apt install default-jre, on Ubuntu 22.04.

...
arm-none-eabi-gcc -mcpu=cortex-m33 -mthumb -mfpu=fpv5-sp-d16 -mfloat-abi=hard -T"build-devkit_xg24_brd2601b/autogen/linkerfile.ld" --specs=nano.specs -Xlinker -Map=build-devkit_xg24_brd2601b/circuitpython_efr32.map -Wl,--gc-sections @build-devkit_xg24_brd2601b/linker_objs -Wl,--start-group -lgcc -lc -lm -lnosys /home/halbert/repos/adafruit/scircu...
#

I did a test build and got this error. Only extra thing I did beforehand was sudo apt install default-jre, on Ubuntu 22.04.

...
arm-none-eabi-gcc -mcpu=cortex-m33 -mthumb -mfpu=fpv5-sp-d16 -mfloat-abi=hard -T"build-devkit_xg24_brd2601b/autogen/linkerfile.ld" --specs=nano.specs -Xlinker -Map=build-devkit_xg24_brd2601b/circuitpython_efr32.map -Wl,--gc-sections @build-devkit_xg24_brd2601b/linker_objs -Wl,--start-group -lgcc -lc -lm -lnosys /home/halbert/repos/adafruit/scircu...
vague geyser
jaunty juniper
manic glacierBOT
#

I am not sure the intent of the CORS implementation in Web Workflow is correct. From my
limited understanding of the feature, it is not for server based request validation but
for informing the client what domains are OK for requests following. As it is currently
implemented in "web_workflow.c:_origin_ok", HTTP requests are rejected if the "Origin:"
HTTP header attribute is set and does not match one of the built-in domains or IP.
Returning a 403 error in this case does not seem correc...

#

Validate Origin/Host headers as follows:

  1. If No Origin: specified, then return OK
  2. If Origin: does not have http: scheme, treat as # 1 above
  3. If No Host given, and Origin: specified, return 403
  4. If both Host: and Origin: are specified, check for hostname match (including port #)
    Origin: is OK if same as Host: --> return Origin:, otherwise 403.
  5. Special debugging case allowed for Origin: equal "localhost" (ignoring port #)

See discussion in #7875 for rationale and deta...

slender iron
#

@brazen hatch I think I found a stack checking bug. It might be related to what you are seeing

brazen hatch
#

Most certainly, cuz I have veen looking everywhere for logic errors

#

where is it?

slender iron
#
STATIC void start_mp(supervisor_allocation *heap, supervisor_allocation *pystack) {
    supervisor_workflow_reset();

    // Stack limit should be less than real stack size, so we have a chance
    // to recover from limit hit.  (Limit is measured in bytes.) The top of the
    // stack is set to our current state. Not the actual top.
    mp_stack_ctrl_init();

    uint32_t* stack_bottom = stack_get_bottom();
    if (stack_bottom != NULL) {
        size_t stack_length = stack_get_length();
        mp_stack_set_top(stack_bottom + (stack_length / sizeof(uint32_t)));
        mp_stack_set_limit(stack_length - 1024);
    }
#

ctrl_init will set the top the stack to where it currently is

#

but we likely want it higher than that if it is fixed

#

this is my corrected version

#

where we set the top explicitly

#

otherwise our checking won't kick in until it is too late

brazen hatch
#

Well, I will try it on all esp.

slender iron
#

(that's in main.c)

brazen hatch
#

kk, thanks!

#

do keep in mind though, I build on pi400, may take an hour or so

slender iron
#

it didn't fix my crash though 🀦

#
(gdb) bt
#0  HardFault_Handler () at supervisor/port.c:313
#1  <signal handler called>
#2  0x98d5613c in ?? ()
#3  0x33f3deee in ?? ()
#

really helpful

brazen hatch
#

yea stack debugging do be like that

#

I still havent gotten jtag working on esp so I can’t even do that

slender iron
#

the panic handler will dump a backtrace for you on esp

#

and there is a way to add code to do it too

brazen hatch
#

yea that looks the same like your snippet though
it’s unknown addresses

slender iron
#

did you use the backtrace decoder tool?

brazen hatch
#

nothing really helps for stack debugging
tried it all

slender iron
#

πŸ‘

brazen hatch
#

it’s been the pot I have been boiling in for 2 months now

slender iron
#

🀞 this stack_ctrl thing fixes it

#

it does look like it applies to esp

#

since esp has a fixed stack too

wraith crow
#

Looks like the crash is gone, the recursion depth hard limit still is there but it probably make sense that this fix doesn't impact that

brazen hatch
#

I’m in kind of a mess, testing will take more

manic glacierBOT
brazen hatch
#

Said mess:
Dynamic console support.
It works, some prints are forgotten though and I need to swap em.
This was a nightmare.

#

Now finally going to test the pystack change.

brazen hatch
#

So my diff is now:

diff --git a/main.c b/main.c
index 6ff6202d6..f41d3956d 100644
--- a/main.c
+++ b/main.c
@@ -153,18 +153,17 @@ STATIC void start_mp(supervisor_allocation *heap, supervisor_allocation *pystack
     supervisor_workflow_reset();
 
     // Stack limit should be less than real stack size, so we have a chance
-    // to recover from limit hit.  (Limit is measured in bytes.)
+    // to recover from limit hit.  (Limit is measured in bytes.) The top of the
+    // stack is set to our current state. Not the actual top.
     mp_stack_ctrl_init();
 
-    if (stack_get_bottom() != NULL) {
-        mp_stack_set_limit(stack_get_length() - 1024);
-    }
-
-
     #if MICROPY_MAX_STACK_USAGE
     // _ezero (same as _ebss) is an int, so start 4 bytes above it.
-    if (stack_get_bottom() != NULL) {
-        mp_stack_set_bottom(stack_get_bottom());
+    uint32_t* stack_bottom = stack_get_bottom();
+    if (stack_bottom != NULL) {
+        size_t stack_length = stack_get_length();
+        mp_stack_set_top(stack_bottom + (stack_length / sizeof(uint32_t)));
+        mp_stack_set_limit(stack_length - 1024);
         mp_stack_fill_with_sentinel();
     }
     #endif

I just merged it with the existing code with my last braincell. It's cooking now. Pi 1fps.

#

Also, that is stack, not pystack.

tulip sleet
#

@brazen hatch do you do make -j4 on the Pi 4?

slender iron
#

it may be limited by the esp stack size now

#

(not sure which stack limits recursion)

#

there is a separate error for running out of pystack though

brazen hatch
#
You are in safe mode because:
CircuitPython core code crashed hard. Whoops!
NLR jump failed. Likely memory corruption.
#

I think I did it wrong.

slender iron
#

I didn't change the max stack usage part

brazen hatch
#

@wraith crow Could you toss me your main.c?

slender iron
#

the c stack still limits recursion I think

#

on some ports it is fixed and others it is dynamic

brazen hatch
tulip sleet
#

i just meant if you're not doing that, try it

brazen hatch
#

I have my 16% overclock enabled too.

#

It's a toasty keyboard.

tulip sleet
#

🚀

manic glacierBOT
brazen hatch
wraith crow
#

I'm actually out at the moment I'll send over the main.c when I'm back home, although I just pasted in Scott's code segment. And yep I was talking about the effective limit of the pystack

brazen hatch
wanton topaz
#

Hi folks, apologies, this is probably a silly question. I am using the latest build (8.0.5) for the Raspberry Pi Pico W. I want to use GPIO24 to tell if the board is charging or not, I can do this pretty simply in MicroPython with charging = Pin(24, Pin.IN) but I can't seem to find anyway to do this with CircuitPython.

When I look at the board pins.c file, I can see that it's not 'mapped' (Sorry if that's not the right term) and then I try to do things like charging = digitalio.DigitalInOut(board.GP24) it errors with AttributeError: 'module' object has no attribute 'GP24', as I would expect it to as it's not in the 'board' class.

Is there any way that I can read if this pin is 1/0 to tell if the board is charging using CircuitPython?

brazen hatch
#

24? I can't see it on the silk screen even.
Perhaps it has to do with the wifi module.

#

Wait let me fetch a normal pico.

#

Uhh, that isn't broken out even on a normal pico.

#

Am I blind or something?

#

Pins of the cpu not broken out, are not defined in the board pin definition.

#

If you want to map that pin, you will have to do a custom build.

tulip sleet
#

on the regular Pico, GPIO24 is used to sense that VBUS (USB voltage) is present

slender iron
#

There is microcontroller.pin which usually has all pins on the microcontroller even if they aren't broken out on the board

tulip sleet
#

there is no microcontroller.pin.GPIO24 on the Pico W; I thought there would be, but there's not, because it's reserved for radio module comm,

#

we didn't bother to make it accessible

tulip sleet
brazen hatch
#
def test(no):
  try:
    test(no+1)
  except RuntimeError:
    print("Done")

a = "for i in range(10): test(0)"
for j in range(3000): exec(a)
#

exec (at this scale) was immediately safemoding on all
Hopping on s2 now.

#

I also did a stack integrity check, pass.

#

Passing diff:

diff --git a/main.c b/main.c
index 6ff6202d6..f007154e1 100644
--- a/main.c
+++ b/main.c
@@ -153,22 +153,17 @@ STATIC void start_mp(supervisor_allocation *heap, supervisor_allocation *pystack
     supervisor_workflow_reset();
 
     // Stack limit should be less than real stack size, so we have a chance
-    // to recover from limit hit.  (Limit is measured in bytes.)
+    // to recover from limit hit.  (Limit is measured in bytes.) The top of the
+    // stack is set to our current state. Not the actual top.
     mp_stack_ctrl_init();
 
-    if (stack_get_bottom() != NULL) {
-        mp_stack_set_limit(stack_get_length() - 1024);
+    uint32_t* stack_bottom = stack_get_bottom();
+    if (stack_bottom != NULL) {
+        size_t stack_length = stack_get_length();
+        mp_stack_set_top(stack_bottom + (stack_length / sizeof(uint32_t)));
+        mp_stack_set_limit(stack_length - 1024);
     }
 
-
-    #if MICROPY_MAX_STACK_USAGE
-    // _ezero (same as _ebss) is an int, so start 4 bytes above it.
-    if (stack_get_bottom() != NULL) {
-        mp_stack_set_bottom(stack_get_bottom());
-        mp_stack_fill_with_sentinel();
-    }
-    #endif
-
     // Sync the file systems in case any used RAM from the GC to cache. As soon
     // as we re-init the GC all bets are off on the cache.
     filesystem_flush();
wanton topaz
#

@tulip sleet @slender iron - Ah right, that makes sense. I had used it with MicroPython on a regular pico so that is why I could do it. No bother. Thank you both so much for your help anyway. I forgot the RP2040 can't just grow extra pins because I want them.

slender iron
#

anyone use a stack larger than 4k on rp2040?

brazen hatch
#

And that was stable.

tulip sleet
wanton topaz
#

I fully didn't explain myself. Long day.

brazen hatch
wanton topaz
#

Ah I can just rely on my voltage figure being > 5v for charging! Sweet

brazen hatch
#

ESP32S2 PASS (12k stack)

#

Do I feel like firing up C3? Naaaahhh.
I do not have a working ESP32 board for now. So can't test that now either.

#

@slender iron I'd say go ahead and pr it.
It's a fix for this much. Since S3 is stable, the others will also be.
S3 was the worst in crashes with big pystacks.

manic glacierBOT
deep acorn
#

missing parameters in wifi module on 8.0.3 can anyone confirm these wifi.radio.connected returns boolean on 8.0.5 or 8.1 eps32boards

slender iron
deep acorn
#

thank you i will proceed to update @slender iron

slender iron
#

they usually match the state of main

tulip sleet
#

there is a selector at the bottom of the left sidebar

deep acorn
#

8.0.5 seems to also be missing the same few params

#

possibly missing from shared-bindings/wifi/Radio.h

jaunty juniper
#

Add wifi.radio.connected, wifi.radio.ap_active

#

prior to that, you would test if there is an ip_address to know if you are connected

deep acorn
#

@jaunty juniper thank you for finding the implantation date. would there be a docker image i could clone down and build main firmware for my feathertftreverse

jaunty juniper
#

was there no docs build for 8.0.5 ? (might not have been needed)

tulip sleet
#

or the doc build failed for some reason

jaunty juniper
deep acorn
#

just to confirm main on git is not the prebuilt 8.0.5

jaunty juniper
#

every merged PR is built and uploaded there

deep acorn
#

yeah i can download whole main which i did last week to read through some source code to compare with micropython i can redownload it but a bit confused on build process as i got me an expressif docker container and was using a one line docker CL to build micropython for S3

#

yeah i have 8.0.5 installed as we speak still no connect boolean

jaunty juniper
#

no I mean you can download the binary for latest

deep acorn
#

bin update?

jaunty juniper
#

what board is it ?

deep acorn
#

feather s2 tft reverse

deep acorn
#

standby for confirm pretty sure its gonna be 8.1.0

slender iron
#

it will be newer than 8.1

jaunty juniper
#

yeah, latest

slender iron
#

(newer than the current beta)

deep acorn
#

we are in there ppl

jaunty juniper
#

a version like 8.1.0-beta.1-23-g0aea45516 means about 23 commits after 8.1.0

deep acorn
#

@jaunty juniper @slender iron @tulip sleet thank all three of you for directing me to various trouble shooting. hope version filter gets fixed on site

tulip sleet
#

the version filter was broken in 8.0.x due to a Javascript problem, but it works with latest, at least for me

deep acorn
#

someone get me a frontend dev lolz jp looks like a jquery not defined error not related to circuitpython

wraith crow
slender iron
wraith crow
#

The size at which increasing pystack no longer increases the recursion limit. I thought you said that the c stack was probably the limiting factor after pystack reaches a certain size.

slender iron
#

ya, I think the cstack is restricting recursion

#

pystack will hold the frames themselves I think

#

the python portion of the frame at least

#

(instead of the heap potentially)

wraith crow
#

That's consistent with what I was seeing when I ran stackless builds. I believe I was turning off the cstack use as well in that case.

slender iron
#

I'm not sure how stackless works

wraith crow
#

I don't think the stackless builds were running into the recursion limits.

brazen hatch
brazen hatch
#

But for anything else, the depth settable pystack reaches, is plenty enough. Except for c3 I guess.

wraith crow
#

Dinner bell is ringing, but I'll build the ESP32 Huzzah image tonight and test on that as well. I'm really excited the dynamic pystack looks like it's getting close to done 😁

wraith crow
brazen hatch
#

hard horrible

wraith crow
#

Feather ESP32 Huzzah ran through the test script without a crash with pystack set to 9000, the recursion limit appears to be around a pystack size of 3900.

brazen hatch
#

Thanks for testing it!
So this issue is entirely fixed by this change.
The effective limit seems so smaller on esp than on rp2.
Perhaps combining 'reserved psram' and 'resizable pystack', we can go even further.

manic glacierBOT
#

Add a new mode to OnDiskGif with the use_palette keyword. Set to true the GIF will decode to an indexed bitmap and a palette. On a 240x240 GIF this saved around 54KB. Yay! I have not done any speed comparisons.

As the GIF gets smaller the palette becomes less efficient because (right now) I do not dynamically change the palette size which I believe can change per frame. So it allocated 256 entries to use (largest possible).

import time
import gc
import board
import...
manic glacierBOT
manic glacierBOT
#

Does TinyUF2 touch the RTC?

After many rabbit holes in the circuitpython code, I didn't find where the default seconds are set. It seems possible that some code path is missing a check for whether to reset the date, or using an unitialized date to set the RTC.

But with more testing, I did find that the issue trigger is more complex. Hard reset always resets the date to 2000, as expected. Early CP8 versions will maintain the year 2000 date after a hard reset and through several UF2 loads...

manic glacierBOT
#

I think there's no reason that synth.envelope = synthio.Envelope(attack_time=37, ...) couldn't be made to work.

If it's what y'all want I can abandon the arbitrary-buffer-based way of this PR as it stands, and switch to something more like the mozzi's envelope generator.

switching synthesis to be based on frequency in (possibly floating point) Hz instead of MIDI notes would be a topic for a future PR.

tall finch
#

Is there any way for me to add a built-in module to a circuitpython uf2? The end goal is to have these custom libraries available to users of a robot I am designing (using Pico W) without having to install the libraries manually/separately.

tulip sleet
tall finch
#

Thank you!

plucky tulip
#

There are two waiting PRs from Neradoc and one from me on adafruit_httpserver, anecdata already reviewed Neradoc's ones. Could someone please review mine and merge them all so that it is possible to continue working on v3.0.0 of the lib?

manic glacierBOT
slender iron
brazen hatch
#

Well, that can already be done the old way.

#

So are you pr'ing the fix?
Or do you want us to handle it?

#

It's your find afterall.

slender iron
#

I was going to include the fix in my dvi PR

#

which is getting closer and closer

#

the c stack size is only dynamic on some ports. I don't think it is on ESP

#

because we inherit the stack size from esp-idf

manic glacierBOT
#
#

If it's what y'all want I can abandon the arbitrary-buffer-based way of this PR as it stands, and switch to something more like the mozzi's envelope generator.

I'm ok with a buffer if it is a shape. I'd avoid a buffer if you'd want each index to mean something else. In that case, it should be named values.

(I leave the synth decisions to folks who've done it.)

crimson ferry
#

@plucky tulip Dan has looked at the code, I will test. Should Neradoc's two be merged, or what's the desired sequence once these three are all approved?

plucky tulip
crimson ferry
#

OK, I'll merge Neradoc's two and use that as the baseline for testing, thanks.

plucky tulip
#

Ok, If there are any conflicts with my branch after merge I will fix them right away, Thank you

slender iron
#

@blissful pollen when testing the gif stuff with dvi did you update to my latest branch?

plucky tulip
#

@crimson ferry Conflicts resolved, thank you for merging and in advance for testing

manic glacierBOT
blissful pollen
slender iron
#

kk. I think I need to update the picodvi module. should be good when you are off work πŸ™‚

#

limor is testing now

thorny jay
# slender iron limor is testing now

Spoiler alert on both "Show and Tell" and "Ask an Engineer"!
Did you ear about an incoming Pimoroni product that use two RP2040 and SRAM chip to do a higher resolution DVI output.
Apparently it permit to dedicate the RP2040 to making the DVI signal, and the added memory use as a buffer where on RP fill in write mode, then the display connected RP read that same memory.
This is what I have understood from a Tom's hardware video.

slender iron
#

ya, I saw mention of that. I was a bit worried when they said "I assume that means they'll support 1080p"

#

my understanding is that clock speed is a limiting factor as well as ram

thorny jay
#

Not sure that is even possible, but all of this is above my league.

manic glacierBOT
blissful pollen
#

@slender iron Got the latest code, haven't done a lot of testing yet but the colors in the GIF using either colorconverter and palettes are off. Trying to figure out why still.

slender iron
#

got a pic?

blissful pollen
#

Left side in using ColorConverter, right using Palette

slender iron
#

hrm

#

the colors that show seem right

blissful pollen
#

The CC one may be something about processing speed. I previously cut CC down to the bare minimum and seemed better. Palette confuses me. The colors are slightly off (compared to the CC colors that are correct but missing in places)

slender iron
#

do you have test code I could run in my branch?

blissful pollen
#

Sure though the GIF palette code is on that PR I just submitted

slender iron
#

I can start with the cc version

blissful pollen
#
import board
import gifio
import displayio
import time
import picodvi
import framebufferio
import gc

display = board.DISPLAY
splash = displayio.Group()

odgcc = gifio.OnDiskGif('/homer-64x64.gif', use_palette=False)
odgp = gifio.OnDiskGif('/homer-64x64.gif', use_palette=True)

start = time.monotonic()
next_delay = odgcc.next_frame() # Load the first frame
odgp.next_frame()
end = time.monotonic()
overhead = end - start

facecc = displayio.TileGrid(odgcc.bitmap, pixel_shader=displayio.ColorConverter(input_colorspace=displayio.Colorspace.RGB565_SWAPPED))
facep = displayio.TileGrid(odgp.bitmap, pixel_shader=odgp.palette)
facep.x = 64

#splash.append(facecc)
splash.append(facep)

display.root_group = splash

# Display repeatedly.
while True:
    #time.sleep(max(0, next_delay - overhead))
    time.sleep(0.3)
    next_delay = odgcc.next_frame()
    next_delay = odgp.next_frame()
slender iron
#

can you send the gif too please?

#

@blissful pollen gif please

blissful pollen
#

Sorry down to one monitor so I can't have discord nicely on the second while testing this

slender iron
#

πŸ˜„ np

#

k, colors are wrong for me too

blissful pollen
#

Okay one small fix line #60 in the shared-module ColorConverter.c it shifts green 12 to the right. It should be 13:
uint32_t g3 = (color_rgb888 >> 13) & 0x7;
That fixed a lot for the palette

slender iron
#

πŸ‘

#

I'm only running the cc version atm

blissful pollen
#

I'll send another picture of the palette only with a larger GIF. There are some missing pixels still but much less

slender iron
#

seems like the black pixels could be incorrectly transparent ones

#

ah, missed a return

#

on line 319

blissful pollen
#

Found it

#

ha was just about to type the same πŸ™‚

slender iron
#

πŸ˜„

blissful pollen
#

Seems to work for me

slender iron
#

ya, me too

#

k, force pushed

#

time to pr

manic glacierBOT
#

PicoDVI in CP support 640x480 and 800x480 on Feather DVI, Pico and Pico W. 1 and 2 bit grayscale are full resolution. 8 and 16 bit color are half resolution.

Memory layout is modified to give the top most 4k of ram to the second core. Its MPU is used to prevent flash access after startup.

The port saved word is moved to a watchdog scratch register so that it doesn't get overwritten by other things in RAM.

Right align status bar and scroll area. This normally gives a few pixels of pad...

blissful pollen
#

I'll try to test it some more hopefully tomorrow

slender iron
#

are you going to show and tell it today?

#

(otherwise I will)

#

I can do turtle instead

blissful pollen
#

I was thinking of it, to show off the Palette PR too

slender iron
#

πŸ‘ I'll do turtle

blissful pollen
#

I loaded up a 320x240 gif that would not otherwise fit

slender iron
#

nice!

#

thanks for the help @blissful pollen

blissful pollen
#

Welcome, thanks for adding the support! You did the heavy lifting. Now to think where I can plug a RP2040 DVI into an unsuspecting TV with HDMI πŸ™‚

slender iron
#

πŸ™‚

manic glacierBOT
manic glacierBOT
manic glacierBOT
manic glacierBOT
jaunty juniper
#

@tulip sleet is it worth adding this to the 8.0.0 release notes ? It's not referenced in any release, and somebody didn't find the info when looking for it

  • The property supervisor.runtime.autoreload replaces supervisor.disable_autoreload() and supervisor.enable_autoreload()
tulip sleet
jaunty juniper
#

thanks toebeans

manic glacierBOT
#

Thanks for this!

I tested with a Metro ESP32-S2 and a Pi Pico W. Works great. I translated my current ASCII passphrase to hex and that worked fine.

I made a couple of changes:

  • Moved validation routine out of argcheck.c, since it was rather specific, and not like the other validation routines. Used unichar_isxdigit() to do hex-digit checking.
  • Added password-length documentation.
  • Ran make translate.

I will merge when/if the build succeeds.

lone sandalBOT
jaunty juniper
#

I just ran into that, maybe the instructions to allow read/write to the actions should be not only in the troubleshooting, but prominent as a step of the setup for releases

manic glacierBOT
#

@sean-bayarea I've tested the connection issue of this example without problems on a Pico W running CircuitPython 8.0.5 and the latest MiniMQTT library. I am connecting to the HiveMQ public MQTT broker.:

image

Below is a copy of the ipynb file you uploaded earlier. At the bottom, I've updated your code snippet a bit to move the try/except inside the while True: loop.:

[CPX_blin...

#

I would say the device/server side is fixed if CORS support is desired (What problem is is solving??? Example, please).

The REST / Serial API is now fully functional independent of device host name used. As far as the web-editor goes, that is not fixed - it needs some code changes to allow normal naming without restrictions.

BTW - not all environments / platforms support mDNS out-of-the-box and/or may not resolve .local names. mDNS does not traverse subnets. However, there exists p...

slim aspen
#

Hi, has anyone programmed the KTIR0711S reflective sensors in CircuitPython? I don't know how to go about it, and I can't find anyone doing anything with them on the internet.

manic glacierBOT
manic glacierBOT
manic glacierBOT
brazen hatch
#

8.1.0 as time goes on is looking tastier and tastier.

manic glacierBOT
#

Tagging @kvc0 and maybe @FoamyGuy:

Compilation generates this mild warning:

../../shared-module/vectorio/Circle.c: In function 'common_hal_vectorio_circle_get_pixel':
../../shared-module/vectorio/Circle.c:26:22: warning: taking the absolute value of unsigned type 'uint16_t' {aka 'short unsigned int'} has no effect [-Wabsolute-value]
   26 |     int16_t radius = abs(self->radius);
      |                      ^~~

The radius struct element is already unsigned:
https://githu...

edgy edge
edgy edge
#

but maybe a lot undone: ifeq ($(MCU_SERIES),L4) # Not yet implemented common-hal modules: CIRCUITPY_ANALOGIO ?= 0 CIRCUITPY_AUDIOBUSIO ?= 0 CIRCUITPY_AUDIOIO ?= 0 CIRCUITPY_COUNTIO ?= 0 CIRCUITPY_FREQUENCYIO ?= 0 CIRCUITPY_I2CTARGET ?= 0 CIRCUITPY_NEOPIXEL_WRITE ?= 0 CIRCUITPY_NVM ?= 0 CIRCUITPY_ROTARYIO ?= 0 CIRCUITPY_RTC ?= 1 # todo - this varies between devices in the series # This slide deck https://www.st.com/content/ccc/resource/training/technical/product_training/98/89/c8/6c/3e/e9/49/79/STM32L4_Peripheral_USB.pdf/files/STM32L4_Peripheral_USB.pdf/jcr:content/translations/en.STM32L4_Peripheral_USB.pdf # cites 16 endpoints, 8 endpoint pairs, while section 3.39 of the L4R5 datasheet states 6 endpoint pairs. USB_NUM_ENDPOINT_PAIRS = 6 UF2_FAMILY_ID ?= 0x00ff6919

#

and bluetooth coprocessor BlueNRG-M2SP

#

any other ports to use as template?