#Nanoleaf beta firmware discussion

1 messages · Page 1 of 1 (latest)

digital star
#

@bleak pond @hardy rivet @normal plank guess Ben threw in the towel 😂

bleak pond
#

That salute means something 👀

#

I don’t mean to read into it too much, but it was legit a one man show in that channel for a while there, only Ben

digital star
#

Yeah I feel like their efforts they are doing now should have been pre launch not 9 months later

hardy rivet
#

Great idea to have a chat for us HA Nanoleaf users. 😃

bleak pond
#

I mean, to be fair we are mostly the only ones providing actual valuable feedback. I guess that’s the perks of HA 😝

hardy rivet
bleak pond
#

New update just dropped, good luck to you unpaid alpha testers 🙃

dense merlin
#

wow i just made the update to 126 and now comes the 134 xD really?

#

in the same minute i starred and finshed the update to 126

normal plank
#

Haha, I hope it’s better

#

Right now the bulbs are not usable for me

dense merlin
#

havent had any issues with essential bulbs but my downlights are fucked up

#

they just no realse an update for them

digital star
bleak pond
#

Not that any of them would ever use HA by the sounds of it

hardy rivet
#

I really don’t know if I should be happy or better cry, when a new update is released…

dense merlin
#

is there a changelog for the beta releases?

digital star
#

Yeah they have one per release

dense merlin
#

u mind linking it

normal plank
#

And let’s hope we didn’t get Benjamin fired 😂

digital star
#

Their whole test setup seems ridiculous. Take the light turning on. It hasn't always happened, but there's no way to report which firmware it started on

bleak pond
digital star
hardy rivet
bleak pond
hardy rivet
normal plank
#

I think they are trying to fix too many things at once right now

hardy rivet
bleak pond
#

That’s honestly sad on their behalf….

hardy rivet
#

I went to bed 10 minutes ago. Good night

bleak pond
#

I honestly don’t blame Ben for dipping out while he can, must be a shitshow behind the scenes

normal plank
bleak pond
#

They are product managers, I don’t think they directly interface with the products

#

Rather are just the middle man, at least that’s the vibe I’m getting from most of them

#

Other than Ben, he seemed to know what he was talking about

hardy rivet
digital star
#

Product managers can definitely be engineers

#

They seem to be fairly well informed

#

Part of the issue is they don't have experienced matter devs (nobody does) but it seems they aren't even doing software engineering 101

normal plank
normal plank
hardy rivet
#

I am a service designer… 😅

bleak pond
#

Yall got these fancy titles, I’m just a uni student 🫣

hardy rivet
hardy rivet
normal plank
novel snow
#

Just updated to 3.6.134, shows as 3.6.-122 🙈

bleak pond
#

Gotta live while ya young you know how it goes

normal plank
bleak pond
#

That’s cooked hey

novel snow
#

Overflow I guess 😅 It did show correct before update

normal plank
normal plank
digital star
#

All I had was a busted ass laptop in school 🥲

bleak pond
#

Couldn’t miss out on the drama in that channel

#

It’s a nice mix between people (unfortunately) complaining about a breakdown in their marriage coz of the bulbs and the echo chamber of some of the valid statements that seemingly go no where

hardy rivet
digital star
#

What did it even fix? The two issues they said for sure they were targeting, one was unsure and one was (fix incoming?)

#

Or were (1) and (2) actual things they were hoping to have fixed

hardy rivet
#

They are hoping. But yes, we don’t know what exactly should be solved.

#

(1) and maybe (2)

bleak pond
#

I guess they are trying to narrow down if it’s an issue with “power” at all, if that be with voltage, or apparently now the capacitors

digital star
digital star
bleak pond
#

Must be a budget constraint of some sort

polar wolf
#

Corp Greed is what it is

bleak pond
#

I mean, Nanoleaf as a company rely on high margins and new products

#

But this is just not good at all

polar wolf
#

I really want them to get slapped with the European consumer rights law or whatever it is

bleak pond
#

Someone has to complain first

#

I mean, I remember a dude in that channel a while ago threatening it, but not sure it went that far

digital star
#

Just raw speculation, but I'd imagine the market for smart bulbs involves a relatively small market but with a potential for many sales

That is fewer customers but they might buy many products

I'm sure their bottom line is hurting, you can just look at the general matter chat where people bought one to test then decided to not buy more

bleak pond
#

Could imagine how many complaints they are getting on their website

#

They have had no live chat feature for a few months now, and no doubt they are inundated with return requests/constant troubleshooting

digital star
#

I really wouldn't be surprised if they step away from thread/matter

#

Skylight was supposed to be a BR and now nothing

bleak pond
#

Weird part is their shapes/lines etc are not actually that bad of a product

#

At least from a stability standpoint in a thread only product

#

There was talks about it being a matter/thread product in an update, but obviously that never happened

digital star
#

I have the elements and they work fine

#

At this point I'm just looking at some matter over WiFi bulb.

If inovelli ships their switches before NL fixes their shit I'm gonna toss them

bleak pond
normal plank
bleak pond
#

Yeah right

digital star
#

So I had an option to update from 3.6.-122 to 3.6.134 and now I'm worried about what's going to happen

bleak pond
#

Strange that this is the first time it’s ever happened in a beta

normal plank
#

I don’t have the problem on iOS though

digital star
#

The android NL app is so terrible

#

Devices are never reachable

jovial raft
#

One day everything will work...

digital star
#

Sigh I can't get any devices to update to latest

digital star
#

Having to point out the matter spec does not bode well

bleak pond
#

its the only version i could get my hands on, 1.2 core you need to be a CSA member

#

i was just going off of the 1.0 spec which is public info 😅

#

but i feel like there is massive internal stuggles, so i mean im happy to help if it means a better end product for everyone

normal plank
#

Is the network diagnostics cluster actually updated after the first connection?

#

Also the reporting of the up address is part of the core sdk, not anything Nanoleaf is doing

digital star
#

If that's the case nobody else is using silabs then or else we'd see the problem elsewhere

bleak pond
#

from a ton of vendors, nanoleaf appear to be the only one not providing some of the diagnostic fields

#

the device type is also "unknown"

digital star
#

Mine come up as routing end device

#

For type

bleak pond
#

i was just going off of this dudes, maybe it was down at the time hence it being unknown

#

its apart of the routing table enum 😅

#

it appears they adhear to the Thread Network Diagnostics Cluster but not the General Diagnostics Cluster?

#

this is way to dank

digital star
#

I don't have sensors like that on mine either so idk

novel snow
#

HA is using NetworkInterfaces attribute from General Diagnostics Cluster.

#

Afaik reading from the node cache. So not typically updated. Unless you trigger interview

bleak pond
#

yeah, so they just must not be loading any types into id 5 or 6, which is the v4 and v6 address respectively with anything, which makes sense

#

oh well, will tell them and will just have to wait and see if they do anything about it

novel snow
#

The cluster is mandatory, and the standard says each logical network interface shall be represented.

#

Since these are connected devices, they must have at least one network interface 😅

#

I am kinda surprised that the certification test harness doesn't catch that.

#

FWIW, Aqara P2 is missing them too. 😢

digital star
#

Yeah missing on my P2

bleak pond
#

what do they use? sillabs?

novel snow
#

I never opened one, but was wondering in the past

#

Maybe today is the day? 🤔

bleak pond
#

i cant see any press release about it

#

asked bard, and didnt say anything about matter

#

but it did mention this

Microcontrollers:

Zigbee chips: Aqara's smart home ecosystem mainly relies on Zigbee connectivity. They likely use Zigbee microcontrollers from manufacturers like Texas Instruments (TI) CC2530/CC2531 or Silicon Labs EM35x series.

#

so no doubt its an issue upstream with silabs if they are working together still

novel snow
#

Ok opened it. I take bets! 😅 l

bleak pond
#

EFR32MG24? the same junk nanoleaf uses?

novel snow
#

Nope

bleak pond
#

sillabs?

novel snow
#

No SiLabs, not this time around 🙂

#

I would have guessed TElink, since Aqara is PRC too, but its not that

bleak pond
#

i mean whats left? nordic or texas?

novel snow
#

Neither of those two 😅

#

NXP

bleak pond
#

yeah right, interesting

novel snow
#

So seems Aqara Door and Window Sensor P2 is using a NXP K32W041AM then (thanks Discord for remember for me 😅 )

#

Datasheet is for K32W041A/K32W041AM

 640 KB flash, K32W041AM has an additional 1 MB data flash
 152 KB SRAM

M has additional flash... Wonder what that M stands for Matter? 😅 🙈

digital star
#

Tuya uses the MG24 apparently

bleak pond
#

Does anyone even have any Tuya matter stuff?

digital star
#

I've heard it sucks and that they were in trouble so I don't

bleak pond
#

I am not at all surprised to hear that

digital star
#

So in the NL discord they posted the bug website. if you look around you'll find their internal product roadmap for devices 🧐

bleak pond
#

Haha no shit, let’s see

#

Oh wait yeah, maybe it’s intentional

#

“Quests”

hardy rivet
#

Uff, Wifi products only.

digital star
#

Looks like switches in Q2 but might be their last thread product for a bit

#

But to be fair, some of the larger products aren't really amenable to thread due to the throughput needed for their scenes and 4D stuff

hardy rivet
#

Yes, that’s true. But a need a Thread Light switch and I do not see the Sense Controls on their timeline.

#

It’s also not Matter certified. So, maybe we will never see it.

#

Instead they have a wired light switch and WiFi light switch on their roadmap. Hm…

bleak pond
#

I suspect they will give up with thread, they clearly don’t seem to be enjoying all of these issues

digital star
digital star
#

@hardy rivet @polar wolf I'm going to file a bug report later today on the NL platform for the light on pop issue

bleak pond
#

Well, Nanoleaf seam to have leaked their hardware beta channel

#

“Blocks” gosh this can’t be good

digital star
#

I'm really interested in the skylight but swore off any more stuff from them until they fix their essentials

bleak pond
#

the skylights are just wifi from what ive seen from other people

#

no matter, no thread border router

digital star
#

Yeah that doesn't bother me

#

Just on principle though

#

They sold me broken shit so why reward them with more sales?

jovial raft
#

I will still be buying the wireless Sense+ controls if Nanoleaf ever releases them.

bleak pond
#

It’s been delayed more times than you can count

#

It was announced at CES 22 yeah?

jovial raft
bleak pond
#

That’s really bad

#

4 and a bit years delayed

digital star
#

I think I saw Q2 2024 somewhere

#

But I also heard 2021 🙂

jovial raft
#

I did find the roadmap, it has lightswitches on it for Q1 and Q2 this year.

bleak pond
#

Yeah, but nothing to do with thread

jovial raft
#

I wonder if they will have Matter Binding

bleak pond
#

Unless they have just given up and made it wifi

digital star
#

All -- trying to track something. If you have bulbs go down try browsing for the _matter._tcp service with some mDNS browser and see if it brings them back. I had it happen twice. Maybe coincidence maybe nor

#

(avahi-browse, bonjour, service browser etc)

hardy rivet
digital star
#

I wonder if polling the devices causes HA to see them or something

#

Like HA isn't always picking them up for some reason

hardy rivet
#

By the way… I already wrote it in the Matter channel. Yesterday evening I migrated my HAOS from a virtual machine to HA Yellow. I had some problems in the beginning, because some bulbs didn’t become available again. Power cycling the bulbs, restarting/rebooting HA didn’t help. Everything was fine in Apple Home. So, I decided to disable the Matter integration for some minutes. After enabling it again, 6 of my 14 Nanoleaf bulbs were unavailable, while all 26 EVE devices worked directly. After the power cycling the bulbs again, they all came back.

hardy rivet
#

Today 2 of my bulbs became unavailable, one for 10 seconds, the other for 3 mins. Much better then 3 or 5 hours. But still too early, to say that there is something wrong with my KVM HAOS. At the moment it at least looks so.

digital star
hardy rivet
#

@digital star Great, at the moment, I am not willing to test/validate this. I am happy that all devices are working. At the moment I want to test, if there is a difference between HAOS in a VM vs. bare metal. When I know, how this works, I can crash my system by a light group including all of my paired bulbs. 😂

digital star
#

Okay I can confirm the mDNS scan on my end

#

Just had two bulbs that came back immediately after calling avahi-browse

hardy rivet
digital star
#

I think NL bulbs are crashing still but there's something strange with the HA mDNS also going on where it doesn't pick up devices

hardy rivet
bleak pond
#

yeah, seems to be the case. as it would make sense why its online in other ecosystems, but not HA

digital star
bleak pond
#

yeah, i have it active in apple home no worries, but not HA (at all)

hardy rivet
#

Yes, my devices always work in Apple Home, when unavailable in HA.

bleak pond
#

yeah, they all on 3.5.45?

#

or is the downlights a version below the bulbs?

hardy rivet
#

I don’t own downlights…

#

All my bulbs (A19, GU10) are on 3.6.136.

bleak pond
#

i wonder if the downlights dont run the same fw/slightly modified version as the bulbs

hardy rivet
#

What do we do with this finding. I am unsure if Marcel or Stefan are reading this thread.

bleak pond
#

not sure. other than wait for the new matter server update, which has a ton of fixes

digital star
#

Not related to mDNS but I did see Marcel added transitions for lights (be warned, they suck)

#

Someone thought it was a great idea to hardcode the number of transition steps based on transition time 🙈

digital star
jovial raft
digital star
jovial raft
#

To update my experience, I'm not using the Nanoleaf beta and I've previously had a ton of issues. However since updating to Home Assistant 2023.2.1 my Thread network is significantly more stable. I still get occasional disconnects but I no longer have the issue where the entire Thread network crashes. Thread devices also reconnect much faster.

#

However sometimes lights take several seconds to gradually turn on to full brightness. I've not had this issue previously

digital star
#

Like it takes a while for them to receive the command or the ramp to on is slow?

jovial raft
#

The latter. The light starts to turn on but it gradually brightens until it's fully on. I know these lights have a native transition but it's much slower than it's supposed to be.

digital star
#

I know they adjusted something with that in the latest beta FW. Something to do with the capacitors and the random turn ons

#

Some of mine pop instant on though lmao

jovial raft
#

I've not had the random turn ons issue. I'm sure they adjusted many things but it's mostly stable for me now, so I prefer not to try the beta.

#

Another strange minor issue, sometimes when I turn off a light with the actual switch (non-smart switch), they flash red before turning off. It's really not a big deal, it just seems alarming.

digital star
#

Yeah but what I'm trying to say is the slower turn in is related to the random turn on fix. I think they will optimize that in the future

polar wolf
#

Have not had it with latest beta tho

#

It was a weird bug for sure. Definitely not normal behavior

digital star
#

Anyone gonna try out the switches?

#

I think the form factor looks unusual

bleak pond
#

yeah, i have signed up. but being australian i doubt ill get selected

#

they no doubt are the wireless/remote ones

digital star
#

Yep you're right I found a picture of them and the hard wired has an actual scroll wheel

hardy rivet
#

I also signed up. I am from Germany

jovial raft
#

I signed up, I was planning to get the wireless ones anyway. I live in a rental so don't want anything hard-wired.

bleak pond
#

I guess, would be interested to see if I get selected or what

#

I stated I got a pretty robust set up full of different ecosystems and devices/BR’s

#

Also yeah, would of been nice if they specified if the device was remote or hardwired 😅

digital star
#

The latest matter server beta should improve things where bulbs are unavailable in HA but available in homekit/Google home etc.

jovial raft
#

Nice great to hear!

jovial raft
# polar wolf I had this issue! I think I had to turn it off for a bit and back on. It was def...

So I did try turning off that slow light earlier this week. It solved that minor slow toggle problem but now the connectivity of all the other lights is awful again!

My speculation is that one always-on light was serving as a Thread Mesh Extender (maybe also the reason it was slow on its own), and turning it off destabilized the entire Thread network...

At this point I probably should try the beta firmware, but I've not had time to deal with it this week.

digital star
jovial raft
#

All my lights are in the Nanoleaf app, they always work with Bluetooth fallback and occasionally with Thread.

#

When the Thread network is healthy most of the bulbs indeed serve as Mesh Extenders.

bleak pond
#

Anyone else get into the sense+ button program?

#

I just got an invite

polar wolf
#

Nope

normal plank
#

Yeah

digital star
#

I didn't sign up since I have no interest in the remotes

jovial raft
#

Not me 😞

polar wolf
#

Anyone else getting chip error

bleak pond
#

w/ the new nanoleaf firmware?

polar wolf
#

Nevermind its just taking a while to reconnect

#

But yooo its super snappy now

digital star
digital star
#

Okay I've got a ton of errors now too

jovial raft
#

Looks like they are working on a good amount of new Thread devices!

hardy rivet
digital star
#

Hows this version for you guys? It's terrible for mw

#

Every light is down and I can't get any of them back

hardy rivet
#

I already wrote my feedback in the Nanoleaf community:

I also had issues with updating 4 of my 16 paired bulbs. But the update mostly worked, after a second or third try. Except one bulb didn’t do anything. I had to reset that bulb to factory defaults. After that I could update the bulb.

Then I was in the situation, that all bulbs were available in Apple Home and all but one bulb were available in Home Assistant. No re-meshing. Power cycling the one unavailable bulb didn’t work. I waited one hour. But it didn’t come back. Besides a lot of bulbs were available, but they didn’t react or it took a lot of time (more than 30 seconds).

So, I decided to reboot my 7 Apple Thread Border Routers. When all devices were back online in Apple Home, I rebooted my dedicated Home Assistant machine (HA Yellow). But 8 of my 16 bulbs didn’t come back by themselves. I waited two hours. So, I am now power cycling these bulbs one after the other and they come back to Apple Home and Home Assistant.

The bulbs that are already available are really snappy and switching them on/off is awesome fast.

After rebooting my Apple Thread Border Routers and my HA Yellow my 32 EVE Matter over Thread devices became available on its own. No power cycling needed.

normal plank
digital star
#

Yeh it doesn't do anything

#

My log is filled with

2024-02-27 10:05:44 core-matter-server chip.native.SC[126] CHIP_ERROR CASESession timed out while waiting for a response from the peer. Current state was 1
#

Other matter over thread devices are fine

hardy rivet
#

All devices and bulbs were up and running after rebooting all TBRs and HA, but now I already have one bulb that became unavailable in HA and also in Apple Home. Uff, never had that before, that a bulb also got unavailable in Apple Home.

#

From the HA log it became unavailable 45 minutes ago.

uneven acorn
#

Has anyone shared a light to HA using the new firmware (3.6.152)? I have two GU10’s installed new, updated. Now if I try share to HA, copy code then enter code in HA, it doesn’t see the device…

Usually, the device that you copied the code for is shown as a nearby device…
Now it’s only shown as all devices… and they all have the same name there so I don’t know which one it is 🥲

It’s as if pairing mode does not turn on.

If I check in the thread section of the NL app, all lights are listed as “Mesh Extenders”, except these two. They are listed as “Child”. Unsure if it’s related, but it’s a difference.

uneven acorn
#

Got it working. I just rebooted the closest AppleTV, and then they showed up as “Neraby Devices”, and it all went smoothly

hardy rivet
digital star
#

I set a deadline for myself, that if inovelli ships their switches before nanoleaf fixes their shit, I'm gonna suck it up and replace them

#

Replace the NL bulbs*

hardy rivet
#

Innovelli is not an option here in Europe, or?

hardy rivet
digital star
# hardy rivet A switch is not a bulb… So, you want to go back to dumb bulbs?

Inovelli has a smart bulb mode where current remains, but pressing the switch sends a message to HA that the switch has been pressed. Then you can use that to trigger an automation to turn the lights on/off.

My reasoning is if, another company has launched a new matter over thread product in the time that NL has been screwing around with their beta program, then there isn't a lot of hope for them

digital star
hardy rivet
digital star
#

Yes correct but I might just get wifi bulbs and be done

hardy rivet
#

Ok, maybe I will also go back to Philips Hue. But I think I can wait for the next 3 or 4 beta firmware updates before I give up. But I understand you. I have better things to do, as fixing something, that is not fixable…

digital star
#

Well three months is a bit ridiculous

#

For something I already paid for

#

It's not something like the matter server which is (mostly) free

ionic bramble
#

We maybe will see bulbs from another big player in the near future. I came accross their product testing application form and you can sign up for GU10 & E26/E27 thread bulbs.

digital star
#

Wow that would be great. I've got some P2 sensors and they have been reliable

novel snow
#

Just looked into an issue with my Nanoleaf GU10 (FW 3.6.136) comming not back online, and it seems it did not refresh the SRP entries on the BR, and let it just timeout:

A3604AF531F86DB7-0000000000000027._matter._tcp.default.service.arpa.
    deleted: true

So no mDNS announcement, and hence our Matter Server did not pick up the bulb. 😢

#

Updating to 3.6.152 now.

ionic bramble
novel snow
#

So for that particular node (Node 39) I did not see any (re)discovered on MDNS message... So mDNS wise, that device was really "down"

ionic bramble
#

Will these SRP entries be created for each matter controller separately? I'm asking because it would explain that bulbs are available in one matter controller but not in another.

novel snow
#

Yes, those are per Matter fabric

#

The device is pingable. I am quite certain it also worked before I restarted the server. So when a Matter subscription is still alive, things is all well and good. The problem starts when trying to reestablish connections with that device.

#

My bulb finally came back online! 🎉

#

Restarting the bulb did not help at first, but later the device suddenly reappeared. A bit weird...

ionic bramble
#

Is your bulb only connected to HA or do you also have another controller active?

novel snow
#

Only HA currently. And Nanoleaf app direclty

hardy rivet
#

I am a bit unsure, what I can do with 4 unavailable bulbs in HA. Power cycling doesn’t help. I think, I have to re-pair them.

hexed cargo
hardy rivet
#

Yes

hardy rivet
hexed cargo
hardy rivet
#

At least, I can wait with two of them. The other 2 are in a woman critical position. 😃

hardy rivet
digital star
#

Here's the situation for me

#

Beta 4 many unavailable

#

Stable, only a few unavailable. The few that are unavailable I can't bring back because they show mDNS (re) discovery but nothing happens

#

Pretty sure turning off the OTBR really destroyed my mesh anyways

hexed cargo
#

and beta 5 ?

digital star
#

Sorry that's what I meant

#

My OTBR is filled with Dropping rx frag frame so I assume I have thread issues now 😵‍💫

hexed cargo
#

yes it looks like it

digital star
#

So my feedback probably won't be useful until that settles down

digital star
#

Okay only took half my sanity to re-pair everything 👌

hardy rivet
digital star
#

I found pairing to be slower than on other firmwares

#

Could be because I nuked my thread mesh though

hardy rivet
#

But after I installed beta 7, I had the Idea to reboot all my Thread Border Routers. But it didn’t help, the 5 bulbs didn’t come back to HA. So I re-paired them. When Everything was fine, I installed the new HA Core. After that I lost some other bulbs. This is crazy! At the moment, I have most bulbs back. But when everything works, another bulb goes offline in HA. I do not think, that my mesh network is re-meshing. It’s always only one bulb. I hope it’s working tomorrow. I need a timeout… 😉

digital star
#

Probably faster when there's neighboring nodes and whatnot

#

I hope soon I can stop dealing with this and instead start hassling Marcel over robot vacuums instead

hardy rivet
#

That maybe true. But I find myself in the situation, where I ask my self, if I should start from 0, too…

#

Keep us updated. I am really interested, if it’s stable for you now. I resetted all bulbs 3 weeks ago.

digital star
#

I basically unplugged all my BRs, reset all thread devices, uninstalled the matter server, then plugged them all in again, reinstalled matter

#

Still get timeouts in the logs

#

But guess the logs will never be totally quiet

hardy rivet
#

Maybe…

digital star
#

If they get to a stable FW, I'm gonna update everything, even put a few spares in a closet, and never update or touch the NL app again lol

hardy rivet
#

Yep…

hardy rivet
#

@hexed cargo and @novel snow When I went to bed, I had four bulbs that were unavailable. Today in the morning all bulbs were available. Yehaaa!!!

So beta 7 seems to do a good job. Let’s wait some days and see what happens when I restart HA. But at the moment I do not plan to reboot/restart HA. 😃

#

@digital star How does it look for you?

bleak pond
#

nanoleaf have really stepped up their game with this 1.2 release i must say

#

now just to see if they can carry this momentum with the new sense+ controller a few of us are getting 🤞

novel snow
#

Which ones are the 1.2 firmwares?

hardy rivet
bleak pond
bleak pond
hexed cargo
#

For your info; I added 3 Nanoleaf lights to my network yesterday and since then I'm seeing a lot of timeouts etc. so I am not so sure still about these NL lights. At least they seem to respond a bit snappier but still I see them dropping off every once in a while. Going to follow this for a few days and otherwise they are going back where they belong again: in a box.

hardy rivet
# bleak pond is that in both HA and another ecosystem? or just a single one?, regardless shou...

You shouldn’t ask this question (insider @hexed cargo). 😉 The bulbs were mostly unavailable in HA. Apple Home mostly works, but I also had two bulbs that were also unavailable in Apple Home since the 3.6.152 update. I also had a bulb that was not available in AH, HA and in in the Nanoleaf app. I resetted and re-added all these bulbs to get them back everywhere.

I already communicated that in the Nanoleaf beta channel and was in direct contact to Nanoleaf Paul.

hexed cargo
#

I rest my case....

image.png
bleak pond
#

At least that’s what we think the main issue at play, poor implementation

hexed cargo
#

Don't get me wrong, they really want to fix this issue and they are actively working with us to get to the bottom of it all but personally I'm dissapointed that this is possible nowadays, companies selling half baken products.

bleak pond
#

Seeing as we are only dealing with the “product manager/s” I doubt they can at all control the sale of the product. But a public apology/acceptance would be nice

hardy rivet
bleak pond
#

I’ll copy it here

#

This sounds related to the GitHub issue reported by @digital star a few comments higher: Home Assistant is using mDNS incorrectly. A trivial dns-sd -B (or Avahi equivalent) triggers them to be rediscovered instantaneously. The reason Home Assistant loses track of these bulbs is those bulbs rebooting/disconnecting-and-reconnecting frequently though. So it's both a bug on Nanoleaf's end (unstable bulbs), and Home Assistant (should be more resilient for unstable devices or unstable networks). The fix has been merged in Home Assistant though, so I bet this problem will disappear soonish 😊

hexed cargo
#

Yes, that is what is available in 5.7.0

hardy rivet
#

But it's definitely too early to say anything about reliability...

hexed cargo
#

Well, I know the answer... and I only have 3 of those bulbs connected. Can't imagine how your log looks like with 30 of them (or how many devices do you have?)

digital star
# hexed cargo I rest my case....

Their app is really terrible. There's been times on past FW where just opening it will crash all the bulbs...

That said your observations about timeouts and whatnot you see in the HA logs are valid. Just wouldn't put too much stock in their app

#

I've got plenty of these

2024-02-29 06:06:17 (Dummy-2) CHIP_ERROR [chip.native.DIS] OperationalSessionSetup[1:0000000000000003]: operational discovery failed: src/lib/address_resolve/AddressResolve_DefaultImpl.cpp:119: CHIP Error 0x00000032: Timeout
2024-02-29 06:06:17 (Dummy-2) CHIP_ERROR [chip.native.DMG] Failed to establish CASE for re-subscription with error 'src/lib/address_resolve/AddressResolve_DefaultImpl.cpp:119: CHIP Error 0x00000032: Timeout'
hardy rivet
digital star
#

Is the part

OperationalSessionSetup[1:0000000000000003]:

003 the node id? Or just part of the error code?

hexed cargo
#

just the error code

uneven acorn
#

I have 13 NL devices, all on this latest firmware. Can’t say I really have many issues with them though, they always work without issues. My Google Nest’s have cause much more chaos than the NL devices ever have 😄

hexed cargo
#

Well, they still reset randomly as it seems but they recover quickly and we also made some changes on our end to pick up the rebooted nodes quickly enough

digital star
#

I have low hopes they're going to fix the reset problem before pushing a FW to the non-beta branch

digital star
#

Anyone else have unavailable nodes? Curious if anyone has a workaround since power cycling doesn't work for me

hexed cargo
digital star
#

Okay 👍

#

Btw I have devices down that are not NL bulbs

#

Even some of my aqara sensors and Yale lock

hexed cargo
#

OK, let me quickly bake a beta for you to test

hardy rivet
hexed cargo
hexed cargo
digital star
novel snow
hexed cargo
#

I'd wish they included the node id in all other logs as the logs are hard to read without

novel snow
#

I think in a lot of places they can't because they don't operate on the node

#

E.g. once you encrypted the stuff, you don't know what it is for..

#

But yeah, I agree, it would be nice if each and every log message could be traced back to the node causing it!

hexed cargo
#

@digital star are you available to quickly test a beta ?

digital star
#

Yeah but I won't able to really check the logs for about a half hour or so

ionic bramble
#

I‘m also in. After restarting the matter server one of my new paired bulbs got unavailable. But at start it could be discovered on mDNS.

hexed cargo
#

I'll ping you guys if the beta is available

hardy rivet
hexed cargo
hardy rivet
#

Ok, I do not have the time for testing today. But maybe tomorrow.

ionic bramble
#

Side question... how can I extend the log size from the matter server? It cuts to soon so that messages from start get discarded.

hexed cargo
#

@digital star @ionic bramble 5.7.1b0 is available, so if you have the beta toggle turned on in the matter server add-on its a matter of restarting the addon to get it installed Please let me know how that looks for you and we're especially interested in the logs as we added some additional logic

hexed cargo
ionic bramble
hardy rivet
ionic bramble
#

two unavailble node after start 27 & 55 - both nl

#

setup of sleepy nodes is way faster compared to 5.7.0

hexed cargo
#

but they should be picked up

ionic bramble
#

node 27 is now online

#

2024-02-29 21:08:11 (MainThread) INFO [matter_server.server.device_controller] Node 27 (re)discovered on MDNS
[Detaching after vfork from child process 435]
[New Thread 0x7fc3fff1e0 (LWP 437)]
[Thread 0x7fc3fff1e0 (LWP 437) exited]
2024-02-29 21:08:11 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 27...
2024-02-29 21:08:11 (MainThread) INFO [matter_server.server.device_controller.node_27] Setting up attributes and events subscription.
2024-02-29 21:08:15 (MainThread) INFO [matter_server.server.device_controller.node_27] Subscription succeeded

digital star
#

Seems I might be suffering from other issues, but this definitely seems to be an improvement

ionic bramble
#

Node 55 is still unavailable but I also had problems before 5.7.1b0 with this device. I'll reset and repair it again. In general 5.7.1b0 improved the startup a lot, especially for sleepy devices like my 9 Eve MotionBlinds and 6 Aqara P2.

digital star
#

Saw this not sure if it's relevant

2024-02-29 15:27:42 (MainThread) ERROR [matter_server.server] Error doing task: Exception in callback AsyncReadTransaction._handleSubscriptionEstablished(8972611)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.11/site-packages/chip/clusters/Attribute.py", line 764, in _handleSubscriptionEstablished
    if self._subscription_handler._onResubscriptionSucceededCb is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '_onResubscriptionSucceededCb'
#

b0 is already way better for me

hexed cargo
ionic bramble
ionic bramble
# hexed cargo yeah, I cant make those any more stable

As I already wrote earlier today, there seems to be something strange going on also with node 55 (NL bulb). It refused to go online but HA couId remove its own matter fabric from the bulb.
2024-02-29 21:49:24 (MainThread) INFO [matter_server.server.device_controller] Removing Node ID 55.
2024-02-29 21:49:24 (MainThread) INFO [matter_server.server.device_controller] Node ID 55 successfully removed from Matter server.
2024-02-29 21:49:24 (MainThread) INFO [matter_server.server.device_controller] Successfully removed Home Assistant fabric from device.

hexed cargo
#

So, with 5.7.1b0 that bulb should be picked up within 30 minutes (or at least retried) - you should be able to see that in the log

digital star
#

I can b0 doing it's best to stem the tide of NLs crap 💪

ionic bramble
hexed cargo
#

I just tried with a bulb that was powered off

ionic bramble
#

With the device earlier today I could confirm the removing of HA as a controller withApple Home

hexed cargo
hexed cargo
ionic bramble
#

Makes sense 👍

hexed cargo
#

For all commands we require an active subscription to the node, except for the remove node, that is issued on a best effort base

#

and succeeded in your case

#

but there was no reason to delete that node

#

you just had to wait long enough

ionic bramble
#

I'll try next time 😉

hexed cargo
#

remember 30 minutes is the magic number - if the node is not (re)detected after 30 minutes has passed, check the logs

digital star
hexed cargo
#

problem is the oh so unstable nanoleaf nodes that make everything noisy

digital star
#

Ah gotcha

hexed cargo
#

5.7.1 has a couple of improvements to help your case and at least make it more obvious in the logs whats happening

digital star
#

People not using HA say this latest FW is great. Not throwing shade at you guys but there working in some context (although IIRC you said apple can be pretty aggressive about things)

hexed cargo
#

My goal is to get complex setups working like yours and hoppel's

#

if that works; everything works 🙂

#

In the meanwhile my 3 nanoleaf bulbs have dropped off the grid completely btw

digital star
#

Do you have any of the mujoy thread bulbs?

hexed cargo
#

They sound very interesting from what I've heard here

digital star
#

Well what I was getting at was it would be great if you had a solid thread bulb to work around/with, then If that worked, you could easily lay all the blame on NL (and spend less time on it)

polar wolf
#

Are they bulbs?

hexed cargo
digital star
#

Well yes but from a very defensible technical position as you can say the server works for sure when bulbs behave well, see brand XX. Cause NL will have plenty of excuses

ionic bramble
#

For me the latest fw also works fine in Apple Home. But I'm using HA heavily for automations. Apple Home is just basic on/off via light switch.

uneven acorn
hexed cargo
#

@digital star so is 5.7.1b0 working well for you ? In the meanwhile we released 5.7.1 stable (which will also be installed if you restart the server with the beta toggle on). The add-on will be updated soon

digital star
hardy rivet
#

Hi, I am still on 5.7.0b7 and had a look at my logbook. Over the day, I had the following unavailabilities. It were always different devices/bulbs:

  • NL A19 E27 - 9:16AM-9:22PM - 6min
  • EVE Energy - 3:17PM-3:18PM - 1min
  • NL GU10 - 3:17PM-4:06PM - 49min
  • NL A19 E27 - 3:20PM-3:54PM - 34min
  • NL A19 E27 - 5:08PM-5:58PM - 50min
  • EVE Energy - 10:41PM-10:43PM - 2min
  • EVE Energy - 10:41PM-10:45PM - 4min
  • NL A19 E27 - 10:43PM-10:45PM - 2min
  • EVE Energy - 12:35PM-12:36PM - 1min
#

Before I installed the Nanoleafs my Thread Mesh was completely stable. Very seldom I saw some EVE becoming unavailable for some seconds.

#

Why takes it so much longer for Nanoleafs to get available in comparison to EVE devices? Where is the difference?

#

The good news is that all devices/bulbs are available after 24hours, even though my thread mesh has some issues.

hexed cargo
hardy rivet
#

Ok, I am going to install the latest Matter server when it’s available officially, maybe tomorrow. I want to enjoy this situation that all bulbs are available and come back automatically when they were unavailable… 😃

hexed cargo
hardy rivet
#

What can I expect from the next version?

hexed cargo
hardy rivet
#

Should it already be available? I still see 5.3.0/5.7.0 on my HA Yellow.

hexed cargo
#

enable the beta toggle

#

new add-on will probbaly be tomorrow

hardy rivet
#

By the way… I ditched my HAOS VM and now have the HA Yellow working since some weeks. Migration was easy and feels great. I do not see any performance difference to my vm. Well done @novel snow

hexed cargo
#

Yeah its good to have a dedicated box for HA - did you get a 8gb cm4 ?

hardy rivet
hardy rivet
bleak pond
#

I mean, if you got the beta toggle on, it just goes to the beta version, updating the add-on just changes the fallback version (aka GA version)

hardy rivet
hexed cargo
hardy rivet
#

Yeah, at the moment I think it was a good decision. I also like that it’s powered by PoE. But it’s so easy to migrate from machine to another. Backups go to my NAS automatically. So, nothing can go wrong… 😃

digital star
#

You guys having the bulbs constantly resubscription in the logs?

#

I know Marcels are already back in the box 😅

hardy rivet
digital star
#

Guys I can't do this anymore. Can any of you recommend good experiences with matter bulbs that aren't NL? Wifi is fine.

Curious if anyone has experience with the tapo but not super optimistic with those since my mini plug from them has issues

#

I want to test something different this weekend

#

(naturally only NL devices are unavailable)

polar wolf
digital star
#

Nah I'm gonna phase them out once I find a good alternative

#

It's been 6 months

polar wolf
digital star
#

Yeah I saw that but in the meantime there's some matter over WiFi bulbs out there. Fortunately I don't have a ton of wifi interference to deal with

#

I'd be happy if aqara just released their motion P2 🥲

bleak pond
#

wiz bulbs if you are in the UK?

hardy rivet
#

Hi @hexed cargo, bad news. I waked up in the morning with 7 devices and one bulb unavailable. I started to power cycle some of them and they came back, but others became unavailable. I did this with some devices until my complete Thread network began to re-mesh.

A lot of logs with this Matter server version. I can give you the logs for nearly 3 hours: https://dpaste.org/TMADr

All paste services crash, when I upload too many log lines... 😂

I disabled the HA Matter integration, removed all my Apple TBRs from current and re-added them to build a fresh mesh. Now all devices are back to Apple Home and I am going to enable the Matter integration now. You will get a new log in an hour or so.

hardy rivet
#

OK, I checked the status 5 minutes after I enabled the Matter integration and all 46 devices/bulbs are already available. What? I never had it that fast. 😄

I do not find the logs for the start of the Matter server 5 to 10 minutes ago. Hm..., maybe the Matter server didn't stop.

Yes, that's the 'issue'. Here is the log from the point where I disabled the Matter integration and the Matter server didn't stop. This typically worked.

https://dpaste.org/wyOnq

#

OK, one bulb is already unavailable since 13 minutes. I will post you another log in an hour so. Hopefully it will come back on its own.

bleak pond
#

no

#

its known about

bleak pond
#

like always, nothing to do with HA, everything to do with NL

#

well, this is before that pull w/ getting it via mdns

hardy rivet
bleak pond
#

oh wait, mb they seems to have only been using local link addresses.

hexed cargo
hexed cargo
hexed cargo
hardy rivet
#

I have the latest HA beta:

Mar 01 07:27:28 homeassistant addon_core_matter_server[547]: -----------------------------------------------------------
Mar 01 07:27:28 homeassistant addon_core_matter_server[547]:  Add-on version: 5.3.0
Mar 01 07:27:28 homeassistant addon_core_matter_server[547]:  You are running the latest version of this add-on.
Mar 01 07:27:29 homeassistant addon_core_matter_server[547]:  System: Home Assistant OS 12.0  (aarch64 / yellow)
Mar 01 07:27:29 homeassistant addon_core_matter_server[547]:  Home Assistant Core: 2024.2.5
Mar 01 07:27:29 homeassistant addon_core_matter_server[547]:  Home Assistant Supervisor: 2024.02.1
Mar 01 07:27:29 homeassistant addon_core_matter_server[547]: -----------------------------------------------------------
Mar 01 07:27:34 homeassistant addon_core_matter_server[547]:   Downloading python_matter_server-5.7.1-py3-none-any.whl.metadata (14 kB)
hexed cargo
hardy rivet
#

OK...

hardy rivet
hexed cargo
#

Do note that we are actually seeing the Nanoleaf bulbs behave really badly so I'd advice anyone to not update to that 3.6 version

#

I fact we're just doing all kind of dances now to work around NL issues

normal plank
hexed cargo
#

really lots, like crazy

normal plank
#

At what log levels do you see them?

hexed cargo
#

ERROR

normal plank
#

thats all my matter server shows for the last 12 hours

hexed cargo
#

This is for instance a Nanoleaf bulb:

Schermafbeelding_2024-03-01_om_10.png
hexed cargo
normal plank
#

and i have 13 nanoleaf bulbs

#

Im on matter server version 5.3.0

hardy rivet
#

Oh man, what shit show...

#

😄

normal plank
#

all nanoleaf bulbs are on the latest beta

hexed cargo
#

ah you mean add-on version 5.3.0 ?

#

those version numbers are confusing at times

normal plank
hexed cargo
# normal plank yes

OK interesting, that is actually the new matter server so what is the difference

normal plank
#

Maybe the addon version should match the matter server version 😄

#

to make it easier

hardy rivet
#

My mesh was in a remesh... So I disable the Matter integration and Matter server again, waited until all devices were back in Apple Home. Re-enabled the Matter integration, restarted HA core, because I got the message to do that. No device comes back to HA. Here is the log:

https://dpaste.org/UxkzB

hexed cargo
#

@normal plank can you tell me about your set-up ?
What border routers etc.

normal plank
hardy rivet
# 
Mar 01 10:05:58 homeassistant addon_core_matter_server[547]: -----------------------------------------------------------
Mar 01 10:05:58 homeassistant addon_core_matter_server[547]:  Add-on version: 5.3.0
Mar 01 10:05:59 homeassistant addon_core_matter_server[547]:  You are running the latest version of this add-on.
Mar 01 10:05:59 homeassistant addon_core_matter_server[547]:  System: Home Assistant OS 12.0  (aarch64 / yellow)
Mar 01 10:05:59 homeassistant addon_core_matter_server[547]:  Home Assistant Core: 2024.2.5
Mar 01 10:05:59 homeassistant addon_core_matter_server[547]:  Home Assistant Supervisor: 2024.02.1
Mar 01 10:05:59 homeassistant addon_core_matter_server[547]: -----------------------------------------------------------
Mar 01 10:06:02 homeassistant addon_core_matter_server[547]:   Downloading python_matter_server-5.7.2b0-py3-none-any.whl.metadata (14 kB)
normal plank
hexed cargo
hexed cargo
normal plank
#

10 eve energy, 12 nanoleaf gu 10, 1 e27 and a few motion blinds and doorsensors.

hexed cargo
#

so I wonder what is the difference

hardy rivet
normal plank
hardy rivet
#

I am going to reboot my HA Yellow now.

hexed cargo
normal plank
#

I have had mostly stable nanoleaf bulbs in my setup for probably the last 2 months after moving my border routers a bit.

#

I have one lamp that gives me problems though because it not letting any signals through. Even had problems with ikea zigbee gu10 bulbs in it.

hardy rivet
#

Hm. I had a stable setup with 5.6.0b7, if I remember right. Yesterday night I updated to the latest available Matter addon and server beta.

hexed cargo
normal plank
hardy rivet
#

Yes, let’s do more tests. But when is the point, where stop testing? Maybe we should wait for the next Nanoleaf beta firmware, before doing more tests. But I am willing to test more HA Matter optimizations, until you completely run out of ideas. I am really impressed by the work and changes you and Stefan do at the moment. 😉

I have Philips Hue lying around. No problem to go that route until next Nanoleaf firmware arrives…

normal plank
hardy rivet
#

I call it 'stable', because all devices came back, when they got unavailable in HA.

hexed cargo
#

OK final round of testing

normal plank
hexed cargo
#

@hardy rivet 5.7.2b1 should be available if you restart - that should give the best balance between speed and stability and allows up to 10 nanoleaf bulbs to be slow in setup stage.

hardy rivet
normal plank
normal plank
hexed cargo
hardy rivet
#

No, no devices came back after reboot of 5.7.2b0. Now I am installing 5.7.2b1.

normal plank
hardy rivet
#

Mar 01 10:41:25 homeassistant addon_core_matter_server[550]: Downloading python_matter_server-5.7.2b1-py3-none-any.whl.metadata (14 kB)

#

Yes, the devices are coming up.

#

9 minutes later, only 7 Nanoleaf bulbs are not available. So, lets give it a bit more time.

hardy rivet
hexed cargo
#

So, the code can now handle this situation much better but to prevent we're doing a lot read requests at the same time we do a bit of throttling to not overload the network. So in theory if you have 10 misbehaving NL bulbs, you need to wait 10 minutes to free a slot to setup another one. But in the end they will all be setup

#

5.7.2b1 looks promising on my end

hardy rivet
#

OK, thanks for the explanation. I still have 7 bulbs that got unavailable 14 mins ago.

hexed cargo
hardy rivet
#

Alle EVE devices are there... 😉

hexed cargo
#

Yes, this is the best we can do for nanoleaf. But still I wonder why some of them are unreliable and some of them not

#

What is the difference

hardy rivet
hardy rivet
hexed cargo
hardy rivet
#

OK, if this will not work, I am going to reduce my 16 bulbs to 7 bulbs in uncritical positions today in the evening.

#

But I could do some further tests, if you need me to do that over the day.

hexed cargo
hexed cargo
hardy rivet
hexed cargo
#

It could also help to just add a few more reliable routers, such as eve plugs 🙂

hardy rivet
novel snow
digital star
hardy rivet
#

34 minutes later, only 2 bulbs unavailable, but one EVE device is also unavailable since 10 mins now.

hexed cargo
hardy rivet
#

Yeah, nearly all my EVE devices are in close proximity to a Nonleaf bulb. I think I have to reposition that EVE Energy. I recorgnized that this EVE device got a bit wonky since the last Nanoleaf beta firmware update. I think I can blame Nanoleaf. 😉

#

41 minutes later, only 2 Nanoleaf bulbs are unavailable.

normal plank
#

So maybe splitting it up makes sense

hardy rivet
#

For both node ids I have the following log:

Mar 01 10:44:21 homeassistant addon_core_matter_server[550]: 2024-03-01 11:44:21 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 135...
Mar 01 10:42:17 homeassistant addon_core_matter_server[550]: 2024-03-01 11:42:17 (MainThread) INFO [matter_server.server.device_controller] Node 143 (re)discovered on MDNS
Mar 01 10:44:39 homeassistant addon_core_matter_server[550]: 2024-03-01 11:44:39 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 143...
digital star
#

FWIW this beta brought back a lot of my NL bulbs as well

#

And I went to bed with 0 available

hexed cargo
#

BTW: It would be nice if you checked nodes 135 and 143 periodically. I'm really curious if they suddenly pop-up after a while or not.
At this time they seem to be stuck in setting up the subscription. It should time out after some while, if not, we have a deadlock 😬
I could put back a timeout just to handle that but the SDK should time out on its own.

hardy rivet
hexed cargo
#

This is the last mention of node 135 in the log you shared above:

Mar 01 10:44:21 homeassistant addon_core_matter_server[550]: 2024-03-01 11:44:21 (MainThread) INFO [matter_server.server.device_controller.node_135] Setting up attributes and events subscription.

We are wondering why is there no follow-up. There should be either a timeout, exception or confirmation that the subscription succeeded.

hardy rivet
digital star
#

I'm grateful for the HA devs hard work but I'm still phasing mine out once I test some new devices this weekend

bleak pond
#

oath, they have had over a month of us helping them and it still is no where near as reliable enough

hardy rivet
hexed cargo
hardy rivet
#

300%? 😄

bleak pond
hexed cargo
hardy rivet
hexed cargo
#

I've been testing a bunch of WiFi Matter stuff and some of it is very cheap and actually very good. Like smartt plugs for more or less € 10 that work really well. The only downside is that it's WiFi tbh.

digital star
#

I'll let you know my experience

#

I have about 20 bulbs which is honestly not many for my unifi AP. I'm assuming the traffic they will generate is low considering thread is like 250 kbit

hexed cargo
bleak pond
#

i mean, i would take the extra power and a little more conjestion over non-working bulbs 😛

digital star
#

I think both the AP and client have to support wifi 6 to reap the benefit correct? I'd imagine most IoT cheap out with just wifi 4

hexed cargo
#

Probbaly best to have a seperate 2,4 Ghz SSID just for the IoT devices and connect your fast devices to a dedicated 5Ghz one

hardy rivet
digital star
#

I also ran a bunch of Ethernet in my crawl space / attic for my media platforms which was fun /s

hardy rivet
#

I did not give up completely on Nanoleaf. 😉

digital star
#

More power to you

#

All my other thread devices are great so I'm keeping them around. Still trying to find a use case for those eve energy as they look neat

hardy rivet
normal plank
hexed cargo
#

In the meanwhile I'm cleaning this list of 100s of devices from all my test attempts.
Why didnt they add a clear all button there ?!

image.png
digital star
hexed cargo
hardy rivet
hexed cargo
normal plank
#

or use the ones you have to set when commissioning a device

hardy rivet
#

Ok, when do you think to release that fixed version? Maybe I am willing to do another test, before I exchange the bulbs to Philips Hue.

hexed cargo
hardy rivet
#

Is that log still relevant to you?

hexed cargo
#

I really want to trace the issue where a subscription setup is started and then seems to never return.

hardy rivet
#

OK, I will not touch the Matter server restart button anymore.

uneven acorn
#

This already known with the latest matter beta?


2024-03-01 13:43:08 (MainThread) ERROR [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/matter_server/server/server.py", line 74, in _handle_ws
    return await connection.handle_client()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/matter_server/server/client_handler.py", line 76, in handle_client
    await wsock.prepare(request)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_ws.py", line 152, in prepare
    payload_writer = await super().prepare(request)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_response.py", line 415, in prepare
    return await self._start(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_response.py", line 423, in _start
    await self._write_headers()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_response.py", line 503, in _write_headers
    await writer.write_headers(status_line, self._headers)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 130, in write_headers
    self._write(buf)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/http_writer.py", line 75, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
[Detaching after vfork from child process 297]
hexed cargo
#

Its a bit verbose but pretty harmless

hexed cargo
uneven acorn
#

Now for the first time, I had 2x lightstrips and 1x e27 that were unresponsive in HomeAssistant and Nanoleaf (although they were shown under the thread network page), but working in AppleHome. Turning the light on and off via Apple Home caused them to become available again immediately for the lightstrips. The e27 has not yet come back to HomeAssistant. But, I just started the add-on like 10 minutes ago, so I'll give it some time.... (it was available before I restarted the add-on anyway)

#

If I look in flame, I see 1 bulb that is not bradcasting the _matter services like the rest are, not sure if this is the problem bulb though, not sure where to match up all the numbers

hexed cargo
#

so for example node 116 is 74

uneven acorn
#

I can't find anything with that node id in _matter indeed. I'm guessing that it still works in Apple because it hasn't had a restart yet (the Apple devices)

hexed cargo
uneven acorn
#

well, I can't see the node id in Apple of course, but I can't find anything ending in that ID in general

hexed cargo
#

problem is that apple will have another node id for this node and apple has another fabric id

#

so basiclaly searching for a needle in an haystack 🙂

#

2 possible things could be going on:

  1. apple still has a connection with the device using its last known ip
  2. the device is no longer broadcasting our mdns name (and maybe only for apple)
uneven acorn
#

so, under _itdpu._udp I still see it (or at least, I have 4x A19 bulbs and I see all 4x) I just don't know which one is which as they all have 7 characters codes, do you know how they match up to the serial number or something?

#

If I can find the code, I can see if it is indeed the one that is not bradcasting the _matter services

#

ah wait, I found it, on the sticker it has an ID

#

k, let me see...

#

yes, that's him

hexed cargo
uneven acorn
#

the little bugger

hexed cargo
#

At least you could count

uneven acorn
#

right, but in Flame I can see the device, and then under it it lists all the services, including that for _matter

#

and for this device, it has none

hexed cargo
#

@hardy rivet did something show up in your logs already ?

hexed cargo
uneven acorn
#

Of course, that's why I'm here 😄

hexed cargo
#

haha right...

uneven acorn
#

I will go power cycle the device, and see if they come back

#

the services that is

#

yes! they came back after the power cycle

#

and HomeAssistant found it again now

#

I see that under addresses they all have a .local name as well as it's ipv6 address, and this was still visible. Is it of any use storing this .local anywhere in order to find its address? Not sure if it matters though if the _matter services are not broadcast.

hexed cargo
hardy rivet
#

I do not see any log for this node id.

#

This is the last log, I see for node id 143:

Mar 01 12:30:58 homeassistant addon_core_matter_server[550]: 2024-03-01 13:30:58 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 143...

#

The other bulb that was unavailable before the restart, node id 135 shows the following logs:

Mar 01 13:31:37 homeassistant addon_core_matter_server[550]: 2024-03-01 14:31:37 (MainThread) INFO [matter_server.server.device_controller] Node 135 (re)discovered on MDNS
Mar 01 13:41:34 homeassistant addon_core_matter_server[550]: 2024-03-01 14:41:34 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 135...
hexed cargo
#

I can reproduce this now - suprise - only with Nanoleaf bulbs.
I have forwarded my test results to them just now

#

I think this is a new issue introduced in firmware 3.6 as I have not seen this one before

hardy rivet
#

node id 143, 177, 181 were unavailable after the restart of the Matter server. 177 and 182 are available now.

#

For node id 177 I see the following in my logs:

Mar 01 13:31:37 homeassistant addon_core_matter_server[550]: 2024-03-01 14:31:37 (MainThread) INFO [matter_server.server.device_controller] Node 177 (re)discovered on MDNS
Mar 01 13:32:30 homeassistant addon_core_matter_server[550]: 2024-03-01 14:32:30 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 177...
Mar 01 13:44:12 homeassistant addon_core_matter_server[550]: 2024-03-01 14:44:12 (MainThread) INFO [matter_server.server.device_controller] Marked node 177 as unavailable
Mar 01 14:05:26 homeassistant addon_core_matter_server[550]: 2024-03-01 15:05:26 (MainThread) INFO [matter_server.server.device_controller] Marked node 177 as unavailable
#

For node id 182 I see the following:

Mar 01 12:27:24 homeassistant addon_core_matter_server[550]: 2024-03-01 13:27:24 (MainThread) INFO [matter_server.server.device_controller] Node 181 (re)discovered on MDNS
Mar 01 12:30:29 homeassistant addon_core_matter_server[550]: 2024-03-01 13:30:29 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 181...
#

Why is Marked node 177 as unavailable? It works as expected.

hexed cargo
hardy rivet
#

Yeah, but I can control it through HA.

#

The light switches on/off.

#

😄

hexed cargo
hardy rivet
#

Yeah, I grepped through the logs. But this is the last line for node id 182. So the restore did not create a log message?

#

Or did the restore happen without the node id in the log line?

#

Can that happen?

hexed cargo
#

unavailable is really unavailable in HA, there is no way that they can be out of sync, really no way.

#

We have released 5.8.0 just now which will be the new stable version.
We have identified some issues with Nanoleaf and forwarded to them. That concludes it for now.
People with issues should remove NL lights from their setups or bring down the amount of them.
Also multi-admin adds an additional layer of stress to these already unstable bulbs so try to avoid that for now.

Thanks you all for the patience testing!

hardy rivet
#

The is from 15:05:26 of the time here in Europe. Now we have it 15:33.

Mar 01 14:05:26 homeassistant addon_core_matter_server[550]: 2024-03-01 15:05:26 (MainThread) INFO [matter_server.server.device_controller] Marked node 177 as unavailable

hardy rivet
hexed cargo
#

its already restored

#

As you can see above, I'm done with Nnaoleaf for now. I stop my support

#

If somebody has issues with non Nanoleaf products I will look into it but the ball is on their end

hardy rivet
#

No, this is the last log line of my Matter server:

Mar 01 14:35:09 homeassistant addon_core_matter_server[550]: 2024-03-01 15:35:09 (MainThread) INFO [matter_server.server.device_controller] Attempting to resolve node 13... (attempt 2 of 2)

So, the log maybe lags some seconds only. Or do you mean the log for this bulb lags behind?

uneven acorn
hexed cargo
#

Yeah maybe, I'm done for this week, what a mess

#

🙂

hardy rivet
# hexed cargo If somebody has issues with non Nanoleaf products I will look into it but the ba...

Yeah, I absolutely understand your point of view. I am in the same situation. We tested shitty firmware the last two days. The good thing is that HA Matter now is very optimized. Thanks again for all the hard work you and @novel snow investigated into this. At the moment I am fine. Only one bulb of my 46 devices/bulbs is unavailable. I think, I will leave it that way. When the situation gets worse, I am going to remove some bulbs and wait for the next Nanoleaf beta firmware.

uneven acorn
#

How would one know which device is being referred to here:

#

2024-03-01 15:41:41 (Dummy-2) CHIP_ERROR [chip.native.DMG] Time out! failed to receive report data from Exchange: 511i

hexed cargo
uneven acorn
#

oh okay

#

😄

hexed cargo
#

If somebody is feeling adventurous, contribute some C++ code to the SDK to print node id's in the logs 🙂

hexed cargo
# hardy rivet Yeah, I absolutely understand your point of view. I am in the same situation. We...

Yes, I think we made many good optimizations in the last couple of days from which many people (with and without NL) will benefit and in the meanwhile we learned something about the NL firmware 3.6. Time to test 5.8 for a while in the wild on various setups.
I think it's safe to say that the NL firmware is still not there where we want it to be and if you have many of them, it destabilizes your network. A good example of that is your setup with many NL bulbs and a destabilized Thread network.

digital star
uneven acorn
hexed cargo
#

First response I got back from NL is that a power cycle should snap them out of this state. The test bulb I have here that was misbehaving indeed came back (after a while) once I did power cycle it

uneven acorn
#

luckily, the two that have done this to me (or at least, the ones I've noticed) are easy to get to in order to power cycle; that's not the case for all of them though 😛

normal plank
uneven acorn
#

Can anyone tell me which Matter command HomeAssistant sends when it does a light.turn_off ?

hexed cargo
#

Plain off command

hardy rivet
#

I also resetted and readded some of my bulbs to get them back to HA. But they became unavailable again and mostly do not come back. Sometimes they frequently come and go. Sometimes they are unavailable for hours, sometimes they do not come back. I also had the issue again, that some of my bulbs are not available in Apple Home. All my 48 Matter over Thread devices are paired to Home Assistant and Apple Home. The firmware 3.6.152 made my Thread mesh unstable. All devices started to remesh, after a day or so. I decided to remove 6 of my 16 bulbs from current. My mesh seems to be more stable now. At least all devices incl. EVE devices stopped getting unavailable since I removed the 6 bulbs 4 hours ago. If my Thread mesh gets wonky again, I am going to remove more bulbs, maybe all. I will do more tests with the next firmware. For the moment it makes no sense to me to use them in my house. It's more bad than good.

ionic crypt
#

@hardy rivet do you think the problem is with too many Thread devices, or too many Nanoleaf thread devices? I'm curious if you could support a larger number of devices overall if more of them were from another manufacturer. I'm not a thread expert, but this sounds like it could be a case where the NL products are keeping track of the network map in a table that is too small for the number of devices. Maybe it can tolerate a few NL products with the too-small storage, but he more devices that lose the routing map the more likely it is to screw up the entire network.

digital star
#

The historical problem has been that nanoleaf bulbs act as full thread devices and potentially route traffic. If they drop connection or crash then thread has to redo the routes. If you have a lot then it can be troublesome

hardy rivet
normal plank
#

This should hopefully do the job.

bleak pond
#

Should be good

uneven acorn
#

Getting a lot of these in my logs this morning:

2024-03-04 07:23:01 (MainThread) INFO [root] Re-subscription succeeded!
2024-03-04 07:23:01 (MainThread) ERROR [matter_server.server] Error doing task: Exception in callback AsyncReadTransaction._handleSubscriptionEstablished(3249868838)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.11/site-packages/chip/clusters/Attribute.py", line 764, in _handleSubscriptionEstablished
    if self._subscription_handler._onResubscriptionSucceededCb is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '_onResubscriptionSucceededCb'
hexed cargo
uneven acorn
digital star
#

So this weekend I removed all my NL bulbs because I'm tired of them. I tested both TAPO and WiZ (Philips Connected) bulbs. For the WiZ I tested both the matter certified ones and an older lamp that was updated for matter support. All these devices are wifi which isn't really an issue for me as I live in a low-moderate wifi interference environment.

TAPO - these are brighter than NL and WiZ (1100 lm at all colors vs WiZ at 800 for their standard bulb and NL is 1100 for whites and 800 for colors)

They were okay. They had some subscription timeouts about every 3 hours or so. I kinda expected this since my tapo mini plug seems to have some issues too.

The Tapo bulbs had no transitions which I didn't like (instant on/off and color/brightness changes). Otherwise they worked fine.

WiZ matter update - I had a lamp that received matter as an update but it had a lot of subscription timeouts (like every 5 minutes). I dunno that I'd recommend their older devices that have been updated for matter, but good that WiZ tried to add support.

WiZ matter certified - These worked great for me. No issues with subscription timeouts. Unbelievably they had light transitions which I didn't think was possible with matter. I guess they modified the SDK to do this. Smooth on/off, color and brightness transitions. My ONLY issue was they didn't respect the "ExecuteIfOff" option meaning they wouldn't turn on with a specified brightness and color (ive reached out to find out if they're going to fix this).

WiZ also has a proprietary API similar to NL, but it's available local only. You can run it with or without matter, too (e.g. you can control using the WiZ integration and matter at the same time).

In my case, I preferred the WiZ local integration as it offered more features (brightness and color from off worked fine) and it had scenes available. Ultimately I ended up buying some open box items via eBay to replace all my NL bulbs with WiZ.

#

@hexed cargo

This happened with all my devices over the weekend every so often (both thread and wifi).

Node 12 (re)discovered on MDNS
2024-03-03 14:06:51 (MainThread) INFO [matter_server.server.device_controller.node_12] The SDK is communicating with the device using fe80::4a22:54ff:fe7d:90c0
2024-03-03 14:06:51 (MainThread) INFO [matter_server.server.device_controller] Setting-up node 12...
2024-03-03 14:06:51 (MainThread) INFO [matter_server.server.device_controller.node_12] Unsubscribing from existing subscription.
2024-03-03 14:06:51 (MainThread) INFO [matter_server.server.device_controller.node_12] Setting up attributes and events subscription.
2024-03-03 14:06:51 (MainThread) INFO [matter_server.server.device_controller.node_12] Subscription succeeded
hexed cargo
#

Yes, its not entirely unavoidable as its all ip based so devices change IP's, have a short communication issue etc.
Point is that we are able to survive all these cases, hence the somewhat extended logging.
We already found a few improvements to the mdns discovery which we'll implement but other that that its perfectly fine to have these messages. It means that everything is working as expected!

digital star
#

Notice that the resubscription is instant and not 40 minutes like NL lol

novel snow
#

@digital star was Node 12 a Wiz node? Did the IP change for the same WiZ node?

raw wedge
#

I have updated a nanoleaf A19 to the latest beta today, and it went unavailable in home assistant. I was still able to control it via the leaf app (app said connection type thread, not bluetooth). Restarting home assistant, matter server and otbr did not remedy the situation. I went ahead and deleted the bulb vom home assistant, reset it and tried rebinding it. It fails every time. Matter Server logs an error:

2024-03-04 15:16:34 (Dummy-2) CHIP_ERROR [chip.native.DMG] Time out! failed to receive report data from Exchange: 30872i```
#

The version the nanoleaf app reported on the bulb after the update is 3.6.152

digital star
digital star
novel snow
#

It seems to me that since 3.6.152 it is essentially impossible to add a Nanoleaf device to HA. My hypothesis is that the new firmware in combination with how we subscribe to the attributes leads to 💥

digital star
#

Lots of people are having issues even with apple and Google home so I'm not sure it's strictly on HA

normal plank
uneven acorn
hardy rivet
novel snow
#

Ok, maybe I did not try hard enough 😅

hexed cargo
#

@hardy rivet please try without any OTBR involved ( so no skyconnect, just only use the homepods) if that improves the situation ?

hexed cargo
#

Anyone who's experiencing issues with Nanoleaf bulbs not being available and a timeout being logged in the logs:
Please test without the Home Assistant OTBR involved. So just Apple BR's or Google BR's (not a mix of them).

digital star
hexed cargo
#

I'm afraid so

#

I can reproduce the locking Nanoleaf's with 100% if I just power on a OTBR

#

Without OTBR all is fine

#

except from some other issues maybe, but that is another story

#

also I'm tracking the rebootcount of nanoleaf bulbs and they do not seem to be crashing

#

at least, not anymore with 3,6

digital star
hexed cargo
#

Well, as long as they stay stable/operational

digital star
#

I had one that something like 700+ reboots sinxe the last reset (like a week) which seems unlikely

hexed cargo
digital star
#

Well hopefully NL can get that fixed, no reason their stuff shouldn't work with all properly implemented BRs

hardy rivet
hexed cargo
hardy rivet
hexed cargo
#

ok thanks, no worries, I'm probably not be able to look at it until tomorrow but I want to verify if I see in your log the same issue as we isolated today with a OTBR based setup

hardy rivet
uneven acorn
#

Ooh, they are now showing the IP addresses

uneven acorn
#

I noticed with this latest beta that for some reason now, the status of the light in HomeAssistant does not (always) match the actual status of the light. Although the status is shown correctly in Apple Home and of course the Nanoleaf app itself.

hexed cargo
#

with latest beta you mean the latest NL firmware version ?

#

Are you also running the latest matter server beta (5.6.1b3) ?

uneven acorn
uneven acorn
hexed cargo
uneven acorn
#

I'll put it on now.

#

The only thing I see for that time period in the MAtter server logs is the following:

# 
2024-03-06 08:43:06 (Dummy-2) CHIP_ERROR [chip.native.DL] Long dispatch time: 120 ms, for event type 2
2024-03-06 08:56:06 (Dummy-2) CHIP_ERROR [chip.native.DL] Long dispatch time: 122 ms, for event type 2
#

Is the resource being referred to the actual light or plug etc?

# 
2024-03-06 09:13:41 (Dummy-2) CHIP_ERROR [chip.native.DMG] Failed to establish CASE for re-subscription with error 'src/protocols/secure_channel/CASESession.cpp:1981: CHIP Error 0x000000DB: The Resource is busy and cannot process the request'
#

I think the issue is... still on the NL side

#

these devices that were not showing the updates, are also struggling to reappear after the matter server restart on the latest beta of the matter server

#

I checked these in mdns, and.... all the _matter services are gone for these lights

#

Was supposed to be fixed in this version, but apparently not

hexed cargo
#

Yeah, the devices I upgraded here to the latest fw version also went (more) unstable

bleak pond
#

yeah, its been reported in their chat, we found the root of it and it has to do with the changes to the off/on ramps we think

digital star
#

This is supposed to be a RC firmware too 😬

hexed cargo
jovial raft
#

The beta sounds really fun.

raw wedge
digital star
#

@hexed cargo since you've been working with them, can you tell us if these are caused by matter, thread, or both? (On their end)

#

I'm curious if they're only really struggling with the matter SDK or if thread is also throwing a wrench in

hexed cargo
#

It's a combination of all kind of factors but most important snowball effects from stability issues in the thread network.

normal plank
#

And this is just really hard to test. Im not saying this is the exact problem, but fixing one thing might cause errors somewhere else completely unrelated.

digital star
#

My tinfoil hat still remains that theres a hardware limitation that's really holding things back

polar wolf
normal plank
digital star
#

I think that NL underestimated (or perhaps silabs overestimated) the requirements for a full thread device with matter and a proprietary protocol

normal plank
digital star
normal plank
digital star
#

Hopefully you figured that out before you went into production. If not then my condolences

normal plank
#

At least i had fun editing linker scripts 😅

empty moss
#

Timeouts in single core systems are nothing new, heck most of the performance work in 2023.3 and .4 is addressing problems caused by the single core python event loop getting unexpectedly blocked and things timing out.

#

Thankfully on our beefy HA systems we can defer the slow stuff to the executor (ie run it in a thread) - but only when we’ve found it

#

But even that means there is follow up work to stop the thread pool getting blocked by hundreds of tasks

jovial raft
#

Is anyone using Adaptive Lighting custom component? It previously worked fine for me with my Nanoleaf bulbs, but has stopped working a couple of weeks ago. I can't figure out what the issue is, since the bulbs themselves work in Home Assistant and with other automations. I know Adaptive Lighting basically just sends turn_on commands to lights but I'm not sure how to debug it.

hexed cargo
#

NL has stability issues enough on its own, do not try to enforce it with hemmering these lights with commands

#

The default settings of Adaptive lights are very, very bad

#

If you insist of using it, set it up to only send a command every 15 minutes or so and use long transition times

#

That is also my advice for zigbee btw

#

People seem to forget that z-wave, zigbee and thread are very low bandwidth networks, you must be careful with sending commands.

#

Thread is a bit better with multiple border routers ofc but still, be careful

jovial raft
#

Thanks for the advice. Another setting I turned on: "skip_redundant_commands: Skip sending adaptation commands whose target state already equals the light's known state. Minimizes network traffic and improves the adaptation responsivity in some situations. 📉Disable if physical light states get out of sync with HA's recorded state."

This one works fine for Nanoleaf, outside the latest NL beta (which I'm not using) which apparently broke on-off state sync.

hexed cargo
#

That said, I saw some comments about issues with transitions in the Nanoleaf RC release

hexed cargo
jovial raft
#

The transition service call with "light.turn_on" never worked for me, I'm not sure if it's even supported by Matter. Or was it added?

hexed cargo
jovial raft
#

Great to hear.

digital star
#

That config did work for me so if it doesn't anymore, blame the FW

#

I used 3s transition

hardy rivet
#

Now where the switch on by themselves issue is back and I also can’t (re-)pair any bulb to HA anymore, I am back to Philips Hue. No Nanoleaf for the moment.

digital star
hexed cargo
jovial raft
#

My issue with Adaptive Lighting turned out to be far dumber, the "Adaptive Lighting Adapt Color" switch was accidentally turned off. Now it works fine. Regardless thank you for the tips.

#

I assume it would work even better now that transitions are supported.

hexed cargo
jovial raft
#

Yep on latest Matter Server, transitions work now with light.turn_on and light.turn_off service calls. Thanks!

raw wedge
#

FWIW, my nanoleaf beta bulb still does not pair with ha on 3.6.161. I'm glad I'm not using it in production. It pairs fine with the nanoleaf hub app and the home assistant otbr by the way!

#

but as soon as I try to pair it with home assistant it also disconnects from the nanoleaf app

digital star
raw wedge
#

app! not hub, nanoleaf app, sorry that was typo!

#

the ha otbr is available on the network and the nanoleaf app apparently finds it and uses it

digital star
#

Okay so it connects to the thread network but isn't controllable/pairable via matter then

uneven acorn
#

I noticed that some devices that are not showing in HA, but work fine in AppleHome show the following in the Matter log for example:

2024-03-07 20:16:21 (MainThread) INFO [matter_server.server.device_controller.node_35] The SDK is communicating with the device using xxxx:xxxx:xxxx:x:xxxx:xxxx:xxxx:xxxx
2024-03-07 20:16:21 (MainThread) INFO [matter_server.server.device_controller] Node 35 discovered using fallback ping

So, if it was discovered… why was it never marked as available in HomeAssistant? I checked, all the _matter services are showing as well…

#

I have this for at least 4 devices at the moment. Reboot of the device doesn’t help either

digital star
#

Likely because it's not responding to HA's request for information. Seems to be a current issue with NL devices

hexed cargo
uneven acorn
hexed cargo
weary merlin
hexed cargo
#

questions about the OTBR can better be asked in the thread channel.

weary merlin
#

Ohh sorry, so many components to this issue 😆

digital star
weary merlin
digital star
weary merlin
digital star
#

Before diving into your settings I would honestly consider either 1) testing with a more stable device or 2) waiting on NL to improve their FW

weary merlin
#

That's fair, if there a official nanoleaf discord link? To where I can stay updated with the works actively happening on the firmware? I'm setting all this up for my not for profit organisation, I'm ready to scale the system, however I think holding off until this is more stable is a good idea

digital star
#

I would certainly not deploy these in a production capacity right now

weary merlin
#

Aa thank you for that, that will help a lot! Do we know of any other matter over Thread lights that are stable as of now?

empty moss
#

Not that match the quality of nanoleaf (build quality and light output). Some people in this discord have actually gone back to hue and zigbee until the nanoleaf situation improves or a decent competitor arrives.

weary merlin
# empty moss Not that match the quality of nanoleaf (build quality and light output). Some pe...

Appreciate the reply, yeah I'm starting to realise that it's a real pain atm, personally I'm a tech enthusiast but not technically knowledgeable, so it's taken me a many many months and research to get to this point, the goal is to convert my NGO to a fully automated facility's so we can focus on other important things. In your opinion would you say namoleaf has a very very long way to go? I hope it's not the hardware itself that's the issue and just the software, if it's software at least that provides some hope for consumers that things will work at some point in the future

Cheers

empty moss
#

Well one thing is the homekit version of their lights had a hardware problem. They were originally meant to be software upgrade able from homekit over thread to matter over thread. But the CPU wasn’t powerful enough in some way, so they stepped away from that promise. But that does mean the current line up should have better hardware for running matter.

#

The beta version of the matter firmware is meant to be loads better, but still not as stable as hue.

#

But they have had a long time to work on the problem so some of their users are starting to give up

digital star
#

I would not personally bank on them fixing their firmware soon. Their release candidate still has a lot of issues

#

I would consider matter over wifi and ZigBee (or even local control of some wifi bulbs) which are going to be more stable

#

(spoiler I'm one of the people that chunked their NL devices so I might be biased but I did also own them for 6 months and beta test them for 3)

weary merlin
#

@empty moss @digital star Appreciate the info, I'll keep an eye out and stay away from scaling up for thr time being, I was hoping to setup 200 iot devices, not sure what challenges I would face other then nanoleafs own issues. I have been frustrated with the issues recently, though I guess everyone's going through it, initially I thought I wasn't doing it properly.

bleak pond
#

you should be alright, but yeah nanoleaf are known to nuke a network and make downstream devices not work

fresh hatch
#

Just to add a bit from my experiences: I hit an issue with subscriptions on Nanoleaf bulbs while attempting to test some other issue, during which I had paired the bulb with chip-tool in addition to HA. This actually turned out to be really reproducible for me; I paired a second bulb with chip-tool, and HA was no longer able to subscribe to that bulb either. I unpaired chip-tool from both bulbs, and HA was able to resubscribe to both on the next retry. So I am pretty suspicious of a firmware issue here…

#

(should note that while paired with chip-tool, i could reproduce the subscription error with chip-tool too)

hexed cargo
digital star
#

This bolsters my theory that it's a hardware issue

#

Multi admin is more taxing on the HW right? I think the chipset simply can't handle it

fresh hatch
#

from what i could see with the message traces from chip-tool and higher log levels in HA, the bulb sends a reportdata command with the MoreChunkedMessages = true flag set, HA sends a statusresponse command, then instead of another reportdata command as expected, the bulb sends a standaloneack for the statusresponse, and just sits there.

#

i think specifically multi-admin requires reserving more ram to track things separately per fabric?

#

it's not really more processing power, but the extra ram usage might cause memory allocation issues with other stuff.

digital star
hexed cargo
#

@fresh hatch thanks for your observations!

#

That is the missing info we so needed

fresh hatch
#

i'll see if I can clean up any sensitive data from my chip-tool logs (i've sent them in full to nanoleaf already) to add to the issue as context.

hexed cargo
fresh hatch
#

(i'll also note that doing a wildcard subscription only on endpoint 0 worked on my bulbs, while doing one only on endpoint 1 failed)

digital star
#

Btw I don't have any of the issues with my other devices (bulbs included); no issue with the latest server version for me

fresh hatch
#

no idea why; endpoint 0 has a lot more attributes and needed to send a whole bunch of reportdata commands, on endpoint 1 it sent a single reportdata containing everything except the manufacturer extension cluster before failing.

hexed cargo
#

We actually switched to one single wildcard (all endpoints) on request of NL

#

Because they had memory issues due to us subscribing to individual attributes

#

(as it should, but that aside)

fresh hatch
#

(that manufacturer extension cluster thing is kinda suspicious to me, tbh, but i have nothing to back that up)

hexed cargo
fresh hatch
#

no idea what the nanoleaf one is for, anyways; it only has two attributes and on my bulbs the first one is just empty (0 bytes long) and the second is 256 bytes of 0s.

hexed cargo
#

It's kind of weird if you imagine that they also communicate directly with the bulbs over their own protocol

hexed cargo
#

at least for Eve we need to manually read the values

digital star
#

Probably some vestigial code someone forgot about. @fresh hatch pointed out they had place holder code in their cluster attributes that they only just removed recently

fresh hatch
#

i should get a complete log of the subscription process with only one fabric to see what the reportdata commands looks like when it works.

#

oh, yeah, the fixedlabel cluster that said room: "bedroom 2", etc :)

hexed cargo
#

Thanks! If you look at the issue report I created on the SDK repo, I attached my log. There is a pattern in the connection. Soo this is 99% sure an issue with the device

#

Holy crap, imagine that I even went wrapping Nanoleaf bulbs in tin foil trying to induce the issue

digital star
#

Have you asked NL for hazard pay?

hexed cargo
#

If this is it, I'm going to send you a beer @fresh hatch !

fresh hatch
#

but yeah, i suspect the fact that they're running multiple competing stacks is the reason for some of the transition jank, like an issue that appeared in a recent firmware where bulbs turned off via matter would sometimes stay on at 1% - the nanoleaf app reported them as on, but via matter they said they were off.

#

setting the on-off-transition-duration to 0 to disable the transition on the matter side made it use the smoother off transition from their own stuff and then it reliably turned off

#

(also, the on-off-transition-duration attribute in their current beta firmwares doesn't behave according to matter spec at all)

hexed cargo
#

Well, we also did add transition in our code since this release. But it works fine with some of the wifi lights I tested and the Hue bridge

#

And I'm pretty sure I followed the spec implementing it 😉

digital star
hexed cargo
digital star
#

They've been really stable for me. I don't actually control them via matter because of that issue but it's the only way I can keep them connected locally without the app

weary merlin
#

@digital star is it just me, or the wiz colors turn bad when you dim them?

digital star
weary merlin
fresh hatch
#

i've seen that on nanoleaf bulbs too; at lower brightness levels you have fewer "steps" in the pwm control available to adjust the different led colors relative to each-other

#

iirc at the lowest brightness, nanoleaf can only do red, blue, or green, since mixing in a second color would make it brighter.

digital star
weary merlin
#

No just Bluetooth, I immediately returned it thinking its just bad quality product

fresh hatch
#

ah, they can do more than just R, G, B, but the exact range of colors does seem pretty limited

digital star
#

I didn't even know you could control them via Bluetooth

#

I also have the latest model so

jovial raft
#

I appreciate how far Home Assistant has come in Matter support since the release of version 2024.2. I am actually not on the Nanoleaf beta so I know the improvements I've seen are all thanks to Home Assistant.

I have steadily increased the amount of light-based automations and the lights are holding up fine for the most part. There are still occasional crashes but Home Assistant is able to quickly reconnect more often than not. I am hopeful that Nanoleaf will iron out the remaining issues.

hexed cargo
#

Good to hear! Yeah, you probably don't want to switch to the NL beta as that does solve (most of the) spontanuous resets of the bulbs but introduces some other issues that are now being investigated.

fresh hatch
#

new beta firmware dropped :) they claim this one fixes the subscription issue … by temporarily removing their vendor specific cluster on the lighting endpoint until they can work out why it broke things.

hexed cargo
novel snow
weary merlin
hexed cargo
weary merlin
#

When you say vendors, that'd be home assistanr, google, Apple etc..

fresh hatch
#

well, in this case the vendor is "nanoleaf"

#

they added an extra cluster to give them the ability to provide/set some information or allow running some commands to control features of their bulbs which aren't exposed through generic matter interfaces.

#

(they have not publicly said what features it was supposed to allow controlling)

bleak pond
#

Got a feeling it has to do with integrating with that sync/4D stuff

fresh hatch
#

i would have expected that to run through their proprietary protocol tho, not matter

weary merlin
#

Thanks for making me understand better

#

I so hope they add over the thread software updates instead of Bluetooth

#

Well that i would make good use of that at least 😅

fresh hatch
#

iirc the "OTA" cluster for doing firmware updates via matter over thread is present on the devices but not yet supported by nanoleaf.

weary merlin
#

As in the hardware supports it, but nanoleaf hasn't implemented yet?

#

Well at least they have the ability to do it, that assures me we will see it soon

fresh hatch
#

unclear. all i know if that they don't currently provide firmware for ota updates via matter, which could mean they haven't implemented some of the software parts, or could mean they simply haven't tested it yet.

weary merlin
#

My goal is a bit different then home users, im looking at it commercially. Not in a sense of doing selling or anything, but running it in a very large setup

fresh hatch
#

of course, HA doesn't support doing firmware updates on matter devices yet either afaik :)

weary merlin
#

Just need to give it time m, fingers crossed 🤞

fresh hatch
#

yeah, for your use case i'd say that if you want to do it right now, you'd probably be better off with, say, zigbee. but if you're ok with waiting, i dunno, 6 months and re-evaluating, maybe things will be better by then?

weary merlin
#

Yeah let's hope, I got some time, my team is still working on the facility, and other areas of the organisation. So not yet ready to scale anyways

#

I was going to make a bulk purchase a month back though, and then some things popped up, and found out that matter is not stable etc etc

fresh hatch
#

matter and thread as specifications are "stable" at least for basic lighting applications, the main issue there is features/device types that haven't been specified yet. the main problem with matter over thread right now is that all the devices and firmware code are still very new and there's not many devices available.

weary merlin
novel snow
fresh hatch
novel snow
#

But you did had an issue during subscription no? The one you were able to reproduce with chip-tool?

fresh hatch
#

yes, i was seeing a subscription issue in HA and was able to reproduce it in chip-tool as well and grab some more detailed logs

#

but i don't recall ever seeing that specific "End of TLV" error message

novel snow
#

Ok I see.

#

Happen to have the logs around what you have been seeing?

fresh hatch
#

(got the same effect with either an all endpoints subscription or just on endpoint 1, so this log is the shorter output of attempting to subscribe only to endpoint 1)

hexed cargo
#

Ah I spot a timeout during the subscription set-up in your log. So basically alomost the same as what I managed to reproduce. Halfway sending the initial response to the read request, something blows up and then things are in a halfbaken state. The subscription has not completed and the case session is reconnecting. I think we should destroy the subscription in case this happens in the SDK (or at least the python wrapper parts of it) in chance another vendor comes by with NL like issue.

hardy rivet
#

The latest firmware 3.6.170 seems really stable to me after half a day. I do not see any timeouts or other issues in the HA Matter server logs. 7 bulbs are connected to HA, Apple Home and the Nanoleaf app. If my log is still that clean tomorrow, I may add more bulbs. How is the latest firmware for you?

hexed cargo
#

We hear good reports about the new firmware version but the problem is officially not yet solved so it needs some more love.

fresh hatch
#

honestly, something that would help is if the HA ui didn't get completely disabled when the subscriptions fail. For example, being able to use the fabric management dialog can get you out of the bad state, if it was due to a multiadmin bug

hexed cargo
#

that is HA core behavior if an entity is unavailable. There is some discussion about that though

#

Like we should probably have a "possible available" and a "really unavailable" state

#

in case of possibly unavailable commands should still work

#

now its all or nothing

#

other protocols such as zigbee and zwave suffer fron this as well

fresh hatch
#

something weird i noticed on android is that it looks like google play services is setting up the device on a google fabric first and then sharing it with home assistant, rather than home assistant being the first fabric

hexed cargo
#

yes that is the framework

#

apple does the same

#

keychain fabric

#

a bit annoying but that is how they decided to implement it

#

we just leverage the phone framework (like api calls on the os level)

fresh hatch
#

which means the devices start out in multiadmin mode when initially set up, triggering this nanoleaf firmware bug on the bad firmware versions :(

hexed cargo
#

For your info, we cant incorporate the sdk directly in our app for certification reasons (and convenience, you dont want the sdk beast in your companion app)

#

so hence we use the iOS and Android api's

#

well, every matter compatible app will do the same

fresh hatch
#

hmm. I think i understand why that's done from a security point of view. It means that the third party fabric never sees the device's initial hardcoded matter provisioning code - it only sees the temporary code generated for adding an additional fabric to an already provisioned device.

hexed cargo
#

one idea I have in mind is to ask the user to remove the intermediate fabric after successful commission to HA

fresh hatch
#

yeah - and doing that would work around the NL multiadmin issues; there you run into the problem is that the UI for doing that isn't visible when the subscriptions fail :)

hexed cargo
#

well, I dont think that we should fix their issue 😉

#

that issue itself should be fixed first

fresh hatch
#

indeed.

#

annoyingly, i'm on a private beta firmware for the bulbs right now to test some other stuff, and it's missing the subscription fix added in the 3.6.170 public beta firmware :( i'll keep bugging them about it…

hexed cargo
#

Yeah, well I still find it super weird that the custom cluster breaks it all. Especially considering there is nothing special in the custom cluster. It makes you wonder if that is just a side effect from something else. Our controller was one of the first to be based on SDK 1.2 and I'm not entirely sure if the controllers from Apple and Google that run in homes are already based on 1.2 (I think they're still 1.1) so we could also look at this from that perspective.

#

Good thing is that we're on speaking terms with them and they take HA users seriously now

fresh hatch
#

the fact that it depends on multiadmin is kinda weird; i know the sdk supports having some types of attributes save values separately for each fabric so maybe there's something set up wrong regarding that on the custom cluster.

hexed cargo
#

Yeah, but so far I have not seen custom clusters being sent in the subscription, only with manual read requests

#

So other vendors are not sending custom clusters in the subscription read request

fresh hatch
#

hmm, it doesn't include them even when the subscriptions succeed? I have to admit that i hadn't thought to check that.

hexed cargo
#

The entire SDK python wrapper cant even handle custom clusters being returned in a subscription. They will not pass certification as that is using the python test framework

fresh hatch
#

bumped up my log level temporarily to check - they are sending the custom cluster in the reportdata commands in response to a wildcard subscription command.

hexed cargo
#

yes and I think they shouldnt

#

🙂

#

at least as a test for now

fresh hatch
#

(huh, i guess the custom cluster must be related to the reason for this private beta firmware; they added more attributes to it)

hexed cargo
fresh hatch
#

er, wait, no, read it wrong. exactly the same attributes.

hexed cargo
#

ah ok

fresh hatch
#

only attributes are 0x115A_0000 which is empty, and 0x115A_0001 with 256 bytes of 0s. no commands.

hexed cargo
#

BTW: if custom attribute data is sent, nothing breaks, the data will simply not be parsed (an error will be logged in fact)

#

did you ever got to reproduce that "End of tlv" error I once got ?

fresh hatch
#

no; i think you said it needed the sdk compiled with some extra debug options to get that?

hexed cargo
#

Nope, I was trying to reproduce that issue so I was even wrapping lights in alu foil etc.
But the moment I got that issue was when the light was connected to a BR that had a weak wifi signal.
Probably that all was just a coincidence but still the symptoms were the same.

hardy rivet
digital star
#

Congrats to you all for towing NL across the line lol

hardy rivet
#

Even though you gave up on them, you also helped… 😉

digital star
#

True but ultimately @fresh hatch found the real problem

fresh hatch
#

Hilariously, I'm now running a private beta firmware on my bulbs... which re-introduces the problem, so I don't get to benefit from this nice new public firmware release :(