Bobby, a new voice assistant and Snowy successor

[karmic lion]

this is probably entirely undebuggable without a bigboard.

and a home-built firmware

so just generally undebuggable

[sick apex]

what's the exact phrasing you guys are using?

I can try it too

[karmic lion]

I guess I could try removing the appglance stuff.

which would be sad

[barren raptor]

if pebbleos has no major bugs why does it keep crashing
checkmate eric magicovsky

[edgy flame]

"set a x minute timer" is my parlance of choice when speaking with robots

[coarse cedar]

"stelle einen fünf minuten timer" (german)

[sick apex]

worked perfectly

[barren raptor]

i also can't replicate it on a pts

[sick apex]

omg the app glance is updating the timer live

[sick apex]

I'm also on PTS (basalt)

[sick apex]

yeah I think I'd butcher that

but anyways, my watch did not crash so 🤷

[edgy flame]

wonderful

our watches are just schizophrenic

[sick apex]

did it twice just to be sure: once from the quick launch and once from the app

[coarse cedar]

well to be fair I am running a fork
and to be not fair, that fork only edits the service

[sick apex]

and also lavender is using stock Bobby I think

[karmic lion]

If it's crashing on launch and also crashing on setting a timer it does seem likely to be appglances.

especially given that we already know them to be broken in other ways

[coarse cedar]

yeah :(

[sick apex]

I wonder what's making it crash some times but not others

on specific watches

[edgy flame]

the humours aren't aligned

[karmic lion]

one could test this in one's fork by removing the one line from app_glance_manager_refresh in app/src/c/appglance/manager.c

[coarse cedar]

can do

[karmic lion]

I wonder whether it's a stack overflow, though I don't know why that would vary by watch. IIRC stack overflows do sometimes take down the whole system

and that method has some giant arrays in it

[edgy flame]

/shrug watch came back up from SOS mode and bobby still had my timer running

[karmic lion]

I wonder whether the system knows that

[edgy flame]

it did! app glance was running and everything

I opened the app and cancelled from the menu

which.. worked

[karmic lion]

maybe I'll try moving the big stack arrays to the heap

[harsh brook]

Does reporting a conversation actually just yeet you back to the app list or is it failing in some way for me? (Trying to report the inability to resume stopped timers)

[coarse cedar]

ok so uhh I cannot build it using rebbletool apparently.

[edgy flame]

idfk, i dont even know how all my apps were still installed and my watch face still had its custom theme after a factory reset

[karmic lion]

rebbletool is many types of broken

[edgy flame]

watch still dies when i tell it to set a timer. idk man ¯_(ツ)_/¯

[karmic lion]

in particular I know that it will misconstruct the menu icons

[sick apex]

then maybe it wasn't a real factory reset?

[karmic lion]

though I'm not aware of it failing to compile at all

oh I think I had to fix two different bugs in it to get it to get that far

so I am aware of it failing to compile at all

[barren raptor]

this has happened every time my watch has factory reset due to crashing repeatedly

it seems like that factory reset isn't a true factory reset

[coarse cedar]

how to fix not being able to build on my system:

push the thing to the repo and let github build it.

[karmic lion]

in the meantime I have a change that shaves 128 bytes off the stack while constructing appglances

which may help

though it's just a wild guess

[coarse cedar]

yep, works when commenting out that line

[wooden wedge]

hmm, my bobby crashed

[south edge]

Katharine

[bobby-assistant:why-are-we-dangerously-rebooting] 1 new commit

e3c9aa5 Try reducing stack usage in appglances. - Katharine

[karmic lion]

CI will spit out a build that might fix it

[wooden wedge]

I tried setting a timer and it kept setting reminders and apologizing for it every time

[rough cliff]

Works fine for me 🙃

[karmic lion]

you should report the bad thread!

[wooden wedge]

I can't if it crashed

[karmic lion]

this is true

[wooden wedge]

I would love to though

[karmic lion]

I like the apologising

"I've set a reminder for you. Wait, that's wrong. Why am I stupid??"

alas, the ponies are not very smart

[wooden wedge]

bad news

[karmic lion]

just trying their best

[wooden wedge]

I tried reporting and it's stuck on running horse

[karmic lion]

are you using cobble?

[wooden wedge]

yes I am

[karmic lion]

I think we already know this to mysteriously not work on cobble

[wooden wedge]

I wonder why that is

[karmic lion]

me too!

absolutely nothing special is happening

[wooden wedge]

it's the same with reminders dialog

that also gets stuck on running horse

[karmic lion]

huh

[barren raptor]

I choose to believe that bobby has a personal preference for the old app

[karmic lion]

but the quota screen works?

[wooden wedge]

yes

[harsh brook]

I'm just getting the app list so I guess my bobby isn't reporting successfully either

[karmic lion]

crashing is not successful, no

also no reports have materialised

I'm glad everything crashes for everyone but me.

[wooden wedge]

that's how it goes with software!

reproducing some things is impossible sometimes

[karmic lion]

the report doesn't even do anything exciting, it just sends a UUID to the phone to do all the work

anyway maybe this one will not crash your watches when you set an alarm

[edgy flame]

let's try Reducing Stack Usage

ded

[karmic lion]

reporting a conversation does use some memory (mostly to show you the warning about reporting conversations) so I guess that could just OOM.

[wooden wedge]

anyway, what happened in that thread was: I asked for a timer and it told me there was an error setting one, then it tried again, then it decided it will set an alarm instead and failed and then set an alarm after saying it will try another approach

[coarse cedar]

cherrypicked the commit, died

[karmic lion]

fuck it, appglances are out

[wooden wedge]

I didn't have to prompt it multiple times! it just did that

[karmic lion]

pebble os is trash

nothing works

[wooden wedge]

oh my god please set a timer once

it keep setting alarms or reminders

[karmic lion]

it'd be cool if you could report this behaviour

but alas, cobble.

[harsh brook]

Ok reporting worked now, only variable that changed I think was my phone being connected to power now after previously being in oxygen os's super low power mode

[karmic lion]

At least the "I will past that feedback on to my development team" is a common lie that they all do

it seems like the watch shouldn't even know baout this

[wooden wedge]

and a crash

because I long pressed the select button

[karmic lion]

fun.

maybe I will just abandon this entire project

my patience for an OS that doesn't work paired to a mobile app that doesn't work is wearing thin

the crash on long pressing select is presumably somehow actually my fault

[wooden wedge]

yeah, that is just confusing

[south edge]

Katharine

[bobby-assistant:why-are-we-dangerously-rebooting] 1 new commit

4c2b302 Death to appglances. - Katharine

[wooden wedge]

the glance is cool

it works just fine for me

[karmic lion]

but it crashes the watch for other people!

[wooden wedge]

that being said, I can only use alarms

am I too nice? I always say please

maybe I should be rude

[karmic lion]

build without appglances

[sick apex]

remember though: this watch doesn't work for a whole 7 days in a row! other watches don't work for only 1-2 days

[karmic lion]

which presumably will not crash

[sick apex]

tf

Bobby sounding like my mom

[karmic lion]

bobby will not cooperate if you're rude

[wooden wedge]

well, it tried when I said yes

but then it set an alarm again

[edgy flame]

works on my watch

[wooden wedge]

told me it set a timer

[edgy flame]

well, shit!

[wooden wedge]

and that didn't trigger a lie detector

[edgy flame]

app glances just don't work I guess

that really sucks. i wish they did work

[sick apex]

it's such a nice feature, too

any way to just make it a toggle for those who's watches it works fine on?

[karmic lion]

the lie detector cannot distinguish alarms and timers so that tracks I guess

[edgy flame]

appglance is a dubious concept for a UI considering that like, it only works if you have the app installed already. and if you have the app installed you can just click it and get 100% of the info instead of the snippet. but it does make your launcher look very nice, and maybe it's more useful for the example app idea which doesn't have a UI except a confirmation or error when you take a one click action

[wooden wedge]

hmm

it apparently doesn't know the difference outside of the lie detector either

[karmic lion]

but if you have four appglances you can receive four pieces of information at once

from different apps!

it's like complications.

[wooden wedge]

idk if it makes a difference, but it seems like local dictation tends to use 'five' instead of '5'

[karmic lion]

it really shouldn't matter

it might matter anyway

but it shouldn't

[coarse cedar]

https://cdn.discordapp.com/attachments/496219707771322379/1289694941366456400/vbn45ehjubrd1.png?ex=67de7dc3&is=67dd2c43&hm=39f6d50bc8b7b82d34e060aca64fc5838770e6c1d22caeb66c9e8fd2cae6e7d6&

[karmic lion]

if you had working reporting this would be much easier to look at

unfortunately, cobble.

[wooden wedge]

I'm in loading hell now, for good reasons

[karmic lion]

I don't understand how it can not work

[wooden wedge]

I should stop trying to insult it

[karmic lion]

for historical raisins, timers are just alarms with is_timer=true

maybe this confuses the LLM but only for you specifically

[wooden wedge]

well, I can try grabbing the other phone with the pebble app

let's see if it makes a difference

[karmic lion]

the historical raisin is that tokens were much more expensive in 2023

if someone for whom apptimers crash wants to try to debug the apptimers, then we can keep having them

but otherwise I think they are dead.

err, appglances

(we have many apptimers)

[wooden wedge]

I don't know if they crash because I can't even create one

[karmic lion]

alarms do also create appglances

[wooden wedge]

alarm didn't crash for me

[karmic lion]

now I wonder whether only timer ones crash

[wooden wedge]

yeah

nope, timers work just fine in pebble app

I should debug cobble for this at some point, this is awful

[karmic lion]

how on earth does cobble impact whether an LLM on a server can set timers

the amount of sense this makes is zero

unless it's trying to set a timer but some miscommunication with cobble is causing that to fail and then it does something else

[wooden wedge]

alarms seem to work just fine

[karmic lion]

those have somewhat different javascript!

[wooden wedge]

apparently

[karmic lion]

if reporting worked this would be very obvious

If you have applogs I could just manually extract the conversation thread

[wooden wedge]

hmmm, I would need to grab my laptop

we used to have logs in the app but that's broken now

[karmic lion]

July!

I guess I really do need to make that cobble warning

if half the buttons in the app don't work and also it somehow influences what the model tries to do

annoyed that I have to build my own UI for this because cobble also doesn't implement the lazy solution.

I have lost interest in trying to ship Bobby in the next day or so, I'm going to go do something else

I guess I'll ship an update that removes appglances first since crashing on timers is pretty bad.

[wooden wedge]

I might look into what's broken about cobble

I would also debug appglances, but I might have trouble dealing with firmware code

[south edge]

Katharine

[bobby-assistant:main] 3 new commits

e3c9aa5 Try reducing stack usage in appglances. - Katharine
4c2b302 Death to appglances. - Katharine
2096e2e Bump to 0.24. - Katharine

[karmic lion]

also it doesn't seem to crash for you

[wooden wedge]

yeah

I do have another watch I could try with actually

[karmic lion]

I am also mystified by how this causes a crash for anyone:

static void prv_select_long_pressed(ClickRecognizerRef recognizer, void *context) {
  SessionWindow* sw = context;
  if (!conversation_is_idle(conversation_manager_get_conversation(sw->manager))) {
    return;
  }
  ActionMenuLevel *action_menu = action_menu_level_create(2);
  action_menu_level_add_action(action_menu, "Prompt", prv_action_menu_query, sw);
  action_menu_level_add_action(action_menu, "Report conversation", prv_action_menu_report_thread, sw);
  ActionMenuConfig config = (ActionMenuConfig) {
    .root_level = action_menu,
    .colors = {
      .background = BRANDED_BACKGROUND_COLOUR,
      .foreground = gcolor_legible_over(BRANDED_BACKGROUND_COLOUR),
    },
    .align = ActionMenuAlignCenter,
    .context = sw
  };
  vibe_haptic_feedback();
  action_menu_open(&config);
}

I have tried with two and neither crash 😄

[wooden wedge]

yeah, weird

I tried another watch and glances do work without an issue

on pebble app that is of course

[karmic lion]

well this is depressing.

[coarse cedar]

sorry for having a weird ass pebble 2 😔

[wooden wedge]

both of those are basalts btw

one is time and the other is time steel

[karmic lion]

I have tried on both without issue

I don't think the watch type is related

Who knows what is

The fact that it always happens for some watches and never for others is weird.

[coarse cedar]

smh shouldnt have gone bankrupt

[karmic lion]

I have no idea what to do with the many types of nonsense cobble apparently provides

The obvious answer is to just ignore it but the everyone who actually uses this thing seems to be be using it.

[wooden wedge]

which part of the logs is going to be the uuid you need

[karmic lion]

I think they both log it?

I know the JS does it prefixed with a "t"

Not sure about the watch logs

[wooden wedge]

0e1388af-718b-4e2b-abcb-9f63df64b987

is that right?

[barren raptor]

this is so dissapointing, damn you Pebble OS.

I'm really hoping that these bugs end up getting fixed someday

i still need to figure out if my quick view bug is consistenly replicable in some way or if it's just my watch

[karmic lion]

yes

[wooden wedge]

surprising that I found it

[karmic lion]

[
  {
    "role": "user",
    "content": "Set a timer for 5 minutes"
  },
  {
    "role": "model",
    "content": "",
    "functionCall": {
      "args": {
        "delay_mins": 5,
        "what": "Timer"
      },
      "name": "set_alarm"
    }
  },
  {
    "role": "function",
    "content": "",
    "functionResponse": {
      "name": "set_alarm",
      "response": {
        "error": "Invalid time string ''."
      }
    }
  },
  {
    "role": "model",
    "content": "My apologies, I seem to have encountered an error. Let me try that again.\n"
  },
  {
    "role": "model",
    "content": "",
    "functionCall": {
      "args": {
        "delay_mins": 5,
        "time": "2025-03-21T21:51:36+01:00",
        "what": "Timer"
      },
      "name": "set_alarm"
    }
  },
  {
    "role": "function",
    "content": "",
    "functionResponse": {
      "name": "set_alarm",
      "response": {
        "status": "ok"
      }
    }
  },
  {
    "role": "model",
    "content": "OK. I've set a timer for 5 minutes.\n"
  },

it's just making up arguments to set_alarm

"what": "Timer" is not a thing

I don't understand why this would be cobble-specific, though.

[wooden wedge]

yeah, that's weird

[karmic lion]

maybe 5 vs five completely changes its understanding of the universe

anyway I guess I know how to fix this

it's just annoying

[coarse cedar]

ask another llm to convert string to int/int to string

[wooden wedge]

a8c1703c-e897-4da6-a565-338a49c236d9

mmmm

[karmic lion]

also I like how it saw that the spec asked for delay_seconds and then figured it could just make up a delay_minutes and that would work

this time it called set_reminder with "what": "Timer"

you can't just call random functions that aren't the right one and ask them to do what you want by passing "what"

this isn't a thing, bobby

[wooden wedge]

aa5c1e2b-ec64-4e70-83e7-22f8c1eeb97a

[karmic lion]

I built this lovely webpage and now I'm reading JSON strings out of redis

it used the "what" parameter despite your request

[wooden wedge]

disappointing

[wooden wedge]

I cannot get it to actually do the right thing

last one: 853d5d3a-b90f-40bf-96e7-b29d279c4774

[karmic lion]

how on earth is this cobble specific though

what is going on

[wooden wedge]

that is a wonderful question

it seems that cobble doesn't have a single "what" in code!

D/Cobble  (20224): [PKJSApp$start$2$invokeSuspend] Received app message: AppMessagePush StructElement(size=1, linkedSize=null, value=18)
V/Cobble  (20224): [AppLogHandler$listenForAppLogs] <1d272bb4-cf00-4e3e-a7f3-7394309e0341> app_timer.c: Timer 188396 does not exist
I/chromium(20224): [INFO:CONSOLE(113)] "Inbound app message!", source: file:///data/user/0/io.rebble.cobble/cache/js/1d272bb4-cf00-4e3e-a7f3-7394309e0341--1.js (113)
I/chromium(20224): [INFO:CONSOLE(114)] "{"isTrusted":false,"payload":{"REPORT_THREAD_UUID":"853d5d3a-b90f-40bf-96e7-b29d279c4774","FEEDBACK_APP_MAJOR":0,"FEEDBACK_APP_MINOR":23,"FEEDBACK_ALARM_COUNT":0},"data":{"REPORT_THREAD_UUID":"853d5d3a-b90f-40bf-96e7-b29d279c4774","FEEDBACK_APP_MAJOR":0,"FEEDBACK_APP_MINOR":23,"FEEDBACK_ALARM_COUNT":0}}", source: file:///data/user/0/io.rebble.cobble/cache/js/1d272bb4-cf00-4e3e-a7f3-7394309e0341--1.js (114)
V/Cobble  (20224): [AppLogHandler$listenForAppLogs] <1d272bb4-cf00-4e3e-a7f3-7394309e0341> ation_manager.c: Sent message successfully.

[karmic lion]

If it weren't for the reminder screen getting stuck I'd wonder whether XHR were somehow broken

though I guess reminders are XHR and that works.

[wooden wedge]

D/Cobble  (20224): [PKJSApp$start$2$invokeSuspend] Received app message: AppMessagePush StructElement(size=1, linkedSize=null, value=19)
I/chromium(20224): [INFO:CONSOLE(113)] "Inbound app message!", source: file:///data/user/0/io.rebble.cobble/cache/js/1d272bb4-cf00-4e3e-a7f3-7394309e0341--1.js (113)
I/chromium(20224): [INFO:CONSOLE(114)] "{"isTrusted":false,"payload":{"REMINDER_LIST_REQUEST":1},"data":{"REMINDER_LIST_REQUEST":1}}", source: file:///data/user/0/io.rebble.cobble/cache/js/1d272bb4-cf00-4e3e-a7f3-7394309e0341--1.js (114)

[karmic lion]

assuming the reminders are actually set

[wooden wedge]

that's all she wrote

[karmic lion]

the reminder ode does not bother checking this.

oh

no, never mind

I don't get it

[wooden wedge]

oh, the feedback screen just doesn't function

it doesn't even give me an option to speak

[karmic lion]

it is plausible I broke that when meddling

[wooden wedge]

I'm still on 0.23 fwiw

[karmic lion]

no, the feedback screen is fine

If dictation_session_start is silently failing, it would appear to just do nothing

that screen is very simple, I do not see much space for it to fail to even start dictation

unless dictation is broken

[wooden wedge]

yeah, so it works, f you start it from the menu without using bobby

[karmic lion]

ahh

I could believe that

I have not tried that

[wooden wedge]

now it works??

[karmic lion]

I mean I can't really believe it because it still makes no sense

[wooden wedge]

weird, it did happen though

I also got absolutely no log when it did

it just reported nothing at all

[karmic lion]

this screen was also randomly crashing on me at some earlier point and then mysteriously stopped

probably memory corruption, I don't know.

if it is there is no chance I will ever find it

[wooden wedge]

did you replace the big bobby image at any point?

[karmic lion]

I did not

I have been busily dealing with it crashing watches and being mysteriously brainwashed by cobble

[wooden wedge]

that's probably a better use of your time

not the cobble part, the other part

if I understood what the actual issue with feedback, reminders and reporting I would go and fix it

alas, I have no idea

I guess I will try placing various log messages until I find something

@upbeat garden ^ I might as well ask if you have any ideas given the logs above and context here

[upbeat garden]

not sure x.x

lots of places it could be being weird

[wooden wedge]

yeah, I will approach it tomorrow then

I have time to spend doing nothing productive

[south edge]

Katharine

[bobby-assistant:main] 1 new commit

61c7765 Split alarms and timers completely. - Katharine

[wooden wedge]

that might help

[karmic lion]

server-side change only. you may as well try setting a timer in cobble again

if it doesn't work now I give up

[wooden wedge]

mmmm

now it's having trouble doing anything!

"got an action: {"action":"set_alarm","cancel":false,"duration":0,"isTimer":true,"name":""}"
in the logs

that's different than before

ad4fff24-835f-414c-a6da-adf2ddc41aab

[karmic lion]

what

oh no that makes some sense I guess

it's just making up a delay_mins parameter that does not exist

because why not

but only if you use cobble!

[wooden wedge]

heyyy, I got it to tell me it will inform you about it!

hm, setting a timer isn't going very well

ayy, a crash!

f941d509-c24e-49cd-aa86-396c9c136be7

[karmic lion]

I went for a walk

I can't look that up, they expire after ten minutes

(I am still out)

[wooden wedge]

that's fine

[karmic lion]

@wooden wedge it'd be interesting to see the URLs it's making requests to (minus the auth token)

maybe cobble is somehow causing it to misconstruct requests in a way that result in the model being given odd instructions

it prints these out in the logs when it makes the request (they start wss://bobby-api.rebble.io/query)

[wooden wedge]

oh yeah, I was gonna ask if you mean websocket connection

[karmic lion]

indeed

[wooden wedge]

I can give that to you tomorrow morning, I wanna sleep

[karmic lion]

it doesn't actually send anything over the websocket to start, only if it's asked to take some action (like set a timer)

fair enough.

[south edge]

Katharine

[bobby-assistant:main] 1 new commit

1a0d7a1 Improve timer setting resilience. - Katharine

[karmic lion]

I have changed the server to tolerate what the model actually does for you

I still do not understand why it does it.

[karmic lion]

Github Copilot ran out of ideas for sample prompts pretty quickly:

What is the capital of Burkina Faso?
What is the population of the United States?
What is the GDP of China?
What is the speed of light?
What is the atomic number of gold?
What is the boiling point of water?
What is the freezing point of water?
What is the atomic weight of carbon?
What is the atomic weight of oxygen?
What is the atomic weight of hydrogen?
What is the atomic weight of helium?
What is the atomic weight of nitrogen?
What is the atomic weight of silicon?
What is the atomic weight of sodium?
What is the atomic weight of potassium?
What is the atomic weight of iron?
What is the atomic weight of mercury?

anyway, sample questions

anyway this is the current list:

What's the weather like?
Set a baking timer for five minutes.
Will I need an umbrella tomorrow?
At 5pm, remind me to go grocery shopping.
Who was the 22nd president of the United States?
Set an alarm for 11am.
What is 50 dollars in pounds?
How do you say 'good morning' in German?
What is 47 squared?
Where is the nearest bar that's open?
What is the capital of Burkina Faso?
What is the population of the United States?
What is the speed of light?

[hollow rivet]

-# You should do "How heavy is 50 single dollar bills, in pounds" as an example

[karmic lion]

"I can't tell you the weight of US dollars, but I can convert 50 US dollars into British pounds if you like. Would you like me to do that?"

[south edge]

Katharine

[bobby-assistant:main] 4 new commits

da9ab1d Sane session window construction. - Katharine
897d618 Add sample prompts to home screen. - Katharine
3818b34 Translations don't need wikipedia. - Katharine
4d6d918 Downgrade fence pony for improved performance. - Katharine

[upbeat garden]

yeah 😔

im trying :p

[karmic lion]

this is the warning screen

[upbeat garden]

yeah

[karmic lion]

it is difficult to convey information in a space that fits in that speech bubble

words like 'unsupported' don't fit at all

[upbeat garden]

nod

this works though!

hopefully it wont be needed for too long

[karmic lion]

monochrome

(cobble also gets you a different background colour here. though it doesn't spread through the entire app.)

hopefully this actually works, I am overriding it to test

[south edge]

Katharine

[bobby-assistant:main] 2 new commits

2ed8a78 Clean up action menu quits. - Katharine
8d03b53 Add a cobble warning to the home screen. - Katharine

[edgy flame]

bobby will probably be really good for testing cobble/coreapp actually

since it's so... it does important things in pkjs and sends a lot of data

and is a very high quality app generally

[hollow rivet]

Plus it implements like every feature of the C API

[karmic lion]

except the ones that make watches reboot apparently

[hollow rivet]

That's... a feature?

[edgy flame]

bobby killed my watch and all i got was this lousy appglance

[karmic lion]

also apparently I can't send double quotes to the official Android app or it drops the message

so that's fun

[hollow rivet]

Could you send a different character instead? Or like... the fancyquote character instead

[karmic lion]

I sent a single quote

[hollow rivet]

Right I suppose that'd work 😅

[edgy flame]

i have been thinking a lot lately about how good bobby is as a pebble app. like it's just wonderfully designed

it does a lot of things, enough things that code size is a problem, but it doesn't feel complicated at all. it's a button an a voice interface and a few menus and it's just really fucking good

[karmic lion]

-------------------------------------------------------
BASALT APP MEMORY USAGE
Total size of resources:        21190 bytes / 256KB
Total footprint in RAM:         43372 bytes / 64KB
Free RAM available (heap):      22164 bytes
------------------------------------------------------- 

send memory 😦

I might still be able to pull off responses with images in them. if the images are small and/or low bit depth

[edgy flame]

it's so good

[karmic lion]

probably ought to start keeping track of memory and throwing away the end of the chat log as needed

[edgy flame]

i hope that bobby-chalk does not get absolutely destroyed by the space limits but i have a feeling something is going to have to give

[karmic lion]

diorite is slightly smaller!

-------------------------------------------------------
DIORITE APP MEMORY USAGE
Total size of resources:        20193 bytes / 256KB
Total footprint in RAM:         43092 bytes / 64KB
Free RAM available (heap):      22444 bytes
------------------------------------------------------- 

slightly.

[edgy flame]

that's like almost a kilobyte

[karmic lion]

it's only 280 bytes in memory

though a bit more when the smaller images are loaded.

it's saved mostly by completely omitting code related to setting colours on things.

[edgy flame]

that is funny

on round we have about yay much space to display actual fullscreen widget content before the next card peek gets in the way. for long text, before the peek is in the way, or if the widget we're looking at is the last one in the stack, we can get some more vertican height because we don't need to fit a whole icon. rectangles for clarity

i think it makes sense to drop the previous peek from my sketches and only do next-peek: it makes the area we're drawing to more consistent, it is easier to reconcile the status bar being at the top, and we no longer have to deal with weirdness around 'should we be peeking top or bottom right now and what does the user expect'

widgets are supposed to always appear at the top of a response, correct? and there are only so many widgets with pre-defined behavior. so i think we can just have widgets always take up a fixed height, always show the peek, and then it's one button press to swap down and view the text response from the model, and the UI never has to worry about peeking to a widget below the text response

[karmic lion]

widgest can technically appear anywhere in a response, though almost always appear at the top

(occasionally they appear at the bottom. very rarely are they in the middle)

(the ones at the bottom are not generally useful)

[edgy flame]

i guess the user's prompt is a text block as well and widgets appear below the user's prompt

[karmic lion]

A response does sometimes contain multiple widgets

(if, for instance, you ask it to "show me all of my timers", you tend to get a widget for each one)

[edgy flame]

yeah

[karmic lion]

But for round purposes, a widget is always one screen high

[edgy flame]

i want to build this out in figma to where i have a component for each widget type, and i can string together an arbitrary stack of widgets

like prompt timer weather timer text lie detector
and have that behavior make sense visually

leaving this for tomorrow

[south edge]

Katharine

[bobby-assistant:main] 2 new commits

1dacd17 Feedback can now be sent conversationally. - Katharine
7aa2cdd Add a note about reporting to the feedback blurb. - Katharine

Katharine

[bobby-assistant:main] 1 new commit

7612c3d Bump to version 0.25. - Katharine

[karmic lion]

there we go.

I am done with this for today.

let me know if it explodes, crashes, misbehaves, or burns your house down

I bet the new LLM-based send feedback still manages to not work on cobble

you know speech recognition is going well when:

[karmic lion]

I asked gemini to make me a marketing website for bobby

behold: https://gemini.google.com/share/3eeace06be46

you can can tell what one image I gave it to work with

also it's in react for some reason

[wooden wedge]

it's not bad

I can give you the font names I used in a second

consistency™️

[coarse cedar]

the reverse pebble

[wooden wedge]

Nunito and Tomorrow

they are both on google fonts

[karmic lion]

informs gemini

it guessed Quicksand.

[wooden wedge]

I used bold variants of both

I think semibold of tomorrow and heavy of nunito

[karmic lion]

https://g.co/gemini/share/af24a8c801e0 tada

[harsh brook]

bobby seems to even be breaking the web for me

[karmic lion]

I think this is probably a dead end.

that's sad

[wooden wedge]

I mean, the design isn't bad

it does feel very google for some reason

[karmic lion]

the design is okay, though I'm not entirely sure it needed to be in react and/or full of animations.

[wooden wedge]

yeah

[karmic lion]

also I don't know what I'd do with it.

[wooden wedge]

of course animate the hell out of it

[karmic lion]

also I'm pretty sure there are some structural parts of this website that are missing.

for instance, anything other than this javascript file it has generated

I do like that it designed a Reverse Pebble

I have asked it to rewrite it without React

let's see what horrors arise

there was an attempt. https://gemini.google.com/share/b638868a767d

it just drew all those icons itself

which is why they are like that.

[wooden wedge]

the reverse pebble exploded

[karmic lion]

it did

and the top one went missing completely

which makes sense because previously it had made a Pebble Scrernshot react component and now it can't just reuse it

[wooden wedge]

if we had renaissance in a usable web font, we could make some nice animations inside watch body

make the text flow in and such

[karmic lion]

could also just screen record it actually running

(I say as if I have ever had it run successfully in an emulator)

[harsh brook]

Oh hey that link now works

[karmic lion]

blah

well I like it conceptually I guess. needs more screenshots in frames and more actually relevant text.

[karmic lion]

and a bobby there, yes

also those icons have "I'm doing thanks how help you" energy

@wooden wedge can you set timers yet? 😄

[wooden wedge]

I have not tried

[karmic lion]

the service now secretly accepts the nonsense that bobby does when you ask it

[wooden wedge]

omg

it works

[karmic lion]

if you want to try something truly wild, you could grab the latest version from the appstore and ask it to send feedback

which may or may not work

but at least it doesn't involve any of the buttons that are broken

(also it unlocks an exclusive new home screen line for Cobble users!)

(maybe)

[sand hare]

Would it be possible to use the buttons for polar questions?

[karmic lion]

I'm not sure what a polar question is

[sand hare]

Yes no questions.

[karmic lion]

Oh

There isn't really a spare button to use

[barren raptor]

Could make it a hold on up and down? nevermind I'm dumb

Or put it in the hold center menu?

[karmic lion]

I could put it in the hold center menu, but nobody would ever find it

and to pull this off I'd have to post-process the reponses, which costs money (quota) and response time

I already post process the responses for the lie detector but I doubt I could combine those passes

(though I guess I could do them simultaneously)

so if I'm going to do that it'd be good if the user could actually discover the menu

interesting concept though

just generally have quick responses

[barren raptor]

I mean, would there be any issue with always showing the options?

[karmic lion]

this still leaves the discoverability problem, though

[barren raptor]

if they're hidden in the menu, it won't matter too much that they're not context sensitive (unless you start adding more than "yes" and "no")

[karmic lion]

The hold menu already has the problem that nobody can find it

[sand hare]

Ai generated quick answers sounds great.

Not only for yes and no

[karmic lion]

I was thinking just hardcoded, but AI generated would be fun. Probably annoyingly slow to appear though.

and, again, how does anyone know they can long press

kinda want to prototype that though

[acoustic wolf]

I mean if the icon can stay "extended" that's a good sign it has long press

e.g. in music app

[sand hare]

Something like a first use page maybe?

[karmic lion]

Bobby's conversation UI does not use an action bar

[acoustic wolf]

right

[karmic lion]

I guess the dot could extend but I'm not sure that'd be effective because it's a pattern used nowhere else

if this could be condensed into five short words it'd fit in Bobby's speech bubble

[acoustic wolf]

"Try long-pressing Select!" maybe?

[sand hare]

A Howto conversation with bobby

[karmic lion]

sadly I think "pressing select" won't fit on one line and "select" is too long for the truncated space on the third line

maybe there just needs to be a tips version of the front page that doesn't have the big clock in exchange for more space

this would cost more code space (especially since figuring out the sizing of that speech bubble is already surprisingly convoluted)

regardless of the answers to this mystery, I am going to bed.

[barren raptor]

"try holding select", maybe?

[karmic lion]

that may or may not fit, but since it doesn't say when you should hold select it's not great

I guess if you hold select on that screen you could get another screen with the rest of the tip

[barren raptor]

I wonder if it's better to say "middle", since some people might not know the middle button is called select

[karmic lion]

I'm not sure it actually is called select

I have checked and it is in fact called select

but possibly

I'm playing with AI generated suggestions and feeling like "Yes" and "No" get you a lot of the way there

On the touchscreen watch it'd be fun to detect that it's a yes or no question and put yes/no buttons on the screen directly

But that is a problem for another year

[sand hare]

Would be interesting what "quick answers" ai gives you, when you maybe ask for a famous personality. There can be a lot for supporting your conversation. Not only for questions. Just don't call it quick answers 😄

[sand hare]

Those overlay buttons could also be visible on our watches maybe. The user has to find out how to use them. Double press or something.

[wooden wedge]

we do have cad models of the watches, I guess we could render them rotating into the website

I have the most cursed of ideas

[kind halo]

not going to lie
I find it kinda funny that the "About" page scroll starts lagging as you approach the bottom

it's 100% worth it though

[hollow rivet]

Is this a public resource? I was trydng to find some online but searching for 3D models just gives me a ton of aggregator sites with barely-relevant results

[edgy flame]

https://github.com/pebble/pebble-3d

[hollow rivet]

Yooo that's awesome thank you so much

[wooden wedge]

there is a more optimized pdc available here, it just needs integrating

[karmic lion]

I did

[karmic lion]

seriously

Well since the cobble warning also doesn't work I guess I will just have to hope that everyone has been ignoring Eric telling them to use cobble

[karmic lion]

Since the optimised pdc still lags, it might be worth seeing if you can render it into a four-colour PNG (black, white, transparent, ?) - or two for Diorite. Loading a bitmap might be the way to go.

[wooden wedge]

I will render that later

[wooden wedge]

oh, I missed that

[edgy flame]

would be cool if we could figure out what tends to make a pdc run slower than an equivalent bitmap, and have warnings in the tools. cc @azure goblet i think we talked about this?

[azure goblet]

Woah… so many messages. I see the mention, do I need to catch up with those 1000+ messages? 😂 is the question just: is a PDC slower to render than a bitmap? The answer is almost always “yes” (on Pebble, especially with PDC using anti-aliasing vs bitmap without blending). How can I help?

[wooden wedge]

do the tools let you check how long it takes to render a pdc to the screen?

I assume this is not really possible to check between different implementations, since the time is going to differ

but having the ability to compare the time between individual pdcs is already useful

[azure goblet]

Why? How would you act on what values?

[wooden wedge]

I mean, if rendering takes more than a frame, it's going to make the interface quite laggy

but in this case I'm just thinking that since I already have a very slowly rendering pdc, I would love to know what it is that happens if I change it in some ways: does it improve the rendering speed or not

[edgy flame]

so, the situation is that we have a large (screen-width) PDC in a scrolllayer, and when the PDC scrolls onto screen, the framerate of scrolling visibly drops. the questions are:

• would a bitmap scroll smoother here?
• is the performance hit because a PDC is on screen at all, or would a simpler PDC draw faster?
• if it depends on the PDC, is the performance hit something we can check and warn for in pdc_tool?
  @azure goblet

[strong ginkgo]

I'd also be interested if there's a nice way to get detailed performance information (~cpu cycles), just for battery-life-optimization purposes.

[wooden wedge]

that's probably a request for firmware/emulator/{p,r}ebble_tool and not for pdc_tool

[azure goblet]

as for profiling, there are is rudmentary support via the "hidden" API calls around __profiler_init, __profiler_start, __profiler_stop , __profiler_print_stats , but I am not sure they still work

[south edge]

Katharine

[bobby-assistant:main] 1 new commit

5b37315 Use a bitmap about pony, it's much faster. - Katharine

[azure goblet]

In most cases, a Bitmap will render much faster than a PDC, especially when blitting bitmap without blending vs. rendering PDC with anti-aliasing.

There are certainly heuristics in the complexity of the PDC that could be applied, but at runtime, there might be more that's going on – including changes in the algorthims.

We have a simialr situation with plain font rendering, which is awefully slow. Any bitmap (even one for just one glyph) should be faster than Pebble's font rendering. For PinyWings, I gained an increasing in several(!) FPS just by rendering the statusbar clock myself – instead using Pebble's built-in clock rendering.

(reminds me of the the 2 FPS speed gain in fastdoom by not rendering the status bar precentage… https://fabiensanglard.net/fastdoom/)

There could be A LOT of speed gain in the firmware itself - espacially when it comes to scrolling, simply by re-using pre-existing pixels across frames instead of re-rendering the entire screen on each frame.

I am stating this last bit since an initial response might be "why not rasterizing the PDC at runtime and treat it as a bitmap thereafter".

If you are interested, I can extend pdc_tool to emit a transparent PNG so the same input/tool path can be used to alternatively use bitmaps (funnily, this is not as trivial as it sounds as Pebble itself cannot render a PDC onto a transparent background so I'd need to do a two-pass render against black and white…)

[south edge]

Katharine

[bobby-assistant:main] 1 new commit

74c8eab Actually remove the removed file from package.j... - Katharine

[edgy flame]

for this specific use case, it sounds like the answer is just to use bitmaps if we are not actually using vector animation or smear effects

[karmic lion]

(This has in fact already happened)

[edgy flame]

thank you for the super in depth answer!

[azure goblet]

What PDC does give you is lower RAM requirement. So for images with large dimensions, this might be a tradeoff worth considering. Especially since in Pebble's model, code space directly compets with available RAM.

[karmic lion]

This is one reason that this image was a pdc in the first place

But a 2 bit palette bitmap will do fine and doesn’t stutter when scrolling.

[azure goblet]

It's really sad… I am very confident that there's a lot of speed gains to have if one were to sit down and do the re-design and work. Both at the drawing level (lines, filling, but also bitmap blitting) but especially when it comes to the layer hierarchy… if only I had the time… 😦

[karmic lion]

now available: 'yes' and 'no' shortcuts on long press

[covert ingot]

Ooh nice

[karmic lion]

the divider between "dictate" and "report conversation" is pebble crimes

oops, did not mean to attach that screenshot

[covert ingot]

I haven't tried yet, when you report the conversation I hope you get

"This incident will be reported" a-la sudo

Crimes?

[karmic lion]

typedef struct {
  char padding[12];
  unsigned separator_index;
} ActionMenuLevelHack;

void action_menu_level_set_separator_index(ActionMenuLevel *level, unsigned index) {
  ActionMenuLevelHack *hack = (ActionMenuLevelHack *)level;
  hack->separator_index = index;
}

as far as I could tell, there isn't actually a way to set it provided.

[karmic lion]

anyway, when the last response is an error, it adds the last prompt to this list as well

I think it's about time to write the Bobby Blog Post and stick a big banner in the appstore.

assuming it is generally working for people who try it.

[south edge]

Katharine

[bobby-assistant:main] 3 new commits

86539c0 Add yes/no/retry quick responses to long press ... - Katharine
653ef1b Keep showing the thinking layer when making a t... - Katharine
4c6f6fc "Try holding select in chat!" - Katharine

Katharine

[bobby-assistant:main] 1 new commit

073c296 Version bump. - Katharine

[karmic lion]

this is the 1.0 candidate, I think

unless anything is particularly broken. on not-Cobble.

blog post time

0.26 incidentally

[wooden wedge]

no website before blog post?

[karmic lion]

I have zero interest in making a website so I guess not

[wooden wedge]

I can make a website real quick I guess

though it is almost midnight

[karmic lion]

I was probably not intending to post the blog post until Monday morning or something

[wooden wedge]

well, I can start out with the gemini output

[karmic lion]

by blog post time I meant "starting to write a blog post" time

[wooden wedge]

right

any preference for ssg?

[karmic lion]

whatever is the least annoying.

my general life preference is to not have ruby in my life but given GitHub, shrug

[covert ingot]

Yeah :/ If it's rebble.io/bobby it should be jekyll

[karmic lion]

it might just be bobby.rebble.io for everyone's sanity.

[wooden wedge]

you can deploy with actions nowadays

[covert ingot]

Makes sense

[wooden wedge]

so you can use any ssg

I do use jekyll every day so that would be my choice but I am open for anything else

[karmic lion]

(more specifically my life preference is to not have to spend time trying to figure out how to have the right version of ruby and the requisite gems)

jekyll is fine

please include a note explaining how I am supposed to convince it to build though

[wooden wedge]

yeah, fair enough

we can specify the ruby version in the gemfile so that it's clear what is currently used to build it

[karmic lion]

unexpected LLM success:

the autosuggested URL https://apps.rebble.io/en_US/application/561960c8a1dd2652af00000d is, in fact, the correct URL to put there

[kind halo]

This is weird lol

[karmic lion]

report the thread!

[edgy flame]

press and hold select

[kind halo]

f- right! report!

[karmic lion]

or just ask it to, that works now.

but pressing the button is easier

[kind halo]

yeah

[karmic lion]

wow that really is it, it didn't even try

I asked it the same question and got:

I can look that up for you. According to Wikipedia, the tongue-twister "How much wood would a woodchuck chuck if a woodchuck could chuck wood?" is used to test a speaker's articulation. The article also mentions that the question is unanswerable, but Cornell University wildlife specialist Richard Thomas determined that a woodchuck would chuck around 700 pounds of wood on a good day.

(bonus: it did not look up the article on Wikipedia, that attribution is a lie)

[kind halo]

gotta love AI

Bobby did not, in fact, wikipedia.

[karmic lion]

I could make it complain about that

it's never lied about that before

(well, not to me)

I wonder whether I should intercept output that includes default_api.something, it does on occasion spit that out instead of actually calling the function

[kind halo]

Bobby did not, in fact, call the function?

[karmic lion]

in this case I would do it silently and tell bobby to do better this time

[kind halo]

in case you haven't realized, I absolutely love the lie detector feature

[karmic lion]

the lie detector is hilarious. just conceptually.

bolt a lie detector on to my computer program, why not

[harsh brook]

lemme know if there's anything I can help with :D

[wooden wedge]

I mean, I am sleeping, so if you want to create a repo and start out, I'm not going to be against it: just post it somewhere so that I can continue in the morning

[coarse cedar]

How could you write that while you are sleeping

[wooden wedge]

well, I am in bed, therefore I am sleeping

that's how it works

[upbeat garden]

factual

[edgy flame]

yeah it's my bedtime therefore im asleep and i'll feel great tomorrow

[wooden wedge]

exactly

[karmic lion]

hurrah I have managed to build rebble.io

victory over jekyll

[harsh brook]

https://github.com/Stubenhocker1399/bobby-website
I have made the basics beginnings where most of the time was spent relearning Jekyll since its been quite a while since I've touched it. 😅

[finite night]

The presumed b-word in question is the same number of letters as the name Bobby, and for some reason my brain found this exceptionally amusing

The end

[hollow rivet]

Easter egg: censor Bobby's name

[edgy flame]

b*bby

[karmic lion]

I have come to the conclusion that automatic language detection works very poorly given only one or two syllables to work with

which makes sense.

I confused poor Bobby

(the speech recognition error didn't help)

I think the flakiness on short messages means I can't ship auto-recognition as the default

but I might ship the new model anyway for higher quality

and make a setting at auth.rebble.io to ignore your language preference and figure it out

[wooden wedge]

wtf is pearl per minute

[karmic lion]

nothing, thus the confusion.

[wooden wedge]

oh

[wooden wedge]

oh, nice, structure!

[coarse cedar]

@karmic lion in the blog post you say "For the time being, Cobble works best with the official Pebble mobile apps."
I think you meant to say Bobby there ^^

[karmic lion]

probably.

[karmic lion]

who wants to try an adventure?

I've just cut all speech recognition over to Google's new recognition model (without letting it try to guess the language)

is it better? worse? does it work at all?

the grandest of adventures

[covert ingot]

I added the watch backgrounds to the screenshots in your blog post

[karmic lion]

yay

I was going to do that but I was lazy.

what did you do for the extremely long screenshots?

[covert ingot]

I was trying to do it a nice fancy way where you could add a css property to an image and get the background, but jekyll makes that hard

The post markdown file ended up having lots of html in it, so for now it's just photoshop

[covert ingot]

Nothing 😁

I'm tempted to make it a scrolling gif

Wait - Are you away from home? Or are you just awake at 4:15?

[karmic lion]

I am just awake at 4:15

because I revamped dictation and then spent two hours dealing with it not working

it works now.

presumably.

give it a shot!

it should be better now.

[azure goblet]

This looks so cool! 🎉

[covert ingot]

It works!

It's definitely better too

One thing - Bobby is a little non obvious when you use it disconnected from the phone. Pressing the middle icon gets you a grey screen but no microphone

[karmic lion]

yeah, I should probably actually handle that

If you turn on error screens for the DictationSession, it displays error messages for every case except that one

that one just silently does nothing

(Well, and presumably returns an error code. That I don't check.)

anyway I should actually go to bed now that my task has taken me three hours longer than forecast

[covert ingot]

You should!

[karmic lion]

if the new asr service is somehow disastrous, you can revert it by changing this line: https://github.com/pebble-dev/rebble-prod-infra-gke/blob/master/cluster/asr.yaml#L64 from asr-test back to asr

the old service is still running.

[covert ingot]

But nice work, I'm gonna test if it performs better when the watch is half covered by a sleeve, that seems to be when it performed badly

[karmic lion]

in case it's a fire while I sleep 😄

[covert ingot]

Haha, thanks

[karmic lion]

I also notice you have it set to en-us while I would've expected you to set it to en-gb

[covert ingot]

So I do!

[karmic lion]

I think the new model is actually entirely agnostic to this

the Universal Speech Model, which is one model for three hundred languages.

but is constrained to assume a given input language because the auto-detection is pretty bad on very short utterances

[harsh brook]

some structure at least for some parts

[harsh brook]

I haven't tried this on previous the previous model but it seems to take room ambient noise happily and try and interpret language, numbers or symbols into it.

Otherwise, it can understand me, but if its better or worse is hard to say. Doesn't immediately feel worse at least! 😅

[wooden wedge]

can it understand me I wonder

[harsh brook]

Oh yeah just send you a collaboration invite on GitHub if you wanna continue on this repo directly.

[wooden wedge]

yeah, I will want to, I'm working through it right now

a bit different icons

idk if better, gemini might have beat me at this game

[harsh brook]

Much nicer than what the llm dreamed up

[harsh brook]

[wooden wedge]

I pushed some stuff, I have to take care of something else

if you wanna, I wanted to write a small piece of html/css to display things within watch frames I added to assets

if not, I'm just gonna come back to it later

[edgy flame]

love love love the currency icon

[wooden wedge]

that's reused quota icon

[edgy flame]

the abacus is really cute too

[edgy flame]

mhm! I love it there too

[wooden wedge]

it's probably better if I had different icons for these

I was thinking of stack of money, but didn't do that yet

[edgy flame]

pebble has a little flag for location that i like a lot - I can check the icon repo. im sure the generic pin is more intuitive though

i use the pin in cobble

[finite night]

sorry if this is a dumb question - can the icons be exported as vectors or otherwise cleanly resized? I like the icons very much, but in this layout it feels like it would be nice for them to be a bit bigger in proportion to the text

[edgy flame]

whichever you think works better

[wooden wedge]

they are vectors

[finite night]

i'd consider making them even just slightly larger, but that's probably down to personal preference 🙂

[wooden wedge]

I don't even know what size they are right now

[finite night]

it's impossible to say

[wooden wedge]

I assume they are smaller than 25x25 since abacus looks bad

[finite night]

yeah, the abacus was the one in particular where i was thinking bigger would be most helpful

[wooden wedge]

https://github.com/Stubenhocker1399/bobby-website/blob/0291444d154fcc34fd6f694cfb9a64b325471553/_sass/components/feature-card.scss#L32

1 rem is 16, so 1,5 is 24

which is a pixel off 25, which means they aren't sharp

[finite night]

Oh, I was joking 😆 nothing is impossible✨

But that does explain the lack of sharpness

[harsh brook]

Looks like the .bundle/config file you've pushed is quite specific for your machine :p

Also I see you removed the blogging functionality; we might wanna keep blog posts around if this page should provide "release notes with pictures" as well

[wooden wedge]

oh I didn't mean to push it

I thought I excluded it, oh well

[wooden wedge]

yeah, I would rather readd it later

I don't think we need it right now for starters, and jekyll makes it rather easy to add it

[harsh brook]

started cobbling something together

[wooden wedge]

that seems nice

I wonder if we could scale the watches down and put them side by side

and have different content in both scrolling through somehow

[harsh brook]

so far ive tried making em dynamically resizable by using aspect ratio and percentages

[wooden wedge]

yeah, that's probably how I would do it too

we can mock the bobby ui in css or just use some recordings of the screen which might be difficult

[harsh brook]

New high score

(I only said "hello," somehow it dictated "call me" instead and Bobby started listing the things it could do instead.)

[wooden wedge]

I wonder if this is a good enough fake

[harsh brook]

things wouldnt be pixel perfect at least

[wooden wedge]

I'm using oswald btw, it's a pretty nice grotesque font

[harsh brook]

did you mean scrolling in the sense of this or it scrolling automatically through some examples

dunno how obvious it would be for users that they can scroll through the screen content 🙈

[wooden wedge]

yes

[harsh brook]

so both?

[wooden wedge]

I mean yes to first

the second is that I don't expect people to know, no

also feel free to request any vectors you may need for this

[harsh brook]

I'm just playing around with it for fun currently

[wooden wedge]

I used cqw/cqh for font size to make sure it's always the size of the container

[harsh brook]

how2 make it scale

Guess we would just not assume someone is browsing the page on their smartwatch browser

[wooden wedge]

I like those new css features

[harsh brook]

speak of the devil lol

[wooden wedge]

we should also make sure the time is set to the current time

[harsh brook]

currently does so in my version at least yea

I didnt have too much success with cqw/cqh but possibly screwed up using it

[wooden wedge]

you need to set a thing on the parent element

[harsh brook]

probably targeting the wrong container

[wooden wedge]

otherwise it won't know what the parent is

it's also probably wise to set font size to something in that root element and then use em sizes within

[wooden wedge]

https://developer.mozilla.org/en-US/docs/Web/CSS/container-type

[harsh brook]

Well if it works fine in your version already then you could just push what you got, including the google font header adjustments n co

[wooden wedge]

I will in a second I guess

[harsh brook]

absolutely no rush

[wooden wedge]

I haven't actually listened to myself and done this properly

[harsh brook]

Like I said, I was just toying around with things

o I see lol

[wooden wedge]

I was also toying around with things

[harsh brook]

🎉

guess one svg that we're gonna need is a nice bobby for the header

[wooden wedge]

yeah, I don't know what that is yet

[harsh brook]

in the reigns of this or better

[wooden wedge]

I think it might be better to use watches there

[wooden wedge]

how bad is using tomorrow everywhere

it does feel like a very pebble font, but it might be a touch much

I pushed what I got

https://tholman.com/cursor-effects/ rainbow or fairy dust?

this is the important part of the website

[edgy flame]

i really don't like cursor effects.. they feel distracting

[harsh brook]

rainbow feels a bit heavy, the fairy dust is a bit more subtle

[wooden wedge]

hmmm

how about a custom cursor then

[harsh brook]

my main concern with using side wide webfonts is them potentially causing layout shifts as they load in, wheras using them in only the pebble frames would keep those to a minimum.

[wooden wedge]

hmm

we can't really avoid layout shift because the Bobby text will be a custom font too

[covert ingot]

Animated Bobby

Why do I feel like this is swaying towards windows 95 vibes

[wooden wedge]

yeah, I am approaching this with way too 90s website vibes

[weary ether]

@karmic lion I do appreciate the new dictation model. Seems to be pretty absent of punctuation. Even though punctuation is nice, the previous model was getting out of hand and just putting it everywhere.

[sick apex]

I think Note To Self broke due to the new model adding punctuation in places, I wonder if it'll work again now

Not that I need that app anymore since Bobby does the same thing!

[wooden wedge]

idk if you are familiar with maya arson crimew and her website, but this reminds me a bit of her website cat https://maia.crimew.gay/

[coarse cedar]

I WANTED TO MENTION THAT ONE

[covert ingot]

Ah nice, I have seen something like that before

Though I dont' think that site in particular

Not suggesting we use it here, but https://alexbsoft.github.io/win95.css/

[wooden wedge]

it's called win95, yet it looks nothing like win95

[covert ingot]

It definitely looks a bit like windows95

[wooden wedge]

I guess one color cards approach the win95 look

they didn't have the gradient technology for win95

that took at least 3 years of development

[covert ingot]

"All" you'd have to do is redraw all of these: https://github.com/adryd325/oneko.js/blob/main/oneko.gif

[harsh brook]

I've pushed my toy-component with an attempt of calculating pixels percentages based on the container width/height. Might be over the top but it's just an idea for a potential approach of how to size mock things. 😅
I did not, however, actually make sure that the things are pixel perfect based on screenshots.

[wooden wedge]

we aren't really aiming for pixel perfect

do you need any vectors?

I think we should have a bit of js that switches between a few screens and scrolls through them

[harsh brook]

Guess it all depends on what we wanna throw into this

advertising the failure screen is probably not one of those things

[wooden wedge]

I mean, it's a cool feature

[harsh brook]

lie detector is funny

[wooden wedge]

also isn't that design used for other things?

I don't actually know anymore, a lot was moved to number display

[harsh brook]

while loading for sure, not sure otherwise

[wooden wedge]

oh yeah, that's true

anyway, I will leave that component up for you, I will try and figure out what I want the image to be

[karmic lion]

Confirmations other than setting a timer still use it

[harsh brook]

speaking of pixel perfect- I am a a bit confused about the skeleton images, aren't both screens supposed to be 144x168? How come they have such differing aspect ratios 🤔

[wooden wedge]

yeah, it's not right, I should modify the images to have the same sized hole

[edgy flame]

hey question

[harsh brook]

Speaking of vectors, a favicon would be a thing thats needed

[edgy flame]

why are we doing all this work instead of just using screenshots

what are we trying to accomplish here

[wooden wedge]

we would rather use videos if possible

[edgy flame]

oooooo

[harsh brook]

one advantage of the mocks is letting user scroll through them & have them be buttery smooth, other than that, very little advantages that I am aware of

[wooden wedge]

I also kinda wanna animate words appearing

[karmic lion]

One could theoretically use the emulator to construct videos

Though either the emulator needs working timeline or source hacking is needed

Significantly less source hacking than what is going on here, mind

[edgy flame]

could fake timeline with emu-insert-pin couldnt we

[karmic lion]

No, timeline needs to actually work or you get stuck at the location consent because it can't boot

This should be an easy fix to pypkjs but I don't think I ever made it

Alternately just edit the source to have your timeline token and skip that step

[wooden wedge]

yeah, that might be a better way to go about it

[karmic lion]

You can skip dictation by adding your prompts here: https://github.com/pebble-dev/bobby-assistant/blob/main/app/resources/text/sample_prompts.txt

Which puts them in the menu on the home screen

(though if you want to reply that doesn't help)

[harsh brook]

@karmic lion it seems to consistently return ] for scratching noises and similar non-word things, maybe something is going wrong with parsing somewhere?

(how I test: have your pebble on a piece of paper, start dication, scratch the paper with your fingernails)

[karmic lion]

It is now a two billion parameter foundation model so it might just be actually doing that

Garbage in, garbage out

[harsh brook]

fair enough

[karmic lion]

There's not much actual parsing being done so I don't know where I could do it wrong

[harsh brook]

I have no reference point on how the previous system handled it, it was just the first thing that came to my mind to test to see how it could break

Other than expecting "missed that, please try again"

[karmic lion]

It probably said that

The new model is much less inclined to return nothing

It can also understand me saying "yes" and "no", which the old one always missed

So that's nice

[harsh brook]

For complete silence it sometimes constructs entire long sentences which is kinda wild

[karmic lion]

Yeahhh

[harsh brook]

Or as silent as my room is

[karmic lion]

This is the thing I don't like about the new approach to asr

Whisper does it too

[harsh brook]

Interesting, it seems to have generated the "I'm not sure if you can hear me very well, but I'm going to try to do my best." again, at least I recall reading something very similar before

[coarse cedar]

I've tried ElevenLabs's scribe model and it's pretty good

[karmic lion]

Time to see whether I can get away with adding an "auto" option and having that actually show up in the menu on the app properly

It looks like at least on android the error behaviour is to show you the language code, which suggests if I send something that isn't a language code it will appear verbatim

[wooden wedge]

hm, is that the same token as in wss query params?

[karmic lion]

Yes

[wooden wedge]

well, I did get to the welcome screen at least

[karmic lion]

I will try getting it to run in an emulator

(I never have so far)

[wooden wedge]

wss://bobby-api.rebble.io/query?prompt=Set%20an%20alarm%20for%2011am.&token=[Redacted]&location=unknown&tzOffset=60&actions=set_reminder,get_reminders,delete_reminder,set_alarm,get_alarm,send_feedback,named_alarms&widgets=weather,timer,number&units=undefined&lang=undefined&version=0.26.0
pkjs> TypeError: undefined is not a function
    at Session.run (./src/pkjs/session.js:61:0)
    at Pebble.handleAppMessage (./src/pkjs/index.js:55:0)

[karmic lion]

Enlightening

Oops, wrong reply

[wooden wedge]

it is not a function apparently

well, cobble works better than this

[karmic lion]

🎉

well I am stuck using the rebble tool, which apparently in yet another thing about it that doesn't work cannot construct a websocket. or report errors correctly. so that's helpful.

¯_(ツ)_/¯

[harsh brook]

Bobby refused to set a 20 minute alarm after a 15 minute one as apparently it overlaps with the 15 minute one.

Reported, if it helps

[karmic lion]

I saw

it actually tried setting two fifteen minute timers

actually I lied

it set two twenty minute timers

it also lied

[harsh brook]

Funny

[karmic lion]

google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'Recognize requests' and limit 'Recognize requests per minute per region' of service 'speech.googleapis.com' for consumer 'project_number:749441128975'.

oh.

[wooden wedge]

hmm, that's not good

[karmic lion]

I have found the error

I made a change that involved making an infinite loop of recognition requests

oops.

[wooden wedge]

is there a repo with rebble logos?

[karmic lion]

not that I'm aware of

there's https://github.com/pebble-dev/iconography/tree/master/hidpi-stickers but

[wooden wedge]

yeah, that's not what I'm looking for

[edgy flame]

thats just the new one for cobble yeah

[karmic lion]

cool, automatic language detection works on iOS and Android

ship it

still has problems on very short entries

but hey

I'm not sure that's avoidable.

(the problem is that the language may not be the expected one)

[wooden wedge]

should the website mention that it's based on gemini?

[karmic lion]

I'm not sure

I'd say yes if Gemini weren't seventeen different products

[wooden wedge]

yeah

[karmic lion]

The app itself does in its consent screen though so may as well

[wooden wedge]

that's true

does that specific gemini have a logo?

[karmic lion]

They all have the same logo I think

[wooden wedge]

confusing

[prime latch]

https://en.m.wikipedia.org/wiki/File:Google_Gemini_logo.svg

[karmic lion]

Or maybe it's this

https://deepmind.google/technologies/gemini/

Or maybe it's that

Everyone's favourite sparkle

Very amused that Google has decided it owns one AI sparkle

[obtuse oriole]

TFW you may as well ask the model what its own logo is since it's too confusing otherwise.

[karmic lion]

Samsung uses four, which is very Samsung of them

Probably shouldn't use the logo, don't want to imply Google endorses it or something

especially since Google has the power to make me extremely sad if I upset them.

[obtuse oriole]

Quick question: Is the new voice recognition engine just for Bobby, or all of Rebble dictation?

[rigid crow]

You mean this isn't going to be the flagship use-case in the next Superbowl ad?

[karmic lion]

all of Rebble dictation

[wooden wedge]

I should make this order make more sense

[karmic lion]

probably wouldn't call out the gemininess as a major feature

[wooden wedge]

I also need a better description for gemini

oh sure

[karmic lion]

also I have shipped automatic language recognition

[wooden wedge]

I might test it in a moment

[karmic lion]

next time your phone checks in with boot (or you rerun it manually) you should get a new "auto" option in the dictation language menu

[edgy flame]

'seamless integration with pebble' is a weird phrase to me

[karmic lion]

then you can confuse bobby by changing language every question

'seamless integration with pebble' is Gemini's doing, I think

[edgy flame]

sounds like it should work with the stock alarm and timer apps and it doesnt

it's a pebble app

it's a really really good app but it's just an app

[karmic lion]

there is no stock timer app

but yes

[edgy flame]

well, as official as the timer app gets

[wooden wedge]

oh wow, the auto language detection is very cool

[wooden wedge]

bobby seemed to fare well with that

[karmic lion]

going to sneak the announcement of this one into the Bobby blog post

[wooden wedge]

seamfull integration with pebble

[karmic lion]

so many seams

[wooden wedge]

enough that some of the seams had to be removed

[karmic lion]

the integration is seamless because Bobby has a tendency to commit crimes and use the open source pebble firmware to do things it's not supposed to be able to

like have a divider in an action menu, which for some reason is not exposed to developers.

[wooden wedge]

having to specify the index of the divider doesn't seem like the best developer experience tbf

[karmic lion]

technically the index of the item after the divider.

but that is true

[wooden wedge]

I do kinda wish bobby had notes of some kind

[karmic lion]

it was once on the todo list

at this point I don't know if I have the code space to build that.

[wooden wedge]

yeah

[karmic lion]

It would be nice though.

I guess I could do a codespace-free implementation of notes as long as I don't give you any way to read them

but I think ideally there would be some app UI

I wonder where they should be stored. Watch? Phone? Server?

[obtuse oriole]

Maybe phone or server, in case you switch watches.

[karmic lion]

currently bobby's biggest problem is that bobby is at least four apps in one

which uses a lot of space

sadly pebble never really did much in the way of IPC

I wonder how much of the reminder stuff I could repurpose for notes

it's basically all already there, except reminders have a timestamp attached and notes do not

[prime latch]

maybe have notes have a timestamp of when it was made if timestamps are required

[karmic lion]

maybe I will have to start actually investigating dynamically loading in the code for the various menus. sounds like a gigantic pain.

[karmic lion]

for that matter, a todo list was once on the todo list

there are event remnants of it

ConversationActionTypeUpdateChecklist exists and has references throughout

no actual checklist, though

[wooden wedge]

I hate the internet sometimes

oh well, firefox is getting a garbage favicon

because it keeps choosing a vector over a pre-made custom pixel perfect icon I made for it

[south edge]

Katharine

[bobby-assistant:emu-hacks] 1 new commit

6f1bb52 Wiki less. - Katharine

[edgy flame]

very bubblegum pink

it's growing on me

[wooden wedge]

I think the bobby text should be even larger

[karmic lion]

yes

[wooden wedge]

well, this doesn't flow very well on mobile

well, I guess I could write a bit more features at the bottom

[karmic lion]

does that copy of the rebble logo also show the current time?

[wooden wedge]

it will tomorrow

I don't think I'm up to spending much more time on this today, since it already is 23:00

[karmic lion]

I don't think it particularly needs to (or even necessarily should), it just seemed like a surprising position for the hands

[wooden wedge]

yeah, that's kinda the reason why they are like that

I feel like I am terrible at explaining why this is good

anyway, time to push this and finish it tomorrow

mmm, yeah

all of my changes are pushed

what is left is basically:

• either recording or mocking up the rest of bobby interface to actually show something off on the watches
• making more and better text
• secret third thing?

[edgy flame]

feel like this takes a minute to get around to what bobby is as an app

proposal: introductory text block at the top of the screen

[wooden wedge]

I added "Rebble AI Assistant" under Bobby later

but yeah, it needs some more clarity

it's kinda why I wanted the watches in the header initially

makes it a bit clearer wtf it is

@harsh brook would you be able to set up a pipeline to build and deploy bobby using github actions?

[edgy flame]

Bobby is a new LLM voice assistant for Pebble Time, Pebble Time Steel, and Pebble 2, included with your Rebble subscription! Bobby can answer questions from Wikipedia, set timers and reminders on your watch, do math with a calculator, and more! <photo of the landing page on a PT>
then you break out the tool cards

[wooden wedge]

would be nice to be able to preview it

[wooden wedge]

where exactly?

under Bobby title?

[edgy flame]

at the very top, under the header

yeahj

[wooden wedge]

sure

[edgy flame]

[ header, big art]
[ big introductory text box] [screenshot of the home page on pebble time]
[tool cards]
[...]

just having a screenshot of the home page goes a long way towards explaining what the app does, i feel. it's pretty intuitive

big mic button

[harsh brook]

yee just didnt want seo to priotize my url in the end hence i decided to hold on that, can do now that its getting into shape

Should still move the repo to rebble group

[karmic lion]

I'm still intending to put a banner on the appstore homepage, so I think the other thing we need is a banner that makes sense in isolation, without the rest of the app description around it.

[wooden wedge]

that feels like it is a lot of content at the top, can we move the artwork lower down somewhere instead?

[karmic lion]

I think probably the big art gets replaced by the home page screenshot, which has a small but also relevant art

[wooden wedge]

yeah

[edgy flame]

up to you! the big thing i want to get at is that the first body content on the page should explain what the app does, first and foremost

[wooden wedge]

yeah

[karmic lion]

fence pony should probably live exclusively in the about page

[harsh brook]

https://stubenhocker1399.github.io/bobby-website/
oh looks like I had enabled it, thought about disabling it and then failed to disable it properly given that this broken version exists

[wooden wedge]

I agree

[karmic lion]

it's less fun hiding at the bottom there if it's also the lead image on the marketing

[harsh brook]

ok now it looks less broken

[wooden wedge]

oops the text is still slightly too large on my screen

[prime latch]

looks ok on mobile Firefox

[harsh brook]

https://jekyllrb.com/docs/continuous-integration/github-actions/

one warning