#Mouse for ingame UI (In Testing)

82 messages · Page 1 of 1 (latest)

wise salmon
#

This one's probably a bigger one than I usually work on,
so, currently, I'm planning to work on it piecemeal

here's some documentation:

I play hollow knight / silksong with mouse&keyboard.
and the menus (inventory, map... etc) are godawful for that control set.
It would be nice if we could navigate it with a mouse

Original Spec (done): #1416743623722139669 message
Testing currently going on: #1416743623722139669 message
bugs/jank list : #1416743623722139669 message
other 'nice-to-have' things: #1416743623722139669 message

Original Specs (complete)
~~ (most-to-least important imo)

  • "free"/reveal mouse, to operate on inventory. Hide when not.
  • happens when menu, or UnityExplorer F7 menu is open?? (happens a different way)
  • if click is bound to attack, don't attack if menu element is clicked (if in menu at all?) - as attack interrupts, and closes the menus.
  • (MVP) clicking left and right arrows (switch screen)
  • clicking tab at top (switch screen)
  • click to equip/unequip crests
  • mouse interact with overmap (mouseover to select region, click to enter, right click to exit)
  • mouse click-and-drag map (move around)
  • mouse wheel for scrolling crests
  • (probably most difficult, so maybe shunt to end) mouse select on zoomed map place to put marker
  • mouse over/click to 'select' in inventory screen (to read descriptions)
    ---- (less important for me) ----
  • mouse wheel for scrolling quests
  • mouse wheel for scrolling hunter's journal
  • mouse hover/select tasks for description
  • mouse hover for crest description
  • mouse hover for select hunter's journal
  • ? mouse click for breaking rosaries / shard bundle?
  • mouse wheel zoom map
  • mouse click "show completed" on tasks page~~
viral sedge
#

this is exactly what we need the current inventory is just too tedious

wise salmon
#

struggling to find how inventory is controlled - documenting what I have so far

DontDestroyOnLoad seems to hold _GameCameras/HudCamera/In-game/Inventory

It seems to contain a Border, which has Arrows (navigation at sides), and the PaneListDisplay (the tabs at the top)

And then Inv/Map/Quests/Tools/Journal which opens the inner screen to different pages

Those pages EACH contain a Cursor, which is your selector thing you move around.
The Cursor has an InventoryCursor class, that has a .currentTarget that can point at things like the Arrows

As for components, it contains EventRegisters for "inventory Cancel", "Open Inventory Map", and "Hero Damaged" (presumably the thing that closes your menu if you get hit) - but they seem to be either events, or listeners? I can't actually tell right now

It also has an InventoryPaneList, which itself has a .paneArrows, that points to the Arrows in the Border

And finally, the inventory has a PlayMaker Finite State Machine (which I haven't been able to pinpoint with https://github.com/nesrak1/FSMExpress/tree/skong (could be my fault) - closest I got is, there seems to a Hollow Knight Silksong\Hollow Knight Silksong_Data\StreamingAssets\aa\StandaloneWindows64\inventory_assets_all.bundle, but I'm not sure it's even the right thing.

And each of the panels also have 2 or 3 FSMs each,
and an InventoryPaneInput class

#

actually, the InventoryPaneInput looks promising.

Thanks, forum rubber ducky

wise salmon
#

Note:
InventoryPaneInput.Update() only runs while the inventory/map/... window is already open
(fine for this mod idea, but not always just opening the menus)

wise salmon
#

Currently, I'm working on the 3rd point (clicking left and right arrows) complete

since that's the minimum viable product/proof of concept imo (I can manually switch 'attack' to different button. as well as temporarily having mouse shown with UnityExplorer)

#

Also, point 1 might be a freebie (I haven't tested it)

#silk-dev-general message

EDIT: Managed to use this, to force the cursor visible while inventory is open.

wise salmon
#

Still working on 3rd point.
So far, I have (very hacky) mouse-over detection

and ability to move the visible cursor. Possibly even select things in individual panes

But still figuring out how the side arrows are actually selected 🤔

dim hedge
#

Most of those features are how it works (kinda) with a controller. Would it be worthwhile looking at how to game treats the controller cursor and then map that to a mouse?

I’m not a dev or anything and you definitely probably already thought of that but I just figured I’d say it just in case

wise salmon
# dim hedge Most of those features are how it works (kinda) with a controller. Would it be w...

how to game treats
gonna assume that's meant to be "how the game treats"

yes and no

You wanna work with the game, and use as many of the things that are re-usable,
to make sense, and make least breaking changes.

But on the other hand, there's no sense in forcing the mouse to be "mouse move right -> shuffle 1 space right". It's the same as the keyboard navigation (janky).

So, in a game explicitly built with directional selection (next element up/down/left/right), it makes sense to peel that layer down for mouse.
Figure out how the things under that layer are addressed.
Add a hitbox of some kind, to detect where the mouse is. (actually, the game already has some. I'm not entirely sure why they're not used, since they're fairly present)
And then on mouse-enters-hitbox, fire that lower-layer element.

I wouldn't actually know all this, if I hadn't gotten as far in trying to figure out how it works, and how best to implement it.

And so far, I've managed to

  • hijack the game's existing 'hitboxes'.
  • make them interact with the mouse
  • found some of how the lower layer works, but not all.

so, at the moment, I have on-mouseover, move the "selection" square... but only the visuals.
currently investigating why that is (seems there are several separate things that manage the actual cursor selection, and I only happened to pick one)

dim hedge
#

Ooooooh okay okay, makin’ progress then!

And yes, that’s what I meant. Sometimes my phone just decides to change previous words, even when they’re spelled correctly

That’s so interesting though. Thanks for the thorough explanation!

wise salmon
#

Well, I'm not sure I'm "doing it right"

But I have managed to get a mouseover on the UI left arrow, to select the left arrow on the inventory pane!

I also have the enabling of viewing the mouse, when inventory opens!

#

Next step:
Mouse-click = ui confirm

dim hedge
#

Sick, this is gonna be one of those, “I don’t play without it” for m&kb players

wise salmon
#

if it sees the light of day 🙏

#

currently, it's in very much baby stages

wise salmon
#

Jank as heck,
and currently only works for one button, on one pane...
but the theory is sound

wise salmon
#

Progress Update:

I've cleaned up my code a bit, and now both clicking left, and right arrows works fine

Down side - I've just noticed that if Unity Explorer UI is off, mouse always resets to the middle of the screen on tab change (less than ideal), and I want to try figure that out next easy fix

#

Next up:
Fixing conflict between 'attack' and mouse-clicking inventory,
causing inventory to close

wise salmon
#

So, it seem that in the controls,

  • the directions are used for movement, as well as menu navigation
  • the jump is UI confirm
  • bind/focus/spell and attack are UI cancel/back
  • a few other buttons that don't matter to my context

... AND THEN, there are additional UI controls from somewhere else?
where MouseClick is allowed on UI panes (menus), and will count as "confirm" (instead of cancel/back, even if bound to bind or attack)

and also the {Enter} button seems to be universally ui confirm, with seemingly no way to bind/unbind it 🤷
same with {Esc}, and UI close (different from cancel/back)

#

Side note:
the esc menu prevents mouseclick='attack' either closing the menu, or attacking in game.
While also respecting the bind = ui back
So maybe exploring how it does it will lead to an easy solution

#

Alternatively, if I could figure out how to break the hard link between "attack", and UI cancel, that would work too
but would be more jank

wise salmon
#

As far as I can tell, attack is also firing in menu.
And the only reason it isn't in game, is cause it's paused

#

I'm gonna temporarily disable it while inventory is open 🤷

wise salmon
#

Ok, also managed to get tab clicking working.
Also, reworked the entire 'click' mechanism to be Unity event based (so I'm not manually doing it)

wise salmon
#

Did a "mini-release" version 0.0.3
#silk-showcase message

It includes the stuff I've done so far (arrows and tabs)

wise salmon
#

Managed to hook the class that's used to desingnate if something is "interactable", and attach my generic mouseover/click class to it

It's working for most of the elements inside inventory/||journal/quests||

||Crests|| and ||rosary bundles|| seem bugged - as if sending "submit" is holding the button down

#

I'd update the released version, if this wasn't such a small bit of code, and so soon

#

wanna see if I can get this hitch worked out, and post another version

#

And as I glance at the FSM, I immediately see the solution

#

Getting ready to push an update

wise salmon
wise salmon
#

known bug:

if you accidentally mouse-over arrows, then an on-screen element, then click, the arrow is triggered.

fixed in the next commit in github source

temporary avoidance: use tab buttons in top right to navigate

#

I've gotten map-clicking-in working "well enough"

#

"clicking-out" is in progress
(I'm taking a break)

wise salmon
#

Didn't mention, but I published 0.1.0 as "map zone clicking"

#

Also, re-prioritised some things

#

Map click-and-dragging has been moved up
so has scrolling crests

#

Also, just done the aforementioned click-n-drag, and will publish

#

Also... added a sort of extra task.
Map zoom in/out with mousewheel

wise salmon
#

hmm, time to sleep I think

wise salmon
wise salmon
#

Morning, and back at it

Did another un-specced task:

When mousing-over the arrows in scrolling windows (crests, journal ...),
the container will scroll in that direction

wise salmon
wise salmon
#

Released 0.1.4 -- mousewheel scrolling

wise salmon
#

Original spec: #1416743623722139669 message
Testing: #1416743623722139669 message
Bugs/Jank: #1416743623722139669 message

Gonna document new-tasks as they come up:

  • Better way of preventing clashing keys https://github.com/Bigfootmech/Silksong_mouse_inventory/issues/21

  • config in general https://github.com/Bigfootmech/Silksong_mouse_inventory/issues/18

    • allow players to adjust scrolling speed etc.
    • allow disable right click = exit
    • allow key rebinding
      • right click
      • mouse side buttons
    • allow disable mouse-scroll

  • Overhaul map control https://github.com/Bigfootmech/Silksong_mouse_inventory/issues/20

    • Reveal map marker selection block in "zoomed-in" / inner map view
      • and be selectable here too
      • mousing over existing pin places a scaled "pin cursor" over it
      • Disable Listen for QuickMap = Cancel in FSM state zoomed in
    • completely rework markers (hold alt to place, don't zoom?)
    • Completely flexible zoom?

  • (nice but unneeded?) mouse wheel scroll tabs

  • option to click once / double click rosaries?

  • control unlock to switch tabs faster

  • remember mouse position between opening and closing inventory? / disable mouse centering on re-enable?
    Done:
    ~~- When mousing-over the arrows in scrolling windows (crests, journal ...), the container will scroll in that direction

  • mouse wheel zoom map

  • Tab to switch markers

  • doubleclick = start placing markers~~

Additionally, we might want compatibility with @noble timber 's mod:
#silk-dev-general message

wise salmon
wise salmon
#

Bugs / Jank:

  • Stop map alphas freezing on early re-transition
  • Certain timers work off of absolute frames. When we now found a Time.unscaledDeltaTime that the game seems to work off of, that might work better.
    • post-scroll control lockout "3 seconds" on tester machine
  • less janky mousewheel interaction with mouse-overing elements that take away control
    • possibly try interrup the FSM that has "control lockout" on other controls?
  • overmap section clicking -- check they don't overlap? / set poly colliders / just remove the whole lot? Change "draw order" if possible? 🤷
    • ||Shellwood > Blasted Steps||
    • ||Hunter's March > Bellheart||
    • ||Sinner's road > Underworks > Citadel||
  • RMB currently hard-bound. Preventing ||Rosary Cannon|| reload if right click = dash
  • Show Complete Quests hitbox only covers the letter itself (rather than key icon) on some machines -- can we make this any bigger? :S
    • try jimmy the collision box twice as big?
  • ||cling-grip (Wall Jump)||, as well as ||Eva Heal|| won't mouseover for whatever reason
  • multiple instances of clicker on some elements
  • possible error on load #1425750833965830166 message

This is also in relation to
Original Spec (done): #1416743623722139669 message
Testing currently going on: #1416743623722139669 message
other 'nice-to-have' things: #1416743623722139669 message

Write-up on Github:
https://github.com/Bigfootmech/Silksong_mouse_inventory/issues

wise salmon
#

0.1.5 DONE (markers), but not released.

Will try to knock last point out

#

should be a small/fast one

wise salmon
#

0.1.6 released

#

it has the marker click

#

kinda wonk, but w/e

#

TIME FOR TESTING!

#

Since all of the initial points have been addressed (even the less relevant ones!)

I'm calling for a 1.0

#

It's a good time to see what issues other people have

#

as well as see if anyone can come up with a better solution to the map -> zoom -> pins control issue :/

tropic pike
#

sorry for the TO, that was automod being dumb, here's the message:

Here are links 0.1.6 - ready to test

https://github.com/Bigfootmech/Silksong_mouse_inventory/releases/tag/0.1.6
https://www.nexusmods.com/hollowknightsilksong/mods/517?tab=files
https://thunderstore.io/c/hollow-knight-silksong/p/Bigfootmech/Silksong_Mouse_Inventory_Use/versions/

@noble timber @austere sequoia @modern pendant @maiden jasper @viral sedge @dim hedge

Please give me your expert opinions

GitHub
Release 0.1.6 - pre-release test · Bigfootmech/Silksong_mouse_inve...

Added a couple things

player markers, or "pins" control. I'm still not happy with it, and want people give me suggestions.
clickable "Show/Hide" Completed Quests button. Al...

Nexus Mods :: Hollow Knight: Silksong
Inventory Mouse Use (Work in Progress)

Clickety click the inventory (WIP)

Thunderstore
Silksong Mouse Inventory Use version history

Version history of Silksong Mouse Inventory Use

wise salmon
#

It's 3am, and I'm wiped, I'll go to sleep, and check back in the morning 😅

dim hedge
#

Is this a call the test out the mod?? 👀

wise salmon
#

it has most of the stuff working... I think

Check it out, and tell me what you think

Anything missing that should be there,
or anything interfering with your controls... etc

dim hedge
#

Cool! I’ll try it out on my next session. Anything I should look out for? (You don’t gotta answer rn, go to bed! lol)

wise salmon
#

Already had 1 report of them not being able to reload rosary cannon, cause I made right click = back out

wise salmon
#

ie: it works ... "ok enough" for me, but it would be good to see how well it works for others

#

@tropic pike thank you again for the save ❤️
you're a legend

#

I'mma go crash out for 8-10 hrs

tropic pike
#

No worries and sorry for that, automod can hallucinate with its automatic spam detection sometimes. Good night :goodknight:!

#

NQN?

#

I think nqn's perms are messed up in the silksong channels zote

limber iris
#

Specifically, by default AutoMod blocks messages that ping "too many" people all at once

maiden jasper
#

If only there was a dedicated skong section of lumafly or scarab or smthn

wise salmon
wise salmon
#

Testing thread: #1425750833965830166 message

Test Results / Fixes required:

Priority:
~~1. (bugfix) Fix ||Rosary Cannon reload|| clash with right-click for xEonlite https://github.com/Bigfootmech/Silksong_mouse_inventory/issues/1

  • 1.1. Make separate class for ||rosary cannon||~~
  1. make ||crests|| selectable https://github.com/Bigfootmech/Silksong_mouse_inventory/issues/2
  • 2.1. button to navigate to that menu
  • 2.2. disable select individual parts
  • 2.3. select whole ||crests||
  • 2.4. (nice) quick-switch ||crests|| with scroll / mouse side buttons

Awareness / look to fix:

  • (bugfix) ||cling-grip? (Wall Jump)||, and ||Eva Heal|| not selectable
  • (bugfix) scroll lock out takes 3 seconds for tester
  • middle mouse button to open crests menu / Show/Hide quests (quick)
  • (mentioned multiple times) Small collision box for Show/Hide quest button :/
  • (thought kicking around) non-interfere keys? (search for player binds, and don't over-bind ?)

Original Spec (done): #1416743623722139669 message
bugs/jank list : #1416743623722139669 message
other 'nice-to-have' things: #1416743623722139669 message

wise salmon
#

Mouse for ingame UI (In Testing)