#Looking for mod ideas that make use of graphics tablet

41 messages · Page 1 of 1 (latest)

deep plover
#

After wasting many hours of my precious life, I've finally managed to bridge OpenTabletDriver to Java (in other words, I can finally read input directly from graphics tablet in Java). You can find the bridge here: https://github.com/nahkd123/inking. Please note that you have to build it yourself in order to build the native library and to build it, you need .NET 8 SDK and gcc/msbuild (the .NET runtime is not needed but it will includes with your .NET SDK anyways tiny_potato).

For those that doesn't know what graphics tablets are, you can just... search it up or see https://en.wikipedia.org/wiki/Graphics_tablet.

Right now, I'm looking for some mod ideas that make use of graphics tablets' features, such as pen pressure, pen tilting and pen hovering distance. I've already came up with some ideas in mind (though I might not be able to do them all):

  • A building mod where player can sculpt the terrain with their tablet. Higher pen pressure = larger brush radius/higher strength. Pen tilting may be used to control the direction of sculpting tool.
  • Another building mod where you can build in the air by hovering the pen. Press the button on the pen (or on your keyboard) to build.
  • Play the game with pen and tablet. Because I can read the physical input size of tablet in millimeters, I can keep the sensitivity across all tablets.
  • Yet another building mod where you can assemble your own brush that depends on the tablet inputs. Brushes can be assembled by connecting nodes/modules together, kinda similar to Blender's Geometry Node.
  • Control the WorldEdit brushes' parameters. Not sure how I can do that because I haven't take a look at WorldEdit API yet.

Another thing to keep in mind is that I might not be able to do your mod idea at all, as I'm quite busy nowadays.

cedar rain
#

One could display handwritten text above a player that says it, as a sort of immersive text chat feature, though it would be really Really niche

neat plaza
#

The player is now controlled via drawing walls to slide on

deep plover
#

though im wondering how would you look around. maybe holding 1st pen button to draw a wall and holding 2nd button to move the camera?

summer hinge
#

Spell casting and rune drawing could be cool?

deep plover
# summer hinge Spell casting and rune drawing could be cool?

like drawing 火 will spawn fire around player and stuff like that? i actually thought of this before, but i feel like drawing with mouse is enough. im not sure which special property of graphics tablet can be used in spell casting/rune drawing

neat plaza
#

@deep plover this might be hard to do but probably possible
Get a machine learning API that can recognize words from doodles and make it so drawing a thing will give you that thing

deep plover
#

🤔

neat plaza
#

I believe google has one

deep plover
#

still feels like it can be drawn with mouse. if you can think of something that requires pressing the pen hard enough at specific part that would be nice tiny_potato

#

or i can just make a "tablet driver for mods" and let others decide whether they should interact with tablets

neat plaza
#

Since the algorithm doesn't know what an eye of ender is

#

You will have to draw Minecraft items that exist in real life

deep plover
#

i'll take a look at this, preferably running the ML model locally

deep plover
#

alright, i just came up with another idea:

  • Play Minecraft with only graphics tablet. Tilt the pen to move towards a direction. Tilt more (towards the top of tablet) to sprint. Tap the pen nib on the tablet for left click, press the pen button for right click, press and hold 2nd button on pen (available on some models only, unfortunately) to open a special HUD.
  • The special HUD allows you to open pause menu, select hotbar slots, swap item to offhand or open inventory.
  • I don't know how player can jump, but some tablets have ability to sense the hovering distance, so maybe I can keep player jumping when the pen is far enough from the tablet surface.

this one uses 2 special parts of graphics tablet: hovering distance sensing and pen tilting

#

might be a nice accessibility mod idea, but i honestly don't know. at least pens are more natural to hold comparing to mouse

drifting snow
#

For the building mod, maybe you could get moulberry to let you work with axiom :P

deep plover
#

there is mod api for axiom iirc, but i haven't take a look at it yet

drifting snow
#

Oh an API? Well that's certainly better xd

unkempt zenith
#

like you place it down, then draw a pattern on it to represent some kind of signal processing

#

i wonder if you could pull off automatic detection of logic gate drawings

#

alternatively, although this is probably harder, have it be that while holding a pen you can simply draw the redstone on any full-block surface

#

i think it'd lead to some neat interactions, like having the potential for a lot of signals in one block, as long as you have a very steady hand

deep plover
#

hmm, like drawing the ink directly from player's view?

unkempt zenith
deep plover
#

yeah i guess tablets are going to be optional for this; players can just draw with their mouse, while tablets should only be used when you don't want to switch between playing and drawing mode

cedar rain
#

I know this is a near-dead thread, but in the shower I was thinking about how obnoxious manipulating orientations for blocks can be, especially tech mods. I wonder if a gesture recognition system with a configurator-type item could achieve this more intuitively, such as with items like stairs, pipes, etc

#

Orientation becomes annoying when I have to get a certain orientation in super compact builds, when I maybe only have a single face of a block to work with

#

Just my little idea

#

Pressure sensitivity could be used to "draw" in 3d space though this would no doubt be a nightmare to implement

deep plover
#

tbf the drawing part can also be done for mouse as well, but basically the idea is you draw the direction to place the block at desired orientation?

cedar rain
#

Somewhat

drifting snow
#

I mean I'm often finding myself out of memorable keybinds. Area-based keybinds would be cool, but gesture-based keybinds would be epic too.

deep plover
#

the default wacom driver have this thing called "wheel menu" where you press the pen button to open it