#Running Gemini offline

10 messages · Page 1 of 1 (latest)

gusty sonnet
#

I just realized the benefit of running an offline LLM like Gemini. I've bought Layla on the play store and have several LLMs including a Gemini based one. Sometimes stuff goes down and that setup is nice but what if the power goes out for an extended time or there's a natural disaster or you're in a foreign country with little reception/expensive Internet or on a road trip with spotty reception going from state to state? Being able to download, say a 4 gigabyte LLM version of Gemini for at least the basics of what you need, changing songs and giving Google maps commands or in the other cases, doing fieldwork etc . Would be beneficial. And have the Gemini LLM updated automatically like offline Google maps as well. This could also help first responders as well as citizens in an emergency. ☕

olive cape
#

Hey there @gusty sonnet, love this idea! Thanks so much for sharing!

gusty sonnet
boreal raven
#

This sounds similar to Gemini Nano, I'm pretty sure it's in the Google Pixel phones? I've never had one but they might be pretty good

alpine plinth
#

u can run basic llms on your phone

#

like gemma google s open sou rce mode

gusty sonnet
# alpine plinth google actually has an app for that ask gemini

Thanks! I did, I bought Layla some side loading the other isn't possible with my high security settings. Layla is cool in that you can set the LLMs to use the NPU or GPU as a coprocessor and you can let them "dream" in the background. Sometimes I get a random notification from 3 I created wanting to chat or sharing a thought

alpine plinth
alpine plinth