Can I use .env.development as local environment variable file?

[mortal plover]

@tender arch can you tell us more about this? Usually .env.development is a file that's committed to the repo, but these local environment variables are different for each developer so they shouldn't be commited. That's why they live in a .env.local file. Do you want to avoid using a .env.local? or are you asking if a .env.development file will be searched for an environment variable?

[dense jackal]

@mortal plover I have a similar question. I store my NEXT_PUBLIC_CONVEX_URL in .env.development and .env.production respectively. When I run bunx convex dev --env-file .env.development, it does the following: Saved the given --url as NEXT_PUBLIC_CONVEX_URL to .env.local.

I find that confusing and don't really see a benefit for me there.

[mortal plover]

@dense jackal What would the ideal behavior be here, is env.development a file that you commit to the repo? The NEXT_PUBLIC_CONVEX_URL for development will be different for each person developing

and each time e.g. an LLM agent checks out the repo to make a PR on a remote machine somewhere

[dense jackal]

@mortal plover Thank you for your answer 🙏 Personally I would prefer the cli tool to not touch any of my environment files or having the option to disable it at least.

In my use case the .env.development is encrypted with dotenvx and indeed committed to the repo. If one wants to overwrite some local environment variable they can via bash or using a .env.development.local (this one is not committed). The settings in .env.development serve as a baseline configuration for all devs.

I didn’t notice the convex cli writing an env to .env.local and later when i switched to a different convex environment, I had to manually remove the .env.local file again.

[mortal plover]

Personally I would prefer the cli tool to not touch any of my environment files or having the option to disable it at least.
Would you want the npx convex dev command to pause and print out the value, so you can copy it into the file? We can offer an option to disable but for most flows this just wouldn't work, or if there is a NEXT_PUBLIC_CONVEX_URL anywhere it might point to the wrong place. We could fail if it's wrong and ask you to change it?

Ah, would .env.development.local be the right file then? We can change this logic, so if this file exists it's written there.

I didn’t notice the convex cli writing an env to .env.local and later when i switched to a different convex environment, I had to manually remove the .env.local file again.
If you can repro this behavior would be good to see, sounds like something is broken. Whenever you run npx convex dev it should write the NEXT_PUBLIC_CONVEX_URL to .env.local so that your frontend knows how to connect to the convex deployment

[dense jackal]

yes, personally I would prefer it to pause and let me know how i should set up my environment. i always like understanding how the pieces fit together. magic can be confusing sometimes 😄

[dense jackal]

in this case i worked with and switched between multiple convex environments. one of them self hosted even. so maybe you wouldn’t reach that edge case in normal use.

to be clear: the convex cli should not look into any .env file and not write to it (if not necessary). this is the responsibility of other tools like dotenvx in my case. it should just use the provided environment variables. helpful notices are welcome but i would be irritated, if i would need to copy the env var into a specific file in order to satisfy the convex cli, if i could also provide the env var through other means (cli arguments, shell env, nix environment, docker env vars, etc). the cli shouldn’t care where or how it is run.

[mortal plover]

I'm half in agreement, it would be better if CONVEX_DEPLOYMENT=dev:pleasant-cat-602 # team: thomasballinger, project: tutorial-97682 was not stored in .env.local and it might move to a .convex.local or .convex/current-deployoment.json or similar someday. That way the convex CLI would never need to read from .env.local or similar, except indirectly via a tool that sources these.

But provisioning a dev deployment produces a new NEXT_PUBLIC_CONVEX_URL that needs to be available to the frontend, it needs to go somewhere where frontend frameworks like Next.js can find it. This is an environment variable that doesn't exist until you run the command, after which point the convex CLI needs to share it with the frontend somehow. That's part of switching between deployments, e.g. the local and cloud deployments.

ah sorry we got off track here @tender arch! Re your question, yes we can support that, but I'd still love to hear more.

[urban musk]

Hey @mortal plover , first thank you for building convex. Then I think I second everything @dense jackal has been saying. On my side, this .env.local behavior is an issue for 2 reasons. I use Convex for a side project (not yet business 😬) but it means I build "locally" so using .local takes precedence and is somewhat a pain. Also in particular I am working in a monorepo (with other side projects) with a unique .env at the very top that can easy handle the deployment etc. I've pointed towards the right ".env" file but it keeps creating the .env.local at the app level whereas I'd only want the "root" one.
Hope it helps

[mortal plover]

Some of the same questions for you, is there another way you'd like to see this done? The reason this writes to .env.local is that convex needs to tell your frontend where the URL (like a connection string) to connect to.

[urban musk]

If you keep considering them mandatory, I would at least skip the writing when the env vars are loaded and when not, suggest to write them in the --env-file parameter when added or to .env.local as fallback. But I think VITE_CONVEX_URL can actually be a default (and is displayed when launching the command) and CONVEX_DEPLOYMENT is not needed in my case as I haven't logged in and that I plan to use the self-hosted version.

Sorry, actually I've tried again. VITE_CONVEX_URL is not mandatory and so I would not force write it at all. And for CONVEX_DEPLOYMENT, it crashes when the var is not found so if it runs, it means it has been found. So I would not write any of them outside of the first execution.

[mortal plover]

I don't quite follow, are you saying we could write CONVEX_DEPLOYMENT and VITE_CONVEX_URL if CONVEX_DEPLOYMENT is not present in .env.local, but if it is present then don't write VITE_CONVEX_URL?

I'm not sure I understand your pain point either, could you say more about .local taking precedence and being a pain? It sounds like you'd like to hardcode the VITE_CONVEX_URL at the top .env instead of using a project-local .env.local file, sounds workable, and presumably instead of editing the convex dev command could fail and point out when these don't match?

[urban musk]

Sorry if not clear. I am saying only CONVEX_DEPLOYMENT should be added and only for the very first execution of convex dev (when the deployment name is actually created). When it runs successfully it means, the env var was found so I consider things are set up as expected and all chances are that you're adding it in a place where it's duplicated.

[mortal plover]

"first execution" meaning only write it if it's not aleady present? Today the presence of CONVEX_DEPLOYMENT is how we determine whether this is the first execution

[mortal plover]

If only CONVEX_DEPLOYMENT is written, goal would be that it prints out what the expected VITE_CONVEX_URL is and you add it manually?

[urban musk]

Is it how you determine the first execution? The first time I got asked some questions but now if the vars are not present it simply crashes. It feels there is additional logic. Maybe the convex folder?

[urban musk]

Yes

[mortal plover]

sounds like a bug, what steps did you take to see this crash? What does your .env.local file look like to get it?

[mortal plover]

@urban musk If there's not CONVEX_DEPLOYMENT the CLI needs to ask you which project to connect to (if using the Convex Cloud product), agree we can add commands to do this manually. To build a workflow you'd like, sounds like it could be something like

# log in, create or choose an existing project to use, store this information in .env.local
npx convex init

# read from this command's output or from the dashboard the dev URL for the DB
echo "VITE_CONVEX_URL=https://happy-animal-123.convex.dev" > ../.env

# crashes if no CONVEX_DEPLOYMENT found, crashes if a VITE_CONVEX_URL is found that doesn't match the expected one, but doesn't write it
npx convex dev --no-write-env

The next step to avoid touching your .env.local file at all would be to stick CONVEX_DEPLOYMENT in a new local temp file