#NvEnc immediately crashes on NixOS unstable

25 messages · Page 1 of 1 (latest)

halcyon dirge
#

Hi, we're currently trying to get ALVR to run on our Desktop and use our NVIDIA Graphics card(RTX 2070 Super).

Unfortunately the stream immediately crashes upon the client connecting with the following error messages:
12:49:18.907 [INFO] Using NvEnc encoder
12:49:18.909 [ERROR] Encoder: cu->cuMemcpy2DAsync(&cpy, cuda_dev->stream) failed
12:49:18.909 [ERROR] Encoder: -> CUDA_ERROR_INVALID_VALUE: invalid argument
12:49:18.909 [ERROR] Encoder:
12:49:19.225 [ERROR] error in encoder thread: Failed to transfer Vulkan image to CUDA frame Generic error in an external library

The connection works when we switch to Software Encoding.
CUDA also works on our device for other applications(e.g. KoboldCPP, LMStudio) and the ffmpeg encoder example from the wiki using h264 works without problems(and uses the GPU).

What could cause this and how we could we debug this further? There's an open ticket on nixpkgs and the ALVR repo but nobody from the Nix crowd seems to have an idea how to debug this.
Could someone point is in a general direction of where in the tech stack the issue may lie? We've built it in Debug mode and are planning to add some more verbosity to the NvEnc C++ code to figure out the cause, but a rough direction would be really helpful. ALVR contains a lot of differernt tech layers that are still hard to wrap our head around.

The Nix buildfile looks like it mostly follows the Building from Source guide, although we're planning to double-check it's dependencies with the dependencies of other distros anyway.
The one discrepancy we could find is that they omit the cargo xtask prepare-deps --platform linux step. Unsure how important this step would be, as the build process runs without any errors.

#

Also, how are the DEBUG and _DEBUG variables in the streamer codebase connected to the rust codebase? Do they automatically get enabled if cargo build in debug mode?

halcyon dirge
#

Thanks for the pointer.
Prepares means e.g. resolving paths etc?

dapper narwhal
#

+1 to this issue, been trying to debug this since november last year

#

People mentioned it's a DRM leasing issue

dapper narwhal
#

Something changed for sure, I ran an update and my old driver cuda and driver config weren’t compiling

#

I still on legacy 535, haven’t tested anything yet, Ill mention it here when I do

#

Probably this weekend

umbral summit
#

would be great if someone could put all this nixos stuff on wiki

dapper narwhal
#

specifically with nvidia cards

#

I've heard report of it working with amd

#

and those for which it worked just installed alvr through the package manager.

dapper narwhal
#

I'm on unstable (the latest nixos) and it doesn't work

#

so idk what that guy was talking about

#

To be clear I mean hardware encoding doesn't work. Software encoding is fine

#

wait nvm, I was still using my overlay

#

Actually didn't get the error in encoder thread: Failed to transfer Vulkan image to CUDA frame Generic error in an external library

#

But I just have a black screen

last boughBOT
dapper narwhal
#

I was getting a black screen because updating reset my settings and I was no longer using my open TCP ports