source-control

[frail condor]

and/or delete depot and make new one

will rider again complain "Unable to Save Settings" ?

[woven sluice]

you're going to get a bit mixed up because the files are still currently marked as read-only, and you'd also have to remove that manually too I guess

[frail condor]

👀 I just avoided the error running Rider with Administrator priviledges

[woven sluice]

[orchid cairn]

I know what .gitignore files are for but what are .ignore files for?

trying to figure something out.

[wide raven]

afaik git uses .gitignore, p4 uses .p4ignore. Perhaps you have some third party program which has created .ignore files, or somebody else has created them by mistake or something?

[orchid cairn]

Unreal engine appears to have autogenerated them when I created the vscode workspace

Still no idea why

[wide raven]

Hmm. what are their contents?

[orchid cairn]

/.git
/.vscode
/Content
/DerivedDataCache
/Intermediate
/Saved

thats the file

[wide raven]

hmm hmm, and it is located where? project root?

[orchid cairn]

yes

Maybe It's some sort of thing for a build tool?

[wide raven]

Very odd. I would say you can just delete it

[orchid cairn]

I don't want to mess anything up

[wide raven]

then cut/paste it to a temp folder for a bit

I'm certain I have no such files in any of our projects

[orchid cairn]

I'd have to test a lot to make Shure it doesn't do anything

I think I'm gonna leave it.

better to leave some things undisturbed.

[wide raven]

Aye whatever. In any case, if it was ignoring something, it wouldnt make sense to ignore Content folder, so I have no idea what that is trying to do

[woven sluice]

maybe some weird thing related to VSCode? (nobody ™️ uses VSCode for unreal, so nobody ™️ would know what that file is for if that were the case)

[orchid cairn]

I use VSCode as I develop on linux

so maybe thats the case

[ornate matrix]

how do you version control a custom editor and the project thats using it ?

do you use one repo for the engine, and one for the project ?

like so:

.
└── ROOT/ <- Git Repo (A)
    ├── Engine/ <- is inside (A)
    │   ├── Binaries/
    │   ├── Build/
    │   ├── Config/
    │   └── ..
    ├── PROJECT1/ <- Git Repo (B)
    │   └── ..

[wind gazelle]

usually just two independent repos (one for engine, another for project)

[ornate matrix]

okay

and how do i select the branch i want to use for my project ?

(or even in my IDE, so i dont see commits past a release)

[wind gazelle]

you make own git branch that is based on a release epic branch/tag

and use it

[ornate matrix]

how

thats the part im missing, how do you decide what branch to use in a project

[wind gazelle]

git branch is just a pointer to a commit

same for tag

[ornate matrix]

doesnt help to setup a project

[wide raven]

For the engine repo, you want to pick a version you want, for example "5.3.2"

For the project repo, you typically have a "main" branch ( a typical pattern is "main", "development" ( as many as you like ), "release" ( often tagged instead of branched ) )

When working on code-heavy projects I would usually branch to do development, and call them "feature-x"

When working with .uasset heavy unreal projects, I rarely do any branching anymore, preferring to make disciplined, granular, iterative changes that retain functionality as much as possible, daily.

[ornate matrix]

i wasnt talking about setting git for a project but how to choose the engine version from a clone

since cloning the repo clones by default all branches and releases version

[wide raven]

checkout the version you want as your starting point from a branch or a tag

[ornate matrix]

i dont use git much so idk what checkout means

in my repo thats forks UE repo, i made a branch thats based on the 5.4 branch
i would like that branch to be the engine version i use locally and inside my project

[wide raven]

well, sounds like you have what you want now

[ornate matrix]

well i still dont know how to chose to only see that branch in my IDE when opening the engine
and i still dont know how to chose that branch for my project

[wide raven]

do you happen to have a git client like sourcetree or fork, which shows you a visual representation of your branch history?

[ornate matrix]

yes i use sourcetree

[wide raven]

so, take a screenshot of the workspace history

[ornate matrix]

im still in progress of cloning the repo tho

got slow wifi here

[ornate matrix]

so its empty

[wide raven]

have you checked out anything yet?

[ornate matrix]

Cloning is in progress

[ornate matrix]

And

[wide raven]

May I ask why you are going into this pretty advanced field of using an unreal source fork engine?

I wouldn't really recommend it if you have 0 experience with version control

[ornate matrix]

i dont have 0 exp in VC

i just dont use every git command, and since now never used checkout

[ornate matrix]

im using a custom engine because i am setting up my workspace now and i know i will have to have a source engine for later

[slender coyote]

Having some trouble with the speed of the perforce server after deleting a user and their shelved files. Network is not an issue. Every command takes a very long time, especially log-in.

Have tried restarting the server, client, hardware with no luck

Are there any good guides or workflows to 'cleaning' the server/client cache and anything which may be slowing it down after any incomplete operations?

Are my log or journal files too big?

[arctic hornet]

that just means you never rotate your journals (which imo is done automatically)... or have a lot of stuff on your server

as for example this is my server for comparison

[slender coyote]

I didn't 'disable' journal rotation, is there a way to enable it?

[arctic hornet]

well, it should do it automatically as iirc it's based on a timer

[slender coyote]

Yes the active depot is about ~80GB (likely ~150GB of total history)

[arctic hornet]

our depot is 400GB+ 🙂 so yeah

either way, you can do manual checkpoint which causes journal rotation

[slender coyote]

Is that an easy process? Can't find a clear way to do it via Perforce docs

[arctic hornet]

iirc it's one command

first of all, check the p4 journals which will give you information about:

• server taking checkpoint
• journal rotation
• journal replay
• checkpoint scheduling

(quoting from p4 docs)

and you should be able to rotate using the p4 admin checkpoint command

it's mostly used to make.. well... checkpoints of your metadata (i.e. the non-versioned stuff)

but it also causes rotation of journals and it's easiest way to do it

[ancient oxide]

Hi, I keep getting Inferno.umap: References /Engine/EngineMaterials/WorldGridMaterial which is not marked for add in source control. (EditorValidator_SourceControl) whenever I try to submit a changelist. It only happens when adding a map to the ShooterMaps game feature plugin folder in Lyra.

Anyone know what's causing it?

[ocean anvil]

only if you set it up that way, otherwise and by default its a manual process.

[arctic hornet]

I don't remember setting it up and it's enabled, though it might have something to do with the fact that I'm running whole cluster for redundancy

so maybe when you have replicas it does it's magic

https://tenor.com/view/the-more-you-know-gif-26317296

[ocean anvil]

🤣 i saw that

[arctic hornet]

I didn't until it was too late

😄

[ocean anvil]

clearly lmao

[arctic hornet]

I blame 2:30 am

[ocean anvil]

its 2:30 AM here too sooooo....

[woven sluice]

i feel like auto rotations might have been a y/n prompt when going through the server setup wizard for linux... can't remember either, goldfish brain

[slender coyote]

I'm fairly confident that I didn't see, otherwise my anxiety-wired brain would have gone down the rabbit hole of DoINeedThis

Rotation was successful. Doing a 'verify' also to see if anything comes up there? Journal is now down to KBs. Will see if performance is restored.

p4 verify -q //...#head,#head

[ornate matrix]

What files and directories should i track so only 1 pc has to build the editor so others can just pull from the repo and use the project using this custom editor directly ?

[wide raven]

Proper way is to use UGS
Another way ( unsupported but works ) is to use Installer builds
Depending on the reach of the source fork engine changes, it is also possible that devs just use the store engine ( if the changes are only needed for certain platforms like mobile, console )

[ornate matrix]

The idea is that:

• i am the dev doing engine modifications
• others devs should pull my compiled engine and use it
• designers uses compiled version of the project

Idk how to define those rules

[ornate matrix]

Thanks, ill look into UGS

[queen sundial]

Is there any reason Epic's own .p4ignore file wouldn't work? I have it setup properly, but it just doesn't work

[wide raven]

try running "p4 set" in the project root to verify the ignore file is correct

[queen sundial]

It's set, and the file is indeed the same name

[wide raven]

looks ok, hmm

sanity check you have "view file extensions" on in the explorer, that the file name is not .p4ignore.txt by mistake?

[queen sundial]

I always have that checked :)

[wide raven]

wise choice. hmm hmm

[queen sundial]

I've spent all day trying to set this up and now that I'm basically done I'm having this issue. I go to add all my project files to the server and it's trying to also add all the Saved and Intermediate stuff despite the ignore file saying it should be ignored

[wide raven]

odd odd... it's worked out of the box for me as far as i recall

[wide raven]

using p4v, restarted it?

[queen sundial]

Yeah. I'll try a full PC reboot

Still not working

[wide raven]

If you look at p4v workspace view, .p4ignore is in project root, correct?

[queen sundial]

yes

[wide raven]

I guess I would keep playing around with it trying to verify it with some test files, something along these lines https://portal.perforce.com/s/article/1282

[queen sundial]

Okay, I tried this .p4ignore file instead and it works https://gist.github.com/stungeye/28e116071c5a7052765d7752968ec56b

I guess the one on this documentation page has something wrong with it https://dev.epicgames.com/documentation/en-us/unreal-engine/using-perforce-as-source-control-for-unreal-engine

[wide raven]

yeah, I was wondering if it could be a syntax thing but I don't know of any ways to verify files like that

[queen sundial]

Nice, I loaded up the project and it automatically connected itself to source control

It's saying I need to create a client with the name of my project/workspace (not sure which one I named them the same thing) to access local files

Oh wait, I think I got it

Yeah, okay. I had used a different workspace name earlier, and it was still trying to use that old one. Just had to update it and it works now

[wide raven]

"p4 info" and "p4 set" are very handly command line calls to verify stuff in this area if there are muddles in future

[queen sundial]

This is entirely on the UE integration side of things, but it's not remembering that I've changed the workspace name when I relaunch the project

Of course, the file can't be changed because perforce has it as read-only

How do I stop this? I'm getting errors about the editor not being able to change stuff in the Intermediate folder, but perforce is forcing it to be read-only

I've checked the allwrite option, but it didn't do anything

I think I got it

[wide raven]

You should not have Intermediate folder versioned

Saved/
Binaries/
Intermediate/
DerivedDataCache/
Developer/
Build/
ArchivedBuilds/

here's a snippet from mine

[queen sundial]

It's not, but that didn't stop it from getting hit with the read-only

Maybe since I didn't have a working ignore file before perforce saw the files? Although I didn't submit anything until I had a working ignore file

Anyway, no errors when I launch the project now. Time for bed

[fleet oriole]

I'm starting to setup UnrealGameSync for our already existing project where we have used P4 for while now. I see that our project is in a depot with depot type "local" like this: //depot/projectfolder and when following documentation it seems like all depots should be of depot type "stream". Is it possible to change our current depots to "stream" or do I have to move the project to a new depot with depot type stream before I continue?

[arctic hornet]

There is no way to switch from local to stream, you have to make new stream depot and then integrate/merge/populate your changes into it

[fleet oriole]

Okay, thank you for the quick answer 👍

[devout walrus]

Not sure if anyone here needs this but a while back when I was working on my website I made a versioning-bump-action for github to handle multiple commits and other edge factors for the most part havn't had issues with it but it's public now should anyone need. https://github.com/Crystalknoct/Version-Bump-Action

[slender coyote]

Thanks again for your recommendation; performance feels snappy again 🙂

I don't know how to set up 'auto rotation' and perforce still scares me, so I'll just manually checkpoint if it gets slow again 😂

[pliant axle]

When using the multi user editing plugin (Default UE plugin) how do I pull the changes if I load up my project without joining the thing everything that I worked on in the server thingy isnt there so I want to pull the changes to be there on my personal project as well.

[fleet oriole]

SOLVED: it was just a permissions issue.
I'm setting up UGS and I am in the middle of running a command in cmd:
C:\Perforce\MyProject>Engine\Build\BatchFiles\RunUAT.bat BuildGraph -Script=Engine/Build/Graph/Examples/BuildEditorAndTools.xml -Target="Submit To Perforce for UGS" -set:EditorTarget=MyProjectEditor -BuildMachine -set:ArchiveStream=//MyProject/Main-Binaries -p4 -submit

The output is this:

Running AutomationTool...
does echo this
Access is denied.

When I'm digging in the scripts it seems it seems like RunUAT.bat call BuildUAT.bat on line 73 and in BuildUAT.bat you can see from cmd output above that it echos the first "does echo this" but it doesn't echo "not echoing this". (check the image to see where I added those)
I see that I don't even have the MyProject/Engine/Programs/Shared directory and I do not have any MyProject/Engine/Programs/UnrealBuildTool/*.cs or **.csproj files either.

Is there anyone here that could say why these files are not here?

[fleet oriole]

Aaaaand it was just permissions that caused the problem....

[civic lotus]

Hey all, is anyone here on Digital Ocean with Perforce Helix Core for Unreal Engine?

Is it normal for my Disk I/O graph to read a constant writing of 30 kB/s, even when I'm not active? It seems my disk space is filling up at a steady 2 GB per 24 hours at the moment even when I've only done a couple of tiny test commits. Also my CPU usage seems off, it seems to be working a lot even when I'm not committing any changes.

[fleet oriole]

I'm setupping UnrealGameSync and when I'm supposed to change the DefaultDepotPath in DeploymentSettings.cs I can't find even find that file. it's supposed to be here:
\Engine\Source\Programs\UnrealGameSync\UnrealGameSync\DeploymentSettings.cs

Does anyone know why it is not there? Lots of other files but not that specific file 🤔

[arctic hornet]

it's entirely possible, as the system itself is doing logs, so does perforce etc...

And CPU usage during commits is absolutely normal, perforce does a lot of computations while comitting, when I'm pushing large changes to perforce (i.e. brand new version of unreal etc...) it can eat all 48 cores of my server's CPU

[arctic hornet]

DeploymentSettings.cs have been removed two or three versions ago, now it's a json file 🙂

or, if you're using horde, it's possible to fetch that config from horde

or to be more precise, the file has moved to UnrealGameSyncShared... but it's now fed from json file which is located at Engine/Source/Programs/UnrealGameSync/UnrealGameSyncShared/Deployment.json

or, if you don't want to modify the json file you can also create new one at Engine/Restricted/NotForLicensees/Source/Programs/UnrealGameSync/UnrealGameSyncShared/Deployment.json as this is a location where internal builds of UGS used at Epic load their internal config, so if you put the file there it will load it from there and you won't have to modify existing files (and since restricted folder is never uploaded to perforce or git, you're fine with it never being overriden)

[fleet oriole]

Wow, thanks a lot for the detailed answer! I really appreciate that! 🤩

[civic lotus]

That makes sense, but these changes I pushed were extremely simple file changes. Like moving a single wall just to test the change system. 😅

I do feel like something ridiculous is happening in my logs, the errors.csv is 2.4 GB

[arctic hornet]

yeeeah, that sounds like a lot of errors that shouldn't be there

check what kind of erros it is

it might be someone random trying to access the server etc... or, there's something broken on your server 😉

[ebon hound]

Hey everyone! From your perspective, which one is easier for designers in your team? Using Git with something like GitHub Desktop or Perforce?

I'm very used to use Git and it's pretty straightforward for me, but I sometimes see my designer friend having hard time with it even with a client application. I've never used Perforce and considering switching, but wanted to ask this first.

[broken loom]

Artists/designers tend to do better with perforce
Git doesn't even have file locking so there's that too
You can consider using UGS with Perforce also
https://dev.epicgames.com/documentation/en-us/unreal-engine/unrealgamesync-ugs?application_version=4.27

If you are forced to stick with Git then at the very least they will need a good visual client, I use and recommend Fork, you can use it for free indefinitely if you can't afford it
https://git-fork.com/

I also specifically dis-recommend Github Desktop and GitKraken

[ebon hound]

Thanks for this! We are about increase our team size so I have an opportunity to make a decision right now. It looks like Perforce will be the choice

[broken loom]

Keep in mind Perforce is only free for up to 5 users
Some people work around this by sharing user accounts and using workspaces as identifiers
But I'm not sure if that violates their terms or not

Oh, and when its no longer free, its expensive (but I don't have numbers)

[ebon hound]

Yes, that’s one of my concerns actually. We could probably afford it but it would be great if there was a public info about pricing.

Thanks for the tips!

[wooden viper]

I'm wondering if someone could tell me the most common C++ static code analysis tool (to check formatting and/or convention problems for the most part) that people use with unreal? I'm looking to add a pre-commit hook to our repo to reject improperly formatted commits.

[full cairn]

Hello guys
I have an issue with installation p4v in arch linux with aur

[wide raven]

I wonder if Rider could be harnessed for this: https://www.jetbrains.com/help/rider/Unreal_Engine__Code_Analysis.html#configure_uht_inspections

[fleet oriole]

Hi again, I located the .json file here as you suggested:
Engine/Source/Programs/UnrealGameSync/UnrealGameSyncShared/Deployment.json.
But in the attached image you can see that the file is empty. I have no experience working with .json files and I can't find any information on how I should change the (previously) DefaultDepotPath in DeploymentSettings.cs to .json format in the Deployment.json.

Could you help me with this or guide me in the right direction please? 🙂

[arctic hornet]

I just woke up few minutes ago and got few errands to run, when I'm back I'll send you how it's supposed to look like 🙂

[fleet oriole]

Thank you dude!! 🥳

[arctic hornet]

ok, I'm here... deployment.json is quite easy:

{
    "DefaultDepotPath": "//depot/InternalTools/UnrealGameSync/bin"
}

well, it can contain more variables like HordeUrl, ApiUrl etc... etc... but I presume you're not dealing with horde right now, so this is plenty to get you started with auto updates from perforce

(obviously change the path to your stream depot etc... this is just what I used before we switched to horde as our configuration and auto-update source for UGS

[fleet oriole]

Thank you, this is perfect! 👍

[civic lotus]

Does anybody know how I should handle the uncontrolled changelist in UE5.2? There are over 30.000 changes in there, mostly related to old path references as this project has been copied/renamed several times now. I've tried fixing up re-directors, but that didn't affect this list at all.

This also caused the errors log on my Digital Ocean / Perforce server to fill up to a whopping 7GB over the course of 2 days... 😵
(it was full of error messages regarding old file paths) How do I clean this up in UE?

[fleet oriole]

Does this look correct to you?

[arctic hornet]

well, it's better to have separate stream for UGS, mostly so you don't mix your binaries with your normal depot

otherwise in theory yes, it would

and honestly, I'm not even sure if UGS does allow for stream depots to be a source for updates

[fleet oriole]

I actually have a separate stream depot which is just:
//UnrealGameSync/UnrealGameSync

I think that's maybe the path I should use

[arctic hornet]

https://dev.epicgames.com/documentation/en-us/unreal-engine/unreal-game-sync-reference-guide-for-unreal-engine#auto-update see this section for how it's supposed to be structured

sadly I don't have any practical examples as we've moved away from perforce for UGS auto-updates

[fleet oriole]

Thanks, I'll take a look 👍

[fleet oriole]

I now ran into this issue. When I choose the workspace (in the "Name:" section) and then the .uproject file (in the "Path:" section) and press Ok, this window pops up 😕 Googling and checking this discord gives me nothing 😕

In Application settings, in the update settings I ticked perforce Server, User and Depot Path seems correct, but when I press Continue it gives me this little window:

The workspace looks like this though and the binaries are not immediately under /UnrealGameSync/Release but under /UnrealGameSync/Release/net6.0-windows/

Could this be a problem maybe?

@arctic hornet Sorry to ping you, I'm quite stuck, do you have any tips regarding the above?

[arctic hornet]

iirc it requires the Release folder to directly contain the files not in the netX.Y-windows subfolder

as described in the link I've sent 🙂

[fleet oriole]

Shit, my bad... Thanks x1000 for helping me again!

[fleet oriole]

Hmm still stuck with the same problem 🤔

I don't see these two files in the Release folder. But I do see UnrealGameSync.exe and UnrealGameSync.pdb

[arctic hornet]

I'm quite sure that's another part of the documentation that is deprecated nowadays

since .config files were generated by old .net framework applications and ionic.zip.reduced was a zip library for again .net framework, nowadays it's a built in feature 🙂

[fleet oriole]

Aa okay! Yeah following tutorials online and documentation and looking at images it really seems like stuff has been changed in many places 😄

[fleet oriole]

Unfortunately I'm still stuck with the same above problems and haven't found anything online that would help me with this exact matter 😕

[long mural]

@fleet oriole

can you show your p4 layout

from within p4v

[teal bone]

Can he what his p4 layout? 🙂

[long mural]

i said it in my head, never typed it out

[teal bone]

😄

[fleet oriole]

This is the UnrealGameSync workspace

That's on the depot side:

This is another depot stream for the engine binaries zipped up:

I have been following Perforces own training tutorial on setting this up from scratch: https://training.perforce.com/learn/course/314/play/1031/property-of-custom-unreal-builds-unreal-game-sync

[wintry goblet]

does anyone know why this keeps happening when I add a fresh project to my GitHub repo?

before it was only the main folders in the content drawer that appear

I've put the git ignore file in and it looks fine so I don't know what the issue is

[woven sluice]

I don't think anyone can figure out what you're asking. What's the problem?

[thick knot]

Not sure if the right channel, but can't find a better option.
I'm trying to locally test Unreal Build Accelerator as a distributed compilation tool. Right now I use 2 computers at home to compile our game and, to distribute the jobs, we use SN-DBS (it's given by Sony within their sdk when you develop for their platform).
My main problem is that I'm unable to find a clear guide on how to install and deploy it in a local network and just for the distributed compilation.
I'm looking at this right now
https://dev.epicgames.com/documentation/en-us/unreal-engine/build-configuration-for-unreal-engine

but aside some flags in the UnrealBuildAccelerator, I can't find a guide that explains how to run it locally etc. I have seen screenshots of a tool that shows how the jobs are being distributed etc, but I have no idea where is that tool or how to enable.

Can someone throw some light for me here please?

Thanks!

I'm guessing you first need to compile this tool

and then enable the flags needed in the build graph?

ok, also found this
https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Programs/Horde/Docs/Tutorials/RemoteCompilation.md

[glacial pecan]

CGZ

[arctic hornet]

If you want to use multiple machines (distributed compile) you will need to have properly set up horde running since it works as governor. If you just want local building using UBA (using UBA even locally on the same machine is still about 10% faster than not using it)... in that case all you need to do is to copy the build configuration from the horde guide and comment out the horde related section

[thick knot]

wait... having UBA in a single machine is faster than not having it? it optimizes the compilation somehow?
I will use it with 2 machines (like I do with sn-dbs now). Ideally, if this works, we might be able to put the system in our build farm. But wanna do a local test first

[arctic hornet]

If we go back to the distributed way... iirc all you have to do is install horde, install horde agents on all the machines in your network, add the machines to a pool in horde and then point the config to the pool and horde url... once that's done if you issue request, horde will copy the UBA agent over and run it... agents will connect to your machine using direct connection etc...

This os really simplified version 🙂

[arctic hornet]

Based on what epic folks have said, indeed, not sure exactly how or why... but it seems like it has more intelligent scheduler and can pack stuff more tightly etc.

[thick knot]

I see. Will try. Thanks a lot for the guidance

[merry verge]

You only need horde if you want to distribute compilation from multiple "hosts". If you want to distribute compilation from just a single host you only need to enable UBA in your BuildConfiguration.xml and then copy the agent to other machines and run it with UbaAgent.exe -host=YourcomputerIP

There's no other setup beyond that unless you are using horde.

[thick knot]

aaah I see. Thanks a lot

[arctic hornet]

I was suggesting use of horde since Bruno mentioned potential use in the buildfarm, and that's something where horde will be necessary... but yes, for single computer manually running the agents is enough 🙂

[pulsar parcel]

At least Horde doesn't require reverse engineering to deploy it anymore (by the looks of it), I'll need to covince work to use it over TC when we actually are using Perforce

[arctic hornet]

Oh yeah, it's decently documented to get basic deployment up and running just fine and if you have any questions or get stuck lemme know, I've helped quite a few people deploy horde already so I know pretty much there's to know about common configuration issues etc.

(and I'm pretty sure other folks here do also know a lot 🙂 )

though such questions would be more suited for #automation

[silver token]

Anyone here run into issues with UGS not triggering scheduled syncs?
We've addressed a weird issue with configuring the sync type (it's no longer a drop down at the top of the dialog but a drop down on the individual project checkboxes).
And we've made sure that the Editor is closed (a common reason I've usually had when a sync doesn't happen, but it results in a fresh log with an error message).

[rustic dagger]

please help, what does this mean?

nvm, this solved it

git config http.version HTTP/1.1

[tame current]

Is Unreal Game Sync only meaningful for projects with custom Engine builds?

If a project uses a release engine do I only need to set up the perforce plugin in unreal?

[woven sluice]

i am 99% certain you need to use a source engine with it but i don't use it yet (or a source engine) either

[arctic hornet]

I'll add that one remaining % 🙂 aa HoJo said, you need to use source build 🙂

[hoary wren]

What are a few lowest-effort-but-still-good approaches I can explore if I want to set up a basic automated build pipeline that will always generate a playable from our latest Perforce changelist? We're a small team under 5 full time folks, so we don't have the most enterprise needs yet.

The idea being that it'd be great for the team to always have the latest build available so that they can play it right away without doing any work on their respective machines.

I heard the TeamCity name mentioned a few times, is that something I might want to look into right now, or is that overkill for my simple needs?

[wide raven]

It's a bit effort to set up but I do highly recommend it ( Jenkins or TeamCity )

Indeed, the idea is usually to have:

1. a nightly build, perhaps uploaded as a zip to google cloud for relatively easy access
2. a monitor build on every change, verifying that each and every change remains packageable ( optimally with a notification to slack, or similiar )

[hoary wren]

Fair to say that TeamCity might be a bit more "cloudy" for those of us who might not want to roll our own Jenkins deployment? Although I'm sure there must be plenty of hosted Jenkins options by now.

[wide raven]

Possibly : I have only used Jenkins, but have heard good things about TeamCity

[hoary wren]

Ok this is going to be horribly naive, but why is testing packaging specifically so high on the list of priorities? Is that something that people get wrong all the time as the team grows?

[wide raven]

Yes, simple mistakes are very easy to make which then can block the whole team from working on the latest commit.

It's technically not a showstopper ( everybody who has pulled the breaking changes can get a non-breaking revision. ) - but it is super annoying.

I find we catch out mistakes in missing files and/or missing changes in submits almost every week with a 10+ team ( though the amount fluctuates a bit, and has lessened over time as people have learned to be more careful )

So the process here goes like this

• person A submits their changes
• autobuilder spots a change and packages
• packaging fails, notifies slack
• person responsible goes "whoops, my bad" and fixes it
• everybody is happy

The main rule is not to submit and immediately head off home, always wait for the autobuilder to show you the greenlight : )

Missing files, broken blueprints, missing c++ headers etc... it's great to catch them out fast

[hoary wren]

That's super helpful, thank you.

[wide raven]

oh yeah, technically you could kick things off with a .bat file and a cron job I guess. But a big part of the advantages are getting notifications for fails and such, which might not be so simple to make with the crude version

[arctic hornet]

Little late to the party, but I'd say Horde is easiest these days, since the basic most setup is really quick to do, it has unreal as it's first (and only) class citizen... and epic also provides a lot of handy build graphs from the start which can be used to run automated testing or do nightly builds etc... (you can set he same scripts in UAT too so you can build the same stuff locally without horde etc...)

The only negative thing about it is that you need to be using perforce and source build of the engine (i.e. not the epic games store version)
So if those are not your limiting factors then horde is (imo) your best bet

if not, then honestly, anything that can execute a script is enough for you, and you can still potentially leverage buildgraph even with softwares like Jenkins or TeamCity

(for example in the studio I work at, we've got buildgraph + horde set up, to run a nightly build that builds the editor and the game, cooks all the assets, packages them, stores our debug symbols on our network storage... runs basic tests for windows and consoles... and also uploads nightly build to epic games store for testing - all the things described are already mostly provided by epic, we've only had to do minor tweaks to customize the stuff for our needs)

[jaunty hornet]

An idiot (not me) reset a computer without seeing whether it had files checked out in Perforce so now I'm stuck with stuff I can't check out

Any way I can force those files to be checked in as admin

[silver token]

I believe that, at best, an admin could forcibly revert them so that they're not checked out any longer by that client.
Unless the changes are shelved the perforce server isn't going to try an reach out to the client to submit it's local copy.
p4 revert with the -C option (not to be confused with the -c option)
https://www.perforce.com/manuals/v24.1/cmdref/Content/CmdRef/p4_revert.html#p4_revert

[jaunty hornet]

Thanks that worked!

[minor ether]

hey how do i push large unreal files to github i saw some dude online he suggested adding this to git attributs
#Added for LFS
UE file types
*.uasset filter=lfs diff=lfs merge=lfs -text
*.umap filter=lfs diff=lfs merge=lfs -text

Raw Content types
*.fbx filter=lfs diff=lfs merge=lfs -text
*.3ds filter=lfs diff=lfs merge=lfs -text
*.psd filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.mp3 filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
*.xcf filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
but it doesnt seem to work on my new project it worked for previous one

[wide raven]

consult documentation I guess
https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage

also note
Every account using Git Large File Storage receives 1 GiB of free storage and 1 GiB a month of free bandwidth. If the bandwidth and storage quotas are not enough, you can choose to purchase an additional quota for Git LFS.

so yeah, the free party is effectively over as far as github is concerned

may use azure instead

[minor ether]

Is azure free?

[pulsar parcel]

yes for up to 5 users

you pay for user 6 and beyond

[thorny temple]

https://www.diversion.dev/?utm_source=ue-marketplace&utm_medium=marketplace

this is great if your a casual team / solo

100gb repo project free for 5 people

[hoary wren]

Fair to say that there isn't much as far as native integration with the UE editor, or is there something pretty solid already out for it?

[thorny temple]

https://www.unrealengine.com/marketplace/en-US/product/diversion?sessionInvalidated=true

[pulsar parcel]

ADO is great for solo teams, this is an unproven git wrapper that has some curious oddities in its ToS

and really, who really wanted a proprietary facade over git?

[modern tulip]

Hi, actually Diversion is built from the ground up, cloud-native, not a git wrapper. We do have an option to import from git, and maybe the wording on the website is confusing - we'll take care of that.

Source: I'm the CTO 🙂

[modern tulip]

Would you mind pointing at the oddities in the ToS? Would love to fix/explain those as well

[pulsar parcel]

9.6 The Services are not intended to be used as storage, backup or archiving services. It is your responsibility to back up your Content and you are responsible for any lost or unrecoverable Content.

so I have to perform my own backups, with a proprietary technology that only runs on cloud

no option to run on prem at all?

[modern tulip]

Just checked other major VCS ToS, and this is pretty standard, you can see for yourself.

We have point-in-time recovery, and on top of that hourly, daily, weekly and monthly back ups. It is as secure as it gets, but as with any software, things are not perfect. If you need extra peace of mind then backing up your content to a storage of your choice is always an option.

As for on-prem, we don't yet offer it but please vote for it on our open roadmap, if it's highly sought after we'll prioritize it higher: https://portal.productboard.com/diversion/1-diversion-public-roadmap/tabs/4-now

[pulsar parcel]

I don't see anything about on-prem there. It's especially hard to trust a new VCS, especially when you barely have any control over your own data. You can also terminate my account for any reason with no notice

[modern tulip]

It's a valid concern, but we would never terminate for no reason. And you can now reach out to me directly with any concerns or issues

And you can suggest new items on the roadmap if it's not there

[teal bone]

Just because you say you wouldn't on discord doesn't mean you actually wouldn't.

It's like adobe saying, "suuuuuuuuuuuure, we don't use your data to train our AIs. Never!"

[merry verge]

It's also a huge concern given the proprietary not self hosted nature of your product.

Git, you can simply take the entire repo somewhere else. Backups are trivial and cover the entire history.

Perforce, it's proprietary but it's trivial to make your own backups of the entire history if you self-host, not to mention lots of existing tools to move data away.

A brand new proprietary VCS that you can't host yourself... data safety and the ability to easily leave to go somewhere else is a huge concern.

[modern tulip]

Would you like to see a takeout feature?

[merry verge]

That would certainly help, yes.

[fleet oriole]

If I want to set a filetype to be -w from -l what command line do I need? The below line only sets it to be -lw but it should remove the -l
p4 reopen -t +w

[fleet oriole]

Or simply, what is the best way of setting all files inside a folder (thousands of files and folders) in the depot to have file attribute -w instead of -l?

[wide raven]

do you want to check them out, or change the permissions aside from control?

I usually just do "p4 edit"

[fleet oriole]

change the permissions

But so that it is kept on the files in the depot

Basically from this:

to this

[wide raven]

and this is on pc, mac, linux?

[fleet oriole]

pc

[wide raven]

icacls
is the command I think

that said, changing stuff that is version controlled past the version control may be a questionable practice - I personally want to see stuff that has changed, but perhaps it makes sense for your cases ( I also revert stuff aafter I have completed a build with custom changes, the idea being to always know if something is 'dirty' )

[fleet oriole]

I definitely see your point

But what is the best practise when having the engine source in perforce available to the team? If someone gets the files from perforce to their workspace and start to build the engine in VS etc it complains that files are set to Read-only.
Is it best if let's say I build the engine and the project on my pc, upload all of those files to perforce and they are locked (with the -l attribute) and then teammembers can get the latest from perforce and the engine and project are "pre-built"? Then no one never needs to rebuild engine and project?

What is like the "best practice" for sharing the engine from source with a team through perforce?

[wide raven]

Sounds like the ignore files may not be working correctly : a cleanly checked out fresh workspace with the unreal engine source fork should be buildable without any checkouts

What files are in question?

[wide raven]

Intermediate/
obj/

Those should be ignored

[fleet oriole]

The engine source was added to perforce and all clients could download those files and build engine locally

[wide raven]

you have a .gitignore / p4ignore file?

[fleet oriole]

Yes

[wide raven]

Let's see the full file

[wide raven]

And next sanity check, is the obj folder nevertheless versioned?

( ignore file looks ok )

[fleet oriole]

That's the folders inside Engine folder

[wide raven]

\Engine\Source\Programs\Shared\EpicGames.Core

[fleet oriole]

I now found an old .p4ignore file that sadly looks like this:

[wide raven]

yarrr

[pulsar parcel]

classic depot is a curious choice in 2024

[wide raven]

I strongly suspect you have all the obj/ folders versioned

[fleet oriole]

Please educate me since I want to learn and do things better

[fleet oriole]

What do you suggest I do now?

[pulsar parcel]

classic depots are old, you typically create stream depots these days

which will have branches that p4 likes to call streams, with at least a mainline

I even did this with pristine engine drops

[fleet oriole]

This depot is a couple of years old now, I just followed a tutorial back then 😅

[pulsar parcel]

that tutorial would've been old hat even a couple of years ago

[fleet oriole]

Okay, damn

[wide raven]

If you haven't got anything critical in there I would consider starting fresh

[pulsar parcel]

The easiest thing has been just copy and pasting Epic's workflows, which is finally documented in one place

[fleet oriole]

Well I have our game project from a couple of years back that the team is constantly working on 😅

[wide raven]

and there are lots of changes to actual source engine code?

[fleet oriole]

aaa, no not yet!

Maybe start fresh with the source engine code?

[pulsar parcel]

https://dev.epicgames.com/community/learning/tutorials/8JYW/setting-up-an-unreal-engine-studio-the-epic-way

This is how Epic does it, we ended up copying a lot of this at my previous studio after seeing an Unreal Fest 2019 talk which covered a lot of this

[wide raven]

yeah, so you should be able to consider it as
a) game
b) source-fork
and keep them separately versioned in any case, I think
but perhaps sswires has more to say

[pulsar parcel]

well you would have

//engine-unreal
//project

engine-unreal is for the pristine engine

ie. a copy of the engine from the licensee P4 or GitHub with zero changes (except for the latter after running Setup.bat and reconciling)

you then initially copy the pristine engine to the game depot, then merge thereafter when performing engine upgrades

the merge conflicts are when your changes conflict with Epic's changes. always make sure to mark up engine changes beginning to end very clearly with comments, and if you cherrypick fixes then mark up with commit/CL you cherrypicked

[wide raven]

I have to say that unless you are certain you need to modify engine, it's a fair bit of extra hassle. If you can just run with store engine life is much easier

[pulsar parcel]

installed builds suck tbh

there is always a time you have to modify engine, even for small things and cherrypicking fixes

a lot of epic tooling is built around being a native build

[fleet oriole]

So this would still be the end result but the pristine engine files are also kept separately in a different depot?

[pulsar parcel]

yes, but ideally the game project is also a stream depot. you have a mainline, but in Epic's workflow, they use release streams pretty often

release streams are used for stabilisation without disrupting development on main

[fleet oriole]

I just wouldn't like to move the game project files to a new stream depot and loose all the years of changelist descriptions etc

[pulsar parcel]

I'm not sure if there's a conversion process but I wouldn't be surprised if there is, but might require your team to recreate their workspaces

But not being able to branch for releases is a pretty serious limitation

[fleet oriole]

Btw, we won't need to upgrade the engine anymore after we now are set to UE5.4.3, only cherrypicking certain fixes if needed

[pulsar parcel]

I would read that Epic way article, since it's pretty much the way to setup P4 for Unreal

Sadly not pinned in the channel

[wide raven]

Epic Official Project Setup Guide
https://dev.epicgames.com/community/learning/tutorials/8JYW/setting-up-an-unreal-engine-studio-the-epic-way

<@&213101288538374145> could we please pin the above link?

[fleet oriole]

Thanks guys, I'm going to look into this more now 👍

Just to confirm, as a big picture, when I have submitted the copy of the engine with zero changes (except for the latter after running Setup.bat and reconciling) to the stream depot, then team members can basically get latest > GenerateProjectFiles.bat > build UE5 > build project
And after that no engine files will need to be submitted, checked-out, added,unless there are manual changes to the engine code from someone in the team (eg a fix is done to a bug in the engine)?

[wide raven]

If you are downloading from Github, then versioning it with Perforce, I would not do anything in between

Then it is 'clean'

Better would be to get perforce access directly I reckon, or just keep the engine versioned with git if you can't do that

Custom licensees get access to our Perforce server.

Talk to your Epic business development manager, or apply at unrealengine.com/license.```

I just keep it in git, since I don't see the point of going over two different versioning processes, but again, perhaps there are other opinions on that flow

[pulsar parcel]

Ideally you don't keep build artifacts in P4 anyway, it's pretty much the reason Epic provides UGS

[wide raven]

If you want it all in one repo

[arctic thicket]

hello i'm trying to use unity vcs / plastic scm for my game project and it uses a writable xlink to another project which holds a code framework. i did some changes on the framework and wanted to commit those from the game project workspace but when i select the framework files i've changed and commit them, i keep getting the error that it doesn't find the repository with that name, any ideas on how to fix this?

[arctic hornet]

You could always make new stream depot and just intrgrate/merge/branch or whatever it's called in perforce into the new stream, those files will start at revision 1 again, but they will be linked to the old files in the depot so you'll ve able to easily see history in p4v etc.... but if you're fine with classic depots you should be fine (I'm just unsure how well do epics modern tool like horde and robomerge handle non-stream depots in case you decide to venture there)

[fleet oriole]

Okay, thanks! I will look into it! 👍

[exotic sundial]

They don't handle classic depots well at all. Horde refused to work with it when I tried it, has to be streams.

[arctic hornet]

Well then, here's your answer directly from Epic.

And also, thank you Ari for all your great work!

[exotic sundial]

You'll get this error, which we got at the start of the Titan project.

[exotic sundial]

My pleasure, and thank you for your help 🤘

[arctic hornet]

I think it's because it should be //Titan/Main in this case

because standard stream depots are two levels deep

(at least)

[exotic sundial]

Correct

We fixed it early on.

[arctic hornet]

ah

[shut hound]

Hey, all. Trying to get a Perforce server set up. When I attempt to log in as a client I get an error message saying "Operator user [name] may not perform operation user-protects." I've given that user superuser permissions, so I'm confused.

Do I need to delete and re-create the user to have superuser permissions from the get-go?

[ocean anvil]

are you certain the user account in question has superuser permissions? doublecheck for any typo's etc.

[shut hound]

Good idea, though if I look at the permissions table that user definitely has 'super' in the first column.

[ocean anvil]

also pro tip to assign your perms to user groups and the groups to users,rather than permissions to users directly. much easier to manage.

[shut hound]

Okay, I deleted and re-created the user and it works as expected now, so 🤷🏻‍♂️

Thanks.

I think what may have happened the first time is that somehow the 'operator' radio button was ticked and subsequently there wasn't an obvious way to change that.

[shut hound]

When I connected from the client machine via Unreal, it threw an error message saying "does not map into this project's directory", BUT Version Control has a green checkmark and it appears I can view changelists.

[tacit forge]

Does anyone know how to make perforce check out action asynchronous in editor just like how it behaves in other IDE. We have a bad p4 authentication environment, checking out an asset takes about 30s to just waiting for authentication. So is it possible to let the editor just try to check out and tell me when it failed even if modify the engine code to achieve this?

[merry verge]

no

you can mark files as writable if you don't care about obtaining a lock and then check it out later (or separately). Or make a plugin to do it for you.

But there's no functionality to try to run p4 checkouts in the background out of the box.

[arctic hornet]

You could also go to perforce (p4v) directly and manually check that thing out

and you could potentially tweak the PerforceSourceControl module, but I'm not even sure where to start

[woven sluice]

30 second ping time is................ wild

[cyan vessel]

Has anyone used this new Diversion version control? Looks like it's some new startup that has built something new from scratch specifically for game devs. https://www.diversion.dev/

Was thinking about giving it a shot to see if it works well. Already using git lfs and azure devops, but their GUI is really nice

[arctic hornet]

not really a fan of their terms of service, so I haven't tried, but few people here were toying with it iirc, so better wait their results
(and personally speaking, I wouldn't decide version control software based on the UI it uses but rather what feature it offers, how well it scales, how secure it is etc.)

These are my pain-points with their ToS - this was originally summary for a friend of mine so it's not a full breakdown etc...

It's mostly these sections that are little scarry, although expected with any cloud service:

9.4       Company has the right but not the obligation to monitor and remove all Content provided by users.

i.e. they are able to monitor and scan your content and remove it if it violates copyright etc... which obviously mean your stuff is not private - this is a deal breaker for any company wanting to use this service to make some proprietary tech that would require any sort of NDA

9.6       The Services are not intended to be used as storage, backup or archiving services. It is your responsibility to back up your Content and you are responsible for any lost or unrecoverable Content.

This one is kinda funny, yes Version control is not a backup... and yes... this only covers their ass in case their servers go to silicon heaven... but still it's like "yeah, we might or might not loose your version control history"

We may terminate or suspend your Account and bar access to Service immediately, without prior notice or liability, under our sole discretion, for any reason whatsoever and without limitation, including but not limited to a breach of the Agreements.

Again, standard clause for ToS, but for important service such as VCS being suddenly cut off from them is another no-go imo

We may amend Terms at any time by posting the amended terms on this site. It is your responsibility to review these Terms periodically.

Your continued use of the Platform following the posting of revised Terms means that you accept and agree to the changes. You are expected to check this page frequently so you are aware of any changes, as they are binding on you.

By continuing to access or use our Service after any revisions become effective, you agree to be bound by the revised terms. If you do not agree to the new terms, you are no longer authorized to use Service.

TL;DR: "We can change these terms at any point in time without prior notice and by not noticing and continuing to use our service you automatically accept them

and that sums up the biggest issues for me with this specific service

[arctic hornet]

Sounds like you have something wrong with your perforce cluster, since even with LDAP based authentication (or HAS) I've never seen 30 second ping nor RTT
main cause might be that the server is too far away from your location and your ISP doesn't have good enough peering to that location - one way to at least partially mitigate this would be to spin up p4proxy or just p4broker closer to your location etc... but it's really hard to say what is wrong without knowing your setup

[woven sluice]

My setup is great. I was reacting to Jenkins Gage's message 😄

[arctic hornet]

my bad, I mentioned wrong person

the point still stands, if there's this large of a latency there's probably something wrong with the cluster... or... network connection between the server and user

[teal bone]

If they change the ToS they should be legally obliged to tell you about the changes and have you accept them again, tbh.

Including a breakdown of what actually changed.

[modern tulip]

Hi, first, thank you for your feedback 🙏🤓 I appreciate your insights and the time you’ve put into raising these items. You’re right about some points however, most of the language included in our ToS is pretty standard for SaaS. For example, did you know that Discord can ban you without any cause or prior notice? This is something included in their ToS and for many other VCSs. One of the biggest VCSs blocked their own founder for a while, and FOSS project owners get blocked out of their own projects often. With our team you have someone to talk to, we want to grow while listening to the community’s needs. Our customers make use of this often when they have questions - we haven’t banned anyone so far.

Regarding the NDA part - I’m not a lawyer but usually an NDA means one party has access to the data of the other party (which is what the clause clarifies), and agrees not to disclose it to a third party. If we didn’t have access to the data, an NDA would not be needed in the first place. So there’s no issue with proprietary data and we are able to sign mutual NDAs.

Regarding the backup clause, this is also standard, but we do have point-in-time, hourly, daily, weekly and monthly backups. If this is a blocker for you, please vote for it on our public roadmap (Takeout feature on the Later tab): https://portal.productboard.com/diversion/1-diversion-public-roadmap/tabs/3-later

Regarding some other points, including the amendment notice - I sent this to our lawyers - we’ll see what we can improve there, expect it to be updated soon.

I understand your skepticism trusting a new player in the VCS market. I recommend initiating a pilot project. You can assess the quality and reliability of the service firsthand before bringing in your core projects. People are still posting issues and questions for VCSs that are decades old so obviously we will not be perfect, but we’re working hard on improving in fast iterations 🏃

[teal bone]

If we didn’t have access to the data, an NDA would not be needed in the first place.
Your ToS literally says you can look at anyone's data, so you do have access...

[arctic hornet]

As I've mentioned in my message I understand that a lot of stuff is common for any service of SaaS to limit liability etc. And it's nothing against your service in general, it's just my distrust in SaaS in general, since for example in my non-game-dev dayjob we work on government related projects and for this reason alone we wouldn't be able to use any service that's not either under our direct control, or encrypted so that the service owner cannot look through our data. Which your ToS explicitly allows you to do.

And also I don't really want to sign any NDAs with my VCS provider, I would like the stuff that I store in my repos to be ideally fully opaque to you (although I understand that in some reagards it's not exactly possible without major changes)

And regarding bans without any cause/prior notice - well, if my discord goes it goes, I don't loose anything valueable.... if my VCS provider just nukes my account and repositories, I loose all my project history... which is not really something that many companies will want to risk

though I am unfamiliar with your VSC storage, since for example with base git I have full snapshot locally so as long as I have my .git folder I have whole history with me etc... so it doesn't matter if github removes my repository for example because of DMCA

in that case I've lost online outlet for my repository, but I still have everything locally (not speaking about git-lfs right now, but about base git)

[modern tulip]

Most features will not be possible without access to the data - for example merging file contents, the backend needs access for that, just like any other managed VCS. I understand that might not fit government needs at this point. I hope we can cater to governments in the future with an on-prem offering or some other solution that satisfies the security requirements

[arctic hornet]

if you'll offer on-premise solution then it's something worth looking into for not only governmental space but also corporations, I gave government contracts as an example since that's the sphere I am familiar with, but I can't imagine rockstar hosting GTA VI source code on some SaaS service for the same privacy related reasons (given as an example).

honestly, that's why big and small studios (and even corporations outside game-dev) pay for perforce... it's fully private, it can run in DMZ, it never calls home and handles any type of content you throw at it with relative ease. and it's got ton of customizability (which might seem complex for many beginners, but it helps a lot). etc...

biggest factor for many is the absolute privacy

[merry verge]

The primary issue for me (even with takeout) is a user's control over their own data, and I don't mean privacy. Takeout helps assuage some concerns (primarily an attempt at lock-in) but the term to me usually means a one-time thing you do when trying to migrate providers.
It doesn't help the case where you disappear tomorrow, or if you lose customer data. I might have an up to date local copy, but it doesn't include the full history.

If GitHub goes down tomorrow, I will still have the entire local history of every repo I have downloaded. If perforce disappears as a company the data is still on my own servers and is still accessible via their software for as long as I still have it (and if I lose that data it's my own fault).

You are limiting your own liability with those terms - and sure, that's perfectly standard - but users of your service are currently unable to limit risk for themselves, and as a result your service becomes a risk. Offering backups on your end is nice, but there's a reason the "3-2-1" backup rule exists: trusting any one provider for backups is a recipe for disaster.

To get anywhere near close to that with a proprietary cloud-hosted system you don't just need to offer backups or takeout, but user-downloadable backups that themselves can act as takeout. This is possible in one way or another with your competitors, but it does not seem to be possible with your service.

I think your service is interesting for those that aren't in the "professional" space but it's hard for me to seriously consider a service with no form of self-service backup strategy.

[modern tulip]

Yeah I agree, some people asked for it via different channels and when visiting our booth at devcom. Others like the simplicity though of not having to setup your own servers and rather being able to start working in 2 minutes with the UE plugin. It's not trivial covering both scenarios though, unfortunately

[modern tulip]

Can you please explain the difference between takeout and user-downloadable backups? Maybe we're talking about the same thing

[merry verge]

It may be, but you have no details on what "takeout" means.
What I mean is a full API for downloading takeout so that backups can be automated. On-prem sort of fills this too since you can just backup the entire set of data assuming there doesn't need to be a persistent connection to a license server (which would be a problem) since you could just restore that data and pull from the on-prem server yourself.

I don't think cloud-hosted services are inherently bad (perforce/assembla even offer that), just that such a service isn't worth considering if the TOS explicitly tells you that it isn't a backup service and then provides no way to create backups yourself.

[teal bone]

I take it there isn't a way to download your data, including the entire revision history?

[arctic hornet]

Also @modern tulip if I can, I'd like to ask who's the data holder since we're already speaking about privacy etc., the ToS and Privacy policy always mentions DIVERSION COMPANY, INC. without any further details (like company id, where the company is based etc...) from your LinkedIn the company seems to be based in San Francisco, but California secretary of state doesn't have such entity in their registry. Upon further investigation there is DIVERSION ISRAEL LTD (which matches the Y combinator profile based on the address) but obviously that is not the same legal entity. So I'm kinda confused in that regard.

[modern tulip]

That was the intention with takeout, yes, I'll add some more to the description

[modern tulip]

It's a Delaware company, you can look it up in the state's public records

[arctic hornet]

Thank you for that, I've been unable to find proper state to look at.

[fiery relic]

Hi, I am not familiar with git lfs, and I am wondering how it works, for example, as an example say I have "*.png filter=lfs diff=lfs merge=lfs -text" in my .gitattributes file. And I ad da png file to my repo. When I commit and push the changes does it get uploaded to my remote origin, or is it just stored in the .git/lfs directory on my local pc?

[proven violet]

if I make a bp project c++ and add a class will I need to worry about someone else being able to pull and open project

[woven sluice]

they will need to be able to compile DLLs also, yes. or you have to set up a way to give them DLLs every time the code updates.

[proven violet]

So If I add to git hub they'll need to build in vs also

[woven sluice]

yes, that's the first option

[proven violet]

Or I include binaries?

[woven sluice]

that's one way to do the second option (kind of ugly and wasteful with git IMO)

[proven violet]

Wym wasteful

[woven sluice]

just can be a lot of binary data to commit... but if you're barely doing any c++ it might not matter

[proven violet]

eh, I can't even figure out these #include errors again. Fixed it once and just keeps messing up

[full cairn]

Hello guys
Have any good resources for working with Perforce?
We have recently switched from Git to Perforce

[pulsar parcel]

yeah the top pin

tells you exactly how Epic recommends doing it

[hoary wren]

I noticed that sometimes I do a rename of an event or a function and my editor doesn't catch something breaking until later, sometimes after I've submitted the changelist. Is there anything I can do to tell the editor to do a full "bp rebuild" to make sure I didn't miss anything accidentally?

[silver token]

Yeah, you can run the CompileAllBlueprints commandlet:
C:\Development\Unreal\UE_5.3\Engine\Binaries\Win64\UnrealEditor-Win64-DebugGame.exe C:\Development\Unreal\SRPG\SRPG.uproject -run=CompileAllBlueprints -log
You'll have to fixup the paths for your particular setup & project name.
There are also other inputs that would allow you to skip directories if you want (like developers).
UGS makes it easy to run this locally before submits.
And if you have automated build machines, this is a great tool to run there after commits.

[hoary wren]

Awesome! Do you happen to know where the log goes for this process? I was hoping to capture it into the terminal I'm running it from but it looks undisposed, I'm assuming it's saving it to a log file somewhere rather than giving me the stdout directly

[silver token]

It's just the regular application log in your project's Saved/Log directory

[hoary wren]

Safe to say that keeping warnings to 0 in one's build is a pretty sensible idea, generally speaking?

[silver token]

That'd be my goal for sure.

[wide raven]

to be more verbose about why this is a good thing

• if you have 0 warnings, you will notice when a new one arises
• even though a warning may not be an issue now, it might be later, or it might be a cause of some anomalous behaviour that hasn't been noticed or traces

[arctic hornet]

It's a good idea in general, since as Karma said, it will alert you when new warnings show up... and to add to that, some warnings can become errors when not building editor... or even crashes while in game but it works in editor etc...

[hoary wren]

Awesome, thank you.

And ok, I'm not bananas for actually thinking that the editor will not always warn you that something is broken until you do a full build externally, right? As in, you can totally change something in one BP and not realize you broke the build until much later?

[silver token]

Yeah, there are some things the Editor will fix up when you open a blueprint but until you save it the blueprint might fail to cook.

[arctic hornet]

Not only blueprints, but any assets can start spewing warnings or errors during cooking... and cooking process in general is the best way (although it takes time) to catch any warnings and errors

since the BuildCookRun process builds the C++ code as it would be for a game (if you are using C++) which will catch weird things like missing ifdefs etc... and it also complains about any assets that are broken or corrupted, including stuff like invalid references in some level you haven't visited in years and is used in the game but you removed assetts for it etc... etc...

in our studio we run nightly builds exactly for this reason - i.e. to catch any outstanding errors/warnings etc... especially because our nightly builds are always built from scratch instead of using incremental building (which can mask a lot of C++ related issues)

ideally I'd like for them to be ran on every changelist pushed, but we don't have compute resources for that

but yeah... TL;DR: BuildCookRun is your best friend for catching weird asset and code related issues

combine that with proper tests and you can catch most of the regressions

[hoary wren]

Is there a baseline set of default flags I can run BuidlCookRun with that will get me started on a good foot? It doesn't have to be perfect right now, but I would love to at least know how to run a basic sanity version of that.

[arctic hornet]

you could do Cook or Package project from within editor directly that's good baseline

BuildCookRun is just the CLI version of that 🙂

[proven violet]

How do I add a folder for github desktop to track

[arctic hornet]

anything under the root of your git repository is automatically tracked (might not bed staged though) unless it's ignored by the .gitignore file
anything outside of the repository root (the folder where you have the .git folder) cannot be added to that repostory unless you copy it into the folder or one of it's subfolders

regarding specific way of doing speciifc operations in Github Desktop application I recommend checking the documentation as it can get you more specific support that I as non-user of git (and especially github for desktop app) can give you... see https://docs.github.com/en/desktop for the documentation.

[pulsar parcel]

https://www.reddit.com/user/CptMarsh/ that's some good astroturfing

[strong sigil]

just makes me want to steer clear of a platform even further when seeing this behaviour

Like this comment doesn't even make sense and totally ignored the top 3 reasons for self hosting that isn't pricing (which is almost the last reason imo)

[golden dragon]

Hey! I'm the community manager at Diversion, which is a position I got since I was one of the most active users, using it for my own solo-indie game, after bad experiences with git and perforce. I still work on it part-time, so not exactly astroturfing 😅😇

I hear you though, I'll be sure to add disclosure to future posts

[golden dragon]

Sorry to hear this makes you want to steer clear of us. If I remember this thread correctly the self-hosting question was weird to me, and I stand behind what I said about self-hosting. I really think that the main reason to do that is pricing, or special security concerns.

[woven sluice]

tbh i don't know why everyone is giving diversion so much flak over its TOS. i read them identically as any other SAAS... if anything it's slightly easier to interpet than most legalese BS, which is not a reason to hate it.

i don't really have any dogs in any fights with all this, although i am rooting for literally any VCS to become a true competitor with perforce, just because i am absolutely appalled by their lack of indie licensing options. git is good at being git, but not good at being a perforce competitor.

(pushing any product while receiving benefits such as employment from that product is definitely astroturfing though 😄 but i'm glad you at least came forward about it and agree to manage it appropriately in the future)

((also i am also not really interested in diversion due to its lack of self hosting - the key thing is that some people want control))

[merry verge]

I don't think the TOS is inherently flawed, it just has issues because of the lack of other features. Having those items on the roadmap is nice but until they exist it's not something I would seriously consider.

[woven sluice]

when my server explodes and i lose all my VCS history data, i want it to be my fault and my fault alone

[merry verge]

Exactly.

[woven sluice]

not everyone is like this, and that's fine, but lots are

[strong sigil]

I work primarily within automotive and aerospace so pricing is usually the least of concern.

But my priorities for self hosting would be

data stays in country (some of my clients have actual government laws prohibiting data residing outside it's country)

Security of IP stays in house and cannot be leaked by third party

Air gapped computers/sites can easily be replicated without needing internet access

Internal design can share data with wider company without a lot of red tape.

Backend can be developed on to hook into JIRA, Teams etc easier than a cloud based solution

[modern tulip]

These are valid security concerns indeed. Though supporting on-premise while offering a very easy to use cloud product is not trivial, probably one of the reasons Google Docs for example is not available for self-hosting - I imagine the backend is very complex but it's so easy to use as an enduser. But I hope we do find a sweet spot, some products did get close like Elasticsearch (but it's still quite an involved process to use their managed service)

[teal bone]

Pretty sure that nothing google does that is online only is that way because of the complexity of it.

It's because they want to keep you in their ecosystem online and collect and sell your data.

Google isn't a search company, it's an ad company. And a data company. And we are the products.

Which I assume is true of most cloud-only companies.

[thin slate]

Perforce won’t let me add files to depot because “file(s) not in client view.” for my unreal engine project. I have a desktop perforce server that I already have a project on for a group game project for one of my classes. The server keeps all of those files in a depot called “depot” because I was new to perforce and I just named it that. After a bit the project grew and I have a full game project stored in there. Since I’m not working on that game project anymore, I’ve started a new project and I wanted to use my already setup perforce desktop server. I’m using Perforce Helix P4V to access the server, its the GUI if that matters. I found it really helpful until I got to the part where you add your workspace files. I created a new depot called ProjectMomentos and created a new workspace with a new folder and copy and pasted the unreal engine project into that workspace folder. The files show up in perforce but their only showing up because I filtered it to “show files not in depot”. Theirs something I’m doing wrong but I can’t figure it out. Whenever I try to add those workspace files, a bunch of warning messages pop up and say this unreal engine asset “file(s) not in client view.” I’ve been trying to fix this for hours but nothing seems to work. It will say “no files opened” “311 warnings reported” before showing all the warnings. If anyone can help, I would seriously appreciate it.

[pulsar parcel]

so is it a stream depot? did you actually setup a stream workspace? workspace mappings won't work with streams

[hoary wren]

Yeah, real bummer about perforce now having real competition at this point, it's good to have at least a couple of products out there pushing each other to be better for their users.

[cyan jay]

FWIW I think Diversion is well placed for small teams and new devs. I've been using perforce for almost 20 years so it would take a lot for me to switch, but if you're a new team looking at the options I think getting started on Diversion looks much simpler. Kinda like the Unity of source control at the moment. Most smaller devs, indies, etc don't really care about where it's hosted, backups, etc.

[teal bone]

Which is bad for Diversion. That's not where the money is.

[cyan jay]

Not necessarily true, depending on overheads. It's a reasonable starting point while they build up their offering. I just don't understand why so much negativity around it.

[teal bone]

I mean, everyone has explained what the negative points are and why it can't be used by any sizeable company.

Nobody is saying johnny random and his friend making a game won't get good use out of it.

But then they have to compete with free perforce setups and such

[cyan jay]

Well I've definitely seen a negative vibe towards it in here over the past few weeks.
Setting up perforce is a huge barrier to entry, but at the same time the experienced users here to help probably wouldn't use Diversion, have not used it, and so can't really recommend it as an option.

[teal bone]

Setting up perforce, even a local perforce, takes about 10 minutes if you read Helix's setup guide.

It's not hard at all.

[vapid hatch]

It's funny you say that, because no one I've ever talked to has said that before you lol

[cyan jay]

It's not hard if you know what you're doing and you have some sysadmin experience. If you're an artist or designer, or you just want to get on with making your game/prototype, then it's hard and hours of work at least.

[teal bone]

I followed the helix setup guide about 2 months ago to set up a self-hosted server and it took about 10 minutes and was very straight forward.

I had no idea what I was doing from a server pov.

[vapid hatch]

I'm glad your experience was good, but speaking from another perspective, most people say the opposite when it comes to setting up Perforce.

[teal bone]

Maybe the guide is new.

Or maybe they didn't look at the helix site for the guide.

Maybe 10m was an exaggeration, but it certainly wasn't hard.

[pulsar parcel]

"most people"

now that's certainly an exaggeration

[woven sluice]

making a server install and run is easy. actually learning how to administrate it over time and handle issues does occupy some time and resources, no need to be silly

in my case i hated the initial setup really really bad but that was because it was also my first foray into linux and it was long enough ago that the install process was a bit worse

[arctic hornet]

yeah, the benefit of SaaS or Managed is that you don't have to deal with any issues and "it just works(tm)" my biggest issue with Diversion was with the privacy and the fact that they reserve the right to change the ToS without any notification and continued use of the service constitutes automatic acceptance - which is actually against the law in EU and there is even legal precedence in US https://casetext.com/case/douglas-v-us-dist-court which says more or less says that parties are not bound by the revised terms of service without notice "a party can’t unilaterally change the terms of a contract; it must obtain the other party’s consent ... such assent can only be inferred after he received proper notice of the proposed changes."

And the fact about privacy imo is what will stop any substantial company from using their service and sadly, that's where the big money is. since you're unlikely to get indie studio that needs 10 TB of space and 1000 seats or something like that.

But yeah, Perfoce is kinda hard to manage if you're not familiar with it, since it's kinda archaic in many ways... but it's super powerful once you know your way around.

[mystic roost]

Hi. I have a project that is located on one server, and we are using Perforce during the project. Now, we are moving to another server. After moving to the new server(i belive we moved perforce to new server) but, when I get the latest updates from the depot to my workspace, the assets inside the project are still referring to the old server. How can I fix this and make them refer to the new location on the new server? The project is identical. we just need to redirect it. Should this be done through Perforce or manually in Unreal Engine? Thanks for any help!

[woven sluice]

Question doesn't make any sense to me at least. You might want to show what's going on

[teal bone]

In the source control settings in UE, I expect.

[silent minnow]

Hello, does anyone know whether it's possible to lock binary files when someone is using them using git? I need the same feature Perforce by default. I don't want anyone accidentally working on a map someone else is already working on

[teal bone]

Git has no lock support at all.

If you want locking, you need to add git lfs.

[silent minnow]

Oh, alright, thank you

[hoary wren]

Fair to say that if I wanted our non-source-building team members to be able to run the game straight out of the perforce folder, we'd want to make sure Binaries\Win64 are in VCS until we've moved off of manual builds?

Also, do we have a channel about building UE? I think this is the closest one to one?

[silver token]

It's doable, but a bit sketchy if you have more than one engineer. Ideal is to use UGS, but if an automated build machine is checking in the Binaries/Win64 directory instead of a human it's a step in the right direction.
#engine-source would probably be closer, but plenty of support for that sort of thing in #cpp too

[near rose]

our artists and programmers would prefer different editor startup levels. from what i can tell, p4 has no way of ignoring individual lines. we obviously need to track DefaultEngine.ini but is there some strategy we can use here? i tried using changelists and that didn't work

[teal sail]

I believe Game/Saved/Config takes priority, so you could ban check-ins from those configs but make locally writeable to allow overrides.

[near rose]

finally came across the setting i was looping for. editor preferences has an option to load the last loaded map. thanks for your suggestion though that is good to know

[tame current]

I searched the channel and don't see a ton of discussion about it but, is there some general consensus on the project borealis git plugin? I'm providing some IT support to a team that consists largely of artists and we're using github/lfs and other than it being expensive so far things have been ok but I fear that without robust locking trouble is brewing, I looked at the repo and the plugin looks like the Real Deal and generally I'm not seeing anything but positive comments about it in this channel. So, generally, did anyone have a particularly negative experience with it? - https://github.com/ProjectBorealis/UEGitPlugin

[arctic hornet]

AFAIK with this plugin your experience will be as good as it can get with git. And I’ve not seen anyone complain about plugin specifically… only sometimes to things that were caused by misconfigured lfs itself…

But at the same time, take my “review” with reserve, since I’m personally not git user (in relation to unreal)

[sick pollen]

does anyone have experience with github desktop and using blueprints because I may have just completely fucked up my project lol I feel ill

I switched branches foolishly before backing up because I didn't expect a ton of issues like this
now I have a ton of merge issues with my code and I don't even remember what these stashed changes are supposed to do

So I was on a branch, changed the branch with stuff. Instead of uploading to that branch again with my changes I switched back to main which is now causing the issues

Very dumb from me to play with github like this and possibly permanently losing hard work, I should have backed up since I lack github experience

but I'm just hoping there's still a way to salvage it..

I don't know the implications of this

I suppose if it somehow lets my still open the blueprints in unreal I could fix it inside of that but i'm worried by commiting i'll do irreversable damage to this progress

that's now in a weird limbo

If I can access the blueprints even if broken I can fix it probably or salvage what i can but I don't know if this will let me or not

alright yeah i think im pretty much fucked nvm

I don't think there's a way I can access the conflicted files

I committed and uh yeah

[pulsar parcel]

uh you don't have Intermediate in your gitignore?

you don't commit those to version control

[sick pollen]

Yeah so I heard 😅

I'll get to making a better github setup for my project

but I wanted to save changes

but i think they're gone

no way to access the blueprints anymore

so i'll have to remake those probably

[pulsar parcel]

the stash is likely a snapshot of everything you had when you switched branches

I'm not sure GH desktop does in that circumstance, it's a pretty bad git UI

[sick pollen]

hmm yeah the stash is now gone anyway

and I don't see the blueprints on my main branch

so I think it's probably a loss

unfortunate thankfully it's only some work

[pulsar parcel]

a lesson to commit and push regularly I suppose

[arctic hornet]

honestly that's a good idea in general even if your VCS is working

pushing small but frequent changes makes it easier to find where stuff went wrong

and also to (selectively) revert that stuff

if you push 100 changed files in one push... it makes it hard to find what went wrong, especially if you know it's somewhere in let's say 5 such commits in a row

[hoary wren]

Is there any solid reading out there I could do on the best way to structure one's p4 depot with CI/CD and art assets in mind? Is there a best practice here that I could learn more about from somewhere?

[arctic hornet]

well with unreal there's only one true structure, especially if you'll want to use tools like Unreal Game Sync and/or Horde etc...

and that is the one that replicates Epic's internal setup, since that's what all the tooling is written for

and thge structure is like this

see the pinned link to an article from Epic that describes everything

and regarding storage of art assets... it depends...

iirc Epic uses combination of regular old depots and new stream depots (legacy and stuff)

but in general you usually also put a folder Art_Source (or whatever) in the root of the workspace (next to the engine and project folders) and import form there, since it's all under workspace root it will always have relative paths and that way you can easily reimport

if you want to have multiple projects you could also do Project/Art_Source or Art_Source/Project etc...

TL;DR: regarding raw art assets there's no real definition or one true way, they're not needed for CI/CD nor day-to-day running of your editor/game , so just put them whereever it suits you best

[hoary wren]

Super helpful, thanks so much 🙏

[hoary wren]

My understanding is that often people will keep the raw art assets in their own stream so that you don't have to sync it down to the machine (whether dev box or ci/cd) unless you actually explicitly want to?

[arctic hornet]

you can have it in separate stream, but that throws the convenience of consistent relative paths out of the window

but if you use/will use Unreal Game Sync, you can setup sync filters which will skip raw assets from syncinng if you don't want them to

and/or will have categories so you can only sync stuff you want/need

or or... you can have them opt-in (as I've said above) but then manually sync only the assets you want directly from p4v for example

[smoky igloo]

Hey guys, what would be the easierst way to setup perforce in VirtualBox (the server).

Just install ubuntu and then perforce or is there a image from them already with the setup?

[arctic hornet]

for basic most setup yes, why'd you want to install inside virtual box though, if I may ask?

[silver token]

Yes, that's what we do. Then there is a virtual stream for artists off the main stream that imports the source art stream.
This resolves the issue klukule referred to since the relative path is baked into the stream and not a separate workspace with it's own root.
It also makes the source content optional without having to deal with UGS sync filters. Since non-artists wouldn't use that virtual stream.

[hoary wren]

When following this particular structure, with Engine + Projects living under the same top root folder, what do you open with Rider when you want to edit one of these projects? Do you open Rider at the very top folder, so that all the projects and the engine are visible, or do you just open one of the sub projects through their respective .uproject?

[silver token]

With Rider I would (almost) always recommend opening from the uproject.
And if not from the uproject, then I'd generate the project specific solutions instead of opening up the UE5 solution that would have all of them
I'd only use the UE5 solution if there's only one game project in that folder.

[queen sundial]

Question: I have Perforce setup and I'm using the in-editor integration to submit changes, do reverts, and all that. If I delete a file is there a way to revert that delete inside the engine, or does that need to be done through P4V?

[arctic hornet]

Sorry for the delay, I’ll mostly echo what Ramius said, as that’s the most convenient way, you could also generate custom sln using UAT that would filter the projects. (Iirc it’s done automatically by Unreal Game Sync too)

So yeah, for rider just open the uproject and for visual studio generate scope limited sln. And full sln is also an option for rider since rider is good at dealing with large solutions, but as said previously, uproject is easier to manage

[arctic hornet]

Sadly unreal doesn’t provide any view for such thing (and honestly neither does perforce unless you’re in depot view and check the “show deleted files”… so yes, you have to do it from perforce

[terse fjord]

Hiya! I'm looking for an old document that I thought used to be in the pinned messages but maybe not? The link led to a page on how to set up Source Control using Github for Unreal Engine and I can't seem to find it now. Is the method obsolete or do people just prefer to use Perforce now?

Thanks in advance for any help!

Edit: Nevermind, it's not here, it's in the general chat pins. Woops!

[arctic hornet]

Glad you found it... though I'll say this: Perforce is first-class citizen in epic's ecosystem since it's used by Epic so all tooling is built around it... and also perforce is sorta industry standard.

But if you want to use git, feel free to use the guide 😉

[terse fjord]

Does Perforce cost money? I'm making my first game so I'm trying to limit costs as much as possible since it'll probably not sell well. ^-^;

[teal bone]

For up to 5 people it's free.

[arctic hornet]

Perforce is free for up to 5 users and 20 workspaces (so virtually up to 20 users/computers at once if you want to share a username)

but you have to self-host it, which is the bad thing

[teal bone]

Beyond that it is ridiculously expensive.

[arctic hornet]

What Daekesh said

🙂

[teal bone]

We were quoted like $10k a year for 15 users.

Upfront.

[arctic hornet]

iirc self-hosted licenses are for a year indeed and there's flat fee per user per month

[teal bone]

Yeah. About $40.

[arctic hornet]

that would be about right... 30-40 USD/whatever I don't remember the curency

but I remember seeing number like this

[terse fjord]

Sorry another newbie question; Can you work in Unreal Engine while Github Desktop is pushing an already committed file to the cloud?

It's looking like it's going to take hours to push the initial commit so I was curious if I could multi task

[modern tulip]

Yes, committed files are stored in .git, so editing your working tree while it's pushing should not interfere with the push

[terse fjord]

[hoary wren]

There's also helix code cloud which is I believe $35ish per user per month, and it's not upfront, but with several caveats.

[cyan viper]

I have reached an unfortunate point where I am getting back to a project after multiple months of not working on it. I’m self hosting perforce and don’t remember how any of it works so while I can connect and still use it through P4V I can’t debug things like the P4 ignore not working. 😦

[hoary wren]

Rider seems to work pretty well from Default.uprojectdirs for me for now, but like you mentioned, we only have one single game for now. No clue what it would do with many games at once, but fortunately we won't have that issue for a while.

Either way the files I see in the Explorer panel seem about the same regardless of how I open the game project, seems like you end up in the same place in the end.

[silver token]

For real production, you (generally) always want to be in that 1:1, game to engine, setup.
For prototypes or tests, you can do X:1.
So at my work, the stream for our game imports the engine but there's another stream that also imports the engine for use by multiple projects. but those projects are all scratch pads for people to mess with unreal. not stuff that would ever ship.

[hoary wren]

Fair enough! I imagine trying to share one modified engine for x games only makes testing everything that much harder, so having one engine per game allows you to not accidentally cause a regression in a game you might not be currently modifying?

[silver token]

Exactly. It also allows the projects to take engine updates on their own schedule (assuming someone's already staged it to a company wide location) instead of trying to coordinate multiple projects to update at the same time.

[earnest helm]

Hail, all! I've managed to get up to the point that I have Horde running and as far as I can tell hooked up correctly for use with Unreal Game Sync. I've gotten a couple of CI builds in horde, and Unreal Game Sync is even showing me the Editor green button (Badge?).

However, sync precompiled binaries is grayed out; I got it to work before when I did the self zip submission stream in P4. Is this option expected to be checkable when using Horde or does it somehow bypass this functionality?

When I try to launch a particular submit number, it will sync okay but then says it needs to build, which is what I would expect if I can't check the sync precompiled binaries option. It does say I am connected to horde, clicking the connected to horde link opens up the correct webpage. I did notice that if I click the Editor badge, it will open up to "https://localhost:3000/blahblah/blah", which maybe is a clue as to what I've done wrong. Manually replacing localhost:3000 with my actual address brings up the expected job, so seems like maybe there is this last configuration somewhere feeding localhost:3000 and that is what is causing the gray out for precompiled binaries as well as the wrong link when clicking the Editor badge? Was hoping someone might be able to help me figure this out.

[arctic hornet]

Hey there, for UGS to work with p4 zipped binaries you have to add few lines to the UntealGameSync.ini so it knows where to look for the things

[//UE4/Main/Samples/Games/ShooterGame/ShooterGame.uproject]
ZippedBinariesPath=//UE4/Dev-Binaries/++UE4+Main-Editor.zip

This is what epic provides in their documentation,so just change stuff here and there to fit what you have

And regarding the localhost:3000 that can be changed by modifying “dashboardUrl” in horde configuration

As that is the variable that’s used for generating links… although it won’t modify existing badges since the URL is stored as a whole inside the database

[earnest helm]

Thanks for the reply! I had zipped binaries working before I switched to Horde, but my understanding is that this is unnecessary if using Horde. Is that incorrect?

I have dashboardUrl defined in the Horde server.json, but that doesn't seem to have adjusted the badge links.

[arctic hornet]

yep yep, existing links are stored as full URls in the database

so you'd have to modify those manually

but new badges will use the new dashboard URL

[earnest helm]

I just had that suspicion and kicked off a new build to check, haha. Thank you! 🙏

[arctic hornet]

and regarding this, it depends on whether you're using horde as the source for precompiled binaries, or P4

if you're using perforce/p4 for your precompiled binaries (PCBs) you still need the ini file thing

and for horde it's kinda annoying to get working especially if you have authentication enabled on horde

[earnest helm]

I do have it using the local horde authentication (set anonymous to Horde in server json).

[arctic hornet]

in that case it should work if your buildgraphs are producing ugs-pcb artifacts... but I don't have any experience with those personally (nor do I know anyone that uses horde for PCBs...) so I can't assist you there sadly.

[earnest helm]

Haha, seemed like you and ss were the most knowledgeable on these subjects, so that's bad news for me 🤣 Build graph is generating pcbs, I get the link to download on the horde page as well as a "download through ugs" downloadable command. I had seen that you had an extra build step for pushing pcbs to Perforce in a random screenshot you had shared, guessing that's to get around using Horde for PCBs? Also saw a mention of some Horde bug that prevented it maybe?

No worries, I appreciate you taking the time to respond. Couldn't have hoped for a better responder (was hoping you'd chime in when I posted). Any clues as to where I should study/look next for investigating further?

[arctic hornet]

well, perforce PCBs came first, horde PCBs are a new thing and I'm not even entirely sure it's a fully cooked feature

so I'd recommend using perforce for pcbs at least until ue 5.5 where new artifact storage is enabled by default and a lot of system rewrites have taken place

I'll be experimenting with horde for pcb storage again when 5.5 is out myself... (which should be sometime this year afaik)

so once I have stuff figured out I can chime in

[earnest helm]

New build job did indeed correctly apply dashboardUrl to the editor badge link, btw!

[tacit ruin]

Hey guys, quick question; if the whole team is compiling from source (game and engine) do I need to keep "Binaries" folders in source control? I'd think not, but none of the documentation shows excluding Binaries, and team repos that I've been on have gone either way

[arctic hornet]

well, some of the tools and stuff in binaries comes precompiled from epic and you don't compile it locally

in terms of engine binaries

the general gist of things is:

• push to source control only what you get after git clone and running Setup.bat
• then push your content Source, Content, Config etc... and skip Binaries and Intermediates

[tacit ruin]

Hm, I was actually hoping for something that could be done after the fact, since I'm actually trying to figure this out for archival purposes, not setting up source control access from the start

[ocean anvil]

so you just want to get rid of binaries from your depot?

[tacit ruin]

I want to get rid of the things which can be regenerated from the project, so I'm not archiving any more than is necessary

So I'm writing a copy script, based on p4ignores, but I was/am unsure of whether I needed to archive the Binaries directories or if I could regenerate everything in them

[ocean anvil]

binaries don't have to be versioned no you can always regen them

[tacit ruin]

Everything in all Binaries folders, or are the files @arctic hornet was referring to going to give me issues?

well, some of the tools and stuff in binaries comes precompiled from epic and you don't compile it locally

[ocean anvil]

from the project itself you want to keep the following in source control:

Build 
Config
Content
Plugins (if you use plugins)
Source (if C++ files are present)
*.UPROJECT file```
when it comes to the source engine i'm pretty sure you can keep everything aside from Binaries and Intermediates

but @arctic hornet should be able to verify that

[tacit ruin]

Yeah, I'm more worried about the Engine folders, as those are most of the binary size. I guess if all binaries are gathered from Setup.bat I could always run that again

Or maybe just run a clean command from VS and then copy Binaries?

[arctic hornet]

sorry, was cooking dinner 🙂 I agree with what WTF sent... although you don't just take whole Plugins, but rather apply the same rules of only taking Build, Config, Content and Source (though plugins usually don't have Build folders)

since as you don't want binaries and intermediates for the game itself, you don't want any of that for plugins (C++ only)

[cyan viper]

I have a perforce server that i'm hosting on AWS and I have configured Swarm. I have 2 computers that I'm trying to connect with P4V. On one computer I correctly see Connected to Swarm Version 'SWARM/2023.4/2546515 (2024/01/24)' and on the other one I see that I need to check the log. The log has: error: Swarm Version of SWARM/2023.4/2546515 (2024/01/24) doesn't meet mimimum version of 1.1 . Why is this error only local to 1 computer?

Also I think unrelated but p4v is ignoring the .p4ignore file in the depot

[arctic hornet]

Make sure your p4v client is sufficiently new

If it’s older than swarm it’s likely going to complain

[cyan viper]

ahh ok

[unreal citrus]

It looks like my .gitconfig file was corrupted. First git stopped working and then when I opene the config file irt was full on nulls signs.
Any idea what may lead to such corruption?

[ancient oxide]

Hi, I keep getting the Cloud not find engine in Perforce relative to project path error when setting up UnrealGameSync. Seems like people have fixed it by making sure Engine and MyProject are in the same folder, but hasn't worked for me. Any ideas?

Is a stream depot required?

[arctic hornet]

I'm pretty sure stream depots are indeed required

they are at least for Horde... and horde shares the same perforce library with unreal game sync... so by extrapolating that, I'm quite sure it's the same

[ancient oxide]

thanks. i'll give that a shot.

[chrome summit]

Hi, I have a problem with UGS and the precompiled binaries. I'm in a small team working on multiples little projects at once and we use Horde and UGS to make and precompiled binaries for the artists. In order to reduce disk space, I've setup all the projects in a single stream so we have a single Engine on disk (with little modification and custom plugins) shared through all our projects. The problem is that now, for each code change in the stream, we have to recompile PCB for all projects, even if the only change is in a projects source code that is not used in the other projects. (I've tried to setup horde to make PCBs only if needed but then UGS don't want to sync if there is no PCBs) I don't want that because 1, the artifacts take spaces on the horde machine, and 2, if a single incremental build take like 5 min, if I have 12 projects, build all of them takes an hour and we only have 1 build agent for now.

Is there a way to tell UGS that if the code change is not related to the project it can use the previous PCBs ? If not, can I setup my streams to have only 1 project per stream but having only 1 Engine folder on disk ? (It seems that UGS need the engine source in the project stream)

Simplified actual perforce stream setup :

[arctic hornet]

It depends, horde doesn’t care much about the project itself it just cares about the workspace you provide it to. if you store PCBs on perforce they are tied only to specific stream. Not sure about horde PCBs though.

One thing to limit/split the repository would be to either set up multiple workspaces with limited view to just engine + project. Or set up virtual stream that does that

And then setting up stuff in horde for each project combo. More or less separating the monorepo to multiple virtual repos

That’s kinda the only idea that I have right now on how to figure this out

Well, unless horde templates have some kind of filter (which iirc they do have) but I’ve not tested that myself so I’m unsure about that area

[chrome summit]

It seems that horde PCBs are tied to a uproject. I've manage to only build PCBs if the code change is in Engine/ or ProjectX/ and then on UGS I can get the PCBs for that project but not the others (that are in the same stream). I will try the virtual stream option, thanks for your answer

[arctic hornet]

Yeah, I just checked the source code and horde is using project as metadata while in horde (and if using p4 for storage rather than horde you map the binaries to the project using ini file)

[ancient oxide]

Is "Main" a stream or depot here? https://dev.epicgames.com/community/learning/tutorials/8JYW/setting-up-an-unreal-engine-studio-the-epic-way

[arctic hornet]

everything epic uses these days re streams

so you can safely assume it's stream

[ancient oxide]

gotcha thanks!

[pulsar parcel]

the upstream engine can even be a separate depot

works in p4

[arctic hornet]

for example this is our server
//UE5/Main is the ingest from epic
//Matcho/Dev-Release-5.4 was/is a merging branch
//Matcho/Main is hte mainline where we develop stuff

[pulsar parcel]

Yeah, done the exact same thing in the past

[ancient oxide]

is Matcho a game?

[arctic hornet]

process of merging for us is:

• sync //UE5/Main to appropriate version
• create merge branch (//Matcho/Dev-Release-5.4)
• merge in //Matcho/Main
• merge in //UE5/Main
• resolve conflicts and run CI and tests on it
• merge back to //Matcho/Main

[arctic hornet]

ayup, it's a game the studio I work in works right now

(it's announced title, so it's not like I'm sharing something NDA 🙂 )

[ancient oxide]

ok cool. if the upstream engine were a separate depot, would that still work with UGS? / in the same directory?

[pulsar parcel]

yes because you copy/merge it to your project depot

[ancient oxide]

ahh ok

thanks all!

[arctic hornet]

more or less this is our project structure

[pulsar parcel]

when you merge it, all the conflicts are just your modifications being flagged

[arctic hornet]

and we ingest limited subset of the UDN repository (or github if you don't have UDN) like this

more or less just:

/Engine/...
/*

i.e. all files in root and engine folder

we skip all the samples, tests, templates etc...

[arctic hornet]

and then just do this

and conflict resolution is done in 2-3 steps:

• auto resolve without merging the code files
• force accept source binary files in the Engine folder
• manually resolve the rest, in case there are some

[ancient oxide]

ok sounds good. i'll use your steps as a reference and give it a shot. thanks again!

[arctic hornet]

just a heads up, use the ingest branch as a base for your game/project branch right from the getgo

since if you do that, and then properly use p4 integrate commands, it will know the history a limit the amount of merge weirdness it tries to do

[pulsar parcel]

it actually works fine if you don't do that since we started with a depot imported from git with a script

[arctic hornet]

since your engine is always based on the ingest stream and that way if only one file changes, perforce needs to only compare one file rather than all files etc... - simply said

[arctic hornet]

first time was a pain for us - mostly the waiting

[pulsar parcel]

p4 copy was used to populate, merging after that

[arctic hornet]

since if perforce sees shared revision it goes and checks... if the revisions are the same the resolve is much faster

[arctic hornet]

we just exclusively use p4 integrate

regardless of direciton and stuff

the only thing with p4 integrate is that in p4d 2024.1 you have to use -F flag since they disabled some behavior by default iirc

but otherwise it's valid approach (even robomerge is using it for example)

[pulsar parcel]

and you should totally use robomerge btw. though is it still annoying where you have to modify the login to get it to work?

[arctic hornet]

iirc there was official OIDC support merged in like less than a year ago

but we're running 3 years old build right now where we did that on our own

so not really sure

[pulsar parcel]

we used release streams, not very many dev streams outside engine upgrades

so robomerge was nice for that

[arctic hornet]

yeah, since we're not released yet we don't have a need for release streams, but we used it for large features like rewriting movement system

which we did for 2 months meanwhile our artists were still able to work and test with the old movement system

but thanks to robomerge we had latest assets all the time

[pulsar parcel]

we actually used release streams for milestones at one point

[arctic hornet]

Either way, this is the setup I usually use to bootstrap empty perforce server:

• create ingest stream
• ingest from UDN or git (with git you want to ingest clean sync + result of running setup.bat - nothing more nothing else)
• create project stream
• integrate ingest into project
• push your project files

[pulsar parcel]

but it was just easier to have a mainline as the default and branch for stabilisation

[arctic hornet]

our current milestone is "release"

yeah, 99% of the time we're working on mainline, since it just works with the way perforce does locking etc...

I like to spin up temporary branches for some large features where I know I want to push often, but at the same time I know it will break everything and project won't compile for a week or so

[pulsar parcel]

when reconciling the pristine engine, https://github.com/brickadia/p4-fast-reconcile will likely help. it's a shame I'm at a git shop at the moment (yeah, we want to move over) so I haven't got to try it

[ancient oxide]

at "ingest from UDN or git (with git you want to ingest clean sync + result of running setup.bat - nothing more nothing else)" do you also add the p4ignore here?

[arctic hornet]

during the initial ingest I disregard any p4ignores

so I get every last file

that way I know everything will work for me 🙂

[ancient oxide]

even the intermediate / saved ones?

[pulsar parcel]

you shouldn't get those

[arctic hornet]

during initial import I don't have those

git clone + setup.bat won't produce those

that's why I'm saying "clean sync"

because git clone + running setup.bat gets you 99% the way to the same content that epic has on UDN

there are some minute differences, but it's almost perfect

i.e. all the binary files, all the assets, all thirdparty deps, all precompiled apps etc... are there... so horde and stuff works seamlessly

no need for any special setup for build agents etc.

since after you add your project, you should just be able to sync from perforce, run UAT/UBT and you should be able to build your project with one command

because everything necessary is there

[arctic hornet]

that's a neat tool, I wrote my "UDNSync" which does take care of the sync from UDN -> virtually sync local -> reconcile -> get platform extensions (since we sadly have to get those like git folks do, due to other weirdnes I won't get to right now) -> and finally push

to speedup reconcile I do the way of just running reconcile on small chunks - folder by folder - which speeds the stuff up due to smaller number of combinations it has to check

but I'm going to take a look at this, since it sounds like proper way to do it, rather than recursive reconcile

[pulsar parcel]

I just remember it being a janky FTP server

much easier to get it from the licensee P4 that's for sure

[arctic hornet]

platform extensions have moved to organization portal, and these ays to dev.epicgames.com

but annoying thing is: the API requires cookies for access and is behind CF... so it's still semi-manual process to get those

[pulsar parcel]

they assume the big boys with automation are using p4 I guess

[arctic hornet]

yeah, our issue is that... I have UDN access as me not as a studio... and studio has NDAs... two distinct legal entities, so epic cannot grant access

[silver token]

There's no way to branch or copy across servers, right? you've got to do a sync from their server followed by a submit to your own?

[pulsar parcel]

yeah, that's usually the process

sync clean engine -> reconcile it on target server -> submit

[arctic hornet]

there are way to sync across servers, but you need service accounts and whitelisting on both ends (more or less making replica) ... which is a no-go with UDN

[silver token]

ah, okay. That makes sense I guess

[arctic hornet]

so as sswires said, in general the sync -> reconcile -> submit is generally the way to do so

[arctic hornet]

but if you're migrating between servers you have control over both of them you can do it in easier way

[pulsar parcel]

I just ran p4 reconcile -I at the end of the day, and it'd usually be done by the next day

so that's why I was curious about that fast reconcile tool

[arctic hornet]

I just do recursive reconcile and it's done in 10 minutes

probably won't catch every move operation, but the speedup is worth it for me

but I'll for sure check the tool

if you're interested, this is the messy/semi-hardcoded thing I wrote in few hours... among others it also does copy the UDN types to my server

it's obviously indended for what we need, and that is:

• sync from UDN
• fetch platform extensions
• reconcile
• submit to our server

and there's a lot of hardcoding in many places (like profile selection in Program.cs) etc...

[silver token]

When you create this merge branch, are you destroying it when it's done?
I've been planning on only 1 staging branch that isn't version specific
We'll have each version in it's own location, but there would always be exactly one path from the stream with the latest engine to the game's stream of the engine.

[arctic hornet]

I tend to keep it around just because I'm too lazy to remove stuff + unless you obliterate you won't get any tangible benefits

but you could have one "merge" stream

and use that constantly

that's actually what most studios do, this is just my personal preference

to always start with clean slate, so I don't have to bother with overwriting last merge etc...

saves me one merge operations (main -> merge stream)

[silver token]

I sort of have that setup for my hobby projects because they use the foreign project setup even though they use engine source.
it's not portable across machines, but I don't need it to be.

[wispy junco]

Hey, so I'm trying to make a game with a friend but when ever one person pushes a change to the git and the other downloads it any assets that were placed in the level are no longer placed, but all the code created or blueprints made are there in the content browser. Does anyone know how we might fix this?

[arctic hornet]

Are you using external actors? If so, are you pushing the __ExternalActors__ folder?

Ugh… I hate discord markdown… instead of underline it should be two underscores at each end

(I just can’t find a way to type code blocks on iPhone keyboard… ugh)

[ocean anvil]

put a \ before the _ it will negate it from being used as formatting.

bit more tedious but it works

[arctic hornet]

I mean… I can’t even find backslash lol… I just recently switched to iPhone from long years of android usage and still getting used to it

[ocean anvil]

eeeeewl

gross

[arctic hornet]

Honestly, it’s not that bad overall… only major gripes are the difference in keyboard layout… otherwise it was cheaper to get iPhone 15 than to upgrade my galaxy note 10… it has all I need etc… but hey that’s discussion for different channel 🙂

[wispy junco]

im pretty new not 100% sure what that is but To test we placed down a pillar it gives you when making a new project and pushed it and the other downloaded it and it was not there, this picture shows all the files that are being committed

[arctic hornet]

Well, I’m on phone (as you could’ve guessed from y previous messages)… so I can’t give much support right now… but… external actors are a thing where instead of storing stuff inside the umap file itself… every actor is it’s own separate file in the external actors folder

If you have that feature enabled it would explain not seeing the stuff if you did not push the external actors

Since all the map does with external actors is to go to the folder and load any uasset files stored in there… (simplified)

https://dev.epicgames.com/documentation/en-us/unreal-engine/one-file-per-actor-in-unreal-engine

This is the relevant documentation from epic about this feature 🙂

[wispy junco]

thank you ill check that out now, I appreciate the help

looks like that is enabled, as for if I'm pushing the ExternalActors Folder, is there a proper way to check? if you dont know off the top of your head thats fine

[arctic hornet]

Well, easy way to check is… if you place new cube into the scene, you will have to push one file inside the external actors folder… if your friend then syncs… they should’ve pulled the same file as you pushed (duh) and the cube should show up

Another potential issue could be engine versions since unreal has a feature where it won’t load assets from newer version of unreal (not just 5.3 vs 5.4 but also changelist number - but not sure if this is also a case for git)

So if you’re sure the files are being pushed and pulled properly, then I’d check the output log for any seemingly related warnings

[ancient oxide]

what does the lower icon indicate? i'm assuming it's a "stream branch?" / how do you create one in p4v?

[silver token]

The top one is a stream-depot, the bottom one is a stream.
I usually create/manage them from the Stream Graph pane (View > Stream Graph) where you can right click empty space and select 'New Stream' or right click an existing stream and select 'Create New Stream from ...'
Maybe there's another way, but I don't immediately see one.

[ancient oxide]

cool, i'll try that. thanks!

[vague whale]

hello quick question to the perforce experts 🙂 simple question, i am trying to rename a directory,

p4 edit .../AbilitySystem/...
p4 move AbilitySystem StormAbilities

.../AbilitySystem/... - file(s) not on client.

i love perforce 🙂 i can never figure out how to do basic stuff

[arctic hornet]

Rather than using wildcard like this, you can use absolute path

I.e C:/MyWorkspace/MyProject/Content/AbilitySystem/…

[vague whale]

thanks!

[earnest helm]

Is it okay to add the ".ugs" folder contents to p4ignore wholesale?

Trying to make sure everything is squared off before I sync binaries, which seems to actually add all that stuff to my workspace (which makes sense). But then reconcile offline works picks it all up -- so seems I have some p4ignore work to do there as well I guess.

[arctic hornet]

.ugs is your local folder containng UGS metadata like configuration etc..., so yes, you can safely ignore it

[earnest helm]

Thank you!

Seems like the UGS clean function isn't taking into account the .p4ignore. It is working in p4 but not UGS.

[arctic hornet]

well, that's because UGS is using customized sync and clean algorithms

even clean syncing from UGS doesn't get you whole stream but only what you need based on sync filters

it does it because you can have multiple projects in one stream and UGS will only sync selected project etc.

[earnest helm]

ok, these guys?

Is the thought that I need to define the use of the .p4ignore using these filters or is it that the .p4ignore itself isn't really applicable to a UGS ecosystem?

[arctic hornet]

honestly I'm not sure, under the hood UGS is still calling p4 api, so it should respect ignores... but I've not checked deep into the ugs innerworkings as I never had the need to 🙂

[earnest helm]

gotcha, was this something you were already aware of, as far as the clean not working with the p4ignore? ie. this is the common behavior vs. I did something wrong.

[arctic hornet]

I don't care that much about what p4ignore cleans, since cleanup unlike reconcile is supposed to just remove any assets like binaries and intermediates

but I've only used it like once or twice so I never noticed

and in the studio where I work, we're running pretty liberal p4ignore anyways

(I should make it more strict... but I barely have time to make the game itself )

[earnest helm]

I feel that, trying to get some boilerplate done so I can actually dev with folks

[arctic hornet]

I recommend to do that honestly, since once the train gets up to speed you won't be able to

lerned that the hard way

(unless you'd be decided to devops etc... but not many people are, especially not in small studio)

[earnest helm]

(want to get out of it being my responsibility as soon as possible, but atm as the tech designer I am the most suited to tackle it)

this whole side quest for p4>ugs>horde feels like a whole job, which of course it is as you pointed out (dev ops). Was thinking I could probably find some contract gigs after I really get it all nailed down to setup for small groups/studios. Really need to find tech design work though for $$ 🤣 😭

[arctic hornet]

I help anyone who asks (for free) 😄 as it's not worth negotiating price for 2 hours of work 😄

since once you know your way around all of that it's literally few minutes to set up

[earnest helm]

To me, seems like the knowledge you have is pretty valuable, even if it isn't difficult for you to do now that you know what you're doing. I've spent weeks banging on all this for instance, weeks I'd have preferred not doing this haha. The knowledge you're sharing with me is shoring up some of the loose logic ends I need to understand it all. I think the value is less the person coming to you to gain the knowledge and more in the person that would come not wanting the knowledge at all - just the use of it.

I've been on the dev user side of horde before, but not operator so I at least know what the end experience I desire is, which is helpful in knowing what I need to figure out next. Looked at the light documentation for it, and t does seem like those sync filters are important, so thanks for pointing me their direction. Annoyingliy, my initial stabs for the custom area didn't seem to affect the clean function itself... will have to investigate more.

Generally it seems I just need to add this new info you gave me into my thoughts on the overall .p4ignore work. In my head I'm trying to get to a state where syncing with ugs (project + pcbs) doesn't leave any clean or reconcile offline work stragglers. May just need to phase it off into the originally intended .p4ignore work, make sure it works with P4V, and not use the ugs clean function until I can figure out how to utilize the filters correctly or to make it work with the .p4ignore file.

An additional abstract thought for you @arctic hornet, I feel like I've scoured the web pretty hard for info on this whole Perforce>UGS>Horde ecosystem, and essentially what I found at the bottom of that rabbit hole was YOU (and sswires)

[arctic hornet]

😄

[arctic hornet]

I've been in game dev as a hobby for more than half of my life (since I was about 10yo... so like 16 years now) so I've gathered quite a lot of knowledge from the internet of the past (i.e. the one where you could actually relatively easily learn high quality information)... so I'm trying to help as many people and to share that knowledge further, since I can... and why wouldn't I...

since these days internet is full of tutorials like "download this template, change the logo, ship it"... but people don't know how it works, or what it does... so that's why I'm trying to do what I do, because I think it's best if people can ask targeted questions and get (relatively) simple and concise answers, rather than something vague

hence why I sometimes attach whole essay to a simple answer of "click this"... because in that essay I try and explain the reasoning and why it is what it is etc... because if you know the underlying concept, you don't have to ask again when the name of the button changes 🙂

[acoustic magnet]

Why is it that the editor source control integration doesn't support undo/rollback? Most of our artists would never need p4v if that feature was added. I'm considering trying to add it myself but im so surprised why it's not already there, maybe I'm missing something?

[unreal citrus]

Is it normal that all my UAsset and UMap files got added to Unversioned after I setup Git-LFS with .gitattributes for my existing repository?
Literally all Content/ was "modified", except I didn't touch it at all. All I did was add Git plugin and setup git lfs 🤔

[arctic hornet]

What source control provider are you using? Because I’m pretty sure that at least perforce integration does have support for reverting changes directly in-engine

[thorn halo]

Anyone used Perforce with GIT (since it says on the website that it is possible), because using the BETA GIT connection in Unreal I feel really slows down the editor compared to when I do work for other companies that use P4 instead. Though I do not know if this works when using it with GIT, since they all had their own Perforce setup. I would just like to be able to have the conncetion between the editor and my source-control since I keep running into issues when I delete something and forgot to FIRST open my source-control software (I am using sourcetree), that it gets added back into the editor.

Sorry... I was dumb. Of-course this will not work, since well, Unreal does a direct perforce connection and not trough P4... Sorry, didn't think about this, I should probably be in bed and sleep... Sorry for the dumb question.

[acoustic magnet]

Perforce. If you know of a way to rollback/undo/restore previous version in editor Id love to know! "Revert" afaik means discarding your local edits, which is possible but not what I'm looking for

[arctic hornet]

ah, yeah, I thought you meant revering local changes

honestly in perforce revering to previous revision is kinda funky as is

more or less what it does is, it syncs to older revision and pushes it again

and yes, there is no way to do rollback to previous revision from within unreal itself

[acoustic magnet]

Thanks for the confirmation. I'm just surprised its not implemented as I'd consider restoring old versions to be one of the primary reasons to use version control. The undo process in p4v is great for people that know what they're doing but imo over complex for most artists use case

[silver token]

To be fair, it's not something that you should really have to do that often.
It's a tool I only reach for if something is broken and the person that would fix their change is unavailable.
It's not a tool I would expect content creators to need to bother with.

[woven sluice]

i can probably count on one hand the number of times i've decided to undo things in the last 10 years

[acoustic magnet]

Interesting. I guess it depends what you use unreal for. We're not a games studio and recently starting using it, and this has been one of the biggest complaints from users who are used to easy access to previous versions

[silver token]

There's a difference between accessing previous versions and undoing.
You can always sync to a previous version of an asset/build in order to view it. Text you can even view without syncing.
Undoing it is very different and may not actually be what they want/need.

[woven sluice]

undoing very specifically means taking an old version and committing it as the next version (in perforce terminology)

[cyan viper]

Anyone have an idea why my projects .p4ignore is being ignored. It keeps trying to commit intermediate files even thought they are explicitly ignored

[teal bone]

Have you set the environmental variable / setting that actually enables the .p4ignore file?

[cyan viper]

i believe so. how can i check (I assume i need to set it on the host not on each individual machine)

[arctic hornet]

if you don't set it and are using p4 that i's 2023.1 or newer iirc perforce automatically uses both .p4ignore and p4ignore.txt files

since before that on windows it only used p4ignore.txt unless the P4IGNORE variable was set

you can check if the P4IGNORE has been set to something other than default by using p4 get P4IGNORE if it returns something it means it's overriden to that value, if it doesn't return anything... then it's using default value

[cyan viper]

i'll check the p4 version as well

[woven sluice]

open P4V
right click on workspace root, "open cmd window here"
type "p4 set"
does it show P4IGNORE?

as a sidenote, my preferred approach is to NOT set P4IGNORE... instead set P4CONFIG and use a config file to specify username, workspace, ignore file name, etc. for each workspace

[cyan viper]

ok but how do you then keep from commiting intermediate and saved files to the depot for workspaces that don't have an ignore?

[woven sluice]

I don't understand what you're thinking happens

The ignore file takes care of that (as you know)

[woven sluice]

(oh, "that don't have an ignore" -- no, you always have an ignore, OR you just rely on stream settings to ignore everything if that's possible)

@cyan viper

i actually don't use any ignore in my current project but the stream spec method isn't as good as an ignore file and not always usable

[cyan viper]

ok that's where my confusion was coming from. When you said you had a p4config that set up ignore file name per workspace. I had horrible visions of someone locally setting their p4 config to just not specify an ignore file. I thought I could just put a p4ignore file in the root dir of the depot the same way I could with a .gitignore

when I type $p4 set I see a config for P4CONFIG, P4PORT, and P4USER but nothing for P4IGNORE

[woven sluice]

well then

[cyan viper]

so I would just create a .p4config file that specifys the ignore file to be .p4ignore and put them both in the root of the depot correct?

[woven sluice]

yeah that's what i do

P4IGNORE=p4ignorefilename

in the p4config file

[cyan viper]

@woven sluice as always tyvm you are a lifesaver.

[woven sluice]

https://tenor.com/view/spinning-swimming-summer-time-lifesaver-happy-gif-14236270

[hoary wren]

Quick question for you all. I'm trying to make sure our non-source-building folks are able to start the editor without having to build anything, and it looks like the plugins are getting in the way, they need to be built as well.

Am I correctly understanding that these are the files (as an example) they would need to have in their game folder in order to be able to run? Minus the .pdb files.

[ashen oyster]

I have some plugins as a submodule on a project which has revision control enabled. SourceTree sees the changes I've made to assets inside those plugins, but in the engine when I right click the asset and go into revision control and press "Diff against Depot", nothing happens. I am also not seeing the icons indicating changes and so forth.
Does UE's revision control not support submodules or am I missing something?

[woven sluice]

yeah the DLL files and maybe the .module files (I actually don't know about the .module files, but i'm too lazy to test so i just included them anyway lol)

and they need to be built under Development Editor config

[arctic minnow]

Question more about github but I can't tell where to ask.
How do you get an upload from previous versions of a plugin? :/
I'm trying to get VRM4U on 5.3 but it seems all files were updated to 5.4.
When I load it, it throws the error that all modules were built in a different version.
Are there specific files that specify the version I can change it to? :x

[teal bone]

Have you tried compiling and launcher from your IDE?

[arctic hornet]

Well, modules have metadata that tell for which version of unreal it's compiled, but that means they have to be compiled to that version because the code in engine changes etc... one way to handle this issue is to either download from marketplace version for UE 5.3... or download it form github (if they're not on market place but on github)

rather than changing the versions

and if you have correct plugin version and are still getting this error, then you'll have to go and re-compile the plugin for your version as Daekesh said

[arctic minnow]

@arctic hornet @teal bone So this was a weird session.
A previous version of 4.27 could be put inside the engine marketplace plugin folder and it ran fine.
Post 5.0, it looks like the plugin was made version agnostic but requires local recompilation in the project itself. It seems that version agnosticism causes this missing modules error since it needs recompilation and it cannot be done on the engine plugins.
Thanks for the hints! 🙂

[arctic hornet]

I believe the switch to version agnostic plugin was simply because unreal is annoying and if you have plugin for ue 5.3 it can sometime start to complain about wrong version if you're running 5.3.1 etc.

I'm personally not using the engine distributed from Epic Games Store, so I'm not really aware of it's limitaitons... but from what it seems, you can easily just download the plugin for your specific unreal version https://github.com/ruyo/VRM4U/releases

if it's the same plugin

[arctic minnow]

Oh my god, that's where that page went

I was looking for that for ages!

ty

[arctic hornet]

if you are using custom engine built from source code rather than from epic games store, then for sure almost any plugin will require recompilation because your engine version will most likely differ locally from EGS one etc... etc...

but I presume you're not using it, so take that as just an information 🙂

[arctic minnow]

I used to but then they release a fix I need and they fix the rest anyways >.>

[pulsar parcel]

and delete the engine version in the uplugin if you want to get rid of any nag screens about engine version

[timid hound]

Anyone run into an issue in perforce where every time another user submits a file and I go to "Get Latest" if gives an error for "can't clobber writable file"? I understand that the work around would be to go into the properties of the file and switch it to read only but shouldn't perforce be automatically doing this upon check in of the file? It seems to be for every file submitted and only started a few days ago after trying to merge two streams and then undoing the merge.

[rotund lava]

Hm, try to check your workspace settings, maybe your entire workspace is set to allwrite by default.
In your P4V go to Connection->Edit Current Workspace->Advanced and uncheck AllWrite if it's checked.
Or, if it's not the case, try to set Clobber to true.

[timid hound]

Based on these settings you think it's alright and I should just set Clobber to true?

[silver token]

This shouldn't regularly happen unless you've selected the 'Make Writable' option in the Editor when working with assets.
When you get the popup, it should give you the option to overwrite it. You shouldn't have to go to the file properties to make it read only.

[timid hound]

Yeah I'm trying to figure out why it's happening. I think its only from one user when I am getting the files but still testing that. Also going to test if its the same for that user when I submit something.

[rotund lava]

You can give it a try.
Also sometimes makes sense to do some reconcile offline work or Get Latest Revision with force update of all files.

[timid hound]

Where would you find the "Force update all files" ?

[rotund lava]

Right click on the folder you want to update or root of workspace, Get Revision
There you'll be able to choose what Revision and a checkbox called something like Force Update.

[timid hound]

I see that, thanks will try when I get a chance.

[rotund lava]

This might take a while, it's basically redownloading a project

[timid hound]

If it fixes having to unclobber everything everytime def worth the wait haha

[timid hound]

Also, where would I check for these settings? When you say in the editor are you referring to UE5 or a setting in Perforce?

[silver token]

One popup would be in the Editor when you save a file. It would ask you to check it out but also gives you the option to make it locally writeable instead.
The other popup is from P4 when you sync the file. It should list the writable files and allow you to select which ones to overwrite.

[timid hound]

Any ideas on this?

Basically I was having issues after merging some dev streams together so I undid it and I think it deleted a bunch of branch files causing some issues with what revision some files were on. Then when trying to submit a changelist it created a bunch of shelved files so I deleted the shelved files and just submitted the changelist now im getting this error with different files. Wondering if resyncing with offline files would get everything back up to date to avoid this issue moving forward?

Any help getting this resolved would be huge.

I guess my question would be is reconciling offline work a way to get the server up to date with what files are actually there as a so called "fresh start" ?

[wide raven]

Take this as preliminary thoughts and wait for second opinions perhaps, but my thoughts are on these lines:

) reconcile offline work compares your local workspace state to the depot. This can certainly be helpful to figure out where you are at, but it doesn't do anything itself, just gives you a list of files which don't match the depot state. What you do from there still depends on the situation

) the options given seem pretty clear. You must resolve or revert. Assuming you have the desired changes still on the streams, I would probably be reverting to a 'clean' state before attempting the merge again

) I assume you are familiar with resolving

[timid hound]

Can we pretend I’m not familiar with resolving. I basically have the project how I want on my local computer but at this point want to just mirror that to the depot to avoid all these “revision unavailable” and “shelved changes deleted” errors. For that specific one I posted I just went into the command line and did the resolve -ay and another prompt chat gpt had me do and it cleared out the changelist but kept my changes locally. I’m assuming others won’t be able to see those changes though. Idk if that updated the depot or not.

[wide raven]

I might consider fetching a fresh copy of the depot to sanity check what state it is in ( all working and fine or messed up )

After a resolve, the changes are still all local - the file has been updated with the changes from depot, and is still checked out and ready to be submitted

[timid hound]

Hmm when i did that resolve -ay thing it made the file disappear from the changelist. Is that wrong?

[wide raven]

hmm. ok
-ay : Accept yours, ignoring changes that might have been made in theirs.

This is not a solution I like to see

I am a bit surprised it would not be in the changelist still

Generally you want a proper merge, not accept yours/theirs

[timid hound]

Could it have had something to do with revert -k?

I believe I ran both of those commands for that specific file

But when I reopened the project from my workspace the Change was still there but nothing to submit

[wide raven]

-k
Keep workspace files; the file(s) are removed from any changelists and Helix Server records that the files as being no longer open, but the file(s) are unchanged in the client workspace.```

so yeah, you should definitely be looking at p4 docs, or using p4v

f.ex https://www.perforce.com/manuals/v22.1/cmdref/Content/CmdRef/p4_revert.html

Try not to accept blindly instructions on what to do, but actively look for the docs to see what is really happening

[timid hound]

Yeah I’m using p4v. This is just my first time using any of these products so it’s def a learning process. I’ve been at it for 10 hours today trying to get this fixed. I really appreciate your help.

[wide raven]

Well, I would say any state should be recoverable with some patience. Just be wary, don't submit more stuff onto the pile before getting a very clear idea of what the situation is

Regarding that, I would again recommend checking out a fresh copy of the depot to see how it fares

Make sure project compiles and the state is in a position you can recognize and work with

[timid hound]

Any steps on how to do that? I would think there would be a way to just upload all my local files to the depot as like a new starting point but maybe not?

[wide raven]

No I mean just creating a new folder and workspace, and getting the depot to that to understand what the state of the project is in Mainline

[timid hound]

Ah that makes sense yeah

[wide raven]

Then see

1. do you need to fix things up
2. do you still need to merge stuff in from the streams mentioned

[timid hound]

How important is merge down vs copy up? I created a development stream under my development stream for someone to work in and tried to merge there stuff into mine. It seemed to work but some of the blueprint stuff they did messed with mine so when I undid it. Seemed to delete the branched files which is when all these issues started.

[wide raven]

So before we dig into the up/down thing, I will now categorically state that using streams with anything else but code is currently a nightmare unless you have very advanced tooling to achieve file locks for the .uassets which are binary blobs and are practically impossible to merge

Most studios just opt for working on mainline

I was shocked too, coming from unity/c# : E

File locking is so crucial to teamwork in Unreal

[timid hound]

The other thing I was trying to figure out like you can “get revisions” from a particular time but then if you “get latest” doesnt it just override it all?

Snap crackle I mean I have like the recommended type map and .ignore files but maybe that’s not enough?

I’m worried that when I undid the merge it deleted all the work the person did so I wanted to recover that stream from the point before I tried the merge.

[wide raven]

Regarding merge down/copy up, it is definitely the best way to do things, and p4v at least tries to enforce the pattern. You can break it for example by cherry picking ( I believe ) but this is again not a recommended pattern for working except in emergency situations

[timid hound]

Does copying up only copy files that were changed or would it overwrite everything?

[wide raven]

Yes, Get Revision can be a bit dodgy, I dont recommend using it for anything except finding the point at which a feature breaks

[wide raven]

Changelists only

[timid hound]

And I’m assuming like merging it tells you if there are conflicts?

[wide raven]

If you "undid the merge", i.e did the "Accept Yours", and then submitted - then yes, you have wiped their work. You should consider Undoing that CL

Undoing a CL is very handy if there's not many changes happened yet, and you are not risking muddling things up even more

[timid hound]

There was like a thousand files or something so it was hard for me to pin point what parts of the merge I didn’t actually want

[wide raven]

There's another point : you should never be doing stuff that complex and massive in one swoop, it's a recipe for disaster...
Now the question is:
) did you submit and overwrite things
) is it still the last change in depot history?

[timid hound]

So basically I should “redo the merge” by undoing the changelist where I undid the merge and then just pick it apart and only remove the parts that overwrote my changes

A lot of it was just added files. Probably like 12 or less conflicts. But there has been quite a few change lists since then.

[wide raven]

1. Recover the depot to a working state
2. Really analyze the mess you are in with conflicts. You need to go over each and every conflict and if they are blobs, decide whose changes 'win' and who gets to redo all their changes

You can always test the Undo locally, and consider what to do with it if it looks good, submit it

Generally speaking I would say the 'biggest amount' of work 'wins', and the smaller amount of changes are redone

But in future one should really never have to be in this situation

[timid hound]

In all honesty would it make sense to just fresh start the server/main stream with my local project that has basically everyone’s work. The other person has their project backed up so couldn’t they just create a workspace in a development stream and upload their project and we merge the differences from there?

I’m also really only coming across issues when either A. Trying to submit something like the image I shared earlier or B. When getting latest I seem to be getting a clobber error where it doesn’t seem like the files are being marked as read only automatically.

[wide raven]

Hmm hmm

[timid hound]

Everything is functioning smoothly when I launch the project from mine or my friends workspace

No errors or anything

[wide raven]

So, I am unclear of what problem remains if both workspaces are now functioning. You have pending CLs on your workspace still, which you are unsure whether to submit?

Question : Is your teammates' work
a) in mainline
b) only on your local merged workspace, in cls?

Hmm need to read your A + B again

[timid hound]

So when my friend submits his changes and I get the latest files it gives me clobber errors for each thing submitted until I go in the properties and make it read only. And there are certain files giving the errors from above image that shows revision unavailable or “can’t resolve shelve change was deleted”

Both of us are working out of a development stream under mainline

[wide raven]

You each have your own streams?

[timid hound]

Same stream separate workspaces

[wide raven]

Maybe the A image issue is this: https://portal.perforce.com/s/article/10830

[timid hound]

These issues only started happening after I tried the merge from someone else’s stream into our stream and then undid it and it seemed to delete a bunch of branches files.

[wide raven]

B : Clobber errors. When I see this sometimes, I usually solve it by navigating to the file in P4V and doing "Get Latest" with "Force Operation" checked. This should restore the file to the latest state and guarantee the p4v managed file locking is in place

[timid hound]

Then today there was basically a large changelist that somehow had all the files also as shelved and couldn’t figure out how to submit it so I deleted the shelved files and just submitted the other files. (Which matched up 1 for 1 with the shelved files.

[wide raven]

Yeah, that is normal.

I mean not being able to submit a CL which has shelves

[timid hound]

But then when I went in and made changes to a blueprint I got that error above saying the shelved items were deleted

[wide raven]

But not sure how the shelves arose : I never see a shelve created without specifically adding them myself

[wide raven]

refer to the link I posted for A issue

[timid hound]

Yeah that could have happened and we just never really came across that so wasn’t sure how to deal with it

[wide raven]

Now I don't know if you have power over this sort of things in your operation but I would recommend talking about this with the team
: most unreal BP/Editor devs do mainline development to avoid messing up with streams
: only pure c++ code work is good in streams because of missing file locking

[timid hound]

I have the power

[wide raven]

so the idea is to do daily commits, always non-breaking. Iterative work direct to mainline

[timid hound]

So our c++ person can still work in their own stream before pushing up to us in mainline basically?

[wide raven]

Yeah, c++ stuff is easy to merge. But the strict rule has to be that they don't touch any .uassets, i.e blueprints etc if they do so

[timid hound]

So if they are editing blueprint stuff they should just jump up to mainline?

[wide raven]

Yes, because of the file locking not working across streams

So they have to be able to see if files are locked by other people, and won't cause merge mess which always results in pain and redoing work

[timid hound]

Makes sense. You are truly my hero right now. I really appreciate your help and time.

[wide raven]

No worries

I got to close up for the night though. I hope you can get things sorted out : ) Good luck, have patience

[timid hound]

Thanks a lot! Hope to chat again sometime!

[woven sluice]

haven't read the whole chat.

99% of the time, writeable/clobber errors are because you (whether intentionally or accidentally through tools) set some files as writeable. this is obviously obvious, but it is what it is.

i don't know if there is any "official" way to reset all accidentally writeable files back to readonly... but you could try doing this:

• Check out EVERYTHING into default changelist
• (Several minutes later) right click on changelist, click Revert Unchanged Files

as an aside, if you want to brute force your way out of this, you could turn off the safety mechanism. Edit your Workspace, go into Advanced settings. there is a tickbox you can turn on to enable automatic clobbering of non-readonly files

[somber merlin]

I have a question i've recently set up a github repository with as base an unreal project that has been created from this free ArchViz Explorer Template. I am certain of the fact that i've selected github's Unreal Engine Gitignore to be included in the repo, yet after cloning the repository i and a few others have these changes present after only opening up the project for some reason? And even if you discard them 1 of the logs and the json file stay there persistently.

Does anyone know if these files should just be comitted, discarded OR added to the gitignore?

[woven sluice]

those entire folders should have always been ignored. if they've already been committed, they will never be ignored until you commit their deletion

[somber merlin]

ah i just didnt add them to the commit for now but i'll add them, thanks

[woven sluice]

i think i hope there was some accidental wrong words in that sentence, somehow 😄

[somber merlin]

oh yeah that sentence is meseed up 😭

i meant add those to the gitignore

[ancient oxide]

I tried to submit a source build to p4 but am getting this error. Anyone had to deal with this / know what nvTextureTools is / know if it's ok to delete it?

Hi, what did you do to get around this?

[long mural]

I had to modify the file type

Of the stored files

[ancient oxide]

Can you elaborate a bit more? Not sure what you mean. You changed .rc to a different file extension?

[long mural]

No the file type in perforce

[ancient oxide]

like in the typemap? what did you map it to?

[tight granite]

Can someone please let me know what the best discord server is for Git?

[arctic hornet]

In what sense? Like discord server focused on git? If so I sadly don’t know any

[tight granite]

@arctic hornet Yes a server devoted focused on Git. I'll keep looking and hopefully someone on here can recommend one 🙂

[mint mountain]

Hi. Is anyone using Perforce with a C++ project on an Apple mac and Xcode ?

[pulsar parcel]

Not sure how that makes a difference, you don't store the Xcode project file

[pulsar parcel]

What is the question?

[mint mountain]

actually everything works, except for the script for the automatic checkout of c++ files. the script is provided by Perforce, but it is outdated. there is a never version on github and at least I could make it to run, but in the log of the script I get the following error: p4 edit ouput: Perforce client error: Failed client connect, server using SSL. Client must add SSL protocol prefix to P4PORT.

checked P4PORT and it has that ssl prefix

[pulsar parcel]

not sure you deleted it and reposted it, but does P4PORT have the ssl prefix set?

you can use Rider on Mac too

[mint mountain]

yes, I think there is something else wrong in the script

so the file is not checked out, but I noticed in the p4v client that the icon has changed (MPTestingCharacter.h)

[pulsar parcel]

Well I don't know what script you speak of

If it's just using environment variables, then use p4 set to make sure P4PORT is set correctly

Rider has a much better P4 integration

[mint mountain]

can provide you the link to the Perforce guide

https://portal.perforce.com/s/article/2997 - each time you intent to change a file in Xcode it is detecting that it's locked, and obviously you can run a script that unlocks the file by checking it out. there is a newer version on github

[pulsar parcel]

so it tells you to add P4PORT to a file

did you do this?

did you put the ssl: prefix in it like you were supposed to (this should match open connection in p4v)

[mint mountain]

I am dumb. 🙄 there that prefix was missing, now it finally works. thank you

[pulsar parcel]

Well at least you got there. I presume that file is highlighted blue because it was made writable but not checked out

[dry urchin]

Hey everyone!

My friend and I are just getting started with Unreal Engine development, and we want to collaborate on a small project together. I'm a fullstack web developer, so I'm already familiar with Git, but I was wondering if anyone could give me some advice on the best way to set up source control for a small Unreal Engine project. What tools and setups would you recommend for a small team like ours to work smoothly together? Are there any guides that are easy to follow that I could use?

Thanks in advance for any help!

[night charm]

I'm trying to push a new UE project to a fresh gitlab repo I just made, but I'm getting an error I've never seen before:

LFS: Client error &{%!!(string=https) %!!(string=) %!!(*url.Userinfo=<nil>) %!!(string=gitlab.io) %!!(string=/es/lyraprojects.git/gitlab-lfs/objects/eb3392bc7fbd300a88406d6e2badfa8c5027ed1b5ce7f38578b87b4a8ecfe34b/149496797) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=) %!!(string=) %!!(string=)}s(MISSING) from HTTP 413
Uploading LFS objects: 100% (8663/8664), 2.2 GB | 37 MB/s, done.
error: failed to push some refs to 'https://gitlab.io/es/LyraProjects.git'

Has anyone seen anything like that before?
I think HTTP 413 is a "Content too large" error. wtf

[outer kiln]

git push -u origin master , getting error code error: failed to push some refs to 'https://github.com/Neighborlee777/smarine.git'
, what would cause that ? < I'm getting the same error you are snipe wth ;())

I have no content in my dir atm so thats def not it