source-control

[arctic hornet]

it gets overwritten when you switch project

so PCB usage on client-side is constant

[long mural]

yeah if you use the same workspace

[amber sentinel]

I can't find any info on how to set info in this p4 info section.
Do I need to run it on the server in some special way (even though i've understood .p4ignore as being client only, not server based?

[arctic hornet]

p4 info is (as the name suggest) info dump of current connection and state etc.

you really won't have any ignore related stuff in the p4 info

if p4 set doesn't show P4IGNORE then it's using default value, which is either p4ignore.txt or .p4ignore - if that file is found it's applied in that folder and it's subfolders depending on the rules

if you however have custom value of P4IGNORE set, then the ignore file has to be named that way

you can also check what rules are applied to given folder by using p4 ignores -v

which gives you output such as this... where you can see implicit defaults... in my case followed by p4config.txt being ignored followed by rules from .p4ignore.txt

[ocean anvil]

Is it pronounced mah-tcho, or match-o?

[arctic hornet]

match-o ... as one of the main mechanics is ithe "match-3"... doesn't have to do anything with macho etc.

(although the pronunciation would be pretty much the same in english ngl)

(wasn't my idea 😄 )

[ocean anvil]

Lol valid, I was just really curious

[arctic hornet]

fair

[amber sentinel]

Yeah, I have the p4 set, set to P4IGNORE=.p4ignore
If I run the p4 ignores -v, it does spit out a ton of extra stuff, and appears to read my custom file.
But, then I can still add files that should be ignored.

Such as my file containing these 2 lines:

/*.sln
*.sln

Someone suggested trying both sytnax's.
yet, I can still add a simple test.sln file.

Both the ignore, and that test file are in my root:

If I find that commands entry for the 2 lines, I get this:

[arctic hornet]

I see the /*.sln/... that sounds like a really weird way to write that.... + ... isn't even valid in a sense of P4IGNORE

P4IGNORE is special... and uses * for non-recursive wildcards and ** for recursive wildcards

so I'd personally use
.sln/ which would ignore any .sln directory (and thus it's files)
or .sln/** to ignore files in that folder

[amber sentinel]

Even a blank file, with legit just .sln
With this output:

I can still add the test.sln perfectly fine.

[arctic hornet]

well that's just he folder
to ignore any sln file you'd most likely use:
/**.sln

or if you want to only ignore sln files in the root (which I recommend, you'd do /*.sln

[amber sentinel]

Oh, so it was incorrect ignore file format this whole time...

Do you have a working ignore file?

(I think I saw one up a decent bit of messages if not)

[arctic hornet]

there is one in disitrbuted engine files by epic

or... actually it's not if you have epic games store edition

if you use source build from UDN/github then it's there

# Here you can specify files to ignore when adding files to the depot.
#
# The syntax for P4IGNORE files is not the same as Perforce syntax.
# Instead, it is similar to that used by other versioning systems:
#
# - Files are specified in local syntax
# - a # character at the beginning of a line denotes a comment
# - a ! character at the beginning of a line excludes the file specification
# - a * wildcard matches substrings.
#
# For example:
#
# foo.txt    Ignore files called "foo.txt"
# *.exe      Ignore all executables
# !bar.exe   Exclude bar.exe from being ignored
#
###############################################################################
# Epic's P4IGNORE.
# P4IGNORE doesn't work like GITIGNORE:
# http://stackoverflow.com/questions/18240084/how-does-perforce-ignore-file-syntax-differ-from-gitignore-syntax
###############################################################################
# Ignore root Visual Studio solution files. We do check in some sln files in subdirectories, so only ignore
# the ones found in the root.
/*.sln
/.p4sync.txt

# Ignore all Visual Studio temp files.
*.suo
*.opensdf
*.sdf
/Engine/DerivedDataCache/*
**/DerivedDataCache/Boot.ddc
**/DerivedDataCache/**/*.udd

# Ignore all Intermediate and Saved directories
*/Intermediate/*
*/Saved/*
# Ignore UBT's configuration.xml
Engine/Programs/UnrealBuildTool/*
*.uatbuildrecord
*.tmp

# Ignore built binaries and temporary build files
*/obj/*
*.csprojAssemblyReference.cache

# Ignore UBT's log output files
/Engine/Programs/UnrealBuildTool/*.txt

# Ignore Python cached files
*.pyc

# Ignore JetBrain's IDE folders
.idea/
!.idea/runConfigurations
.gradle/

# Ignore autogenerated files from HoloLens WMRInterop
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/packages/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/Generated Files/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/ARM64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/ARM64/*

# Ignore files added by Finder on Mac
.DS_Store

# Ignore all .code-workspace files
*.code-workspace

This is our ignore - it's the same as the one provided and used by epic

it's quite lenient

since it's expected that developers push only what they need

• in some parts it expects source build, hence paths starting with /Engine etc.

[amber sentinel]

Ah, many thanks.

I must've gotten some bad one from somewhere (maybe I got the github one, rather then perforce one by accident)
But, that fixes it for me.
Thanks. 🙂

Hmmm, maybe not?

I see this entry:

# Ignore all Intermediate and Saved directories
*/Intermediate/*

The cmd command reads it.

Yet, I can still add files from those directories (assuming I did it by accident for instance)
Or, do I miss-understand the syntax?

[arctic hornet]

you might want to do /** at the end, not really sure I just use what epic provides, simple as that 😉

[amber sentinel]

lol
No worries

I'll mess with the syntax then, to get it how I want/ expected it.

[ocean anvil]

if you are manually selecting specific files and marking them for add you can sometimes bypass the ignore file.

instead select your depot>right click>reconcile

[amber sentinel]

It looks like it was just wrong format for some reason.
I swapped the format, and got it how I wanted it.
Thanks for extra info about the reconcile

[outer lava]

was the Source folder supposed to be versioned? I created a workspace, copied files from my UnrealProjects folder into my workspace, then generated project files and opened UnrealEditor from uproject, connected to perforce and submitted everything

[amber sentinel]

Unless you included it in your ignore by accident, but yes

[arctic hornet]

Source folder is quite important... unreal editor by default keeps track only of the assets... so that might be the reason why

I'd probably go in to your Workspace tree, selectively click on "Build", "Config", "Content" and "Source" folder and hit "mark for add" and submit that straight from within p4v

never push Binaries, Saved or Intermediates

[arctic hornet]

as for plugins, same rules apply, go to each plugin (if you have any) and submit the uplugin file, "Source", "Content" and "Config" folders

[outer lava]

i did that recently but I left out the build folder, should I have included it?
I have included the whole source folder even though somethings in it arent supposed to be versioned

[arctic hornet]

Source folder should be versioned completely, there's nothing in there that's not supposed to be
Build folder generally contains things like helper and automation scripts, application icons etc. and so that one should also be versioned

[outer lava]

i must have misunderstood Epics p4ignore file
Thank you for the advice

[arctic hornet]

Epic's P4Ignore pretty much just ignores Intermediate and Saved folders

and certain autogenerated files

[outer lava]

i misunderstood this block

[arctic hornet]

yeah, that's in engine and those are auto-generated files (as you can see in the coment for this block)

[amber sentinel]

Its also ignoring those specific files/ folders.
Not the entire Source folder

[ocean anvil]

show the entries in your ignore file.

[green berry]

I was just assigned a project using git lfs, after only having used perforce for a very long time. I am running into issues when merging lfs files. After the merge, it seems to only keep the LFS pointer, not the actual object. I assume we are missing some config?

[long mural]

if the file is already marked for add, then it will carry on

you need to remove it from the depot/unmark as add

[amber sentinel]

If you read down more.
Someone helped me figure it out.
I was miss-understanding the ignore file formatting.

[white badge]

the non-versioned files in my perforce workspace are read-only and I cant change it, I have the workspace set to write and have noallwrite on, what else should I do?

[silver token]

What do you mean by "non-versioned file in my perforce"?
Either they're in perforce and versioned or they're not in perforce and therefore not versioned.

[teal bone]

Perhaps in his perforce directory?

[silver token]

yeah that's what I figured but didn't want to assume

[teal bone]

Unversioned files do show up in perforce too, so they can be in perforce and unversioned.

[silver token]

They can show up in the workspace view sure, but if they're not in the depot view they're not in perforce.
/pedantic

[teal bone]

Depends if you mean perforce as the version system or perforce the application! (Being pedentic, that'd be p4v (or the helix client now or whatever), but in the context of somebody who doesn't know much about perforce, you'd easily call it that.)

[white badge]

i mean the files that are in my workspace but are not in my stream depot

[silver token]

Then the only way to mark them as writable is to edit them through the file properties.
I guess you could add them to your depot and then revert the add (careful not to allow p4 to delete them)

[white badge]

i cant change the read-only property, it reverts after I change it

[teal bone]

Perforce won't change the read-only property of unversioned files, will it?

[silver token]

No.

[teal bone]

Are they files in, say, your UE content folder and is the editor running?

[silver token]

that's bizarre. Something on your machine must be modifying that file after you change the flag.
I don't know how you find that app. Maybe anti-virus (grasping at straws)

[white badge]

it wasnt until I installed the p4vs plugin to visual studio, which kept freezing when I tried to connect
Should I just make a new workspace

[teal bone]

vs might be screwing with you.

[white badge]

editor and vs are closed, everything except vsconfig and project.sln are read-only

i can check out the versioned stuff and edit them but everything else I cant edit

[teal bone]

Have you tried simply rebooting?

[white badge]

no, ill try that

[deft herald]

Hello everyone, question regarding revision control.

What is the best revision control for indie non profit hobbyist ? Me and friend working on a project together for small cinematic. We would love to have revision control ot help us share and keep track of files.

We dont wanna spend on P4 and some cloud server for this.

We cant create our own server as we both live in diferent timezone and having PC run 24/7 for the other person to sync would suck.

Anyone with some ideas what is best approach here ? 🙏

Was looking into Github with teams ( 8 bucks ) that has up to 250gb but not sure how good it would be for revision control 🤔

[arctic hornet]

I would always say perforce (p4) is the way to go with unreal, other variants are nowhere near the level of integration that you get with perforce, it's also free for up to 5 users and 20 workspaces...

but since you've said you don't want to get a server to host that and can't host yourself... then I'd say... go with git (doesn't matter if github, gitea, bitbucket etc...) as it's the most used version control other than perforce so you can find a lot of tutorials on the internet....

as for git provider I'd recommend checking Azure DevOps since they provide 250 gig private repos for free afaik

Also you might want to check out https://github.com/ProjectBorealis/UEGitPlugin which is alternative implementation of git for unreal which is apparently much better than the default one - but I'm not sure whether it's been updated to UE5 and/or whether it's good for real (as I've only ever used perforce)

Hope this helps

[deft herald]

Thanks! I was just checking the Azure DevOps but the fact they want my card even if I want just use the free scares me a bit 😅 But might go that way. Git seems as the last think we can actually try. Just suck that it seems too complex 😅

I would love to use P4 but as I wrote, there is the fact we both live too far from each other and I would need run my PC whole night for him to be able use it which I dont want as I already have high electricity bills 🫠 Plus the fact last time I had issues with port forwarding and could not let anyone join me so idk if I would be able to run it anyway.

[white badge]

my new workspaces are read-only as well, should I just version everything so I can check them out and write to them when needed

[arctic hornet]

perforce doesn't modify anything that's not checked in... and it only modifies the flags when you check out (it makes the file read-write) or when you check-in/submit (makes the file read-only)

removing the read-only flag directly in windows can make those files writable

if the files are still read-only then there's something breaking that outside of perforce and so pushing that to perforce wouldn't solve the issue

[white badge]

it reverts back when I try and change the read-only flag in properties
it started doing this when I installed the p4vs plugin to visual studio and tried to connect to my p4d server

[arctic hornet]

p4vs only calls p4 client and as I've said perforce itself modifies the flag only when explicitly told

so it's an issue unrelated to perforce

because if it's immediately being reverted it's something else on your machine forcing that read-only flag

[woven sluice]

show us what you're actually seeing. my guess is either a) you're checking properties of a folder, not a file, and misinterpreting what you see or b) you've got your workspace set up inside a OneDrive folder or something similar to that which is externally affecting the files

[arctic hornet]

Just a quick heads up... if someone would require my wisdom (not sure who, but still) ... I'll be mostly gone for next 20 hours... and then slow to respond for next ~2 weeks as I'm flying to US for couple of hours and then staying there for 2 weeks. I'll show up here and there, but no consistent presence like usual.

[narrow depot]

can anyone help setting up perforce? I have got connected via administration window but I cant seem to connect in the p4v window

[narrow depot]

here it says 1666, but idk if its just rerouting it to me
server adress says Amir-Laptop:1666 , does that mean this is what I have to give to other members of the team?
instead of perforce:1666 im assuming

[teal bone]

Assuming they're on a lan with you, sure. That's most likely your local network computer name.

If not, you'll need to set up a port forward on your internet router and give them your internet ip.

You can just google "what is my ip" and give them that. Or set up a dynamic host. You can google those.

[foggy oar]

We're getting pressured out of our long lasted contract of Plastic SCM (now Unity VCS <_<") and was wondering what you guys all use. I'm considering of running P4 on some internal server and just back that up, because normal git integration has no (?) extensions for file locking

[teal bone]

Git lfs supports file locking iirc?

Not saying it's any good, mind.

Perforce is great, but expensive.

[foggy oar]

I'm mostly fishing for suggestions and experiences currently, we're probably hosting that ourselves if possible, I'm just not ready to pay hundreds of dollars for repos with no traffic on them 🙃

[teal bone]

When we talked to perforce, they wanted $40/user/month upfront for a year.

And the discount for self-hosting was about $2.

If you can afford that, go with perforce. If you can't, git lfs?

If you have a small team, you can self host perforce for free.

[foggy oar]

yikes

[foggy oar]

we have a small team, I didn't find out what small means in their case.

[teal bone]

It was kinda funny. There was a "we'll host you with full support", "we'll host you with no support" and "you can self host with full support", but no "just let me run the damn software and figure it out myself" tier.

Small team, in pretty much every program I've come across, means up to 5 people.

[foggy oar]

Too bad!

Thanks for all your info however!

[teal bone]

Np

[thick knot]

Hello everyone!
In our project we are unable to Diff State Trees against the Depo within the editor. Any other asset/blueprint works just fine, we can see the diffing tool in editor.
We use Plastic SCM (now known as Unity SCM) and we are uncertain if it's a problem with unreal not being able to diff state trees or if it's a local issue.
Can anyone using SCM and State Trees confirm if they are able to diff them please?
Thank you

[foggy oar]

I'm getting struck by clown behavior.
Apparently their Domain lease just expired and plasticscm.com (that is the domain for all UnityVCS) is now ... parked. 'das right, everything is broken because their domain expired.

[foggy oar]

It's such a beautiful picture, they just didn't renew their domain.

[ocean anvil]

🤔

wonder if they realized and quickly re-purchased it, (if so why is it not just forwarding to their new URL?)
Or if someone decided this is a great opportunity to do a lil scamming

[unreal mirage]

Hey, I am currently trying to setup perforce for my project - including a binary build I made using the engine source.
But it seems like I cannot submit the entire engine at once. (even tho it is just 22GB)
One way around this would be to split up the changes and submit them as smaller portions, however that would take ages to do.

Is there maybe anything i forgot to specify or setup in p4 ?
1.7TB estimated seems a bit unrealistic, also as I dont need any history for the binary build at all

the server im hosting p4 on, has around 250GB

[foggy oar]

They confirmed to me that they had it expire and fixed it

Didn't help that our DNS cache was poisoned

[arctic hornet]

are you trying to push things like intermediates and binaries? those take TON of space... although I've never seen local repo take more than 300 gigs

[unreal mirage]

well the binaries yes, I dont want others having to compile the engine from source themselves

[arctic hornet]

in general that's a bad idea as full unstripped binaries can be 150 gig+ and it's hassle to keep them properly updated

I recommend taking a look at Unreal Game Sync and Precompiled binaries (PCB)

which in it's basic most form is a 2-4 gig file which UGS automatically downloads from perforce insteand of every using compiling the binaries themself

[unreal mirage]

AFAIK UnrealGameSync only supports syncinc binaries for uprojects and not the engine source itself

[arctic hornet]

well, yes and no... there are really no engine specific binaries you might need that are not part of the game at the end of the day

engine modules that are required by the game are part of PCB

[arctic hornet]

These for example are our PCBs... if I take this and put it to the source-only downloaded through UGS I'll get fully working game and editor (well... in our case, since we're building eidtor and game eexecutables for our PCBs, epic by default does editor only iirc)

[unreal mirage]

hm got it, so I would rather sync the engine source code but not the binaries at all.
And my team members can get the compiled binaries through UGS later then ?

[arctic hornet]

yes, it's a toggle in UGS, if you do not check the "Sync precompiled binaries" you will get the source code and UGS will automatically run local build after sync (if you let it to)
and if you choose to downloaded PCBs instead of compiling it will fetch the binaries

here

[unreal mirage]

it will fetch the binaries

Im not sure if I understood that part yet, you are saying UGS can fetch the binaries from somewhere?
But from where? If I havent uploaded them to my perforce server,

Like, where does UGS get the binaries from if i didnt upload them somewhere

[arctic hornet]

you upload them, just in a specific way where they're stashed separately from the main stream

and you can upload them to perforce, horde or cloud ddc (although cloud ddc will have this feature only from 5.6)

if you're using perforce for the storage then you just tell UGS where to fetch the binaries from like this

[unreal mirage]

alright thanks got that.

Just to be sure, that means I still upload the raw engine source code.
But to prevent others from having to build the engine from source themselves, i send them the compiled binaries through UGS which they then simply have to sync down to their machine ?

[arctic hornet]

and UGS then just fetches this automatically based on the description - the +F32 flag on the file itself means that it will only keep last 32 versions of the file and the rest will be removed (metadata will stay, but you won't be able to sync to anything older than that since the file no longer exists in perforce)

[arctic hornet]

you upload the source code in clean form (no Saved, Intermediate or binaries folder other than the ones that came from github) and they sync through UGS and get binaries if they're available (they're usually build automatically by some form of a build server that monitors changes in perforce)

and you sync from UGS too but let it compile locally

if that makes sense

[unreal mirage]

hm yeah I understand that so far, but I dont have a build server setup.
Would that mean I have to do the job of a build server on my machine and upload them manually

[arctic hornet]

you can mostly automate it by using buildgraph (epic even provides an exampe of that)

so you'd just need to execute one script and wait

[unreal mirage]

yes of course.

Alright but I think i got it so far

thank you very much, I appreciate your help !

I guess Ill try my luck and come back if anything doesnt go as planned

[arctic hornet]

I'm currently abroad for next two weeks so my availablility might not be the greatest, but there are others who can help you.

[unreal mirage]

Hm, Im also a bit confused about workspaces, should I rather have on workspace that depends both Engine and Project depot or for each depot one separate workspace?
How exactly is that intended to be used?

[arctic hornet]

general use-case I advocate for is this one, which closely follows Epic's style... one depot per major project where you have both the engine and such project (and arbitrary number of secondary projects)

and you then just sync this one stream using one workspace through UGS

although tehre are other styles (feel free to read the history of this channel) where you have engine and game separate and import the engine or game stream to the other etc... and share the engine source between multiple projects

[unreal mirage]

hm yeah, I have seen on depot per project plus one depot for the engine a lot of times.
I just wasnt sure about it, as I would then make multiple workspaces in technically the same location for engine and project.
Which kinda made the idea of having those workspaces sound a bit redundant in that case

[silver token]

What do you mean? You'd still only have 1 workspace for each project.
You can do 1 workspace with an engine and multiple projects but it's not recommended for shipping projects. It's fine for R&D, tests or prototypes, but once you're working on the project in a way that you want to treat it as a shipping product it should be isolated from other projects.

[unreal mirage]

Well if I am not mistaken it is required to have one workspace per stream?
Or at least I wasnt able to have one workspace that includes 2 depots working on the "Main/Dev" stream for each of these depots

[silver token]

That's true, you only work in one stream at a time.
However in the stream's advanced properties you can "import" other streams to a location within the workspace.
So your workspace stream would be your project and it can import the engine stream.

[unreal mirage]

ahh got it, so if i would be working on my game project I should just import the other stream that contains the engine source?

[silver token]

right.

[unreal mirage]

hm hm, this may not be the best place for that question but I'm just assuming you may know something about that.
If I have my project located in the same directory as the Engine content it gets treated as a "native" project.
So what happens is that it doesn't generate its own visual studio solution but rather use the existing UE5.sln

Is there an appropriate way around this?

[silver token]

I believe there is a switch you can pass GenerateProjectFiles to produce a project specific solution.
You could wait for VS to roll out their support for working with the uproject file instead of using the sln (though I don't know what the timeline is for that)
You could switch to Rider (or if you already do switch) and open the uproject file instead of the sln (Rider already supports this workflow)
Just live with it. I worked on projects that way for quite a while and it's okay. Not great but okay. Usually you just have to make sure you set the startup project correctly when you open the sln.

[unreal mirage]

alright got that

thanks for your help then !

[unreal mirage]

Just a quick question again, so the PrecompiledBinaries for both the UnrealEditor but also MyProject shouldn't be directly uploaded to perforce right?
So I would just put the Binaries+Intermediate folder into the .p4ignore ?

[silver token]

Sort of, it depends a little. You wouldn't have the Binaries or Intermediate folders in p4 that's true. You might still have them in a perforce location, you just wouldn't get them when syncing latest on your workspace.
This is one of the ways UGS can work for distributing binaries.

[teal bone]

There are p4ignore files already made for UE.

Even on the helix website.

[unreal mirage]

I found that one https://gist.github.com/jase-perf/b15b57d72fa8095c732fdbc4ca948903

[unreal mirage]

Welp I'm just a little confused in differentiating what it means having the Binaries / Intermediate folders in my p4 workspace but not syncing them?
Wouldn't I, when submitting the root folder, also upload the Binaries / Intermediate folders to p4 then too?

[wide raven]

Ignore them

[silver token]

No you wouldn't submit those folders (and they'd be in the p4ignore).
Usually what happens is that some other automated process would sync that stream, build the game, zip the binaries and put them in an entirely different stream. One that you're not importing from your project stream.
If you can't set something like that up, it might be easier to setup non-engineers with an easy way to sync and build locally (everything they need is freely available) than to try and push binaries and support both engineers and content creators.

[wide raven]

Since our project has relatively rare c++ changes, we have a cleanup .bat script, and all devs set up to compile the game when occassionally needed. It's fine for us, about a dozen devs. Haven't bothered with the hassle of setting up more advanced systems so far

I would presume it becomes more important if compile times start being atrocious, say 1 hour +, and happening on a daily/weekly cadence

[silver token]

Yeah, making everyone do local builds is a perfectly viable path. We're a dozen people and do have those systems.
But we do make lots of C++ changes (not to the engine).

[wide raven]

Do you have any rules for how that works, like only submit c++ end of day or such?

[silver token]

Those systems also make cooked builds.

No. Why would we?

[wide raven]

Hmm was just pondering if it might be a hassle if people happen to sync something that forces a rebuild, even if it was only 15 minutes or whatever downtime

[silver token]

Well it only affects engineers to start, so a sub-group of the whole team.
But engineers are generally cognizant of that cost and factor it into when they decide to sync.
UGS also has automation, so it will have sync'd and built for me before I come in.
You don't have to sync frequently during the day. Once in the morning can be enough. Maybe right before checking in, depends on the change, to build and try compiling all the blueprints.
Maybe you'd sync if the build is broken and you need the fix.

[wide raven]

Ah yes, clear

[narrow depot]

day 3 or 4 of having perforce issues

port is open but still cant connect to the port, OR my own ip address

this is getting exhausting lol

[arctic hornet]

I recommend removing this image too as I've said on the other server since posting ip/domain of unsecured perforce server is a bad idea 🙂

[narrow depot]

mb sorry Im sleepy lol

but thank you for the reminder!

[unreal mirage]

Hm so, I am currently trying to upload the Engine files to p4, I didnt checkout any Binaries yet..
But what about the .dlls from Plugins?
Is it safe to upload them or should I use UGS for those later as well ?

[silver token]

Dlls are the binaries, so you won’t submit them for a plugins either.

[unreal mirage]

oops, it appears I already did that..
I thought I would be good to go with using the .p4ignore from Epic Games to just naively submit the plugins folder

Well how would I submit the engine files then if just submitting it isnt the right way?

[pulsar parcel]

typically the engine on p4 will just be a clean clone with no build products (if you get the engine from github, then it should just be the state after running Setup.bat)

and then you'll use something like UGS to deliver binaries to non-coders

[unreal mirage]

yes but I built the engine myself as I thought that would be required to have the binaries that UGS then can sync for the others

or e.g. here https://training.perforce.com/learn/courses/314/working-with-custom-unreal-engine-builds-unreal-game-sync/lessons/1027/custom-unreal-builds-building-submitting I was following this guide

[pulsar parcel]

When you're using UGS it'll grab the binaries either from Horde (if you use it) or a zip file located somewhere external to the depot on P4

and yeah I'm not going to wade through a whole course to see what it's telling you to do

but what's shown in the screenshot ain't it

[unreal mirage]

Hm got it,
so just for the record, UGS syncs both Engine + Project binaries ?

[pulsar parcel]

yes, it expects you to be using a native project. the artifacts it expects to sync are the relevant engine and project binaries

there's even an example buildgraph script somewhere which shows most of the steps

https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Build/Graph/Examples/BuildEditorAndTools.xml

[unreal mirage]

right, i see

[unreal mirage]

For some reason I see others having a Native unreal project and still being able to generate a separate MyProject.sln file, rather than for me It appears to make use of the existing UE5.sln solution.
Is that the intended way or did I mess something up?

[pulsar parcel]

you should just not generate a project-specific sln and just use UE5.sln

you can open the uproject file in Rider, just make sure there's no "EngineAssociation" value in it

[unreal mirage]

yeah I think I got that

well alright, gotta try it out myself now

thanks for the help !

[unreal mirage]

Im currently trying to submit the engine source code to p4 for the first time.
But I am very unsure on how to do so.
Should I just clone it from github and after running Setup.bat simply submit it?
It seems like that way I am trying to submit too many files at once, p4 complains about some estimated file size is too big for what I have.
Splitting it into chunks would work tho.

But I thought I am surely not the only person who ever tried that so there might be a better/correct way?

[silver token]

Yes, you want to submit everything from github + what Setup.bat downloads.
Yes it's a lot of files. That error sounds like you don't have enough space at the destination and splitting it up would just result in a partial version of the Engine in source control.
I have to split it up across multiple checkins at work for other reasons, but for that error we've also had to bump up our AWS drive space.
I've never had to at home (the server is on my development machine in that case).
Welcome to engine source development. Mo' source, mo' problems.

[unreal mirage]

well I have around 250GB of space, I thought that would be enough

I dont need to sync the entire thing, It is fine for me to just upload a binary build of the engine with no history

[silver token]

If you're going to do that you're probably better off with the installed build of the engine. That's a weird approach.
I'm not sure how the space translates. I think my 5.5 stream is taking up 22GB of space just on data. I think there's more in the journal or something.
Maybe post the actual error message that you're getting.

[unreal mirage]

Thats the error it gives me

The thing im trying to upload has around 21GB

I dont know how p4 comes up with 1.7 TB

[unreal mirage]

The engine source basically

[silver token]

Based on that I don't know why it thinks you need that much space available. You could try from the commandline I guess.
Or you can try to upload it in batches ... but be ready to obliterate it if you can't get everything in for some reason.

[unreal mirage]

hm could it be that I didnt add a .p4ignore before submitting the engine?

[silver token]

Nothing in the git clone + setup should be ignored.

[unreal mirage]

at https://training.perforce.com I followed a guide basically step-by-step, I noticed that when he submitted the engine source it was way less files than for me.

I also setup my typemap following the epic games docs

so I honestly dont have any idea what I am doing wrong at this point

for reference it was this course

[arctic hornet]

the perforce training courses don't really follow best practices for seamless integration into epic's tooling

not sure if it got changed since way back in the day... but way back in the day they pretty much just did "git clone -> p4 submit" and never ran the setup.bat etc... so it complicated the use of UGS etc.

[unreal mirage]

So the "best" way to start is by git clone -> run Setup.bat -> submit everything to perforce ?

[arctic hornet]

everything except the .git folder itself

but otherwise yes

[unreal mirage]

hm, and that is around 40GB then, right?

[arctic hornet]

without debugging symbols that sounds about right

with debug symbols it can be around 100 to 150

iirc

[silver token]

Oh, I always forget about that folder 'cause I'm downloading the zip instead of doing any cloning.

[arctic hornet]

but you don't get debug symbols from git 🙂

[unreal mirage]

yeah It wont be necessary for my team

So I think thats the better way

[unreal mirage]

So I would just submit the entire engine at once ?
Im not sure if this will work because with my previous attempt, p4 complained about estimating the desired space of around 1.7 TB (if you haven't seen my message)

[arctic hornet]

doesn't matter if it's submitted in chunks or as one big change, although one big change makes it more conveniet

and honestly, god knows whether 1.7 TB calculation comes from

[hollow drum]

hey @arctic hornet when setting up p4 for UGS, you said to Merge from //UE5/Main to /ProjectName/Dev-EngineMerge, but it gives me this error Invalid parent field '//UE5/Main'. Check stream, parent and type.. I assume this is because I am trying to do a Stream to Stream copy with unparented streams, so should I do it with "Specify source and target files" ?

(thanks a lot for the help the other day btw!)

[arctic hornet]

there's the magical flag -F which forces this integration to happen, because normally perforce tries to enforce some opinionated rules that I recommend disregarding in this case

[hollow drum]

ahh the UI doesn't have it hahaha

[arctic hornet]

CLI for the win 🙂

although, honestly now that I think about it... if you'd make //UE5/Main into a //ProjectName/Dev-Ingest or something and make it of type "Release" etc... one could use default integrate/copy ruleset

I'll have to check that out properly when I'm not busy 🙂

[hollow drum]

I'll try that instead, since the -F flag somehow gives me the same error

[arctic hornet]

what is the whole command you're running?

[hollow drum]

p4 copy -Af -S //ProjectName/Dev-EngineMerge -P //UE5/Main -F -r or p4 merge -Af -S //Project/Dev-EngineMerge -P //UE5/Main -F -r

copied it from the log in P4V and added -F

[arctic hornet]

it should be copy or integrate rather than merge

copy, integrate and merge are three different operations

they at the end result in similar outputs though

[hollow drum]

in any case, the copy command gives the same error, which is annoying

[arctic hornet]

either way, it should be p4 integrate -F //Source/Stream/... //Destination/Stream/...

[hollow drum]

ah yeah so that's not stream to stream, that's file to file

-S would be needed for "stream to stream"

[arctic hornet]

I mean... this is the way you merge whole stream using filespec

[hollow drum]

yeah, I was trying with a stream spec

[arctic hornet]

my usual flow is:

• virtual sync of the ingest stream (p4 sync -k)

• reconcile offline changes on the engine source (p4 reconcile -m //UE5/Main/...)

• submit

• integrate main to staging

• resolve

• submit

• integrate ingest to staging

• resolve

• submit

• test things out, fix issues

• submit

• copy staging to main

• submit

[silver token]

There shouldn't be any differences if you use the stream spec vs a file spec. The stream spec should just allow you to specify less information (the destination or source is implied I believe)

[arctic hornet]

iirc yes, though I've never used the stream specs

maybe only when copying from staging to main

[hollow drum]

the stream spec complained about using the wrong parent, even with -F so the filespec worked better for me hahaha

[arctic hornet]

yeah, that's the perforce opinionated rules about integrating and copying certain ways between mainline, release and development branches

using file spec with -F you can do anything your heart desires

[hollow drum]

virtual sync was quite nice to copy without downloading everything (though I do need to download it now, but it's faster than download + submit)

[unreal mirage]

Sorry again, just to make sure, I upload everything except the Binaries folder?

[arctic hornet]

For clean upload from git, you want to also include the binaries folder since that contains ton of support executables

But for game no you skip binaries, intermediate and saved

[unreal mirage]

well for me "clean" as all I did was running Setup.bat

[arctic hornet]

Yea “clean” I.e sync + setup.bat

But nothing else ran

Since just running generate projects etc… will introduce ton of files that you don’t really want

[unreal mirage]

alright got that, thanks

[oblique palm]

hi there - im having a bit of trouble setting up perforce version control

can a workspace root directory be anywhere on the drive? or does it need to be under program files > perforce > xxxx

[ocean anvil]

it can be anywhere, preferably the full path to your root directory isn't too long as ue has a max character length for filepaths

[oblique palm]

cool yeah i might try to go shorter

[oblique palm]

if i want to move assets into a new folder in a perforce version controlled project, i can do that without breaking it right?

[silver token]

Yes, just be sure to move them in the Content Browser. Don't do it directly in perforce or in explorer.
It won't immediately look like the file has been moved because the old file will become a redirector and a new file will be added, but that's what a move looks like.
Eventually you can fixup the redirector to make sure any references point at the new location and delete the redirector.

[oblique palm]

excellent thank you

how do i fix up the redirector?

typically when i move stuff in content browser, it just moves with no redirector made

[silver token]

You can run a commandlet, you can right click a specific redirector in the browser or you can right click on a folder.
If there aren't any references it might not create one.

[oblique palm]

cool i'll look into that

thanks!

[oblique palm]

hi again - i made a perforce account for a collaborator to log into but when they try to log in it’s saying invalid pw

a friend and i were both able to log into this account so we’re not entirely sure as to why it’s saying invalid for them

the server info and usernames are both correct and the pw they’re inputting is correct

they aren’t based in the us like us though so could that be the issue?

[arctic hornet]

if it's saying invalid password it gotta be invalid password, there's not much else to it... if they're copying it, make sure they're not copying any whitespaces at the start and and the end (that goes both for username and password)

[dry marten]

Hi !
Speaking for a small (3) group of friends getting into unreal.
We're looking for a way to share our files, ideally free.
So far we came across Perforce (not free) SVN (must setup a server so not free) and Github.
Github is ok for now since our repo is only 2gb large, but will stop being free when LFS is needed.

We are willing to put the money into a tool if it is really needed, but want to consider all options first.

So,

• What source control would you recommend for a small team new to UE5 ? Most of us have IT background so familiar with programming/web hosting/git
• Are there any good practice guide for keeping repos light ?

Any help appreciated, thx 🫶

[long hull]

Perforce is free for <5 users IIRC. But you need to host it somewhere. If you're looking to get into gamedev, it might be worth investigating, as Perforce is the standard in the industry.

If it's more of a hobby thing and you're all already familiar with git, it could make sense to just use that, it does the job. For free hosting, people prefer Azure DevOps as it has like a 250GB limit, which is way larger than GitHub.

To keep repos light, consider using One File Per Actor, as it avoids having huge map files, as only modified actors needs to be submitted, and not the whole map (also makes concurrent work easier). LFS also makes it easier to reduce the size on disk on the client side, as you can just prune everything and only keep the working copy.

[dry marten]

Yep for perforce I meant "the hosting has to be payed for", sorry for being unclear

Thank you for your answer, really helpfull ! We'll make a decision and go for it

Other related question.
While looking for tools/guides on source control for UE5 I red a lot about Anchorpoint.
I started a free trial just to see and it is pretty handy, but is it really needed if everyone in the team know the basics of commit/push/merge etc ?

As I see it for now it is just a clickable interface for git

[woven sluice]

hosting is up to you. you can run perforce on one of your local PCs and set up a VPN via tailscale/zerotier to join everyone into the network. but obviously you'll want to keep that PC on 24/7... so this might only be a good option if you don't have expensive electricity where you are

amongst the other giants there is also Diversion (i am unaffiliated, i have never used it, i have no opinion on it) but from what i know it's a different service, you won't host it yourself, it also will become paid as project grows, etc.

[dry marten]

Considered self hosting but electricity is a little too much sadly 😅

[dry marten]

And thx, I ll check it out !

[haughty yoke]

any suggestions for a perforce host that has larger storage for a team of 3?

like 80gb?

all the ones I know only run to like 10gb on a budget

[arctic hornet]

If you are EU based you could try hetzner, they have servers in Helsinki and Frankfurt… and servers from server auction can be like 40€ a month for quite a beefy server

[woven sluice]

and for 3 people and 100 GB you only need a potato, not a beef

i'm using gandi.net but i think if i was setting up new today i would be shopping at Hetzner and Linode

or did linode get bought out too, now it's "akamai" ... annoying guys doing good startups and eventually getting bought out and the service turns to ass

[quartz patrol]

After Connecting perforce with Unreal engine then unreal starts freezing when performing blueprint saving or other basic operation. Any suggestion how to fix it?

Thanks for your help.

Do you have any solution for this?

[arctic hornet]

Well… at hetzner its either 40 bucks and potato … or 40 bucks and relatively beefy… so Id opt in for beefier… and linode is still linode even though it got bought by Akamai… One of my companies has about 300 servers there since before it got bought and the only thing that got better is halving of egress pricing

[arctic hornet]

Well… that usually indicates slow perforce server… because unreal is doing and “fstat” query (file status) on the file etc… and it has to wait for the result… there’s not much you can do

[pearl star]

Howdy has anyone tested Git filelock in a project?
If so does it work with IDE and other tools like Blender so you can just rightlick and lock files similar to perforce or is it CLI only ?

[teal bone]

I do not, sorry.

[unreal mirage]

Hey, so I am currently still trying to make UGS work for my team.
We do have a custom engine source build, however I am only distributing an Installed Build that I previously created using the BuildGraph.
Now when I try to create and submit PrecompiledBinaries for UGS it seems like a bunch of scripts are missing in the Installed Build.

Is this generally the wrong attempt that I am doing?
Or does UGS require to have the actual source build and not the installed version of it?

[arctic hornet]

UGS is wrapper on top of perforce… people sync your depot with source build and then either build locally or let UGS fetch corresponding precompiled binaries

[unreal mirage]

meaning that uploading an engine Binary Build is not an option ?

hm

okay got it

so I guess I have to obliterate my current engine depot

[arctic hornet]

The intended way is:

• clone from git
• run setup.bat
• push it all to perforce (excluding.git folder if exists)

And then its just PCB (precompiled binaries) and UGS

Also obviously upload your project

Where in general ignore Binaries, Intermediate and Saved folders… for all modules and plugins… unless they contain external dlls that are not built directly from source

[unreal mirage]

But wouldnt that mean that UGS would upload engine binaries every single time even if I didnt touch the engine source ?

[arctic hornet]

UGS is only responsible for downloading the binaries… you’re responsible for building and uploading them (well… usually its your CI solution that does it)

And you’re only required to submit new binaries for changes that are tagged as “Code”

It’s not necessary to build new binaries for “content” only changes

[unreal mirage]

yes right, I think I may have not expressed myself correctly,
for instance when I want to submit stuff to my binary stream, I can do this by running the BuildEditorAndTools.xml BuildGraph script.
which wants me to specify the -set:EditorTarget= argument.
So lets say I made changes in my projects code, when running the script, wouldn't it also compile the engine + project even if in some cases the "old" engine binaries are still up-to-date ?

[arctic hornet]

PCBs are one big combined archive for Engine, Editor and Game modules

[harsh tree]

Hello I'm trying to find a good free program to use for a group project. I was going to use git but figured I would ask in here if there was anything better/easier to use.

[arctic hornet]

If you don’t mind hosting yourself then I’d recommend perforce… its free for up to 5 users and it’s de facto industry standard

Otherwise probably git is your only other option

[harsh tree]

I would do the git but I have more than 5 people

[wide raven]

I would say perforce is the only way to go for a serious project, but indeed not free above 5...

[unreal mirage]

So I think I made progress with UGS.
However now that I am trying to build and submit my project to perforce it seems like it cant find MyProjectEditor target.

Engine\Build\BatchFiles\RunUAT.bat BuildGraph -Script="Engine/Build/Graph/Examples/BuildEditorAndTools.xml" -Target="Submit To Perforce for UGS" -set:EditorTarget=MyProjectEditor -BuildMachine -set:ArchiveStream=//MyProject/Main-Binaries -p4 -submit

with error message

Couldn't find target rules file for target 'MyProjectEditor' in rules assembly 'UE5Rules, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'

Location: F:\p3\UE5\Engine\Intermediate\Build\BuildRules\UE5Rules.dll

im not really sure how to resolve this issue tho, I cant seem to pass in the full path of my editor target in the -set:EditorTarget= argument

[warped loom]

Hey, been trying to set up source control the "unreal way" for a team, wondering if there's any docs on how to actually set up robomerge? I can see there's a docker file for it, but do I need any config set up outside of that or authentication etc.?

[unreal mirage]

well it seems like I forgot to GenerateProjectFiles after adding my project to the workspace

[marsh portal]

What's a good strategy for taking in engine changes and moving a source build from 5.5 to for example? It seems like the the merge base is actually quite old and these releases diverge a lot.

I'm using a submodule for the engine source and I plan to simply repoint that to latest but I'm curious if there is actually a merge strategy that is expected?

[wide raven]

There's several strategies, I think they have been mostly gone through in #engine-source channel

[marsh portal]

ah ty

[wide raven]

sadly searching on disco can be a bit tricky, we should really probably try to pin some of the most commonly accepted ones

@marsh portal
Here's one of mine in this channel : P
Looks like we are a bit mixed up on this area.
Mine is not the most common, but I think there were several options before and after this post
#source-control message

[wide raven]

I think robomerge is kind of advanced stuff, maybe not used by smaller teams

[arctic hornet]

Unless you are actively working with multiple streams robomerge won’t be of any use to you… but otherwise there’s not much to configure… point it to perforce server and set up branchspec on there based on the example provided in there

And possibly also just update the links for slack, swarm etc. in the past those were hardcoded but iirc nowadays stuff is configurable through env vars but don’t quote me on that

[arctic hornet]

Robomerge doesn’t really rely to team size… but rather its related to how many streams you have… the moment you work with more than one stream at once, it’s worth setting it up

I should add “multiple streams for the same project”

[vague mulch]

Hello. In 5.6 in the lower right corner actions menu, when I would make a change to a file "submit changes" would appear. It no longer appears and I need to go to view changes to submit the change list. Is that a new behavior?

Sorry in versions previous to 5.6.

[vague mulch]

I’m still trying to resolve a Game Feature Plugin editor issue with asset validation and source control. In a GFP I am referencing assets like /Engine/BasicShapes/Cube and /Engine/EditorMaterials/GridMaterial. When the validator runs I get <an asset> References /Engine/MapTemplates/SM_Template_Map_Floor which is not marked for add in source control. (EditorValidator_SourceControl). Any ideas? This worked fine without validation errors before in other projects and I am wondering if there is a way to fix it here.

[arctic hornet]

Are you using source build of the engine… or one from epic games launcher?

[vague mulch]

From the epic launcher and I am also inherting stuff from Lyra.

[arctic hornet]

Ok, that’s probably the issue… since validators are written in a way where they expect engine code to also be in version control - so it will complain any time you use engine assets…. I’m not sure if there is a way to change it ignores the engine content (as I never had to check for that)

[vague mulch]

huh interesting. I had the same thing workin in 5.5 with the same GFP and Perforce being used and it wasn't causing the validation errors.

I did copy and paste the GFP into the 5.6 project. IDK if that could be causing the issue now.

For future reference; I figured it out. It's actually inside the LyraEditor validation files. Line 53 of EditorValidator_SourceControl.cpp.

That section of the code (accourding to comments) checks for assets that are submitted to source control that references assets that are not.

Commenting it out will kill the validation rule for assets in that type of situation but it does allow assets to be submitted via the editor.

[faint cairn]

Fwiw you shouldn't reference engine content.

If you need that cube etc, copy it into your project instead.

[arctic hornet]

There’s nothing wrong with referencing engine content… albeit it can be risky if not using source build

[faint cairn]

Okay, sure you can reference it, but there are often cases where content gets modified. Such as a Material being adjusted, a StaticMesh getting updated Collision and what not. And while you can you shouldn't really modify that content.
It's way safer and saner to simply duplicate the given asset into project level. That resolves any future accidental changes and what not.

[arctic hornet]

In certain cases that’s wanted behavior… but yeah understandable… if you don’t know your engine perfectly then it’s probably better idea to copy.

[faint cairn]

Gotta say, in the past 10+ years, I have yet to encounter it to be wanted behavior to modify Engine Assets instead of just copying them.

If I want reusability I use a Plugin that holds the content.

[silver token]

Yeah, you shouldn't edit engine content but a copy-on-edit strategy should be fine.
With some perforce setups you can even prevent checkins of engine code/content.

[faint cairn]

Pretty sure the project I'm currently working on even has validators that prevent Engine content usage.

Not saying that's the norm though

But not really something worth arguing about tbh.

[arctic hornet]

What ramius said, but also I’ve done it couple of times especially when doing engine tweaks and optimizations deep within the bowels of the engine, as that sometimes requires also changing assets… but not many people tend to do open heart surgery on their engines

[faint cairn]

To be fair, if you modify the Engine and that results in some of the default assets to require being fixed up due to those changes, then that's a totally different story.

Like, that's completely expected and isn't what the initial suggestion to not directly reference Engine content was about.

[arctic hornet]

True, I’ve got quite offtopic… my bad 🙂 but yeah we generally practice copy or edit… especially for prototyping etc. but I’m pretty sure that all of our production assets at the end of the day are custom assets

[cursive mulch]

Do you guys use any automated way to version your project with Git?
I'm evaluating some alternatives.

[arctic hornet]

what do you mean by "automated way to version"

versioning is generally done by the users, since you need to push and pull etc... so there's not much to automate

unless I'm misunderstanding your question 🙂

[cursive mulch]

https://en.wikipedia.org/wiki/Software_versioning

[arctic hornet]

still, in what way... like automatically creating branches, raising the numbers or simply appending commit hash to the version.... (most of these are done during CI rather than in git/version control itself)

though let me answer all three:

• branches: don't recommend doing that nor do I know any way to "automate" that
• raising versions: don't really recommend that since version is generally tied to unreal engine version nor is there any sane version to do it directly in git... with CI pipeline you could more or less consistently raise your numbers by modifying build.version and couple of other files (or calling UAT script)
• appending hash: if you're using epic games store engine build there's nothing youcan do, since the version is hardcoded in the binaries... if you're using source build it should get auto-appended whenever you build as Major.Minor.Patch+GitSHA iirc.

[teal sail]

has anyone encountered p4merge having an empty base?
I have streams:

- main
    -> ThirdParty (dev)

ThirdParty is coming in just as a big unversioned dump, and I'm trying to merge it to main as the histories have diverged, but only by a couple hundred lines or so. When I do so, the merge utility just leaves the base field empty for the 3-way merge, and so the diffs are trashed.

My understanding was that perforce should use the last common revision in main as its Base, since the streams are related to each other. What am I missing?

[candid cobalt]

Hey guys. I updated my project to 5.6 Now I want the new project to be in my repo instead of the old one. Can I just swap out the entire project in my local repo and push the "changes"? Will this cause any trouble?

[pulsar parcel]

?

when you upgrade, you should be converting in place, and just pushing that

[candid cobalt]

Unreal prompted me to make a copy of the project before opening it in 5.6 and I did, just in case...

[pulsar parcel]

your "just in case" is your source control

[candid cobalt]

yeah ik x). lets call it force of a habit.

So I can just connect my upgraded project to source control and push the new project into my repo

[pulsar parcel]

well it would've been easier to convert it in place

[candid cobalt]

I'll do that next time I upgrade. for now just replacing the old project and connecting and pushing the new project worked.
thank you ❤️

[pulsar parcel]

for the most part, a conversion is just changing the EngineAssociation in the uproject file

[finite quest]

Hello everyone. I need some advice and help.

I am trying to set up soure control for one of my projects, which is very large in size (over 30 GB), and I would like to do it without purchasing any subscription. I have found that Perforce can allow me to locally host a server and make it act as a source control but it has a file limit of 1000 per push afaik but is there any guide that someone can provide helping me set up my UE5 project with perforce on my old PC that will be acting as the server?

Or is there any better alternative option, I have heard Microsoft Azure is good but I have never worked with that

[silver token]

Where are you getting this 1000 per push file limit?
I have perforce set up this way for my hobby projects and the 5.6 upgrade involved submitting 300,000 files at once.
Worked fine.

[finite quest]

oh chatgpt told me 😂

but if thats the case then hell yeah im using perforce I love it

[wind gazelle]

are there any tricks breaking down repo clone, like partial.
git clone constantly interrupts due to poor connection during "Receiving objects"

$ git clone https://gitgud.io/xxxxx/xxxxxxxxxxxxxxx --depth 1
Cloning into 'xxxxxxx'...
warning: redirecting to https://gitgud.io/xxxxx/xxxxxxx.git/
remote: Enumerating objects: 850, done.
remote: Counting objects: 100% (850/850), done.
remote: Compressing objects: 100% (820/820), done.  
Receiving objects:   0% (8/850), 932.00 KiB | 41.00 KiB/s
// after 10 or so minutes of "receiving objects"
error: RPC failed; curl 56 OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0
error: 16451 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

i've tried also the git init to make repo, add remote and do fetch, but still end up with various SSL errors

[stark oracle]

Are there any known issues when using the perforce server in unicode mode?
I did not found any information beside @klukle's recommendation for a non unicode setup
I'm using perforce to store the engine and projects. Additionally I am using Horde and UGS

[pulsar parcel]

might be an urban myth? but looking at the docs, the only reason you'd need unicode is if you need to store metadata in unicode (usernames, CL descriptions, etc), it doesn't affect the ability to version text files as unicode

[arctic hornet]

yeah, as sswires says, it's for supporting unicode characters in file names, which in most cases is not really necessary...

at the end of the day it doesn't really matter that much... I just recommend non-unicode because A) I've had random weird issues with unicode in the past (long long time ago, like 10+ years at this point) and B) Epic is running non-unicode... and since my goal is to provide stuff as close to epic's... it makes sense to go non-unicode

[stark oracle]

Thanks, that helps!
As some users would like to use unicode in CL Descriptions, I will clone the server and change it to unicode to test for some time, lets see how it goes 🙂

[arctic hornet]

just be careful, some of epic's tools don't really like anything that's not basic ASCII and will through garbled mess - so just a heads up 🙂

[arctic hornet]

<@&213101288538374145> ^

[pastel coral]

I've been looking at completely restructuring my git repos and want to keep my raw assets files as a separate submodule to keep them out of the main repos but I kind of actually don't really want source control for the raw assets, I really just want a cloud-based backup where they can be pulled from as needed. Should I just use something like gdrive and API access to pull them instead of wasting git lfs storage in that case? I would love source control for them but not the way git handles it; I'd want delta diffs and I only know of unity devops that can do that and really don't want to pay for that right now.

[ocean anvil]

use p4, set them to only keep 1 or 2 revisions it will function more like a cloud storage/short term back up.

[pastel coral]

It's not much storage out of the box from what I see in pricing. Do you happen to know how much additional storage costs?

[ocean anvil]

depends on the host, if you host it locally its however much storage you can cram in an old pc lol.

[pastel coral]

I definitely want cloud storage (or hosted rather) for now. I was figuring gdrive because they throw tons of storage dirt cheap and I can use their API to pull via scripts.

[pastel coral]

I saw mention somewhere that Epic uses Amazon S3 as a CDN sort of for their binary engine content

S3's prices aren't too bad either

[long mural]

anyone eve seen The filesystem 'UnrealEngine' has only 1.3T free, but the command estimates it needs at least 1.8T available.

its not 1.8tb

[arctic hornet]

unless you're trying to push 1.8 TB it shouldn't be an issue

I've only ever seen it once from someone, not sure how/if they fixed it

[long mural]

i had to break the commit to smaller chunks

was weird

was 150gb commit on lot of files

but wanted 1.8tb

confusing

[unreal mirage]

yeah me too

No idea why though

I also split it up into smaller chunks, very annoying

but unfortunately no idea on how to fix it

i just accepted it..

[arctic hornet]

<@&213101288538374145>

[arctic hornet]

@unreal mirage @long mural could you tell me what p4d versions you have… and what file systems etc… I’ll try to dig bit deeper… as it could ve a bug in certain releases of p4d… filesystem compression or something weird like that

[unreal mirage]

I am on the latest p4 version (2025.1/2761706) and my p4 server is using ext4

[arctic hornet]

Hmm… I’m not aware of any known issues of this type In latest version of p4d … so it is indeed weird… are you running in a container or on physical host etc?

[unreal mirage]

welp, yeah basically a VM somewhere in the cloud

[arctic hornet]

Yeah... honestly... weird... I'm running zfs and currently on 2024.2 p4d and I've never ran into this issue until now

could be a bug in 2025.1 version but I doubt that

[unreal mirage]

hm its very annoying honestly

It surely did work by just splitting it up into smaller chunks but it was a big time waste imo

[arctic hornet]

I know that 2024.1 works fine with large commits since I've done UE 5.6 merge which was 120k files, although my p4 server has about 2.8 TB of disk available so it's hard to say if I'm having the same issue

[grim heron]

Github recently changed their model to metered instead of packets. I noticed I am now being charged per day for storage. I'm just wondering what kind of costs people are seeing for downloading files from their repo?

[slow quiver]

Ellow there fellas
So our team is currently exploring the use of Perforce as our primary version control system for Unreal Engine development. We’ve successfully set up a local Perforce server hosted on a PC and connected our development machines over a LAN. Now, we’re planning to upgrade to a dedicated mini server to store our projects.I don't have much knowledge about servers, so I would love to get your opinion on the server configuration below.

[teal bone]

An Epyc seems overkill for a perforce server.

An Epyc with only 64gb of ram seems a waste.

Is that a single stick of memory?!

[vale bronze]

what do you need epyc cpu for?

Its a cpu for large multithreaded operations, perforce just needs IO, fast internet mostly

[arctic hornet]

There are cases where s lot of cpu horsepower is useful…. Especially when doing large commits… and its 32 core (64 threads) “only” so its not overkill depending on team size etc.

[arctic hornet]

What is your team size…specs seems mostly adequate for larger teams but kinda overkill for smaller team

Also I’d highly recommend getting SSDs rather than HDDs because perforce performance will depend A LOT on the speed of the drives and spinning drives even in raid usually isn’t enough

Also also… might be better to get at least two sticks of memory or ideally 4… epycs love their multi channel memory

[sand rover]

Hey, what's the recommended way to manage different UE source streams in P4? I'm thinking about project specific engine modifications and more studio global modifications to be used across multiple projects.

I'm currently thinking there would be a general stream at say //engine/Engine/... and then project streams at //project1/Engine/... and //project2/Engine/... containing both the Engine and project folders. Then there'd be some merging action going on between projects and the general engine whenever the need arises.. does it sound stupid?

[silver token]

The top pinned message has a link to Epics tutorial which should be your starting point.
But in general no, it doesn't sound stupid and is more or less what Epic recommends.
If you have questions after checking out that doc, we'll be happy to answer any questions but you should really read that first.

[sand rover]

I might be missing it but I don't think it mentions any strategies for juggling different versions of the engine. I mean it's easy to just let each project maintain their own version but I'd also want to have one project agnostic version I guess, to act as the base version for any new projects etc.

[silver token]

Yeah I guess not that specifically.
Okay, one sec ...

Here's what I have at work (& sort of at home):

Stream 1: numbered versions of the engine source I've downloaded. So //stream1/5.5, //stream1/5.4, etc.
Stream 2: a pristine copy of the engine that is the current latest that I've downloaded. So //stream2/ (which is identical to //stream1/5.6 right now)
Stream 3: a merged copy of the engine where company wide Engine modifications are introduced.
Stream 4a, 4b, etc: a staging copy of the engine that has company and project engine modifications
Stream 5a, 5b, etc: development copy of the engine that has all modifications and is what the rest of the team uses and builds against.

This setup is intended to limit the number of reconciles that take place (because we usually only take 1 hotfixed version)
So new release occurs, I download that and but it in a subfolder of Stream 1
I do a p4 copy of that subfolder to Stream 2. This generates a CL that is the 5.X -> 5.Y diff/reconcile.
Merge Stream 2 into Stream 3. Do engine update tests here with a project and company plugins to update to new engine version.
Merge Stream 3 to a project's Stream 4. Merge Stream 5 to Stream 4 with updated project changes. Make the project work on the new engine.
Eventually copy Stream 4 back to Stream 5 (along with other project streams) as the roll out to the rest of the team. This is the step that is really the "project has been upgraded to the new version).

Of course you don't have to go nearly this far. Lots of people skip Stream 1 and start with Stream 2 and then reconcile the new engine source code/data against the version in perforce.

[sand rover]

Thank you! That is similar to what I had in mind then. I wonder why you want to keep all downloaded versions in stream1 though? Is there a workflow benefit? Otherwise I'm thinking the downloaded versions could be tagged/labeled in stream2 (to get to a specific downloaded version)?

Do you also submit modifications directly into stream4 and stream5 (projects) and cherry pick changes into stream3 ("general" engine)? Or is the flow always from Epic downloads -> stream1 -> stream2 -> 3 -> 4 etc?

[silver token]

It makes it easier to compare across versions I guess. Makes it easier to create a project on a specific version by branching a specific folder instead of trying to sync stream 2 to a specific CL. But we haven't had to do anything like that. But mostly because of the skipping reconciles which take take a long time especially over a large depot like the engine. If you do the upgrade in place, you have to sync 5.5, replace it all with 5.6 and then do a reconcile off-line work. If that's okay for you, do it.

Stream 5 gets project specific modifications to the engine as part of regular project work. Stream 3 gets modifications merged from project Stream 5's (in theory after some sort of review to approve it for company wide release, but we're too small for that and only really have 1 project). Stream 4 primarily gets changes from merges of Stream 3 and Stream 5 into it. But it's where any project specific Engine changes may have to be updated to work with the new engine version.

[sand rover]

Thanks, gotcha. Yea the reconciles with UE are really slow.. I've recently been testing this tool which is quite a lot faster than the regular reconcile, and has worked well so far: https://github.com/brickadia/p4-fast-reconcile

Ok that sounds like what I have in mind then, thank you so much for the info I really appreciate it!

[silver token]

Yeah, I've seen that tool as well. We'll probably use it if we ever have to take a second hotfix version.

[slow quiver]

Yup it was, now we tried making it 4 sticks of 32gb

[slow quiver]

Actually we are a research lab, so our current team size is around 10 members. We basically want to store all our projects into it.

We also changed the config to include 4 sticks of 32gb ram

Also my apologies for late response

[woven sluice]

perforce is going to use approximately 1% of your RAM and 0.5% of your CPU. maybe you intend to use this as a dual-purpose perforce server + build agent to justify this though?

[arctic hornet]

Our server we use for everything has 48 threads (24 cores), 192 gigs of ram and 10 TB SSD raid... we're running whole studio infra on it: perforce, horde (server), cloud ddc, UBA cache, jira, confluence, vpn server, bunch of other internal tools and we barely use 25% of our CPU on average

RAM... yes... we do use that, because databases like to eat ram for breakfast

[arctic hornet]

I wouldn't go build agents + perforce, since build agent will take every single ounce of resources you give to it... which will wirtually kill perforce

but also, it's nice to have some peak compute power available for perforcfe server, when doing large commits perforce if you let it... can eat almost everything too

This is perforce server load when pushing 300k+ files at 5+ GBPS speeds

[woven sluice]

hopefully virtual machines to divide up the perforce and build machines. but yeah i mean otherwise agree on all that.

5 GBPS must be nice, but also barely needed 364 days of the year for most people. you'd fill up your 10 TB storage in a few hours of "work" so obviously you don't really need it but i bet it sure feels nice. lol

uploading to my absolute potato perforce server at 0.1 GBPS here ¯_(ツ)_/¯

[arctic hornet]

it's really nice for ingesting new engine releases... but yes, 99.9% of the time the server just chills at least as to what perforce stats go...

and that's also why the same server is hosting the rest of our infra... which albeit dangerous - both from security and also failure prevention standpoints - at least utilizes the hardware a little

I've bought that server refurbed for ~1k USD about 5 years ago, shoved couple hundred bucks of SSDs into that server and it's been chilling ever since

[woven sluice]

hey if project zomboid made it work (at least initially) by storing everything on a single laptop that got stolen then us today indies can make it work with a single standalone worker machine

[arctic hornet]

I mean... I've had one time when the raid card decided to blast garbage all over the raid and thus kill it... luckily even though I'm running everything on one machine for prod, we keep offsite backups

so we lost like 2-3 commits thanks to our low rate of daily commits

well... I do keep perforce backed up... the rest is just yolo

since nothing much happens if I loose my jira tasks

I'd by happy for that actually

[amber sentinel]

Hey all.
I'm not having any luck with google, since it keeps on thinking I'm preventing adding, and to use the .p4ignore (already tried, and doesn't have desired outcome)

How do you either:
Prevent an modify action/ reconcile offline work, from detecting files that change all the time (aka, .dll build files)
Or
Basically same outcome, but always force the revision #1 to download, as to now constantly get latest random extra files, such as .dll from builds changing like 1 bit in them?

[wide raven]

You should be ignoring stuff like build folders, intermediate and binaries?

[amber sentinel]

I did.
But, its required to include some, from the initial engine download, so building works fine, since the raw engine source on Github includes some binaries content.

So, those couple ones that are required to be included, trigger modify actions.

Its correctly ignored, since I can't add any new binaries content, but any that were required to be included, still popup when modified.

[wide raven]

Post a path example?

[amber sentinel]

I assume you mean a modify entry, such as:
e:\Asharavel_Engine\Engine\Binaries\DotNET\AutomationTool\EpicGames.Core.dll

Hmm.
Now that you make me double check.
I see thats missing from the engine source on github.
But, I do have an ignore setup. Odd

I'll need to go over it again then.

[wide raven]

Ok, so we are talking about your engine project, not game project right?

[amber sentinel]

Yeah, my engine.
But, looks like I maybe did a step wrong, when adding my engine source to, to my source control, and maybe added at the wring step.
hence all the extra .dll stuff

[wide raven]

yeah, sounds like something is bork

[amber sentinel]

100% out of my element for how Perforce works.
and, also just it being new to me.

But, that previous attempt, I just did some scuffed copy/ paste of folders.

But, now I'm trying the "official" route of 3 separate repos, for base engine, merge, final modified engine.

And, I just don't quite understand how to merge/ resolve.
I start a merge, but it starts to merge 100% of the project.
I don't quite understand how to just merge my file edits. Since, I have a clean engine source, and just want to merge my couple custom engine edits.
But, it starts to merge 100% of the entire engine.

[sand rover]

Is the target stream/folder already populated with files? If not that might be what you are seeing (p4 populating the target destination with all the files from source)?

[amber sentinel]

Target is populated with the clean engine, and source is my currently modified/ semi messed up engine, with only a couple source changes

[sand rover]

Hmm why do you want to merge your messed up engine to the clean one?

[amber sentinel]

Its messed up with what file are version controlled.
But, it has a couple engine C++ edits.

So, if I can see the modified files, aka the like 8 C++ edits, I can merge them, then just upload that on my main, so that its all version controlled properly

[sand rover]

Aha hm, somewhere in the merge dialog there are options to specify merging specific submits .. and I think options to limit the merge to specific parts, like just the Source folder or such

Are your c++ edits in their own submits? Or submitted together with all the DLLs etc?

[amber sentinel]

Randomly spread out submits, as I figured out how to make those edits.
I don't think there with other stuff, but I still need to re-find them.

Surely theres a way to just do like if files are different, add to pending, else do nothing with it

[sand rover]

Yes you mean the "Reconcile offline work" option maybe?

[amber sentinel]

Thats for my current work.
But, i'm talking about current main edits stream -> clean engine stream

[sand rover]

That's what the merge/integrate option should do with detected changes between two streams or folders

The diff between source and target, placed in a pending changelist

[amber sentinel]

Is it some setting I'm miss-understanding?
I had also assumed the merge would do that by default.
But, using the merge right now, does the entire stream -> entire stream

[sand rover]

Yea it does the entire stream by default

But in that dialog where you select source and target the are more options to slim it down

You can also edit that pending changelist and remove all files that aren't .cpp or .h etc

To me it looks like either your clean target stream is not populated (can you sync files from it?) or you have 200k "wrongly" added files in your modified source stream and the merge wants to add them to the clean target stream

[amber sentinel]

1. I don't see any dialog to slim down the merging, at least that direct suggest like only keep changed files for resolve sort of shit.
2. Editing the changelist is a valid idea, just assumed there was some default like only keep files that are different
3. Not sure what you mean by sync files in this context. 😛 But, the files exist in my Workspace, and in my Depot UI.

[ornate willow]

if it helps, .p4ignore is for linux, on windows it looks for p4ignore.txt. not sure if thats been changed in recent years. i use the typemap to keep engine, project and plugin binaries writable for others without VS, along with other always changing files like DCC content, they don't get added to CL unless changed

[amber sentinel]

Unsure if I changed that myself, but the ignore is working, since it says I can't add, because of my ignore file.
But, I guess ignore is for adding, not editing.

[sand rover]

If you view that pending changelist, does it contain files that exist in both source and target, and marked for edit and not add?

So in your depot ui for the clean target stream, you see that it contains a bunch of files?

[amber sentinel]

If I single out 1 file, it shows this.

And, if I run the merge tool, it shows the 2 different files, and picks them up correctly.

[sand rover]

So that's a valid change you want to merge?

[amber sentinel]

That has no change at all.

Simply just showing it off, how it wants to merge it, yet theres zero differences.

Lemme go single out a file that I did make an edit too.

[sand rover]

Wait that looks like it doesn't exist on the target

[amber sentinel]

Left says its source
Middle is "base" (so, I guess previous revisions?)
Right is Target.

As shown here:

[sand rover]

Hrm tricky to follow on phone

[amber sentinel]

Yeah, when I run it on known different files, it shows the difference, and its attempt at auto resolving.
So, it knows theres changes.

[sand rover]

But middle is empty which makes it sound like they don't have a common base version? Are the streams completely separated or is one branched of from the other?

[amber sentinel]

There I guess "childs" of the mainline stream, and there Development streams.

A quick google suggests "base" is some form of like "previous revision" sort of shit, given:
I'm told that Base is the common ancestor of your version and the version you are merging. And Theirs is the version you are merging.

It sounds like base is used, when merging within the same stream, such as 2 people doing C++ edits

[livid rover]

Hi, quick question here, is svn still good for version control or is it getting too old ? I think I’m starting to truly hate perforce

[wide raven]

Some beginner friends of mine tried to use it with unreal a year back and ran into trouble with it quite quickly, but I was too busy to untangle them at the time. I would say just stick to perforce. It's the best tool for unreal because of the inbuilt locking. Also pretty much industry standard for big gamedev companies.

What sort of issues are you having? I've been using it with unreal for the past 3.5 years, dev team over a dozen strong, and can't really remember having any trouble...

[amber sentinel]

Probably the same noob confusion as me. 😛

[livid rover]

I probably just need a break but I’ve spent 3h deploying it on a server I have and after setting everything up and creating a workspace I kept getting the same error message on unreal saying that’s the modification cannot be added something something (I left an hour ago so I cannot check the message). After searching on the forum I basically saw that this error was apparently already a thing 6 years ago. I’ve tried a few fixes but nothing seems to have worked

[pulsar parcel]

This setup doesn't look particularly typical?

[amber sentinel]

Is this not what UE does?

[pulsar parcel]

Not really in application, no

I usually mirror the clean engine in an unrelated depot, like //engine-unreal

That can be copy/merged to a project stream

And it's always a lot easier to create a new dev stream for each engine upgrade too

Something like //project/dev-engine-upgrade-5.6 was a typical stream name

[amber sentinel]

Shouldn't my current setup, even if a bit wrong, still be "correct"

Clean engine download
A stream for merging the new engine, and custom edit
A working and ready engine

But, even if setup wrong or not, I still appear to miss-understand, or use the merge/ resolve tool wrong.

[pulsar parcel]

I've never run into bogus merge conflicts this way

[amber sentinel]

Thats not the issue.

[pulsar parcel]

The only merge conflicts I've got is from actual modified files (in project)

[amber sentinel]

Or, I guess maybe that is my issue?

IDK. lol

There is no conflicts, if I look at the files with the resolve/ merge tool.

But, it still marks every individual file to be merged.

[pulsar parcel]

Yeah with this method, you only get "file needs resolve" on actual project-modified engine files

[amber sentinel]

This method, as in, what you said, or it should be like that, with what I have setup?

[pulsar parcel]

The engine upgrade process is usually creating an upgrade stream populated from main, merging the new engine to that stream, dealing with conflicts, submit

And you can merge across depots in P4, that's just allowed

[amber sentinel]

Perhaps thats the issue?
In this case, I did in in reverse, because of all my extra files I added to the repo.

(I uploaded the clean engine to the merge, then attempted to merge in my engine edits.
and, it wanted to constantly merge all like 200k files

[pulsar parcel]

The engine should initially be copied from the clean stream, which like I said I've always kept in a separate depot

Then you can accumulate edits on it

You want to upgrade the engine? Update your clean stream from the licensee P4 or GitHub after running setup.bat, reconcile, submit

And that's what you merge in

The only things it'll flag then are project specific edits

[amber sentinel]

I didn't run setup.bat, just from thinking that also did .dll files, which was the origional issue.

But, I did do the rest.

Downloaded UE source
Uploaded to a clean stream
then, copied it to the merge stream
Then, attempted to merge my main engine (with its couple of C++ edits) onto the merge stream, and it was trying to merge every file.

[pulsar parcel]

Yeah that's definitely wrong. It's helpful if your engine source always came from this method

When I migrated a Git team to P4, I really just went through the engine edits manually after copying over a clean engine

Each engine files should have its roots in the clean stream

It sounds like you previously had a project with engine source, got a clean engine, tried to merge that on top. No shit that it's going to flag every engine file

[amber sentinel]

Now you lost me?
Why would it flag every file, if the engine version is the same, except for like 6 C++ files?
And, what issues does a previous project with engine source cause?

[pulsar parcel]

Because they don't have the same base

Which an initial p4 copy would've done

With your method, to P4, they're just separate files that happen to have been created with the same name

It doesn't know what an engine version is

[amber sentinel]

So, then. No joke.
Enlighten me? 😛

I had assumed that creating child streams of the development type would've been exactly what I needed?
Instead, do I need seperate whole streams, that aren't childs of the Main one?

[pulsar parcel]

This has been the attempt to enlighten

You start with a project with no engine source

[amber sentinel]

Damn

[pulsar parcel]

You copy the engine to the project stream

[silver token]

It doesn't matter the relationship of the streams, it matters the source of the files.
There's a big difference between submitting files to two streams and submitting files to one stream and branching.

[pulsar parcel]

As in p4 copy

Then you can freely modify engine source in your project stream

When you upgrade, do the stream creation from the updated clean stream, merge it, then the only conflicts are project edits

That has been a method I've seen successfully deployed everywhere, the key difference is that your engine edits were branched off from the clean engine

[amber sentinel]

I don't doubt your method, I'm new after all, just trying to understand where I'm lost at.

You said my engine edits are branched off from the clean, but they aren't?

it goes like:

Complete and setup engine
Merge of the 2
Clean base UE

[pulsar parcel]

The r1 of that engine file in your project stream must be the one you copied from the clean stream

Then merging can work properly

[amber sentinel]

Ok, so its a copy issue, that controls merging.
I had assumed merging does just compare each same name/ path file.

[pulsar parcel]

I always used copy at first then merge every subsequent time. Copying will overwrite everything

Applying your engine edits after this step will result in nice, easy merges every subsequent upgrade

[amber sentinel]

Yeah, I didn't have this in mind, or that prior knowledge of copying.

Since, my inital full engine was just a generic add command on the engine folder.
The merge, and clean engine is a recent addition, to fix an early fuckup

Well, thanks for the info, and context.
I had assumed merging did legit just do direct file contents == file contents sort of shit.

So, hopefully whenever I do have to actually upgrade the engine, it should work fine. 🙂

[arctic hornet]

perforce is kinda funky in this regard, anything that doesn't have shared history needs resolving for perforce to understand how it's linked together since perforce generally does stuff based on file revision numbers...

even if it has the same contents

[sand rover]

Hey! I've got a basic setup to build and submit precompiled binaries from a server into a separate folder in the project stream. I'm now wondering what's the best way to "map" those binaries into the /Engine folder for everyone who aren't building the engine themselves.

I thought I could do it with the stream path setup but it seems to replace my entire /Engine directory with the contents of my /PrecompiledBinaries directory (so a bunch of engine files are removed).

I've read about being able to use symlinks but I don't really understand how that would be setup?

I'm currently running a robocopy script to move files from /PrecompiledBinaries into /Engine but it requires that users remember to run it, which feels a bit risky. Any advice welcomed! (I prefer to avoid UGS)

[arctic hornet]

What is your reason to avoid UGS if I may ask, since this is exactly what UGS was made for, to handle PCBs and simplify syncing.

[sand rover]

I'm not completely against it but I want to see what's possible without it. It's just another thing that I have to teach people and another thing to maintain etc. If a good enough solution is reachable by just using P4 then I'd prefer that.

I basically want something similar to this https://jonasreich.de/blog/005-ue-deployment.html

[arctic hornet]

The way UGS does it:

• binaries are built and zipped
• they are uploaded to separate stream with description containing the version they're built for
• upon launching UGS finds closest code change (since you only need to rebuild for code change)
• it downloads the zip
• extracts it in place
• keeps a track of files it extracted
• when you sync next time it removes all the files it previously extracted

and afaik there are no post-sync triggers in p4, at least not client-side ones like git has, so unless you make your users use some client (like UGS) which does it automatically you'll have to make them launch something manually

[sand rover]

Yea that's the tricky thing, making the client aware that it should update the files without having to use a separate tool. The article mentions "…locally remapped to the regular engine directories using symlinks for everyone else" but I don't really understand how that would be done.

Got a stupid idea just now that I could maybe edit the editor bootup to check for new files and if so then update and restart..

[woven sluice]

i am not using a source engine and i'm in a tiny team of revshare weekend warriors, so my complexity is way less, but we just keep compiled project binaries right in the mainline. everything is stupidly simple and fast this way. coders use a virtual stream which cuts out the binaries. designers use a virtual stream which cuts out the unneeded source code files. CI machine uses a virtual stream which cuts out bulk art assets or content it doesn't need.

i do think UGS is probably worth considering though as soon as you have both a source engine and perforce...

[sand rover]

I mean to me that sounds way better than the ugs bloat. Though my situation is also a small team <50 ppl and we're not doing daily modifications to the engine, so binary update frequency is quite low.

For the project DLL this is what we're already doing, just submit the dll to p4 on each change (can be done by both coders and build servers depending on need for the specific change)

I've recently used UGS in a team of 500 ppl and I get the value it brings, but I also value workflow simplicity a lot and in this case just using p4 feels a lot leaner.

[woven sluice]

my value into this conversation isn't super high but i have a P4 trigger that tells teamcity to run; the build server asks P4 for a change list and if any .cpp .h .cs .uproject etc files are in it, it launches a build... starts by posting a discord message that building is in progress, and then posts a completion message a few minutes later. design team has simple instructions: don't sync perforce while the build state isn't green (finished with success)

works flawlessly for a micro team, would eventually fall apart for large team of course lol

i feel like i could extend the same setup into source engine binaries if i wanted, but IDK

i wrote a "bot" app in C# to manage some of that initial logic because i'm a masochist

[sand rover]

Yea that's similar to how we do it but we also let ppl just submit the dll straight in, if its tied to content changes for example. To solve the issue where ppl sync to a changelist that doesn't have the correct binary built.

[woven sluice]

technically i could just sync a DLL straight in too but i don't actually know why that would ever be necessary, content changes don't affect DLLs. this is just an academic sidequest topic though lol

[sand rover]

I mean you might have code changes that expose some blueprint functions, if you submit a BP using those functions without submitting the dll then someone might sync to that changelist and get BP compilation errors.

[woven sluice]

takes a serious chaotic evil coder to do that lol

[sand rover]

The content might depend on the code in that sense

[woven sluice]

coder works on code changes, finishes code changes, submits code changes (maybe also including blueprint changes with it)

CI says "don't sync for a bit gents, i'm still making the DLLs for y'all"

artists pull when it's ready

[again this is just how my micro-sized team works, not a good setup for everybody, but great for us]

if a coder is pushing the blueprints mentioned in the first step without submitting the code changes, that's fireable offense

[sand rover]

Yea in your situation you know that everyone should see that message and not sync, so that works for you.

I'm saying without that message, a coder could submit the code change together with BP changes using that new code. Then someone might sync the changelist before the build is ready. But you've got the messaging and workflow setup to make ppl aware to not do that. :)

[arctic hornet]

that's one more thing that UGS solves, since it won't let people who use binaries sync until the binaries are ready for that build, people who compile locally can sync just fine

[sand rover]

I basically want to put the responsibility on the one submitting code changes instead of the messaging.

[arctic hornet]

so at the end of the day you'd pretty much writing UGS on your own - although obviously I'm not saying "USE UGS", I'm just pointing thing sout

also UGS is incredibly easy to set up and doesn't require horde or anything, just perforce and UGS, also UGS is engine version agnostic etc. and then to build the binaries you either do it locally whenever someone submits a code change or from whatever CI solution you'd be using.

at the end of the day I've honestly found that running UGS makes things so much easier even with small teams and the maintenance cost compared to the rest of our CI pipeline is virtually zero

[sand rover]

Well yes and no imo, I want to use 5% of UGS functionality (sync binaries) without having to use (and teach and maintain) a separate program.

[arctic hornet]

I wouldn't say that binary syncing is 5%... it's like 90 or 95%... the rest you'd have is the badges, which is just nice to have for CI overview and then maybe installing some tools

but primary objective of UGS is to handle binary distribution

though I fully understand your reasoning and obviously the final decision is up to you...

[sand rover]

It might be that we'll need to "revert" to UGS but I feel that for our situation there should be a simpler approach. I mean I think I have something almost there, I just need to figure out how to easily get files from one folder to another 😅

[woven sluice]

tbh i just don't know why what you're describing is a useful solution in any way

how is that better than just committing them straight

[sand rover]

Even the current solution of telling ppl to use "StartEditor.bat" to open the project (which makes sure the latest binaries are used) might be OK but I want to explore improving it

[arctic hornet]

the only benefit I can see is that you don't get "bloat" directly in the tree and don't have to keep the files writable so programmers can easily use it (unless you do some selective opt-out through stream view)

although without properly configured p4ignore you risk someone pushing those copied binaries directly into the tree anyways by mistake

[sand rover]

Committing them straight might produce some version mismatch between locally built and precompiled versions, but I'm not sure how much of an actual problem that is

[woven sluice]

if that were true, then two coders could never work on the same project

[sand rover]

It's also easier to exclude the prebuilts if they are in a separate folder?

[woven sluice]

(you're not also making coders work with the CI server's DLLs, are you?)

[sand rover]

No that's what I mean I want to avoid, sorry if I'm unclear. It's really late here and I'll need to head to bed. Thanks a lot for your input and ideas, Ill do some thinking and experimenting 🙏

[narrow depot]

Hey all

So for perforce users, if you add a file to the engine from outside perforce, how do you have it update to the depot for all the other users?

We had a map that we did that with, we brought is outside the engine and worked on it

But it's not updating on my side now

It's says last checked out June 18th

[woven sluice]

you modify a file in your workspace, you add it to a changelist, you commit (push) the changelist

[wide raven]

When I Add something in Editor and save it, it shows up automatically in my P4V Pending view, and from there can be submitted. Is your case somehow different? ( I assume this works the same in Unreal version system, but I don't really use that except to confirm checkouts )

[narrow depot]

I see

Well they submitted the file, but it doesn't show on my system

They even sent the file separately and it's not changed

It might be that they change da different file

[narrow depot]

Well this file is from outside the workspace

[woven sluice]

you're going to have to show some info, you're saying conflicting things (your coworker submitted the file? so it must be in their workspace. but then you're saying it's from outside the workspace...)

[narrow depot]

My coworker submitted the file from outside their workspace into their workspace

I can explain more after work

[spare tundra]

Hi, I havent explored source control options, but have gotten interested in them recently, since, for my project, it is absolutely needed.

All I have seen is people recommend Perforce and then other people saying they used Subversion, but that it had issues with binaries

if anybody could tell me their experience, I would highly value it.

[ocean anvil]

the main 2 options are Perforce and Git. perforce integrates better with the engine and any additional tooling released by epic for the engine. git lacks some features and long term convenience.
Perforce is a bit of a pain in the ass to set up initially and get used to in the beginning with a steeper learning curve, but once you get it fully set up and running its smooth sailing mostly.
Git is a bit easier to set up but due to its inconvenience remains to be a bit tedious to do certain things with.
Perforce is free for up to 5 users but you need to provide your own hosting, which could be anything from an old laptop to a cloud based server.
Git is free to use and you can get free hosting through azure devops.

[spare tundra]

how is perforce in terms of perofmance cost?

I have ordered a NAS which supports running docker containers.
Obviously, it would be better to just run it on a proper server, but the NAS does have 16gb ram, and a somewhat usable intel cpu.
For an entire team, that wouldnt be sufficent, but its only going to be me and a 2-3 extra people with no more than 2 concurrent users.

I am wondering if that is sufficent

I also would like to hear experience with using subversion

because I heard that is able to handle a 1tb large project fine

according to people on the internet

[arctic hornet]

perforce is absolutely free for up to 5 users (or 20 workspaces with account sharing)... afterwards it gets quite costly, iirc it's like 30-40 USD per user per month (you pay up front per year)

but for small team if you have somewhere to host it I'd go with perforce

and the NAS seems like it would be enough for small team, as long as it's accessible from whereever others are...

although ideally you want fast disks in it since perforce can be quite I/O heavy

As to SVN, it handles larger projects better than git that's for sure, but I've not really tried it at all with unreal so I can't say how well it'll go... all I can say is that perforce has the most seamless integration out of all of the version control softwares since that's what epic themself use

[spare tundra]

IO heavy specify please

[arctic hornet]

well... it benefits tremendeously from using fast disks

if you're on slow disks it will behave fine, but it's much more snappy when running on SSDs

especially with unreal projects that can get quite large

[spare tundra]

mine should be able to do fast, and from my home, it wont be an issue to run it fast, but then working remotely will be bottlenecked by the speed I get from my ISP

since it is gonna run from my home

(as far as I understand it, but maybe there is more things to be worried about)

[arctic hornet]

yeah, another thing is whether you have necessary public IPs and port-forwarding, some ISPs don't allow this feature so your machines at home are not reachable from the internet, which obviously would make it hard to use your perforce without something like tailscale or other form of vpn/remote access software

[spare tundra]

yeah, my router shares a public ip with other ones, I would need to request a unique one

port forwarding does not work

but I still am able to host a webapp on it

using cloudflare tunnel

maybe cloudflare tunnel might bottleneck it as well

[arctic hornet]

well, hard to say how well it'll go with perforce, since perforce isn't a web app, but rather it uses custom TCP based protocol (usually running on port 1666)

but hey, try it and you'll see

[spare tundra]

yeah, it does support different protocols

but I am more worried about it being slower

worst case scenario I will request a dedicated ip

do they charge extra for that usually?

[arctic hornet]

who knows, I know that at least where I live they do, but I only know part of one country in europe and not the whole world

[spare tundra]

how much do they charge?

[arctic hornet]

there's not standard, for me public ip was roughly 1 USD per month... but ISPs especially in US are... quite annoying and like to charge people arm and a leg.

so ask your ISP

[spare tundra]

I live in europe, and I know a single isp isnt a good way to measure the average cost, but I just wanted to know how much should I be prepared for

[spare tundra]

yeah that is not terrible

[wide raven]

I would advise to stick to perforce : it's the best match for unreal, and support file locking out of the box

[inland bay]

Subversion is fine for unreal. I've run 500GB+ repositories with 30+ users and over 100,000 commits and it's just fine with it. It outperforms git and is on par with Perforce. It lacks some of the integration of UE & Perforce but for most teams its just fine.

[spare tundra]

which specific integration?

I almost feel like "use perforce" is a redditor opinion

and it isnt necessary for most projects

[inland bay]

What I mean is all the secondary tools like UGS do not support subversion. Perforce is a great choice if you want everything and you want "industry standard" experience.

[loud flame]

We use git. It is free and for a team of two it is enough. Of course you can't easily see what the deleted file was,it is a little inconvenient. But we manage

[spare tundra]

thank you for clarifying

[spare tundra]

I thought of using git lfs

I used it originally

[loud flame]

Ah yes. For UE, you might need lfs

[spare tundra]

I started using it but hit the free limit for github

so I stopped

[loud flame]

We have 38 repositories ( all uefn )

[ornate willow]

github is a terrible hosting option for unreal, github lfs charges storage per GB each month AND download bandwidth per GB under a pay as you go plan. if two people each pull your 50 GB unreal project you pay for 100 GB of bandwidth.

yes its true perforce is free for 5 people, and under the 5 free users could have 20 workspaces, each to 1 real person. but its actually a pain to manage when it comes to locking and merging.

But the real indie secret sauce is to use git + gitLFS with azure dev repos. completely free. azure git repos have unlimited storage with no cap at all. I have almost 3 TB worth of assets, projects, and personal stuff stored to azure repos with git lfs.
DO NOT use git inside of unreal editor, it has always been too slow to be usable.
either use git CLI or use a git GUI like sourcetree or fork.
pushes need to remain around 5gb, but it's easy to have scripts chunk up the pushes

[woven sluice]

you also might just try setting up a VPN with something like zerotier [or whatever the other common one people use is -- tailscale?], if you go this route

[spare tundra]

thank you. did not know about this

I never used the ue git inside of the editor anyway

my pushes were under 5gb anyway

my game isnt that asset heavy

thank you for letting me know about this too

[spare tundra]

I will look into it. I dont have much experience either self hosting yet

thanks

also

I have my project code conviniently split up into mostly reusable plugins

is there a good way to develop these and store them as a sep repo and use the game project to test them?

because id like to version them

[sand rover]

Something like git submodules maybe?

[spare tundra]

is it possible to handle multiple repositories inside of a single project?

[long hull]

That's exactly what submodules do

[teal bone]

Or use different project modules with their own repositories.

[woven sluice]

It's also easy to do with perforce too via imports

[spare tundra]

thanks

[spare tundra]

great!

[quaint cedar]

anyone know a good way to share files amongst a small studio? figure 8 people rn we're working on the same project with different tasks and and update eachother on which files were working on and when theyre finished we upload them to drop box and everyone updates weekly

[woven sluice]

none of you have any experience with any version control systems yet?

perforce is going to cost you like 40/person/month so unless you have money and are super serious you're probably going to end up choosing git. everyone will need to learn how to use it, someone will need to take charge of managing the central server or setting it up through a hosting service, and you'll want to use the project borealis git plugin for unreal.

[sand rover]

One thing I'm struggling to understand is when I should be using an Installed Build (via InstalledEngineBuild.xml) and when I should be using just the built binaries (via BuildEditorAndTools.xml). I mean I see the difference in terms of what files are generated etc but I don't understand when I would need one over the other. If someone doesn't need to build/code anything, are they fine with just the BuildEditorAndTools.xml binaries? I guess it might be missing some sample/default assets and such that could potentially be needed?

[wide raven]

Since we update the source engine very rarely, we are considering distributing installed build to devs instead of the whole source project. I think that is the use case, but not 100% sure

[sand rover]

Yea that's how we've done it now .. either an installed build or source (git + setup.bat) + BuildEditorAndTools.xml generated binaries .. but not sure if that's the way

[gaunt lily]

Hello guys, if I get latest from perforce, should I always hit regenerate vs project files with visual closed, or can I always just re-open the .uproject

[wide raven]

just reopen .uproject. You will only need regen and rebuild if your c++ code has significant changes.

[foggy niche]

I've noted that P4 is notoriously bad at merging cpp (and the merge tool from p4v is very confusing to me, I never understand what is circle, what is triangle and what is square)

I am considering adding git as an external merge tool just because i feel git has better diffing and if not, at least the ours vs theirs logic with the <<<<=====>>>>> thingies work for me.

Has anybody done something similar before? 🤔

(can I change P4 diffing algorithm? Maybe I am making this harder than it should?)

[silver token]

I can't say I've ever had any particular difficulty with the p4 merging. It's usually fine until there a conflict.
The top panels should show you pretty clearly what each shape/color represents.

[foggy niche]

The shape confusion happens on the third window, where clicking a shape sometimes lights it up, sometimes removes it completely, some weird time gives me a striked out shape 😅

Also, I feel P4 gets into conflicts that shouldn't. The lines were VERY far away, why was that a conflict? 🤔

[silver token]

The lit up shapes are the ones that should be included in the final out put. When you click and it gets striked out, all the text from that file should be removed from the lower pane.
I can't say I've even encountered an issue with conflicts that you're describing. Maybe you're misunderstanding the UI, but it's hard to say without looking at the same thing.

[foggy niche]

I'll try yo recreate some conflicts later today to see if I can recreate the crossed-out shapes 😅

[arctic hornet]

by default perforce considers a "conflict" any times there's no direct relation between the files even if they're the same...
imageine separately uploading to StreamA and StreamB, then trying to merge A to B or vice-versa... perforce can't cleanly decide what to do with the two history "graphs" so it will ask you to resolve the "conflict" in general I first do "Automatic without merging" which takes care of 99% of the problems... and the rest I either chose "Use Source", "Use Target" based on what I need... and lastly I go through the remaining files and manually merge them using p4merge

p4merge is kinda confusing tool at first, but once you get to know it a little, it's sooooo good for merging things

[foggy niche]

People keep hyping up p4merge for merges and I still don't get it 😅
Most likely an skill issue on my part 😛

[unreal mirage]

for some reason after getting the latest revision of my module file from perforce it seems to have it formatted into mac file format or something.
Anyone knows why this happened?

No one was using mac at all.

[winter parcel]

hi! is it possible to manually setup a revision ( change list ) in p4 ?
e.g. I have current latest revision 50 and next push will be 51 but I need force it to e.g. 60

[foggy niche]

no idea what is happening, but "mac" format should be unix, should be less characters per line break, not more 😛
(windows uses CR + LF, Mac and Linux use only LF)
🤔

[wide raven]

does it matter if it is a specific number, as long as it is the newest one?

[wide raven]

Open the file in Notepad++ and check line endings?

[foggy niche]

@unreal mirage 👆
or in other editor that shows what unicode is that weird box cross thingy

[arctic hornet]

perforce is so old that it has knowledge of the original MAC encoding which is CR only... linux/unix was LF only... and so windows decided to do both because compatibility so we got CR+LF

and they don't mean "MacOS" but "Mac" 🙂

and it sometimes happens if there's mismatch between two clients where one pushes CR+LF saying it's LF and other pulls LF and converts to CR+LF only or something like that, which causes CR+CR+LF for every line terminator, which is often visible as every other line being empty etc.

it's caused by the fact that perforce does translate line endings on the fly through some black magic I've yet to understand after 15+ years of using perforce

(I understand how... don't understand when or under what conditions and how it can get fucked up etc.)

[foggy niche]

daaaaaaaaaaaaamn... and I thought that git "normalizing" line endings was bad 😅

new fear unlocked

[woven sluice]

years ago i encountered some sort of corruption like this just by messing around putting emojis and other special chars in source files, for UE logs. i still joke about it occasionally here (and basically ban myself from using anything beyond ASCII lol)

i was too dumb at the time to try and figure out what actually happened. well, i'm still too dumb today actually.

[foggy niche]

I haven't embraced the emoji comments yet... but I could 👀

[unreal mirage]

well i set the file format to CRLF by default, for some reason it changed it by itself to LF

no idea, I didnt touch it at all

[spare tundra]

is perforce really still 40 per user per month even on self host?

I see how this could work for game studios with 50+ employees and an estabilished presence

but if I had more than the free 5 users, and had to pay that much for a small team

(especially if most of that small team only worked part time)

I also tried subversion, but only just trying to use it was already a pain, and I almost feel like buying 5 hard disks and just storing copies of the project would be simpler than using subversion

So I am really inclined to using git then

but I am a bit reluctant as people did mention issues with binaries

and I really do not want to deal with problems midway through development

I am talking about always self hosting the versioning software

I like perforce but its paid and I cant pay 40 a month per-user since its a hobby project

[foggy niche]

I've realized that most mid sized studios just pay something like assembla or some other saas that resells perforce 👀

The only problem with git is locking and unlocking is slow

Other than that, git is amazing 🤷‍♀️

Gitea is usually what is used for self hosting

But you could use git lab ce, I guess 🤔

If you are going to locally host git, then the locks time goes down (but is still slower than P4, since git still locks files one by one)

[spare tundra]

how is the pricïng there?

[foggy niche]

I don't pay 😅
I just use it, sorry

[spare tundra]

makes sense

is there a clean setup for subversion?

or is it just so archaic nobody even bothers

[foggy niche]

I know Subversion exists... I never used it, tortoise was a meme by the time I joined the industry 😅 😅

[spare tundra]

I havent even really "joined the industry"

I am just a hobbyist I guess

I just tried setting it up

and It just felt like it was gonna waste my time more than help me

[wide raven]

If you get even halfway serious about making a project all the way to the end, version control will save your ass regularly over time.

• mess something up, and want to compare changes? Diff
• mess something up, and want to roll back changes? Revert
• want to work on something in several locations without actually pushing it to mainline? Shelve

[spare tundra]

yeah I get what its good for. I am talking speicifically about subversion

[foggy niche]

Now that I think about it... I am not sure how P4 differs from SVN except on tooling (ugs)

Maybe it's about the streams and depots and swarms?

[arctic hornet]

P4 and SVN are closer than GIT and SVN... main difference is that P4 isn't a DVCS (Distributed Version Control System) so you need to be always connected to perforce server for it to handle stuff like checkouts and checkins etc...

(well... excluding reconcile offline changes etc.)

and Streams are just wrapper on top of good old depots - which is pretty much an SVN workflow

and Swarm is just external code-review tool with integration into P4V, but nothing special inside perforce itself pretty much

(that's why even Swarm needs it's perforce extension to be installed)

[pulsar parcel]

I would say self-hosting is the norm over Assembla

[arctic hornet]

a know a lot of mid-sized studios that use assembla

it's the kind of studios that can afford p4 licenses, but want the flexibility of A) monthly payments and B) if you need more/less users, just pay more or less next month etc.

but couple of years ago assembla was super insecure and allowed for exfilling data without auth iirc.

(and their whole infra is really weird... as when I looked into it, it looked like they had one 250 user license that they just multiplexed onto multiple servers and just provided a p4proxy as a gateway or something weird like that - but that's been like 2-3 years ago and I don't really remember the specifics anymore)

[fading horizon]

hey guys I have a general question i would like to ask about between P4V and UGS

context: initially i was coming with intention of using P4V to sync both source code and uasset, and manually generate projects and compile (through the use of GenerateProjects.bat and Build.bat)

1. Are they more or less the same or is there some differences?
2. I had an issue where what I was syncing with P4V , that CL i was in did not reflect correctly with UGS, even though the workspaces are the same.

[silver token]

1. UGS uses P4, it's not a source control system itself. UGS will do the GenerateProjectFiles for you when syncing. It can do a build for you as well, but that's a checkbox in the UI.
2. UGS will only show you syncing information based on syncing you do through UGS. It can't tell what you're sync'd to if you sync through P4v

[fading horizon]

so in theory if I rely on using p4 sync , GenerateProject.bat and Build.Bat, technically it does the same thing UGS correct?

@ point 2: yea that's unfortunate. having a mismatch bothers me a little

[silver token]

If you're building from source yes. However the benefits of UGS come from it's being the front end for non-engineers to get precompiled binaries and as a dashboard for automated builds.
There's some other tooling you can make easier from UGS as well, like ours has a tool that will do a compile of all the blueprints in the project so that you can easily check work that may have broken them.
Unless I'm syncing specific files for some reason, I do all my sync to CL and Sync to Latest type operations from UGS.

If the mismatch bothers you, sync from UGS. What you're hoping for is literally impossible for UGS to figure out. Did you sync everything up to a CL? just the CL? some random selection of files to a random revision?
It's just not feasible to figure out.

[fading horizon]

no that is fair. it's just a matter of decidcing to stick with p4v or ugs (and not ended up using both) in this case. I'm fine with building from source

[silver token]

You have to use both. You can't do submissions from UGS.

I mean if you're the only person working the project you don't really need UGS. that's fair. I don't use it for my hobby projects.

[fading horizon]

alright, in this case, my question then would be why not I opt to use p4v entirely as a preference?

[silver token]

But I probably wouldn't organize a team without it (unless the artists were really comfortable doing local builds)

I think it's easier to do the "sync project to latest" through UGS.
The build status badges (if you have any automated builds) are super useful.
As a list of changes to the project, it's a more easily digestible list than manually filtering the submitted changes in p4.
The tools you can link to a make available in the header are super helpful

[fading horizon]

ah alright, that is useful answer. thanks Ramius.

[foggy niche]

That last part is what confused me the most when changing from git to P4.
On git, a commit is a snapshot of the entire project. All the files.
On perforce, a Change list only lists the files in that change, it says nothing about other files in the project.

[arctic hornet]

it's because perforce versions each file separately, a list of individual file changes is then grouped into, you guessed it, a changelist

[verbal laurel]

Hi, I recently upgraded my project from UE5.5 to UE5.6.
I'm using a third-party plugin called UEGitPlugin.

I ran into an issue where I couldn't commit the Skybox actor because its material uses the Engine content "WorldGridMaterial". To fix this, I copied the material and related textures from the Engine content into my project. and use the copied material for that Skybox actor.

After doing that, I encountered a new issue: when I click Revision Control → View Changes, I get Git error messages, and around 7,000 Engine content files appear in the Uncontrolled Changelists. As a result, I'm unable to submit anything.

I have a video recording that shows exactly when the error happens:
The moment the 7,000 files appear in Uncontrolled Changelists:
https://youtu.be/AB2FArMXnBQ?si=HwAr1AUBQrbDT_RB&t=1945

[foggy niche]

I ran into an issue where I couldn't commit the Skybox actor because its material uses the Engine content
this kinda doesn't make sense? 🙃

To fix this, I copied the material and related textures from the Engine content into my project. and use the copied material for that Skybox actor.
Not too bad, but shouldn't be related to git in any way...

After doing that, I encountered a new issue: when I click Revision Control → View Changes, I get Git error messages, and around 7,000 Engine content files appear in the Uncontrolled Changelists. As a result, I'm unable to submit anything.
Oh... unless you have the entire engine in source control (can happen if you are working with a source build) you shouldn't be seeing engine files in the source control window. Ask Borealis because that could be a bug of the plugin 🤔
If you do have the engine in git... then it might be true that you touched a file that dirtied 7k files? 😬
just in case, if you can git status and put the output I can try to guess what's going on for real with git 😛

[verbal laurel]

I ran into an issue where I couldn't commit the Skybox actor because its material uses the Engine content
The issue happened when I tried submitting LI_Light (a Level Instance that includes the Skybox).
The submission failed, as shown at 2:05 in the video:
https://youtu.be/AB2FArMXnBQ?si=FZwc67im9lZG9zYs&t=125

I checked the log and saw error messages (around 4:46 in the video).
It turns out the Skybox actor is using WorldGridMaterial, which is Engine content and not tracked by Git.
So I figured I needed to copy the material and its related assets into my project folder.

Oh... unless you have the entire engine in source control (can happen if you are working with a source build) you shouldn't be seeing engine files in the source control window.
No, my project and engine are in separate folders, so the Engine content shouldn't be under source control.

Ask Borealis because that could be a bug of the plugin 🤔
Yes I also ask the Borealis team, since this might be a bug in the plugin 🤔
There’s another user (DinoLampa) who had the same issue, but they didn’t find a solution either.
https://github.com/ProjectBorealis/UEGitPlugin/issues/167

just in case, if you can git status and put the output I can try to guess what's going on for real with git 😛
git status looks fine (see screenshot). I think the Uncontrolled Changelist is a UE feature and not related to Git or Perforce, since there’s an option in the Project Settings called "Enable Uncontrolled Changelist."

I have an idea that might fix it: reinstall UE5.6.🤔

[atomic saffron]

does anyone know how to open any game in the unreal engine editor because I would like to edit hello neighbor 2 late night shift fan game and I would like to edit a few things I tried to open somehow through uproject and there is no source folder somehow it didn't work for me and of course I mean the project from the unpacked pak file and does anyone know how to do it And then for example export it so that I could change it in the game or does anyone know anything on this server?

[wide raven]

please do not cross-post.

[spare tundra]

Uploading LFS objects: 0% (0/8602), 0 B | 0 B/s

how long will it be stuck like this?

For context, I setup up a gitlab ce server inside a docker container

it now works just fine, but when I try pushing, it just holds at 0 B/s for wayy too long

and I dont know if this is just the way it works, that its doing something in the background

or if this is an error

[spare tundra]

okay fixed it

I am hosting it under a subdomain and I forgot you need to configure it

[foggy niche]

test that your lfs is pointing correctly to your S3 👀

if your git server loses the handles to the s3 files you have all the files and you can't use any of them 😅

[spare tundra]

wait you cant just store the files on your own server?

[foggy niche]

you can 🤔
the idea behind LFS is to send the heavy files to a cheaper server 😅

[spare tundra]

tbh I didnt even know what S3 was

[foggy niche]

(also, LFS brings file locking, good for teams)

if you don't need file locking, and you don't need to offload files to a cheaper storage bucket, you can avoid LFS 🤷‍♀️

[spare tundra]

I want to host everything on my own hardware

so I dont have to rely on external storage and definitely not pay for it

Besides the electricity bill that is

[foggy niche]

are you working with a team? do you need to "lock" files so you let the rest of the team know that you are using that file?

[spare tundra]

well, it would be helpful

not always, but sometime I need it

[foggy niche]

then you do need LFS 😅

you can host the files in the same box that has the git server, yeah 🤷‍♀️

[spare tundra]

I seemingly got it to work but it is so painfully slow that I couldnt bother waiting for all of it to upload.
(like 40-100 KB/s)

[foggy niche]

LFS grabs your "Large Files" and dumps them somewhere on a disk. It's up to the git server to remember how to get back the correct file from that "somewhere on a disk" 😛
If your git server doesn't do this remembembering correctly, you end up with a project with no files 😛

[spare tundra]

good to know that

[foggy niche]

try with a smaller project, put some lfs files into it, send it to your server, try clonning back

and check if you got your large files

or you only have pointers 😛

[spare tundra]

but I am such a noob at networking that I dont why that is

I am using cloudflare tunnel since my isp doesnt give my router a dedicated ip

[spare tundra]

Ill try it

[sand rover]

When using UGS, is it recommended to always submit engine+project binaries together in the zip? Feels wasteful if it's just the project DLL that has changed? 🤔

[arctic hornet]

UGS expects whole package every time

[frank silo]

hello! I have been trying to setup perforce and made the mistake of setting my server to unicode, I guess my only option now is to uninstall the server and delete my P4Root to get it to be non unicode?

[arctic hornet]

You indeed cannot switch from Unicode to non-Unicode while keeping the data… so yes… that’s the easiest way to do it… (you could go, remove P4ROOT and change configuration too)

[frank silo]

awesome thank you, I have been having non stop issues trying to get my team mates to be able to build the engine through UGS when they sync, between translation errors and other things its been a bit of a nightmare, I had no idea this channel was available until now

[frank silo]

so I reset the server and Unicode but when I try to set my typemaps I get the error telling me the server has to be unicode. this is probably why I added Unicode in the first place, any ideas?

[woven sluice]

Type in p4 set

Do you have p4charset set?

[frank silo]

I solved it, but the problem was the P4_Server.ddns.net:1666_CHARSET it was set to auto, setting it to none worked

Thanks for the help

[ornate willow]

I've tried this many different times throughout it's development, including getting detailed setup instructions from it's creator. it's unfortunately never been usable for us, and the lfs locking didn't work and created merge conflicts. If someone has a modern vouch, used with multiple people, I would love to give it a try again. But git from cli or simple gui is simply the fastest way to work with big plentiful files from unreal, after perforce

[sand rover]

How do people usually set up build jobs with UE5? Talking about both cooking/packaging the game but also building precompiled binaries. I'm using Jenkins now and just doing the "auto cleanup and sync" option takes 40-50 minutes just to reconcile and clean up any local changes. Then there's the engine rebuild on top of that, so the build times are quite "insane" for even small changes. Maybe I shouldn't clean and just go with incremental builds? Scared of some old junk causing troubles every now and then..

[wide raven]

We use the p4 "sync only"

[sand rover]

Do you ever run scheduled cleanup jobs?

[silver token]

We do incremental builds. We can kick clean ones manually if there are weird results.
Worse part about incremental builds is that they can have some annoying variances in time if you have multiple build machines, depending on how they get allocated.
Since an incremental build should only have to build new changes, you could have builds taking a few minutes and then have one that takes 45 because it hasn't been used in a while.

[arctic hornet]

We use horde… and horde has custom “have table” logic implemented where horde locally keeps track of what it synced and to what revision and then can clean up much faster… usually such cleanups take between 40 to 60 seconds, which is much better than your 50 minutes

Perforce is generally super slow… especially if you’re not rotating your journals often… or if you’re not using partitioned workspaces

[wide raven]

Except for some build artefact folders, no, havent really needed such, and have been running the builders for 3 years.

[tulip ledge]

I'm trying to set up our build servers to work on a second unreal project, and I'm running into disk space issues. Fixing that isn't too hard by just giving them all more disk, but that won't scale up if we want every box in the pool to be able to build every project since keeping the artefacts around means 250+ GB/project

How do y'all handle that? Lock nodes to a specific project?

(this may be a question for automation, not this channel, I'm not quite sure where to put build server questions)

[arctic hornet]

Since you did not specify what CI solution and what version control, I'm going to speak for what we are using, which is P4 and Horde...

we generally have a pool per project like GameName_Win64 etc. and combined with custom properties and conditions on the pool (Platform == Win64 && WithGameName == true ) we lock the nodes to projects... if nodes have enough space, we add them to two projects for example etc.

Another way you could go about this would be to overlap your workspaces so multiple streams could share the same physical location, horde and perforce can deal with that just fine (it might raise your job times slightly since it will have to conform the workspace, but generally it should be fine) - but at the end of the day we did not have to fully deploy this since our buildmachines have more than enough space for us

[arctic hornet]

CI jobs generally go to #automation but there's overlap, so even here is fine

[slender coyote]

Where is a good forum or place to post for detailed perforce questions?

I have been working all day on a script to try and clean up old revision history (server is now too large), but it is horrendously slow. Want to post the script for others to use but also some perforce-chad to help fix

I am also making a weekly 'ZIP' backup script which i want to share with the community, where is a good place to post that

[lucid laurel]

These are files within the Developers folder that are ignored by the .p4ignore file. Any way of having the Uncontrolled Changelist ignore the Developers folder?

[slender coyote]

setup a p4ignore

[arctic hornet]

Unreal keeps track of any touched files that are not controlled by version control... using it's own "uncontrolled changelists" so there's generally way

(and I don't even think there's a way to disable uncontrolled changelists, although don't quote me on that)

[lucid laurel]

Then my editor fails to check what is ignored. I double checked within p4v and the developers content is ignored.

[arctic hornet]

editor doesn't even check what is ignored, it has simple logic:
does source control want ot have anything to deal with with this file? Yes: Shove it to SCC, No: Uncontrolled

so if perforce has it ignored, it doesn't want anything to do with it then unreal will shove it to uncontrolled

[silver token]

For what it's worth, we've never ignored the Developers folder.
I think it helps to have it in perforce. It makes for a very useful place where people can coordinate on experiments without affecting the main game.
It also means that content changes can keep that dev work working instead of breaking references or api changes.

[lucid laurel]

That’s seems unintuitive. I thought the uncontrolled changelist is for keeping track of modified files that haven’t been checked out.

[arctic hornet]

that's the general usecase, but really it's for any file that's outside of VCS

i.e. ignored, outside workspace etc..

[lucid laurel]

Understandable

[narrow depot]

Hey guys, my coworker has an issue with submitting through p4v, it keeps giving her an error saying to reconcile or merge

[silver token]

That means there's a conflict. So they need to sync and reconcile the changes they made with the changes to the same file someone else made.

[narrow depot]

Whenever we try to sync on unreal ( either me or her) it crashes , so should we sync via cmd, if so how?

[silver token]

Wait, is it source code? or an asset?
Just close unreal and sync with p4v

[narrow depot]

An asset

Got it

[silver token]

Do they have it checked out? p4 shouldn't have let them do that if someone else was editing (or not let the other person do it).
For an asset, they'll have to discard their changes, sync and remake them.
There's no merges for binary data.

[narrow depot]

So they told me they imported the assets to unreal , so just tell them do remove the assets, sync then import?

[silver token]

are they importing or re-importing? Why would they have a conflict on an import? did someone else already import them?
bad naming conventions?

[narrow depot]

the y imported, and the changelist that has the imports does not submit

tbf it has a ton of other stuff in it

that she didnt even touch

[narrow depot]

Ah ok, its not a import confilct ots a submition colflict

[silver token]

Well they have to find the file that’s conflicting and resolve it somehow. Merge, discard their own changes, whatever. Maybe figure out why there’s a “ton of stuff” checked out they didn’t mean to.

[thorn harness]

I'm an artist who is starting to dip their toes into some coding. I've got a P4 server with UnrealGameSync setup with 5.6 built from source. I have a few other people I'm working with who I'm trying to have use PCB so they don't need to get Visual Studio setup.

Is there a way to generate new PCB for JUST code changes in my project? For example, adding a new plugin, or a Blueprint Function Library, has UGS complain that I need to generate new PCB for everything, including the editor. I'd love to be able to just recompile what is needed for the project and not push new editor binaries so frequently.

My goal is to keep the editor and projects somewhat agnostic so that I can have multiple uprojects run through UGS off the same editor build. Any tips?

[silver token]

While you can have UGS support multiple projects simultaneously, Unreal is not designed to support that sort of separation of Editor and Game. You can reuse the same engine binaries (sort of) but ultimately they will be packaged individually with the project specific binaries.

[thorn harness]

Gotcha. I was hoping to emulate how working with the Launcher is, where I maintain a source build of 5.6 with minimal code changes. In terms of the project binaries - there's no way to build and package those separately from the editor ones? I'm mainly trying to avoid having tons of revisions of editor PCBs since its like 2gb and my hosting space is limited. I also plan on really only iterating on the project source as opposed to editor.

Or is there a way for me to just push my project binaries to P4 and not have UGS dirty the CL as being incompatible?

[silver token]

There’s not really a difference in the way that you mean. There are only the project binaries. They may be configured one way or another(ie Editor or Shipping) but it’s a single code base. Editor is just dll’s and Shipping is monolithic.
So if the projects share an engine you’ll save on compiles (it won’t rebuild engine binaries for each project) but when it packages up PCB’s the Engine binaries are included since with a source build the next version’s Engine binaries could be incompatible if you made any changes.

[thorn harness]

I guess UGS might be overkill for my needs and I should just push the compiled binaries to P4. I love UGS as a frontend launcher, but it seems like it I'm fighting with how it's meant to be used.

[silver token]

There’s a perforce setting for the number of revisions for a file if you don’t want to keep the full history. Like you could say only keep the last 5 or 10 instead of every version of PCB zips.

[thorn harness]

I did look at that, I think something in the BuildUAT script overrides it because it went back to 32 revisions after I ran it again.

[silver token]

If you have (or plan to have) multiple engineers at any point, manually just pushing binaries is a potentially very error prone process. Less error prone with a build machine, but not great.

[thorn harness]

At the moment I'm moonlighting as our only engineer 🙂 it's a small personal project and almost entirely Blueprint based outside of a few plugins and small bits of code I'm implementing. Ultimately I don't expect to have to recompile binaries too often so it likely won't be an issue once the project is up and running. Appreciate the help!

[narrow depot]

Hey all

Me and my coworker are still having trouble

She has a folder on perfore and she pushed it, but I haven't received it and don't t have the folder

We're stuck and don't know what tondo

Mark for add is also greyed out

And she tried reconcile and merge with source

Their also not on the depot

[sand rover]

For people not using UGS to distribute binaries, how do you handle the engine BuilId version? Do you have it set to something manually or do you have a specific process when submitting new (mainly project, not engine) binaries/DLLs?

[wide raven]

In P4V, you can view both depot and local folder structure. Have the folder and contents arrived in the repo?

[narrow depot]

Nope

That's what I was trying to get at

Does she have to merge or?

[jagged gull]

I have this really wierd error when trying to submit specifically headers and cpp files, and I can't figure out how to solve it. If anyone has a clue on what that might it it would be very appreciated
open for write: /tmp/tmp.3755.140365656233216.812: No such file or directory

[arctic hornet]

are you working in different streams or depots?

if you are then yes, you will have to do merge... if you are not, then just using "Get latest" will work

by default you only see your local copy, if you switch to the "Depot" tab you'll see what's on the server

[narrow depot]

So even her depot doesn't have the folders

Lemme see if I can get her to chat here

[arctic hornet]

well then that can only mean she did not push

but probably only created a numbered changelist

[narrow depot]

Huh

So she recently submitted all the changes she had

[arctic hornet]

if it's submitted then it has to be there there's no way around it

[narrow depot]

So she should click the folder and submit?

[arctic hornet]

she should open Pending tab, and then submit whatever she needs to

since you can have multiple changelists opened at once and as long as they're in "Pending" then they're not submitted

submitted changelists are visible in "Submitted" tab

i.e. this (you can see I have multiple changelists)

and these are changelists that I've submitted to the server - i.e. they're available for others to sync

[narrow depot]

Got it!

[narrow depot]

One last thing, if both our streams are blank would that mean their the same stream?

[narrow depot]

so we just looked into this

the changes are blue on my coworkers side but red on my end

I have no idea what that means

[arctic hornet]

you mean the triangles?

in pending?

[narrow depot]

yup

and my cowoerkers is

should I have her resubmit the pending ones?

[woven sluice]

if it's in a changelist, then it's never even been submitted before

[narrow depot]

thats weird

then how does my cowerker have it submitted on their end?

[woven sluice]

changelists are just "modified files lists" they're a staging ground, you mark what things to send to the server

[narrow depot]

so is this a perforce error?

[woven sluice]

like what's inside of the oldest one, 31 Wall_Placement

[narrow depot]

oh

there empty

[woven sluice]

ok so cleanup first, might as well just delete empty changelists

[narrow depot]

got it

cleaning now

ok it cleaned

but the changelists are still there

[woven sluice]

new screenshot of all changelists from your end?

[narrow depot]

right one sec

[woven sluice]

alright what's in 74

[narrow depot]

still nothin

[woven sluice]

nothing...... Chandell should be able to delete this one too

94, 96, 98, 99, any contents?

[narrow depot]

yup all of them are empty

on chandells side they say sumbitted

[woven sluice]

no they don't

submitted changes are in a different tab in P4V

[narrow depot]

oh

your right mb

[woven sluice]

klukule posted a screenshot above... see the circle triangles

[narrow depot]

yup saw that

[woven sluice]

so now, in the continued interest of "let's just clean up current status", Chandell (and/or you) could try right clicking on your project folder and Reconcile Offline Work ... let it run ... does it find anything changed?

[narrow depot]

will do

its greyed out

oh wait

[woven sluice]

yeah switch to workspace view if you didn't figure already

[narrow depot]

yeah right i did that lol

hence the oh wait

ok this will take a while

uuuh

so this came for cleaning

[woven sluice]

wtf

might have found something wrong, this is not reconcile offline work

[narrow depot]

yeah no this is clean

apparaently i canceled the last clean

[woven sluice]

reconcile offline work will be comparing your own local workspace files to the server to try and find anything different. if you modified something it should help you add it to a changelist as "modified". if you created a new file it should help you add it as "added". if you deleted something it should help you add it as "deleted"

[narrow depot]

got it

so I shouldnt clean? and reconcile instead?

[woven sluice]

clean is for nuking your local work to make it match the server

[narrow depot]

oh

got it

well the files I need are not even in the depo

for some reason

and chandell couldnt even submit them to the depo

so would reconcile even work in this case? by server you mean depot correct?

[woven sluice]

a depot lives on the server, kind of yes (a server can have multiple depots)

a reconcile will at least give you some idea of what's going on... like if it finds every file then it either means you didn't submit anything to teh server (can't be true, i see versioned files) or a workspace setting is screwed up

i also wonder, if you set up a classic depot instead of a stream depot... the root folder setup for classic depots is kind of stupid in some way and easy to mess up. stream depots are a tiny bit more intuitive

[narrow depot]

huh

for the stream we leaved it blank

reconciling now

[woven sluice]

i think you might have made classic depots then, FWIW. stream setups will have an icon,