source-control

[silver token]

There is dark mode support the ue5-main branch (as of Nov 2024), but it hasn't made it into a public release branch yet.

[dense fulcrum]

Looking forward to that. One more reason for us to move forward in engine versions in the near future

[ornate matrix]

what is the proper way to build binaries on a repo (im using azure repos)
i guess this is the correct pipeline

• clear all files in Binaries folder
• run build command

for pulling, i guess developers should have in their gitignore set to ingore Binaries folder, and for designers they dont ignore it ?
also, i would like to know if there is a safe thing to do to avoid issue if a designer pulls the repo while the pipeline is building the project (because there will be missing files since Binaries folder will be cleared), something like "put in a buffer all new generated files, once completed clear binaries folder and place the buffer files in the folder"

[outer hinge]

Is there a way to have a certain map not check files into source control?

Like I want to test PCG generation and every time I go to save I have to spend 8+ minutes after I hit Save All out of habit waiting for source control changes to take place.

[proven violet]

Can anyone help? I have a gitignore file suppose to ignore intermediate and saved folders but it appears to not be ignoring these folders

[pulsar parcel]

<@&213101288538374145> another one

[near granite]

i need help
in setting up some form git server
for 2 ppl
Using Unreal Engine rn
what's the way to set it up with file control?

[arctic hornet]

<@&213101288538374145> another one

[pulsar parcel]

I would've thought there would be something detecting multiple identical messages in different channels by now

seems like a multiple times daily occurrence now

<@&213101288538374145> same scam

[ocean anvil]

add it to your *ignore file

[ocean anvil]

best way is to watch/follow some tutorial.

i'd start here: #ue5-general message

[outer hinge]

Thanks but this sadly was not enough. I ended up disabling source control integration

[arctic hornet]

Automod is good at catching these but it relies on keywords, so if it’s something new or not added to automod there’s not much… and I don’t think there are any discord bots cross checking messages across threads which would be a good heuristic… 🤷‍♂️

[earnest helm]

UGS seems to want to build the editor when I haven't specifically sync'd the last editor code change and thus gotten the binaries associated with that change.. Example:

1. Change 53 was a code change, I sync it, get binaries.
2. Change 54 55 56 were content changes. No sync.
3. Change 57 was a code, new binaries built by Horde.
4. Change 58 was content changes.
5. I sync from 53 directly to 58

This makes UGS build the editor.

Is there a way to get UGS to download the last good binaries if it detects that your current binaries aren't the latest? I feel like this used to just work, but this is the behavior my team is seeing now. In the example, when I sync from 53 to 58, is there a way to get it to pull the latest available binaries in change 57 knows that binaries were built for 57 to use with the content only 58 change?

If I sync 57 then go to 58, it works as expected, but this behavior of needing to sync the last binaries built change list before getting the latest content change list is not a great workflow. Hoping I missed something and can make it behave as described.

[brisk pond]

Does anyone have any resources for making Perforce suck less? Whose idea was this crime against humanity? No, sorry, you can't edit a simple text file without clicking through our stupid GUI or manually setting it to edit through a command. Oh sure you can edit it by clearing our read only flag (we set this because we think you are too dumb to make changes safely, you're welcome), but we won't track your changes unless you explicitly tell us to

With p4 I cd into a dir that has a """"workspace"""" and p4 doesn't even know about it? I have to manually p4 set the client? Or workspace? Or whatever they call it?

What was wrong with cd into your dir and pressing git status, and wow look at that, there's all the information I need?

[silver token]

Why are you doing any of that? IDE's like VS and Rider have integration with perforce to do those check outs for you. The Editor also has p4 integration to do everything through the Editor including checkouts when you make edits.
It sounds like you're making it harder for yourself than you need to.
You can't apply git workflow practices to p4.
Much of this is because p4 is centralized version control while git is decentralized. So you need to go through an extra step to make sure that you're allowed to safely make edits to the file. Binary files are nearly always exclusive checkout to prevent you from losing changes while I'm making changes to the same file. Sometimes other files might be exclusive checkout if something unique is going on that might result in lost work.

[brisk pond]

I am not always using an IDE, and I don't want the IDE to be abstracting away commands anyways. Is there a reason p4 can't just know what workspace I'm in based on where directory I'm in? My current plan is to just maintain a list that I can look up in my poweshell profile to automatically switch the workspace

"It sounds like you're making it harder for yourself than you need to" this is totally subjective though, isn't it? I don't want to constantly have the GUI open, it is a waste of space and they designed it poorly. I also have no interest in GUI for git. This is a valid feeling.

[silver token]

Theoretically multiple workspaces could map to the same local disk drive. It's not common, but I've done it before, and it means that you can't assume the workspace based on the current directory.
I suppose it is a little subjective, but there's no getting around having to check things out before editing stuff. If you don't like the GUI (it's not perfect for sure) you can use the command line but nothing's going to be automatic.
I guess the only thing to say is that this is how perforce and centralized source control work and you've got to figure out how to work within those constraints.
I'm not sure why you care about the IDE doing this for you. It does plenty of other things that you probably don't care about. You're the first person that's I've ever heard of having a problem with it.
Maybe there's another GUI out there, p4v has never bothered me enough to go looking.
Personally it's either the IDE, Editor or p4v (if I'm not using one of the others). I will use the command line for something that p4v doesn't support or that are large operations that work a little better from there (usually when doing engine upgrades). And everyone I've ever worked with has worked with p4 the same way.

[uneven storm]

Is there a way to get UGS for a perforce project without having the engine inside the perforce depot? It doesn't seem that way but wanted to know if there any known workarounds.

It's for a small hobby project with a small team so we wanted to save on storage costs and just have the main project folders by having each of us have a locally built engine in the proejct directory, but ignored by P4. And just have Engine Changes in a pinned thread, since those will be relatively minimal.

[silver token]

No, you need the engine in perforce because it basically requires a build machine using horde or something else.
And that build machine would need to be able to sync the engine and build.

[woven sluice]

to be fair, VS's perforce integration always was fucking ass 🤣 not sure how it is today but probably still the same

i don't even notice it in rider, just automatically checks out to default changelist when i save

and... well, the few times i do use p4, i do so by launching cmd from P4V so that it already has the workspace setup

[pulsar parcel]

P4VS can be configured to be less shit, the defaults are pretty much assuming you're on the same LAN as the server

[lament hinge]

Hello
In Perforce, does anyone have some material to learn how to do branching?
if I want for example to have a commit my changes to lets say like Git a different branch until it's ready to be pushed onto the main branch, is that doable and how please?

[silver token]

There isn't because generally gamedev and more so unreal development does trunk based development.
Everyone is submitting to a mainline/trunk. Branches are used for releases or staging upgrades. Rarely are they used for isolated development like you would in git.
The biggest issue is the wide array of binary assets that cannot be merged between branches which can make that workflow more trouble than it's worth.

[brisk pond]

I can live with manually having to check things out, but if I manually clear the read only flag to edit things, is there a way I can get p4 to tell me which things I've updated so that I can add them??

[arctic hornet]

"Reconcile offline work"

it goes into the selected folder and it's subfolders and checks any writable files for changes based on modify timestamp and file hashes

and then it tells you what files were added, changed or removed and then adds them to the changelist

in general anything in perforce is a user-initiated operation rather than being implicit, so perforce won't do anything unless you tell it to.

[silver token]

However, the large the scope of the reconcile (number of files in folder and subfolder) the longer it's going to take.

[arctic hornet]

for that I can recommend third-party tools such as https://github.com/brickadia/p4-fast-reconcile/tree/main due to more optimized caching and whatnot it can do reconcile on full unreal code-base (~300k files) in about 30 seconds

we use it heavily for ingesting new versions... speeds up that stuff by literal hours

[silver token]

Doesn't it get it speed up by doing things like ignoring possible moved files? instead treating those as independent deletes and adds?
It's not a deal breaker. I also don't tend to need to do reconciles in the first place. Even our engine source in p4 is laid out to not really need it most of the time.

[arctic hornet]

iirc it handles moves too, biggest speedup comes from caching the digests afaik, since normal perforce does it the most naive way ever... check every file against every file while calculating the digest every time you access the file... which for large group is... SLOOOOW...

nevermind, it doesn't... either way... I gladly take the speedup over move/add/delete tbh...

(and one could probably add that as second layer after you resolved the files, it's decently easy to check all the pairs - especially as you will already have the digests cached - and convert that to move/add/delete pairs)

[silver token]

Like I said, not a deal breaker. Generally I just work (and usually recommend) working in ways that don't require reconciles. RiderLink and thirdparty plugins are the exception and those are small enough to not need a separate tool.
But if I ever have to it on the engine, it was definitely on my radar.

[brisk pond]

Reconcile listed tons of things that I didnt actually change, not sure why. I think it's also including files that are not part of the workspace, but are in the directory?

I'm just looking to get a list of files that are in the workspace, in the directory, and have changed

[arctic hornet]

well, reconcile checks for the files that are not on the server but are there locally, for modified files and for files that are removed locally but are on the server... you could use CLI command p4 reconcile which you can limit what kinds of changes it's looking for

[brisk pond]

Right. I am using CLI.

[arctic hornet]

but don't take me wrong, this looks like a workflow issue and is highly likely to cause issues the moment multiple people start using the same stream/depot

[silver token]

the only real difference between workspaces and directories is if you have a p4ignore file setup.
Otherwise p4 will just assume all the files in a directory should be in source control and generate adds

[brisk pond]

It's an idealogy issue. I am willing to be wrong. If so, I will pay the price. I shouldn't be forced to work in a certain way, ever.

I am a grown man and I have been coding since I was 13. I should be allowed to edit files without someone telling me no, you can't

[arctic hornet]

I mean yes... but also, every software has it's workflows it's designed to do... you can hammer a nail using screwdriver... but it's easier with hammer

[silver token]

I think you're going to be disappointed in using unreal if you approach things that way. it will definitely force you to work in a certain way.

[brisk pond]

I don't think I am asking for anything outlandish. Just a list of files I changed.

[arctic hornet]

Also, when you make new workspace you can check the "All writable" flag which will disable the read-only protections (which are really good to have... though I've been using perforce for 15+ years now, so I'm used to it and probably biased)

[silver token]

that should be what you get with reconcile. without more details we can't really tell you what's "wrong" with your list.

[arctic hornet]

and then regarding the changed files, p4 reconcile is your friend, I recommend checking the documentation for all the different CLI flags

[brisk pond]

Yeah, that's the plan. Thanks for the advice, also the fast reconcile you linked looks promising

[arctic hornet]

from what I'm getting bunch of "random" files that would be marked for add since they're not pushed to the server probably...

[brisk pond]

^

Our workspace is in our engine folder, but we only upload our plugins

[arctic hornet]

hence my suggestion to limit what kind of things reconcile looks for by using the switches/args/flags

[silver token]

you can also tweak p4ignore to filter things out of the workspace from the directory

[arctic hornet]

either way, I highly recommend doing at least a research into perforce workflows in conjunction with unreal... since doing the "Epic Way (tm)" is by far the easiest way to use unreal and surrounding ecosystem of tools.

The moment you start to diverge and do weird stuff, things tend to break... a lot...

(speaking from experience)

[brisk pond]

Great, I would rather learn that the hard way

Than be forced into SLAVERY

Just kidding

[arctic hornet]

my general mindset in this is.... I'm lazy... people from epic are paid to make what's best for them, and thus by proxy for anyone using unreal... so why would I spend my time thinking of things I don't have to... I just take what epic does, scale it down to fit our scale and we've been happy for past 6+ years on our current project (nad me personally 13+ years in unreal ecosystem in general)

[silver token]

that's a little overly dramatic

[arctic hornet]

either way, good luck with your experiments, may they lead you to success rather than nuclear armagedon 🙂

[brisk pond]

I'm also lazy, that's why i want less windows to manage (p4v), less commands to type, less assumptions about what editor im using

But yeah enough whining about it for now

Cheers

[arctic hornet]

pretty much any editor other than notepad has perforce integration... iirc vscode, visual studio, rider etc... have them and at that moment it just becomes implicit feature and no need to switch windows etc. 🙂

[brisk pond]

Lol

All of those take a long time to open

[cerulean hamlet]

Does anyone have a good git/P4 ignore file? i keep having a ton of changes like crashdumps and logs that i dont feel like i need to commit everytime the engine opens up

[wide raven]

here's mine #source-control message

[cerulean hamlet]

Thank you so much!

[wide raven]

( edited, removed the .uproject ignore, that was a temp thing )

[cerulean hamlet]

Is workspace needed in the ignore? I'm trying to setup the ignore for a group project and a couple of people keep having merge conflicts all over the place

[wide raven]

If you are using perforce, you should have file locking -> no conflicts

[cerulean hamlet]

I'm relying on git sadly

[wide raven]

I have heard rumors of git lfs supporting locking... but yeah. Locking is pretty critical for smooth work with unreal, since so many of the files are .uasset blobs.
Not sure if there are any really good workarounds. I would consider switching to perforce, unless somebody can recommend ways to work with git

[cerulean hamlet]

Yea, plan is to move to perforce later on

[wide raven]

Next best thing would be to try to compartmentalize work, have very clear comms on who is working on what and so forth. Big hassle without auto-locking

[copper mountain]

Is there a way in Perforce / Helix V4 to simply use my local workspace and completley replace whats on the Server? Im hosting it locally on one of my servers but im new to any sort of version control and cant seem to wrap my head around just replacing whats in the server with what i have in my local workspace. Ive deleted lots of files, restructured a bit and its throwing a complete tantrum about it all

[silver token]

You can use "reconcile offline work" to let p4 figure out all the add/removes that are required to make what's on the server match what's on your disk.

[copper mountain]

Oh ok that sounds exactly what i need

Hmm it still failed

Says there is nothing to reconcile, is that because i have a submit pending?

I feel like i need to nuke it all and start again

[silver token]

Maybe 🤷 or you're not selecting the right folder for the reconcile or folder's not visible to the workspace

[copper mountain]

So pretty much this is only me using this, and i just checked out the whole UE Directory so i can do what i want with it, and there are like 3k files / diffs to go back to the depot

[silver token]

Sorry, I'm not sure I can help you much more remotely.

[runic vault]

anyone know how to get past this error?
It happens when I try to open a project created from epic installer with my build from src setup, I'm not otherwise using perforce at all.

like it says consider disabling.. yah its disabled thats why it is not there

the project doesn't have it specified as enabled either

oh I got, had to add this to the uproject file
{ "Name": "PerforceSourceControl", "Enabled": false }, { "Name": "PlasticSourceControl", "Enabled": false }

[late summit]

Hey guys I need just a little help please, I am trying to set up plastic source control but I cant seem to to get it to work, I have the addon for unreal, I have the main app, I have plasticFS agent, I cant seem to upload my unreal to the version control

I tried clicking the revision control login > plastic SCM >rename workspace and Repository >Added my Server url address> initial checkin on> create a new unity version conrtol workspace for the current project.

but it says
ogSourceControl: Warning: RunCommand: 'makeworkspace AetherWorlds . --repository=rep:AetherWorldss@repserver:{{MYNAME}}@cloud' (in 29.618s) output (42 chars):
Error: The remote host closed the socket
SourceControl: Error: Command: MakeWorkspace, Error: Error: The remote host closed the socket

do I need the path to cm?

maybe in unity I need an address for my org?

perhaps even services or tools in the unity cloud?

I made sure I "bought" the free unity cloud plan

hmmm, is it only for c++?

[astral girder]

Hello!
I got a problem with Git lfs. When changing a file, it doesn't get checked out and the other team members don't get any warnings when opening the same files nor do they see an indication that it is checked out.

How can I fix that? Btw total source control noob here, never done this

[frank hedge]

How is that Github for free, its so much useful and powerful

[red kiln]

Anyone?

[idle urchin]

Hey All, sorry if this has been asked before, but is it possible to grab the P4 user that is authenticated in the Revision Control UI? I'm trying to prevent a double auth before running some python scripts.

[wide raven]

hmm not sure, maybe something from here
https://help.perforce.com/helix-core/server-apps/cmdref/2023.1/Content/CmdRef/commands.html

[floral owl]

How do you suggest storing an installed build from Engine source for my team to use? I was looking for a way to put it inside Azure but am unsure about how to go about the process

[silver token]

Generally you have an automated build process that stick the build "someplace". That can be a network drive or perforce (but not in the same place it exists when you build it locally). Using UGS makes distribution easier/safer. But you have to be using a source build for that solution.

[floral owl]

We are on a very tight budget or we could have gone with perforce? No way of getting it up on Azure? LFS is unlimited there which is why we have been using it for our project

[silver token]

Oh, sorry I should have said "source control" there. UGS needs perforce, but an automated build could stick it into git if you're not using that.

[floral owl]

What about the gitattributes and all? I was planning to make an isolated repo which my team can clone from and save space on their hardware. 100 plus to 45GB as of now

I was planning to just move the Localbulds content elsewhere to push

[silver token]

That I don't know. At work it's perforce and for hobby stuff I only keep source in git.
Are you the only person building? or are there multiple people that would cause changes to the binaries?

[floral owl]

As of now I am the only one. We only intend to make minor changes to the engine as of now, maybe more if we find a good reason or an engine dev to help us there

I was thinking of putting a source fork on Azure tho for future modding

[silver token]

But it's not just the engine. If you're doing any C++ development for the game itself or custom plugins, you need to distribute those as well for anyone not able to compile.

[floral owl]

We can compile on all systems. We wanted to save space and time in case someone accidentally cleaned the engine. So it would be like the others devs are using the launcher build

not everyone has that much free space and a powerful enough pc to build source that frequently

[silver token]

I'm not sure how that ends up affecting the build process. The usual setups are 1) all precompiled binaries 2) installed build, compile project stuff, 3) source build, compile everything.
I'm not sure you can sync precompiled binaries & build local changes the way that you're suggesting.
The build process may not recognize the sync'd binaries as matching the local source and decide to rebuild them anyway.

[floral owl]

Also, how can I make a fork of my local branch for Azure?

[silver token]

🤷 I'm a super basic git user. Sorry

[wispy marten]

I am at my wits end, if anyone is a perforce genius:

Error:
Client does not have P4ALTSYNC configured, required for this workspace. This error happens during any time I try to add or submit to the Depot.

SS: attached.

I cannot for the life of me get my workspace to sync to the depo due to this error.
Image

So altync is fine in client options. Trying to set up version control to begin the course.

I have googled and see to edit this file for the client to alysync for noaltsync, but that is already how it is.

[arctic hornet]

p4altsync shouldn't be required unless you're using P4VFS which is not recommended with Unreal Engine

[wispy marten]

Any idea how I dont use it then?

I was not aware anywhere I turned it on.

[arctic hornet]

Client: lukas.jech_ANDROMEDA_Matcho_Main

Update: 2024/12/22 16:02:03

Access: 2025/03/01 19:37:38

Owner:  lukas.jech

Host:   ANDROMEDA

Root:   E:\P4\Matcho

Options:        noallwrite noclobber nocompress unlocked nomodtime rmdir noaltsync

SubmitOptions:  submitunchanged

LineEnd:        local

Stream: //Matcho/Main

View:
        //Matcho/Main/... //lukas.jech_ANDROMEDA_Matcho_Main/...

This is my usual workspace when made from scratch, the only differences I can thus see are altsync vs noaltsync and different submit options, but submit options won't play a role in this

[wispy marten]

So uncheck altsync here

Which I just did

[arctic hornet]

yes, uncheck there, or when running p4 client add the no in front of the altsync and just save the notepad and close it

make sure you are editting the same workspace in p4v and in p4

since p4v can work on different workspace than what you have set for your terminal

[wispy marten]

I was following a perforce youtube who said to turn it on.

Thanks I think it may of fixed it.

[arctic hornet]

youtube tutorials are kinda hit and miss... especially with these random settings

I believe that I've never seen any good tutorial for perforce... ever... especially combined with unreal ecosystem

which is funny, because what's recommended and used by epic is pretty much the default configuration 🙃 - and by "pretty much" I mean that most of their defaults were set to what was default and recommended around the year 2000 when epic started using perforce afaik... and then they just started using streams instead of old depots

either way... I'm going to stop rambling now 🙂

[wispy marten]

No you are so right tho.

Its funny to me because this is my 2nd depot for a new stream/project.

My first one works great...

Then I spent 6 hours today just getting this going.

Lol

And now for some reason unreal wont find the workspace but it is syncing in P4V fine... The battle continues.

I have it all on the same drive sooo hmm.

[arctic hornet]

yeah, honestly to put it really simply, this is what I recommend to most people:

• ideally linux based host (cheaper due to licenses + in general more performant)

• case-insensitive

• non-unicode

• steal some random p4typemap and p4ignore from epic's documentation

• simply create a //MyProject/Main (for your project) and //MyProject/Dev-EngineMerge (merging your project and new engine releases) and something like //UE5/Main (clean releases from epic)

• sync unreal engine from github, run setup.bat... and do nothing else

• submit everything from that repository 1:1 except ignoring .git folder to //UE5/Main

• p4 integrate //UE5/Main to //MyProject/Dev-EngineMerge

• p4 copy //MyProject/Dev-EngineMerge to //MyProject/Main

• submit your game code to //MyProject/Main/MyProject/(MyProject.uproject)

Done... on engine changes, sync from git, push to UE5/Main, integrate Project Main to Dev-EngineMerge Integrate UE5 Main to Dev-EngineMerge... resolve conflicts, copy back to Project Main

This is really really abbreviated way to do pretty much everything and to make everything work "The Epic Way (tm)"

[arctic hornet]

unreal sometimes refuses to use workspaces with long names and stuff, check the output log for warnings or errors

if it's filtering the workspace out, it will complain in the output

but even if it's doing that, you can still type it in there manually

[wispy marten]

ah it threw an old address from a prior host in

Just needed to update IP

We are all set.

I almost dont feel like designing today that took it out of me 😛

[arctic hornet]

and that's why devops is a fulltime job for some

[wispy marten]

Damn right man

Thanks for everyting and being responsive you guys are the true heros for people learning.

[idle urchin]

Hey, these are the P4 commands, which work, but they don't seem to interact with the login through the UE ui. I still have to log in through the CLI separately, which I am trying to avoid. UE seems to hold on to the connection/auth data somewhere I just want to see if we can access it.

[arctic hornet]

technically speaking everything perforce uses the same login tickets stored in p4tickets files, but every window/process can have it's own active connection at that time, which unreal does based on what's set in the Source Control Settings...
the only direct way to interact with this is through the SourceControl module in UE using C++

although the data are stored on the disk so you can manually read and parse the ini file to get what the user has set in there

Saved/Config/WindowsEditor/SourceControlSettings.ini is the location of the file

though keep in mind that for proper handling of this you also need to use the p4config file if that is set to true etc... so it can be quite a bit of work... but... yeah...

[wide raven]

if all that is to be achieved is to save a couple of seconds from a login, I wouldnt bother

[granite junco]

hi, i made an upload of my entire project

but I dont see it being actually uploaded here

what did I actually achieve by doing this?

is my project actually uploaded? becase I dont see any files in, I just see the name of it that I assigned it

[arctic hornet]

not familiar with diversion, but to me it looks like you did not upload anything, maybe just the .dvignore file... I don't know of many people using diversion on this server, so it would be best to ask on official server if there is one etc.

[granite junco]

but theyre pretty much all the same aren't they?

and looks kinda simple

[arctic hornet]

no version control is exactly the same, every one has it's one peculiarities and workflows... so knowledge from one doesn't really translate to the other. I'm not saying diversion is bad or anything, all I'm saying is that you will have hard time looking for help here as there's not many people using diversion (and if they are, they are not super active here) so you will probably get better help related to diversion on their official servers/communication channels, since they will be more knowledgeable about their software

and for example, if I told you some perforce commands which I know would help you debug this, they would be useless to you since you'd need diversion specific commands etc... same for git etc... they all share one common thing: storage of versioned files... but their aproach and the way you control them etc. is different

[granite junco]

what kind of commands do you use usually?

[arctic hornet]

depends of what I'm trying to do... but in this case to see what files I've pushed I'd use commands like p4 describe to check what was in the changelist I submitted, or p4 files to see what files are on the server (or p4 fstat) or simply just by looking at the P4V client honestly.

like I've said, these are pretty much specific to perforce, so they won't work for diversion I'm 99% sure of it

[granite junco]

perforce? oh

i think its pretty much the same for diversion except you type dv

[arctic hornet]

yeah, as I've said, I've never used diversion, I'm a long time perforce user and occasional git user. And that's why I was saying that you'd probably want to ask on diversion discord, since you'd be able to get better guidance. Since most people that are active in this channel are either perforce or git users

[granite junco]

but diversion gives me up to 100gb free

[granite junco]

is that also the case for other version control systems?

[arctic hornet]

well our perforce server after all the years is about 10 TB in size and I've had my perforce server running for 15 years meanwhile diversion is new thing.

[arctic hornet]

not sure what do you mean by the question.

[granite junco]

is ti free?

thats the question

lol

I use perforce cause its free

[arctic hornet]

git is free, perforce is free for 10 users, it's just not cloud based like diversion, so you have to provide your own server.

either way, I'm really busy now, so I'll excuse my self and I'll go start working on my work 🙂

[pulsar parcel]

Though they did do sort of a cloud thing but it's really just a semi-automated VPS on Azure

[idle urchin]

I don't mind interacting with C++, it's a custom plugin anyway. However, I can't find much documentation about the Source Control Module, I'll look again. Good idea about the source control ini. I had forgotten about that.

[arctic hornet]

there's not much documentation about it, you'll have to go by the source code more or less

[idle urchin]

It's more about ensuring people are logged in and that tooling can work. It's easier if they have one point of entry.

[idle urchin]

Understood. thanks for your help!

[arctic hornet]

your entry point will be ISourceControlModule::Get() and then it either has CreateProvider or GetProvider, since you can crete separate provider from the global one

and then you either use the SourceControlHelpers class or use the raw ISourceControlOperation::Create<T> function

TUniquePtr<ISourceControlProvider> FMiniDumpDiagnosticsApp::CreateSourceControlProvider()
{
    FString PerforceServerAndPort;
    FString PerforceUserName;
    FString PerforceClientName;
    FParse::Value(FCommandLine::Get(), TEXT("P4User="), PerforceUserName);
    FParse::Value(FCommandLine::Get(), TEXT("P4Client="), PerforceClientName);
    FParse::Value(FCommandLine::Get(), TEXT("P4Port="), PerforceServerAndPort);

    if (PerforceServerAndPort.IsEmpty() || PerforceUserName.IsEmpty()) return nullptr;

    FSourceControlInitSettings SCCSettings(FSourceControlInitSettings::EBehavior::OverrideAll);
    const FSourceControlInitSettings::EConfigBehavior IniBehavior = FSourceControlInitSettings::EConfigBehavior::ReadOnly;
    SCCSettings.SetConfigBehavior(IniBehavior);
    SCCSettings.AddSetting(TEXT("P4Port"), PerforceServerAndPort);
    SCCSettings.AddSetting(TEXT("P4User"), PerforceUserName);
    SCCSettings.AddSetting(TEXT("P4Client"), PerforceClientName);

    TUniquePtr<ISourceControlProvider> SCCProvider = ISourceControlModule::Get().CreateProvider(FName("Perforce"), TEXT("MinidumpDiagnostics"), SCCSettings);
    const ISourceControlProvider::FInitResult ConnectionResult = SCCProvider->Init(ISourceControlProvider::EInitFlags::AttemptConnection);

    if (!ConnectionResult.bIsAvailable) return nullptr;
    return SCCProvider;
}

This is an example of explicit creation of perforce provider using CLI arguments (this is from my app to analyze minidump files from crash reports)

    FCrashInfo& CrashInfo = CrashDebugHelper->CrashInfo;

    const TSharedRef<FDownloadFile> DownloadFileOperation = ISourceControlOperation::Create<FDownloadFile>(TEXT(""), FDownloadFile::EVerbosity::Full);

    if (SCCProvider->Execute(DownloadFileOperation, InFile, EConcurrency::Synchronous) == ECommandResult::Succeeded)
    {
        const FSharedBuffer Buffer = DownloadFileOperation->GetFileData(InFile);
        FString FileContent = FString(UTF8_TO_TCHAR(Buffer.GetData()));
...

And this is how to use it for example using raw operations

again, I recommend checking unreal source code for references, but yeah...

[idle urchin]

This is above and beyond. Much appreciated.

[granite junco]

help?

[woven sluice]

You may have realized this by now but there's no such thing as "logging in" with perforce. Every command is authenticated. There is no persistent connection.

It's (optionally) a ticket system. The first time you authenticate you get a ticket. There's a file on your drive that contains a ticket string, and perforce will use that ticket string to authenticate commands for as long as the ticket is valid (server setting)

[arctic hornet]

<@&213101288538374145> another one

[gusty lion]

SVN users, do you use the "trunk branches tags" root structure? UE5's docs seem to imply an empty repo.

[peak siren]

any Perforce users got a crash issues when using the Tools > Sync Content button?

i keep getting this, and i guess using P4v manually gets the job done but it's irritating to have a crash on a feature that should be reasonably easy

[arctic hornet]

We don’t really use that button, since we usually sync through UGS… also if you could share some details about the crash might be able to help more

[peak siren]

actually, can i jump into one of the VC's with you? i think screensharing might make it a bit easier

i honestly don't really know what i'm looking for

[arctic hornet]

I’m at my dayjob… so randomly helping on discord is fine but I can’t justify vc 🙂

[peak siren]

for sure

let me pull up the crash log

LoginId:af5583f1465ff3c89f9d4b9f85e49724
EpicAccountId:6b274b21f1274d3d88ec2fd40c9017f5

Assertion failed: !GDefaultMaterials[Domain]->HasAnyFlags(RF_NeedPostLoad) [File:D:\build\++UE5\Sync\Engine\Source\Runtime\Engine\Private\Materials\Material.cpp] [Line: 757] 



UnrealEditor_Engine
UnrealEditor_Engine
UnrealEditor_Engine
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_CoreUObject
UnrealEditor_UnrealEd
UnrealEditor_UnrealEd
UnrealEditor_AssetTools
UnrealEditor_AssetTools
UnrealEditor_SourceControlWindows
UnrealEditor_SourceControlWindows
UnrealEditor_MainFrame
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_Slate
UnrealEditor_ApplicationCore
UnrealEditor_ApplicationCore
UnrealEditor_ApplicationCore
UnrealEditor_ApplicationCore
user32
user32
UnrealEditor_ApplicationCore
UnrealEditor
UnrealEditor
UnrealEditor
UnrealEditor
UnrealEditor
UnrealEditor
kernel32
ntdll

this is from the crash reporter

the logs are pretty immense and i dont really know what i'd be looking for in them to solve that issue

i looked online, other people seemed to have this problem (https://forums.unrealengine.com/t/perforce-sync-crash-on-ue-5-4-2/1897779/8)

but i didnt find relevant fixes

[arctic hornet]

OK… so… without debugging symbols its hard to say really …. But this is probably some contention where something’s using the material you’re trying to change etc… so there’s nothing we can do about it as its deep within unreal

Honestly, I don’t think even epic is using this button ngl

[peak siren]

i didn't change any material or anything

like this may as well be a fresh blueprint preset project

I happen to have added one plugin and one library asset pack, but the issue existed before that already

[peak siren]

Can you define Debuggin symbols for me (or tell me how to find them?) this might help me figure out where the issue lies

[arctic hornet]

Debugging symbols is about 100 gigs worth of pdb files you can install from epic games store as an addon to the unreal installation

What it does is that in case of the crash it will show function name and line where it crashed and not only just the module name

So it can help debug

[inland bay]

I do. I use tags for "points in time of interest" and branches for releases. This is mainly useful for tracking down new bugs.

[broken patio]

Hey, does anyone use Git as their Source Control? If so what cloud solutions do you use to host it? And what storage capacity do you use

[woven sluice]

if you're a small team you probably want to use Azure Devops for as long as you can (i don't have any XP with it myself though)

[broken patio]

How do you mean for as long as you can?

[woven sluice]

team size limits on free tier

(free tier sounds very generous)

[broken patio]

But what about storage?

[woven sluice]

sorry, what about it?

[broken patio]

I'm just looking into the Azure Devops. Trying to understand what these rate cards means.

[woven sluice]

if you're over 5 users (out of the free tier), i have no honest idea if AD is still the best option

[broken patio]

Ah okay yeah we are a bit over 5.

Currently we have a server with 2TB's and 15 users. Just finding options that can cut some costs

[dire helm]

Hello everyone ! quick question regarding things like GIT for projects.

1. Is there a way to Reduce hashfiles when making a new map file to reduce the file space usage?

Me and my bouddy run out of space when i make pretty small changes... is this normal? is there a way around this issue? thanks!

[long hull]

Use One File per Actor

[dire helm]

Any info i can read about this?

thank you for replying

[long hull]

https://dev.epicgames.com/documentation/en-us/unreal-engine/one-file-per-actor-in-unreal-engine

[pearl star]

Would anyone have a good best practise workflow site to share on perforce how to use it with multiple diciplines (artists and coders) branching / merging / creating releases .. etc 🙂 we not using unreal engine but im sure atm that dont matter as long as there is good material to read up on

[wide raven]

The thing is, with unreal most just do mainline-development. I have talked about that a bit before and can try to dig that up if needed. After doing that for 3 years I am actually starting to feel that may be a pretty good workflow even outside of unreal engine.

In a nutshell, a broad generalistic view ( of mine )

• branching is best for pure c/c++/whatever coders working on complex features ( but we still don't really use it with Unreal BP heavy project )
• for everybody else, mainline dev is fine
• feature flags are super useful ( don't break existing functionality, hide your refactoring / new broken stuff behind a flag )

For releases, we just used to tag them in git. Perforce doesn't really have the same functionality sadly, second best is just to name a CL as "Release x.y.z" perhaps...

A hierarchy of dev -> staging -> release branches is pretty much industry standard in any case

[silver token]

Here, you're only going to get advice that are best practices when working with Unreal, not necessarily for general software development.
You should find a community that is closer to your use-case and ask there.

[pearl star]

There was UnrealSync workflow 🤔 seemed interesting

[pearl star]

Ya it can be general UE workflow

Since its still game development only difference can be integrated tools but general best practice would be good to mirror n compare

Prolly gotta google a perforce coming from git workflow overview 🙂 sonce we bit too big of a team to work in mainline so feature developing somehow xD

@wide raven have you tested p4 tag or labels or is that in any shape comparable to a git tag

[wide raven]

tag was per file ( so that's a no go, I don't wan't to be tagging a zillion files ), labels i haven't checked yet

I think label has similar per file ideology - seems like git and perforce are fundamentally different beasts in this area. Git tag is just a pointer to a commit

[pearl star]

Okay yea that sounds like ass, was hoping for tagging a CL and call it a day 🥹 , guess the Helix management tool would be better for managing a release

Had to google what it was called Helix Plan

[white bone]

For those that use Perforce, I have a question.
I have this error when I try to submit my work:
Perforce The filesystem 'TEMP' has only 203.9M free, but the server configuration requires at least 250M available.

I tried a couple of things, including
p4 filesys.TEMP.min 500M
But nothing worked out for me... I feel like I'm missing something stupid...
I hope this Screenshot helps as well:

Please TAG @white bone me so I can see your message.

[arctic hornet]

well, the issue is that you have only 200 MB free on the "TEMP" and the minimum is set by default to 250... by setting it to 500 you're making the limit higher rather than loweing the limit... also 200 MB even for temporary folder is really small. and perforce might fail while transferring large files

[white bone]

Right, well I want to increase it is the point, and I don't know how.

[arctic hornet]

well, tempfs is by default the size of half the RAM in your machine iirc... been some time since I've dealt with tempfs directly

(since we just mount TEMP to regular disk instead)

[white bone]

Ah, the machine being where the perforce is installed in, right?

[arctic hornet]

yes, this is all server-side

[white bone]

Ok, that's helpful, thanks!
I suppose the last resort is to "obliterate" the TEMP file. (I guess?)
Though even for that, I have no idea how to do it.

[arctic hornet]

tempfs technically isn't a file on the disk, it resides in RAM (in reality it is mounted as a file, since everything on linux is a file in one way shape or form)

and that's why it's generally set to be up to 50% of your total RAM size

[white bone]

aah I see.
So... assuming I restart the perforce server, technically the RAM would be... freed?
Idk if that'd fix the problem but... it almost sounds... right?

[arctic hornet]

no, you ideally need machine with more ram in general

512 MB of ram is quite small

[white bone]

oh wow, ok I see now.

Alright, I'll try to work with that, thanks a lot for now!

[arctic hornet]

perforce itself doesn't take that much ram iirc... but if you make the ram of the VM to be 1 GB at least, suddenly the tempfs will be 512 MB which would be above the minimum limits

[fierce cosmos]

Hey, I was wanting to set up a local Source Control using a NAS, what would be the best way to go about with this? I'd be the only user of this

[willow plank]

Could someone help me understand how Git works for UE? I'm really struggling to understand why it thinks it needs to upload 13GB of data, when my content folder is only 6.9GB...

Additionally, I'm slightly concerned that LFS hasn't configured properly

at this point I'm starting to wonder if I'd be better off using local backups & my Onedrive

[strong sigil]

Looks to me like you haven’t set your .gitignore list properly to exclude the ancillary folders

[faint cairn]

Local backups and one drive is such a terrible idea. You can get a free perforce setup if you are just a handful of peeps.

And git works fine too if lfs and the gitignore are set up correctly. Been working with GitLab for quite some time now. Only thing needed is the git plugin by the community instead of Epic's

[teal bone]

The gitlab interface is so poop

😦

[wide raven]

heresy! ( seriously, version control is the way to go for any selfrespecting dev, even solo )

[faint cairn]

Yeah but you get used to it. I barely interact with it anyway

We are setting up a combination by now. Perforce for active development, GitLab for storage

[willow plank]

so I looked into GitHub, and it seems you only get 1GB of LFS, which is nowhere near close enough for my needs. I need something measured in the terabytes, which is why a local NAS system would work very well for me. You still suggest the same?

[faint cairn]

If you can afford a NAS system you can also "just" get a cloud server with perforce.

Decent NAS setup, with GitLab potentially needing a second server to run on, depending on what your NAS uses, is gonna cost a bit.

Saying that cause we recently got one, but we also additionally have one setup on Hetzner in the cloud for backing up most of our servers.

Saying cloud but that thing is not accessible from outside anyway

[willow plank]

Yeah, except GitHub charges you $5/month for 50GB of LFS. So if I were to need 10TB of storage, that's 200 upgrade packages, which is then $1,000/month. I can buy a very high quality NAS system that will last me years for less then the price of a single month. I can't find any cloud services for anywhere near a similar price.

I hate subscription based models, and only use them in a very few instances, so Onedrive (because I use a lot of the office suite), and then Discord Nitro because I like the perks. Other then that, I see it as cheaper to buy once, and prefer to actually own my stuff.

Is there any specific reason why I should be using systems like GitHub? A backup is a backup, one can keep the last 2 stable versions, don't need to keep the whole thing

[arctic hornet]

well, generally you want to keep 3 copies of data, on two different media and at least one in different physical location to cover all your bases...

Also, regarding the "10 TB" I've not seen any project that needed more than 1 TB of storage at most except for AAA productions... who at that point usually just self-host (either on premise on in cloud) whole cluster using Perforce rather than git/GitHub...

And if you really want to self-host on your NAS (not that I recommend it without external backup solution), I'd recommend setting up Perforce, it's fast, you don't have to bother with LFS etc... and it's free for up to 5 users

And reason for using systems such as GitHub (or to be more precise git) or Perforce etc... is simple, unlike manual backups, which I don't see you making manually after every change... is that they are all so called VCS or version control system... which allows you to see any version of every file, easily traverse back and forth, revert changes, create multiple branches etc... etc...

Also if you're individual or smal lteam (up to 5 users) you can go and try Azure DevOps, it's still git, but microsoft gives you 250 GB limit instead of the 50...

and most if not all small to medium scale projects can easily fit into 250 GB (honestly, so they can into 50... but who's counting)

[woven sluice]

if you want to use git, azure devops should be free up to 5 team members, FYI
(i don't know why i replied to that specific message)

[faint cairn]

I never said GitHub.

I said GitLab.

And we are using it self-hosted, which despite the server cost is free.

[long mural]

06:13:26   Submitting change 60.
06:13:26   Locking 1 files ...
06:13:26   add //XX/Editor-Binaries/++XX+Main-XXEditor.zip#1
06:13:26   Change 60 submitted.
06:13:26   Took 1.130,16s to run p4.exe, ExitCode=0
06:13:26   Submitted CL 60 which became CL 60``` why did this take so long?

1.6gb zip file, but server has 500mbit upload speed..

[willow plank]

that's nice, just intrigued though, as everything I've been able to find says that DevOps is limited to 10GB hard limit per repo.

[still quarry]

What are the advantages of using Git File Locking? Should i be constantly using it? Can you give me some tips what to care while using Git LFS, and github desktop?

[faint cairn]

The advantages that Perforce Checkout/File Locking has. If two people were to modify the same Asset (so non-text files), one of them would have to give up their changes because you can't merge those files.
That's why the first one to modify it would lock the file, so the other can't submit it. With the proper UE plugin you'd also get a message in the engine if you were to try to modify it.

[strong sigil]

Hey @arctic hornet sorry for the ping but you seem very knowledgeable about the ins and outs of P4 servers and administration.

I've got P4D running as a service on Windows with it pointed to a file share (NAS connected over SMB and mapped in Windows) and its working for the most part.

but when the team tries to submit a large number of files / folders we get this error. ~
Frustratingly if we move those same files and folders to another sub location it will submit just fine.
If we submit smaller changelists with a handful of the folders it submits fine

it's not a file or char length issue as we've got deeper nests uploading fine. All other areas of Perforce seem to be working except this and there is no pattern as to when it appears. It looks completely random.

I'm going crazy over it. My initial thought is the network is being choked trying to submit a couple hundred folder creations etc at once after the submission. So I decreased a lot of the parallel sync and multi threaded stuff and it still appears.

We have one depot with multiple mini streams. MOSTLY .uasset files. No code nothing, it's a linear content render server and not gamedev related

even Perforce themselves are like "wtf?"

[faint cairn]

Can you try to find out if this is a perforce issue or not? To me it sounds like the SMB connection may be temporarily dropping and then when perforce tries to make those directions it fails to find the SMB share. Can you try rclone or rsync with a large number of directories? Though rclone/rsync may be able to detect connection drops and then wait prior to trying to make files/folders. Do you have any network/SMB logs you could look at to see if it's losing connection? What are you using for the NAS? Perhaps there's logs on the NAS side that's useful?

@strong sigil

(That was my wife using my account btw. 😅 Told her someone has issues with P4 + NAS, cause she's setting up a similar thing for us currently, and she then wanted to write in case she can help.)

[still quarry]

I have added GASP to my project, and overriden LevelPrototyping materials. And when my friend tried to pull.
We got this error.

BTW; changes are not my friends changes. Those were mine on the push..

[pearl star]

Free perforce 👀 ?

[faint cairn]

Your friend has to make sure that they close the editor before pulling

Otherwise Git can't write the files

[still quarry]

Hmmm, so everytime we pull/push stuff, I will make sure we close editor, rider etc.

[faint cairn]

I haven't done so myself, but lots of people have said that you can use Azure's free Server to host your Peforce Server on it.
I would assume there is some form of storage limit eventually, but yeah.

Rider doesn't matter.

[pearl star]

Aaaaah like that 🙂

But sont perforce still require a license per server

Dont*

[faint cairn]

Your friend can in theory also pull from within the Editor if they have the better plugin installed.
But even then it's probably better to Close -> Pull -> Open, cause changing Asset while the Editor is running is also often crashing.

Your Friend should also make sure to Stash their changes before pulling, to avoid unneeded merges.

Outside of UE we use SourceTree to manage the GitLab Push/Pull/etc.
We usually do this:

1. Fetch to see if there are any changes.
2. If there are changes, stash your current changes.
3. Pull the new changes.
4. Apply the stash back onto current.

That way you'll get a local "resolve your files" in case the stash/changes touch the same text files.

If you don't stash, Git will see this as merging two branches, and you will get a setup where you need to still do the file merge but then basically sub a merged version of the changes and yours.
The "line" that indicates the branch will also have that jucky merge indicator then iirc.

Here is the thing though: I would ALWAYS suggest to use Perforce, if you have 5 or less people working on a project for the forseeable future.

While Perforce isn't perfect and has its fair share of headache, for UE it's vastly better than other Source Control Solutions.
Part of that comes from Epic heavilly leaning into it, and parts come from other SC Solutions not having certain features.

With Perforce you usually get:

• Automatic File Locking and Unlocking based on "Check-out" and "Check-in", where "Check-ing" is the submit/commit.
  • Git offers Locks, but they are general Locks on files, no matter if you have altered the file or if you committed it. Once you lock a file you gotta unlock it by hand again. The unofficial Git plugin (and potentially the official one too) has some stuff builtin to automatically trigger the Unlock on commit, but when you commit outside of UE you will need to manually unlock them. We have a script that we run that unlocks all "Unchanged Files". Very annoying to do, especially if someone forgets it before clocking out.
• Pre-Commit Reviews
  • Git offers Pre-Merge Reviews, but branching with Binary Assets is, in general, a headache, as you still can't merge them. If someone modifies a Binary Asset after you create your Branch and before you commit your own changes to it, you are either in need to override yours or theirs. Best-case would be being stopped from altering it on the branch if it's modified on a different branch or locked, but well.
• Stronger integration into UE and UE Programs, such as Horde
  • Horde, in addition to the generic Unreal Build Acceleration stuff, offers things like "Preflights", where you can have your Build Machine compile a Commit that isn't even submitted yet, to ensure it compiles against all targets and doesn't fail any functional tests etc.
  • UGS for non-programmers.
• There are a lot more things I could probably list, but it's getting quite the wall of text already.

We made Git(Lab) work for Evershire with 4 Programmers and 1 Artist. We had some headache sometimes, especially because GitLab had a bug for quite a while where merging wasn't possible unless all locks are released, even locks of files that weren't modified in the branch the merge originates from.

After they resolved that it got easier to branch out for some new features and merge back in later with a Pre-Merge code-review.

Still with the * that Binary files have to be handled with care, but that also applies to Perforce when having multiple Streams.

[faint cairn]

Pretty sure Perforce is still free for 5 or less people.

[pearl star]

Thank you 🙂 i might setup my personal test env to see what workflows work and automate scripts for releases and general workflows with streams

[faint cairn]

One benefit of GitLab was the builtin runner setup for Automation.

[still quarry]

Yeah, I actually want to try SourceTree.
One thing I dont like in Github is, (idk if there is a feature), that my friend can directly push, before fetching (he is artist), and this is causing automatically merging, or aborting it.

[faint cairn]

Well GitHub is an issue anyway, cause they have a limit LFS storage iirc?

We didn't use GitHub and I wouldn't, unless it's something small or a code-only thing, like a plugin.

We have a self-hosted GitLab instance.

Git is just the thing that drives this. There is more than just GitHub out there making use of Git.

[faint cairn]

In theory you could maaaybe add a Pre-Commit to do something before the Commit (e.g. Pull), but this is ultimately more a thing of your friend having to learn to not just Push.

They probably don't need to Stash -> Pull -> Unstash, cause it's unlikely with the locking system that 2 people touched the same binary file, but yeah.

Git isn't that ideal for UE :/

[still quarry]

Yes, we have already LFS data packs which 5dollar for month of 50 gb. Can we also have smth. like LFS for SourceTree? Which service do you use for repo?
For example, SourceTree as handler, and what about REPO? Bitbucket, still github etc? I dont think i can handle self-hosted things, or servers to be setup.

[faint cairn]

SourceTree as a Software is just a UI for using Git. You can also use the CommandLine/Git Bash or any other Software you are more comfortable with that has the feature you need.
The repo itself is, as already said, a Self-Hosted GitLab instance.

It's hosted on a Hetzner Server that we pay for.

[still quarry]

<hmm, so It might be better we use SourceTree, and Github for the service. Can still we use Github LFS?

[faint cairn]

SourceTree has no effect on you being able to use GitHub LFS or not.

But I stand by my suggestion to use Perforce on a free Azure Server or whatever else you can afford, if you are 5 or less peeps anyway.

Even with more than 5 peeps one can cheat a bit by using Workspaces of the same account.

[strong sigil]

Thanks Cedric and Cedric’s wife.

I’ll take a look at the logs and see if there’s anything anomalous in there.

It’s a large team as well and no one around the same time is complaining about connectivity so it could be a packet drop internally. Will see what the logs look like

[arctic hornet]

Hey there, sorry for the delay… just woke up… I’ll pretty much echo Cedric(s wife) here… smb is quite flaky on best of the days. And this usually means that smb connection dropped or some smb daemon overload or whatever… the error itself is coming directly from windows when perforce says “make me a folder” and windows literally goes “sorry, can’t…”

I personally always run perforce on local ssd based raid … since otherwise you’re leaving ton of throughput and latency on the table (though I obsess about both since I host perforce on 10g lan and you need to have fast and low latency storage to take advantage of speeds higher than 1g)

[long mural]

yeah perforce submission is slow

for the binaries

[woven sluice]

my up is 150 mbps and i'm pretty sure i upload to my p4 server at 150 mbps but i've never actually done a measured test

[long mural]

my up is 600mbs

and im sending to it at 3mbs

:/

something wonky with networking i think

[long mural]

[  5]  22.01-23.00  sec  1.62 MBytes  13.7 Mbits/sec
[  5]  23.00-24.01  sec  1.38 MBytes  11.4 Mbits/sec
[  5]  24.01-25.01  sec  1.12 MBytes  9.47 Mbits/sec
[  5]  25.01-26.01  sec  1.38 MBytes  11.6 Mbits/sec
[  5]  26.01-27.01  sec  1.12 MBytes  9.46 Mbits/sec
[  5]  27.01-28.01  sec  1.62 MBytes  13.6 Mbits/sec
[  5]  28.01-29.00  sec  1.50 MBytes  12.7 Mbits/sec
[  5]  29.00-30.01  sec  1.88 MBytes  15.6 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-30.01  sec  56.9 MBytes  15.9 Mbits/sec                  sender
[  5]   0.00-30.05  sec  56.0 MBytes  15.6 Mbits/sec                  receiver``` something 100% is wonky

[pulsar parcel]

Parallel sync on?

[long mural]

yeah doesnt matter its not strictly perforce

every traffic to it is slow

contacted the provider

[pulsar parcel]

Yeah if that's the case they're the only ones who can help

And that's for everyone right? Not just your connection?

[long mural]

yh

should be 1gbit up/down even with fair use policy which halves it to 50% thats 500mbit

im getting like 15...

[arctic hornet]

yikes, thats slow... good luck...

[willow plank]

Just in response to what you said earlier about me using Azure DevOps. Everything I've read online seems to suggest there's a hard 10GB per repo limit, is that actually true? 250GB will suffice for a single project, but 10GB won't

[arctic hornet]

there's difference between repo size (code) and LFS size. as least as far as I understand the vague writing microsoft is using

[willow plank]

yeah, MS is notorious for vague info. So setting up DevOps, does it take care of large files uploaded automatically, or do I need to implement a custom LFS solution, in which case, how?

[arctic hornet]

afaik you just need to setup your git config files properly, LFS is implicit feature for both azure devops and github... although I don't really know how to do it as I've never used LFS with github or devops (I self-host perforce and have been for past 15 or so years)

[willow plank]

so the git config files would tell it where to upload to, in this case being Azure rather then GitHub?

[arctic hornet]

not really sure about that, as I've said before I've not really used lfs ever. Just try and experiement is all I can say probably.

[willow plank]

are you recommending that I use Perforce on Azure DevOps then?

[arctic hornet]

no, all I'm saying is that I personally use Perforce rather than git... but I've heard that Azure DevOps is better for smaller team than github since it gives larger LFS storage for free.

and that's also why I can't help you with LFS setup, as I've never used it

[wind totem]

Hey @arctic hornet

I saw you posting about perforce set-up earlier, I wonder if I can pick your brain. If it's outside the scope of this channel we can continue elsewhere (or I can leave you alone of course).
I have my projects set-up the "native" way (with the projects inside the engine), but because I have a lot of side projects, I don't want to have 10 versions of the same engine downloaded - so right now I import the engine stream into each of my projects, and set them up in the same workspace root.

The benefit is that I have one copy of the engine for all of my projects, which is always up to date when you sync the project workspace. The negative is that each workspace has its own understanding of the state of the files, so if I am not careful they can get out of sync (a good example is when I update the engine, each workspace has to sync the engine to know its on the correct version).

Can you see a better way of setting this up? I assume this is the best I can do unless I want to have duplicate engine folders.

[arctic hornet]

In general, I tend to have one depot (engine + game) per major project... let's say a big game I'm working on... but then I have ton of smaller prototypes or projects in the same depot just sticking around... one engine install with multiple projects... so you could make one depot let's say "Sandbox" and then have your prototypes there

and if they get large enough to warrant their own depot, you can just copy them over and spin their own engine instance

[wind totem]

How would you deal with the 'out of sync' issues? I find myself doing a p4 sync -k between my workspaces when I update so they are aware that they have the correct files, but this situation is fallible

[arctic hornet]

for example here, I have one main project in /Titan and then project for testing random things that I can push and not break main project inside /Sandbox/Core/DevIteration_Titan

[arctic hornet]

Honestly, I'm personally against one shared engine depot and then just "importing" things as it's annoying

[wind totem]

I basically have my depots like this:

• Unreal Engine
• Project A
• Project B
• Project C

And then each stream has the Unreal Engine and Project X imported

[arctic hornet]

yeah, this one is annoying to deal with honestly.

I usually have "Clean Engine" and then I merge the engine directly into the project (as you can see in the screenshot)

and if I have smaller projects, I just have multiple in my "Sandbox" depot with one engine etc...

it avoids issues when syncing similar to what you're describing and I know it works fine with Horde and UGS etc.

[wind totem]

Yeah honestly Horde is one reason why I set it up like this. I used to have it the non-native way, where I just had a standard depot with the engine, and a separate depot for each project. But I didnt like having to sync the engine and project separately.

But I did like only having one version of the engine. I guess I just need to make a choice whether I care more about storage or protection hah. At least storage is cheap

Alright, thank you - have a good evening

[arctic hornet]

Although here on the server there's couple of folks who succesfully work with shared engine depot, so they might be better source of information than me when it comes to that way of structuring things.

[wind totem]

Okay! If anyone doing that is lurking and wants to chime in, I would be grateful.

[arctic hornet]

I am personally advocating for "have everything in one place" rather than sharing engine since as you've said, storage is relatively cheap + it allows us to have separate engine versions for every project etc. etc. (+ it's the way epic does it, so it's easy way to making everything "Just Work (tm)")

[wind totem]

I am not a programmer so I don't make engine changes at least. I am a tech designer however so I use the Unreal Engine AS version that I used at Hazelight, so I like to build to keep up to date.

[arctic hornet]

I've been part of projects that had a lot of engine tweaks (especially during early UE 4 era) but nowadays we pretty much have clean engine install... but in the end it was always easier for us to have separate engine for each project, since it was easier to come back to older project and everything was still working the day we left it rather than things being broken since we moved 5 engine versions in the meantime

[silver token]

@wind totem FWIW, I've had no problems with using engine streams and importing those to use with projects.
However, I'm still doing what klukule described where each "real" project has a separate stream that is branched off for their exclusive use.

Version Stream -> Latest Stream -> Company Stream -> Project Staging Stream -> Project Live Stream -> Release Stream.

I submit new versions (or apply a patch version) to Version Stream. Copy to Latest. Both of these are pristine copies of Epics code.
Merge to Company Stream (this introduces company approved engine changes).
Merge to Project Staging Stream (introduces project approved engine changes).
Merge down/copy up with Project Live Stream.
Copy to Release Stream.

[rancid steeple]

how to clone project that using source engine?

copy paste will missed up the project

on normal situation i could right click the project in epic launcher and choose clone

[silver token]

How is this source control related? messed up how?
You should be able to make a copy of the project directory.

[ocean anvil]

as long as you copy paste it while it is not in use it should work fine.

[ocean anvil]

i know i'm a bit late to the party for this but i had similar issues trying to put my perforce depots on my NAS, SMB will constantly cause issues as it will randomly block(delay) access to files/folders as they are being read/checked/verified etc. for varying reasons, windows file transfer generally handles this just fine and lots of other software simply reserves files that are currently within their scope which works great with SMB. perforce does not and if it cannot access a file or location on the first attempt it throws errors and breaks.
if your NAS supports setting up an iSCSI storage location that should work, iSCSI is more like attaching a hard drive directly over ethernet. there are pro's and cons to it compared to SMB but for perforce it should work.
(did for me at the time untill i upgraded the storage of my perforce server and now i only write my back-ups to the nas.)

[tulip pulsar]

Hello anyone can help me setting Perforce? I'm trying to using my pc as server

I still don't get what I need to type in server 😅

[wide raven]

well, do you have a server set up? I would presume Helix would have some setup docs?

[tulip pulsar]

I think I'm too dumb cause I haven't found yet how to do it, I looked in the docs and followed few videos but they all starts with the server already running so I guess I missed something

[wide raven]

not dumb! bad docs if they don't go through things comprehensively

might be something like "localhost:1666"

but I haven't done this, so I am just taking a stab in the dark here

[tulip pulsar]

It still return errors
Connect to server failed; check $P4PORT.
TCP connect to localhost:1666 failed.
connect: 127.0.0.1:1666: WSAECONNREFUSED, No connection could be made because the target machine actively refused it.

[tulip pulsar]

Are you using Git in your setup?

[wide raven]

no we have perforce from some big cloud service

[wide raven]

Time to do more search-fu, just start looking at more videos, looking for more docs...
https://www.youtube.com/watch?v=P8DKbF6aQfk

[arctic hornet]

where do you have your perforce server installed? server filed is filled with {{IP_OF_PERFORCE_SERVER}}:{{PORT}} or ssl:{{IP_OF_PERFORCE_SERVER}}:{{PORT}} where port is usually 1666... you can also use domain instead of ip address if you have one pointing at your server

if you don't have a server running, you need to install "P4D" (i.e. perforce server) first

what you have right now is just the perforce client

(similar to how for git, you have the server... and then the git client)

(but unlike git, perforce always works against some kind of server, server rules the whole perforce ecosystem.)

[tulip pulsar]

Ah perhaps that's what I'm missing! Thanks!!

[arctic hornet]

you usually install P4D first but to be also fair, you don't usually use your own machine as perforce server in the first place

you usually host it somewhere else

so perforce setup is pretty much just being angry about that

[wide raven]

Aye, you really oughta have a little 'box in the corner' to be the server, not your dev machine

But I guess pure local is still massively better than nothing

[arctic hornet]

technically speaking P4V contains something called DVCS which is like lightweight perforce server, but it doesn't use normal TCP connection and so it's limited only to the P4V client

so if you just want to use P4V locally, you can... but it won't work with unreal or any other app

[wide raven]

without auto-file locking, kind of pointless

[tulip pulsar]

hmm I see...

[arctic hornet]

well, DVCS works like normal perforce server, so including file locing, versioning etc... but again, as it's limited only to p4v itself you'll have hard time doing anything with it

[tulip pulsar]

thanks for the insight guys!!

[arctic hornet]

although maybe things have changed, it's been about 5 years since I last even looked at DVCS 🙂

[arctic hornet]

Good luck! 🙂

[long mural]

@pulsar parcel thats more like it

was some QOS software installed on the agent 😦

[pulsar parcel]

It must've forgot what the Q meant

[long mural]

lol

did upgrade our server though, not the fastest in the world, but its great for what i need lol

[arctic hornet]

nice nice

meanwhile I'm fighting to keep my agents performant enough since horde can eat all the disk I/O it can and even NVMe disks aren't enough to keep up under certain conditions

[long mural]

:/

[wide raven]

https://tenor.com/view/dungeonkeeper-dungeon-keeper-videogame-fantasy-gif-22645979

[teal bone]

Good ol' Dungeon Keeper.

[rugged roost]

I'm trying to setup UE5.5 100gb project with Github. Does Github allow 100gb+ repos? If it's too big can I pay them to host the project?

[arctic hornet]

github limits repository size somewhere between 2 and 5 GB depending on what you have https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage iirc you can pay for more, but I don't really know much about that, so I recommend checking the documentation/pricing tables etc.

[rugged roost]

Yeah I found the price charts, surprisingly reasonable. Especially for a small business.

[hidden night]

This is such a small thing, but have others found a difference in how UE moves files in between 5.3 & 5.5?

It used to be that I'd move files, then update references, then delete the directories.

And source control would fight me all the way and sometimes things just wouldn't work or I'd duplicate folders.

Now I just... move the files & remove old references & it jsut seems to work...
The Redirector Update Report is 100% new

[ocean anvil]

U really shouldn't use github for hosting UE5 projects though.

It does not like large files, and is primarily made for code, not binary files or other bulky filetypes you would use in UE5 development (textures, video, animations etc .) u're better off hosting on other platforms, if u want to use git then azure devops is generally recommended. (and free)

[dim glade]

Hey all. I have a general source control question for you. My project is intended for release on both Android and iOS. My intention is to work from my PC, push changes to my source control, then pull on my mac and just build. My question is, should I leave files like this out of the push? Or maybe set my mac up to ignore these files when I pull, so that individual configurations for packaging don't get messed up when I make updates?

[arctic hornet]

In general you don’t want binaries, intermediate and saved folders in your version control since they only take space and are just a build products, there are never any manually modified files in there

And if you use version control with support for file locking it can cause another set of issues

[dim glade]

That's what I thought, and my .ignore file has builds in it so I was confused why that .build file made it into the list. Very strange.

[cloud grove]

Hello, I have a question about perforce:
In UEFN I noticed there is many assets in "Read-Only" that we cannot download, is there a way to do the same thing with perforce?
I've read about Virtual Workspace Stream, but it seems to not prevent from exporting a uasset, I am wrong?

[silver token]

Perforce will make things read-only until you check them out (or manually clear the flag through the file system).
There's not really any concept of "can't download" (assuming you have a workspace that maps the depot/stream to your drive somewhere)

[cloud grove]

Yeah thank you, I used the wrong term: I was looking for a way to have the uasset visible on my collaborator workspace, but the source being protected from any asset export / not being on the collaborator workspace, I'm not really sure if it's possible.

[arctic hornet]

I’m not quite sure what do you mean… if you are speaking about selective access on per-file basis (lets say give them access to uasset but not fbx) that can be done in the permissions settings of the server

[cloud grove]

I've been looking for on the documentation and it might not be possible, I'm very bad to explain it but it's like: allowing the collaborator to see the visual of fbx, through an actor (as example) but not having the fbx downloaded on its local workspace

[arctic hornet]

If its two physical files on the perforce server then you can limit permissions on per-file basis… although keep in mind that with unreal you can always export the uasset as fbx anyway

[cloud grove]

Ok got it, thank you

[gusty pilot]

Coming from a traditional non game cs background i'am very familiar with git etc. I see that this is not really applicable with lots of binaries. Perforce seems to be the Industrystandard here and also what epic "propagates". Are there any Talks/Articles that go a bit more indepth about organisation/version control on professional larger projects?

[silver token]

Check the pins. The top pin especially.

But also this talk from Riot about multi-project studio setup has a lot of good info coming from a user of the Engine and not just Epic.
https://www.youtube.com/watch?v=9xxwQVdwcTQ

[gusty pilot]

Thanks, just watching the toppin right now! thats great.

[still quarry]

Are there any possibilities that we can work on same Level with myfriend, without conflicting? We are already trying not to work on same level directly. But Is there anyway of doing that? For example, when I am playing items to my level, he will be editing landscape of our level etc.

[wide raven]

Legends tell of something called "One File Per Actor", but I cannot vouch for it, never used it

[arctic hornet]

if you're using perforce or some other form of source control that works well when multiple people push things at once (git is not really great since you have to merge the trees etc... etc...) OFPA is really the best way to go as Karma said

it pretty much converts your level to just be a container, and every actor placed in that level is it's own separate file... so when working on the level you usually just check out the actors you move/add/delete etc... instead of the whole level

which allows massive levels of collaboration.... it's been successfully used during project titan on massive scale (8x8km world) with ~400 people working on the same map and OFPA was not an issue

I've been personally using OFPA for every project of mine be it with or without world partition (since it's primarily made to be used with WP, but it works even with the old sublevel approach)

OFPA has been production ready since UE 5.0 when using with World Partitioned levels and production ready for non-partitioned levels since UE 5.3 iirc... so it's not even some experimental technology

[strong sigil]

What would you say are your own prefs and advantages for OFPA when using the old sub level approach?

I come from Virtual Production so we would use a master level and then sub levels for everything (lights, geo, particles, animation, cams etc)

I’m still not seeing any tangible benefit for me to make that switch while we still use the old sub level approach. World Partition isn’t a use case for us right now as there is a strange bug or feedback loop when using USD and stage layers with WP (we get a critical crash when trying to load a level again after an import has taken place).

Since we use the sub level approach I mainly try and keep commits and change lists centered around the sub levels

@arctic hornet

[still quarry]

Well, Git is actually doing that too, I think, but with really shitty UNKNOWN names like "CASLKEUQWYUI312089731.uasset." So that i cant really see what i have made changes on.

[arctic hornet]

that's normal, the names are hashed on the disk

even with perforce

[arctic hornet]

I recommend using built-in Revision Control windows in Unreal, as that can decdode the names and so it can display it in more detail

[still quarry]

Aha, than I can also make this work on Github Desktop, but do we need activate it per level? I'm pretty use ThirdPersonMap have it as default so everytime I do change there, It shows files like that.

[arctic hornet]

it's a per-level setting, yes

[still quarry]

What's sub-level approach? Having multiple sub levels in a level or smth?

[arctic hornet]

I'm generally not fond of github since you always have to pull before push etc...

[still quarry]

Yeah. Just for now. I have to use it a while.

[arctic hornet]

you split your level into sublevels based either on geographical location (to facilitate streaming in/out) or by discipline (geometry, lighting, vfx etc...) or both

[still quarry]

Alright, then we manually activate it for levels. And i can debug on the revision control tab on the UE side. Thats looking good actually.

So for example I have InteractionMap, can I just activate it just for that map?

[arctic hornet]

discipline splitting was mostly because two people can't work on the same level at once, and there's low chance that 2 lighting artists will work on the same chunk at once, but much larger chance that level artist and sound artists will work on the same level at once

although with OFPA multiple peopl,e can work on the same level

[arctic hornet]

yep yep

check the docs for OFPA process is described there

[still quarry]

Thats really relieving thing for me. Always scared about making changes in the same level, even if we do completely different things. But still, If we add some Actors in map, wouldnt it count as "Level Change" and shows up as InteractionMap.umap in the github?

[arctic hornet]

without WP most benefits of OFPA are thrown away other than the "multiple people working on the same level at the same time"... although even at our small size (6 people at the studio) we've had moments where two people wanted to touch the same sublevel (we use both geographical and discipline based sublevels) so having OFPA resulted in easier work for us since we did not have to deal with juggling things between two people

[arctic hornet]

nope, assets are only stored in the external actors folder and when the map is loaded it checks for any assets in there and loads them as actors

the only time you change the level itself is level blueprint, world settings etc..

and even then, it's the level asset and not the actors that are locked

the only "negative" is that you'll be juggling ton of small files with random hashes, but as long as you have git lfs set up and are using locking you should have not that many issues even when using git (other than the usual git related struggles 😄 )

[still quarry]

Thanks sir. Yeah. I have Git LFS actually. Not using Locking for now but going to combine them all i think.

[hot ivy]

hi.. I just got a TortoiseSVN repository (on my machine).. I had been commiting the last 3 days but it ate the niagara system I was working on, and it wasn't in the repository anywhere. Fortunately I was able to replicate my work manually. For future reference, if you create a new file in a project do you have to add it manually to the SVN using Add? Its now added anyway.

[inland bay]

Anything you create in UnrealEd is automatically marked for Add by the editor in SVN (or any other VCS as it's part of the plugin interface). Make sure you've got your Revision Control settings set up correctly. I just tested creating a Niagra System in Unreal5.3 and it's was marked for add.

[hot ivy]

strange that it ate mine.. I find I have to manually add.. maybe something wrong with my config.. is there a way of reconfiguring it? See it goes Workbench>Workbench(Project folder)) and the nesting might be messing it up

[hot ivy]

I did the same, made Testhing.. this is how it shows

[hot ivy]

still stays unticked even after a commit

also, I'm not using a plugin.. didn't think there was one for TortoiseSVN..

[inland bay]

UnrealEd ships with an SVN plugin. If you don't configure that for your project then it's not going to add assets automatically. Make sure that is configured correctly.

[hot ivy]

was just reading about this.. thanks.. hmm

[arctic hornet]

You don't need "TortoiseSVN" plugin, since tortoise is just an SVN client and as dredgrave has said, unreal has built-in svn support, so I recommend setting it up, it's really convenient

(I'm speaking generally since I'm not svn user, but generally the integration helps a lot with any workflow you might have)

[hot ivy]

hmm there is a field called Unity Source Control in project settings..

Unity Version Control...

but I'm using Tortoise hemmm

[inland bay]

I think you need to read and understand https://dev.epicgames.com/documentation/en-us/unreal-engine/using-svn-as-source-control-for-unreal-engine before you go any further.

[hot ivy]

yep just reading that now

[inland bay]

specifically "Connecting to Subversion from Unreal Editor". I think that's what you are missing.

[hot ivy]

yep. my repository is local though, not a server

[inland bay]

I've never done that personally but usually for a local repo you use file:///<full path to repo> note the three slashes.

[hot ivy]

going to try a local:// thing

read my mind thanks

[inland bay]

In the long run you should consider having a server though... but it's your call.

[hot ivy]

after getting this working (if possible) I'll look into it

I am doing this for address, but it bounces: file\\E:\SVN_Repos

(four \ \ \ \ discord alters it)

file:\\E:\SVN_Repos

interesting

it's actually reading it and getting an error

svn: E170013: Unable to connect to a repository at URL 'file:///E:/SVN_Repos/Workbench/trunk/Workbench2/Workbench2'

[runic vault]

why are you trying to change the url, you already had the url setup

the fact that the checkout worked is proof of that

now your trying to change something from a working state to a non working state

[arctic hornet]

Unity version control is new name for "Plastic SCM" which was a version control that Unity has bought few years back.

[hot ivy]

ah yes

[analog lodge]

Just linking this here, its more about plugin dev, but includes some perforce concerns....
So if you know stuff about how to handle external / global plugins using source control, feel free to answer as well...:)
Pls ping when replying!

[silver token]

From a source control perspective you may or may not be able to get away from the extra copies. The reason being that you'll want to isolate ProjectA from changes ProjectB makes to that shared content.
You can still put that stuff into shared plugins and even have that in a shared stream, but then each project would have their own version of the stream that it uses. This allows each project to make expedient changes to the plugin which can eventually be migrated up to the shared version and then back down to other projects as they are able to update. This may not work as well for content plugins as it does for code since assets are mergable, but the theory is solid.
For prototype projects that you don't care about breaking so much, sharing a single plugin is probably fine.

[analog lodge]

Well, the whole point of having a global content IS that its supposed to be shared across projects.
So from that PoV, its NOT desireable to have the changes isolated.
I´m always trying to update especially the Master Materials in a way that allows changes to be compatible (defaults not changing etc) and if they would, I start a new version of the Master material and either convert/edit all instances or successively use the new master material for new projects.
Other stuff in the global content folders is reusable effects and blueprints, that are usually only changed in the projects as copies or all edits are made via sequencer/level and not in the assets themselves.
I´ve given up on working in streams for changes, usually I dont have enough time between project to manage the merging process between projects and then it starts to become messier than just using WIP assets and refining them/finalizing them along the way, especially because as you said: Most of these shared assets are binaries.

I´m still struggling a bit understanding how exactly the plugins should be handled in source control.

Rn, I THINK I only need to worry about that when updating stuff in these plugins, by simply switching workspaces once a day and making sure I submit changes for both.

But particularily with redirectors it seems to be tricky, when moving assets from the plugin folders to the project folders and vice versa.

So far, it seems like references still seem to update properly, but yeah.

[silver token]

You do you. I can only tell you what the best practices are for doing multi-project development. If you think those don't apply to you, 🤷

[analog lodge]

Oh, I don´t doubt it makes sense in most cases. We´re just not most cases and 90% of the advice I get are always targeted towards games and then I have to try and filter out the relevant bits of the information, by understanding what exactly the reasoning is and how that applies to our situation...:)

[silver token]

I don't think the advice has anything to do with games though.
Again, if someone made a change to the shared content for needs of ProjectA that were undesirable for ProjectB, they may not know that until someone sees something wrong in ProjectB. And if ProjectB is 24hrs from an important deadline that could be a huge problem. It doesn't matter if you're making games or a tv show or a movie.

[analog lodge]

The reason I´m not "agreeing" with is that you are saying the problem is that shared content should be project specific/isolated:
Our previous/current setup had ONE mastertemplate that was copied per project using import in perforce.
So in theory we already had that problem and it wouldn´t change with switching to a content plugin setup.
Only I was able to submit content to these folders using import+, reducing the amount of human error.
We also don´t usually work on too many projects in parallel, at least not on the Unreal side of things:
Large volume of single projects with similar content and a small user base using them.
Since we barely ever had any conflicts there, I don´t see how THAT would change with a plugin concept.

The problem I am apparently already seeing now is redirectors that can´t be checked out.
Maybe its more about this one folder right now, gotta do more testing, but yeah: as I´m working in ONE project, thats source controlled in ONE workspace and the plugins are in a different workspace, it makes sense that redirectors can´t be checked out.

So, the big difference is in fact more about source control and how to set that up?

I just checked another asset that I moved: It opens fine and all asset references across the different new content plugins seem to have updated properly.
Is still can´t delete the redirectors from the original folder from the previous imported template folders, but it may have to do with me previously not fixing all redirectors when moving the assets I was testing right now, before moving the global asset folder.
I only added the new plugin folders to their own depot today, that was a bit of an oversight.

So, gonna test what happens if I force delete the left over redirectors of a few of the assets giving me that error now.
(unlocking/deleting them through windows explorer).

[strong sigil]

You could just have streams that point to specific locations within the engine install for master templates.

Your secondary streams for plugins in the additional directories you can filter who gets access to edit them and who doesn’t. A “read” only flag.

You would of course be best developing some validator tools in engine to ensure at the point of rendering that all materials and assets that you want locked down haven’t been changed by anyone

[analog lodge]

What would be the purpose of having streams pointing to specific locations within the engine install?
Vs the current "additional directories" I´m having in their own depot?

[strong sigil]

Like I said in plugin dev reply to you,

You could put your master template location into a folder in the engine install. So everyone gets it but you can then independently update it between projects without anything breaking

It’s using both the template system in unreal for projects

And the additional plugin systems for best of both worlds

[analog lodge]

ah, sorry, haven´t noticed that yet

[woven sluice]

i didn't read this whole novel but i will say this:
We´re currently using a Mastertemplate that we import+ using perforce in all our projects.

import+ is different from import

if i were doing this (trying to set up a master shared thing and avoid having intermediate copies per project) i think i would be using import not import+

[analog lodge]

Yeah, sorry, import is for all other users, import+ just for me who is maintaining the template/plugin stuff.

[analog lodge]

Ok, sorry for crosspinging, just trying to keep thing separate...

So, in your setup: How do you handle source control?
Like I´ve mentioned: My issue now seems to be, that while working inside Unreal and connected to the projects depot/workspace, I can´t fix redirectors pointing to or from the plugin folders, as they are not checked out and I can´t check them out from within the project.

I CAN then go though pv4s GUI and just switch to the plugin workspace and manually check out the assets in question and THEN I can go back to unreal and fix the redirectors just fine.

But thats already sounding very dangerous in a day to day scenario....
I could try and just check out ALL plugin folders before I start working on a project and set it to also keep files checked out after submitting changes, so I could just work in the unreal projects in peace and would always be able to fix redirectors etc.
Just seems like...not the way its meant to be used...

[wide raven]

without knowing the full context, the manual checkout angle seems to me the correct way of things, unless this is something you are spending way too much time on per day, in which case i might consider building some editor tooling around it

[strong sigil]

Yeah we explicitly have a super user check out
All our plugins and we also have write access removed from all standard users.

They can still mess with it in engine but then our validators flag that at the moment of rendering with a log as well as burn in on WIPs to indicate non conventional assets

[analog lodge]

I see, but do you just NOT update plugin content constantly, as we do, or how do you handle changes happening while working on a project, that need to be updated to plugins folders? Like, new templates/reusable assets/material instances etc, that are NOT project specific?

Like, 80% of our work is medical and while we have a large archive, it just continuosly grows.
And right now the only way to deal with this in a clean, corret way, seems a bit unusual to me (keeping plugin content constantly checked out).

In the end, I´d still want standard users to also be prohibited to make changes and I´ve also previously set it up, so I wouldn´t change much there.

But for myself I´d like to have a solid setup, where I can just work on stuff without constantly having to switch workspaces etc.

[arctic hornet]

I'm not 100% following the discussion, but what I'd personally do is to use one big depot/stream with multiple projects inside instead of importing modules multiple times... that way everything's everywhere only once and you can then use tools like UGS to automatically download specific project only iirc...

• if you want some files read only for specific users, just use perforce's ACLs

[silver token]

You could probably create a virtual stream off the one everyone else uses. The default stream has the import, your virtual one changes it to an import+

[arctic hornet]

or just in general, leverage perforce permissions to deny access for normal users by setting permission for let's say //.../SharedPlugins/... to readonly and then set it to write for the "Maintainers" group or something, no need to then deal with import vs import+

that way you can have single setup for everyone goverened by the groups they're part of

[analog lodge]

Yeah, permissions aren´t the issue rn.
Its how to set up plugins so users (including me) can continuosly work in ONE workspace, without having to constantly switch workspaces (well, for me, users shouldn´t be allowed anyways) to make changes like moving stuff from a project folder to a plugin folder.

I haven´t used virtual streams yet, gotta look into what exactly they are.

[arctic hornet]

virtual streams are simply streams that are purely defined by the imports etc... they don't have physical files of their own

they're usually used for narrowing the scope of parent workspace(s) etc.

[analog lodge]

Our initial setup involved just ONE large ueproject and then all subprojects resided just in their own folder inside that one project and could be synced separately.
But that had multiple issues I don´t remember in detail right now...:)

[arctic hornet]

in general I'd say "just use one stream" for everything, that way you won't need to sync multiple workspaces

though every project would still be it's own uproject

and if using UGS you can then specify custom sync filters so you can sync just engine + project A or engine + project B etc.

so you'd be able to limit the scope

and when changing sync filters UGS only does the bare minimum to get from from state A to state B

so it won't redownload everything etc.

[analog lodge]

I might have a slight misconception about streams...
I thought I need one stream per unreal project?
not sure if this isn´t derailing the conversation, maybe I have to read up a bit more on that...

[arctic hornet]

stream is just like a branch on git, whatever you put into it is in it

unreal does inherently support mutliple projects per stream

[analog lodge]

I mean "import" does exactly that, it points to a different location in the depot, but that was part of the original problem, as we kept making new projects with new workspaces and every one then had the import content copied to its unreal content folder, which lead to tons of unecessary data.
And the "plugin from additional directory" promises to fix that problem, but not requiring a sync to each new project for each new workspace.
So, without trying to understand how else a stream might be used completely:
Does that help you answering, wether your idea would also adress that issue?

[arctic hornet]

if you want to deduplicate the amount of data, best way to do it would be to have one big stream aggregating everything together

since by that logic user would then only have one stream and thus workspace to sync

and you could use selective syncing in UGS to sync engine + shared and project specific files only without requiring syncing all the projects

[analog lodge]

But would that locally need to then also be in ONE big space?
Because we currently separate that, each unreal project also resides in its own folder structure per project.

And as far as I understand, you can import from different depots residing on different server locations,but they all need to sync into the same workspace, aka local directory?

[arctic hornet]

it would have one common root, yes...
let's say
//MegaDepot/Main/...
synced to D:/P4/...

and the mega depot would then have:

Engine
SharedPlugins
ProjectA
- Source
- Plugins
- ProjectA.uproject
ProjectB
- ...
ProjectC
- ...

and UGS by default should only sync "Project + Engine" iirc... and even if not, set up custom sync filters... so if a person would be working on ProjectA, they'd only sync Engine,SharedPlguins and ProjectA folders

if that makes sense?

and you can switch between projects just by changing the sync filters, which is couple of button clicks away

still one workspace, would allow for arbitrary selection of projects to be synced at once etc... but at the same time everything would be on user's machine only once

and if you already have existing projects, you could always build a "SharedStream" by using bunch of import+ and aggregate it that way so you wouldn't even have to do much on the server/devops side of things

and honestly, iirc UGS already does the "per project" sync filters automatically

by default it just syncs Engine + project folder

[analog lodge]

Hm, yeah, sounds exactly like we initially set up. We still use this for larger projects, like a tv show with multiple episodes, where each episode then just resides in its own folder.
But that just doesn´t make sense for many smaller projects that are completely unrelated in content, except for the shared content like the master materials/basetextures etc.
For archival reasons alone its just much safer to have every project in its own self contained directory, without having to reference a "megadepot" folder, which also includes maintaining/archiving the depot.
I think that was one of our inital concerns, runnin out of cloud space to quickly and having to then manually sort through everything in that megadepot and make sure we don´t delete stuff used across multiple projects, breaking references etc.

[arctic hornet]

you could use the "mega depot" approach just for easier sync by using it like an aggergate by importing other streams into it

[analog lodge]

TBH, haven´t even looked into Unreal Game Sync yet.

Hm, I see...thats an interesting idea too.

[arctic hornet]

ugs just makes it more convenient to sync since it does bunch of filtering (both built-in and user configurable) and does present the changes as simple list where you jsut click "sync" at it works*

but it requires engine source code to be on perforce too, since that's the "Epic way"

[analog lodge]

Not sure how that would play out though...you´d start the project in its own project directory and then go to the megadepot project and import that project into the megadepot workspace, but only temporarily, as long as you need it?

[arctic hornet]

I'd say more, import it there until you have to archive it

[analog lodge]

Right.
So, you´d basically ALWAYS work inside the "Megadepotproject", but the files themselves would actually sit in a different location.
Thus leaving everything in the "masterdepotproject", including all the content in a "masterenginetemplate" or "content plugins", as I was planning now...

[arctic hornet]

pretty much just unifying into one big stream/workspace for ease of syncing, yes.... and the way you achive that behind the scenes - be it by physically having the content there, or by importing it - that's up to you

[analog lodge]

Yeah, NOT having the content physically there was the main reason we stopped doing that.

Shit, seems like a very stupid oversight to not think "in reverse" when trying to figure out how to manage shared content...

[arctic hornet]

yeah, I was just sayting that the way you achive the "monorepo" like state is all up to you

bunch of imports seem quite a bit easier to manage with your existing structure + you can pull stuff in and out as necessary

I don't have a real example of this, but as long as your streams follow this structure (by having Engine in the stream) then you can use UGS, which would simplify the syncs for you... you can have mutliple "Matcho" (name of our game) projects next to each other in folders and UGS will only sync engine + selected project (and whatever else you tell it to)

since in UGS you get promp like this

so you can select "ProjectA" or "ProjectB"

and since it's all based on sync filters UGS will automatically clean what you are no longer syncing etc.

[analog lodge]

Dang. Yeah, it seems very easily manageble, compared to what I´ve been cooking up previously and even better than what I was coming up with now.
Major concern than would just be maintaining the monorepo, by regularily archiving stuff, so it doesn´t completely explode in scope.
But that should be as easy as simply deleting the "import+" path mappings to the projects themselves and then disk space should be freed up or would need to be manually backed up/archived/deleted.

[arctic hornet]

combined with custom Sync filters for the "SharedPlugins" folder which is like 1-2 lines in INI config file you can easily do it 🙂

[analog lodge]

But the monorepo itself would basically be the same as my master template is now and continuously grow with shared content.

Well. Its not all lost yet. I still have the current master template.
And even if its too much work rn to also get Game sync working...the result would be mainly the inconvenience of having to make sure to go to your current project folder before clicking "get all", so you don´t explode your hardrive with ten terabyte of projects being imported...

Which also shouldn´t happen anyways, if we at least maintain the import mappings regularily, so at most the 2-3 projects we work on in parellel would get pulled...:)

[arctic hornet]

yeah, main benefit of UGS is that it does the filtering for you...

[analog lodge]

Dang it.
This sounds both too stupid to admit AND to good to be true...:)

[arctic hornet]

• it presents things in nice and ordered fashion 🙂

[analog lodge]

no doubt.

Ok, maybe one more question before I go pester chatGPT again...:
So, whats the requirement on perforce end for this to work?
As I wanna import all the projects into the megadepotproject then, I would
A) Set up a stream depot for the megaproject.
B) Set up another stream depot for the projects (or multiple, as we do now, we set up one per year).
Or would they all need to reside in the same streamdepot?
Stupid question, as its the exact same now, just in reverse and thats probably the answer....

I´m just really confused as why this never once occured to me or was suggested...

[arctic hornet]

whatever depot you use/create can be used, the only requirement is that it's a stream-based depot... then create the stream, use bunch of import+ or import statements to aggregate all your projects into one big pile

that's it

pretty much:

import+ //Shared/Engine/... /Engine/...
import+ //Shared/Plugins/... /SharedPlugins/...
import+ //ProjectA/Main/... /ProjectA/...
...

or whatever your structure is

[analog lodge]

Alright man.
This came at a very good and very bad time.
Very good, because I haven´t actually finalized any restructuring, very bad because I now wanna try this out and iron out potential issues as fast as possible, but I´m supposed to stop working and go home...:)
Thanks anyways, hope its ok if I ping you again, if I encounter something I can´t think of right now...:)

[arctic hornet]

😄 good luck and feel free to ping me whenever you need to (although best to go through here at first, since I have my DMs set to be ignored by default so there might be higher delay before I answer for the first time)

[livid bane]

is this important?

I mean that uassets

[silver token]

yes, those uassets should be in source control.

[analog lodge]

Oh yeah, no worries I´d only ping you here, not dm you, what am I, a stalker? 😉
I did have a bit of a brain fry moment now while trying to get ChatGPT to reason me out of my anxiety of breaking thing worse with this workflow and I THINK I got it sorted out again, but its something still floating in my brain right now....
Mostly caused by the confusion of setting up a workspace thats essentially empty, before filling it with content from the "Masterdepotproject".
Like..."but there are no other folders in that project workspace to reference anything, no redirectors will work etc".
Yeah, dumbass, because you never open the project from that folder, you always work from the "Masterdepotproject" and all the folders are THERE, including any redirectors.

Only way to actually break things would then be to reference assets from the project folder in the mastertemplates global content folders.
Which might still happen, but it would´n´t break things worse than the other way around (master template changing and breaking stuff when resyncing an older project in my current workflow).

Anyways.
Thanks again, I´ll get to work tomorrow...:)

[livid bane]

i hate git sometimes

i mean git is mostly fine, but lfs is straight ass

[faint cairn]

We are doing fine with it. I'm also advocating for switching back to P4 on our end.
We aren't enough people to fill out the 5 users limit, so it would be free (minus Server hosting, but we have Servers for that either way).

But on the other hand, Git(Lab) offers basically the same support that P4 + Horde offers (and one can use Horde's UBA without P4), and if one wishes to branch, then P4's branching is absolute "ass".
Not sure we had many problems with LFS. I would need to ask again if we did anything special there.

Biggest loss, if "you" have a team is UGS I guess. But that can be automated for Git too fwiw, just not with the same nice UI.
There are people working on this though, and I assume some Git Clients can help with that.

--

Speaking of Git Clients, those of you who are using Git (Hub/Lab/etc.) for UE, what Desktop Client, if any, are you using?
I'm not the biggest fan of CLI stuff, as I don't want to remember all the commands. I've been using SourceTree for a while, mainly due to its LFS support.
But that's also an absolute pain sometimes, where it locks up, fails to authenticate properly, etc.

Currently looking at GitKraken (they have a nice sale for their Pro License, which allows connecting to our self-hosted GitLab instance).
Anyone having any suggestions or some experience with GitKraken or similar to share?

[wide raven]

Fork! ( Disclaimer, haven't used LFS )

[livid bane]

default windows Git Client

but honestly I just add files from terminal, easier to filter stuff out than click and select stuff manually

https://tenor.com/view/the-road-to-el-dorado-both-both-is-good-gif-8304204

With LFS I just can't setup Repo xD

LFS is just for Large File Storage, which just keeps files as pointers and stores them on different server

[inland bay]

GitKraken is good and scales pretty well for power users, Fork is good and better for non-technical users imho. however in my experience git+lfs doesn't scale as the team grows beyond 5 or so asset creators. Invest in something that handles binary files well like P4 or subversion.

[teal bone]

Unfortunately p4 is just stupidly expensive.

[faint cairn]

Are you sure your Client actually supports LFS? That can totally explain why you have trouble :P

[faint cairn]

We were working with 5 users via self-hosted GitLab and LFS. Despite the fact that Locks aren't directly tied to the file being altered (unless using the UE plugin) we didn't notice any issue.

[livid bane]

I was able to push few files and pull in ither directory

I used termianl, so not sure what client you meant 😄

[faint cairn]

Ah you meant the CLI. Well, I specifically wrote that I'm asking for non-command-line solutions :P

[woven sluice]

i am not a git pro at all. but my only beef with gitkraken has been that it could barely at all handle interacting with the UE repository. like, it can clone it just fine, but then after that it was almost impossible to check out specific commits or something from it. i can't quite remember my exact problem... it was last summer. i just recall like it was choking when trying to interact with such a massive repo. which could be kind of a hugely serious problem, but not one that affects me 99.9% of the time with what i normally do so i've otherwise been happy using it

[woven sluice]

i think i was trying to right click on a branch or tag on the left side and it just wouldn't open the context menu (like it would but it would take 7 years) and in my cruddy little 200 commit repos they were responsive just fine. i don't have a full source engine clone right now to try it again since all i use now are shallow clones

[limber tartan]

Because you asked. You can look at Anchorpoint, which has the option to mark all binary files as LFS automatically and has an Unreal plugin, so you can checkout assets with file locking. It requires a subscription for features like file locking. I am also one of the devs of Anchorpoint, so happy to help.

[faint cairn]

Pretty sure Anchorpoint was a full replacement for Git/P4 or not?

And if I'm not mistaken Anchorpoint has no self-host option, or?

Not sure that was the same thing I remember

[limber tartan]

It's an addition to Git. It pushes files to GitHub, GitLab etc. It manages metadata in it's cloud servers. Metadata is used for tagging of assets, file locking or for setting up a centralized Git config. But production files are in a normal Git server.

[faint cairn]

Yeah source tree does the same. It just dies if you click on something with a lot of files

[faint cairn]

Hm right. But no self hosting option, or?

[limber tartan]

There is. You can also self host the metadata server. But 95% of our users don't do this unless they work in industries that require strict compliance such as automotive or engineering etc. And for production files, you can push them to your self-hosted GitLab or whatever.

[strong sigil]

Hey @limber tartan does the team plan come with the asset management?

And what is the definition of user?

With perforce we’re able to use service accounts for automation and scripts. Is there similar in Anchorpoint?

Similarly is there user management levels inside it that we can utilise for permissions?

Never mind for the team plan question. I see the self hosted is limited to the studio. So the team plan for me is moot

[limber tartan]

Hmm....sorry I did not understand the question fully. So the Team Plan and Org Plan are bound to a workspace. It's not like a Perforce workspace. It's more like a Slack Workspace or Discord Server. When talking about service accounts, do you mean generic accounts that are not user connected?

[strong sigil]

Yeah,

They aren’t tied to a license since they’re restricted to a set of commands etc

With it being bound like discord / slack does this mean 1 user can potentially log into a laptop and workstation under the same license? Only asking to get clarity since that would likely be a question our cybersec team would ask so I’m pre empting it

[livid bane]

I think I fixed it actually

I runed out of space when commiting 🤔 so something went wrong

<@&213101288538374145> solved

[frank hedge]

i have 4 waiting to be pushed, but all the time i have the internet error, i would like to push them 1 by 1, is that possible?

[limber tartan]

Yes, one user has a license that he/she can use from any machine with a username/ password. It's like a floating license. For machine licenses we usually don't require a user license.

[strong sigil]

Nice thanks for that,

I’ll book a call for the custom option. I’ve got linear content creating clients that don’t use the typical gamedev oriented setup for perforce and git so this with the DAM system could be quite a nice setup

[livid bane]

do you guys commit solution file ?

[silver token]

No because it’s not useful to. You can create it using the batchfile that is in source control if you need it.

[livid bane]

I know I cn create it :d

[frank hedge]

git push origin master --force --depth=1

anyone know this command?

[teal bone]

I'm sure a lot of people are familiar with git.

[still quarry]

How can i activate OFPA just for a level that I define, I did Use External Actors. But what about conflicts now? How we are handling conflicts since those binary files are hard to read? For example, If I make changes in scene, dragging an item to left which is BP_Spear, it looks like ASD12309JLK3128.uasset, but is this defined code for that asset? for example next time i drag this spear, is it same .uasset again so that i can know that remote also changed spear

[ocean anvil]

you shouldnt have to handle conflicts as there should not be any. you want files that are being worked on to be locked out so only 1 person can work on them at once, the reason u use OFPA is so that more people can work in the same area without having to lock out the exact same thing as someone else as often.

resolving conflicts in binary files is basically never done.

also as far as i am aware OFPA is activated on a per level basis by default.

[arctic hornet]

As WTF said OFPA is indeed set per level (see the documentation for OFPA, it’s well described there) and the IDs are unique per actor, so if you drag new spear into the scene it will have different ID… so there should never be any collision unless multiple people are modifying same actor, which is where locking comes into play

[still quarry]

Thanks! But if we edit same Spear at the same time, would this shown as same ID in both user? I am its a unique id that is not changing everytime?

I'll look into file locking with Git LFS.

[dense fulcrum]

Any ideas why UGS hangs on downloading files labeled as unicode in P4?

And if it's to be expected, any ideas how to rapidly replace the unicode file type in P4 to text without having to do it manually for 100s of files?

[wide raven]

maybe this https://stackoverflow.com/questions/35526466/notepad-convert-to-utf-8-multiple-files

[dense fulcrum]

It's not the actual encoding I need to change, just the way P4 sees them

Most of our issues seem to come from the /Engine/ThirdParty/ folder, so I'm considering just making everything in there be seen by P4 as binary+w

[wide raven]

https://stackoverflow.com/questions/74987979/can-you-set-the-perforce-filetype-especially-revisions-stored-of-a-file-in-the

[dense fulcrum]

Alright, got that last one to work as a dry run, now to wait for the revert of another attempt to be done so I can run this and see if it works

[livid bane]

ask question maybe? :d

[pulsar parcel]

classic https://dontasktoask.com/

I was unfortunate enough to use Git at my previous studio, and really the only times I had to force push was after rebasing

I don't think --depth does anything here

[frank hedge]

i was expecting someone to say like " yes, this command is doing .... X thing "

[livid bane]

if you don;t know what force does I think you should avoid using it 😉

[frank hedge]

But now i follow some recommendation and try to use devops

but then i have this

I need to "split" it i guess but im not confuse

[livid bane]

xD

i guess 5GB is the limit

its not even git related

[frank hedge]

Yes, but how can i do

[livid bane]

do what?

your project is too big

bro

or maybe just push

[frank hedge]

Yes i understand, but how do i send it to azure anyway? too big, okay, but how do i split it

[livid bane]

do smaller commits?

[frank hedge]

since i turn on azure, i have to send all the commit again (wich is okay) but the full think is more than 5gb, and its aparently not possible in 1 time

so i need to send like 1st half and then 2nd half maybe? But if so, how

[strong sigil]

Push smaller commits with sub folders of things

[frank hedge]

my problem is i dont understans exactly what it means

[wide raven]

You have a big project with lots of stuff ( hopefully arranged in some semblance of order )

• add + commit all the audio folders
• then all the texture folders
• etc

[frank hedge]

You mean i need create a new project ?

[wide raven]

No... I am a bit rusty on git, but I assume you could just go Project/Content/Audio, and just add that folder for example

[frank hedge]

thats the thing i dont understand.. How do i do that ?

[wide raven]

I presume you have

• an unreal project, unversioned
• a git project, blank

[frank hedge]

github desktop connected to azurez

[wide raven]

so now try just adding the README.md, nothing else
https://docs.github.com/en/repositories/working-with-files/managing-files/adding-a-file-to-a-repository

[frank hedge]

github link, is it same for azure then?

is it like.. manually upload the file?

if yes, and if for exemple I manually upload ALL the files ( i wont, its for the exemple )
will my github desktop understand it and says that all the commit are done?

[wide raven]

azure doesn't care what you use as the git system : commandline, or a git visual client like SourceTree, Fork, etc etc

[wide raven]

essentially yes

[wide raven]

yes, but since a lot of hosts don't like receiving x Gigabytes at a time for some reason ( timeouts? ), it is best to break up the initial add+commits into manageable chunks

[frank hedge]

So.. I can manually upload some of the folder ( such as .. Music folder for exemple ) then github desktop will understand by itself that I upload it (?reallly?how?) And so when i will want to commit, it gonna send only all the other folder, and so reduce the amount of file i send in 1 time
is that what we are saying?

[wide raven]

github desktop is a git visual client itself if I remember correctly, you should be able to add + commit + push from there... i think that was the git chain, i am on perforce now so i am a bit rusty

[frank hedge]

or i dont get it, or we are talkint about the following screenshot, but then its just a push of all the commit and i dont know how to split them ( and so its not working )

[wide raven]

hmm, so you have a lot of local changes already committed, and waiting to be pushed i think

hmm hmm

I think I will step down and let somebody who is more fresh with git to advise you from here

[frank hedge]

So i was pushing all my commit at the same time i commit them
But then i didnt have internet, and so i have 5 commit waiting to be push, didnt work
So i move from github to azure,
So i have to push ALL my commit, but its too heavy and not working

and then im just lost

[wide raven]

I think I would do it like this

• take a backup copy of the whole project just in case
• delete the .git stuff, files and folders
• create the local repository again
• start adding things in pieces from there

[frank hedge]

Thats what Im doing now, seems to work

i probably need long time but at least it seems to be working

[wide raven]

good to hear : )

[frank hedge]

i have my first commit on azure now

but

should it not be invisible here after i commit it?

or the " change " will always show all the file

i dont remember

[wide raven]

hmm. Are all those files in the repo now?

[frank hedge]

also, weirdly i have the " vehicle template " folder waiting to be commit/push.. but i dont have this in my project

[frank hedge]

You mean the repo in azure ?

[wide raven]

Changes (+) would seem to indicate that they haven't yet been added, commited, and pushed i think

[frank hedge]

Im maybe confuse with these name, im pushing some thing and do some test and come back here to ask if my question still exist

[wide raven]

Once all Changes have been committed, they shouldnt show as changes anymore

[frank hedge]

yes right

gotcha

and so yes

i commit some new and then okay

waw

im happy i didnt have problem with my project

[wide raven]

I think you are on the right track. Not sure about that vehicle thing though, that seems odd

[frank hedge]

apparently.. i solve my problem

But i also realize i was able to solve it earlier without doing all this

But anyway i learn something now

Once i finish to upload everything, i will try to "pull" to see if everything i save can actually save me lol

• I usualy also send my code ( not the 3D model etc ) to googledrive

in case of worst scenario, getting back at least my code will be enough to not kill myself

[wide raven]

if this is your first time setting up version control, totally normal to struggle a bit
it's a good thing, it will save your ass many times over your career in dev : )

[frank hedge]

I had github before, but was not able to push for some reason, then someone told me to switch to azure

But even i never pull on github

but i realize by doing what you said that i could had fix my github issue with the same system

but.. anyway, if azure is better according to most of people, i lost 2 days of developing but at least im MAYBE more safe now ahaha

(i was unable to push since 6days now)

[wide raven]

It is indeed a good idea to test a pull of the whole project to another machine to sanity check you have everything and can boot up the project from scratch as needed

once you have done it once, it should be fine for the future

[frank hedge]

hopefully

but I still use google drive in parallel, just in case

[wide raven]

well, that seems like overkill to me

[frank hedge]

one of my friend lost his project because he lost his account for safety reason apparently

[wide raven]

Ouch!
but on this general subject, one thing i heartily recommend as the next step on the gamedev pro level journey is to consider at some point setting up an autobuilder ( Jenkins, TeamCity etc )
This will allow you to have a builder machine packaging on every change, verifying that there are no breaking changes going in by mistake
On a bigger project with multiple devs, this is pretty much industry standard.

So the idea is that when it comes time to package a demo, or whatever, you never get a nasty surprise : P

[frank hedge]

Im actually surprise that source control is not more "popular" for beginner
there is nothing speaking about it on youtube if you not look for this specificaly
i could have finish a game without knowing the existence of that if i was not on this discord probably lol

then i realize how important and powerful is it

[pulsar parcel]

It's not a sexy subject but it's vital

And a necessary skill to work professionally

[wide raven]

Yeah, it is indeed. I am also shocked how often indie devs are like "what is dat"
So I have taught some gamedev courses now and then, and I always make it mandatory to submit work through version control

[frank hedge]

they should create an Unreal Engine cloud
even if its paid
would be so much easier to set up and to use

[pulsar parcel]

a common story on this Discord is:

1. someone describes a problem easily remedied with source control
2. someone replies "have you checked source control?"
3. OP responds "what's that?"

[frank hedge]

Sorry, as we are here, i prefer to ask now
what is the " proper way " to pull a full project from azure ?
im afraid it conflict with my existing project when i try to use and so not being able to see if it REALLy work
Maybe i can use just another account on my PC ? what u guys think

I want to push all my project, when its done i want to try to PULL and open it again from scratch, such as if i lost everything, or if my computer explode lol

[frank hedge]

this is exactly how i discover source control.. I was asking what people use as backup, because i use 1 more save in my computer + 1 FULL PROJECT in google drive every 2 or 3 days

[wide raven]

Just create a new blank folder somewhere, and ~~pull ~~ clone it there

[pulsar parcel]

you clone it in another location

that's it

[wide raven]

classic : )

[frank hedge]

oh yea right
I actually dont even need another machine, i will be able to see if my files are there, its probably not even useful to actually OPEN the project in UE5, isnt it?

[pulsar parcel]

you'll want to ensure your project compiles (if there's code) and can be opened without issues on other machines

[frank hedge]

right

[wide raven]

optimally it's another machine to verify everything is tip-top, but local other folder is next best thing

[pulsar parcel]

what if the storage on your machine just breaks overnight? a fresh clone is what you'll be getting

so if you're unsure, you'll definitely want to verify it

[hoary wren]

Could use some advice. Do folks here have a format for their perforce changelist descriptions that they really like and that's worked well for them in the past?

I want to make them consistently and maximally useful to the team when looking at changelist history, but without making them real onerous for the team to write every time they have to make one.

[pulsar parcel]

not really other than making the Jira task really visible

[wide raven]

We do use something like this

ADD: Anything involving new files added into the project
BUILDS: Build scripts etc
CHANGE: Adjusting logic/content to fit new specs
CLEANUP: generally should be non-destructive and very safe
DELETE: Deleting stuff, potentially dangerous
FIX: Bug Fixes
REFACTOR : Concrete changes affecting code flow. possibly risky

Some studios indeed have the link to a project management task in the description, we tried that but didn't really use it much. However, the converse : when a ticket is moved to QA, we prefix the ticket with the corresponding CL number so QA doesn't have to wonder whether the changes are in the nightly builds or not

[pulsar parcel]

we tried that but didn't really use it much
we actually forced it to exist in the commit message or have some exemption tag like #nojira

and it's nice since in UGS, Swarm, etc you can just click the link and go straight to it

and with Swarm and its Jira integration, you would also end up with backlinks to the commits and code reviews

[wide raven]

Yah, if it can be really short link its nice, and the bigger the team the more likely to be useful i imagine

[lucid laurel]

Anyway of filtering specific kind of changes? By just compiling a control rig, Source Control says that every link has been moved, cluttering everything. I want to filter out "Link moved" changes.

[frank hedge]

finalyyyyyyyyy

[wide raven]

gratz : )

[frank hedge]

on the 6th day god finish to upload everything, and going to rest on the 7th

[smoky igloo]

Is it possible with Perforce, that I can just revert a file in the current change list temporary?

I'm not sure what introduced the bug and now I want to check it file by file.

[silver token]

Yes, you can use the shelving feature to save the revisions and revert the changes.
Shelving is not available on the default changelist so you have to move changes to a numbered CL first.

[smoky igloo]

i created a new change list and now have right clicked and shelved hope that was right ^^

No the files are still there

Also it is not possible to check blueprint changes right? 😅

[arctic hornet]

While shelving a file you can check these two checkboxes which will locally revert the file(s) to the state they're on the server while in the shelve they will still be changed

You can then either Unshelve or delete all files by right clicking ont he "Shelved files"... or do the same with specific file by right-clicking on that file

one changelist can contain only one shelf, but you can have multiple "WIP" changelists and thus multiple shelves

shelves are also useful for transferring things between users or machines, since the data is stored on the server, so if the shelf is visible to all users they can unshelve from it (or so can you on different machine for example)

[arctic hornet]

in case you forget to check those while shelving the files, you can also safely then just revert the file in the changelist locally and it will stay in the shelf

[smoky igloo]

Thats really cool to know!

[smoky igloo]

Ohh I thought they both where checked, but maybe not and I was to scared to then revert the locally still checked out files, maybe I should play around with it to be more confident about that for the future 😅

Thanks a lot for the clearing things up for me (and the tip!) 🙂

[hoary wren]

I'm trying to see if we can distribute internal playtests of the game through Steam, before we even have an official coming soon page up and running. I'm hoping to be able to use Steam's rails to deliver the latest build to the team and also to a few internal playtesters. We already have a default and development branches populated with the builds.

What's the best way to make this happen in your experience? I've seen Steamworks recommend that all of your team members are added to the Steamworks account, but also that you can give out up to 2500 keys: https://partner.steamgames.com/doc/features/keys

An official playtest is probably not a fit here since we aren't public yet.

Seems like there are a ton of options, curious what others have explored?

[wide raven]

Use the password protected branches i reckon.

[dense fulcrum]

Anyone has a successful setup of the Metadata server for UGS? I'm trying to understand the UGS Reference instructions page for it and it's not working

[arctic hornet]

UGs metadata server has been deprecated for many years, UGS is meant to be connected with horde which provides metadata server implementation inside

[dense fulcrum]

The system that allows to flag builds as good, bad, etc. is deprecated?

[arctic hornet]

the standalone metadata server hosted in IIS is deprecated, yes...

as I've said, it's now part of Horde itself

since horde job can automatically output <Label> which has UgsBadge="XXX" attribute... and all the other features like marking builds good, bad, manual tag pushing etc etc... are also implemented in there

although if you're not using Horde (or are not planning to) I'd probably recommend rewriting the metadata service using .net core or something more modern rather than ancient .net framework version dependant on IIS

[dense fulcrum]

We'll be moving to horde eventually so I'll shelve that

[tiny epoch]

Hey folks, I wonder if anyone here has any insight into this. I'm developing in UE5.4 and use Perforce as source control. Sometimes I want to "replace" one asset with another (let's say I have SM_PlayerGun, and I import a new gun asset, and I want that to become SM_PlayerGun). Ideally, I want to replace the asset and commit the change in a single changelist. But, if I delete the old asset and want to add the new one, then the asset is marked for delete in Perforce, and if I revert the delete action, it reverts my new asset to the old one.

So deleting an asset and adding a new one of the same name in the same CL isn't supported in Perforce (AFAIK). Normally, the process for replacing files is to check them out and replace them on disk, but if I try to do this with the .uasset files, UE5 doesn't recognize the changed assets when I launch it (the files no longer appear in the Content Browser). So it seems like UE5 expects me to delete/add files, whereas Perforce expects me to check out / overwrite files.

There are workarounds, such as just using a different asset name, or submitting an intermediate CL that deletes the asset (before adding the new version in a subsequent CL). But neither of these are ideal. Any ideas?

[arctic hornet]

well... if it's a static mesh, just reimport it instead of removing and readding it...
another way is to checkout the file, remove it on the file system and then import your new file
anotherr way is to remove old asset on the filesystem, import new asset and then reconcile offline changes

but preferred way would be to just reimport

but a TL;DR is ... you want to convince perforce it's an "Edit" operation rather than Delete/Add etc... which can be achieved in the ways I've outlined above

[strong sigil]

yeah how I would do this would be a "reimport with new file" if it's a new mesh or whatever. Then explicitly describe that new asset in the changelist so that I can go back through months later if needed, find it and revert that changelist back so I can get the old mesh (providing I have enough history to return to)

[tiny epoch]

Ah yep, good points. I suppose static mesh wasn't a good practical example but this could happen with any .uasset. Reimport with New File makes sense when I have a source asset to import. Sometimes though it may be an asset that I haven't imported from a source asset outside UE (let's say it's a mesh in an asset pack from fab or something). In that case I guess the process would be to export the new version of the asset, then Reimport with New File on the old version of the asset (or via @arctic hornet's method).

Maybe a better example of the problem is for something like a data asset, where there's no source asset or export/import option. I have DA_PlayerStats and DA_ExperimentalPlayerStats, and I want to replace the former with the latter.

Ideally, there would be a safe way to to just replace the underlying .uasset file on disk. Then you could just check out DA_PlayerStats.uasset and copy DA_ExperimentalPlayerStats.uasset over the top.

(although there may be other reasons not to use that as a workflow, it might break referencers and so on)

Aha, actually... Not sure why I didn't see this before, but this seems to do the trick: Check out OldAsset in P4, delete OldAsset.uasset on disk, then open UE, rename NewAsset to OldAsset.

[silver token]

One trick I've found for this sort of swap without a delete: Add a core redirect for current type of asset -> new type of asset. Open up the asset(s) and resave them. Close the editor and remove the core redirect.

[still quarry]

Why cant we reset those all changes? Unreal Engine is already closed, nothing is happening when we discard...

[flat holly]

How does Unreal and the Perforce integration deal with renaming uassets, does it handle it gracefully and retain history or are there any caveats we have to be aware of?

I can't find any conclusive info, only some older threads about issues that were never clarified

[hoary wren]

Do you all know if it's possible to have Perforce pre-fill the changelist description with some template text?

[woven sluice]

"form-out trigger"
https://portal.perforce.com/s/article/2534
https://help.perforce.com/helix-core/server-apps/p4sag/current/Content/P4SAG/scripting.triggers.forms.html
never done it before

[hoary wren]

Thank you!

[woven sluice]

kind of complicated since UE also has redirectors... if you actually rename an asset and fix up redirectors and reconcile all that properly within a single change, it should usually detect it as a rename. IIRC it has some logic that if a binary file is only slightly changed it will still assume it's the same file even if the name is different

if you leave the redirector then i expect it probably breaks the history but i've never checked. it isn't that big of a deal normally since you can just go to the next commit back before the "new file add" and keep tracking a file's history if it moved anyway

(at least, isn't a big deal IMHO)

[arctic hornet]

Although perforce does support the "move" operation using "move/add" and "move/delete" combo which preserves the link... but as HoJo has said, unreal doesn't really like physically moving files from A to B without resaving them (which changes their content etc.)
and in general, even if you fix redirectors it usually becomes 2 operations "delete old asset" "add new asset"

[woven sluice]

well i'm saying it will commit a straight delete+add as a move if the files are identical. and i'm 90% certain it will also detect it even if the files are very close (which is what actually happens to .uassets). but what i think it definitely can't do is detect a move if you rename a file and leave the redirector in place. then you clearly get a modified original file and a new file

the problem is, in my limited experience you usually want to leave the redirector in place and have occasional massive "pause all work and cleanup all redirectors" days

which means... yeah, broken link. lol

¯_(ツ)_/¯

[flat holly]

the behaviour HoJo describes is described in this perforce article as a feature of p4 reconcile; the article doesn't sound very conclusive if it's behaviour that occurs on any submitted change list

I can't find any more recent info than this forum update for UE4.10 which at this point is ancient

it's rather disappointing that the redirectors make renaming such a pain

[woven sluice]

if you want to avoid it, then you'd have to only ever move files when you can check out all files that reference that file. and then fixup redirectors in the whole project every time you move a file.

let's face it: epic will never design a better system

[merry verge]

it does an integrate to retain history iirc. It's not the cleanest but history is retained - it just takes some know-how to view it.

The revision graph in p4v does a good job of showing history for integrations. The editor tries to also show integration history for assets but I've found it screws up showing that as often as it works ¯_(ツ)_/¯

[hoary wren]

So let's say I made a swarm review for a change I'm making to BP_MyCharacter and now it's shelved and ready for the reviewer to unshelf locally and diff. How do I prevent someone else from checking out that file while it's in this shelved state and making changes to it while it's actually pending a review?

[wide raven]

Shelving does not mean you have to revert : it remains checked out.

[hoary wren]

But then I don't believe anybody else can unshelve it locally in order to review the changes, no? Or is there some way to make it happen?

[silver token]

Nothing really. You’d have to keep it checked out until the moment they’re ready to unshelve and review the changelist.
I don’t think the editor has a was to diff a shelved file right now but it might. It’s not that different than diffing two versions in the file history which it can do.

[slate pulsar]

Question: Using github desktop, how do I keep the old version of the file instead of the new?

[pulsar parcel]

no discard option? at least do yourself a favour and use Fork

[wide raven]

Another can unshelve on p4v without you reverting. ( at least we can, maybe some settings involved? )

[warm pagoda]

Is it good practice to force revision (full overwrite) periodically? Working with small team on basic unreal project.

[hoary wren]

Oh I wonder if maybe you have your typemap set up in such a way that even binaries can be checked out by multiple people at once?

[wide raven]

Possibly. Unreal doesn't allow checkouts of locked files, but in P4V we can do what we like ( of course we always check with the current 'owner' of the file

[hoary wren]

Does Swarm leave a pending changelist hanging around in perpetuity for every Swarm review that ever gets created, even when archived?

[merry verge]

owned by swarm, yes. it stores all metadata about a review as part of a change it owns so it has to.

the only way to get rid of it would be to delete the review (and all history of it) entirely. Which I'm not sure you can do in swarm itself.

[arctic hornet]

you can obliterate the review, which iirc does exactly this

[peak nacelle]

on Plastic SCM, how to find a file across every branches?
cm find "file where name = 'filename.extension'"
this doesn't work

[smoky igloo]

I have my perforce server running in a VM and had a few crashes, hopefully, the RAM upgrade now fixes it, but is there a way, to check if the data on the server is up to date and everything is ok there?

[arctic hornet]

I've personally never seen perforce server crash due to OOM in the 15 years I've been using it.. since it doesn't really utilize that much memory...

either way there's bunch of commands you can run like p4 dbverify which will verify integrity of the database files and iirc thre's also something to validate hashes of the archives etc. but I don't really remember those from top of my head

[smoky igloo]

I'm not sure, it is running on a nuc and had about 4GB of RAM and the crashes didn't really happen while actively doing something, maybe it was also a error accessing the harddrive, but yea i hope the ram upgrade helped

[arctic hornet]

4 gigs of ram should be more then plenty, either way... you'll see after trying. Good luck

Also, just remembered the command for verifying archive files p4 verify 🙂

check the command help or documentation for more details about the two commands

[smoky igloo]

But running dbverify or verify, will not then maybe sync corrupted/missing files to my local machine right?

[arctic hornet]

nope, in perforce land command only does what you tell it to... so p4 sync syncs... p4 verify verifies, p4 dbverify verifies the database

TL;DR: no, it won't

[smoky igloo]

cool thanks! 🙂

[pseudo granite]

Does p4v have proper linux support?

[pseudo granite]

yea, i should have checked first.
hopefully its easy to install.

but will hold off since i dont want to worry about it until i understand ubuntu better

[modern tulip]

Would you mind sharing an example please? I’m checking internally, so far it seems in many months now we’ve only posted from our official account or with full disclosure - we are taking these comments very seriously (I was looking for our discussion about the ToS because someone at GDC asked what did we change, and found the quoted message)

[devout quail]

Y'all fw Diversion??

oh hey mr co-founder

[pulsar parcel]

maybe if there's ever an on-prem version

[unreal citrus]

How do you properly mange upgrading engine version and merging conflicts (when changes are made to the engine)?
I have this issue where merging 5.5 into my custom 5.4 made me fix like 200 conflicting files even though I literally changed only a single line in a single file.

[pulsar parcel]

is this p4? git?

at least with p4 use Epic's strategy, it's a proven method

[wide raven]

here's my current strategy

Fork unreal-<version_you_want> to a new repo  ( I call mine unreal-54, unreal-55 etc )

Clone the new fork in git or git-client ( I am using Fork )

Create and checkout branch <mygame-mods> ( the branch with all the modifications for <mygame> )

Add remote <my_unreal-fork-old_version>

merge branch <mygame-mods>(old) to <mygame-mods>(new)

resolve conflicts

stage, commit, push```
I like this way because it leaves me with the old version at hand while getting the game on the new one up to speed - have two separate projects, current dev and upgrade so I am able to keep working on the current dev while on the other hand solving the worst of the inevitable multitude of upgrade issues

this is epic docs
https://dev.epicgames.com/documentation/en-us/unreal-engine/updating-to-the-latest-changes-from-epic-in-unreal-engine?application_version=5.5

[unreal citrus]

I'm using git.

[wind gazelle]

the strategy i prefer is fetching new release branch and rebasing local branch with engine changes onto new release branch.
since my commits are pretty much self-contained (fix aaa, fix bbb, pick ccc) - can manually adjust things on each commit and cherry picks can be skipped

from original state: branch release-5.4, branch custom-5.4
make temporary branch or tag at current commit as "custom-5.4-marker" for later
fetch release-5.5
interactive rebase custom-5.4 onto release-5.5 while resolving little merge conflicts, skipping cherry picks, since commits are small merging usually simple
rename rebased branch custom-5.4 as custom-5.5
rename custom-5.4-marker as custom-5.4, sometimes useful for history or when need to rollback
push branches to origin

now you have rebased 5.5 and 5.4 versions of your custom engine
5.4 retains all picks and changes of 5.4
5.5 retails all changes, skips picked commits (history shorter)
can push fixes to both branches if simultaniously working on both

it is a bit more work to do rebase than merge, but dealign with conflicts is significantly less stressful

[prime crescent]

Hey, I ran into an issue while merging. It’s showing file conflicts, and it seems like Git is resolving them by deleting the conflicted files for example, it removed BP_ThirdPerson, which is now causing a lot of errors in the project.

Has anyone dealt with this before or knows how to fix it without losing important files? Would really appreciate any help or suggestions!

[wind gazelle]

blueprints are binary assets, and if you have conflicting changes the only way to try resolve is using tools in ue editor or just picking one of the files as correct and redoing changes to blueprint from another branch.
it is not "removing" them but most likely temporary renaming until merge conflict is resolved.
it is easier to see if you use some kind of ui frontend for git (like tortoisegit or flow etc)

[prime crescent]

So i need to redoing all the conflict files manually from the correct one?

[wide raven]

optimal situation with .uasset files like blueprints :

• don't get conflicts. When using perforce, files are locked so conflicts are rare

in the case you do get a conflict:

• look at the work done on version A vs version B. Which changes are easier to manually copy over? Accept the more tricky one to main, then copy over the changes from the easier one

[hot ivy]

Hi. I've been working on a project in UE and C++. I had been using TortoiseSVN to try and manage all the updates I'd been making. I made an error in a file and decided to roll back. The entire project subdirectory vanished. It was called GraphMiniGame. I really don't know what I'm doing with the settings in TortoiseSVN. I had just rolled back by right clicking on the log and reverting to it. I can't find the files anywhere.. I do have a backup independent of the VCS of the main source file NewBipartiteGraphActor.cpp and .h... but there were also a bunch of meshes and materials that I was using with it. I have no idea what to do to find these files. There are very 'dense' tutorials out there, but also I'm afraid of doing something that might destroy the files completely. This represents a fortnight's work..

the only thing left in the directory when I've tried to recover it has been the contents of the black screen shown.

[wind gazelle]

you can open git console, run git status or git reflog and see what state your repo at.

check git stash, usually autostash is on (there is a checkbox "show all branches" or context menu "view stash")

if you ever did commit your files - you can use reflog to find and switch to commit with your changes, also technically you can pull files out of git objects folder (but again, only if you ever committed them locally)

tortoisesvn delete files to TrashBin, you can check it too

[inland bay]

The first thing I would do is try to checkout the repo again to a different location, this is the easiest way to make sure you've still got the project files. If that works, in the new copy try reverting to the revision you desire and see what happens. It should work. You didn't technically do anything incorrectly.

[hot ivy]

thanks.. I'll have a go with your suggestions.

[hot ivy]

@dredgrave i checked out to a different directory.. the only file from the project created is ".dtbcache.v2" in the subfolder DesignTimeBuild.. This isn't the first time TortoiseSVN has deleted my files, and the whole point in me using it is to avoid this situation. I'm thinking of manually zipping project files in future because it's screwed me twice. I'll have another look in the recycle bin - I found an old version of some code before, but the meshes and textures are missing it seems.

[hot ivy]

could not find the files.. manually got a working version from bits and pieces not smited. Now, I'll make a zipped up directory as backup every session. defeats the purpose of a SVN but there we go.

[woven sluice]

a good reminder that source control is not a form of backup

[teal bone]

Source control provides better granularity and documentation than backups.

[hot ivy]

unless it deletes everything! I might see if there is a way of scheduling zip-ups of folders to automate it. programming can be a dangerous sport.

well i have a working version of the graph code.. the two materials and two meshes were just for nodes and edges, so it was trivial to restore them. unfortunately the nice gaussian post-process blur material vanished to the ol' dev/null but it could be worse.

[teal bone]

If your SC is deleting everything, that seems like either user or application error.

I doubt it's the SC itself.

Unless you meant the Tortoise SVN server, not the client?!

It may even be hardware failure.

[amber blaze]

I keep ending up with the error "The system cannot find the path specified" on commit/update/cleanup, which I think is caused whenever I make changes to asset locations. I move assets, update redirectors and submit from unreal. Then later the commit will not work because it can't find a file. Whether or not the file is actually there has no bearing on the error being produced. What is the safest way of moving around assets in unreal with svn? anything specific I should be aware of?

[wide raven]

That seems unusual. You could try just a rename first, without fixing redirector, check whether that works.

Sanity check : you may not do renames which only change a letter case ( upper/lower ), that will be a recipe for trouble

[oblique comet]

anyone have any tips for adding all the dependencies of an asset to git easily? in this specific case, i want to add a niagara system from a marketplace asset without adding the entire content pack, but the NS depends on a few textures spread throughout the content packs directory

[arctic hornet]

not specifically a git related question, but easiest way would be to spin up separate project, import whole asset pack in there and then use the Migrate feature, unreal will go through the assets and will select all necessary dependencies recursively and you can then set your main projectas a target

that way you won't drag any files that are not necessary for that niagara asset to work

[wind gazelle]

again, there are tons of things you could've tried.

if you had unsaved changes they could go to stash unless you did explicit "Revert" (but it would simply change newly added files to not versioned and keep them)
if you were switching to another branch usually autostash is on as well, unversioned files would remain as is
after "revert to" action files usually go to tash bin.
if you had local commits that were forgotten - you could use reflog to view them

unlike git console or others tortoise removes files to trashbin when user agreed
each "Commit" is a saved version which can be switched to

[gusty lion]

UE5.4 doesn't show any changes in "View changes" with SVN, yeah? I'll probably switch to p4 as a solo dev just for that window. I think it used to work with git.

[inland bay]

This is true. View Changes is always empty with the SVN plugin.

[gray sonnet]

Hi.
I can't figure it out what service/deploy i need to use if i want perforce with oracle.

[strong sigil]

https://dev.epicgames.com/community/learning/tutorials/1lV4/unreal-engine-perforce-setup-on-oracle-cloud-free-tier

[mighty beacon]

I got UGS set up and working with precompiled engine builds. the editor itself seem to be working (e.g. running UnrealEditor.exe and creating a new project works as expected).

However, actually trying to compile my own project fails with The batch file does not appear to be located in the Engine/Build/BatchFiles directory. This script must be run from within that directory.
It definitely is located exactly there.
Looking at Build.bat, it seems to check for the existence of an Engine/Source directory, so the error is a bit misleading. That Engine/Source directory does not exist, I am trying to avoid putting the engine source code in the workspace after all which is why I have PCBs.

Just creating the /Source/ directory allows me to proceed, but then it fails to find C:\PerforceTest\Engine\Intermediate\Build\BuildRules\UE5Rules.dll.

Do I need to include that in the PCB? Including Intermediate seems very wrong.
Why do I need any of that to compile the project in the first place? I do have a InstalledBuild.txt, so UGS should be aware that there already is a compiled Engine ready and waiting

[pulsar parcel]

UGS is designed exclusively to be used with a native builds, PCBs are merely a convenience

[mighty beacon]

So I do need the entire engine source in Perforce and on every user's computer? That seems very inconvenient

[pulsar parcel]

that's is generally how it works, yes, but you can make use of sync filters

there's even a built in one for source code iirc

[mighty beacon]

I suppose I could also just put the project binaries in perforce, so that no one else has to compile the project in the first place?

[pulsar parcel]

PCBs will be the engine and the project if you base the PCB build process on the sample build graph script

[mighty beacon]

Yeah I had it modified to specifically not do that lol.
But thanks, gonna revert that and live with having to compile the project a bit more often I suppose

[pulsar parcel]

Well this something your CI should be doing without much thought

Any time there's a change that UGS classes as "code"

[naive plover]

Quick question, If I have a project on source control with the standard gitignore for unreal. Does everyone who wants to open the project need visual studio and/or vsbuild tools or can UE still compile the source without needing to open and build it through an IDE?

I'm asking because for me I always open the project through Rider however I'm not sure if the non coders in my team should need an IDE installed in order to work on the project

[wide raven]

If it's a c++ project, I think yes... ( at least once - after that, can just double click .uproject )

not 100% sure of that

[naive plover]

Okay 👍🏼

[silent minnow]

Hello, how do people deal with baked shadows and VCS? Anytime we push the map with new bake, it increases the repository size by more than 1GB. In overall, only the bakes are going to be more than a half of the total repository size (including the history). Is it possible to somehow not to save the bake in the history, but still keep the latest version it in the repository? I'm using Perforce by the way

[cyan jay]

You can tell perforce to only store the last N revisions of a file by changing the filetype of the file and setting +S10 for example, which would store only the last 10 revisions. You can do this in p4v via Change filetype.

You must delete the exiting history manually using obliterate.

[silent minnow]

I got the filetype changed, but what about obliterate? Do I need to do that using cmd?

[arctic hornet]

obliterate is available either from p4admin or from cli

but be careful, obliterate will completely nuke the file(s) out of existence

including any and all history and metadata, like the file never existed in perforce in the first place

and in our case, before we switched to fully realtime solution (lumen) we just dealt with lightmaps by just letting them be, we've got plenty of space on our perforce server 🙂

[silent minnow]

I don't have that much space, so I would rather not have any history. It can be rebaked at any point either way, but having the latest one is good

[silent minnow]

Thank you, I found that in p4admin 😄

[woven loom]

hey so im using github with git LFS to store my project. my git LFS has run out of free space (1GB) and i went to purchase a data pack and got this error. DOes anyone know how i can resolve this so i can purcahse a data pack for git LFS?

Thank you

[dim elm]

Anyone have experience splitting the project into Code and Art/Content? I was mostly considering git right now.
Basically have all the C++ code in a repo, that defines lower-level classes etc. Which then the Art/Content repo uses, with blueprints, 3d assets, etc. to make the "full game". This way a Code commit could be pinned in Art repo and artists could download binaries pretty easily.

I have 2 problems with this approach,

1. the .uproject file, which seems to be needed in both cases? making it a bit akward, e.g. I could host it in the "Code" repo, but then artists cannot add plugins (for better or worse), there is NO way to compile modules without .uproject?
2. hot reload/live coding, no idea how that would be setup, unless programmers make a weird merged repo with both Code and Art

[silver token]

Everyone needs to work with both code and art. Artists might be able to work with code as precompiled binaries but the engineers need the art too. It wouldn't be a weird merged repo, it's the one they need to be using.

[dim elm]

Yeah I get that part, I'm not saying that I want programmers to work without art. But I want Art and Code histories to be separate. So that for example, programmers won't need to download latest heavy assets to pull changes of other programmers.

And it makes sense to me, considering that Code shouldn't ever depend on content/art. It seems like it's closer to the engine.

[silver token]

That's the thing, they might need to. If you're talking source assets, that's one thing. But project content changes should be in the same history as the code that uses that content.
Otherwise it'd be all to easy to sync to code or content that is incompatible with the other.

[silver token]

It sort of can. Like when using the BindWidget meta or when looking for a component you expect to be on an actor that the game won't run properly when it's missing.

[dim elm]

I don't know, there could be a commit "pin" somewhere in the content repo, that binds the code version. This could be compared e.g. at runtime to the existing binaries to make sure nobody is running old code. Maybe that's complicating a bit, but I'm curious whether it's possible. It would make even more sense if we were to use Perforce, since Art could be on Perforce while code stays on Git.

[dim elm]

Dunno what the BindWidget meta is, but looking for a component isn't really a direct dependency. More like "I require the thing that's going to use this class, to behave in a certain way", I don't really bind the code to any "concrete" blueprint or asset, that's what I meant.

Like, any assets should be referenced in code by parameters that can be changed in blueprints etc.

[silver token]

The engine already sort of does this with a version number, but it assumes that code and content are in the same stream of changes because it basically uses (at least when using p4) the changelist number. If code and content are coming from different repo's with unrelated numbers. At least not without going to a bunch of extra effort just because you wanna do a weird setup.
And if art's already on p4, put code there too! You should be able to do a single sync and be able to run or build your project. Not multiples from different places. It would also make automated builds a nightmare.

I'll just say that you're trying to do something very non-standard even when using git (from my understanding).
At best when people want to separate some code from the project you push it to plugins. But the project code and content always live together.
You're playing with fire (and there won't be a lot of people with experience doing it that way because basically no one does it that way)

[dim elm]

Hmm, okay, thanks for advice. I guess I'll go with the combined approach for now, and maybe come back to this idea somewhere in the future.
How about a custom engine build/source then? It's sort of a similar dilemma, does the Engine sit in right with the code and art? Or is it shipped and built separately ? For now I don't imagine the engine changes to be frequent.

[hot ivy]

I decided to Occam's razor the situation. Twice bitten, Thrice shy. yes. I checked the recycle bin. all that was left when I reverted to earlier versions of the project was a database file. I am not sure if I accidentally commited the parent directory rather than the individual files. I recall my boss at the job that used it running into difficultly too. Whether it's my own lack of understanding of how TortoiseSVN works or some other reason, it's the second time now so I doing it the old fashioned way; zipping up the project directory and putting the date on it. Given I'm a sole developer it seems to be the safest way. I'll still use the SVN, but this way it'll be a safety net.

[wide raven]

I seem to recall having trouble with svn on various projects over the years. Git ( not great for unreal ) and Perforce ( the match made in heaven ) are my go-tos these days, though I did have a big project in SVN still not that long ago... Maybe ten years : E

[hot ivy]

Perforce costs shekels though, eh? 😆

[wide raven]

you can host a small own server for free

separate server box + buildmachine box would be optimal if you get serious

theoretically svn should be fine, but i know a starting group had a lot of pains with it recently

truly streamlined easy source control, claims have been made for Plastic and Diversion would be simpler beasts but I have only briefly tried plastic with unity, so can't vouch for either

It's a wonder nobody has made/tried to make(?) a FoolProofGit client : )

[silver token]

It's roughly the same idea. In the case of perforce, the server or depot is the thing that should be the same. They don't have to be in the same stream. I'm not sure of the best practices for git that provide the same benefits, all my dev is in p4.
If you haven't looked at the top pin, you should though it also it p4-oriented because that's what Epic and most studios use.
If you're on engine source, it's built and distributed at the same time as any game binaries that you build. It all needs to be done at the same time. You can't just run any version of your binaries with any version of the engine binaries so they get bundled together when they are built and then stay together.

[woven loom]

?

[inland bay]

Seems to be a live issue atm, maybe follow this thread. https://github.com/orgs/community/discussions/156401 Also check out git+lfs on azure? People often talk about getting more storage for free using this. I don't use git though.

[woven loom]

oh wow, that was posted after i looked it up so i didnt see it😰

thank you so much

[ocean anvil]

u generally shouldn't use github for UE projects. better to use git on azure devops as indeed suggested above.

[woven loom]

ah okay, i did not know that, this is my first unreal project. I will try to migrate to azure devops, thank you guys!

[dim elm]

I'm trying to decide between git with LFS, Plastic SCM, and Perforce, for binary assets only (code already uses git),

AFAIK git LFS has the giant flaw of not doing any kind of compression on the assets, which is really bad for uncompressed binaries, which is the case in UE. + I recall it's quite buggy.
Perforce I've set up and tried to use but got some random issues related to the server hanging up on revert, as well as I've heard the costs are about 40$/user/month above 5 users, which is quite pricey.
Plastic SCM I have not used, but I'm seeing really mixed opinions on the internet, seems like it's more affordable as well.

Can anyone confirm my fears concerning LFS, that the disk usage is going to basically explode, as well as give me advice concerning Perforce vs Plastic SCM?
I know Perforce is The Epic Way ™️ , but I don't mind tinkering a bit with custom tools or plugins.

[long hull]

I can only speak about Git LFS, but are you worried about local or remote disk size? Locally, you can prune everything to only keep the working directory, without any of the history (that's what I do). On the server side however, AFAIK you're correct: there's no compression and everything in history is stored as-is, so it can add up.

[dim elm]

yeah remote side mostly

the solution I'm planning right now is: have the "game project" on Perforce, have the "Source" directory on git, which is optional for non-programmers, and have the UE engine code on git as well
and then a config which pins the versions of the code and engine, to e.g. a git branch or specific commit, and a script which downloads a build based on the pinned commit from e.g. Gitea, with CI/CD building it automatically

and fortunately a relative dir can be used for EngineAssociation, so UnrealEngine is generally going to be a subdirectory of the project. Only downside is that it's not a "native" project, but I've read that this is mostly an issue for Epic's tools, which I don't plan to use right now

[woven sluice]

Sounds like hell. Use one or the other.

[arctic hornet]

I'll echo the sentiment of others, use one or the other, you'll have TON of weird issues while mixing VCS + there will literally be 0 tooling available around that setup.

I'd personally recommend perforce.

The "freezes" usually happen if the server has large journal (the larger the journal the slower perforce gets, P4 installations since 2019+ or something like that have nightly journal rotation enabled at 4 or 5 am UTC every day). Or, they can happen if you're running for example on spinny HDDs rather than SSDs, in the past when we were running our server on HDDs we've had issues with things freezing etc, but it was simply due to us maxing the throughput and iops of the disks... switching to SSD based storage allows us to easily saturate ~8 gbps both ways and everything's snappy.

Obviously the thing with perforce is the cost, there's not much one can do about that sadly, especially as the licensing costs have to be paid per-year since that's the duration of the license.

I can't speak about git lfs for unreal projects as I've not used git for that purpose in a loooong time, but all I can say that you'll most likely have to self-host (at least your LFS) because all the cloud hosts like github, devops etc. provide relatively small LFS space and then charge you arm and a leg for more.

Pros of perforce: best compatibility with unreal ecosystem, most of epic's tooling is built for or around perforce
Cons of perforce: you have to set it up yourself (unless you go with providers like Assembla which is managed perforce hosting - among others) you have to self-host and obviously license costs

Pros of git: free (unless you count LFS), you can use cloud providers or self-host
Cons of git: from my (limited) experience, there's a big issue with working in teams since unreal is built around trunk based development which is not something git is best at so prepare for merge conflicts, LFS from cloud providers can get expensive real fast

[white badge]

hello, im trying to install perforce on an oracle VM running Ubuntu 20.04, and im following this guide: https://dev.epicgames.com/community/learning/tutorials/1lV4/unreal-engine-perforce-setup-on-oracle-cloud-free-tier
I'm unable to install helix-p4d, I get a message saying N: Ignoring file '' in directory '/etc/apt/sources.list.d/' as it has no filename extension
E: Unable to locate package helix-p4d
Should I just try using a newer version of Ubuntu?

[arctic hornet]

seems to me like you borked something while making the "perforce.list" in the sources.list.d as the SQUARE isn't valid filename and that's what APT is complaining

if you navigate to the /etc/apt/sources.list.d/ folder and use ls you should pretty much have one file, which is perforce.list (unless you added other sources, for example docker, stuff from microsoft or whatever beforehand)

[white badge]

i think I need to use a newer version of Ubuntu or switch from aarch64 to x86_64 and try again

[arctic hornet]

it shouldn't matter, 20.04 is supported and so is arm64, but I'd probably go to 22.04 or 24.04...

[white badge]

its working fine now, I terminated the old VM and switched to Ubuntu 24.04 and from Ampere to AMD

[earnest beacon]

i'm using git and github desktop for source control and i connected this I think properly with the tutorial but for the past days i'm getting errors with locations that don't exists anymore and i feels like it slows down loading of my project horribly

does anyone know how to fix that?

[earnest beacon]

looks like deleting intermedtiates and binaries helped

[olive bramble]

Can anybody help me with Perforce Revision Control?
I am unable to contact a team member on the project and they had some files checked out on revision control so it's locked for everyone else. I'm very new to P4V and admin controls.

Is there a way to unlock the file or force a revert to get it unlocked?

I tried right clicking the file in workspace -> Open Command Window Here -> and running the command:

P4 -c workspacename -H computername -u username revert filename

But I'm getting this error:

Path 'C:\my_workspace_path\file.uasset' is not under client's root 'C:\their_workspace_path'.

[pulsar parcel]

it's p4 revert with -C, the capital letter is important