source-control

[long hull]

I think I remember why I ditched branches on sub: both merge and rebase had nasty side effects. Example: I'm doing some stuff on a sub, moving across branches to test stuff. But if the tracking is set to master and I left the sub on test28, next time I run the submodule update, it'll (silently) either merge or rebase those two branches, which is absolutely NOT what I wanted.

That's a disaster when you have forks (which I do, but I'm guessing that's not common)

[faint cairn]

sigh

What a headache this is tbh. Like, Git is such a leading thing, and at the same time it s*cks so hard

At least for UE I guess

[long hull]

I think subtrees have the inverse problem: pulling is very easy, and keeping things in sync is trivial. But pushing is hard. So if you're more interested in having people pull, and less push, they might be interesting.

[faint cairn]

I will have a look, thanks!

[arctic hornet]

Epic did make their own VCS, it's called Skein... or more precisely Unreal Revisoin Control (URC) but it's still currently being developed and it's used only by UEFN right now, but it will come into mainline unreal one day. Although I should also add that Epic still uses perforce internally.

[long hull]

Yeah I mentioned that in my message. I had hopes for Skein, but when I realized it was for UEFN, I basically gave up hope.

I don't understand how Epic can rely so heavily on a single company for VCS. They don't even own a stake in Perforce, when they bought full or partial stakes in many other major tools (e.g., Houdini, RAD).

[arctic hornet]

I mean, perforce is what most major studios use because it's been established for a long time (Epic themself are using perforce since the year 2000) and perforce doesn't call home or send any telemetry, it works fully offline etc... as long as you have the licenses you're golden

and it's decent in all the avenues you need... it's not the best in any single one, but it does everything great overall

[tranquil tiger]

has anyone tried to host a Zen Server over the internet rather than over LAN? Is that intended use? I couldnt really tell from the documentation :/

[arctic hornet]

Zen is indended to be used either as local cache (on your PC) or "studio"/"office" level cache (i.e. LAN) ... for WAN usage there is Unreal Cloud DDC (don't let the "Cloud" fool you, it can be fully self-hosted)

so in ideal scenario you'd have following setup:

• Local Zen
• Studio Zen
• Cloud DDC

or in most cases for distributed teams it would be:

• Local Zen
• Cloud DDC

[formal perch]

Hey all
Does anyone have any good ideas for beginner videos on push/pull process using P4V and Unreal?
I'm hosting at home have given my work partner VPN access.

Here are our troubles:
We're both new to using Perforce Helix style checkout, push, pull. It seems like some of our files are "exclusively checked out" and maybe not getting pushed?
When we add a new file or asset in Unreal how do we make sure it gets added to our depot to push? From Unreal or P4V?
Is there anywhere in P4V that we can filter to see new, or exclusively checked out items quickly to troubleshoot?

Thanks for any tips or ideas on how to resolve our learning gaps!

[arctic hornet]

so... the "exclusive checkout" is for binary assets (like uasset files in unreal) which cannot be merged, this prevents two people working on the same files at the same time to prevent overwrites or collisions... it gets locked when you checkout the files and gets unlocked once you either checkin (submit) the files or revert them

and you can see them in perforce in the Pending tab among your checked out files, usually in the default changelist

if you expand the filters you can remove the filter for "Current user" and that way you'll see all users (except their default changelist which is local only)

you can use commandline commands such as p4 opened -a to list all the checked out files by all users

when you add new file (like an asset) if you have your under connected to version control (bottom right corner) it will automatically get added to your changelist... files can be marked for "add" (which means newly added files), "edit" (file that's been sent to server at least once and you're modifying it) or "delete" (you locally removed the file and once you submit it, it will get removed on the server too)

I recommend checking the stuff here https://www.perforce.com/blog/vcs/how-use-unreal-engine-5-perforce and also the getting started series of videos on perforce's channel (i'll get you the link in a sec when I find it)

https://www.youtube.com/watch?v=jIQEjDiSe0g&list=PLH3pq2J85xsPYn71_yzzsZQKvalTW-duE
https://www.youtube.com/watch?v=LITmtpuLrQQ&list=PLH3pq2J85xsMDY2AAkzUhTmHIg1S7WN1f

one series is for how to use perforce as a user and other one is for managing the server as an administrator

hope it helps

[formal perch]

THANK YOU!!! Big help getting pointed in right direction.

[kindred compass]

Hi! Attempting to set up source control for my UE5 project. I found digital ocean's pricing to be reasonable, and found they had a one-click droplet for easy setup of perforce https://marketplace.digitalocean.com/apps/perforce-helix-core. Sounded pretty sweet and easy, so went with that. I've gone through the steps of creating the droplet, but am now stuck on actually connecting to it.

SSHing from my local into the droplet "works", but I get a message of Helix Core is being configured. Try again in a few momemnts... and then the connection gets terminated. Reasonable at first, but its been 30 minutes now and I'm beginning to wonder if something has gone wrong in droplet creation or if I should reboot the droplet. Is configuration lasting this long normal? Thanks

[kindred compass]

Update: Still in the same state. Just tried turning the droplet off and back on, and get the same message still...

Spoke too soon. Turning it off and back on again did fix it. Finally have access

[gentle pond]

hey all, i think i majorly messed up by saving and submitting assets in a newer engine version and now i get this complaining message and things won't build of course:
Package <asset path> contains a newer version than the current process supports. PackageVersion 1,012, MaxExpected 1,008 : LicenseePackageVersion 0, MaxExpected 0.
i'm using perforce. i undid the changes and saved them to a changelist to see if it works before i submit it. the project is still complaining about the same assets tho for some reason. i'm pretty sure the undo should've worked locally and very confused about why it doesn't. how fucked am i? any idea on how to fix this?

[woven sluice]

in theory if you do it correctly, just undoing changes should completely pull you back to the previous history state... did you submit multiple commits since you changed engine versions, and only ran undo on one commit, maybe?

and it is actually your intent to go back to a previous engine version for sure?

[gentle pond]

thankfully the fuckup is in the 2 most recent submitted commits. i undid both of them together.
and yes, i do need to go back to previous version for sure.

[woven sluice]

if i were you i would just do a test to gain confidence: make a new workspace, sync it to the last good commit, and see if that workspace all works as expected

assuming that works (certainly hope it should) then maybe you can show exactly how you did the undo process or try it again

[gentle pond]

fingers crossed. it's building rn 🤞

[gentle pond]

welp, turns out i pulled a user idiocity error the problem assets were submitted in the 3 most recent commits, not 2. helps to have more complete commit messages for sure 😅
thank you so much for your suggestion! it helped me find what was actually happening

[jaunty hornet]

I'm wondering, I'm approaching a testing phase with my project. Is there a way to get the current git revision of the project's repository?

As in, in an Unreal project as a string or name

[sinful jungle]

You mean something like git rev-parse HEAD?
This will give you the commit's SHA1

[wind gazelle]

there are many

• either by querying git in a process from build.cs that will create define
• running prebuild script (uproject or build.cs) that will generate header
• reading .git folder and creating define in builld.cs

[jaunty hornet]

Yes, but from inside Unreal itself. So when I make a package I can implement the SHA1 somewhere

[jaunty hornet]

Ok great thank you very much!

[tough vector]

hey people so I wanted to switch my project to like azure devops and I tried to like upload my project on to it and this happens, allat for 1 hour of not doing aynthing productive 😭 🙏

[lilac flame]

What's the command to .gitignore a folder that is already on the repo?

[ornate matrix]

Just type it manually in the file 😆

[arctic hornet]

it depends, if you just want to stop pushing further changes to git, just add it to .gitignore file as Fishy said... but if you want to remove it... well, you first have to remove it, push the removed files to git and then add .gitignore

because .gitignore literally just ignores the folder and doesn't scan for any changes

[lilac flame]

Yeah but there is a command to do it all at once. Without the need to delete the file first then gitignore afaik.

Didn't end up applying cuz turns out I already ignore the folder

[arctic hornet]

Heh, today I've learned something, I've been mostly perforce user for 10+ years so I'm not that versed in git specifics 🙂

[ornate matrix]

how would you share a source engine with your designers ?
do you send the bins of the engine ?

[arctic hornet]

since we use perforce we have it relatively easy because of the tools that Epic has made over the years

we use Unreal Game Sync for day-to-day work which is a wrapper on top of perforce that simplifies the work... combine that with Horde, which is a build server... UGS can automatically download precompiled-binaries for given changelist but still keeps them separate from main source code

so programmers sync whole source code and build it locally, artists sync just the content (uassets etc...) and skip the source code... and then automatically download precompiled binaries that are then extracted

sadly both Horde and UGS are perforce only.

but the general aproach is:

• server (or someone) builds binaries for given change

• zips them up

• stores them somewhere

• when someone else syncs to new change previous binaries are cleared

• new version gets synced

• binaries get downloaded and extracted

• list of extracted files gets stored inside somewhere so it can be later deleted

you could easily replicate that with whatever CI/CD solution you're using and do the downloading in pre-sync and post-sync hooks (I'm pretty sure these are not correct terms for git, but I'm also pretty sure there are hooks like this in git)

[ornate matrix]

thanks for the details

[spare oriole]

for some reason im stuck here for so long

im setting up perforce server on a ubuntu machine

[spare oriole]

Perforce Server starting...
Perforce server warning:
Pid 75820
Operation: topologyRegistration
No entries made in db.topology for server address: '1666', dest address: '' and serverID: ''.
ServerID for the server should be set and a server restart is required.

im having this issue while setting up perforce

[arctic hornet]

that should not cause it to fail starting up, it just can't use the clustering feature

when the server is running you can do p4 serverid NAME_HERE and it will assign it an id

obviously you need to have your local p4 configured properly to communicate with target server etc.

[hoary wren]

Is one workspace mapping multiple streams in perforce generally undesirable? Do folks do this in practice?

[silver token]

No, I don't think perforce would let you actually do this.
You can have one stream import other streams (like an engine or plugins stream if you want to keep those separate).
But your workspace would still just be the stream importing the others.

[arctic hornet]

as Raimus said, workspace can only be mapped to one stream at a time... but you could use stream-level mappings to combine or filter streams together, for example I'm using filtering to filter our AutoSDK setup like this:

we have global AutoSDK stream that contains all versions of Windows, Xbox and PS5 SDKs etc... and then for build-machines I just filter it to currently used versions to save on space (and it also results in fastrer syncs because it's less data)

And this allows you to combine streams together, or load non-stream depots into a stream

here's the documentation: https://www.perforce.com/manuals/p4guide/Content/P4Guide/streams.paths.html

[steel berry]

Hi there, I am just begining with Unreal Development and have been following this guide to setup Azure DevOps, https://docs.google.com/document/d/1-NOspTVKX2eRgzmPKzoBmmnEqdPn-5G_IXt32IldB2c/edit?usp=drivesdk

Im at the section that goes into adding collaborators to your repo using Azure DevOps.

It says that its a Work In Progress WIP

Just wondering if this section will be finished off? Not sure what the user is here but they say to ping them on Unreal Slackers to see if they can finish it.

Their Reddit username is UltimateGamingTechie

[wide raven]

https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/add-organization-users?view=azure-devops&tabs=browser

[untold bobcat]

Does anyone know any tutorials for pushing and pulling projects using github? I am looking to access my unreal engine project on my laptop when I am out somewhere. It is a huge project that is about 200 GB so I don't know if that is possible. Could anyone help me?

[teal bone]

You can't stick 200gb on github without paying a lot.

I would just copy it to google drive and download it from there.

Or just copy it over a local network.

The code you can put on github (c++) no problem.

[untold bobcat]

Okay, but how would that work without downloading the whole project every time? I just want to be able to download the updated project file

[teal bone]

You can get an azure devops (I think?) thing with basically unlimited space.

Git is still going to go apeshit if you try to upload 200gb at once.

[wide raven]

If you have folders and folders of assets of which you only use a fraction, I would strongly recommend you clean up your resources folders to only have stuff you really use

[frank hedge]

Hi everyone, I just learn about Source Control and I'm wondering if anyone can give me a few brief about what is it exactly and how to use it (I can find tutorial of course but i have to understand the real purpose, if I just follow tutorial without understanding what is it about its probably useless)

[arctic hornet]

well, I'll probably direct you simply to wikipedia for the basic most and most unbiased overview of what version control (as the concept) is https://en.wikipedia.org/wiki/Version_control afterwards if you have any more specific questions (let's say regarding source control in relation to unreal... or questions about specific source control software etc... etc...) let us know

[frank hedge]

https://docs.google.com/document/d/1-NOspTVKX2eRgzmPKzoBmmnEqdPn-5G_IXt32IldB2c/edit?tab=t.0 Im reading this right now ( I found this link after posting my question )
I guess its a good information

[wide raven]

tldr : you have a running backup, minimum daily commits preferably, so you don't lose work if something goes horribly wrong, trace problems, work remotely, collaborate, enable autobuilders...

[steel berry]

Im having an issue publishing to Azure Dev Ops. If I post the error could someone have a look at it for me.
Unsure what to do to remedy it.
This is my first time using Git Hub Desktop and Azure Dev Ops.

[wide raven]

Best to just post the error immediately
as code snippet
or .txt file log

[steel berry]

Locking support detected on remote "origin". Consider enabling it with:
$ git config lfs.https://[e-mail]/Ragnar-1/J Train/_git/J Train.git/info/lfs.locksverify true
Fatal error: Server error &{%!!(string=https) %!!(string=) %!!(*url.Userinfo=&{Ragnar-1 false}) %!!(string=dev.azure.com) %!!(string=/Ragnar-1/J Train/_git/J Train/info/lfs/objects/3c210e7532acad12694909091a88bad71f5fda2b05e13fa6bf147d93bb5713c1) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=) %!!(string=) %!!(string=)}s(MISSING) from HTTP 503
LFS: Client error &{%!!(string=https) %!!(string=) %!!(*url.Userinfo=&{Ragnar-1 false}) %!!(string=dev.azure.com) %!!(string=/Ragnar-1/J Train/_git/J Train/info/lfs/objects/346f6f46895ebbbf15d3aa82322bb77854f9d0f8c6dbc5f608f8d27a154727ea) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=) %!!(string=) %!!(string=)}s(MISSING) from HTTP 413
error: failed to push some refs to 'https://dev.azure.com/Ragnar-1/J Train/_git/J Train'

Sorry dont know how to do snippet

[wide raven]

Is your path Ragnar-1/J Train/etc?

Hmm, haven't messed with git for years

[steel berry]

I think thats my Primary Remote repository

[wide raven]

something wrong with the lfs setup it seems

[frank hedge]

revision control = source control = version control ?

just the name change ?

[wide raven]

Yeah it's all the same

[frank hedge]

ok thx

[wide raven]

I would consider trying to make a small test workspace first, just some .txt files, no LFS

Once you have that working, can do the next one with LFS ( or add LFS to that )

[frank hedge]

stupid question but how do i find the path?

[steel berry]

Ok I will try a test, thanks.

[wide raven]

Whereever you have installed git : this will depend on your choice of git client ( if any ) and platform

[frank hedge]

C:/Program Files/Git/bin/git.exe looks this

what do im missing

[wide raven]

Are you followin the linked google doc?

[frank hedge]

yes

im at this ste

step

oh i misread

i think

gimme a minute

[wide raven]

I gotta head off, good luck : )

[frank hedge]

thank you haha

i guess the first time is long as its have to check ALL the files I have in my project ??

im not sure to understand everything from tutorial 😦

I cant mark for add, what do i miss?

whats wrong :((((

I can create a new blueprint, its then has the ?, I save and then has the + , i submit but it doesnt showup here
I feel like its not going to the good one

i tried to create a new repo, but i think UE5 is looking for the path to another repo, that is deleted

[kindred compass]

CommandMessage Command: MarkForAdd, Error: (filepath).uasset - ignored file can't be added.

Why is it an error to try to add an ignored file? Given Unreal auto adds the ignored files in the 'Submit Content' dialouge, this seems like kinda bad UX. Am I missing something? I literally have to go through and manually unselect every ignored file, which kind of defeats the point of the .p4ignore...

Note, p4ignore is setup properly. if I use the P4V client, it works fine

[thick marten]

@frank hedgeif you created a repo in your project folder, and then removed the old one. how did you remove it? it sounds like it might still be there if you didnt delete the .git folder. you might need to remove them from your git program you are using. delete the .git folder from the project. then go back into the git program and initialize a new repo in that folder.

[frank hedge]

i now have this

[thick marten]

yep so thats all the changed files, you would add them then commit and push

[frank hedge]

i come back when its pushed

thanks

The surprising things is its not my whole project

it should be right

?

[thick marten]

well whats your whole project?

[frank hedge]

the ..full project ?

im not sure to understand your question

[thick marten]

well you have an ignore file, so its going to ignore some things that are not needed so your full project wont get pushed. just what is needed. I dont know if there are any other commits ahead of this that have the rest of the project

your also packaging your project into your project folder, which is now getting upload, which is... not normal

[frank hedge]

why?

[thick marten]

do you need to keep a copy of the packaged project in source control? the one you package to test/give out to other people?

[frank hedge]

I would like to have my full project in Github in case i lost everything

and then everytime i edit something, i just push for the specific thing

is that not how it works?

[thick marten]

project yes, packaged project no. everything you need to package the project is what you add to source control

[frank hedge]

But there is no packaged project?

[thick marten]

you .exe files, your .pak files, all of the things that are not needed in source control are being pushed to source control

your screenshot says there is, that is what you are commiting in that screenshot. look at the files

[frank hedge]

oh right
thats because I push the whole project, including the packaged .exe that i did yesterday

[thick marten]

yes and I am saying you should not be pushing your packaged project

[frank hedge]

should i cancel?

or i just wait and then i remove it and not push it the next time

[thick marten]

you can do whatever you want, but ive never seen anyone push a packaged build with their source. normally you build your project into another folder

its basically just wasted space and time

[frank hedge]

it was my first packaging i didnt know

[thick marten]

unless I guess you want people to be able to download the packaged project for some reason from source control, but its going to hit your storage limit quick doing that

[frank hedge]

but yes i understand what you say now

Im not wanting to upload the .exe i didnt know it gonna do it

[thick marten]

well its in your project folder, and you didnt tell it to ignore it, so its going to do that and it will tell you that its going to

[frank hedge]

In any case
At the end of that i can even delete this .exe i dont need it anymore

But now, as i have different depo, how do i ask UE5 to upload to the good one,

[thick marten]

ue reads the repo that is in the folder for the project, so if this project is now using this repo it should be using it. disable source control and re enable it and it should pick up the settings

[frank hedge]

If i didnt packaged anything
I should NOT have any .exe in my project folder, right?

[thick marten]

correct

[frank hedge]

im trying to clean my project now first and then upload it again instead

the binaries can be removed ?

or it contain some project file?

and how can i check that by myself without asking?

[thick marten]

config, content, saved, .uproject are all that is needed. your .png should probably be saved as well

the docs tells you what is needed and what is not, and your .gitignore file tells you as well

the git ignore is set up to just ignore the folders you dont need for the most part, but it looks like its missing BUILD

[frank hedge]

I will put back a backup file (i create one more backup with the current folder) that was here before the packaging, so i will have a clean project ( i guess )

[thick marten]

the build folder is needed if you build to other platforms like mobile, but for just windows it can be ignored

[frank hedge]

can i delete all of them

[thick marten]

if you dont want them anymore, sure

[frank hedge]

how do i even do that lmao

[thick marten]

if your just setting this up still and you still have your original files, you arent going to harm anything removing old repos

I have no idea, i dont know what you are using. probably in the settings for them

[frank hedge]

lol ok yes

to not make mistake
how do i safely rename a project?

can i rename any of them and then UE5 will show the renamed ?

[thick marten]

rename what? the name of the project you see in the project, or just the folder

[frank hedge]

the name of the project i see when i open UE5

to be sure im on the correct one ( the clean backup )

[thick marten]

well the .uproject file there is what you see when you open the editor. the project name inside of the project in project settings is the name of the game itself when its running

[frank hedge]

so if i rename this .uproject MATHEWW
when I open UE5 and i have to chose wich project to use, it gonna displayh " matheww" and thats the only thing that changed?

or it gonna break other stuff

[thick marten]

when you open the launcher? it will update eventually yes when the launcher library refreshes

[frank hedge]

ok i gonna try

Ok i have now a clear project

a clear githubapp

and a clear github

in here i chose then go to my project folder right? @thick marten

[thick marten]

I dont use that client, but you should be looking for initializing a repo in a folder which this might be? I dont know if this is looking for one that exists or creating a new one

it looks like if you try and add to one thats empty, it will then ask you if you want to create one

[frank hedge]

edit : i have to pick my folder project to be able to see that

wich is the correct way i guess

[thick marten]

yeah I think doing what you are doing there creates a new folder in your folder

[frank hedge]

probably this yes

[thick marten]

no, it says Prototype/Prototype

your folder is Prototype, and your repo is named Prototype so its makes a new folder for it

but if it created the .git in the correct folder your fine

[frank hedge]

looks good

[thick marten]

so if im reading it right, your local path would be up to Unreal Projects and your name would be Prototype and it would create the .git repo in your Unreal Project/Prototype folder which I assume is where your project is already

yeah that looks like it found it all. you can open up the project and enable source control and see what it shows

[frank hedge]

why it makes me swet lol

i will try to create a Blueprint empty

[thick marten]

seems right

[frank hedge]

good

so now i will publish

[thick marten]

I personally dont use the built in revision control inside of the editor, it tends to slow it down especially if your solo. Theres nothing wrong with just loading up your software after your done and committing/pushing. but thats just me, I dont like the interface in UE

[frank hedge]

probably gonna take a while as its the full project uploaded for first time(again) and then, it gonna upload only the changes

if i understand well

[thick marten]

correct

[frank hedge]

and that gonna avoid me 3h of uploading everyday hahaha

[thick marten]

you should see a number of commits now equal to the # of commits you committed lol

then in the future you add files to a new commit and publish just those changes

[frank hedge]

(I knew it was not correct but as it was a little file before, it was ok to do so on google drive, then it become very too long and so yesterday I ask, people suggested i should use github and here i am)

[frank hedge]

Not sure to understand

you mean you prefer commit from the gitapp instead of from ue5?

[thick marten]

yes

its just easier to read for me than the tiny dialogue box inside of UE

[frank hedge]

and when this is done

that should show up here?

[thick marten]

yes

[frank hedge]

but i dont have a data limit on githu?

[thick marten]

I have no idea, I assume there is one

[frank hedge]

i dont succed to upload on GitHub it fail on the way but maybe my project too big
Can I upload in multiple part?

[urban breach]

i have this error on unreal it says no assets to check when i go to revision control and go submit content

[frank hedge]

try create a blueprint empty and see if then you have it

[thick marten]

I would just try again, remote disconnect could have been a timeout due to size.

[frank hedge]

i tried 4 times already, im sure its due to size
If i upload manualy from github will it work?

i mean, yes it works to upload, but will the system understand they are there already?

[thick marten]

what system? and ive never uploaded to github so I dont know

i would check the logs and see what it says is the issue

[frank hedge]

where can i find the log?

[thick marten]

it should be somewhere when you are uploading

[frank hedge]

it just says the disconnected

[frank hedge]

just this

[thick marten]

there should be one somewhere showing you things like uploads and other loggy stuff

[frank hedge]

i dont find any

[thick marten]

or maybe it doesnt, thats annoying

To access the log files go to the file menu in GitHub Desktop and select Help > Show Logs.

[frank hedge]

my problem with github is i cant upload my full project at once
so i have to do folder by folder, wich is.. ok
But im afraid that then It doesnt work as it should when i edit a file because im afraid the girhub app doesnt understand its the same folder in github

[thick marten]

could be related to size yes, cant say for sure

https://github.com/desktop/desktop/issues/19023 might have some suggestions to make it upload

Ive never had to upload anything to git manually, ive always used an app and had no issues so I couldnt say. but i also dont upload large files normally

[frank hedge]

git config --global http.postBuffer 157286400 apparently i have to use this command

I understand nothing

now i have this

Why is it so complicated
I just want upload my folder in a safe place
Im trying since 11am, its 4pm now

[thick marten]

it probably wouldnt be complicated if it was done from the start

[frank hedge]

sure, but at the start i had no idea about that

Now i dont know what to do

[thick marten]

and I cant say I know what that error is unless you uploaded something by hand

but it saying your remote repo doesnt match the local one and that you need to update the local one by getting the changes that the remote has

[thick marten]

the git command you posted earlier goes into command prompt, yes

[frank hedge]

now i have this

i pressed fetch

[thick marten]

so all you did was that, or did you do anything else?

[frank hedge]

but same

only this

before that, i was able to push, even if it fail later

[thick marten]

did you have github desktop closed when you did that?

[frank hedge]

no because i need to e on the app to do that

[thick marten]

you didnt

[frank hedge]

the command console is in the app

[thick marten]

its a command prompt, its part of windows

[frank hedge]

but i open it from the app

[thick marten]

you did, but that doesnt mean you had to

[frank hedge]

i mean, the app was not close anyway

[thick marten]

anyways restart the app and see what happens

[frank hedge]

ok wait

same

i fetch, it load

and same again

i do enter it properly right?

[thick marten]

if it didnt error, yes

thats a change to git in general

it shouldnt have done this, its saying your source on github has a new commit

[frank hedge]

i have absolutely no idea whats wrong and how to fix it

[thick marten]

what does it show for commits on github website

[frank hedge]

i deleted the manual uploading ( as it was going to fail )

so it should be clear now

[thick marten]

you have 14 commits

[frank hedge]

i uploaded 14 files, then i understand its not going to work like that
I deleted them
So my github is empty

yes?

[thick marten]

well its not empty, you have 14 commits or changes

[frank hedge]

its commited as delete

[thick marten]

yes so that doesnt match whats local

[frank hedge]

how

local = my computer

[thick marten]

your local git repo

[frank hedge]

thats the local

[thick marten]

your remote one on the internet doesnt match now, your commits are different

[frank hedge]

how do i fix that

[thick marten]

i dont know if you can since you uploaded files then removed them, I dont know what that will do if you try and sync them

[frank hedge]

• it probably make sense but not for me ( as i dont understand probably ) but i just delete things from the Github, to have it empty, so i can try to push everything again

i created a new branch

and it works

lets see if it stop again

[thick marten]

source control isnt just a bunch of files is the issue, its a list of things that change from one commit to the next so you cant just have things out of sync and do things at different times and places and then hope it magically works back together

a new branch might fix it, you just have to remember to always use that branch or force that branch into the main/master

[frank hedge]

im now waiting to see if it going to send me the error

again

oops

wrong picture

[dont take care message above)

im waiting to see if i have this message again or not

[thick marten]

check the log

[frank hedge]

im at the writing part

i have more issue trying to save my game than making my game

😦

(well, for the moment it still uploading)

[frank hedge]

AAAAAAAAAAAAAAAAAA

https://tenor.com/view/give-up-tantrum-office-time-gif-14456507

i spend less time uploading the full project on google drive for 4hours lol

than trying to upload on github

i have no idea what should i do now

Writing objects: 98% (4361/4420), 4.91 GiB | 15.00 MiB/s
Writing objects: 99% (4376/4420), 4.91 GiB | 15.00 MiB/s
Writing objects: 100% (4420/4420), 4.91 GiB | 15.00 MiB/s
Writing objects: 100% (4420/4420), 4.91 GiB | 4.78 MiB/s, done.
Total 4420 (delta 804), reused 268 (delta 55), pack-reused 0 (from 0)
fatal: the remote end hung up unexpectedly
Everything up-to-date

(The error was parsed as 5: The remote disconnected. Check your Internet connection and try again.)
2024-11-20T05:03:01.962Z - info: [ui] credential-helper: found GitHub credential for https://github.com/ in store
2024-11-20T05:03:46.312Z - info: [ui] Executing getAllTags: git show-ref --tags -d (took 1.225s)

it says nothing specific

Is there a github alternative ?

[thick marten]

i mean it says it was complete, maybe it finished

[frank hedge]

but it didnt

i try again now

[thick marten]

and if you push again, what does the log show since it uploaded it all?

[frank hedge]

its starting again

still just 1 branch

im waiting again and come back to see (probably to say it fail again lmao)

[thick marten]

its going to fail again

my only other thought is to get this initial one in chunks. you can choose what you commit.

so only commit a small bit, a directory or 2 for example, and see what happens.

[woven sluice]

Lots of people use azure devops instead when starting out to avoid getting hammered by github fees, I don't know if one is actually easier than the other

[thick marten]

your pushing almost 5gb which is going to cause problems in the first place with git according to their docs

[woven sluice]

and are you using LFS?

[thick marten]

they arent since thats pay

[woven sluice]

It isn't really an option not to, tbh... Sooner or later you'll have a 100 MB file, lol

[thick marten]

Just trying to help with what they're trying to do. I know that was suggested to use azure

But it's more work than just Github. So shrug

[frank hedge]

I will use Google drive and upload only my édit
With 3 backup
And 1 time per week 1 upload the full project

Less pain

Unfortunately technology beat me today

[wide raven]

Please do consider returning to the battle. Version control is a backbone of proper development

[frank hedge]

I will have to at some point of course
But everything i tried didnt work

Last solution maybe is to create a new project
Copy/paste my current project inside it slowly and uploading them slowly at the same time

[wide raven]

Like I said before, do a dry run with a totally blank workspace, add a dummy text file and so forth. Things will be easier once you get that rolling

[frank hedge]

I did with .ini file

I succeed uploading
But not my full project

[wide raven]

And yes, you can then push stuff in smaller chunks, I recall git can be a pain in the butt with big commits

ok so you are close

[frank hedge]

But after failing again after 5h trying.. pfiou

[wide raven]

I can't recommend using version control highly enough : I always make it mandatory when I occasionally teach gamedev

remember the gamedev motto : Pain is Fun

[frank hedge]

When its about working on my project yes

When its about trying to save it - while i can save it already - no

[pulsar parcel]

another benefit of ADO: it isn't there

github just made up something to monetise

[pulsar parcel]

you don't know you need source control until you need source control, so it's better to be proactive

also you will need to know the basics to work at any game studio

[frank hedge]

i would like to but i cant manage to make it work !! 😦

[pulsar parcel]

at least get rid of GitHub Desktop, the WOATed client

Fork is pretty nice, it's nagware if you don't pay for it, but pretty infrequent popups

and I would suggest you use ADO so you can use LFS for large files

[frank hedge]

what is ADO?

[arctic hornet]

Azure DevOps

it's microsoft's platform for general devops stuff (duh) and it includes among others, much better priced Git hosting service which afaik has unlimited LFS unlike github (which, funnily enough, is also owned by microsoft)

[frank hedge]

where to start?

[arctic hornet]

https://azure.microsoft.com/en-us/products/devops on their website

if that's what you mean

[frank hedge]

Im on it thanks
I will watch tutorial in the next days

[pulsar parcel]

you can create a repo there, and it's free up to 5 users

[frank hedge]

to be honest Im okay to pay for service as long as they are very helpful

[frank hedge]

but its also working from github desktop??

[arctic hornet]

don't know, never used github desktop

[frank hedge]

i check a tutorial on youtube and to upload on azure he use github desktop

wich i dont want as i think its not working because of it

[arctic hornet]

pretty sure it should be, since it's all just a git client behind the scenes.

[frank hedge]

can you explain quickly how it works from your UE5 project (inside UE5) to your azure?

[arctic hornet]

I don't use git or Azure/ADO at all

so I can't help you there sadly

[frank hedge]

you dont use any source control?

[arctic hornet]

I do, I use perforce

[frank hedge]

How good is it?

according to where i am right now, im ok to use anything that works lol

[arctic hornet]

in my opinion it's the best way tehre is for working with unreal... but the biggest negative is that you have to host it yourself, there is no service for it like github etc.

[thick marten]

If you looking for an alternative to GitHub desktop look at source tree or fork, I personally use fork. Both of them have free versions to use. Your issue is not with GitHub desktop it's with GitHub itself and the size of your repository.

[arctic hornet]

and it can get quite complex unless you are willing to spend a lot of time in documentation

[frank hedge]

So I guess I can check for this fork also

fork is also github?? wtf i dont understand anything lol

[arctic hornet]

Github is the service...
it is using GIT (protocol/standard) under the hood
ADO has GIT hosting service (alternative to github)
Github Desktop is a git client made by github
Fork is another git client made by someone else

[frank hedge]

https://docs.github.com/fr/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo this doc confuse me

as fork is also the name of a mechanism

[arctic hornet]

Yes, because forking is a github specific thing where you copy someone elses repository etc etc etc

I presume @thick marten was speaking about https://git-fork.com/

[frank hedge]

thanks for clarification
i willl try in the next days to git myself

[thick marten]

yes, sorry i didnt clarify I was on the phone. they got it correct and yes its confusing and dont you love technology

this is the other app I used to use, https://www.sourcetreeapp.com/, which works well but would choke on the epic repo due to how deep and large it was

azure is probably the smarter one for people to use with real repos due to less strict size issues, github is fine if you want public hosted code easily used. I personally store all my free stuff on Bitbucket (a github alternative) since its free for private for larger within reason. I use perforce for my business stuff since it works well. I use github for my public stuff that I give out for learning materials. there is no one answer if you notice lol

[ocean hearth]

Hello ,i have a basic question. i would only need sourcecontrol as a “backup function”. Since online storage costs money, how can I calculate approximately how much storage space I need? And above all is there even an “offline” version? P.S.: is there any performance impact using source control? sorry but i try to understand something as much as im capable before using something ^^

[arctic hornet]

in general there is no runtime performance cost to using source control since all your files are stored locally at that point

obviously there is some cost associated with pushing and pulling the data to and from such source control regardless of which server/product you'll be using

cost as in compute/time

most services like github, ADO etc... are usually a wrapper on top of underlying technology like git... so yes, you can absolutely use git locally or even self-host on antoher computer on local network

same can be said for perforce, it can run as a service on background of your computer... runtime costs are almost zero as long as you're not actively pushing/pulling etc.

hope that makes sense for you.

And regarding calculation, in worst case your storage can be at least 1:1 of what you have locally (if no compression is used etc...) but many services utilize plethora of compression algorithms, like gzip for text files, which can reduce the size by 75-95% depending on the content... and/or delta compression, which means that newer versions of the same file only store the bytes that have changed etc...

but in general, if your project is 30 GB in side, you'll ideally need at least 30 GB of space on the server, and the usage will grow as you push more and more stuff to it over time

For example practically speaking, our games files are about 76 GB in side (The versioned ones) and on our perforce server, after 6+ years of development we're at ~2 TB of used space (though the server is hosting ton of other things)

[teal bone]

So I have a git problem. I have a branch, "development", that's meant for coders. I have another branch, "prebuilt", meant for artists. The code from development is synced to prebuilt and then compiled and the dlls are added to the prebuilt branch. They are not tracked on the development branch because codesr are just building their own. The artists have a nice friendly environment to work from and don't need to build anything. This works great.

The issue I'm having right now is how do I get an artists to actually submit anything? If they make a feature branch from the prebuilt one and merge it into development (so coders get assets too, this is automatically merged back into the prebuilt branch as well) then it brings over the dlls from prebuilt into the development branch, which we don't want.

Is there some selective, advanced merge I can use?

I guess a cherry pick of just the new commits could be made...

But that's not something gitlab mr system seems to want to let me do.

Can create a cherry pick merge request fairly easily from a single commit,b ut that's not l ikely to be the case.

(I mean from the gitlab web interface)

Getting artists to squash commits is gonna be a pain.

[woven sluice]

i am a far cry from really knowledgeable in git, but since nobody is talking... i feel like the idea may have been flawed from the beginning. if i had to set up unreal dev using git, i would probably first be trying to set up a dumb trunk development scheme, no branches, and no binaries in git. have binaries be stored and distributed by teamcity. the repo would contain a script that pulls repo updates from git + binaries from teamcity. coders would just pull updates from git like normal, other devs would pull using the script. this simple setup would break down at some scale but for a small team probably work fine. for a little bit of safety the script could try to abort updating if the build server had a build in progress, or a chat bot can notify team members when binaries are updating/updated

[teal bone]

That's probably good idea.

Thanks!

[timber drum]

I use digital ocean for server setup. Pretty easy to get the ball rolling with their tutorials

[arctic hornet]

yeah, for me I host at my home office on my physical server(s) so I was going there from that perspective of manually installing p4dctl and configuring p4d and handling bunch of that stuff 🙂

[median crystal]

HI, i need to include a native unreal plugin into source control. I have the plugin running locally (in project plugins folder) and that's fine. But, if the engine plugin gets updated and i want to fetch it. Do i have to move my local version, fetch the new plugin version to then merge manually and then swap local back in again? Is this the workflow, anyone?

Thanks in advance if anyone knows

[teal bone]

Does it have its own git repo?

[median crystal]

No, it does not

[teal bone]

Oh that was a silly question. You've copied an engine plugin into your project so you can edit it?

[median crystal]

Correct

[teal bone]

I don't think there's a good workflow for that. 🙂

[neat grotto]

The latest version of Git LFS is out with some pretty significant updates.
https://github.com/git-lfs/git-lfs/releases/tag/v3.6.0

If you're using LFS, this is worth considering.

• Support for multi-stage authentication with Git credential helpers (requires Git 2.46.0) and
  relative worktree paths (requires Git 2.48.0)
• A new object transfer batch size configuration option
• Better path handling when installing on Windows
• More POSIX-compliant hook scripts
• Improved performance with sparse checkouts, partial clones, and Git remotes with large numbers of tags.

[median crystal]

Alright, thanks for the input 🙂

[teal bone]

I was thinking (and this is probalby a bad idea) to, isntead of copying to your project, symlink it to your project, so you can just edit and update in-place in the engine.

[median crystal]

Hehe, wouldn't verifying engine files cause issues?

[teal bone]

...probably 😄

I didn't think very far ahead with the idea!

[hoary wren]

Is it typically a bad idea to re-use the same folder & Perforce workspace for different streams, editing the workspace to switch between them? Will that confuse Perforce and potentially lead to unpleasantness?

[arctic hornet]

Not really, UGS allows for such switching

Might be some additional checks in UGS, but stream switching under the same workspace works there just fine

[silver token]

You should only reuse the workspace to move within the same stream tree. So you'd move the workspace between multiple streams for a particular project, but you should create a separate workspace for a separate project that has it's own tree of streams.

[kindred compass]

wasnt able to connect to my perforce droplet, went and looked at the stats and it had 100% cpu utilization O.o

rebooting the droplet fixed it btw. but gotta wonder wtf was going on there

[kindred compass]

uh, what? I'm literally connected...

[dire latch]

# Unreal Engine file types.
*.uasset filter=lfs diff=lfs merge=lfs -text
*.umap filter=lfs diff=lfs merge=lfs -text

# Raw Content file types.
*.3ds filter=lfs diff=lfs merge=lfs -text
*.bmp filter=lfs diff=lfs merge=lfs -text
*.exr filter=lfs diff=lfs merge=lfs -text
*.fbx filter=lfs diff=lfs merge=lfs -text
*.jpeg filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.mov filter=lfs diff=lfs merge=lfs -text
*.mp3 filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
*.obj filter=lfs diff=lfs merge=lfs -text
*.ogg filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.psd filter=lfs diff=lfs merge=lfs -text
*.tga filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
*.xcf filter=lfs diff=lfs merge=lfs -text

# Anything in `/RawContent` dir.
/RawContent/**/* filter=lfs diff=lfs merge=lfs -text

Is this the recomended GIT LFS setup? I mean, everything here is managed by lfs basically, so it's super easy to go beyond the free 1gb github gives you? Just wondering why I'd make everything managed by lfs and what are the advantages. Paying 5$ a month is not a lot.
I'm having some issues with blueprints that are being saved by unreal every now and then when I didn't change them, which makes it easier to get merge conflicts, maybe with lfs that wouldn't happen? idk

[pulsar parcel]

Don't even bother with GitHub with LFS

Use ADO or something with an equivalent offering

[dire latch]

it's the first 3 lines

the problem with ADO is that it's not free beyond 5 users

with github I could pay 5$/month for 50gb and this is it

[pulsar parcel]

Yeah and what is? You'll easily exceed 1GB and be paying out of the arse for "data packs"

[dire latch]

I'll not use 50gb, but certainly more than 1gb

devops for 10 users would be 30$/month

github would still be 5$/month because I'd be paying for the storage only

[pulsar parcel]

Well you also have to pay for data transfer and Unreal is very data hungry

[dire latch]

I have been using that setup with unity and never had problems, but yeah, maybe unreal uses more

ADO is harder to use also, specially considering the artists

[pulsar parcel]

I'd probably self host something at that point

[dire latch]

what about diversion? I just came across it

it's more designer friendly I guess

[pulsar parcel]

I don't really trust it and you have no ownership of your own data

They're very effective at astroturfing though

If I was setting up a studio I'd likely rip the bandaid off and license P4

[dire latch]

what exactly do you mean? is it something they say in the terms of service or something?

you mean you can't host your own server or something?

[pulsar parcel]

Well it's all magic fairy pixie dust in the cloud, no option to self host

No way to run your own backups

If the company disappears, your entire project history disappears with them, which is more likely since they look like a startup

At least with Git, the remote is essentially a redundant backup anyway

[dire latch]

technically you can use diversion and every now and then someone also pushes to git

it would be nice if they had an option to export your commits to git, they have an option to import git repository with the history

self hosting is a terrible idea unless you have someone experienced with security related stuff

that + backup, redundancy, etc

[pulsar parcel]

Well do what you want because I don't find those difficult

[woven sluice]

just be like me and run teamcity and jenkins on your best dev machine both exposed fully to the world wide web, nothing will go wrong

[ornate matrix]

what would be a correct way to host my source build binaries (and anything that needs to be used to correctly load and run a project from a designer perspective)

from the current host i am using for the project i cannot store the soruce engine on it, so im thinking about having a project repo and a engine repo on 2 different providers

since engine changes would be rare i can tell my designers when to pull the new binaries from the engine repo, im more annoyed about the potential size of a compiled engine

[arctic hornet]

the way it's usually done is to have both engine and project in one repository and artists just sync everything as programmers would do, but they then automatically download precompiled binaries from external source (i.e. build server) which makes sure everyone has everything necessary but designers don't have to spend 2 hours compiling engine from scratch

having two repositories sounds like pain and you'll probably have hard time making sure everyone is on correct version etc.

[ornate matrix]

yeah i know its not ideal but i dont have the budget for perforce or any sophisticate tool

[smoky igloo]

what is the easiest way to setup perforce just for local usage?

also is there a easy way, to get a perforce server running in a virtual box container?

[kindred compass]

if its just you (well, 4 or less people), perforce license is free and hosting is fairly cheap depending on where you do it

[smoky igloo]

I feel like it depends, if it's just a hobby and you gotta pay 20-30€ a month its not that cheap. Just wondering if there is a cheaper way then DO.

[ornate matrix]

yeah im kinda on that rn

i can host big repos on azure devops, so i dont want to pay anything elsewhere

if i can host my engine source bins on it its fine

[smoky igloo]

I probably need to get on it again, but I wanted to run a perforce server on my second PC, but there sadly is not an "easy" way of just downloading a image and put it on virtual box.

But as soon as you add some mega scan assets you easily need 100GB storage and thats getting quite expensive for a personal/hobbyiest

I had that running for a while but then after 2-3 months sparsely doing some work it felt quite expensive (just to justify it 😅)

even though I really don't like it and need to get it setup again because if feels so unsafe ^^

[kindred compass]

DO's cheapest option is $6 a month which is what I'm personally running. Comes with 25GB storage

[ornate matrix]

25GB is terribly small for UE dev

[kindred compass]

from a quick google search it looks like helix core supports Docker, no?

[smoky igloo]

there is just a helix swarm image or something at least thats what i found

[kindred compass]

https://www.perforce.com/blog/vcs/how-docker-works this implies core works with it too

[kindred compass]

fair. i did tack on a 100GB volume

[smoky igloo]

yea it was the same for me

[kindred compass]

found this reddit thread https://www.reddit.com/r/unrealengine/comments/18erk9x/has_anyone_successfully_installed_perforce_helix/

which lead me to this guys guide for setting it up on docker https://clait.sh/posts/unreal-version-control/

[smoky igloo]

https://hub.docker.com/u/perforce

but where is the helix-core image it is just swarm stuff

[kindred compass]

Quoting the second link I sent here:

Although Perforce doesn’t offer a pre-made Docker image, we can follow the official documentation to create one.

so it is build your own

he does walk through the entire process tho. Havent personally gone through it obviously, so dont know if the guide is actually fruitful. but seems like a solid one at a glance

[smoky igloo]

But I don't really care too much about docker, I'm also fine running a ubuntu instance that hosts the server

Btw how would I go about running perforce locally can i just install it add everything and then I'm ok?

and it is tracked?

and later on if i have the server i can push it?

every time I'm trying to get into it I'm kind of overwhelmed, it feels so much more complicated in comparison to git and I fear fucking something up 😅

[arctic hornet]

there are two ways of doing local only perforce

either using DVCS which is part of P4V (locally ran, lightweight variant of p4d)

or just simply installing p4d (perforce server) on your local machine

[smoky igloo]

what would you recommend?

or is it better to get the perforce server setup running on my 2nd PC and then connect to it?

[arctic hornet]

I presonally would recommend full p4d server as you can later just copy the P4ROOT folder to another machine/os and it'll work

I'm not sure how/if the files made by DVCS are compatible with p4d

[smoky igloo]

and one more question about perforce

[arctic hornet]

if you have spare separate machine it can be useful to have it separate, since it can serve as a source of backup in case your main machine dies... but you can easily just run it on your local machine (the benefit will be since it's localhost only you'll not be limited by your NIC speed)

[smoky igloo]

If i have the repository locally and my server burns down or vice versa then I'm fine everything on the other PC is still there

[smoky igloo]

Yea thats my thinking

[arctic hornet]

unlike git, with perforce you locally only have the snapshot of what you have currently synced

but server has whole version history and everything

git pretty much works like a mirror/copy of the remote repository... perforce works on client <-> server model

so you have to be connected to the server any time you want to do anything

[smoky igloo]

What does snapshot mean?

And can i work locally and later on push it to the server or do i have to have a active connection?

[arctic hornet]

snapshot was probably not correct wording on my end, but it simply means that if you sync to changelist 123, your computer will have local state same as the server has in changelist 123... but your client doesn't have any knowledge about history or the future, that's all on the server

i.e. perforce doesn't do .git folder

[smoky igloo]

okok so locally you are basically just on HEAD all the time

[arctic hornet]

well, you can technically work "offline" and then later "reconcile offline work"... but in general you want to be always connected to the perforce server especially since server is keeping track about who has what files checked out (opened for edit) etc...

[arctic hornet]

not really, you are wherever you want to be in the whole history, but you only have locally what was the state at that point in time

with git you usually have whole tree offline, so you can go back and forth in the history - or browse the history - even witohut being online

with perforce, you just have the current files, nothing more, nothing less

[smoky igloo]

Perforce is frying my brain, but it's probably just because I'm so used to git

[arctic hornet]

yeah, it's quite different compared to git

[smoky igloo]

But it's reasonable right? Because files are bigger and thats a better way of handling it right?

[arctic hornet]

indeed, since you have only one revision locally your project size is consistent give or take, and doesn't grow just by adding more stuff to the history

either way, I'd recommend checking these two playlists, it might give you some insight into perforce:
https://www.youtube.com/watch?v=jIQEjDiSe0g&list=PLH3pq2J85xsPYn71_yzzsZQKvalTW-duE

https://www.youtube.com/watch?v=LITmtpuLrQQ&list=PLH3pq2J85xsMDY2AAkzUhTmHIg1S7WN1f

first one is from the point of the user, so explains the p4v UI, how to work with it etc... and then there's the second series which is focused on administering the server

[smoky igloo]

I think I watched the first one a year ago, need to rewatch it.

What I still remember the workspaces it's called right?

[arctic hornet]

that's one of many terms used by perforce, indeed 🙂

[smoky igloo]

I didn't understand them, why do i name it like natschz/perforce_12341234 or something

is it like a branch

is it just like a checkt out commit

why do i have the user name in there even if it's on my local machine

is this just a local thing or does it matter to the server

[arctic hornet]

so, let's say it like this:
on top level there are depots - which roughly equals to git repositories (if using stream depots - non stream depots are more like SVN)
second level inside the depot is the stream - which is like a branch.... inside the stream you have your files
then there are workspaces which defines how the files from the server are mapped to your local machine (i.e. your local work space) - it usually just consist of you selecting local root folder and what stream you want to sync

perforce is built on usernames simply because unlike git, where if you see the repository, you have access everywhere, perforce has full permission/ACL system, so different users can see different parts of the streams, or even different depots entirely

let's say you have two depots, //UE5 and //Fortnite
Fornite will be visible only to users in group "Epic Employees" meanwhile UE5 will be visible to both "Epic Employees" and "Public" (giving hypothetical example)

it's still the same server, but what user sees differs on who they're logged in as

[smoky igloo]

Ok yea I was not even thinking about the permission/ACL stuff since I'm solo rn

[smoky igloo]

I feel like that cleared up things a lot! thanks!! 🙂

Would files between streams be shared?

[arctic hornet]

Honestly, I'm not sure about that, iirc they're usually separate/copied over

[smoky igloo]

also I think one really important question, how does perforce do commits/diffs?

if i change a file, whill a full copy of it be stored?

[arctic hornet]

but there are new things like sparse streams etc etc which do some magic behind the scene

[smoky igloo]

meaining if i change a 1GB file 5 times it will use 5GB on the server?

[arctic hornet]

depends

text files are stored using deltas, butr binary files are/were stored as a whole

but last time I checked that was like 10-12 years aso

ago*

so stuff might've changed since

[smoky igloo]

because I feel like you can blow up your depo size pretty fast

[arctic hornet]

oh, that's for sure, but at the same time it allows perforce to be blazing fast with ton of files

because it doesn't have to do ton of runtime computation to get you that file

[smoky igloo]

True

is there a way to remove some history from a depo?

[arctic hornet]

in general it's not recommended, but yes

there are two general ways... obliterate - which means "nuke it out of existence"

but you can also set special flag for given file to for example keep only last 32 revisions and delete the rest

[smoky igloo]

I mean I wish so much that I could use git but I also see the issues with that and there probably is a reason perforce is the standard regarding things like that

[arctic hornet]

but still keep the metadata

[arctic hornet]

perforce is de-facto standard for the fact that it's fast, handles large binary files just fine and... came out waaaay back in the day

git didn't even support LFS natively until relatively recently

perforce did handle large binary files since day 1 which iirc was before year 2000

not sure about the exact dates

[smoky igloo]

Yea and still until this day if you have a 50+GB repo LFS wil struggle with it

[arctic hornet]

meanwhile my/our perforce server is currently 6 TB in size (hosting multiple company and personal projects over 10+ years of history)

and it's running like on day one

[smoky igloo]

Thats awesome

but 6TB feels quite small for 10+ years

[arctic hornet]

we're a small indie team

[smoky igloo]

I'm using some mega scans

this adds up GB's so quickly

[arctic hornet]

oh yeah, that can easily do that, we're not using megascans since our game is being stylized and previous games were 2D platformers etc...

• perforce does server-side compression, so text files are gzipped - which reduces their size by like 75-95%

[smoky igloo]

I mean if i get the perforce server setup on the 2nd PC it's fine

[arctic hornet]

and every saving like that can stack up quite easily

[smoky igloo]

yea

I mean you can get a 5TB HDD for ~100€thats ok

so if i get the setup done on my 2nd PC its ok

[arctic hornet]

we have SSD raid with redundancy, but yeah, still cheaper than hosting in the cloud ngl

[smoky igloo]

yea

I wish I could cloud host that

I'm also thinking about a SSD since the HDD still is a bit noisy

[arctic hornet]

I highly recommend SSD for your perforce server if you can, since especially with large unreal repos you'll easily get bound by the random read/write performance of your HDD

and for solo project 1-2 TB SSD is plenty for quite a long time imo

[smoky igloo]

Yea gotta think about that

[arctic hornet]

and even if it's not, migration is as easy as running cp /oldDrive/p4root /newDrive/p4root and you're pretty much done

[smoky igloo]

How easy would it be to move a perforce server to a different hard drive?

just copy the folder?

[arctic hornet]

there's no special sauce, perforce stores everything in the root folder where you tell it to

[smoky igloo]

ok thats awesome

[arctic hornet]

everything is file

[smoky igloo]

Yea gotta look into the ssd thing, since my 2nd pc is a thin client 😅

[arctic hornet]

archives is the bulk data/files themself
journals is the history/checkpoints
logs are... logs
root contains the database/state of perforce server

and that's all, you just point your perforce server to this folder and that's it

[smoky igloo]

thats awesome

so I could do the setup and then just copy it over

I mean since I would set it up in a VM anyway i guess i could just copy the whole vm 😅

[arctic hornet]

true

[smoky igloo]

but might be good do know for backing up the server

[arctic hornet]

We're hosting our perforce server (and whole studio infra) on physical server in my home-office since it's cheaper in a long run and gives us enough perf for crazy things (like trying to push whole unreal update there on 10gbps uplink )

[arctic hornet]

backup procedures are quite easy, since it's just files, you can just robocopy/rsync/whatever your archives folder and if you have latest journal, you can always restore from the journal, since those are like checkpoints for the metadata

[smoky igloo]

I totally get that, i mean 5TB would be 500€ on DO each month if I'm not wrong

[arctic hornet]

oh yeah, for sure... not counting the VM itself and the egress pricing

[smoky igloo]

yea

[arctic hornet]

like our perforce server's monthly traffic is about 70 TB

and we're just 6 people

[smoky igloo]

that is insane, I mean I get that having it in the cloud and the safety and if your physical server burns down its bad

but you probalby have the projekt on another machine

[arctic hornet]

we have secondary replica of the server in different physical location for backup purposes

just in case

mostly not caring about perforce failure, hasn't happened yet, but I've had my fair share of raid failures over the years

[smoky igloo]

but 500€ just for the storage is quite a lot especially for a indie team, a few months and you have a PC that has better specs then the cloud and you just paying for electricity

[arctic hornet]

mostly older raid cards giving up and blasting all disks with random data so the raid was unrecoverable (was painful friday afternoon)

[smoky igloo]

uff :/

[arctic hornet]

cloud is extremely costly

[smoky igloo]

ofc things like that have to happen on firday 😅

[arctic hornet]

there are some providers like OVH etc where the costs are better but it's still 100+ euro a month

[smoky igloo]

yea, I mean I would not want to host a webserver or game server on a local machine but for things like a perforce server

I mean a thin client + a 2tb ssd would cost about 500€ ?

[arctic hornet]

black friday deals are actually decent ngl...

[smoky igloo]

could even put one into another physical location

[arctic hornet]

it's 6 core/12 thread, 32 GB ram, 2x1TB SSD and 1gbps (unlimited) uplink for 77 euro a month - but that's without VAT

[smoky igloo]

yea but afterwards

[arctic hornet]

yeah, as I've said, cloud gets costly... self-hosting has higher barrier for entry because of the costs necessary to purchase the hardware... but it balances over time

our yearly costs for the perforce server (excluding perforce license) is about 500 USD because of the power requirements

with initial cost of 3000 USD worth of disks and hardware when we bought it 6 years ago

[smoky igloo]

and i gotta say, I thought the same thing, 20€ its fine have a DO perforce server, but then I had it running for 2-3 months where I was just sparsely working on the project and then when you are not doing things on it for a month you start wonder

[smoky igloo]

Yea but than take 500-600€ a month on DO

thats 7000+ a year same I'm thinking about the 20€ it's 240€ a year

and for my I have a thin client for smart home stuff and that certainly has some resources left so

[smoky igloo]

I'm currently trying to setup perforce but getting:

Connect to server failed; check $P4PORT. connect: 192.160.0.2:1666: WSAETIMEDOUT, A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

I have create a rule for port 1666 is there something i need to do, maybe since it's a local server ssl is an issue?

or was there some thing where it needs to be an ssl connection and since i don't have a ssl certificate by a know authority I need to create a custom certificate but then i need to install the certificate on my machine as well and keep it up to date ? :/

[arctic hornet]

perforce can handle self-signed certificate just fine, you'll just get prompted by the "Do you trust this connection?" window

and if you're using ssl, then make sure to use ssl:192.160.0.2:1666 etc.

but otherwise it's hard to say what's going on

[smoky igloo]

I'm trying to connect like this

and I'm not sure if I'm using ssl just defaulted it

[arctic hornet]

depends on how you installed it

• what OS etc...

[smoky igloo]

ubuntu

[arctic hornet]

are you using p4dctl?

[smoky igloo]

what is that?

https://help.perforce.com/helix-core/quickstart/current/Content/quickstart/admin-install-linux.html

I followed this guide

[arctic hornet]

well, one of the last steps is using p4dctl 🙂

it's the control plane for the services

i.e. start/stop.. etc.

[smoky igloo]

ohh no i assued i can just connect remote ^^

[arctic hornet]

well, you need to make sure you start the server first and that it's running

as that is the last step of the guide 🙂

[smoky igloo]

*not numbered step 😅

but I've to say, its clunky now i want to do p4dctl status but it does not work

i just want to know if its running or not ^^

[arctic hornet]

🤷‍♂️ well, it needs to be run as perforce user iirc

i.e. sudo -u perforce ....

[smoky igloo]

ohh i see

yea but it is running i says

[arctic hornet]

try connecting locally and double checking your firewall

although, now that I've looked at the time, it's almost 3:30 am, so I'm going to leave it here for tonight, good luck!

[smoky igloo]

yea i should go too 😅

but thanks for the help and was a nice conversation 🙂

[smoky igloo]

If I want to add a existing project to a perforce server, how would I go about it?

Just create a depo and stream and then copy the project into the local folder and push the changes? (Probably need a perforce ignor file as well right

[silver token]

You can point the workspace root at the current location of the project if you want. You don't have to move it any place special.

[pulsar parcel]

Though I would suggest having it in a subdirectory so you have the potential to use a source build later without having to rearrange your project stream

[smoky igloo]

you mean .../workspace_directory/game_directory instead of having the game directory directly as workspace directory?

[pulsar parcel]

Yes because if you do end up using a source build later then the project doesn't need to move

[smoky igloo]

Cool thanks for the info! 🙂

Also there is a perforceignore file similar to the gitignore right?

Do you have a good one for ue5?

[pulsar parcel]

Well a lot of the public gitignores are also wrong

But Intermediate, DerivedDataCache, and Saved are a good start

Then IDE files, and Binaries if you don't store build artifacts directly in the depot

[smoky igloo]

Ok cool

and what are the workspace mappings?

should i exclude all depos besides the one i want the workspace to be in?

[smoky igloo]

I've a depo I create a workspace and now clicking add won't do anything 😦

[pulsar parcel]

You don't use workspace mappings with streams

Please say you at least made a stream depot

[smoky igloo]

Yea I did

[pulsar parcel]

Then you don't use workspace mappings at all

[smoky igloo]

But when creating the workspace, I need to set them

[pulsar parcel]

No you don't

They don't work with stream depots

And mappings will be read only

[smoky igloo]

so i should remove all of themß

[pulsar parcel]

You select the stream from the stream field and that's it

[smoky igloo]

how do i create the stream? ^^

ohh ok got it for that i need to be in p4v

[pulsar parcel]

You haven't created a mainline yet?

[smoky igloo]

what is a mainline?

[pulsar parcel]

A branch that you usually call "main"

[smoky igloo]

no

[pulsar parcel]

Then make it from the streams tab

And then create your workspace to use it

[smoky igloo]

is it ok if the workspace is already created?

[pulsar parcel]

Well you'll need to edit it instead

But the stream creation needs to be out of the way first

[smoky igloo]

ok i created the stream ^^

[pulsar parcel]

Alright, edit the workspace and edit the stream field so it points to the main stream you just created

[smoky igloo]

ok but populating the stream with the wizard i can just neglect right?

[pulsar parcel]

Not sure what wizard that is since it's been a while, but it's easy enough to populate yourself

I probably should have my personal project using P4 at this rate since work is stuck on Git for the time being

[smoky igloo]

dude I feel like it's so clunky, if everything is setup it's probably ok, but now i have a workspace, if i click edit workspace a popup opens where two workspaces show up where i can change a checkbox and if i click ok thats it

but i also cant remove the workspace i created

[pulsar parcel]

Screenshots probably help

And you can just delete workspaces anyway

[smoky igloo]

I just realized edit recent workspace does mean that we just edit the list 😅

how can i edit the stream for the workspace

[pulsar parcel]

Well I just use the menu option to edit a specific workspace

[smoky igloo]

what menu option do you mean?

[pulsar parcel]

Like I said I don't have P4V in front of me to tell you step by step

Tools menu maybe? It's one of the top ones anyway

[smoky igloo]

npnp i think i found it ^^

now the + also shows up when adding a file ^^

btw thanks for your help

is it possible to delete added file from change list?

[pulsar parcel]

yeah, revert

[smoky igloo]

@pulsar parcel since my workspace folder is not the game folder, will this cause issues with rider to check that it is in a perforce workspace or in the ue editor with the plugin?

[pulsar parcel]

?

[smoky igloo]

in git IDE's have some issues when not opend with the root directory, wondering if this is the case with perforce (currently waiting for the stuff to be pushed thats why I don't want to mess around to much 😅)

[arctic hornet]

perforce does not care about your folder, it keeps data stored in environment variables

[smoky igloo]

ok cool 🙂

[woven sluice]

thx perfarce 💩 https://gyazo.com/13f441b6accd42f558ab4432f8b21a3d.mp4

[woven sluice]

if that isn't a load of fucking horseshit. so i tried to switch my workspace from a mainline onto a virtual stream, then when it asked me if i wanted to get latest for the stream i hit no, and then it aborted, shelved all my WIP into this, and ... hid it

great, slashes no allowed in depot path, that's definitely normal

I'm probably going to have to ssh into the server and do open heart surgery to put these shelved files data somewhere accessible again. This is the counter argument to "oh perforce is easy to admin, I just install perforce and it's been running for a couple years hassle free"

[teal bone]

Lol

[woven sluice]

REMARKABLY, i updated P4V and now it works normal, amazing

good thing i threw up my hands and went to bed instead of reformatting all in anger

oh, no wait. it was there and then i started doing stuff to restore the data and it resumed 🤣

[pulsar parcel]

Well you did leave an unpatched TeamCity exposed to the public internet...

A more serious comment is was the server ahead of the client version? You can actually configure the server to reject old client versions

[woven sluice]

if it was, it wasn't grossly different. i'm never using the workspace stream switching feature ever again that's for sure

all of my "add" files are on the server stuck in this corrupt changelist's shelved area... the changelist has a few other "edit" and "delete" entries that seem to be messing it up

like there's one of my new added files lol

i wonder if p4d has the changelists in any actual physical files human readable

[merry verge]

it sort of does

the depot directory in p4d has files in a readable format, it's kinda weird

I think it acts as backing file storage for the actual database that deals with revision history

[woven sluice]

ah, maybe i won't be able to cut the tumor out of it that way then

probably just stick to my original plan of copy/pasting out the .gz files for the files i want to salvage and ... re-embedding them somehow

or just pull them back out to my windows machine, dump them back into my project, and try adding again as part of a new change, maybe

tedious but it's only like 20 files or less

this is two ways i've managed to break perforce changelists. first way being using Jenkins' perforce plugin with parallel submit enabled. am i the only one this good at breaking things?

[teal bone]

Some people are wrecking ball operators.

[silver token]

I remember having this happen once when streams as well. Sadly I don't remember what I did to resolve it.
I think I resorted to the commandline (p4v limitations aren't always the same as the actual command limitations).
I wouldn't have had any sort of special access to the server or disks.

[woven sluice]

hmm thanks maybe i'll try for a few more minutes to mess with that first

p4 submit: change belongs to another client
p4 info: perfect match??

[woven sluice]

ah i think i got something! p4 reopen -c default //.../file

moves the file out of the fudged changelist into default changelist

[woven sluice]

now i just can't delete that changelist 401 lol

nvm, just had to switch to admin account and start blastin'. alrighty thx perforce there's 2 or 3 hours of my life i'll never get back, thanks all for helping me rubber duck 😄

[arctic hornet]

it's not perfect match, p4 switch either got confused or created a client name which contains the client name + client stream concatenated (unless it's default behavior - I've never used p4 switch)
so client Megacity_KWilcox is not the same as Megacity_KWilcox//Megacity/....

apparently p4 switch is supposed to unshelve the files when you switch back to that stream etc.

[woven sluice]

Yeah seems like it must have some internal thing to create hidden workspaces

[foggy juniper]

I've just started using perforce and I've set everything up. However I keep having one issue, after I submit my work and I try and compile I get this error in rider
11>LINK: Error LNK1104 : cannot open file '\Binaries\Win64\UnrealEditor-X.dll'

It seems to be because I don't check out the \Binaries folder, however is there a way to make Rider check it out automatically?

[silver token]

Generally, binaries aren't supposed to be in perforce.

[wide raven]

https://forums.unrealengine.com/t/best-p4ignore-for-populating-perforce-with-ue4-engine-source/555073/2

It would be nice to have an official boilerplate ignore file pinned, I wonder if one exists

[ocean hearth]

Hello quick question for some people with expirience with that. The Price design not easy to understand, because where it says free of charge it says “start free of charge” and I'm always rather cautious about that. I have read from several sources that Azure Devops as a source control provider is free. Does this also include storage space. if so, how much? will it remain free indefinitely?

[wide raven]

At the moment at least, they are betting on people that use their free services to grow up and use their paid services, for example Github Enterprise ( which gives larger LFS ) etc

I haven't heard what people expand to in the case of Unreal, but if I had to guess I would imagine Azure Pipelines ( CI/CD system ) might be a natural next step

That said, perforce remains the industry standard for Unreal devs, I am not aware of large studios using git

[ocean anvil]

no because this is different depending on your set up what software you use, whether ur project is C++ whether you have additional folders for art, whether you have plugins etc.

[ocean hearth]

so is there any budget solution then? since i dont like to pay montly something. is there an offline solution like with an nas?

[wide raven]

"Boilerplate code is computer language text that you can reuse with little or no alteration in several different contexts."
The idea is to serve as a good startingpoint.

[wide raven]

Azure seems to be the most often recommended zerobudget option

SVN might be another angle

[ocean hearth]

i dont find any specific source where i can read about the possibilty of free azure :/ or rather whats not possible ^^

[foggy juniper]

The problem is I'm the only programmer on the project and I don't want people having to compile the whole project and a lot of custom c++ and we don't have a build machine setup

[silver token]

Well, there's no way that Rider will check them out for you.
At best you could change their filetypes (in perforce) to have the +w option. This leaves the files write-able on disk.
But you then have to figure out what binaries to check in when you're done making changes.
This generally doesn't scale beyond one programmer though. It can stretch to two if both are very conscientious about their checkins.

[wide raven]

Once the first build is done, its mostly plain sailing. If you really do a lot of cpp changes which cause a need for rebuilds, can make a .bat script which cleans intermediate+binaries and regens solution. Then noncoders just need to run that and press play in ide. Its not so bad.

[foggy juniper]

Okay thanks, I'll look into it some more!

[pulsar parcel]

Did you not use the typemap that Epic provides? Pretty much every executable is +w in it

[wide raven]

Perforce setup : typemap and ignore https://dev.epicgames.com/documentation/en-us/unreal-engine/using-perforce-as-source-control-for-unreal-engine

already pinned, but at the very bottom ( start ) of pins

[ocean anvil]

i am aware what it means.

[woven sluice]

https://landelare.github.io/2022/09/27/tips-and-tricks.html#automatically-update-c-binaries

highly recommend setting up a big complex CI system but, you can probably get quite far along with just this

[pulsar parcel]

I'm not sure I would recommend that approach since it can be error prone (eg. artists not installing exactly the right dependencies, and also could be in violation of the VS Community EULA if there's enough of you)

CI isn't that scary as long as you don't let anyone and their port scanner have access

[ornate matrix]

Any resources on that ?

Im thinking about editing some plugins source code but still thinking about the setup (the engine source will be one repo from one provider, and inside (ignored by the engine repo) i will have the project repo host on another provider)

I still have to define whats the minimum i need to not ignore in my local engine source so when i compile the latest engine version and commit it to the repo i don't keep unnecessary files

And how to handle cases where designers loads the project without pulling the latest update

[woven sluice]

teamcity is a small behemoth, just start by watching any tutorial videos you can find, install it and play around with it (do NOT expose it to the internet)

[foggy juniper]

Aw this could be useful! Thanks!

[crystal merlin]

When using ISourceControlProvider::Execute(FCheckIn, PackagesToSubmit) from the code, is there a way to force the submit to fail if any of the provided files cannot be submitted?
Currently using P4.

E.g. When I pass these files, I want the entire action to fail instead of submitting A.txt/B.txt:
//Repo/A.txt - Ok
//Repo/B.txt - Ok
//Repo/C.txt - Doesn't point to a proper path, file doesn't exist.

The call to the function returns Failed code, but it still creates a changelist and submits it with partial list of the provided files.

[arctic hornet]

Changelists in p4 are all or nothing in a sense that if something fails to submit while changelist isn’t submitted…. But at the same time perforce can’t begin submit process without making numbered changelist first.

You could probably split the operation in perforce into multiple steps and only try submit if you have any files to be submitted (I.e verify that they exist etc first)

[crystal merlin]

Yup, it's all or nothing, though when it makes the numbered changelist and it fails to reopen the invalid paths, it still submits the remaining valid paths (at least with the unreal integration).

Guess I am going to do manual reopen on all of the paths before doing checkin.

[arctic hornet]

Also I recommend checking the ‘USourceControlHelpers’ class which as name suggests contains more high level wrappers for buch of operations and might prove useful

[blazing arch]

Hey I 've never used source control before and I was going to be using Git but I also heard Diversion is really good, which is best or a solo dev (all I want to do is be able to roll back in case of crashes)

[pulsar parcel]

I'd still probably go with Git for peace of mind

[blazing arch]

what do you mean peace of mind?

[pulsar parcel]

Git is a mature open source thing

Git is a DVCS so even if your remote disappears you still have your data

[blazing arch]

I see

[blazing arch]

The reason I asked is because I heard uploading UE files some times is slower on Git than Diversion

[pulsar parcel]

"I heard"

that thing again

[blazing arch]

Yeah

[pulsar parcel]

and it depends who's hosting your repo

[blazing arch]

I wacthed a Cobra Code video on it basically

[pulsar parcel]

who the hell is that

[blazing arch]

https://youtu.be/GDgEzTxAGSA?si=2bbrL_0V4to3mdMe

[blazing arch]

this guy

[blazing arch]

What do you mean?

[pulsar parcel]

you don't know what a host is?

your remote could be anyone (github, azure devops, gitlab, your own self-hosted thing)

[blazing arch]

TBH I found out about vesion control yesterday

[pulsar parcel]

also this video is a poorly disclosed ad (which diversion are good at). the diversion link in the description has tracking on it

[blazing arch]

the guy also has a Git tutorial

But does Diversion commonly run ads like this?

[pulsar parcel]

they're good at astroturfing on Reddit from what I've seen

[blazing arch]

How can you tell it has tracking?

[pulsar parcel]

utm codes in url

[blazing arch]

Are there any advantages to Diversion over Git?

[pulsar parcel]

no idea, with being burned by plasticscm a few years ago when that was new, I have an automatic distrust for a brand new SCM

didn't help when I saw the astroturfing, their original ToS, and the lack of self-hosted options

[blazing arch]

I see

Thanks for the insight

[modern tulip]

Hi, he clearly states in the video (at about the 1:00 mark) that it's sponsored, plus it's marked as shown in the screenshot. We are always making sure to add a disclosure if the referral is coming from us. UTM codes are a standard method to measure referral performance, it's aggregated and non-personal data.

[modern tulip]

Would be happy to take care of any astroturfing if you can please dm me examples. The ToS were fixed, some of it thanks to feedback from you among others 🙏 . Self-hosting I hope we can offer in the future, in the meanwhile I agree it's suited to devs/artists that are after a more "hands-off" solution without maintaining their own infrastructure

[devout walrus]

Anyone got a good alternative to sourcetree? It was working fine but recently just started acting funny on my one pc and I can't seem to get it to work no matter what I try. Tried github desktop and it worked instantly but ideally I want something similar to source tree. GitKraken looked alright.

[wide raven]

Fork for the win

technically a paid thing, but so far they have allowed people to run it free with practically no nagging for buying. I bought it after using it for a year

[modern tulip]

Another option is Rider, their source control integration is very good, and it's free now for non-commercial use

[wide raven]

interesting, I've never tried the integrated Rider thing

[modern tulip]

Do you have something like this in DefaultGame.ini?

[StartupActions]
bAddPacks=True
InsertPack=(PackSource="StarterContent.upack",PackName="StarterContent")

[arctic hornet]

There is plenty of cases this can happen, most common reasons for invalidation/resave are:

• underlying format change (I.e epic changes something under the hood and miss few files that then get shipped with older version and need to be resaved)
• in case of materials it can be the usage flags (I.e the “Used with nanite meshes” for example)
• or you just change something which propagates through…

The file you mentioned is used as a background in the editor windows (not level viewport but the others, like material preview etc) which gets driven by “preview scene” settings/presets, so there could be something there that’s causing the invalidation

[modern tulip]

If it happens mostly when you start UE, removing this section from the ini file might solve it. Though klu's suggestions could also be the cause of course

[smoky igloo]

hey guys as far as I understand, history in perforce is only stored on the server and locally I only have the current revision right?

If I want to checkout a revision from the server, I just do get Revision and locally the files are reverted to that point in the history?

[wide raven]

Yes. There's a couple of checkboxes that can adjust the workings.
The annoying thing is that nothing actually shows as checked out, or shows that your project is at the specific CL effectively. So the project is essentially in a weird state that way. I generally use it to pinpoint the CL in which something went wrong, then make sure to be back on the head before changing stuff around.

If somebody has more pointers in this area I would love to hear

[smoky igloo]

awesome thx 🙂

[spice imp]

Hey. Kind of a weird question. Currently my project needs to be built locally by each individual that clones our perforce repo. This is a huge pain in the butt and not really required by anyone except two programmers. What files and folders do I need to make available in our p4ignore file to ensure people can just clone and fire up the editor? Right now I'm currnetly excluding:

*Intermediate/*
*Saved/*
*DerivedDataCache/*
*Build/*
.git/*
.github/*
.idea/*
.vs/*
*Binaries/*
!Plugins/MeshMorpher/Intermediate/*
!Plugins/MeshMorpher/Binaries/*

/*.sln
/*.vsconfig
*.VC.db
*.opensdf
*.opendb
*.sdf
*.suo
*.xcodeproj
*.xcworkspace

[cyan jay]

The Binaries folder is the important one. Checking binaries in comes with lots of potential issues, but is an OK solution to get started.

[spice imp]

Is a built binary folder the only thing that is needed to make sure they can run it? Everyone is on windows, but there are different windows versions (10, 11, etc)

[cyan jay]

Yes, should be. Also if you have code pluggins in the project, they each have their own binaries folder that's required.

Each user will.need to run the prerequisites installer at least once too.

[spice imp]

Okay, I can also include the code plugin binaries folders easily. By prereq installer do you mean right clicking the uproject file and clicking generate?

Also thank you for the info, I've tried googling but I couldnt find targeted information

[cyan jay]

If you're not using a source build, then each user needs to install unreal themselves, in which case prerequisites are installed automatically.

[spice imp]

Ohhh, gotcha. Luckily we're not using a custom build so they should be fine there. I will make these changes and give it a test. Thank you 🙂

[smoky igloo]

Sometimes I randomly lose connection to P4V and then after a while I notice that UE is not asking me to checkout the files, so thats when I restart/reconnect and I can continue.

I just wonder, what if there are files that are not checkedout and have changes, how will PV4 know about them, can I check for all those files, or is there somethign to prevent this?

[pulsar parcel]

a good ol' reconcile on the content directory

though it's odd that that'll happen

[arctic hornet]

check the logs, the reason could be that your session expires, unreal kinda doesn't handle re-login and expects you to do so beforehand, for example by using unreal game sync, or just p4v to sync

[smoky igloo]

But is Perforce smart enough, to see that I created/changed files?

[arctic hornet]

well, perforce is peculiar beast... locally it doesn't know anything... everything is stored on the perforce server itself.

so unless you're connected, it won't know about the added/changed files

to tell perforce that you've done some changes while offline ,you need to run the "Reconcile Offline Work" as sswires said, which tells perforce to go and check local state vs what server expects

[smoky igloo]

Because I just noticed that I add/change files and no checkout message is coming on saving, and then i reconnected it's fine, but some of the files where just recognized after i switched in the folder.

So my biggest fear regarding this is, that some things are not on the server that should be

[arctic hornet]

reconcile offline work is your friend 🙂

[smoky igloo]

Ahh ok thats awesome! 🙂

[arctic hornet]

just run in on your Content folder in p4v, it'll churn through and find the changes

it's located right here

[smoky igloo]

Aweomse thanks, that will ease my mind a lot!

[arctic hornet]

though if you run it on workspace root expect it to take relatively long time, as it does a lot of computation under the hood

so it's best to run it on limited context

[smoky igloo]

And then I sse the files that are not in depo and if i hit reconcile they will be added right?

[smoky igloo]

Yea i see that, otherwise it might be a overnight thing

[arctic hornet]

Although it can be relatively fast if your files have not modified, because for speed it first checks the "Last Modified" time on the file... if it's the same as the server expects it will not compute the hashes of the file (which is why it takes so long in large scope)

[arctic hornet]

if used in p4v it will first prompt you with the changes and then does the reconcile again after you confirm your selection

if used from CLI (p4 reconcile) it directly adds the found stuff to the default changelist

[smoky igloo]

Thats awesome, I wondered how it was so fast on the content folder ^^

[smoky igloo]

ok thats perfect 🙂

Thanks a lot for the help!

[arctic hornet]

you will get presented with this window 🙂

[smoky igloo]

Yea exactly and I hit reconcile now

[arctic hornet]

as you can see, I modified the BPT_Credentials.txt without doing checkout first... and then I added a file without doing p4 add - so it found one file that needs to be added and one that has changed etc.

[arctic hornet]

indeed

[smoky igloo]

I also see them in my pending change list

[arctic hornet]

yep, that's correct, it will add files to the pending changelist, so you can decide what you want to do

perforce never does anything unless you tell it to

so, if you tell it to find offline changes, it finds them, when you say "yes, I want to reconcile these" it will add them to your changelist, so you can decide if you want to submit them, revert them, etc etc

[smoky igloo]

Thats nice, I still need to get used to it, having used git so much, it's a little bit different, but I also understand that perforce has to handle a different thing than git.

[arctic hornet]

perforce is quite different from git, but once you'll get used to it I find it much more convenient and scalable (though I'm biassed as I've been using perforce as my primary version control for more than 10 years 🙂 )

though one thing I'll say is that perforce can feel old-fashioned in some ways... and that's because it is... 😄

(initial release was 1995)

[smoky igloo]

I'm just used to

git status
git diff
git add . 
git commit -am "Commit message"
git push

[arctic hornet]

yeah... this could map to p4 commands too, it's just that due to the locking you have to checkout stuff first... 🙂

p4 add ./new-file.txt to add new file to default changelist
p4 submit -m "Message" to submit default changelist with description "Message"
p4 checkout //depot/file.txt to checkout the file
p4 revert ./file.txt to revert checked out file
p4 diff ... to diff whatever (I omited the args)
p4 delete ./file.txt to delete a file
🙂

[smoky igloo]

Yea and in the beginning I thought why do I have to do this ...
But after I was trying git as VCS and saw that it will just fail with such big files, I understand that it's a different thing.

[smoky igloo]

But you have to be aware of the files you are changing.

*kind of

[arctic hornet]

indeed... but that's also a benefit if you look at it from the big picture

because you have to tell server that you're working on the file, server knows it... and so do other people working on the project

which limits the number of collisions since binary files have exclusive locks (only one person can work on the file at the same time)

[smoky igloo]

Yea thats one thing thats not even relevant to me, that it also handles that.

in git you have the merge conflicts, but for binary files this won't work i guess 😅

But I'm getting used to it, and just thinking about adding a few mega scans assets, git will 100% fail, you can buy LFS but even then and all the workarounds you have to do for it.

[arctic hornet]

yeah, another large benefit (and another reason why it's used by large game studios) is that you have much better permission system... in git if you have access to the git repository, you see it all

in perforce, you can have per user and per group permissions, which can limit stuff up to the scope of single files

so you can say, "this is read only for some", "this is something half the people don't even see" etc. etc.

[smoky igloo]

Thats quite interesting, the different needs and what it can do.

But knowing such things also makes you appreciate the tool more

[arctic hornet]

indeed, there was never a need for such things in git, since git was developed for linux kernel, which is open-source... so you mostly handle text files and don't have to care about hiding stuff etc...

[smoky igloo]

yea also I feel like, thinking about software, you would always have a module/package/..., which could be isolated and it's own git repository and you would have to have access to the whole thing anyway.

and then you get permissions to a repo and changes will be checked via a PullRequest.

just interesting seeing the different approaches emerging from different needs

[arctic hornet]

indeed.

[valid moat]

What is a good easy source control solution for one dev and a laptop?

[pulsar parcel]

git or perforce. perforce requires you to setup your own server though

[valid moat]

I have used perforce before but it was in no way easy 😔

[pulsar parcel]

what parts were you struggling with

[valid moat]

setting it up and everything, i don't want to do it again

[wide raven]

2nd time should be a bit easier now that you know your way around a bit... perforce is absolutely the best for unreal i reckon

[valid moat]

it was like 6+ years ago. I hoped there would be simpler solutions by now

[woven sluice]

the easiest solution is the one you learn how to use - yes there is a learning curve to this, yes you need to deal with it

other than git/perforce, maybe you'd like to try diversion, but it's a startup that works entirely differently and might have its own caveats (i have never used diversion - and i am not endorsing any solution here - every solution sucks in some way or another)

[cyan jay]

You could also try Helix Cloud if you're prepared to pay

[ornate matrix]

i've been looking around how DDC cloud works
is there any known provider that supports this ? i guess perforce do ...

[arctic hornet]

If you mean Unreal Cloud DDC… then that’s entirely separate thing that’s not connected to source control in any way… you just set it up somewhere, connect to it and start using it as last layer of your ddc chain

[ornate matrix]

Is there any git comand to clone/pull only specific folders/files ? (A sort of local gitignore)

[modern tulip]

You can git clone with the --filter=<spec> flag to get a partial clone. See the possible specs in the filter section of the git rev-list manual: https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt
It sounds like you probably want to combine it with the sparse checkout feature and pass a --filter=sparse:oid=<blob-ish> parameter. This means you will need to have a sparse-checkout file committed first (similar syntax to gitignore), you can find more info in the sparse-checkout manual: https://git-scm.com/docs/git-sparse-checkout

[ornate matrix]

Thanks

[long hull]

Ari's Git rant: https://twitter.com/flassari/status/1863621606980927887

I'll have to heavily disagree. It's like back in those days, when people using SVN were complaining that Git is bad because they were just trying to copy/paste their SVN workflow onto Git. It's the same with Perforce, if you use your Perforce workflow with Git, you're going to have a bad time. But somehow, the whole dev world (except gaming) manages to use Git just fine, and all the points raised by Ari are very common dev problem, and not gaming specific in any way.

So it's just the matter of using the tool in the way it's meant to be used, and not being stuck in another tool's workflow.

Ari Arnbjörnsson (@flassari) on X

Developers sometimes ask me if they should use Git with Unreal Engine projects. This is my response to that, I give you my "Git rant" 😅 #uetips

This is a short clip from "Lessons Learned From A Year of Unreal Engine AAA Development": https://t.co/TPCKpQPZ1W

Twitter · Dec 2, 2024, 4:29 PM

[pulsar parcel]

on the other hand, I'm very aligned with what Ari says in this and the Epic way presentation

and I'm currently using Git at work and hoping and praying that they find a way to make the P4 switch

[arctic hornet]

I'd generally agree with Ari and sswires... git isn't really suited for the expected way you do unreal based development... there has been so many times outside game dev where I was forced to merge master into master even when the two commits were completely distinct etc... and that's something that really annoys me... obviously if you make different workflow that is more suited for git, then feel free to use git... but honestly, I feel that git isn't suitable for Unreal in general, even if we gloss over the workflow issues... you have to set up and configure LFS, and even then with large number of changes and billions of files git gets super slow... another thing could be cross depot merging (in git case cross repo merging) (i.e. if you have engine repo and project repo) in perforce, p4 copy or p4 integrate and bam done, in git... not sure how to approach that honestly (speaking about raw git, not github etc.)
and there's ton of other things... but I will also say, that I'm kinda biased as I've been using perforce for 10+ years, so I kinda consider what perforce does a must have at this point.

[modern tulip]

In git I think the equivalent of cross depot merging would be to set up a subtree (not to be confused with submodule) in one repo that is based on another repo (for example project GameProj will have the dir /Engine as a subtree, pointing at the root of the UnrealEngine repo). You can then commit changes to files inside and outside of /Engine as part of GameProj (unlike submodules) and when you're ready, tell git to push from the subtree /Engine to UnrealEngine.
But I agree - it's not meant for such projects, it will be slow and non-trivial to set up

[arctic hornet]

Thanks for explaining, I didn't know such a thing even existed, I've always used submodules at most... I'll have to check that out.

[long hull]

I feel that git isn't suitable for Unreal in general
I fully agree with that, but it's very much by design: Unreal was made with and force Perforce. So it's a bit rich having Epic say "Git isn't well suited", when they made exactly no effort whatsoever to maybe somehow investigate the possibility of keeping in mind that Perforce isn't the only VCS out there.

Epic could make Unreal work fine with Git, they just decided not too. But blaming Git for that? It's like blaming Unreal if your game has poor performance and you've never opened Insights once.

It goes back to my usual rant: if Epic had any stake in Perforce, I would totally understand their decision. But at the moment, it seems they're putting all their eggs in someone else's basket.

[arctic hornet]

A) Epic has been using perforce since March 2000 - way before git was a thing
B) Git isn't suited for unreal by it's design, not by unreal's design... git doesn't handle binary files super well as git was made for linux kernel - mostly text files...
C) I've had the opurtunity to see under the hood of many of the "AAA" game engines and Unreal is nothing unusual in it's design... ton of engines rely on mostly binary formats, be it for legacy reasons or just simply for the fact that it unifies (de)serialization etc etc

So I wouldn't say it's Epics fault git doesn't wor super well with unreal by default... yes... they could put more work into the git integration... but epic doesn't do anything that doesn't make it money... or... epic doesn't do anything that doesn't benefit them

Epic internally uses perforce, all their large partners and other studios also use perforce... so it makes sense for epic to focus on perforce

that's the same reason why all epic's tooling has existing integrations for AWS, Slack and Jira... and nothing else works or is maintained

epic uses it, so they maintain it

What I've been told by epic developers directly: "If someone wants something, they can develop it and maintain it.... they can also try to merge it into mainline unreal... the caveat thre is, we will only merge what we can and will maintain, as anything that is merged becomes our responsibility"

[woven sluice]

can and will maintain
hot reload has entered the chat

[arctic hornet]

Hot reload is used heavily at epic. and it works... no hot reload in C++ land will ever be perfect, especially not in such complex codebase as Unreal.

[merry verge]

It isn't though, live coding is. And it still has massive asset-corrupting issues if you're not careful.

Hot reload is the old even more broken system.

[arctic hornet]

And epic has stated that it's designed more for small-scale itteration to function bodies rather than adding/removing full new classes.

ah... my bad then

[woven sluice]

i did it, i successfully derailed the chat 🥳

[arctic hornet]

👏 congrats

[merry verge]

Anyway, git's intended lightweight branching workflow is inherently bad for any project where you have a lot of assets that aren't mergable (i.e. binary files, so all unreal and source assets). It's usable, but it's the wrong tool for the job.

[hoary wren]

Do folks typically check their Developers folder into source control? I was reading https://dev.epicgames.com/documentation/en-us/unreal-engine/developers-folder-in-unreal-engine?application_version=5.4 and it seems like the answer is most likely that you want that content in VCS, but you just want to make sure not to package that in your build to avoid wasting space and compute on something you don't be using in the final build anyway?

[silver token]

There's really no reason to exclude them from source control. It's an easy way to share experimental assets between developers and it makes it easier to keep them up to date with other changes like asset moves that other team members might make.
As long as they're not packaged (and you're setup to prevent "real" game assets from referencing stuff from the Developer's Directories) it doesn't really matter if they're in source control.

[hoary wren]

Awesome, thank you. Any chance it's possible to change the Developer's individual folder name rather than using whatever the OS account name is? I haven't found an answer on the forums yet, seems like the answer is no short of modifying the engine.

Unrelated, but is it possible to inspect a packaged game to verify which maps got bundled with it, to sanity check that you didn't accidentally include a bunch of stuff you never wanted in there?

[pulsar parcel]

I guess Epic's assumption is you're always connected on a Windows machine that's joined to a domain huh

[silver token]

I don't know if you can change how the individual name is made, but I do know you can just make folders on your own just like any other directory in the content browser.
So you could make one and move stuff if it really matters. If you renamed the folder it would probably remake the default

[arctic hornet]

Not strictly a domain, but just username being firstname.lastname … sadly it’s the only fully consistent way of uniquely identifying users

Can’t rely on source control username because it’s not always connected etc

[ornate matrix]

when you use a installed build for your project, do you stop tracking your .uproject because of the engine association key

[arctic hornet]

Hard to say what exactly are you asking for as its not written as a question… but… You should always track your uproject file, the engine association for builds from EGS is just a version number… and if you’re using custom engine I recommend leaving that variable empty and just have engine located next to it as epic recommends

[ornate matrix]

idk if a installed build works well if a project is placed inside, will try

if i place the project inside MySourceBuild/Engine/LocalBuilds/Windows/ idk if it will be overwritten when rebuilding

[wide raven]

My method is to keep it tracked, but move the local change to a named CL. Every now and then, if .uproject changes, will need a resolve.

[ornate matrix]

what do you mean with a named CL

[wide raven]

When you check something out, it goes to default. Do "Move to another changelist"

[ornate matrix]

im not using perforce

[wide raven]

not sure if git has an equivalent

[ornate matrix]

should the Intermediate folder in a installed build be shared to team members ?

from my knowledge it shouldnt, but im unsure if installed builds have different behaviors

[wide raven]

no

I am not running installed builds at the moment but when i was, there was no differences to normal operations that i remember

[midnight goblet]

Hi all. Is source control on a Helix Core instance enough to enable multi-edit mode?

[arctic hornet]

what do you mean "multi-edit"? like One File per Actor or something else?

[midnight goblet]

I mean this.
https://interactiveimmersive.io/blog/unreal-engine/collaborative-world-building-in-unreal-engine-with-multi-user-editing/.

But I would be open to hearing about your one file per actor suggestion too.

[arctic hornet]

ofpa is intended for easier collaboration on large worlds by multiple people at once by storing each actor in separate file so only specific actors get locked meanwhile in traditional aproach whole level gets locked... what I believe you're looking for is some form of realtime collaboration based on the things you've sent?

but neither OFPA nor Multi-user editing is tied to one specific version control system, although from personal experience, perforce is best suited for use with unreal simply because it's what epic uses, so it's most polished

so yes, multi-user editing will work with perforce (helix core)