#source-control
1 messages · Page 4 of 1
And submit it.
Then they´dneed to log it somewher, so that I can review that, if need be sort things into correct folders etc
And then migrate from the temp library into the master library.
Which leaves way more admin duties to me than I´d like to, but I just don´t have teh expertise to build a system thats "fool proof" and automatically sorts assets into correct folders and makes sure nothing breaks, lol.
The other problem though is, how do I layout the rest of our repository?
Lets say we have 100 projects named "2001", "2002", "2003" etc.
they are all separate projects without any interconnections.
So, do I now have 100 separate depots?
It comes down to how much you trust your artists, really.
Not at all, lol.
If you think they're going to check in shit work that needs to be reviewed and checked constantly, it's going to be hell.
Or if they are going to fail to understnad where to place assets etc
I mean, on our current server, working with 3ds max etc, we have ONE folder per year and then subfolders for each project.
And at the end of the year, we archive the whole folder.
We don´t need versioning history for any of our project, once they are archived.
So I´d just back up the local workspace folders manually on our big NAS and delete the depots in perforce.
I´m hoping that after a certain educational period they´ll start getting used to it. We usually don´t have a lot of exchange inside projects, most projects are self contained and are being done by ONE artist.
Its when people get sick or we have bigger projects when things tend to get messy.
And Unreal kind forces you to be a bit more rigid in how you use your files.
You can´t simply copy and paste shit in the explorer and expect things to work (which they often also don´t do in other DCCS).
Did I miunderstand the conversation that happened before, or did you say that git LFS keeps versions of files across all branches? so you can't have different LFS files in different branches?
(i was just scrolling up and reading random chats xD)
Thank you for your reply. I'm quite new to this. Woukd you mind telling me the benefits of using git over the zip? You are referring to gitbash correct?
well, any form of git, not just git bash. then you have actual source control. you can make changes and share them, have a log of them, be able to control them, incrementally download new changes by others, etc.
So, what you're saying it, you have a lot more control... of your source!
mcoms outlines the reasons why source control in general is awesome. It's an invaluable tool!
Okay thank you. I take it the changes will appear on the website?
so, earlier in the day, my game wouldn't boot up (picture 1), so I followed the advice of someone else and verified the engine files. After it gave me the error as shown in picture two and when clicking the error message i get told that there are corrupt game files. How can I fix this without losing my two months of progress?
I was kinda being facetious when I suggested Source Control - it will help the next time this happens.
But like, Install Failed? Have you clicked the links to see what the error codes mean?
yep, this is what it shows
and the tutorial it shows doesn't work
Copy your assets out of your content directory. I assume it's all BPs?
Just back them up somewhere.
And then delete them from your content folder, see if it runs.
what?
sorry, im new to unreal, and building a game off of tutorials
so i dont know much
Go to your project folder.
It will have a directory called "Content"
That's where all your stuff is.
Copy and paste it somewhere.
Can even just copy+paste it to your project fodler and call it "Content2" or whatever.
Alright.
that will have my whole game, including the maps?
Yes
Now delete things (the .uasset files) in the original content folder. Like all of them.
cause i CANT lose this game
And try to open your project again.
k, their still copying
As long as your copypasta fu isn't awful, your game isn't going anywhere.
lol
You can either delete assets 1 at a time until it loads.
Or delete all of them to see if it loads and then add them back in from your backup 1 at a time.
okay
Until you find the issue.
ill try the deleting and restoring one at a time
This is the issue source control would fix - you would have a backup of every file from, say, yesterday.
And you could just restore that state.
i dont but okay
I never thought to make backups, which now i realize is very stupid
You can literally just zip your content folder up every week if you want, not as effective, but avoids catastrophic failure.
yea
imma delete everything in the content folder, then reboot the unreal engine to see if its a system problem
gotta repair the engine itself first tho
If it doesn't boot with nothing in your content directory, you have far more serious problems.
and those might be?
same error, nothing in content directory of the game
i moved the backup of the content folder somewhere else in case that was the problem
and i still got the same error
Try rebooting
okay
okay im rebooting the launcher
still wants me to repair it so i clicked repair
and i got the same error
You should probably do that.
Or actually remove it and add it again.
That will take ages, but yeah.
okay
yeah ive clicked repair 4 times and i sends that error every time
okay its uninstalling
and i dont really care how long it takes, as long as the game gets back online
@teal bone i got the same error and it stopped the re-install. What should I do?
nope
imma try again, and if it dont work imma remove all the files that are there
it may not be the assets that are the problem
I think that's definitely the case.
okay
i removed all the vast ruin files and it still gave me the error
should i just try uninstalling the launcher itself?
No idea sorry mate.
Oh?
i uninstalled the launcher and everything
and i couldnt delete this one folder
this is in it and this is the error i get
Ha
now if i can get it off my computer that would be great
So that plugin from right back at the start. 🙂
yep
it was in the main engine, not my game
now if i can get it off my computer, that would be great
rename EpicGame to DIEPROFILEWIZARDDIE and be done with it.
i dont particularly want it corrupting my whole pc
Your pc is already corrupted. It might be a sign your hard disk is failing.
I would do a chkdsk
idk what that is but okay
also, whats this do
okay...
and it deletes corrupted files?
ah
okay
shouldn't have
my computer works fine
Hard disks degrade over time.
well that i know
When they start failing, they automatically hide the broken bits.
interesting
When they run out of ways to do that, it's already pretty fucked.
why is that?
Because you have a billion billion bits on your hard drive and a few million of them failing isn't a problem.
what the hell does this mean
okie ill be back in a sec
"sec" 😂
It's giving me loady wheel
And its checking the disk
(the cords are to a second monitor)
K logging back in
So now I should be able to delete it, correct?
YES IT DELETED
I just did shift+delete so I don't have to go through the trash bin
Okie imma go reinstall unreal engine
I'll be back later!
Good luck.
lol this has been running for 3 hours now 🤣
nah its this. Its done now, im just waiting for quixel to finish before i launch it
cross your fingers
It works!
Hello, I am having a really hard time with Plastic SCM
not a whole lot to go on there sarge
Maybe they just want a bit of sympathy and some moral support
Cheerleader squad, assemble
Hello everyone. I was hoping I could get some help with an issue regarding git, git lfs, and project boot times that are way too slow.
Setup:
UE5 project (hosted on a different drive than the UE5 engine)
git installed
git lfs installed
git init run in project repo
Initial commit made (all files in repo committed)
suggested .gitignore added to repo base (attached)
suggested .gitattributes added to repo base (attached)
git source control enabled in project
Issue:
Every time I open the project, it gets to 91% and then spends >40 seconds flashing a message about adding to source control. (gif attached)
The source control log shows a ton of errors related to the fact that I have ignored starter content. (file attached)
The source control log also shows fatal errors related to Engine Content being outside the repository (file attached)
The errors are annoying but the startup time is what kills me.
Any help getting rid of that 40 second constant flashing would be very appreciated. Happy to provide any extra details needed.
Apologies if there is a better way to display the pictures so it is not so obnoxious spacewise, maybe I should have made a collage....
I also looked in the history of this channel and found some similar error messages but they seemed tangentially related to note having pushed content, which I'm not sure is an issue here.
I appreciate the response. 🙂 That's what I'm doing now. 😦 Separate command-line window always open. Just a bit sad I can't revert in engine.
Oh, I'm much more comfortable with command line. More-so, I meant that it's nice not to have to close the engine if I need to revert a change. SourceTree is definitely a good suggestion though. 🙂
Yeah, that's why it's a charitable suggestion. I'll admit that I'm less familiar with source control interfaces on Windows, so your response was a useful one. 🙂
@worn pebble use the new git plugin
Is it safe to use the same workspace for different perforce streams?
When talking about StateBranches, does anyone have some insights on the CI pipeline required to auto merge stuff from, e.g., main into dev?
// Order matters. Lower values are lower in the hierarchy, i.e., changes from higher branches get automatically merged down. // (Automatic merging requires an appropriately configured CI pipeline) // With this paradigm, the higher the branch is, the stabler it is, and has changes manually promoted up.
(using GitLab, but I assume that the CI part is just generic Git stuff I can add there)
I saw that I would need to have an SSH Connection to the Runner so it can execute the Merge via the CI/CD stuff
There were some SO questions about it
Haven#t looked into it yet though
All I#m working with atm is this
And to a very simple extend
Something something
Not sure what trains are yet
(bob is the build machine. don't judge)
Alright, will keep that in mind when tackling it
why does it lead to broken branches
how would this be relevant in this status branch context, where branches don't fork and it's just a straight line of branches?
but i don't understand how it would be possible for merge related breakages to occur if it's just merges from release->main->dev
because you merge up manually the other way?
again, this is status branches we're talking about
what do you mean by "missing some changes"
but people work on release, they work on main, they work on dev
i feel like maybe you don't know about the status branch workflow? it was shared by epic regarding how they work on fortnite
safe? yes. practical? maybe not. if the difference between the two streams is very minimal then changing between them frequently might not be a nuisance. if there are many GB's of data different between them then each switch is going to take ages.
Thanks for the clarification and we don't intend to merge these streams.
not merging
when you switch a workspace from one to the other, the physical files on your disk have to change
your workspace has to match the new stream
Makes sense and I don't think there will be frequent switching. 🙂
is there any way to find a word in a list of revisions in P4?
I removed something I shouldnt (a struct declaration) but it exists on previous revisions.. though not sure which file
gotta find and copy paste it back to an another file
one other thing: before switching workspaces, you have to have a clean workspace with nothing in any change lists. p4v won't let you switch if it's going to destroy pending work
I learned that the hard way but it was only a small change in one file 😅
hi, can anyone show how to set P4ROOT using p4dctl? (digital ocean, perforce)
I tried to put P4ROOT like this
#-------------------------------------------------------------------------------
# Environment variables common to all servers
#-------------------------------------------------------------------------------
Environment
{
P4DEBUG = "server=3" # Embedded = requires quotes
P4LOG = log
P4CONFIG = .p4config
P4ROOT = /mnt/volume_lon1_01 /perforceserver/root # ADDED THIS LINE
}```
but it does not work 😦
I am basically trying to move the p4root to volume i purchased.
Also treid restarting the services
p4dctl restart -a
that root looks very wrong
what's with the space?
there is no space
/mnt/volume_lon1_01 /perforceserver/root is a typo?
yes
next, why are you setting the root in your global environment section? It's specific to each p4d instance
generally p4dctl has a config file per-server in p4dctl.conf.d/<server name>.conf
and it'll have something like this in it:
p4d servername
{
Owner = perforce
some other options
Environment
{
P4ROOT = /path/to/root
some other options
}
}
ok knew that was wrong.
p4d perforceservice
{
Owner = perforce
Execute = /opt/perforce/sbin/p4d
Environment
{
P4ROOT = /mnt/volume_lon1_01/perforceserver/root
P4JOURNAL = journal
P4PORT = ssl:{ip}1666
P4SSLDIR = ssl
PATH = /bin:/usr/bin:/usr/local/bin:/opt/perforce/sbin
}
}
is this right?
that looks correct, yes
p4dctl restart -a
error
p4dctl error:
Service 'perforceservice' p4d: Directory '/mnt/volume_lon1_01/perforceserver/root' does not appear to be writable.
p4dctl error:
Configuration for "perforceservice" p4d has a conflicting parameter "name".
Stopped 0 services.
p4dctl error:
Service 'perforceservice' p4d: Directory '/mnt/volume_lon1_01/perforceserver/root' does not appear to be writable.
p4dctl error:
Configuration for "perforceservice" p4d has a conflicting parameter "name".
Started 0 services.
p4dctl error:
Not all services started successfully.
will this work?
sudo chmod +rwx directoryname
@merry verge
tried this
sudo chmod ugo+rwx foldername
and the directory is now readable but still getting error about the name
Configuration for "perforceservice" p4d has a conflicting parameter "name".
Thanks for your help @merry verge were nearly there!
You have defined "perforceservice" twice in your p4dctl config file
perforceserver is not in p4dctl
there was just the global environment block.
and bunch of commented example blocks.
I am pretty sure perforceserver is there only once.
include /etc/perforce/p4dctl.conf.d
aaah
yes
i guess .d is default file
should I be modifying this?
include /etc/perforce/p4dctl.d (includes *.conf files from the directory)
ok
@iron bronze thank you very much
Yes the .d suffix in linux environments indicates a folder
It's a good habit to put your own .conf files in there so anytime you update your server you won't have to resolve the default configuration file (that might get changed by the package manager)
Thanks a lot for this suggestion! I'll check it out. 🙂
Anyone?
I tried p4 grep but it searches for files instead
P4v Time-lapse view is good for this.
thanks i'll take a look
hi! how to clean server space in perforce and remove old revisions or clean files history ?
e.g. I have revisions 1-1000 and I want to keep only 100 last and keep at least latest possible file .
p.s. this one https://i.imgur.com/8omyJg4.png
is not working this way because it will erase all revisions and if at if any file was not changed in last 100 commits - it will gone and project will become incomplete.
expected result - clean up server space and have each file from depo residing in before this clean up.
rider added a plugin to my project under the Developer I wonder if I can't move it to the engine so that it doesn't pollute our git
FileSystemMappings.ini keeps getting changed every time a different user opens the project which creates an extra modification to commit every time as well
(and it shows the installation path of Unreal as well as the project path)
The traditional way to do this is to set the filetype of the files. You can edit the typemap to automatically apply the same settings to any new files
This will indeed limit your ability to go back in time so I wouldn't do it if your project is mature, but it can be handy during alpha/beta dev in my opinion
(Perforce, DigitalOcean)
Hi, I am trying to set the P4ROOT to volume.
- edited
/etc/perforce/p4dctl.conf.d/perforceservice.conffromP4ROOT = home/perforceserver/roottoP4ROOT = mnt/volume/perforceserver/root - copied the contents from original directory to new directory.
but when I do step 1 I get this error:
'perforceservice' p4d: '/opt/perforce/sbin/p4d -p ssl:1666' exited with status 255.
in the docs it is mentioned that the error is shown when there is issue with privilage, ownership or configuration.
I checked the privilage and ownership, using
-ls -l root for both new and old, and compared it is same.
so can anyone point me in the right direction. thanks in advance.
that directory and everything under it should be owned by the user as specified in your .conf file
also, you can manually run p4d for that server to see what the specific error is
su perforce
cd /path/to/the/server/directory
/opt/perforce/sbin/p4d -p ssl:1666
ya 'perforce' is the owner . it was root previously but changed it to perforce
how can i see the logs? if that will help.
running p4d manually should log things out to the console
ahh ok
and hit ctrl+c to exit (if it doesn't exit on its own due to an error)
i'll try
/bin:/usr/bin:/usr/local/bin:/opt/perforce/bin:/opt/perforce/sbin
this is the path right?
I am not sure what the path to the server is.
this is the directory I want the P4ROOT to be set.
so I use this?
yes
thanks
cd to that directory when you're the perforce user, run p4d, see what error you get
open for write: ../journals/journal: No such file or directory
wait a second to i have to copy from home/perforceserver/
rather than just home/perforceserver/root?
I have no idea what your filesystem is like
hmm
but you need all files from the server, and yes there are generally files above the server root directory
perforce@UTMserver:/home/perforceserver$ ls
archives journals logs root
ok i am moving archive, journals, and logs as well
Listen 1666 failed.
P4SSLDIR not defined or does not reference a valid directory.
the directory ssl is there, there are two files privatekey.txt and certificate.txt in it.
P4SSLDIR = ssl
Yes, but those are when running under p4dctl. Set that env var manually when testing yourself.
Or just see if p4dctl works now.
ok
thanks for your time siliex.
it worked.
sorry, dont got you here.
I am asking about how to keep e.g. only latest revision per each existing file in ue project.
I have a file history like that https://i.imgur.com/x3yfO1u.png per each changed file in the project and I 100 percent sure I dont need file history for 1 or 2 years ago.
e.g. I want to keep only latest one, or few latest ones for specific revision or date etc. but this tool https://i.imgur.com/8omyJg4.png is not working properly.
right click on a file, press change filetype. google perforce docs about it for more info
its impossible to change it per file. Its thousands of them! will dig it more )
Is anyone successfully using git lfs with a largish repository? Something in the 100-500gb range.
in total history or repo footprint?
My git repo is ~500GB, but it's all spread across submodule, so no individual repo is >20GB. Not sure if it helps...
Hi, I'm not so sure where to ask this, I can't seem to find the appropriate channel, so I'll ask here.
So, regarding project repositories, I'm currently using TortoiseSVN and at very rare occasions it just breaks and I need to download the whole project again.
So my question is... is there a way I can let the SVN know that I already have the project files downloaded? so that I don't download it again?
(I'm referring to the hidden .svn folder).
If you know the answer, please please @white bone Tag me.
Repo footprint. Trying to figure out how viable it is compared to perforce/plastic for a large project
i dont see any reason why it wouldnt be able to scale
when i moved my project to 5.1, every time i open it now, source control takes like 40 seconds to specifically restore my StarterContent folder. does anyone know why this is happening and how to make it stop? 😭
remove
[StartupActions]
bAddPacks=True
InsertPack=(PackSource="StarterContent.upack",PackName="StarterContent")
from DefaultGane.ini
Does anyone know what this error means?
Well it's a platform not supported exception. So what OS are you running? And what git tools have you installed?
Actually it's a dot net issue.... Strange. What os? And what versions of dot net do you have installed?
yeah, i don't quite understand it, cause setup.bat supposed to download the dependences, not sure why it is complaining about platform
@vale basin Windows 11
i didn't install the DotNet, i believe it install when i installed the Unreal Launcher
git is just the git bash 2.39.0.2 64bit
I try to remove everything from my OS, from VS 2022 and all the packages and reinstall everything, still getting the same error
all these also installed
VS 2022 v17.4.3
there is this error as well
when i try to GenerateProjectFiles.bat
DotNet v6.0.302
thank you so much!
is there anything better storage wise than droplets if i want to keep perforce server costs under like $20/month
@haughty yoke a droplet is their marketing term for a vritual machine (running on shared hardware)
waht you're interested in is block storage: https://www.digitalocean.com/products/block-storage
if cloud hosting is too expensive, run the server off your PC and use port forwarding on your router
Contabo Storage VPS is one of the best value options I found.
i would but my home internet is still on DSL
max down/up for my home is 300kb/s / 45kb/s
fiber lines have been laid to the area but ISP has not told us that we can order it yet
i have a nice pc with an 8tb HDD that would be great for running a remote server as its already up 24/7
its other hardware is crap
but its dual core 1.3ghz, and 12gb ram
Hey, please @white bone tag me if you know the answer to this.
Is there a way to use Totoise SVN with Google Drive as a repository?
Probably not.
Maybe you can mount a Google drive on a linux vps but that sounds like it might just be a good way to have a frustratingly slow version control system
Careful. We have to tread the fine line between making this sound bad enough that they know they shouldn't do it, but not bad enough that they decide not to try it
You can use azure Dev ops
Free
And no size limits
Just make sure you set up Git lfs
Yea I meant it more as an alternative, but true the ultimate cost reduction from something paid is something free 😄
it is
it's not free past 5 people, and you have to pay for the initial 5 at that point
That's false
There's no limit on data as you provide your own server and storage
20 workspace limit is true though.
If its 20, theres no way i will hit the limit
And the rig i plan to host on has 8tb
Don't forget that because the server keeps the version history for each file you might end up hitting that limit sooner than you think
Only $120 for each additional 8tb
It does it for all graphics as well?
Ye you can set it per-file though. Maybe you don't want to keep all revisions of specific assets but only the last 2 versions or something like that
Should be fine. Perforce isn't compute heavy
I'm new to using source control and github. I just created a repo on my PC and pulled it onto my laptop. I get this error when I try to open the project:
Can anybody help?
Happened after enabling LFS
Anyone knows of a good P4 tutorial that clearly shows how to manage updates of the source with local changes?
Do you have the same version of the engine insatlled on both?
Hey, i plan to work with a team on perforce, i need a easy use onlmine service for perforce (i need 500gb storage) anybody know a service like that?
Assembla is the only managed hosting provider for perforce and it is not cheap.
would be much cheaper to buy a NUC or something with a 1tb ssd
Cheapy second hand office thinkstation or dell from eBay, Craigslist, fb marketplace. Good value for your first server!
Has anyone set up perforce core on an EC2 instance? I don’t have extra hardware laying around for a server
Of course, you can run perforce on pretty much any VM.
just make sure the VM is running on top of an x86 CPU, I don't think there's an official ARM build for p4
anyone else have an issue with uassets not being part of the project (files are still there) after pulling down from git?
the project works fine on the computer I originally worked on - it's like the level/map didn't save also
oh, nevermind, I opened the wrong project 🤣
carry on
@Forgemaster_Alex#4890 🤣🤣
Hey! I'm looking to redux my workflow with source control as I've always been a bit unorganized. Do ya'll generally keep art assets away from engine assets? Or do you put files like your blender/maya project files in source control next to your UE projects? Opinions on git or perforce welcome, I have projects in both.
Depends on the size of the team. I've heard of larger teams having a repo dedicated to source assets. Smaller teams i've worked on just have a folder in the same repo dedicated to source assets
this is in P4 btw, so you can just strip out source assets from a workspace view, so people that don't need them don't see them
That's interesting, right now I just have a massive NAS sync folder that everything lives in but I want to seperate everything out into their own project disciplines.
I prefer git for code, but P4 is great for uasset file locking. I'm just afraid that .blend files are going to baloon a git repository pretty fast with LFS
I think I like your idea of having a repo specialized for art assets
it's probably the more scalable option to have separate repo
For perforce I don't see a reason to use separate depots for different source assets. Just throw things in another folder.
Makes it easy to see when someone is making changes across different types of things.
Git may be another story depending on your setup.
yep and don't have sync issues with the 2 repos being at different version levels
But i think from juggling storage and future maintenance it may be a bit easier to have separate repos. Really depends on your team, workflows and sysadmin skills
Git is amazing if you have your own LFS file server. If you're hosting on something like Github, you'll be paying for cloud storage pretty quick.
I disagree with it being "amazing" for the purposes of gamedev, but that's not what I was talking about.
This is my opinion. I want to cause as little work for future me as possible
The least amount of work for a solo dev is almost always going to be a single repo. Just be organized in how you store things.
If you have multiple interrelated repos it's just going to be annoying to manage. You don't get any benefits of splitting things up that way.
I mean same with p4 and assembla. That's just an on-prem vs cloud and managed services discussion. Not really relevant to where source assets should go
We're a small team, but I've been using unreal engine since you had to pay monthly to use it in the 4.0 days, so that's a lot of baggage to maintain
I'm not sure how that's a lot of baggage to maintain.
Unless you're specifically complaining about people having to download assets they don't need - which is solvable depending on what source control you're using.
Perforce has virtual streams and workspace views. Don't need art source? Don't map those folders to your client.
Git is more complicated.
Larger studios generally work in a single repo (also see UnrealGameSync for working with perforce - it helps with this and more).
Does Git+LFS support "views" so you can ignore subfolders? That might make single repo tricky. That's a great feature of P4 so you can narrow scope of repo's locally for people
What I mean by baggage, and I meant potential baggage in a repo, is 8 years of blend file changes that would have compounded in repository size. I'm basically trying to find a solution that lets me do version control while leaving myself open to managing space requirements down the line.
Separating out assets wouldn't help with that
In perforce you can set filetypes to restrict the number of revisions on specific types of files (or even just in certain folders)
Git I don't have a good solution for you but a separate repo doesn't help either.
This is true, I think I have that setup for binary builds of projects.
no reason I couldn't set that up for art assets
hmmmmm
an 8 year running project sounds problematic. If you have a single repo with MULTIPLE projects, i'd separate those out. If otherwise your project is really bloating over that, you might consider something like starting a new repo when you do an engine change or something, somewhere sane to clip history and leave the old repo online for a while
each project has its own repo. The art assets for all projects are laid out in a massive folder stored on a NAS
Ok, that's a whole different story
if those assets are in a shared library (sounds like the are), seperate repo sounds good. If they were highly project specific, with the project sounds better
I would store source assets with the projects themselves. And maybe a "shared" depot mapped into client views if you want to share things across projects.
And use file types if you want.
I'm currently separating everything out to their own projects that they are a part of and I'm looking to do version control of some type to facilitate non-destructive changes
Thanks for the feedback, sorry I wasn't clear lol.
I think the solution for us is to have a dedicated art depot in P4 with each project seperated out for version control.
That'll let us trim out old revisions when space fills up.
(again talking only art assets that are attributed to specific projects, and a depot for art assets that span multiple)
does git and github work well with ue5?
hello, any solutions for this problem?:
Question on perforce - say my entire project is 10GB
If I make a change and push the changes into my main stream, does it make a copy and now my project stores 20GB (because it saves history of project?) or can someone explain to me how that works
It saves the full history of each file individually, as any individual files are pushed/updated
there's compression + deduplication so it wont literally double the size
Use LFS storage for binary assets. Github works, and it works pretty well, at least for small projects. You'll run out of space pretty quick, though if you continue to edit large assets (Which games tend to have)
thanks! it will be medium sized, LFS is also on github?
Yea! If you go to your profile settings, billing, and scroll to the bottom, you can see how much LFS data you have on Github. I have a private repository on my home NAS so I have as much LFS storage as I want, but with Github, it's like any other cloud storage provider.
'Codespace' is where things go that are not on LFS, which is much more limited because it's meant for.... code lol, classic git repository situation
big thanks
npnp
Hey guys how can I "obliterate" a submit: in P4V
I tried different variant of p4 obliterate...
ask chatgpt and it worked... for those wanting to do that:
p4 obliterate @revisionNumber
I just set up a repo using GitHub that contains three projects for our dev team to work on. As of now, we are using simple good ole github desktop to work with everything. Is there a better way to do this that allows developers to check out/in assets? I have tried to use the built-in function from UE5 but it does not seem to do much at all, any help would be extremely appreciated as there is barely anything about UE source control online
I would personally advise one repo per project. But from what i hear the gitlfs plugin for UE by mcoms(sp?) is much improved over the default one
The project is divided into three separate parts which are technically separate projects
Beta, Design, and Main
For this system, would you still recommend one repo per project? We chose to use one single repo so that we can move finished assets from the two predeccesor projects (Design and Beta) to main within one single repo
that's an interesting workflow.... Can't really offer advice on that one
Yeah I know its a bit unorthodox, yet we found it to work best for the team we have currently and what were going for
what can cause this problem?(fixed - wrong mapping)
set P4ROOT maybe? or cd to your workspace
what exact command should I use then?
Easiest way to ensure environmental variables are set is to use P4V. Right click in the workspace view and choose Open command window here
I have a window like this, but I don't know about these commands at all
can use UEGitPlugin and PBSync
also id recommend migrating to a branch workflow, might be painful with the asset paths you have
curious how many ppl use classic depots vs. streaming depots in p4?
i would bet that 95+% of the people who know how to use streams depots use streams depots
how is locking binaries across streams handled? (when developing with vanilla versions of the engine)
you don't lock across streams
Which is the correct way to do things. Assets should not be modified concurrently across multiple streams that need to merge together.
But you often need to make modifications on a stream that does not merge back into your main branch (for example, a one-off bug fix on a release branch). Locking would prevent this and make it impossible to make patches on release branches.
In general, asset modification should only be done on your main branch or on branches that will no longer merge from main.
With an exception for new assets being created on development branches - they're new so they won't cause problems, and often new content requires work done on the development branch anyway.
it's worth mentioning that there is a way to code in stream names for the editor to display asset statuses from other streams. so if your team actually does multi-stream dev, if someone modifies an asset on any stream, someone working on the other stream will at least see a warning icon on the asset indicating that it's been modified elsewhere. but this doesn't change the ideal situation, which is what siliex said - mainline contains 100% of your forward-moving binaries, everything else is short lived
question. im using github and i was wondering if you can setup two repositories with one project. for example i have a private repository for my project that contains everything, then i want a public repository with a separate gitignore file so it doesnt push stuff i dont want others to have/see. is this possible?
not exactly, can do this instead each time you wanna publish to the public one https://stackoverflow.com/questions/1662753/export-subtree-in-git-with-history
I have a lot of annoying small issues with the Git Plugin lately.
- Files being Checked Out but not marked writeable, randomly failing to save and having to be marked writeable by hand.
- Files not being removed. They are removed from UE, but not from the Window File Explorer, as well as not listed for committing. It takes a restart and a second wave of deleting them to really get rid of them.
- Files not checking in when committing. I'm committing to a StatusBranch (dev) and some (or all?) files that were checked out stay checked out and have to be manually reverted (which often crashes the editor, cause UE loves reverting files).
Anyone else having issues like these? Is that a setup problem or is it just unstable?
I gave up on the Git plugin (before mcoms' fork), but yeah, it had quite a few quirks back then. Now I only enable it when I have no choice (e.g., managing shared collections), and I just use the CLI (which is what I do for my non-UE projects, so it's actually better for me).
Well it is ProjectBorealis one that I use :P
I'll take a look when I'm back from vacation!
Thanks, enjoy your vacation
Not to invalidate your experience by any means, but I don't think I've seen these issues before. In any case, the plugin needs to be robust and handle problems like this. So I will see what I can do
Yeah that's why I said maybe it's a setup issue on my end
If you could, please make issues on the GitHub regarding these and provide any more information about your setup that you may have
If not, I'll be able to make the issues when I find the time
Will see if I can find time. I just noticed the readonly problem seems to occur when moving an Asset and fixing up redirectors. The "new" asset is marked for add but not checked out and there is no option to check it out.
And it stays readonly fwiw
that's the only case, or is it possible you encountered this problem for other cases in the past?
ill assume marked for add assets are made read only at some point because they're not checked out for now
and see if i can solve the problem with some quick edits
@quaint obsidian I don't want to post "maybes", as I dislike getting Jiras like this, so I will only report things I noticed with a high consistency.
So this is the only thing I noticed in terms of repro steps.
It's def marked as "checked out" because I can't mark it as that anymore. Yet it's read only.
I did have cases (not sure if same repro) where it was marked for add but not checked out and I could resolve the saving/read only issue by checking them out.
But most of the time that option doesn't exist and I could only revert, which is of course also not ideal
okay will check it out, thanks, a git status --porcelain could also help to see what the exact status is in git for me to debug, if you're okay with sending to me
Sure, next time I encounter it I will try that
hello, i setup a perforce server and yes its really good to collaborate with my friend, but the login time out is way too short, after a bit of googling i couldn't find a setting to tweak, anyone know where this is set?
Hi, i wanna ask a questions. We are working with azure git repo but thats not the perfect tool for game development. Is it easy to setup perforce? And is there any storage limit while working perforce. For example we can not work with github because storage limit is not enough for game assets.
Default session time for me is 24 hours
24 hours is too short
If you want to switch to perforce be prepared to pay for hosting or host it yourself
not sure if its easy, that depends on you
Im not an expert on networking or administration 😦 i think azure is the best option for now, thank you
what sort of issues have you experienced with azure?
We couldn't connect it with unreal engine.
the git plugin is not working?
While working with github, it was okay but with azure we couldn't make it. I don't know it is about our knowledge or azure repo options.
You can change login expiration time in p4admin
hey folks i have a question ^^
im using git for version control and its pretty simple what i want.
my goal is to keep the folder Final_Game tracked and the other 3 Folders i need to work with excluded and i tried it inside the .gitattributes file with this commands:
but it wont work and its pretty hard to find something not totaly confusing online so can anyone explain me in short whats the prob and how i can achieve what i want ? thx alot 🙂
can you use .gitignore for those folders?
its inside my Content folder so im not sure its useful ?
alright worked like a charm thx ^^
Anyone using GitKraken ever encountered client not opening at all? I contacted their support (I have pro version) but got no response since a week
I will keep looking but I can't find it
Look in group permissions
Oh group, weirdly I hadn't looked there
How do you use source control to collaborate with others? I'm trying to work on a game with someone and I'm not sure how we can both make changes to the project at the same time. We're both new to Unreal and GitHub which is what we're using
Code is pretty easy and what Git was made for. You should do some googling and self-research, but the gist is communication. Work on separate things, and when someone makes changes you pull their changlist and 'merge' it into the branch it belongs. Once you satisfy any merge conflicts, you can then push the now-merged branch to github. You can also rebase someone's branch which is a more complicated topic, but its basically taking all of your changes and putting them after another's. Again, google and do some self research as code is what Git was made for.
Use git-lfs for binary assets like .uassets. Git is weak in this, make sure only one person is modifying a .uasset file at a time because you cannot merge those. It's one or the other. Perforce has a technology built for this which is 'file locking', something that git-lfs isn't fully ready for yet. In perforce, you checkout a binary file and no one else can touch it until you check it back in.
Complicated subject but that's why every team has a standardized workflow.
Alright thanks I'll research into that
What's "git=lfs"? Is "lfs" an acronym?
https://git-lfs.com/
Oh this?
Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.
Yeah that's it. it's an extension to solve the problems of working with large binary files in git.
Question of my own: I don't usually connect UE to a git source control as I do it manually via terminal, but I'm trying it out. Every time I launch UE5.1 it spends a large amount of time "Checking source control status of <asset>" and seems to work through my entire content tree. Is this normal behavior and is there a way to reduce it?
Git LFS has locking and works fine
are you using the git plugin built into the engine?
There are still situation in which locking is malfunctional, epecially on mac. Or at least it was broken last time I tried it. It may be good now
This looks phenominal, I will check it out (heh) next time I get a chance, thanks!
I am setting up a self hosted -perforce- source control. I have the core and p4V but could not connect from client to my localhost:1666 . Do I need to identify the server or is there any specific settings to let few users use my local disk as a server?
If they're on the same network LAN then they should be able to connect via <your machine IP>:1666
If they're not on your network, or connected via VPN, then you'll need to configure your router to forward requests to your local machine. External users would then connect via <your router public ip>:1666 or whatever port you use for the port forwarding.
I finally bit the bullet and moved from Git to Perforce for Unreal development.
I documented how I set up a new Perforce server for a Lyra 5.1 project in case it is of interest to anyone:
Including exact p4 commands that need to be run:
https://x157.github.io/Perforce/New-Stream-Depot-Procedure
Thank you! It seems we will need a static IP.
Or is there a way to turn around the IP problem?
just making sure: you're aware you can do all of this without touching the command line right? you're just using command line as personal preference?
I much prefer the command line. Especially for documentation. 👍
You can use a free dynamic dns and a small script that runs regularly to update the dns record. I have a similar setup and it works great.
Or best solution IMO is a cloud host.
did you try adding the files to your .gitignore
does anyone know how to remove this read only popup on rider? I'm using perforce
Perforce is marking the files as read-only since you don't have them checked out.
You can either use the 'using file system' option, or you can enable Rider support for Perforce. (VCS -> Enable, choose Perforce).
If you enable Perforce support, you'll still get the same popup from Rider, but you will be able to choose the "Using version control integration" option and choose the specific changelist you want to put the file into.
The only other way to get rid of that popup is to forcefully remove all read-only flags from your source, which could mess up Perforce. (Not sure tbh)
(Presumably if you check the "dont show this again" box it will just always clear read only on the filesystem? Probably not a great idea)
No you can override this at workspace level (Advanced Tab, check AllWrite)
Ah, you're right! Lots of interesting options there. Thanks! 👍
I could use some help. We tried to set up Helix Perforce, for Source Control, but while the other two developers could connect just fine, I could not, and the only difference is, I am IPv6. We tried to solve the solution, but it did not seem to work. Also, it seems that while connected to Source Control it really slows down the UE for other developers.
What I am looking for, and we do not care about price, is a Source Control that is easy to install, use, and can accept larger file uploads and I mean large I would like to have more than 15GB upload capacity. Any suggestions?
Please tag me or DM me Im in a lot of Discords
it's already enabled
i can see settings here
and I checked AllWrite on p4v but the "using version control integration" is still unavailable
if it bothers you, why not just add to default changelist and then you can sort in p4v later? (this is what I do, because yes I got annoyed by that popup too, and I'm very confident at using P4V to manage my workspace and commits/reverts nicely)
allwrite is a terrible solution, DO NOT use it
Do you guys have any tips to not have a metric ton of files using github source control and world partition?
should I include in git .vs folder? in case I do have c++ project?
Moving an asset seems to always result in a different binary content. Does it mean it's basically impossible for VCS to detect an asset move in order to construct full history of the asset?
Anyone willing to share a .p4ignore for a source engine?
As silly as it seems you can actually use the .gitignore that comes with the source build for perforce as well.
At the very least it's worked out okay for me. I didn't even bother renaming it.
Really? I didn't know they were interchangeable, good to know.
If there are, I haven't noticed. But my entire data is is 1: me, so take that for whatever it might be worth.
What size on Perforce should I expect the engine source to take? Having compiled bins would be nice to have for designers/artists, who will not be compiling themselves.
Guys, has anyone tried pipeing on batch to svn command?
Right now I'm trying to figure out how to make this work
where /R "%cd%" *.uasset | svn unlock --force
I want to unlock a folder with subfolders(1st layer is only subfolders without any files) but when i use svn unlock i get svn: E155008: The node 'Path_To_Folder' is not a file
also, wow, i didnt even think of that 😄
anyone else having trouble with plastic svn from github,its unpacked into project dir, but on starting project from launcher : ---------------------------
Missing MyProject Modules
The following modules are missing or built with a different engine version:
PlasticSourceControl
XmlParser2
Would you like to rebuild them now?
Yes No
, is this a knownissue or do I have it installed wrong....HMM
building of course does no good
if I have to build manually anditworks, so be it
oh sorry, version 1.7.0 zip
into project folderf
folder
Have you tried to just generate project files and then build the project with it?
gen project files ?
UGH@!Q
SO sorry,,I flat MISED there were versions for each engine LOL!!!
der der
o_0 🙂
gah, ok Plastic 1.7.0 installed in 5.1 project, but engine can't find CM command line tool, is that a common error ?
I see no cm.exe anywhere ininstalled path for plugin either
Failed to launch Plastic SCM 'cm' command line tool. You need to install it and make sure that 'cm' is on the Path and correctly configured.
mine is just in project folder which is enough supp0osedly
moved this to automation
I don't see it there
Newb here. I love using git. I want to use git for my code, so it can be hosted by itself on github for collaboration, etc. But I also wanna use perforce, to learn it and because perforce.
Stupid question: Is there a free cloud hosting for perforce? Lollllll spoiled by github
well I have cm status showing a running head, whew finllay, but ue5 project still can't SEE 'cm.exe', ????
sigh
source control is a giant headache sometimes
cm as in plastic svn installed, but in ue5.1 , trying to connect to plastic SC, never can find 'cm'
yet in cmd, cm worksfine
unfortunantly, unless you self host, im not aware of a free hosting for it, it's only limit for the free version is 5 users and 20 workspaces as long as it's self hosted, otherwise is azzure, aws google and i beleive digital ocean, but none of those are free for hosting perforce helix
ah so it is only free for self-hosted? Not only is the cloud server cost incurred, but also a licensing cost for helix?
no
helix core is free for use of 5 users and 20 workspaces
but you incure hosting costs
What’s the easiest way to have source control between just a pc and a laptop they will be on separate networks often they won’t really be working on the project at the same time. Preferably free
thank you droid
to be honest, the easiest and quickest, free way would be something like syncthing or nextcloud, neither of them are true source control but work for syncing files between clients/servers and you can set version keep times too. otherwise the easiest and also free way is to self host a gitlab instance on a spare computer, it's free, but it takes time to set up and works exactly like the paid service or github but it's under YOUR controll
anyone have experience with helix4git?
what is c++ source control like in perforce/UE to begin with? I would wanna use git for code if I can
no problem, one thing about perforce though, depending on your actual needs, can be a blessing or a curse, HARD file locking, it's nice in some cases but can be a pain too, it means you have to actively checkout each file before you can make edits, not make your edits and ask the server for permission to upload.
same goes for the c++ programing, you have to checkout the file you want to edit before y;ou can.
yeah I wanna learn the workflow in case I wanna get a job in game dev lol
gotcha, not a bad idea. personally, as a developer, i hate perforce specifically because of the file locking, the workflow isn't that different from git, it's ui is different, the terminology is different, but does almost the exact thing gitlab/github does, they just added hard file locking and instead of having code everywhere, all the files are stored in one place, not across the users and verified by the server after the fact. also a blessing and curse, if that one directory goes down, noone can edit, but at least the code is secured when it works.
yeah how does the code/committing normally work with perforce compared to git? I kinda wanna use git for code alone, whether via addon helix4git or just by clobbering things together with the typemap. thoughts?
the commiting is similar, but you have to do some steps in reverse, instead of creating the changelist and present your changes for review by git, you have to ask permission to edit every file you think you need first and make the change list before you even get your permission to edit, and only when thats done can you edit the files you specified, then you can check in the files and they are added to a cue for review and whoever has admin rights can do a merge to finally commit the files
as for my thoughts on git vs helix using git for code and helix for large files and basic transfer.... it could work, just seems more fragmented and less efficient, maybe thats where that helix4git plugin comes in, idk, but seems unwise to seperate a project into two version controls unless it's to duplicate it and have a copy
m2c
depending on the size of your project, if it were me, i would have a folder for each, copy the entirty of it to each and work through the workflow on each to see what's best for you, that way, you don't affect one code base by editing on a diferent workflow where something could go wrong
i do like perforce's large file hadleing better than git's git-lfs but it's not enough for me to give up gitlab's "personal opinion" superior redundency if used properly
interesting idea I might try that if I can manage the space and mental overhead lol
thank you very much for your thoughts
can i host gitlab on my desktop that im using unreal on?
im also not opposed to something hosted if its like 10 bucks a month
Is 50 gb enough for a perforce server?
That depends entirely on your project
If it's a small project, yeah, but you might have to obliterate versions older than some timeframe in the future though to clear up space if there's heaps of revisions and a long development time frame
Yes, however, if you have others working on the project, if the pc is turned off, they can't commit changes
It will be purely myself
What’s the average project size in terms of source control? I wouldn’t need a ton of older versions but usually I’m doing a lot of asset importing does that affect it much?
I was looking at the like 15 dollar version of digital ocean that gets you 50 gigs and I wasn’t sure if that was enough
Yes, that would be fine for a small project, just be careful digital Ocean has bandwidth limits on top of storage limits
To give you an idea though, my project for a simple mp survival is around 20 gig just for the code and assets, that doesn't count the actual builds(release candidates)
And another example, the first versions of apex and ark were around 114 gig
And to answer your questions about the asset imports, no, it wouldn't affect it much as once it's imported assets are rarely touched and since unreal engine 5, the one file per actor feature makes the changes small and fast, only if you have something like a 1gig skin for your character and your editing it alot will that ever to be a problem
Also, if your work flow makes use of kubernetes, you would do well to apply for the credits on Google cloud kubernetes, it's 400 bucks in storage and compute (spinning up vms and such for the builds), the only catch with Google's is that you have to use the 400 bucks credits in 9 months
Google and gitlab have a working relationship to make the credits viable
having the same issue on a brand new 5.1 source build, keeps failing here. How did you get past that?
Perforce's workings is based on the premise of Central storage, your client essentially downloads a shadow copy of the files that you "check-out", edit and have perforce verify changes, that way the source code is secure and Noone can open/edit without permissions where git, everyone has a full copy of the project and can edit as they please but needs permissions to commit the changes
What has that to do with what I said?
He said his projects are on different networks that, I'm assuming, can't talk to each other.
And he wanted his central storage to be on one of them.
That sounds like a chatgpt response. 😦
I was mearly pointing out that you don't actually share anything with a perforce SC and it sounded like he wanted the files on both computers, and as long as the perforce server has access to the internet it doesn't need to be on the same network
Still don't see how that's got anything to do with him being on unconnected networks.
Anybody know the command for p4 / perforce / helix core for force submitting a change to master and ignoring errors? (artist did an overhaul and deleted files without p4 use, so I'm just at the point of wanting to say "use their local, and ignore all errors trying to locate files"
They can use the Reconcile Offline Work functionality in p4V
Channie I could kiss you! Thank you for saving us time, and for teaching us for the future!
should I put .sln and .vcxproj on gitignore?
Yes
The one that comes with unreal source does this and is a good starting point
if I enable a theme plugin (from the engine), will other collaborators need to have that theme aswell?
I want to enable electronic and darker nodes, just want to make sure
Shouldn't.
alright
also idk whats doing it but sometimes my github changes are using \ instead of / so i think when that happens it's ignoring the gitignore?
for example .vs/ is ignored but it pushed .vs\... anyway
the path was too long, so I pull everything in my root directory and build the engine
I just had to regenerate files apparently
since theyre plugins, yes
were those changes tracked before you added the .gitignore?
then you need to remove them from tracking first
Does anybody here have a setup with perforce tracking the entire project overall, and git repository for the source code, inside the perforce depot? I'm trying to wrap my head around a way this could work, so that coders can use their typical git workflow, but then when the changes are checked in, it doesn't break. lol.
I know that if we perfectly track the .git folder's contents, whatever .git folder is submitted by a coder is guaranteed to be a faithful representation of the contents of the git repo at the time they submitted it. How can that be harnessed to allow perforce to keep track of which git "snapshop" to use, and to make sure the git repo actually reflects that state and there are no unstaged changes left dangling by accidentally mixing and matching files from the repo with a different .git folder.
?? 🧐
What did I just read
Did you say that you have a git repo inside of your perforce depot?
I do not yet have any perforce depot but I'm trying to figure out how to do that. The git repo would ONLY track the c++ files.
Normally you don't do this because it's the wrong way to do it. Usually you would use just perforce for code and assets, or you would use git with lfs for code and assets. You don't nest source controls
Any particular reason why you want to do it this way?
one reason is to make it easy to distribute the code via github. Also, because coders very much are used to git and not perforce. also, perforce does not seem like a very good tool for working with code. Is it? git seems to have better tools but correct me if I'm wrong.
FYI, many projects are indeed collections of nested git repositories, so the concept of a repo inside a repo is not insane
but, perhaps, it is with perforce lollll
many other c++ software projects, I mean, not UE projects
Perforce is basically the standard for game development. Git is more software development focused and is overall great for text files. My advice is to tell your programmers to grow up if you are complaining about perforce. Either git+lfs or perforce. There is also plasticscm if you want to explore that and I think svn has some support in the editor but I'm not sure what it's like
If you are talking about git modules that's completely different
I mean, yeah perforce is great for binary assets sure...
with the typical perforce workflow, do coders checkout individual files ahead of time? exclusively?
Not exclusively but yes it's checked out ahead of time. It took me a little bit to get used to it but you can merge them the same as git
I mean a git repo that has many other git repos inside of it, all tracking their state
Only your binary assets should have an exclusive lock because they can't be merged
ok so it sounds like perforce will be a nightmare for merging
I don't know if that's a real thing but I can't check right now. It sounds stupid af
that's why I wanna use git, so that the code can all be managed separtely but then referenced by perforce
it's not only real but very common
and a git submodule is basically exaclty nested git repos, but the main repo is "aware" of all the other repos
in the cases I'm thinking of, there is no use of submodules. Instead, the repos are just dropped righ inside, and each repo is tracked and managed separately.
This would, eg, separate the modular parts of your project. IIRC, google does this alot
If you are 100% sold on using 2 different and incompatible vcs for your project, you are going to need a custom CI/CD pipeline to build and deploy your project so people can use it. Otherwise all of your artists and designers are going to need to know how to use both vcs and know how to build your project correctly
hmm, I don't think that's correct
as long as the code in the depot is the intended code, what's the problem?
it adds complexity for people managing the code and/or the project as a whole, but why would artists care about a hidden folder .git?
the nice thing about git
is that you can delete the .git folder, and it just becomes a normal folder
all the files are there still.
building the project only needs the current, correct state of the intended git repo. nobody needs to change it if it is left in its correct state
I'm brainstorming here, please provide me with harsh criticism lol
I already told you that it's not a good idea and gave you alternatives
I'm not looking for alternatives though. I'm positive I want to do this. I know other game studios have done this.
I'm not even a game studio lol I'm just messing around and learning
so it's not critical if it works or not
Do you want your source code in perforce and git?
I think so
I imagine the perforce tracking of the code would be much less granular than the git tracking, tho.
So like, the coders go and do all their commits, their branches, their merges whatever. When the code is deemed appropriate for the depot, the changes are submitted to perforce. Perforce will track each larger set of changes as a whole, with a reference to the exact state of the git repo. The git repo is tasked with tracking all the tiny, many changes inside only the git repo
I'm thinking of making the .git folder treated as binary in perforce. Do file deletions get tracked reliably in perforce?
and can the entire contents of a folder be mapped to binary, regardless of file extension?
I'm not sure you want the entire .git in perforce.
In fact, you probably don't wnat any of it. Just stick it on ignore.
Typemap:
binary //depot/dev_projects/...*
??
no I do really want to track the .git folder
that is the most important part to track. It represents the exact snapshot of the code at the time it was submitted to the depot.
But it's already being tracked by the git server. Why track it twice?
to track the intended state
because like a git repo can have many branches or states of the branch
the .git folder tracks the exact intended state, in relation to all the other states
this way perforce doesn't need to track all the little code changes just like
each large change to the code is tracked
Let's say the designers are like
Coders, we need feature X. Make X.cpp and X.h plz
coders are like ok. They make a branch of the current code, make some changes, test it out, and submit a their code on their test branch
to perforce, a WHOLE TON OF THINGS just changed at once, files added, changes made in places perhaps.
not granular at all.
But the git repo tracked all the little changes the coder made along the way, and tracks which branch of the code is submitted
It would be granular if you made code changes directly through perforce.
yeah
I much prefer the git workflow, but it's entirely possible to do it on p4 and it's not hard at all.
I imagine there is definitely a way to do it
and I will learn it because I will use it for binary assets and all other text assets besides the c++ code
but in order to share the project code easily on github
and for coders to be able to use git as they want, with pull requests and merges and diff tools like we all know...
And think of it like this way also
You like giving yourself a headache trying to maintain 2 VCS instead of 1?
Now the coders get their nice clean design history without a bunch of stuff about art assets
and the designers get a nice clean and concise design history in perforce, with large single changes to the code at one time
You can have a nice clean history without a bunch of art assets in p4 already
oh really ok good
You have per-folder (inc subfolder) and per-file history views. It's quite useful.
How so?
tracking changes in code is usually more about tracking the snapshot across multiple files at once, not individual little changes to each file
That's where change lists come in.
Tbh, you just sound like you haven't used p4 before and you're trying to shoehorn your git workflow into it.
and this does not seem hard to do... unless I'm missing something
It's not worth the hassle. Just learn how to use p4 effectively.
how is it not worth the hassle to allow greater control, and to onboard a more diverse set of coders onto the project easier?
u know how many c++ coders normally use perforce?
the number of c++ coders who are forced to use it
and none more
You know how many c++ coders can learn how to use p4 in about 10m with a little instruction? All of them.
that's not the point
I can learn to use a hockey stick to do surgery but it doesn't mean it's gonna be easy or fun or the best way
Except that's exactly what you are doing.
nah I'm trying to use a specialized scalpel alongside the standard set of tools
like, I'm learning p4
No, you're used to using a mallet and you're trying to use it with a scalpel because you haven't learned to how to use the scalpel properly.
nothing wrong with that
except... I'm not ignoring p4
p4 will be tracking the code too
wat is the problem u forsee?
If it's not gonna be a problem, just set it up and get it working.
lollll
It's simple, right? You just add the ,git folder to p4 and off you go.
there will be specific ways to do it that will work better than others. That's why I'm here to discuss it
it's not tho. the typemap
I'm tlaking about it from a p4 perspective here
we need to set up the typemap so the .git folder is tracked 100% accurately, and the contents of the repo in terms of the actual source files are always ensured to be kept to what the .git folder says
what that means, really, is that multiple coders could submit all sorts of random source code files, jam them up into p4, and as long as the .git folder represents the actual intended state of the code, git can ensure that the files match that.
I understand, you can achieve similar functionality with p4 alone
But, then, you can't track it with git
and I want to track it with git because I know lots of people who want to track code with git
and then the managers can track the code in p4 too, and they won't be saddled with the hundreds of commits that the coders would want to be making
and the code can easily be shared
someone can modify it, with no knowledge of the p4 depot, and the code can easily be integrated in with all changes tracked
p4 can still see "Code was like this. Then code was like this. Then code was like this"
but how the code got to each of those points can be tracked by git
The code on the p4 depot would be more nimble. Large changes to the code base could easily be swapped in and out with git, all managed in branches, so that you don't need a million streams in p4
basically, my question kind of comes down to, how to manage the checking in and out of the .git folder. Because it has to be tracked in its entirety accurately
everything else remains exactly the same in terms of p4
It must be disallowed for a coder to make a change to a source code file without also submitting the full .git folder
and, really, that means they may potentially be changing the entire codebase 😐
in the typical p4 workflow, is there like a pull request kind of convention or mechanism?
who approves the changes to be actually commited to a stream?
Ah, thanks ChatGPT:
Yes, in Helix Core, you can enforce that entire folders be checked out or checked in at one time by using the "atomic checkouts" feature. This feature ensures that all files within a folder are checked out together and checked in together, so that the contents of the folder always remain in a consistent state. To enable atomic checkouts, you will need to configure your Helix Core server to use the "atomic checkouts" option for the specific depot or folder that you want to enforce this behavior on.
so I would want to make the entire git repo, including the .git dubdirectory, to be an atomic checkout/checkin
but without exclusivity, so other people can also do the same
done
There is in the Epic workflow, I'm not sure if their system (horde) is publicly available, though.
nice thank you
thank you for entertaining my zany ideas
but I'm pretty sure I heard someone talk about doing this in an unreal talk at some point
using git inside of p4
the great thing about setting this up will be that it can be abandoned at any time by simply deleting the .git folder and moving on
so it would make a good experiment with no real risk
I would still not track .git inside p4, except maybe an initial state just so new people don't have to init the repo
they wouldn't have to
if you don't have git, the .git folder is just a small hidden folder
it does nothing
It's not a "small" hidden folder. It can get quite big.
I mean, with binary assets
with a normal sized repo it will be fine
like code-based
because it stores binary blobs
it is very efficient at storing changes to text
286 MB .git folder for an entire software project
although
if p4 starts tracking changes to those binary blobs themselves it would get expensive on storage on the p4 side, but that doesn't happen I don't think. Maybe we would only track the files in the .git folder for a few versions
Can it be easily changed later in the typemap, the number of revisions that are kept for a binary file? Like if I decided to keep them all at first, and see if storage becomes an issue, can I easily change it down to keep like 10 versions only of those binary blobs? I don't know exaclt how git stores those blobs but I doubt they are changing much once they are saved. They either exist, or don't, IIRC.
this leads to my other question how well does p4 track file deletion?
Why not just track the actaul source files?
Let git worry about git.
You don't need the git housekeeping files in your p4 repo.
My thought for how to do it (I haven't actually tried it yet):
P4 ignores .git/* database
P4 ignores all files stored in Git
Git ignores everything other than text stuff stored in Git
In this way any time you want to edit binary stuff, you're editing P4.
Any time you're coding, you're using Git.
And for non-coders you can do a source code update every so often.
The problem with this is all the P4 people now need to git pull every time they update P4
I'm not a fan of that. Maybe there is a way to automate that inside P4?
Well it's not just CPP that IMO should be kept in Git
The .uproject should be kept there too
It's text. Zero reason to keep it in P4
You can keep uproject on p4, because non-coders need it too.
Right that is why P4 users would have to do a git pull
To get the .uproject, .uplugin. .ini files, etc
But p4 users should never have to use git.
Which is why I haven't actually done it this way yet.
What's the point of this setup if p4 users have to use git?
Why not just use git and lfs?
Exactly
I would put just source/ in git
Then you have to edit both Git and P4 if you're a developer, any time you want to change the uplugin/uproject/ini files.
The point of this is to make things easier/better for c++ devs, not to throw more road blocks at them.
Maybe there is a better way to do it. Given my limited knowledge about P4 this was the best solution I came up with if you really want to use Git for C++.
And I don't particularly like it so... 
Do you really change those things all that often?
I don't change the plugin/project files all that often but I do try to keep things in INI as much as possible.
Honestly, I don't think I've ever changed anything in an ini file and committed it.
I don't work on games, though.
Anyway I definitely think you do not want to store the .git database inside P4, if you decide to combine the two.
P4 is already keeping copies of the files, it doesn't need copies of the (supposed-to-be distributed) Git database.
That theoretically exists on many remote machines anyway, P4 would be needlessly redundant.
Indeed.
The "state" he talks about is literally the files themselves, not the housekeeping files.
All the files inside .git are the Git state, to allow you to swap from one version of the file to another without contacting the server.
It's one of the reasons Git is so much more vastly superior than P4 for C++ work.
I'm talking about a different type of state.
I think we're probably trying to say the same thing in different ways.
He mentions the "intended state of the files" and I'm saying, the intended state of the files is the files themselves in the current checkout situation.
You don't need anything else to tell you that.
Right, well that is P4's philosophy about the state of files anyway.
One of the major differences between working with P4 and working with Git.
From my usage of p4 the main difference is branches vs cls, though I don't think we use p4 to its fullest potential in that sense.
The state of files is pretty much the same, just one is kept on the server (p4) and one is stored on srever and client (git)
interesting thank you xist for giving your thoughts
I'm going to try doing it with p4 tracking the .git folder, so that you don't have to lol
and if it is crazy I will let you know
the reason I really want to include the .git folder is because it keeps direct link to the actual state of the repo it's from. without it, it's just a bunch of files in p4.
My concern there is the .git/objects folder contains every revision that has ever been saved in Git.
Which if you copy that into P4 is going to be lots and lots of redundant copies.
No major issue, but a lot of file bloat.
Maybe in P4 you should link it not as an actual Git repo, but as a Git submodule.
That would allow you to tag a particular Git revision.
yeah there will be some bloat in a sense, but the .git folder only stores the local state of the repo when it was submitted to perforce. So, it will be small at first, and only grow as the project does. Then, it can be cut down with a hard fork
but this way it will be able to recreate perfectly the state of the repo and branch, if the files get messed up somehow or just to be sure
and you can just open up git gui or bash and find out exactly where in the git repo the code is from
if you don't have git? Just ignore the .git folder and don't modify code
tracking uproject files and ini files is interesting. I suppose that would depend, for me, on how closely it is related to the work of the people touching them. If they are primarily for coders, keep it in git too, but if not maybe just p4 for simple things
since everything is tracked in p4 and git, we won't really lose sight of things. But, it's important to note that p4 is the primary source control. Git alone won't tell you much about the rest of the project. Commit messages could include "soft references" to p4 changelists or streams or something. p4 changelists should definitely reference the commit hash of the git repo if relevant, just in case
also, back to my question, how does p4 handle file deletion? The .git folders could be deleted or re-added at any time to the p4 depot. If the file deletion is smartly tracked, bloat can be managed if need be.
But compared to the bloat of binary art assets and their iterations, this seems like no concern to me really
the main reason I want to do this is to make it easy to release code on github open source, and then other people can easily use it. If the code is only in p4, it's gonna be a bad time. lol
If you are not selective about what you pull, that "local state" will be "everything ever"
yeah that is true
p4 should be smart enough to not store the same data multiple times though, right? as long as the files don't change
git definitely creates unique binary blobs. the files are never written over, afaik. so the perforce storage footprint will be 1:1 to the size of the .git folder. right?
it's just like having the .git folder in your depot and it doesn't change. it just gets added to.
As long as the files doesn't change, p4 will not make additional copies.
so this should be fine I think
just have to ensure "atomic checkout" is enforced on anything tracked by git. the whole repo.
As far as I understand it, the files in /object won't ever change, they'll just multiply.
and then make p4 only take what has actually changed to reduce bandwidth
but make it so when someone checks it out, it's not exclusive
since git is tracking the code, who cares how many ppl change code or when they submit it to p4. it will always be coherent
I mean, how it relates to the rest of the project is up to the management of the project as a whole lol but at least the codebase will be the exact codebase that the coder thought they were submitting
and people will know who and how many ppl are currently working on the codebase
and then they can check github for the activity of those ppl, in their branches
I'm starting to think bot might actually be a bot
I'm ChatGPT 3.5
Actually ChatGPT has way better grammer, more coherent thoughts and a better sense of... smartness
Oh, nevermind, I think bots can spell grammar.
Bahaahah
my point exactly
to be fair
I asked ChatGPT if this was a good idea and they thought it would be fine
Yes, it is possible to use Perforce to track the version history of the project as a whole, including binary assets, while using GitHub to track changes and version control for the source code text files.
Perforce is a centralized version control system that is well-suited for managing binary files, as well as large files and directories, it can handle large amounts of data and is optimized for performance. GitHub, on the other hand, is a distributed version control system that is well-suited for managing text files, such as source code files, and it has a strong focus on collaboration and open-source development.
In this setup, you could have a Perforce repository that contains the binary assets and other large files, and a GitHub repository that contains the source code text files. The two repositories would be separate, but the source code could be included in the Perforce repository as well.
It is also possible to have a Git repository reside within a Perforce depot and be tracked, including the Git head, blobs, etc. This can be done using Helix4Git, which is a plugin for Perforce that allows you to use Git as a client for Perforce. Helix4Git allows you to use Git commands to access and manage files in a Perforce repository, and also allows you to use Git's branching and merging functionality with Perforce.
However, it's important to note that this kind of setup can be complex, and it would require a good understanding of both Perforce and Git, as well as how to configure and manage them. It's also important to test and evaluate the performance of the setup before using it in a production environment.
lol just noticed it called git GitHub loll
same thing
and I think it lied about Helix4Git
that's not what it does LOLL
fraggin bot
This can be done using Helix4Git, which is a plugin for Perforce that allows you to use Git as a client for Perforce. Helix4Git allows you to use Git commands to access and manage files in a Perforce repository, and also allows you to use Git's branching and merging functionality with Perforce.
this is actually quite literally what it does
is it?
I mean, I think it's kinda accurate about what helix4git is: an interface layer from git to p4. But I don't think it achieves having a git repo inside a p4 repo, but using p4 as normal otherwise
yeah thats what I thought. That might be useful in some scenarios but is not this lol
Your original plan to combine the 2 was so that your programmers didn't have to relearn p4.
no not really
the primary reason is to be able to easily share code on github
and with that come some nice tools and familiar tools for coders
the ppl using git will 100% have to also be using p4 all the time as much as they would if git was not being used
but they can use git to further track granularity of their code changes, and have more branches in an more efficient way than lots of p4 streams
you're probably going to give yourself more headaches rather than less with this
The headache argument doesn't work. 😦
Often you want code and asset changes in sync - otherwise you have to come up with a solution for making sure builds have the correct changelist and commit. Sync content but forget to pull code? Oops you overwrote an asset with old data.
That's the whole reason helix4git is a translation layer rather than a git repository inside perforce.
I'm not going to tell you which one to pick, but I am going to tell you that using both is likely going to be worse than choosing just one. I'm not a fan of using git for assets but Git LFS does exist. Perforce won't work with github but swarm exists for code reviews and "lightweight" task streams exist for at least some of git's branching functionality.
I agree with you in some senses, but I really feel like there is a compelling reason to go through the slightly extra steps to be able to have full control of the code alone.
The assets and code will be kept in sync by p4. We are not removing any tracking from p4. But we are giving the coders the ability to more easily jump around in their multiple branched codebases, submit exactly whatever code they want to p4, have that exact state tracked alongside the art/binary assets.
If it wasn't for my desire to potentially release some or all of the code as open source, then I probably wouldn't go through the trouble. But, if the codebase is all tracked in a separate git repo, or multiple git repos -- one for each code module. then those code modules can more easily be shared and incorporated into other unreal projects, and incorporated into their p4 history, while retaining 100% of code history across projects
but there are some huge drawbacks for larger projects: security
so yeah this is definitely not a good option unless the code is meant to be shared and not highly protected
If it wasn't for my desire to potentially release some or all of the code as open source
it's not that hard to mirror commits from p4 to git by writing some scripts (if you don't find some premade ones).
But, if the codebase is all tracked in a separate git repo, or multiple git repos -- one for each code module. then those code modules can more easily be shared and incorporated into other unreal projects
You can already do that with perforce, arguably easier than with git. You can map multiple depots into a client view. So you could have a shared code depot. If you need per-commit control over when those shared plugins are integrated into a project then you don't map those depots into a client view but instead just treat them as separate branches and integrate upstream/downstream as normal.
afaik, p4 history is only relevant to the server it was made on. no history can move over to a different server
Depot != server
git history is portable
I'm not sure why you're talking about multiple perforce servers
because if the code is open source, then another team with a different server might use it
they want the full code history
not starting from scratch
I didn't say use perforce for open source work.
they also wanna be able to pull in any code changes
it's not that hard to mirror commits from p4 to git by writing some scripts (if you don't find some premade ones).
mirroring commits does not preserve history
Sure it does.
it creates a new fake one
it seems like the same amount of hassle as using git like I suggest
P4 is your source of truth. Github is a nicety.
Then I'm not sure what you're even saying you want to do
the code is also tracked separately so it can be portable.
Picture the code as one giant binary asset
Err, what
we wanna be able to take it and use it in another project
You can already do that with perforce, arguably easier than with git. You can map multiple depots into a client view. So you could have a shared code depot. If you need per-commit control over when those shared plugins are integrated into a project then you don't map those depots into a client view but instead just treat them as separate branches and integrate upstream/downstream as normal.
i mean not literally like a binary asset lol
I'm not sure exactly what the downside of doing as I suggest is
but I see lots of downsides of custom scripts mirroring commits to GH lol
Honestly at this point I'm not even sure what you're suggesting. You want to mirror portions of your perforce depot into git which is exactly what I've said you can do.
not mirror them.
Yes mirror them. Even helix4git is "mirroring" them because perforce itself doesn't use the same concepts as git.
if the git history is fully respected and not just imitated, then users across teams from many different projects can all cherry pick individual changes they want, and see the full history
He wants to add the entire .git folder into p4 for "reasons"
how familiar with git are you guys? just curious because this does not seem crazy lol
very
a .git folder is not a big deal
It's just totally pointless
no it isn't!!
do you ever write anything down about your project on a piece of paper?
you're now describing mirroring a git repository into perforce rather than the other way around. By making perforce commits every time the git repo changes.
no, they aren't
at the same time
you make changes to git files and then commit them into perforce
these are two (potentially more) distinct operations
someone could make 10 commits in git then submit that as a single CL in perforce
except that won't work
granularity in git, and easy overview in p4
if someone else also makes changes in git, syncing the .git folder won't magically resolve the changes
why won't it work?
it will if the entire repo is set as atomic checkout
that way the whole repo will be compared to the p4 depot
and ALL changes, especially to the .git folder, is submitted
as long as the hidden .git folder is accurate, we can know what the code is supposed to be and we can resolve or prevent issues
Surely in this scenario it's down to the person making the p4 code update to make sure the code is what it's meant to be.
yes precisely
it's up to them which branch of the git repo
or what they wanna submit
except from what I know that there are a number of files in .git that cannot be merged on a p4 sync.
the git repo allows them to manage that
they do not need to be merged
except they do... or stomped
if they are text files, they can be merged fine. other than that, they are binary blobs
the binary blobs don't change
yes, and stomping those loses data for someone
stomp -> replacing the file rather than merging
there are a number of files in the main .git directory (not subdirectory) that are, to my understanding, required for git to operate correctly
ah yeah
well I have a very good understanding of how thos files work
and I can tell you it should be fine
if p4 can track text files
and p4 can add binary files with no file extension
then we should be fine lol
it should be fine as long as we atomically checkout the .git folder
the entire folder needs to be checked in and out exactly as it is at any moment
it might be needed to disable compression on the binary blobs
What I suggested was init the repo and add that to the p4 (if you want to do the crazy p4/git plan) so the coder only has to pull and not init the repo themselves.
easy with typemap
the coder doesn't need to pull unless they want to
the entire state of the repo is in the .git folder
one danger would be the potential for a coder to submit untracked changes, and then the changes could get lost for git... but the history on p4 is fine
that's not the only danger
The other issue here, siliex, is that bot doesn't actually know how to use p4.
well yes I do
I have spent the past three days studying it lol
does it sound like I know nothing about it to you?
the issue isn't about "using p4"
it's about workflow best practices
Three whole days?
hmmm
Sure. It takes a coder 10m to learn to use p4. Create CL, add code, submit, whatever.
You aren't a coder, you are a project manager.
Or a DevOps guy.
sounds about right
there's a massive difference.
i mean, I use git to modify and release my own versions of open source software
soooo there is that
kinda makes me a coder?
That's not the point. You can also be a coder, but your job on this project is not coder, it is manager.