#source-control

1 messages · Page 4 of 1

analog lodge
#

So if they create a new asset that should go into the master Library, they can migrate it to their local copy of the temp library.

#

And submit it.

#

Then they´dneed to log it somewher, so that I can review that, if need be sort things into correct folders etc

#

And then migrate from the temp library into the master library.

#

Which leaves way more admin duties to me than I´d like to, but I just don´t have teh expertise to build a system thats "fool proof" and automatically sorts assets into correct folders and makes sure nothing breaks, lol.

#

The other problem though is, how do I layout the rest of our repository?

#

Lets say we have 100 projects named "2001", "2002", "2003" etc.

#

they are all separate projects without any interconnections.

#

So, do I now have 100 separate depots?

teal bone
#

It comes down to how much you trust your artists, really.

analog lodge
#

Not at all, lol.

teal bone
#

If you think they're going to check in shit work that needs to be reviewed and checked constantly, it's going to be hell.

#

Or if they are going to fail to understnad where to place assets etc

analog lodge
#

I mean, on our current server, working with 3ds max etc, we have ONE folder per year and then subfolders for each project.

#

And at the end of the year, we archive the whole folder.

#

We don´t need versioning history for any of our project, once they are archived.
So I´d just back up the local workspace folders manually on our big NAS and delete the depots in perforce.

analog lodge
#

Its when people get sick or we have bigger projects when things tend to get messy.

#

And Unreal kind forces you to be a bit more rigid in how you use your files.

#

You can´t simply copy and paste shit in the explorer and expect things to work (which they often also don´t do in other DCCS).

limpid obsidian
frank veldt
#

Did I miunderstand the conversation that happened before, or did you say that git LFS keeps versions of files across all branches? so you can't have different LFS files in different branches?

#

(i was just scrolling up and reading random chats xD)

hybrid pine
#

Thank you for your reply. I'm quite new to this. Woukd you mind telling me the benefits of using git over the zip? You are referring to gitbash correct?

quaint obsidian
teal bone
#

So, what you're saying it, you have a lot more control... of your source!

#

mcoms outlines the reasons why source control in general is awesome. It's an invaluable tool!

hybrid pine
scarlet dust
#

so, earlier in the day, my game wouldn't boot up (picture 1), so I followed the advice of someone else and verified the engine files. After it gave me the error as shown in picture two and when clicking the error message i get told that there are corrupt game files. How can I fix this without losing my two months of progress?

teal bone
#

I was kinda being facetious when I suggested Source Control - it will help the next time this happens.

#

But like, Install Failed? Have you clicked the links to see what the error codes mean?

scarlet dust
#

and the tutorial it shows doesn't work

teal bone
#

Copy your assets out of your content directory. I assume it's all BPs?

#

Just back them up somewhere.

#

And then delete them from your content folder, see if it runs.

scarlet dust
#

what?

#

sorry, im new to unreal, and building a game off of tutorials

#

so i dont know much

teal bone
#

Go to your project folder.

#

It will have a directory called "Content"

#

That's where all your stuff is.

scarlet dust
#

okay

#

i see it

#

that has my whole game in it?

teal bone
#

Copy and paste it somewhere.

#

Can even just copy+paste it to your project fodler and call it "Content2" or whatever.

scarlet dust
#

okay

#

im putting it in a folder i called VastRuinsBackup

teal bone
#

Alright.

scarlet dust
#

that will have my whole game, including the maps?

teal bone
#

Yes

scarlet dust
#

okay

#

good

teal bone
#

Now delete things (the .uasset files) in the original content folder. Like all of them.

scarlet dust
#

cause i CANT lose this game

teal bone
#

And try to open your project again.

teal bone
#

As long as your copypasta fu isn't awful, your game isn't going anywhere.

scarlet dust
#

lol

teal bone
#

You can either delete assets 1 at a time until it loads.

#

Or delete all of them to see if it loads and then add them back in from your backup 1 at a time.

scarlet dust
#

okay

teal bone
#

Until you find the issue.

scarlet dust
#

ill try the deleting and restoring one at a time

teal bone
#

This is the issue source control would fix - you would have a backup of every file from, say, yesterday.

#

And you could just restore that state.

scarlet dust
#

i dont but okay

#

I never thought to make backups, which now i realize is very stupid

teal bone
#

You can literally just zip your content folder up every week if you want, not as effective, but avoids catastrophic failure.

scarlet dust
#

yea

#

imma delete everything in the content folder, then reboot the unreal engine to see if its a system problem

#

gotta repair the engine itself first tho

teal bone
#

If it doesn't boot with nothing in your content directory, you have far more serious problems.

scarlet dust
#

and those might be?

scarlet dust
#

i moved the backup of the content folder somewhere else in case that was the problem

#

and i still got the same error

teal bone
#

Try rebooting

scarlet dust
#

okay

#

okay im rebooting the launcher

#

still wants me to repair it so i clicked repair

scarlet dust
teal bone
#

Create a new project?

#

You verified the engine, right?

scarlet dust
#

i did but then i keeps saying i need to repair it

teal bone
#

You should probably do that.

#

Or actually remove it and add it again.

#

That will take ages, but yeah.

scarlet dust
#

okay

#

yeah ive clicked repair 4 times and i sends that error every time

#

okay its uninstalling

#

and i dont really care how long it takes, as long as the game gets back online

scarlet dust
#

@teal bone i got the same error and it stopped the re-install. What should I do?

teal bone
#

Install 5.0 instead? 😦

#

You're not out of disk space or something are you?

scarlet dust
#

nope

#

imma try again, and if it dont work imma remove all the files that are there

#

it may not be the assets that are the problem

teal bone
#

I think that's definitely the case.

scarlet dust
#

okay

#

i removed all the vast ruin files and it still gave me the error

#

should i just try uninstalling the launcher itself?

teal bone
#

No idea sorry mate.

scarlet dust
#

okay

#

imma try uninstalling and erasing everything but my content folder

scarlet dust
#

@teal bone I FOUND THE FOLDER

#

I FOUND THE CORRUPTED FOLDER

teal bone
#

Oh?

scarlet dust
#

i uninstalled the launcher and everything

#

and i couldnt delete this one folder

#

this is in it and this is the error i get

teal bone
#

Ha

scarlet dust
#

now if i can get it off my computer that would be great

teal bone
#

So that plugin from right back at the start. 🙂

scarlet dust
#

yep

#

it was in the main engine, not my game

#

now if i can get it off my computer, that would be great

teal bone
#

rename EpicGame to DIEPROFILEWIZARDDIE and be done with it.

scarlet dust
#

i dont particularly want it corrupting my whole pc

teal bone
#

Your pc is already corrupted. It might be a sign your hard disk is failing.

#

I would do a chkdsk

scarlet dust
#

idk what that is but okay

#

also, whats this do

#

okay...

#

and it deletes corrupted files?

#

ah

#

okay

#

shouldn't have

#

my computer works fine

teal bone
#

Hard disks degrade over time.

scarlet dust
#

well that i know

teal bone
#

When they start failing, they automatically hide the broken bits.

scarlet dust
#

interesting

teal bone
#

When they run out of ways to do that, it's already pretty fucked.

scarlet dust
#

why is that?

teal bone
#

Because you have a billion billion bits on your hard drive and a few million of them failing isn't a problem.

scarlet dust
#

what the hell does this mean

teal bone
#

(exaggeration mode engage)

#

That means it will do it on the next boot.

#

Just hit y

scarlet dust
#

okie ill be back in a sec

teal bone
#

"sec" 😂

scarlet dust
#

It's giving me loady wheel

#

And its checking the disk

#

(the cords are to a second monitor)

#

K logging back in

#

So now I should be able to delete it, correct?

#

YES IT DELETED

#

I just did shift+delete so I don't have to go through the trash bin

#

Okie imma go reinstall unreal engine

#

I'll be back later!

teal bone
#

Good luck.

scarlet dust
scarlet dust
#

nah its this. Its done now, im just waiting for quixel to finish before i launch it

#

cross your fingers

scarlet dust
#

It works!

stiff vessel
#

Hello, I am having a really hard time with Plastic SCM

woven sluice
#

not a whole lot to go on there sarge

mighty light
#

Maybe they just want a bit of sympathy and some moral support

woven sluice
#

Cheerleader squad, assemble

worn pebble
#

Hello everyone. I was hoping I could get some help with an issue regarding git, git lfs, and project boot times that are way too slow.

Setup:
UE5 project (hosted on a different drive than the UE5 engine)
git installed
git lfs installed
git init run in project repo
Initial commit made (all files in repo committed)
suggested .gitignore added to repo base (attached)
suggested .gitattributes added to repo base (attached)
git source control enabled in project

Issue:
Every time I open the project, it gets to 91% and then spends >40 seconds flashing a message about adding to source control. (gif attached)
The source control log shows a ton of errors related to the fact that I have ignored starter content. (file attached)
The source control log also shows fatal errors related to Engine Content being outside the repository (file attached)

The errors are annoying but the startup time is what kills me.

Any help getting rid of that 40 second constant flashing would be very appreciated. Happy to provide any extra details needed.

#

Apologies if there is a better way to display the pictures so it is not so obnoxious spacewise, maybe I should have made a collage....
I also looked in the history of this channel and found some similar error messages but they seemed tangentially related to note having pushed content, which I'm not sure is an issue here.

worn pebble
#

I appreciate the response. 🙂 That's what I'm doing now. 😦 Separate command-line window always open. Just a bit sad I can't revert in engine.

#

Oh, I'm much more comfortable with command line. More-so, I meant that it's nice not to have to close the engine if I need to revert a change. SourceTree is definitely a good suggestion though. 🙂

#

Yeah, that's why it's a charitable suggestion. I'll admit that I'm less familiar with source control interfaces on Windows, so your response was a useful one. 🙂

quaint obsidian
merry dust
#

Is it safe to use the same workspace for different perforce streams?

faint cairn
#

When talking about StateBranches, does anyone have some insights on the CI pipeline required to auto merge stuff from, e.g., main into dev?

    // Order matters. Lower values are lower in the hierarchy, i.e., changes from higher branches get automatically merged down.
    // (Automatic merging requires an appropriately configured CI pipeline)
    // With this paradigm, the higher the branch is, the stabler it is, and has changes manually promoted up.
#

(using GitLab, but I assume that the CI part is just generic Git stuff I can add there)

faint cairn
#

I saw that I would need to have an SSH Connection to the Runner so it can execute the Merge via the CI/CD stuff

#

There were some SO questions about it

#

Haven#t looked into it yet though

#

All I#m working with atm is this

#

And to a very simple extend

#

Something something

#

Not sure what trains are yet

#

(bob is the build machine. don't judge)

#

Alright, will keep that in mind when tackling it

quaint obsidian
#

why does it lead to broken branches

#

how would this be relevant in this status branch context, where branches don't fork and it's just a straight line of branches?

#

but i don't understand how it would be possible for merge related breakages to occur if it's just merges from release->main->dev

#

because you merge up manually the other way?

#

again, this is status branches we're talking about

#

what do you mean by "missing some changes"

#

but people work on release, they work on main, they work on dev

#

i feel like maybe you don't know about the status branch workflow? it was shared by epic regarding how they work on fortnite

woven sluice
merry dust
woven sluice
#

not merging

#

when you switch a workspace from one to the other, the physical files on your disk have to change

#

your workspace has to match the new stream

merry dust
#

Makes sense and I don't think there will be frequent switching. 🙂

bitter sun
#

is there any way to find a word in a list of revisions in P4?

#

I removed something I shouldnt (a struct declaration) but it exists on previous revisions.. though not sure which file

#

gotta find and copy paste it back to an another file

woven sluice
merry dust
tidal skiff
#

hi, can anyone show how to set P4ROOT using p4dctl? (digital ocean, perforce)

I tried to put P4ROOT like this

#-------------------------------------------------------------------------------
# Environment variables common to all servers
#-------------------------------------------------------------------------------
Environment
{
    P4DEBUG     =  "server=3" # Embedded = requires quotes
    P4LOG       =  log
    P4CONFIG    =  .p4config
    P4ROOT      =  /mnt/volume_lon1_01 /perforceserver/root # ADDED THIS LINE
}```

but it does not work 😦

I am basically trying to move the p4root to volume i purchased.
#

Also treid restarting the services
p4dctl restart -a

merry verge
#

what's with the space?

tidal skiff
#

there is no space

merry verge
#

/mnt/volume_lon1_01 /perforceserver/root is a typo?

tidal skiff
#

yes

merry verge
#

next, why are you setting the root in your global environment section? It's specific to each p4d instance

#

generally p4dctl has a config file per-server in p4dctl.conf.d/<server name>.conf

#

and it'll have something like this in it:

p4d servername
{
  Owner = perforce
  some other options

  Environment
  {
    P4ROOT = /path/to/root
    some other options
  }
}
tidal skiff
#

ok knew that was wrong.

p4d perforceservice
{
     Owner          =        perforce
     Execute        =        /opt/perforce/sbin/p4d
     Environment
     {
        P4ROOT      =        /mnt/volume_lon1_01/perforceserver/root
        P4JOURNAL   =        journal
        P4PORT      =        ssl:{ip}1666
        P4SSLDIR    =        ssl
        PATH        =        /bin:/usr/bin:/usr/local/bin:/opt/perforce/sbin
     }
}

is this right?

merry verge
#

that looks correct, yes

tidal skiff
#

p4dctl restart -a
error

p4dctl error:
        Service 'perforceservice' p4d: Directory '/mnt/volume_lon1_01/perforceserver/root' does not appear to be writable.
p4dctl error:
        Configuration for "perforceservice" p4d has a conflicting parameter "name".
Stopped 0 services.
p4dctl error:
        Service 'perforceservice' p4d: Directory '/mnt/volume_lon1_01/perforceserver/root' does not appear to be writable.
p4dctl error:
        Configuration for "perforceservice" p4d has a conflicting parameter "name".
Started 0 services.
p4dctl error:
        Not all services started successfully.
#

will this work?
sudo chmod +rwx directoryname

#

@merry verge

#

tried this
sudo chmod ugo+rwx foldername

and the directory is now readable but still getting error about the name

#

Configuration for "perforceservice" p4d has a conflicting parameter "name".

amber wing
#

Thanks for your help @merry verge were nearly there!

iron bronze
tidal skiff
#

perforceserver is not in p4dctl

tidal skiff
tidal skiff
#

aaah

#

yes

#

i guess .d is default file

#

should I be modifying this?

#

include /etc/perforce/p4dctl.d (includes *.conf files from the directory)
ok

#

@iron bronze thank you very much

iron bronze
#

Yes the .d suffix in linux environments indicates a folder

#

It's a good habit to put your own .conf files in there so anytime you update your server you won't have to resolve the default configuration file (that might get changed by the package manager)

worn pebble
bitter sun
#

I tried p4 grep but it searches for files instead

cyan jay
bitter sun
#

thanks i'll take a look

winter parcel
#

hi! how to clean server space in perforce and remove old revisions or clean files history ?
e.g. I have revisions 1-1000 and I want to keep only 100 last and keep at least latest possible file .
p.s. this one https://i.imgur.com/8omyJg4.png
is not working this way because it will erase all revisions and if at if any file was not changed in last 100 commits - it will gone and project will become incomplete.
expected result - clean up server space and have each file from depo residing in before this clean up.

fluid kestrel
#

rider added a plugin to my project under the Developer I wonder if I can't move it to the engine so that it doesn't pollute our git

#

FileSystemMappings.ini keeps getting changed every time a different user opens the project which creates an extra modification to commit every time as well

#

(and it shows the installation path of Unreal as well as the project path)

woven sluice
#

This will indeed limit your ability to go back in time so I wouldn't do it if your project is mature, but it can be handy during alpha/beta dev in my opinion

tidal skiff
#

(Perforce, DigitalOcean)

Hi, I am trying to set the P4ROOT to volume.

  1. edited /etc/perforce/p4dctl.conf.d/perforceservice.conf from P4ROOT = home/perforceserver/root to P4ROOT = mnt/volume/perforceserver/root
  2. copied the contents from original directory to new directory.

but when I do step 1 I get this error:
'perforceservice' p4d: '/opt/perforce/sbin/p4d -p ssl:1666' exited with status 255.

in the docs it is mentioned that the error is shown when there is issue with privilage, ownership or configuration.
I checked the privilage and ownership, using
-ls -l root for both new and old, and compared it is same.

so can anyone point me in the right direction. thanks in advance.

merry verge
#

that directory and everything under it should be owned by the user as specified in your .conf file

#

also, you can manually run p4d for that server to see what the specific error is

#
su perforce
cd /path/to/the/server/directory
/opt/perforce/sbin/p4d -p ssl:1666
tidal skiff
#

ya 'perforce' is the owner . it was root previously but changed it to perforce

#

how can i see the logs? if that will help.

merry verge
#

running p4d manually should log things out to the console

tidal skiff
#

ahh ok

merry verge
#

and hit ctrl+c to exit (if it doesn't exit on its own due to an error)

tidal skiff
#

i'll try

#

/bin:/usr/bin:/usr/local/bin:/opt/perforce/bin:/opt/perforce/sbin
this is the path right?

merry verge
#

that's the PATH environment variable

#

not relevant to testing this

tidal skiff
#

I am not sure what the path to the server is.

merry verge
#

you literally have it in your config file

#

/mnt/volume/perforceserver/root

tidal skiff
merry verge
#

yes

tidal skiff
#

thanks

merry verge
#

cd to that directory when you're the perforce user, run p4d, see what error you get

tidal skiff
#

wait a second to i have to copy from home/perforceserver/
rather than just home/perforceserver/root?

merry verge
#

I have no idea what your filesystem is like

tidal skiff
#

hmm

merry verge
#

but you need all files from the server, and yes there are generally files above the server root directory

tidal skiff
#
perforce@UTMserver:/home/perforceserver$ ls
archives  journals  logs  root
#

ok i am moving archive, journals, and logs as well

tidal skiff
merry verge
#

Yes, but those are when running under p4dctl. Set that env var manually when testing yourself.
Or just see if p4dctl works now.

tidal skiff
#

ok

tidal skiff
winter parcel
# woven sluice The traditional way to do this is to set the filetype of the files. You can edit...

sorry, dont got you here.
I am asking about how to keep e.g. only latest revision per each existing file in ue project.
I have a file history like that https://i.imgur.com/x3yfO1u.png per each changed file in the project and I 100 percent sure I dont need file history for 1 or 2 years ago.
e.g. I want to keep only latest one, or few latest ones for specific revision or date etc. but this tool https://i.imgur.com/8omyJg4.png is not working properly.

woven sluice
winter parcel
restive ferry
#

Is anyone successfully using git lfs with a largish repository? Something in the 100-500gb range.

quaint obsidian
#

in total history or repo footprint?

long hull
#

My git repo is ~500GB, but it's all spread across submodule, so no individual repo is >20GB. Not sure if it helps...

white bone
#

Hi, I'm not so sure where to ask this, I can't seem to find the appropriate channel, so I'll ask here.

So, regarding project repositories, I'm currently using TortoiseSVN and at very rare occasions it just breaks and I need to download the whole project again.
So my question is... is there a way I can let the SVN know that I already have the project files downloaded? so that I don't download it again?
(I'm referring to the hidden .svn folder).

If you know the answer, please please @white bone Tag me.

restive ferry
quaint obsidian
glad kindle
#

when i moved my project to 5.1, every time i open it now, source control takes like 40 seconds to specifically restore my StarterContent folder. does anyone know why this is happening and how to make it stop? 😭

quaint obsidian
digital flint
#

Does anyone know what this error means?

vale basin
#

Well it's a platform not supported exception. So what OS are you running? And what git tools have you installed?

#

Actually it's a dot net issue.... Strange. What os? And what versions of dot net do you have installed?

digital flint
#

yeah, i don't quite understand it, cause setup.bat supposed to download the dependences, not sure why it is complaining about platform

#

@vale basin Windows 11

#

i didn't install the DotNet, i believe it install when i installed the Unreal Launcher

#

git is just the git bash 2.39.0.2 64bit

#

I try to remove everything from my OS, from VS 2022 and all the packages and reinstall everything, still getting the same error

#

all these also installed

#

VS 2022 v17.4.3

#

there is this error as well

#

when i try to GenerateProjectFiles.bat

#

DotNet v6.0.302

digital flint
#

looking at the Setup.bat line 13 is executing the GitDependencies.exe

haughty yoke
#

is there anything better storage wise than droplets if i want to keep perforce server costs under like $20/month

limpid obsidian
#

@haughty yoke a droplet is their marketing term for a vritual machine (running on shared hardware)

#

if cloud hosting is too expensive, run the server off your PC and use port forwarding on your router

cyan jay
haughty yoke
#

max down/up for my home is 300kb/s / 45kb/s

#

fiber lines have been laid to the area but ISP has not told us that we can order it yet

#

i have a nice pc with an 8tb HDD that would be great for running a remote server as its already up 24/7

#

its other hardware is crap

#

but its dual core 1.3ghz, and 12gb ram

white bone
#

Hey, please @white bone tag me if you know the answer to this.

Is there a way to use Totoise SVN with Google Drive as a repository?

teal bone
#

Probably not.

woven sluice
#

Maybe you can mount a Google drive on a linux vps but that sounds like it might just be a good way to have a frustratingly slow version control system

woven sluice
#

Careful. We have to tread the fine line between making this sound bad enough that they know they shouldn't do it, but not bad enough that they decide not to try it

nova heath
#

Free

#

And no size limits

#

Just make sure you set up Git lfs

nova heath
#

Yea I meant it more as an alternative, but true the ultimate cost reduction from something paid is something free 😄

haughty yoke
#

I thought perforce was free up to 5 people

#

We currently have 2

merry verge
#

it is

#

it's not free past 5 people, and you have to pay for the initial 5 at that point

merry verge
#

That's false

#

There's no limit on data as you provide your own server and storage

#

20 workspace limit is true though.

haughty yoke
#

If its 20, theres no way i will hit the limit

#

And the rig i plan to host on has 8tb

mighty light
haughty yoke
#

Only $120 for each additional 8tb

haughty yoke
mighty light
haughty yoke
#

Epic

#

Once isp hooks up my fiber i have a dedicated desktop

haughty yoke
#

What hardware is needed for a server?

#

I have 1.3ghz dual core with 12gb ram

mighty light
cunning stream
#

I'm new to using source control and github. I just created a repo on my PC and pulled it onto my laptop. I get this error when I try to open the project:

#

Can anybody help?

#

Happened after enabling LFS

bright quarry
#

Anyone knows of a good P4 tutorial that clearly shows how to manage updates of the source with local changes?

teal bone
quick gorge
#

Hey, i plan to work with a team on perforce, i need a easy use onlmine service for perforce (i need 500gb storage) anybody know a service like that?

merry verge
#

Assembla is the only managed hosting provider for perforce and it is not cheap.

mighty light
#

would be much cheaper to buy a NUC or something with a 1tb ssd

vale basin
native igloo
#

Has anyone set up perforce core on an EC2 instance? I don’t have extra hardware laying around for a server

cyan jay
iron bronze
#

just make sure the VM is running on top of an x86 CPU, I don't think there's an official ARM build for p4

summer horizon
#

anyone else have an issue with uassets not being part of the project (files are still there) after pulling down from git?

#

the project works fine on the computer I originally worked on - it's like the level/map didn't save also

#

oh, nevermind, I opened the wrong project 🤣

#

carry on

quiet stratus
#

@Forgemaster_Alex#4890 🤣🤣

urban hare
#

Hey! I'm looking to redux my workflow with source control as I've always been a bit unorganized. Do ya'll generally keep art assets away from engine assets? Or do you put files like your blender/maya project files in source control next to your UE projects? Opinions on git or perforce welcome, I have projects in both.

vale basin
#

Depends on the size of the team. I've heard of larger teams having a repo dedicated to source assets. Smaller teams i've worked on just have a folder in the same repo dedicated to source assets

#

this is in P4 btw, so you can just strip out source assets from a workspace view, so people that don't need them don't see them

urban hare
#

That's interesting, right now I just have a massive NAS sync folder that everything lives in but I want to seperate everything out into their own project disciplines.

I prefer git for code, but P4 is great for uasset file locking. I'm just afraid that .blend files are going to baloon a git repository pretty fast with LFS

#

I think I like your idea of having a repo specialized for art assets

vale basin
#

it's probably the more scalable option to have separate repo

merry verge
#

For perforce I don't see a reason to use separate depots for different source assets. Just throw things in another folder.

#

Makes it easy to see when someone is making changes across different types of things.

#

Git may be another story depending on your setup.

vale basin
#

yep and don't have sync issues with the 2 repos being at different version levels

#

But i think from juggling storage and future maintenance it may be a bit easier to have separate repos. Really depends on your team, workflows and sysadmin skills

urban hare
merry verge
#

I disagree with it being "amazing" for the purposes of gamedev, but that's not what I was talking about.

urban hare
merry verge
#

The least amount of work for a solo dev is almost always going to be a single repo. Just be organized in how you store things.

#

If you have multiple interrelated repos it's just going to be annoying to manage. You don't get any benefits of splitting things up that way.

vale basin
urban hare
merry verge
#

I'm not sure how that's a lot of baggage to maintain.

#

Unless you're specifically complaining about people having to download assets they don't need - which is solvable depending on what source control you're using.

#

Perforce has virtual streams and workspace views. Don't need art source? Don't map those folders to your client.

#

Git is more complicated.

#

Larger studios generally work in a single repo (also see UnrealGameSync for working with perforce - it helps with this and more).

vale basin
#

Does Git+LFS support "views" so you can ignore subfolders? That might make single repo tricky. That's a great feature of P4 so you can narrow scope of repo's locally for people

merry verge
#

Not natively

#

partial checkouts are a separate concept

urban hare
#

What I mean by baggage, and I meant potential baggage in a repo, is 8 years of blend file changes that would have compounded in repository size. I'm basically trying to find a solution that lets me do version control while leaving myself open to managing space requirements down the line.

merry verge
#

Separating out assets wouldn't help with that

#

In perforce you can set filetypes to restrict the number of revisions on specific types of files (or even just in certain folders)

#

Git I don't have a good solution for you but a separate repo doesn't help either.

urban hare
#

no reason I couldn't set that up for art assets

#

hmmmmm

vale basin
#

an 8 year running project sounds problematic. If you have a single repo with MULTIPLE projects, i'd separate those out. If otherwise your project is really bloating over that, you might consider something like starting a new repo when you do an engine change or something, somewhere sane to clip history and leave the old repo online for a while

urban hare
merry verge
#

Ok, that's a whole different story

vale basin
#

if those assets are in a shared library (sounds like the are), seperate repo sounds good. If they were highly project specific, with the project sounds better

merry verge
#

I would store source assets with the projects themselves. And maybe a "shared" depot mapped into client views if you want to share things across projects.

#

And use file types if you want.

urban hare
#

I'm currently separating everything out to their own projects that they are a part of and I'm looking to do version control of some type to facilitate non-destructive changes

#

Thanks for the feedback, sorry I wasn't clear lol.

#

I think the solution for us is to have a dedicated art depot in P4 with each project seperated out for version control.

#

That'll let us trim out old revisions when space fills up.

#

(again talking only art assets that are attributed to specific projects, and a depot for art assets that span multiple)

daring carbon
#

does git and github work well with ue5?

olive stump
#

hello, any solutions for this problem?:

native igloo
#

Question on perforce - say my entire project is 10GB

If I make a change and push the changes into my main stream, does it make a copy and now my project stores 20GB (because it saves history of project?) or can someone explain to me how that works

woven sluice
#

It saves the full history of each file individually, as any individual files are pushed/updated

urban crystal
#

there's compression + deduplication so it wont literally double the size

urban hare
daring carbon
urban hare
#

'Codespace' is where things go that are not on LFS, which is much more limited because it's meant for.... code lol, classic git repository situation

urban hare
#

npnp

somber kraken
#

Hey guys how can I "obliterate" a submit: in P4V

#

I tried different variant of p4 obliterate...

#

ask chatgpt and it worked... for those wanting to do that:

p4 obliterate @revisionNumber

digital shadow
#

I just set up a repo using GitHub that contains three projects for our dev team to work on. As of now, we are using simple good ole github desktop to work with everything. Is there a better way to do this that allows developers to check out/in assets? I have tried to use the built-in function from UE5 but it does not seem to do much at all, any help would be extremely appreciated as there is barely anything about UE source control online

vale basin
digital shadow
vale basin
#

that's an interesting workflow.... Can't really offer advice on that one

digital shadow
olive stump
#

what can cause this problem?(fixed - wrong mapping)

mighty light
olive stump
#

what exact command should I use then?

cyan jay
olive stump
quaint obsidian
#

also id recommend migrating to a branch workflow, might be painful with the asset paths you have

limpid obsidian
#

curious how many ppl use classic depots vs. streaming depots in p4?

woven sluice
#

i would bet that 95+% of the people who know how to use streams depots use streams depots

limpid obsidian
#

how is locking binaries across streams handled? (when developing with vanilla versions of the engine)

merry verge
#

you don't lock across streams

#

Which is the correct way to do things. Assets should not be modified concurrently across multiple streams that need to merge together.

#

But you often need to make modifications on a stream that does not merge back into your main branch (for example, a one-off bug fix on a release branch). Locking would prevent this and make it impossible to make patches on release branches.

#

In general, asset modification should only be done on your main branch or on branches that will no longer merge from main.
With an exception for new assets being created on development branches - they're new so they won't cause problems, and often new content requires work done on the development branch anyway.

woven sluice
#

it's worth mentioning that there is a way to code in stream names for the editor to display asset statuses from other streams. so if your team actually does multi-stream dev, if someone modifies an asset on any stream, someone working on the other stream will at least see a warning icon on the asset indicating that it's been modified elsewhere. but this doesn't change the ideal situation, which is what siliex said - mainline contains 100% of your forward-moving binaries, everything else is short lived

sinful heath
#

question. im using github and i was wondering if you can setup two repositories with one project. for example i have a private repository for my project that contains everything, then i want a public repository with a separate gitignore file so it doesnt push stuff i dont want others to have/see. is this possible?

quaint obsidian
# sinful heath question. im using github and i was wondering if you can setup two repositories ...

not exactly, can do this instead each time you wanna publish to the public one https://stackoverflow.com/questions/1662753/export-subtree-in-git-with-history

faint cairn
#

I have a lot of annoying small issues with the Git Plugin lately.

  • Files being Checked Out but not marked writeable, randomly failing to save and having to be marked writeable by hand.
  • Files not being removed. They are removed from UE, but not from the Window File Explorer, as well as not listed for committing. It takes a restart and a second wave of deleting them to really get rid of them.
  • Files not checking in when committing. I'm committing to a StatusBranch (dev) and some (or all?) files that were checked out stay checked out and have to be manually reverted (which often crashes the editor, cause UE loves reverting files).

Anyone else having issues like these? Is that a setup problem or is it just unstable?

long hull
#

I gave up on the Git plugin (before mcoms' fork), but yeah, it had quite a few quirks back then. Now I only enable it when I have no choice (e.g., managing shared collections), and I just use the CLI (which is what I do for my non-UE projects, so it's actually better for me).

faint cairn
#

Well it is ProjectBorealis one that I use :P

quaint obsidian
faint cairn
#

Thanks, enjoy your vacation

quaint obsidian
#

Not to invalidate your experience by any means, but I don't think I've seen these issues before. In any case, the plugin needs to be robust and handle problems like this. So I will see what I can do

faint cairn
#

Yeah that's why I said maybe it's a setup issue on my end

quaint obsidian
#

If you could, please make issues on the GitHub regarding these and provide any more information about your setup that you may have

#

If not, I'll be able to make the issues when I find the time

faint cairn
#

And it stays readonly fwiw

quaint obsidian
#

ill assume marked for add assets are made read only at some point because they're not checked out for now

#

and see if i can solve the problem with some quick edits

faint cairn
#

@quaint obsidian I don't want to post "maybes", as I dislike getting Jiras like this, so I will only report things I noticed with a high consistency.
So this is the only thing I noticed in terms of repro steps.

#

It's def marked as "checked out" because I can't mark it as that anymore. Yet it's read only.
I did have cases (not sure if same repro) where it was marked for add but not checked out and I could resolve the saving/read only issue by checking them out.

#

But most of the time that option doesn't exist and I could only revert, which is of course also not ideal

quaint obsidian
#

okay will check it out, thanks, a git status --porcelain could also help to see what the exact status is in git for me to debug, if you're okay with sending to me

faint cairn
#

Sure, next time I encounter it I will try that

vague whale
#

hello, i setup a perforce server and yes its really good to collaborate with my friend, but the login time out is way too short, after a bit of googling i couldn't find a setting to tweak, anyone know where this is set?

slim junco
mighty light
vague whale
mighty light
vague whale
slim junco
quaint obsidian
slim junco
quaint obsidian
#

the git plugin is not working?

slim junco
iron bronze
quick swift
#

hey folks i have a question ^^
im using git for version control and its pretty simple what i want.
my goal is to keep the folder Final_Game tracked and the other 3 Folders i need to work with excluded and i tried it inside the .gitattributes file with this commands:

#

but it wont work and its pretty hard to find something not totaly confusing online so can anyone explain me in short whats the prob and how i can achieve what i want ? thx alot 🙂

quaint obsidian
quick swift
#

its inside my Content folder so im not sure its useful ?

quick swift
bitter sun
#

Anyone using GitKraken ever encountered client not opening at all? I contacted their support (I have pro version) but got no response since a week

vague whale
iron bronze
vague whale
#

Oh group, weirdly I hadn't looked there

drifting mason
#

How do you use source control to collaborate with others? I'm trying to work on a game with someone and I'm not sure how we can both make changes to the project at the same time. We're both new to Unreal and GitHub which is what we're using

urban hare
# drifting mason How do you use source control to collaborate with others? I'm trying to work on...

Code is pretty easy and what Git was made for. You should do some googling and self-research, but the gist is communication. Work on separate things, and when someone makes changes you pull their changlist and 'merge' it into the branch it belongs. Once you satisfy any merge conflicts, you can then push the now-merged branch to github. You can also rebase someone's branch which is a more complicated topic, but its basically taking all of your changes and putting them after another's. Again, google and do some self research as code is what Git was made for.

Use git-lfs for binary assets like .uassets. Git is weak in this, make sure only one person is modifying a .uasset file at a time because you cannot merge those. It's one or the other. Perforce has a technology built for this which is 'file locking', something that git-lfs isn't fully ready for yet. In perforce, you checkout a binary file and no one else can touch it until you check it back in.

Complicated subject but that's why every team has a standardized workflow.

drifting mason
#

Alright thanks I'll research into that
What's "git=lfs"? Is "lfs" an acronym?

#
Git Large File Storage

Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.

vale basin
#

Yeah that's it. it's an extension to solve the problems of working with large binary files in git.

urban hare
#

Question of my own: I don't usually connect UE to a git source control as I do it manually via terminal, but I'm trying it out. Every time I launch UE5.1 it spends a large amount of time "Checking source control status of <asset>" and seems to work through my entire content tree. Is this normal behavior and is there a way to reduce it?

quaint obsidian
#

Git LFS has locking and works fine

quaint obsidian
urban hare
quaint obsidian
#

use this

urban hare
jade current
#

I am setting up a self hosted -perforce- source control. I have the core and p4V but could not connect from client to my localhost:1666 . Do I need to identify the server or is there any specific settings to let few users use my local disk as a server?

cyan jay
fair monolith
#

I finally bit the bullet and moved from Git to Perforce for Unreal development.

#

I documented how I set up a new Perforce server for a Lyra 5.1 project in case it is of interest to anyone:

jade current
#

Or is there a way to turn around the IP problem?

woven sluice
fair monolith
cyan jay
cyan jay
#

Or best solution IMO is a cloud host.

crystal prawn
wheat mica
#

does anyone know how to remove this read only popup on rider? I'm using perforce

fair monolith
#

You can either use the 'using file system' option, or you can enable Rider support for Perforce. (VCS -> Enable, choose Perforce).

#

If you enable Perforce support, you'll still get the same popup from Rider, but you will be able to choose the "Using version control integration" option and choose the specific changelist you want to put the file into.

#

The only other way to get rid of that popup is to forcefully remove all read-only flags from your source, which could mess up Perforce. (Not sure tbh)

#

(Presumably if you check the "dont show this again" box it will just always clear read only on the filesystem? Probably not a great idea)

iron bronze
fair monolith
tidal cosmos
#

I could use some help. We tried to set up Helix Perforce, for Source Control, but while the other two developers could connect just fine, I could not, and the only difference is, I am IPv6. We tried to solve the solution, but it did not seem to work. Also, it seems that while connected to Source Control it really slows down the UE for other developers.

What I am looking for, and we do not care about price, is a Source Control that is easy to install, use, and can accept larger file uploads and I mean large I would like to have more than 15GB upload capacity. Any suggestions?

#

Please tag me or DM me Im in a lot of Discords

wheat mica
#

i can see settings here

#

and I checked AllWrite on p4v but the "using version control integration" is still unavailable

woven sluice
slate pulsar
#

Do you guys have any tips to not have a metric ton of files using github source control and world partition?

crystal prawn
#

should I include in git .vs folder? in case I do have c++ project?

crimson lynx
#

Moving an asset seems to always result in a different binary content. Does it mean it's basically impossible for VCS to detect an asset move in order to construct full history of the asset?

mortal vale
#

Anyone willing to share a .p4ignore for a source engine?

silver token
mortal vale
silver token
#

If there are, I haven't noticed. But my entire data is is 1: me, so take that for whatever it might be worth.

mortal vale
#

What size on Perforce should I expect the engine source to take? Having compiled bins would be nice to have for designers/artists, who will not be compiling themselves.

pearl egret
#

Guys, has anyone tried pipeing on batch to svn command?

#

Right now I'm trying to figure out how to make this work
where /R "%cd%" *.uasset | svn unlock --force
I want to unlock a folder with subfolders(1st layer is only subfolders without any files) but when i use svn unlock i get svn: E155008: The node 'Path_To_Folder' is not a file

pearl egret
outer kiln
#

anyone else having trouble with plastic svn from github,its unpacked into project dir, but on starting project from launcher : ---------------------------
Missing MyProject Modules

The following modules are missing or built with a different engine version:

PlasticSourceControl
XmlParser2

Would you like to rebuild them now?

Yes No

, is this a knownissue or do I have it installed wrong....HMM

#

building of course does no good

#

if I have to build manually anditworks, so be it

#

oh sorry, version 1.7.0 zip

#

into project folderf

#

folder

pearl egret
#

Have you tried to just generate project files and then build the project with it?

outer kiln
#

gen project files ?

outer kiln
#

UGH@!Q

#

SO sorry,,I flat MISED there were versions for each engine LOL!!!

#

der der

#

o_0 🙂

outer kiln
#

gah, ok Plastic 1.7.0 installed in 5.1 project, but engine can't find CM command line tool, is that a common error ?

#

I see no cm.exe anywhere ininstalled path for plugin either

#

Failed to launch Plastic SCM 'cm' command line tool. You need to install it and make sure that 'cm' is on the Path and correctly configured.

#

mine is just in project folder which is enough supp0osedly

jolly fog
#

moved this to automation

outer kiln
#

I don't see it there

jolly fog
haughty wedge
#

Newb here. I love using git. I want to use git for my code, so it can be hosted by itself on github for collaboration, etc. But I also wanna use perforce, to learn it and because perforce.

Stupid question: Is there a free cloud hosting for perforce? Lollllll spoiled by github

outer kiln
#

well I have cm status showing a running head, whew finllay, but ue5 project still can't SEE 'cm.exe', ????
sigh
source control is a giant headache sometimes

#

cm as in plastic svn installed, but in ue5.1 , trying to connect to plastic SC, never can find 'cm'

#

yet in cmd, cm worksfine

jolly fog
haughty wedge
#

ah so it is only free for self-hosted? Not only is the cloud server cost incurred, but also a licensing cost for helix?

jolly fog
#

no

#

helix core is free for use of 5 users and 20 workspaces

#

but you incure hosting costs

lapis lintel
#

What’s the easiest way to have source control between just a pc and a laptop they will be on separate networks often they won’t really be working on the project at the same time. Preferably free

haughty wedge
#

thank you droid

jolly fog
# lapis lintel What’s the easiest way to have source control between just a pc and a laptop the...

to be honest, the easiest and quickest, free way would be something like syncthing or nextcloud, neither of them are true source control but work for syncing files between clients/servers and you can set version keep times too. otherwise the easiest and also free way is to self host a gitlab instance on a spare computer, it's free, but it takes time to set up and works exactly like the paid service or github but it's under YOUR controll

haughty wedge
#

anyone have experience with helix4git?

#

what is c++ source control like in perforce/UE to begin with? I would wanna use git for code if I can

jolly fog
# haughty wedge thank you droid

no problem, one thing about perforce though, depending on your actual needs, can be a blessing or a curse, HARD file locking, it's nice in some cases but can be a pain too, it means you have to actively checkout each file before you can make edits, not make your edits and ask the server for permission to upload.

#

same goes for the c++ programing, you have to checkout the file you want to edit before y;ou can.

haughty wedge
#

yeah I wanna learn the workflow in case I wanna get a job in game dev lol

jolly fog
#

gotcha, not a bad idea. personally, as a developer, i hate perforce specifically because of the file locking, the workflow isn't that different from git, it's ui is different, the terminology is different, but does almost the exact thing gitlab/github does, they just added hard file locking and instead of having code everywhere, all the files are stored in one place, not across the users and verified by the server after the fact. also a blessing and curse, if that one directory goes down, noone can edit, but at least the code is secured when it works.

haughty wedge
#

yeah how does the code/committing normally work with perforce compared to git? I kinda wanna use git for code alone, whether via addon helix4git or just by clobbering things together with the typemap. thoughts?

jolly fog
#

the commiting is similar, but you have to do some steps in reverse, instead of creating the changelist and present your changes for review by git, you have to ask permission to edit every file you think you need first and make the change list before you even get your permission to edit, and only when thats done can you edit the files you specified, then you can check in the files and they are added to a cue for review and whoever has admin rights can do a merge to finally commit the files

#

as for my thoughts on git vs helix using git for code and helix for large files and basic transfer.... it could work, just seems more fragmented and less efficient, maybe thats where that helix4git plugin comes in, idk, but seems unwise to seperate a project into two version controls unless it's to duplicate it and have a copy

#

m2c

#

depending on the size of your project, if it were me, i would have a folder for each, copy the entirty of it to each and work through the workflow on each to see what's best for you, that way, you don't affect one code base by editing on a diferent workflow where something could go wrong

#

i do like perforce's large file hadleing better than git's git-lfs but it's not enough for me to give up gitlab's "personal opinion" superior redundency if used properly

haughty wedge
#

thank you very much for your thoughts

lapis lintel
#

im also not opposed to something hosted if its like 10 bucks a month

lapis lintel
#

Is 50 gb enough for a perforce server?

vale basin
#

That depends entirely on your project

#

If it's a small project, yeah, but you might have to obliterate versions older than some timeframe in the future though to clear up space if there's heaps of revisions and a long development time frame

jolly fog
lapis lintel
#

I was looking at the like 15 dollar version of digital ocean that gets you 50 gigs and I wasn’t sure if that was enough

jolly fog
#

To give you an idea though, my project for a simple mp survival is around 20 gig just for the code and assets, that doesn't count the actual builds(release candidates)

#

And another example, the first versions of apex and ark were around 114 gig

#

And to answer your questions about the asset imports, no, it wouldn't affect it much as once it's imported assets are rarely touched and since unreal engine 5, the one file per actor feature makes the changes small and fast, only if you have something like a 1gig skin for your character and your editing it alot will that ever to be a problem

jolly fog
#

Google and gitlab have a working relationship to make the credits viable

bold hawk
#

having the same issue on a brand new 5.1 source build, keeps failing here. How did you get past that?

jolly fog
#

Perforce's workings is based on the premise of Central storage, your client essentially downloads a shadow copy of the files that you "check-out", edit and have perforce verify changes, that way the source code is secure and Noone can open/edit without permissions where git, everyone has a full copy of the project and can edit as they please but needs permissions to commit the changes

teal bone
#

What has that to do with what I said?

#

He said his projects are on different networks that, I'm assuming, can't talk to each other.

#

And he wanted his central storage to be on one of them.

#

That sounds like a chatgpt response. 😦

jolly fog
#

I was mearly pointing out that you don't actually share anything with a perforce SC and it sounded like he wanted the files on both computers, and as long as the perforce server has access to the internet it doesn't need to be on the same network

teal bone
#

Still don't see how that's got anything to do with him being on unconnected networks.

oak dune
#

Anybody know the command for p4 / perforce / helix core for force submitting a change to master and ignoring errors? (artist did an overhaul and deleted files without p4 use, so I'm just at the point of wanting to say "use their local, and ignore all errors trying to locate files"

iron bronze
oak dune
ashen plaza
#

should I put .sln and .vcxproj on gitignore?

mighty light
#

The one that comes with unreal source does this and is a good starting point

ashen plaza
#

if I enable a theme plugin (from the engine), will other collaborators need to have that theme aswell?

#

I want to enable electronic and darker nodes, just want to make sure

teal bone
#

Shouldn't.

ashen plaza
#

alright

#

also idk whats doing it but sometimes my github changes are using \ instead of / so i think when that happens it's ignoring the gitignore?

#

for example .vs/ is ignored but it pushed .vs\... anyway

digital flint
bold hawk
#

I just had to regenerate files apparently

quaint obsidian
quaint obsidian
#

then you need to remove them from tracking first

haughty wedge
#

Does anybody here have a setup with perforce tracking the entire project overall, and git repository for the source code, inside the perforce depot? I'm trying to wrap my head around a way this could work, so that coders can use their typical git workflow, but then when the changes are checked in, it doesn't break. lol.

I know that if we perfectly track the .git folder's contents, whatever .git folder is submitted by a coder is guaranteed to be a faithful representation of the contents of the git repo at the time they submitted it. How can that be harnessed to allow perforce to keep track of which git "snapshop" to use, and to make sure the git repo actually reflects that state and there are no unstaged changes left dangling by accidentally mixing and matching files from the repo with a different .git folder.

?? 🧐

mighty light
#

What did I just read

#

Did you say that you have a git repo inside of your perforce depot?

haughty wedge
#

I do not yet have any perforce depot but I'm trying to figure out how to do that. The git repo would ONLY track the c++ files.

mighty light
#

Normally you don't do this because it's the wrong way to do it. Usually you would use just perforce for code and assets, or you would use git with lfs for code and assets. You don't nest source controls

#

Any particular reason why you want to do it this way?

haughty wedge
#

one reason is to make it easy to distribute the code via github. Also, because coders very much are used to git and not perforce. also, perforce does not seem like a very good tool for working with code. Is it? git seems to have better tools but correct me if I'm wrong.

#

FYI, many projects are indeed collections of nested git repositories, so the concept of a repo inside a repo is not insane

#

but, perhaps, it is with perforce lollll

#

many other c++ software projects, I mean, not UE projects

mighty light
mighty light
haughty wedge
#

I mean, yeah perforce is great for binary assets sure...

#

with the typical perforce workflow, do coders checkout individual files ahead of time? exclusively?

mighty light
haughty wedge
mighty light
#

Only your binary assets should have an exclusive lock because they can't be merged

haughty wedge
#

ok so it sounds like perforce will be a nightmare for merging

mighty light
haughty wedge
#

that's why I wanna use git, so that the code can all be managed separtely but then referenced by perforce

haughty wedge
#

and a git submodule is basically exaclty nested git repos, but the main repo is "aware" of all the other repos

#

in the cases I'm thinking of, there is no use of submodules. Instead, the repos are just dropped righ inside, and each repo is tracked and managed separately.

#

This would, eg, separate the modular parts of your project. IIRC, google does this alot

mighty light
#

If you are 100% sold on using 2 different and incompatible vcs for your project, you are going to need a custom CI/CD pipeline to build and deploy your project so people can use it. Otherwise all of your artists and designers are going to need to know how to use both vcs and know how to build your project correctly

haughty wedge
#

hmm, I don't think that's correct

#

as long as the code in the depot is the intended code, what's the problem?

#

it adds complexity for people managing the code and/or the project as a whole, but why would artists care about a hidden folder .git?

#

the nice thing about git

#

is that you can delete the .git folder, and it just becomes a normal folder

#

all the files are there still.

#

building the project only needs the current, correct state of the intended git repo. nobody needs to change it if it is left in its correct state

#

I'm brainstorming here, please provide me with harsh criticism lol

mighty light
#

I already told you that it's not a good idea and gave you alternatives

haughty wedge
#

I'm not looking for alternatives though. I'm positive I want to do this. I know other game studios have done this.

#

I'm not even a game studio lol I'm just messing around and learning

#

so it's not critical if it works or not

teal bone
#

Do you want your source code in perforce and git?

haughty wedge
#

I think so

#

I imagine the perforce tracking of the code would be much less granular than the git tracking, tho.

#

So like, the coders go and do all their commits, their branches, their merges whatever. When the code is deemed appropriate for the depot, the changes are submitted to perforce. Perforce will track each larger set of changes as a whole, with a reference to the exact state of the git repo. The git repo is tasked with tracking all the tiny, many changes inside only the git repo

#

I'm thinking of making the .git folder treated as binary in perforce. Do file deletions get tracked reliably in perforce?

#

and can the entire contents of a folder be mapped to binary, regardless of file extension?

teal bone
#

I'm not sure you want the entire .git in perforce.

#

In fact, you probably don't wnat any of it. Just stick it on ignore.

haughty wedge
#
Typemap:
        binary //depot/dev_projects/...*

??

#

no I do really want to track the .git folder

#

that is the most important part to track. It represents the exact snapshot of the code at the time it was submitted to the depot.

teal bone
#

But it's already being tracked by the git server. Why track it twice?

haughty wedge
#

to track the intended state

#

because like a git repo can have many branches or states of the branch

#

the .git folder tracks the exact intended state, in relation to all the other states

#

this way perforce doesn't need to track all the little code changes just like

#

each large change to the code is tracked

#

Let's say the designers are like

Coders, we need feature X. Make X.cpp and X.h plz

#

coders are like ok. They make a branch of the current code, make some changes, test it out, and submit a their code on their test branch

#

to perforce, a WHOLE TON OF THINGS just changed at once, files added, changes made in places perhaps.

#

not granular at all.

#

But the git repo tracked all the little changes the coder made along the way, and tracks which branch of the code is submitted

teal bone
#

It would be granular if you made code changes directly through perforce.

haughty wedge
#

yeah

teal bone
#

I much prefer the git workflow, but it's entirely possible to do it on p4 and it's not hard at all.

haughty wedge
#

I imagine there is definitely a way to do it

#

and I will learn it because I will use it for binary assets and all other text assets besides the c++ code

#

but in order to share the project code easily on github

#

and for coders to be able to use git as they want, with pull requests and merges and diff tools like we all know...

#

And think of it like this way also

teal bone
#

You like giving yourself a headache trying to maintain 2 VCS instead of 1?

haughty wedge
#

Now the coders get their nice clean design history without a bunch of stuff about art assets

#

and the designers get a nice clean and concise design history in perforce, with large single changes to the code at one time

teal bone
#

You can have a nice clean history without a bunch of art assets in p4 already

haughty wedge
#

oh really ok good

teal bone
#

You have per-folder (inc subfolder) and per-file history views. It's quite useful.

haughty wedge
#

yeahh lol

#

that's the problem

#

the per file thing is kinda nuts

teal bone
#

How so?

haughty wedge
#

tracking changes in code is usually more about tracking the snapshot across multiple files at once, not individual little changes to each file

teal bone
#

That's where change lists come in.

haughty wedge
#

yeah

#

but git is literally better lol

#

let's be real

teal bone
#

Tbh, you just sound like you haven't used p4 before and you're trying to shoehorn your git workflow into it.

haughty wedge
#

and this does not seem hard to do... unless I'm missing something

teal bone
#

It's not worth the hassle. Just learn how to use p4 effectively.

haughty wedge
#

how is it not worth the hassle to allow greater control, and to onboard a more diverse set of coders onto the project easier?

#

u know how many c++ coders normally use perforce?

#

the number of c++ coders who are forced to use it

#

and none more

teal bone
#

You know how many c++ coders can learn how to use p4 in about 10m with a little instruction? All of them.

haughty wedge
#

that's not the point

#

I can learn to use a hockey stick to do surgery but it doesn't mean it's gonna be easy or fun or the best way

teal bone
#

Except that's exactly what you are doing.

haughty wedge
#

nah I'm trying to use a specialized scalpel alongside the standard set of tools

#

like, I'm learning p4

teal bone
#

No, you're used to using a mallet and you're trying to use it with a scalpel because you haven't learned to how to use the scalpel properly.

haughty wedge
#

nothing wrong with that

#

except... I'm not ignoring p4

#

p4 will be tracking the code too

#

wat is the problem u forsee?

teal bone
#

If it's not gonna be a problem, just set it up and get it working.

haughty wedge
#

lollll

teal bone
#

It's simple, right? You just add the ,git folder to p4 and off you go.

haughty wedge
#

there will be specific ways to do it that will work better than others. That's why I'm here to discuss it

haughty wedge
#

I'm tlaking about it from a p4 perspective here

#

we need to set up the typemap so the .git folder is tracked 100% accurately, and the contents of the repo in terms of the actual source files are always ensured to be kept to what the .git folder says

#

what that means, really, is that multiple coders could submit all sorts of random source code files, jam them up into p4, and as long as the .git folder represents the actual intended state of the code, git can ensure that the files match that.

#

I understand, you can achieve similar functionality with p4 alone

#

But, then, you can't track it with git

#

and I want to track it with git because I know lots of people who want to track code with git

#

and then the managers can track the code in p4 too, and they won't be saddled with the hundreds of commits that the coders would want to be making

#

and the code can easily be shared

#

someone can modify it, with no knowledge of the p4 depot, and the code can easily be integrated in with all changes tracked

#

p4 can still see "Code was like this. Then code was like this. Then code was like this"

#

but how the code got to each of those points can be tracked by git

#

The code on the p4 depot would be more nimble. Large changes to the code base could easily be swapped in and out with git, all managed in branches, so that you don't need a million streams in p4

#

basically, my question kind of comes down to, how to manage the checking in and out of the .git folder. Because it has to be tracked in its entirety accurately

#

everything else remains exactly the same in terms of p4

#

It must be disallowed for a coder to make a change to a source code file without also submitting the full .git folder

#

and, really, that means they may potentially be changing the entire codebase 😐

#

in the typical p4 workflow, is there like a pull request kind of convention or mechanism?

#

who approves the changes to be actually commited to a stream?

#

Ah, thanks ChatGPT:

Yes, in Helix Core, you can enforce that entire folders be checked out or checked in at one time by using the "atomic checkouts" feature. This feature ensures that all files within a folder are checked out together and checked in together, so that the contents of the folder always remain in a consistent state. To enable atomic checkouts, you will need to configure your Helix Core server to use the "atomic checkouts" option for the specific depot or folder that you want to enforce this behavior on.

#

so I would want to make the entire git repo, including the .git dubdirectory, to be an atomic checkout/checkin

#

but without exclusivity, so other people can also do the same

#

done

teal bone
haughty wedge
#

nice thank you

#

thank you for entertaining my zany ideas

#

but I'm pretty sure I heard someone talk about doing this in an unreal talk at some point

#

using git inside of p4

#

the great thing about setting this up will be that it can be abandoned at any time by simply deleting the .git folder and moving on

#

so it would make a good experiment with no real risk

teal bone
#

I would still not track .git inside p4, except maybe an initial state just so new people don't have to init the repo

haughty wedge
#

they wouldn't have to

#

if you don't have git, the .git folder is just a small hidden folder

#

it does nothing

teal bone
#

It's not a "small" hidden folder. It can get quite big.

haughty wedge
#

I mean, with binary assets

#

with a normal sized repo it will be fine

#

like code-based

#

because it stores binary blobs

#

it is very efficient at storing changes to text

#

286 MB .git folder for an entire software project

#

although

#

if p4 starts tracking changes to those binary blobs themselves it would get expensive on storage on the p4 side, but that doesn't happen I don't think. Maybe we would only track the files in the .git folder for a few versions

#

Can it be easily changed later in the typemap, the number of revisions that are kept for a binary file? Like if I decided to keep them all at first, and see if storage becomes an issue, can I easily change it down to keep like 10 versions only of those binary blobs? I don't know exaclt how git stores those blobs but I doubt they are changing much once they are saved. They either exist, or don't, IIRC.

#

this leads to my other question how well does p4 track file deletion?

teal bone
#

Why not just track the actaul source files?

#

Let git worry about git.

#

You don't need the git housekeeping files in your p4 repo.

fair monolith
#

My thought for how to do it (I haven't actually tried it yet):

P4 ignores .git/* database
P4 ignores all files stored in Git
Git ignores everything other than text stuff stored in Git

#

In this way any time you want to edit binary stuff, you're editing P4.

#

Any time you're coding, you're using Git.

teal bone
#

And for non-coders you can do a source code update every so often.

fair monolith
#

The problem with this is all the P4 people now need to git pull every time they update P4

#

I'm not a fan of that. Maybe there is a way to automate that inside P4?

teal bone
#

Or just distribute binaries.

#

(through p4)

fair monolith
#

Well it's not just CPP that IMO should be kept in Git

#

The .uproject should be kept there too

#

It's text. Zero reason to keep it in P4

teal bone
#

You can keep uproject on p4, because non-coders need it too.

fair monolith
#

Right that is why P4 users would have to do a git pull

#

To get the .uproject, .uplugin. .ini files, etc

teal bone
#

But p4 users should never have to use git.

fair monolith
#

Which is why I haven't actually done it this way yet.

teal bone
#

What's the point of this setup if p4 users have to use git?

#

Why not just use git and lfs?

fair monolith
#

Exactly

teal bone
#

I would put just source/ in git

fair monolith
#

Then you have to edit both Git and P4 if you're a developer, any time you want to change the uplugin/uproject/ini files.

#

The point of this is to make things easier/better for c++ devs, not to throw more road blocks at them.

#

Maybe there is a better way to do it. Given my limited knowledge about P4 this was the best solution I came up with if you really want to use Git for C++.

#

And I don't particularly like it so... lol

teal bone
fair monolith
#

I don't change the plugin/project files all that often but I do try to keep things in INI as much as possible.

teal bone
#

Honestly, I don't think I've ever changed anything in an ini file and committed it.

#

I don't work on games, though.

fair monolith
#

Anyway I definitely think you do not want to store the .git database inside P4, if you decide to combine the two.

#

P4 is already keeping copies of the files, it doesn't need copies of the (supposed-to-be distributed) Git database.

#

That theoretically exists on many remote machines anyway, P4 would be needlessly redundant.

teal bone
#

Indeed.

#

The "state" he talks about is literally the files themselves, not the housekeeping files.

fair monolith
#

All the files inside .git are the Git state, to allow you to swap from one version of the file to another without contacting the server.

#

It's one of the reasons Git is so much more vastly superior than P4 for C++ work.

teal bone
#

I'm talking about a different type of state.

fair monolith
#

I think we're probably trying to say the same thing in different ways.

teal bone
#

He mentions the "intended state of the files" and I'm saying, the intended state of the files is the files themselves in the current checkout situation.

#

You don't need anything else to tell you that.

fair monolith
#

Right, well that is P4's philosophy about the state of files anyway.

#

One of the major differences between working with P4 and working with Git.

teal bone
#

From my usage of p4 the main difference is branches vs cls, though I don't think we use p4 to its fullest potential in that sense.

#

The state of files is pretty much the same, just one is kept on the server (p4) and one is stored on srever and client (git)

haughty wedge
#

interesting thank you xist for giving your thoughts

#

I'm going to try doing it with p4 tracking the .git folder, so that you don't have to lol

#

and if it is crazy I will let you know

#

the reason I really want to include the .git folder is because it keeps direct link to the actual state of the repo it's from. without it, it's just a bunch of files in p4.

fair monolith
#

My concern there is the .git/objects folder contains every revision that has ever been saved in Git.

#

Which if you copy that into P4 is going to be lots and lots of redundant copies.

#

No major issue, but a lot of file bloat.

#

Maybe in P4 you should link it not as an actual Git repo, but as a Git submodule.

#

That would allow you to tag a particular Git revision.

haughty wedge
#

yeah there will be some bloat in a sense, but the .git folder only stores the local state of the repo when it was submitted to perforce. So, it will be small at first, and only grow as the project does. Then, it can be cut down with a hard fork

#

but this way it will be able to recreate perfectly the state of the repo and branch, if the files get messed up somehow or just to be sure

#

and you can just open up git gui or bash and find out exactly where in the git repo the code is from

#

if you don't have git? Just ignore the .git folder and don't modify code

#

tracking uproject files and ini files is interesting. I suppose that would depend, for me, on how closely it is related to the work of the people touching them. If they are primarily for coders, keep it in git too, but if not maybe just p4 for simple things

#

since everything is tracked in p4 and git, we won't really lose sight of things. But, it's important to note that p4 is the primary source control. Git alone won't tell you much about the rest of the project. Commit messages could include "soft references" to p4 changelists or streams or something. p4 changelists should definitely reference the commit hash of the git repo if relevant, just in case

#

also, back to my question, how does p4 handle file deletion? The .git folders could be deleted or re-added at any time to the p4 depot. If the file deletion is smartly tracked, bloat can be managed if need be.

#

But compared to the bloat of binary art assets and their iterations, this seems like no concern to me really

#

the main reason I want to do this is to make it easy to release code on github open source, and then other people can easily use it. If the code is only in p4, it's gonna be a bad time. lol

teal bone
haughty wedge
#

yeah that is true

#

p4 should be smart enough to not store the same data multiple times though, right? as long as the files don't change

#

git definitely creates unique binary blobs. the files are never written over, afaik. so the perforce storage footprint will be 1:1 to the size of the .git folder. right?

#

it's just like having the .git folder in your depot and it doesn't change. it just gets added to.

teal bone
#

As long as the files doesn't change, p4 will not make additional copies.

haughty wedge
#

so this should be fine I think

#

just have to ensure "atomic checkout" is enforced on anything tracked by git. the whole repo.

teal bone
#

As far as I understand it, the files in /object won't ever change, they'll just multiply.

haughty wedge
#

and then make p4 only take what has actually changed to reduce bandwidth

#

but make it so when someone checks it out, it's not exclusive

#

since git is tracking the code, who cares how many ppl change code or when they submit it to p4. it will always be coherent

#

I mean, how it relates to the rest of the project is up to the management of the project as a whole lol but at least the codebase will be the exact codebase that the coder thought they were submitting

#

and people will know who and how many ppl are currently working on the codebase

#

and then they can check github for the activity of those ppl, in their branches

teal bone
#

I'm starting to think bot might actually be a bot

haughty wedge
#

I'm ChatGPT 3.5

#

Actually ChatGPT has way better grammer, more coherent thoughts and a better sense of... smartness

teal bone
#

Oh, nevermind, I think bots can spell grammar.

haughty wedge
#

Bahaahah

#

my point exactly

#

to be fair

#

I asked ChatGPT if this was a good idea and they thought it would be fine

teal bone
#

Asking ChatGPT for advice on programming won't end well.

#

(or source control)

haughty wedge
#

Yes, it is possible to use Perforce to track the version history of the project as a whole, including binary assets, while using GitHub to track changes and version control for the source code text files.

Perforce is a centralized version control system that is well-suited for managing binary files, as well as large files and directories, it can handle large amounts of data and is optimized for performance. GitHub, on the other hand, is a distributed version control system that is well-suited for managing text files, such as source code files, and it has a strong focus on collaboration and open-source development.

In this setup, you could have a Perforce repository that contains the binary assets and other large files, and a GitHub repository that contains the source code text files. The two repositories would be separate, but the source code could be included in the Perforce repository as well.

It is also possible to have a Git repository reside within a Perforce depot and be tracked, including the Git head, blobs, etc. This can be done using Helix4Git, which is a plugin for Perforce that allows you to use Git as a client for Perforce. Helix4Git allows you to use Git commands to access and manage files in a Perforce repository, and also allows you to use Git's branching and merging functionality with Perforce.

However, it's important to note that this kind of setup can be complex, and it would require a good understanding of both Perforce and Git, as well as how to configure and manage them. It's also important to test and evaluate the performance of the setup before using it in a production environment.

#

lol just noticed it called git GitHub loll

#

same thing

#

and I think it lied about Helix4Git

#

that's not what it does LOLL

#

fraggin bot

merry verge
#

This can be done using Helix4Git, which is a plugin for Perforce that allows you to use Git as a client for Perforce. Helix4Git allows you to use Git commands to access and manage files in a Perforce repository, and also allows you to use Git's branching and merging functionality with Perforce.
this is actually quite literally what it does

haughty wedge
#

is it?

#

I mean, I think it's kinda accurate about what helix4git is: an interface layer from git to p4. But I don't think it achieves having a git repo inside a p4 repo, but using p4 as normal otherwise

teal bone
#

It does not.

#

It means you can use your git clients to interact with p4

haughty wedge
#

yeah thats what I thought. That might be useful in some scenarios but is not this lol

teal bone
#

Your original plan to combine the 2 was so that your programmers didn't have to relearn p4.

haughty wedge
#

no not really

#

the primary reason is to be able to easily share code on github

#

and with that come some nice tools and familiar tools for coders

#

the ppl using git will 100% have to also be using p4 all the time as much as they would if git was not being used

#

but they can use git to further track granularity of their code changes, and have more branches in an more efficient way than lots of p4 streams

merry verge
#

you're probably going to give yourself more headaches rather than less with this

teal bone
#

The headache argument doesn't work. 😦

merry verge
#

Often you want code and asset changes in sync - otherwise you have to come up with a solution for making sure builds have the correct changelist and commit. Sync content but forget to pull code? Oops you overwrote an asset with old data.

#

That's the whole reason helix4git is a translation layer rather than a git repository inside perforce.

#

I'm not going to tell you which one to pick, but I am going to tell you that using both is likely going to be worse than choosing just one. I'm not a fan of using git for assets but Git LFS does exist. Perforce won't work with github but swarm exists for code reviews and "lightweight" task streams exist for at least some of git's branching functionality.

haughty wedge
#

I agree with you in some senses, but I really feel like there is a compelling reason to go through the slightly extra steps to be able to have full control of the code alone.

The assets and code will be kept in sync by p4. We are not removing any tracking from p4. But we are giving the coders the ability to more easily jump around in their multiple branched codebases, submit exactly whatever code they want to p4, have that exact state tracked alongside the art/binary assets.

If it wasn't for my desire to potentially release some or all of the code as open source, then I probably wouldn't go through the trouble. But, if the codebase is all tracked in a separate git repo, or multiple git repos -- one for each code module. then those code modules can more easily be shared and incorporated into other unreal projects, and incorporated into their p4 history, while retaining 100% of code history across projects

#

but there are some huge drawbacks for larger projects: security

#

so yeah this is definitely not a good option unless the code is meant to be shared and not highly protected

merry verge
#

If it wasn't for my desire to potentially release some or all of the code as open source
it's not that hard to mirror commits from p4 to git by writing some scripts (if you don't find some premade ones).

But, if the codebase is all tracked in a separate git repo, or multiple git repos -- one for each code module. then those code modules can more easily be shared and incorporated into other unreal projects
You can already do that with perforce, arguably easier than with git. You can map multiple depots into a client view. So you could have a shared code depot. If you need per-commit control over when those shared plugins are integrated into a project then you don't map those depots into a client view but instead just treat them as separate branches and integrate upstream/downstream as normal.

haughty wedge
#

afaik, p4 history is only relevant to the server it was made on. no history can move over to a different server

merry verge
#

Depot != server

haughty wedge
#

git history is portable

merry verge
#

I'm not sure why you're talking about multiple perforce servers

haughty wedge
#

because if the code is open source, then another team with a different server might use it

#

they want the full code history

#

not starting from scratch

merry verge
#

I didn't say use perforce for open source work.

haughty wedge
#

they also wanna be able to pull in any code changes

merry verge
#

it's not that hard to mirror commits from p4 to git by writing some scripts (if you don't find some premade ones).

haughty wedge
#

mirroring commits does not preserve history

merry verge
#

Sure it does.

haughty wedge
#

it creates a new fake one

merry verge
#

So?

#

It's effectively the same thing.,

haughty wedge
#

it seems like the same amount of hassle as using git like I suggest

merry verge
#

P4 is your source of truth. Github is a nicety.

haughty wedge
#

that's exactly what I intend too

#

p4 is the source of truth for the project

merry verge
#

Then I'm not sure what you're even saying you want to do

haughty wedge
#

the code is also tracked separately so it can be portable.

#

Picture the code as one giant binary asset

merry verge
#

Err, what

haughty wedge
#

we wanna be able to take it and use it in another project

merry verge
#

You can already do that with perforce, arguably easier than with git. You can map multiple depots into a client view. So you could have a shared code depot. If you need per-commit control over when those shared plugins are integrated into a project then you don't map those depots into a client view but instead just treat them as separate branches and integrate upstream/downstream as normal.

haughty wedge
#

i mean not literally like a binary asset lol

#

I'm not sure exactly what the downside of doing as I suggest is

#

but I see lots of downsides of custom scripts mirroring commits to GH lol

merry verge
#

Honestly at this point I'm not even sure what you're suggesting. You want to mirror portions of your perforce depot into git which is exactly what I've said you can do.

haughty wedge
#

not mirror them.

merry verge
#

Yes mirror them. Even helix4git is "mirroring" them because perforce itself doesn't use the same concepts as git.

haughty wedge
#

if the git history is fully respected and not just imitated, then users across teams from many different projects can all cherry pick individual changes they want, and see the full history

teal bone
merry verge
#

oh geez

#

uhh yeah that sounds like a horrible idea

haughty wedge
#

how familiar with git are you guys? just curious because this does not seem crazy lol

merry verge
#

very

haughty wedge
#

a .git folder is not a big deal

teal bone
#

It's just totally pointless

haughty wedge
#

no it isn't!!

#

do you ever write anything down about your project on a piece of paper?

merry verge
#

you're now describing mirroring a git repository into perforce rather than the other way around. By making perforce commits every time the git repo changes.

haughty wedge
#

it's not mirroring

#

both the git and p4 histories are created together

merry verge
#

no, they aren't

haughty wedge
#

at the same time

merry verge
#

you make changes to git files and then commit them into perforce

#

these are two (potentially more) distinct operations

#

someone could make 10 commits in git then submit that as a single CL in perforce

haughty wedge
#

yeah

#

exactly

#

that's what I want

merry verge
#

except that won't work

haughty wedge
#

granularity in git, and easy overview in p4

merry verge
#

if someone else also makes changes in git, syncing the .git folder won't magically resolve the changes

haughty wedge
#

why won't it work?

#

it will if the entire repo is set as atomic checkout

#

that way the whole repo will be compared to the p4 depot

#

and ALL changes, especially to the .git folder, is submitted

#

as long as the hidden .git folder is accurate, we can know what the code is supposed to be and we can resolve or prevent issues

teal bone
#

Surely in this scenario it's down to the person making the p4 code update to make sure the code is what it's meant to be.

haughty wedge
#

yes precisely

#

it's up to them which branch of the git repo

#

or what they wanna submit

merry verge
#

except from what I know that there are a number of files in .git that cannot be merged on a p4 sync.

haughty wedge
#

the git repo allows them to manage that

haughty wedge
merry verge
#

except they do... or stomped

haughty wedge
#

if they are text files, they can be merged fine. other than that, they are binary blobs

#

the binary blobs don't change

merry verge
#

yes, and stomping those loses data for someone

haughty wedge
#

they only are added

#

why are they being "stomped?"

#

what is stomping

merry verge
#

stomp -> replacing the file rather than merging

#

there are a number of files in the main .git directory (not subdirectory) that are, to my understanding, required for git to operate correctly

haughty wedge
#

ah yeah

merry verge
#

and do change

#

and are not all text

haughty wedge
#

well I have a very good understanding of how thos files work

#

and I can tell you it should be fine

#

if p4 can track text files

#

and p4 can add binary files with no file extension

#

then we should be fine lol

#

it should be fine as long as we atomically checkout the .git folder

#

the entire folder needs to be checked in and out exactly as it is at any moment

#

it might be needed to disable compression on the binary blobs

teal bone
haughty wedge
#

easy with typemap

#

the coder doesn't need to pull unless they want to

#

the entire state of the repo is in the .git folder

#

one danger would be the potential for a coder to submit untracked changes, and then the changes could get lost for git... but the history on p4 is fine

merry verge
#

that's not the only danger

haughty wedge
#

one danger

#

ONE

#

UNO DANGERO

teal bone
#

The other issue here, siliex, is that bot doesn't actually know how to use p4.

haughty wedge
#

well yes I do

#

I have spent the past three days studying it lol

#

does it sound like I know nothing about it to you?

#

the issue isn't about "using p4"

#

it's about workflow best practices

teal bone
#

Three whole days?

haughty wedge
#

yes like literally

#

didn't you say it takes a coder 10 minutes to learn p4?

#

lol

teal bone
#

Sure. It takes a coder 10m to learn to use p4. Create CL, add code, submit, whatever.

#

You aren't a coder, you are a project manager.

#

Or a DevOps guy.

haughty wedge
#

sounds about right

teal bone
#

there's a massive difference.

haughty wedge
#

i mean, I use git to modify and release my own versions of open source software

#

soooo there is that

#

kinda makes me a coder?

teal bone
#

That's not the point. You can also be a coder, but your job on this project is not coder, it is manager.

haughty wedge
#

it's both actually lol

#

and yes

#

I'm looking at it from that point of view

#

that's why I've spent three whole days learning and I'm only beginning