#source-control
1 messages ยท Page 54 of 1
Have you already considered simply adding an entry to your stream to pull the plugin content from elsewhere into your game project stream?
Not entirely familiar with that method, but in this case we're running this batch script as a part of a jenkins build, so we copy the plugin over only if it builds successfully.
The copy running will then ideally trigger a build on the target project as well
hence auto-submitting
@woven sluice Do you know how the HLOD system gets pulled into source control - I'm having proxy meshes built and the HLOD uasset is checked OUT of source control but when I reopen the editor I find like half of my proxy meshes need to be built again
Even though I haven't moved them
When I migrate the assets into a brand new project with no source control, they persist fine
(@quick slate oops I vanished) ok, you probably don't want to just simply import the plugin stream then. Not sure how to use copy then sorry ๐ฆ If it was any easier to figure out how to do this, you could set up a Release stream for your plugin and import that stream into projects... and then either manually or with Jenkins merge your plugin's Mainline stream into its Release stream when appropriate (i.e. when it succeeds at a build). This is just pushing the same problem into maybe a different alley... not sure
no idea sorry blake
@woven sluice no worries I really appreciate you taking the time ^_^
if a temp file is made, how could source control interfere??
this is how I set up my plugins, they're in their own depot, and imported into our game project. I actually set it up bidirectionally so I can work on the plugin either in the game project or in its own project. different than what you're proposing (and more chaotic but works great for tiny team use)
@woven sluice This was in my output log when I built one of the problem Proxy meshs in the HLOD outliner - LogSourceControl: Attempting 'p4 fstat -Or E:/LiveSimMainline/Content/Maps/HLOD/ModNeighborPack_OptimizedVersion_0_HLOD.uasset E:/LiveSimMainline/Content/Maps/ModNeighborPack_OptimizedVersion.umap E:/LiveSimMainline/Content/Maps/FactoryLevel.umap'
That just simply means it's trying to save it because it's a source controlled file - (those assets are checked out)
we essentially have that setup too.
Plugin (and dev project) in its own stream tree (with mainline and dev) then any other project we work on has its own (main and dev) which we merge into or from as needed. We've always done merges/copies manually in either direction, but are now looking at how to make jenkins auto-copy out from the dev project as needed, its just perforce's CLI is a pain in the butt
Hey guys, I'm using Git / LFS for my source control of various UE4 projects and have 100 GB storage. to avoid having to buy new data packs I wanted to remove a few commits from old projects in my repositories. how can one do that in Git ?! and does it actually help me save space ?!
for instance for an old project I want to remove all the commits and only keep the important milestones of every month.
You can use https://github.com/newren/git-filter-repo to rewrite history and only keep some commits, but that in itself won't remove LFS files, which are what are using space. To clear the then orphaned LFS files, it depends on your LFS host (e.g. Gitlab, Github).
Oh thanks I'll look in to it. for removing orphaned LFS files (I'm using github) is there also a tool to handle those automatically or should it be done manually ?!
If you're not using GitHub Entreprise, I'm afraid you're out of luck https://docs.github.com/en/free-pro-team@latest/github/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository
To remove Git LFS objects from a repository, delete and recreate the repository
For Entreprise:
To remove Git LFS objects from a repository, contact your GitHub Enterprise administrator to archive the objects. Archived objects are purged after three months.
If you've set up Git LFS for your repository, you can remove all files or a subset of files from Git LFS.
did plastic SCM change their offerings?
do they not off a personal free option anymore?
Oh crap, unity bought them out and killed Plastic SCM for me ๐ญ
It seems there's still a free version, but it's cloud-only and 5GB of storage.
yeah that's not gonna cut it for my projects in all likelihood. I just wanted to store it on my local machine.
if an entire project has been checked out in p4v how do you get them all back in to unlock the files so other people can edit them?
You can "Revert Unchanged Files"
If you want to give all the files back, you will either need to submit a changelist with the changes in, or revert them back to what's in the depot, losing your changes
there are no changes, so you would just right click the source folder > Revert unchanged files, and then click submit?
is there a setting in unreal editor when using perforce to have it auto add a file I just made to the default CL?
I seem to have to always add it manually when I create a new blueprint for example
Anyone knows how can I submit those part by part?
Github says you cant submit more than 100mb
uh, i haven't used github but isn't that a per-file limit of 100mb?
are you not using lfs?
@bitter sun
Yeah, it was per-file, I misread the warning in Github
And solved it, but now something like this happening, do you have any ideas about that?
afraid not I'm not good with GIT, have to wait and see if someone else shows up...
Okay, thanks ๐
Does this mean I tried to push a file too big?
Most likely yes. I have seen that error when trying to push more than 2GB up to the remote at once.
Ah, okay. I'll try splitting it up
If you run self hosted you can tweak some things to increase that
but ultimately you shouldnt be doing mega sized pushes
you can use SSH Protocol instead of http, so you can push large size
If you are using LFS then SSH isnt supported, the SSH agent just makes http connections which makes pushes slower
Hey everyone, does anyone here have experience setting up Perforce on AWS? Need help!
I was importing assets so that why it was so huge
I was wondering if anyone of you has a solution to do exclusive checkouts with the GIT beta source control in unreal?
we are thinking about writing our own solution
Doesn't the built-in git provider support LFS locks
if not maybe I'm thinking of one of the third-party git plugins
we are using git bash and the git beta plugin - no lfs though since we host on the google cloud which replaces lfs in its functionality (for free btw)
Then no. LFS is the solution to locks with git.
Use a provider that supports it. You should be using it regardless for uasset (and related) files.
There are a number of git providers that are free and support LFS
@merry verge do you know if theres a public repo for the source data of the git unreal integration?
i tried finding it on github but maybe my search skills are too bad
Damn. you're not seriously thinking of abandoning LFS and going home brew are you. Sounds like a good way to sentence a project to die a long slow painful death 
@woven sluice We first thought the same
But when you use source cloud from google
it doesnt let you use lfs since it does its own "oh this is a big file" check
it replaces the core functionality of lfs in that regards
Is it storing uasset files outside of git then?
this is where our repository is hosted
If your provider doesn't support LFS, then it's not a good provider and you should avoid them until they do. LFS is standard across all the major git hosting services.
It's not
we used lfs before
LFS is the standard for large files and locking in git
and its not like its perfect xD
Are binary files in a git repo or not tho 
If you're going full proprietary anyway you might as well use perforce at that point, which does a much better job with large files regardless.
so far we have tested nearly every source controll and our current one is the most STABLE compared to the rest
yeah perforce was a money issue atm
that's fair
team is at 10 persons +
i asked them if they do a "non support" licence that cost less but nono
Yeah, perforce isn't friendly to indies
I'm a little out of touch but I'd probably easily prefer to try SVN over git without LFS
i dont say its too expensive
yeah we have 10+ years exp on our programmers side with git
so they dont feel to comfy with other tools
and my responsibility is to get a workflow going with them that reduces the damage ARTISTS can do xD
you can say 1 merge conflict can be up to 30 minutes of troubleshooting
thats why we are going to have a database set up that just blacklists and locks files and use a git hook to dissallow the "add" while and also immediatly on loadup mark files that are "blacklisted by being checked out"
and thats only for binaries
in theory it sounds easy enough
lets see how far we get i guess xD
Sounds like a good way to burn a couple weeks of programming time reinventing the wheel to me but I'm a dick. Good luck ๐
Thanks man ๐ Its a cost benefit thing i guess xD Source Cloud Storage is tremendously inexpensive xD
and super SUPER performant
For my perforce server I was using DigitalOcean at about $20 a month. Idk the limit on the amount of users but I had 4GB Ram, 2 vCPUs, 4TB Transfer, 80GB SSD, and my monthly sometimes would be higher or lower than $20 depending on usage
we are currently running 3 different game developments on our system and will be adding more repos over time Iยดd say we are sitting at a similar cost as you where looking at except we dont have access to any cpuยดs from how i understand it. At least not in the classical terms. The cool thing about the workflow that we currently have is its so damn simple to set up - only issue we had was that we couldnt see in engine who had what checked out. Looking at the last update to plugin from epic for git i think they just dont have time to polish this up more and we decided to move forward with editing the existing plugin ๐
meaning we store around 100-150gb minimum of data atm excluding backups @thick belfry
ah yeah and currently we are at 5 user that have access but that will ramp up this month to 15
Oh dang
The price of running perforce is primarily user licenses, not hosting ยฏ_(ใ)_/ยฏ
yeaup ๐
perforce is a really nice tool to use if you know your project wont ever grow over 5 users
damn then its sexy af
but it makes zero sense to use it if you plan to scale your production up or are already scaling up while preparing early access for example or other "no budget" "low budget" scenarious
It's sort of insane to me that perforce doesn't try to capture some of the lower-budget market but it seems like they just don't care. They have the free license (which is great) and they have the multi-thousand-dollar licenses (which makes sense since it's largely aimed at larger companies that use support contracts), and nothing in-between.
They really don't care about anyone who doesn't have $$$ which is just such an odd thing to me. indies don't exactly have much money to throw around, but it's still money p4 is leaving on the table. And I'm sure they could come up with some license that's limited to some gross income or something so their current large customers would still be paying the same. It'd cost them practically nothing since I assume a lower priced license wouldn't come with support.
its hard to compete with git, especially with lfs support
svn is an option too, which isn't that unpopular amongst indies I know
then there is plastics, which was bought by unity, and will probably become the default solution for unity projects after some time
there isn't much of a market for perforce to expand on imho
@merry verge I sent this to Mark Ties the Ceo and President of Perforce a whle back xD
obviously no response even though i sent it through linkedin pro/plus/prime (forgot what the paid version was called xD)
If you work for a studio that uses perforce and you do a project on the side most ppl will default to what they know which is perforce in that case. It really depends though, I think we all know that any programmer starts with GIT just naturally in their study period ๐
I think free license covers that space rather well. The amount of projects that doesn't qualify for the free license, but would still want to use perforce, but can't afford the regular license at the same time, should be fairly small imho
if you look at from an indie perspective, the amount of unity users dwarfs everything else, and perforce isn't well known in that space, and historically unity used to have their own vcs and now plastics
I was freelancing for around 1.5 years now. Do you want to know how many source controls got into from devs that used the same seat different workspace technique in that time?
and that was not only solo or small team devs
that also happened at a studio i worked at ...
its just ... i dont know
ppl always risk things and i never wouldve want to do the same choices which is why any solution with git rn seems to be the thing ill be using for building up my company at least. Its scalable, it works, it doesnt cost ... as much by FAR ๐
btw about plastic
this year i already encountered one ue team that uses it there
so it seems to work for ue too Oo
never touched it before though
I don't know, I think svn works pretty well for most games. I would use svn rather than juggling p4 workspaces tbh
git only works in small teams imho
yeah honestly i only defaulted to p4 bcs of the first contact in a studio
and then transitioned over to git
you mean it has issues with teams larger than 10?
for solo projects git + lfs + local remote works flawlessly, but anything over 5-10 it breaks down
tell me what so i can adjust my negative bias xD
especially if you have multiple artists that want to work on the same content
ahyeah
but i mean perforce doesnt rly change that, i remember two artist in a studio sitting next to each other working in the SAME scene to just discover "oh i cant push this its checked out by someon already"
and ive seen that happen more than once xD
the difference in svn/p4's case is that the artist didn't use the tool correctly, and on git the artist used git correctly
one can be fixed pretty easily, the other is the intended default
we had a slack channel for artist so they could notify others about not changing assets, mainly materials and maps, but also models and textures would also count
so whenever they wanted to change something they had to check whether someone was already doing something with the asset in question
there were a few material mishaps and lost work on maps due to that
if the team was using svn/p4 correctly instead of git none of that would've happened
ah yeh - about that - currently letting the eu evangelist reach out to ryan durant who created the git integration asking if hes currently working on an update. if not we will be definitly taking a shot at creating a realtime integration that shows you if a file is already checked out. we would be using "always exclusive" as a rule for artists meaning someone has something checked out gives you "you cant touch this and a name"
what you described is definitly the biggest issue i have with git too, just fixed two merge conflicts yesterday xD
the cool thing with git is you can create your own hooks and if you have a good programmer its definitly something you "could" engage on ;3
well, I would rather use svn than to mold git to my preferences, but that's just me ๐
yeah i think its rly a luxury thing rn
i mean us thinking about reinventing the wheel just cause xD
Git itself cannot tell you whether a file is checked out because it's not built for that - it'd break the distributed model git uses. No git integration can change that.
The only reason LFS can is that it adds an additional centralized server into the mix for storing LFS-controlled files. Which allows them to build an exclusive lock system on top of it.
So you're just going to get told to use LFS or build an entirely new system yourself.
has anyone else had issues with perforce in the new engine version?
right now in the engine it won't let me check files out or show if files are being checked out.
never mind I was logged into the wrong workspace :/
I think you are not following xD We are using a centralized checkout system in addition to some intelligent use of git hooks to "simulate" the same behaviour that an "exclusive checkout" would be doing. So basically when you save something it does a git add the hook listens for that and checks "hey is this file blacklisted rn" (meaning added by someone else to their local repo already) and if its the case it prevents the add for the blacklisted file.
its a simple database of "local adds" tracking
and its only targeted at binary files
Looking for UE4+Perforce expert for (paid) help
I have teams of 20-25 on git just fine. Above that id be a little bit leery though. It requires a decent amount of work to get it usable in "larger" projects, but its entirely possible. Actually number of people isnt a good measure, Project size/size of assets is the main driver of it becoming less usable.
I would imagine you could create a decentish workflow to make git usable, especially if the people that have to interact with assets is limited, but I would far prefer p4 or svn for anything over 10 devs. Maybe if the team is very programmer heavy things would be smoother though.
What was the question?
do you use file locks with Git?
to prevent people editing the same file
We do
You mean git lock?
yeah
git-lfs lock
alright
Could you please explain more how to prevent editing with this?
Only one thing Id warn about the locks, it can prevent merges - and the unreal plugin is not super polished atm
@jolly fog you know how centralized source controls locks files to prevent editing files?
so if i have a file X, i can tell the system "i am editing this, nobody else touch this"
yes with the command I posted
and then only until i have unlocked it, nothing can be done with the file
In fact im new to unreal but interested
git-lfs lock && git-lfs unlock
U know
yep
i am working on a solo project atm but need to look into git lfs lock and unlock lol
LoL
what you mean? git, git lfs?
yea
git is a software for decentralized version control
helps you manage the history of your project
I think he means why do you care about locking?
yea thanks
as a solo dev
ah yea
exactly right
Hi, I have merged two projects together so there is quite a lot I need to commit (using perforce) am I correct in thinking I should go into perforce > right click the route directory > mark for add > click submit
I turned off my droplet hosting my p4 in order to upgrade my disk space from 80 to 160GB and once I turned it back on and tried to connect to p4 I am getting this error
Connect to server failed; check $P4PORT.
I stopped using git with Unreal as it doesn't seem to be all that supported. I think switching over to Perforce is definitely the way to go
But switching over to Perforce has its own frustrations. I accidentally added the Intermediate folder -_-. Also I have no idea why Perforce put read-only permissions on every file in the proejct
Anyone think the read-only permissions is intentional and a good thing, perhaps I'm doing something wrong?
Does Perforce set files to read only until you check it out or something?
Btw pretty sure you can retroactively remove Intermediate folder and other folders that are not supposed to be there by first marking for delete and second adding a .p4ignore file. But I believe you also have to manually set P4 to look for that .p4ignore file by running a p4 set P4IGNORE=.p4ignore
This is correct. The intended workflow with perforce is that you check-out a file before editing it.
Keeping things read-only until then makes it so perforce can check for locks on the file before allowing you to edit it, and the workflow makes it so perforce doesn't have to scan through all your files to find the edited ones (you can tell p4 to do this manually... but if you have a large project it can be slow).
If you're using visual studio, install P4VS to have VS automatically prompt you for check-out when editing files. For Rider, there's a built-in plugin.
In unreal editor just enable the perforce integration - the editor will prompt you to check-out files when trying to save.
thanks. I'll also try that extension out
Hi, I have an ongoing submit in progress, I realised that its going to include a bunch of folders that I do not want such as Intermediate & DerivedDataCache. what happens if I stop it now?
It should stop the submit without issue then you can setup a p4ignore OR deselect what you don't want pushed
Hello everyone,
So our current games are based on BP and C++ and we have a team of a couple programmers and artists. We are currently using Perforce and it works great but we're out of user licenses and workspaces and the pricing is quite high. I'm just wondering, out of SVN and Plastic SCM (Paid but reasonable) which one would you guys suggest?
I can see that SVN is centralized like Perforce so it can lock files for editing and avoid the mess of multiple edits, but here are a few questions:
1- Is SVN fully free? Apart from a server hosting fee.
2- Does UE's integration auto-check out/lock files on edit like Perforce?
Right I have messed this up ๐ how do I get rid of a pending action?
I dont want to overwrite my local changes, I just want to remove all the currently marked for add's that exist as they are against files that are now included in the p4ignore.txt file
Right click the files in the cnangelist you don't want to add, hit revert, make sure you do NOT check the box that says "delete files opened for add".
I stopped it many times it didn't matter
Yes it auto-checks out and locks files with Perforce. It is fully integrated with Perforce, epic games uses it. So if ur using perforce + UE4 then ur gonna use a battle tested integration by thousands of Epic Games employees
I highly recommend Perforce
They're asking if SVN has integrated locking (I don't know and am curious too since it doesn't look like perforce will ever be viable for large indie startups)
Yes that was my question, I'm considering other alternatives other than Perforce seeing that the license price per user is really high
4.26 chaos ...4.26
@lament hinge If you want a simple setup Iยดd advice you to use GIT + source.google cloud storage. Working with it rn and the only downside atm is the lackof exclusive checkouts. We have around 400gb and more than 10 teammembers on the repo (several projects) and pay 17 bucks a month for everything
Not having exclusive checkouts is a pretty big dealbreaker if you aren't willing to write your own locking system ๐
Hey ive got a somewhat old version which im turning into a lightweight version of UE4 for mobile and stuff. Is there a way i can turn "4.17.2" to "LW.4.1" or something?
Exclusive checkouts are a must for us. We used to use GIT and as the team got larger it becomes a massive time waste to have people enforce exclusivity.
Perforce is saying it has long file name support enabeled by default https://www.perforce.com/manuals/p4sag/Content/P4SAG/install.long_filenames.html but I am getting an error when I try to submit a file due to it being too long
Hi everyone. We are using git for our project. In the case two of us worked on the same binary file and it generates a conflict, we use the File->Diff method and copy all the stuff that was added. But to be honest we are almost on the mental boom level ๐ What is the best way to work with this kind of conflicts? Hope there is a good option. Thanks!
@ivory hinge literally read the previous page or two worth of discussion, TLDR why aren't you using LFS
^
The answer is LFS and LFS locks. Don't work on binary files at the same time. There is no automated solution if you do.
How doth one have source build show in marketplace so I can add assets to it easily
Hi
Im new to source control. Should there be an option to check out and check in assets?
I only get a check in option if i enable auto checkout on modification
I am using git
When i used perforce, there was an option to check in n out
So i guess the real question is, did they remove those options in 4.26 or is gits beta status the obvious answer
Nm i scrolled the page and got the answer
Git is beta
What is the correct way to restore just one item on a perforce changelist? You want to revert just it but not the whole changelist? It's better to do it outside the editor while it's not running right?
In perforce, right click on that item and choose to revert. Beware! If item is marked for Add a window will pop up asking if you want to delete files marked for Add. Make sure that option is not checked if you want to keep those files
Or right-click on the asset in the content browser and go to source control -> revert. The editor will reload the asset after reverting.
Did you set p4ignore in the command line?
Thanks!
Uh, shit. How does anyone handle merging assets from one stream to another?
Specifically, the assets are failing to open in another stream because the CL# check between the editor binary and the assets detects that the assets are from a newer CL.
That is technically true... but not actually meaningful.
If I have a release stream and do a copy up from the main stream, the release stream will of course have a higher CL#.
I Can't Publish My Project
You need to pull first.
oh okay thanks
Hey I'm trying to work on a project with another friend, I've uploaded my entire project to my GitHub (with a UE gitignore)
However, when he pulls it, he only has these files:
I'm guessing its because of the gitignore but how exactly are people supposed to work on the same project? whats the idea behind it?
I understand theres something called LFS, and I'll get to that most likely but how exactly do two people do this? Does he create his own project and just copy these files or?
He needs to generate and build the project himself. Everything necessary for that should already be there.
LFS is just a way to store large binary files (assets), but it doesn't change how you use the project.
Also, it looks like they don't have the uproject file associated with unreal. Make sure they installed the version of the engine you're using
Once they have, they should be able to just open the uproject and it'll prompt them to build it (they will need visual studio installed as well)
FYI you should be ignoring the Binaries, Intermediate, and Saved directories too. You should be able to find a proper git ignore just by searching for "ue4 gitignore". Once that's done, delete those directories (or move them outside the project temporarily) and then commit.
It looks like you don't have a gitignore file at all in that directory so you'll need to create that.
can we force the perforce component of the editor to only check-in when i actually want to check-in everything instead of on every file save?
Does anyone know how to work with git lfs?
@merry verge Thank you
Its the same workflow as git?
@raw tundra in the past you had to do a command line and add any folders you want LFS to handle
after that it's much the same
hey guys
I am having an issue with perforce. I am trying to have my team access the workspace but it's only available for me. anyone know how to fix this?
Workspaces are the local copy of the Depot on your PC
each client would have their own Workspace
client/user
Just updated our base project files with our latest editor and git config that powers collaboration at scale with a UE4 repo on git shared with 70+ people with tens of thousands of asset files and 50+ built data+levels
@quaint obsidian documentation to see how it works?
Hey guys I have LFS set up for my Github, however when I try to load a uasset file it says this
heres my gitattributes, how do I work with this?
no it's more of a look and see thing for now. we do have this blog post:
https://projectborealis.com/git-distributed-collaboration-at-scale
as well as a wiki on the repo above
@quaint obsidian Thanks!
not using source for half life feels like a betrayal 
anyone know a good free version control solution/host for a small hobby gamedev team? (like if you were in a gamejam or whatever, what would you use?)
This is what I typically use for VCS for jams. Self-hosted/local Git for Solo, Perforce for Teams (free up to 5) @full raft
my local .git folder size is completely out of control. someone else added a bunch of stuff to it and it ballooned in size. 180gb for a project nowhere near that size. so i did some googling it was recommended to run git repack -a -d --depth=250 --window=250 and all that did was balloon it another 25 gb and fill my drive. The .git folder is about 120gb.
is there some way to keep only like the most recent or two commits locally?
Welcome to why git (on its own) is horrible for games.
Make sure your host supports LFS, then go look up how to set that up.
Assuming most of those files are uasset or similar binaries.
https://notiz.dev/blog/migrate-git-repo-to-git-lfs#1-download-and-install-git-lfs-extension
You'll want to track *.uasset, *.umap, and any other large binaries you have stored in the repo (image files, audio files, model sources, etc).
thanks I'll dig into that, I'm fairly sure we've got LFS turned on since we don't seem to have any issue with large files, but it's probably not set up to handle them properly.
LFS just being on doesn't do anything - you need to explicitly tell it what types of files to track. If you're having issues with git history taking up lots of space then you either have massive amounts of small files/lots of changes (LFS won't really help with that) or large files that aren't tracked by LFS.
Alternatively, someone somehow submitted LFS-tracked files without using LFS. You can use LFS migrate to erase them from history and move them to LFS storage.
yeah i dont' think we set up anything specifically to be tracked by LFS.
I'll probably have to do that.
I'm getting failed to push some refs on attempted initial push to remote
~2500 objects, might be too big?
Using default gitignore that the UE project creator set me up with, c++ project
an empty UE4 project probably has something like 5 to 10 files. add whatever is in your Source and Content folders to that count
So the 2500 file that its trying to push are including files that should be ignored
quite likely
Is there a reference .gitignore I could look at somewhere?
just googling UE4 gitignore should find some results
or if you search this channel for "perforce ignore file" you'll get one of my perforce ignore files... the syntax is 80% identical, if you know git's syntax you could cherrypick from it
Hi everyone, I'm new to version control and I want to work with a friend of mine in another city on a simple project.
Which version controller is the most preferable for this propose?
the project can have about up to 5 gig of assets
If you do mostly code, git. If you're both mostly working with binary files, probably better to go with perforce
git is extremely easy to set up in cloud environment though, haven't used perforce myself but it seems a more complicated setup at first
my friend will do the art and i will handle the code side
is it going to be problematic for artists to use git + a user friendly GUI like source tree
cuz I have heard git is not compatible with uasset
define not compatible
binary files can't be merged, which uassets are
but neither can they in perforce afaik
but in perforce you can lock them so no one else can touch them
you can also do so with git lfs depending on which git hosting you use
should definitely set up git LFS as well though if you're going to choose git
by not compatible I meant getting a lot of conflicts
alright, so the git LFS is giving me the option to lock the thing I'm working on?
I thought it's for handling some large file only(cons of being new to something ((: )
it is for handeling large files, but it also has locking
it's not fantastic though
and it's not supported by al git hosts
thank you so much for information๐ .
@rotund bobcat Just read about Subversion source controller, I guess it has the locking system as well, do you think that it can be a better choice for me in this case?(an artist and a coder)
Could work too yeah
but I think if you're looking at something like svn, you're very close to perforce anyway
since perforce is free for <5 users
might as well make the jump then
aha, so I have to pay for server as well here(for svn i meant)
idk if there's free cloud hosting, never done the svn setup myself, only used it shortly
maybe someone else can help you better with that
I'm a bit of a git fanboy :p
I have used git a little bit, I like it as well
but I was worried if it can be problematic for the artist who works inside the engine
tbh you'll probably have very little issue if you're only working with 2 people
the problems only start to arise if multiple people are working on the same things
aha, got it. thanks.
you can give it a try, and if it doesn't fit your style you can always just swap to something else I guess
sure thing.
what's your preferable GUI for git?
I like sourcetree
but use your own 64 bit version of git maybe instead of the embedded git, and turn of their LFS media adapter or whatever it's called
aha, got it.๐
technically I have not encountered to large file problems using git, but as far as I know you can't push a file larger than 100mb, is that true?
not necessarily, again depends on the host, git in itself doesn't really have size limitiations, but like github or sourcetree or whatever probably do
someone told me github desktop does not allow to push a file larger than 100 mb so I thought it's something to do with git
what teams do with these limits?
git lfs ๐
lfs = large file system
Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.
basically throws your big files on another "server" and keeps pointers to those files in your repository
so many useful info (: tnq
what source control is most commonly used? for both small indie studios and big game companies.
Git or Perforce mainly
Git for very small teams, as it doesn't deal well with binary assets
SVN and PlasticSCM are also used. I believe someone who works on PUBG said the other week that they use SVN
Plastic seems very promising, but I've never used it though
I'm going to have a look at Plastic and SVN to replace Perforce at some point next year. Our current Perforce licenses run until October
what are the pricing for these options?
depends on what you pick, for git it depends by what host you use and how much data you need and such
perforce is free for self hosting up to 5 users, but after that gets really expensive
SVN is free if you self host afaik
no idea about plastic
is that 5 users per repo? i mean if i have 2 projects with two diffrent groups of people (less than 5 ppl each) can i create 2 repos free or will i have to pay?
not sure, I don't use perforce myself
@sturdy lake do you know the answer to my question above?
It's 5 for the whole server
You could possible set up 2 servers with 5 different people on each...
okey
what do you mean by self hosting?
You run the server yourself
^
Not for 5 users/logins, if you want to run it on a PC you already own
well you neeed a machine to host it on
but it could be your workstation itself technically
Is it possible for a team of 2 to use this feature?
By my guess, 2 is probably less than 5, so it should work
as long as you both have a way of connecting to this machine yeah
e.g. over lan or whatever
hmm I was thinking about how my friend can have access to my machine
Most people try to host it in a VPS as it provides slightly faster/reliable access for remote members but if you're in a pinch local hosting does work
is there any tutorial that I could use to learn how to do it?
Perforce is complex so expect some learning curve if you choose to try it
They have lots of articles and video tutorials on everything, plus YouTube should have a few
I guess it's better for me to start with git then move on to others
Everyone should learn how to use GIT. no programmer/developer has ever learned GIT and said "I wish I never learned how to use it well"
Everyone should learn how to use git on the command-line.
Using a GUI is fine, but it doesn't help you understand what's going on as well
Nah, I exclusively use git from the command-line but I don't think you learn more by doing that. Probably much better to just read some short articles on how to do basic stuff and understand the fundamentals of it all.
Seconding ^
Working exclusively with a UI is fine, you can learn the fundamentals just fine that way (push/pull/commit/fetch/etc).
God forbid you had to teach the command line to an artist (not that git via a UI is great either)...
Hi, I'm using git + SourceTree. I'm practicing how I can go back to my previous commit(let's say I want to go back to a specific commit because I'm unhappy about my commits).
here is the picture to demonstrate what I want to achieve.
I want to go back to 3rd commit, I used Checkout... button and "Head" is now on my 3rd Commit. But it seems everything in my project remains the same. Is there any additional step?
The only thing is that every asset I have added after my 3rd commit has a star mark(unsaved) in Content Browser.
I don't use SourceTree but you are looking from something like reset to here (this is from git fork)
also stash any local changes first if you want to keep them
stash button seems to be disabled, but sometimes it's enabled. is there any rule for stashing?
There needs to be something unstaged to be stashed
I clicked on a commit I want to go back to, I used Reset To this commit option and it gives me 3 options.
which one should I use?
If you are OK losing all local changes (you have them committed or stashed) then choose hard. This is a full reset to that checkin
but if you have something you need to keep, you have to stash, commit or at least copy and paste it somewhere first ๐
you can see from those descriptions what the 3 options do
so first i should stash, then i should use the Hard option
yes
you can un stash into any branch so if you move forward from the older commit, you can then put your stashed changes into that new branch
I used Hard option, now my local master is on my 3rd commit. but the origin/master is still on the latest commit
yup it would be
you would have to do the same thing on the remote. I don't have a 'spare' project here to test on but you have to either do it on the remote which I think is the same process or you have push you reset to the origin, you'd need to search for how to do that
looks like you can do a force push if you are very sure
thanks!
what's the difference between Reverse Commit and Reset Current Branch To This Commit ?
some people suggested to use reverse commit
reverse commit will create changes that do the inverse of everything from now back up to that commit
reset the branch to this commit will just reset the whole state of your project tot that commit, ignoring any history that came behind it
afaik
@rotund bobcat the thing is when i use reset commit option it resets the local branch to that commit
but the remote one still remains in the latest commit.
I can't find a way to reset both remote and local to a commit I want
I found something about git push -f which forces the branch to a commit
yeah
a force push overwrites the remote state with your local state regardless of history clashes and stuff
What's the difference between Merging and Pull Request?
it seems they both are used to merge a branch into master branch
merging is just merging it in, creating a pull request is creating a request to merge your changes in, giving you the opportunity to do e.g. code reviews or automated builds or whatever before approving the request to merge
so it's like a safer way to merge our branch to the master
but in the basics they are doing the same thing
Dose that mean I should never use merge in teams?
the final goal of a pull request is to perform a merge
depends on your workflow
a common workflow is to have a "main" branch, where no one can just push to, but is only possible to be updated through pull requests. Developers then work on their own feature branches, where they can do whatever they want, then create a pull request of that branch into main, perform code review and automated tests and once approved let it auto merge into main
got it thanks!
Hello, stupid ask probably dropped 20 time, but can't found the answer anywhere :
I've got a project, unreal engine is from launcher, project is c++.
We use perforce for the team.
Issue : Designer can't rebuild the project cause obviously, they don't have anything for.
How we can send (or what folder) we should send to avoid them having to recompile the project everytime a change is made in c++ ?
the binaries folder
but you should thinnk about how you approach it
cause if you just add the binaries folder to source control, your devs will always have open changes to those, which might be annoying
e.g. we have an automated build machine that creates the binaries on every push to the main branch and puts it in a separate folder, and then on designer/artist machines, we have a git hook that copies it from that folder to their local binaries folder
Hum, i checkout the binary folder, by look like they should recompile it even with this, should is share intermediate too in this case ?
after not a lot of concern cause we don't have only two dev.
it's really focused for artist
and game designer
just the binaries folder should be fine normally
but you have to compile in Development Editor btw
those are the binaries that the .uproject uses when opening without vs
Understood, i will try to checkout the whole folder.
unfortunally, still need to recompile
ยฏ_(ใ)_/ยฏ
guess it's time to give up on life then
there's also UGS, but I've never used it
https://docs.unrealengine.com/en-US/ProductionPipelines/DeployingTheEngine/UnrealGameSync/index.html
An overview of UnrealGameSync (UGS), an internal tool used by developers to sync their Workspace with a project's stream.
Dieter is right - just submitting (all) Binaries folders is all you need for artists.
haha, i will throw an eye into it.
Are you using classic depot or streams depot?
classic depot
is that an expression I don't know of or do you just randomly have eyes laying around
So with a classic depot each person is in charge of their own workspace mapping. Coders will generally want to exclude all binaries folders from their workspace, except one coder dedicated to updating the binaries for artists. Normally you will also set up a continuous integration system to do this for you
But one step at a time
What happens if I'm on branch A and I'm pushing a commit but I check mark Master or other branches' instead of only check marking A
(I don't know if anyone mentioned it btw, but if you're working in a team branching is useless for Unreal dev. Good practice to learn the basics anyway)
What's the reason?
branching is still good for code
or for other stuff as long as you don't create binary conflicts tbh
so artists or whomever works inside Unreal should work in master branch right?
we had a master branch with latest release, then main branch with latest stable update, then an art branch for arists and designers, and then developers just created feature branches for code
Totally disagree. Uassets normally depend on that code.
Everyone working in main is sadly the most robust way to work. Don't even try branching unless everyone is super familiar with both unreal and git (or rather, with your vcs)
Alternative logic: you can use branching, but as a team everyone should be working in the same branch at the same time
E.g. Everyone works on dev branch, or everyone works on release branch
When you switch, one branch can just totally override the other
I'm still learning about git so I could not understand what you meant here, let's say I have a material in my branch A, based on what I understood I'm not gonna be able to merge my branch A into master?
if there's a change to the same material on master
that you did not have when you started working on it on A
then you would have a conflict
cause it changed in both master and A since their common history point
with code you can resolve this with merge tools or manually
but for binary files, you're fucked
now I understand
so the good practice is not to change anything on master
only merge the final results to it
Is there any guide line about this?
As long as you don't modify a binary file more than once in different branches you won't run into this challenge
A brief introduction to common Git workflows including the Centralized Workflow, Feature Branch Workflow, Gitflow Workflow, and Forking Workflow.
but basically yeah binary conflicts are the devil
Also I marked master to see what will happen, nothing happened
what do you mean with marked
in that push ddialog?
that would just mean you push your local changes to master to the remote version of master as well
see "remote branch" next to it
I added a sound cue in engine, pushed it in branch lets say A, and at pushing time I marked both A and master as I have demonstrated in that picture
that would do nothing to master if you didn't change anything on your local master
but sound cue is not pushed into master
to do that you would need to check A and change the remote branch to master
then you would push your commits of A on master if possible
but that would be a weird thing to do tbh
you should use either merges or rebasing
yeah I was just wondering what will happen, just experiencing
This is an expression haha :p
The issue of the rebuild can be cause i've got symbol in my unreal launcher version and they dont ?
i think it's a no, but i really struggeling on this one.
ah wait
they don't have VS at all on their pc maybe?
because you do need some VC++ redistributables by default to work with unreal and stuff
maybe that's the problem?
I think so yeah
not 100% sure
We just installed VS on arists pc's as well
sometimes handy when you need to walk overr to their desk and do some local debugging ๐
i will try, thanks !
How do you force add an ignored filetype in perforce? I keep getting "ignored file can't be added.", and now even removing it from the ignore file list I still get that...
scratch the last part I also ignored the parent folder, but I would still like to know how to force without unignoring then reignoring
ahh, command line p4 add -I "filename", too bad can't do it in p4v...
yeah I usually just rename my ignore file, add it, unrename it lol
if you have other edits and run Reconcile Offline Work it has an option to turn off the ignores for the reconcile, but you can only pick it if the window pops up, so if all edited files are ignored you can't do it that way because the window won't pop up 
So is set up github for my ue4 project and what does my friend need to do to open it an actualise my project
?
Is it ok to merge a pull request from Bitbucket itself? or it's just better to merge within source tree itself after the approval of pull request?
someone on Youtube said it's not recommended to merge from Bitbucket but didn't say why.
i have a query....i have configured a svn server for working with my team....and i have tested it....all i want to know is now...which are the files that we should share on version control system....i am unable to find and specif answers at the forms and other locations so i rather tried it on the entire folder that contains my project...which is quite dumb i know....but if anyone has any idea about it...i would be happy to learn more.....
Doing the merge in bitbucket is absolutely fine, but I would always recommend merging your develop branch into your feature branch locally so that you can test that itโs all good first. And then the merge on bitbucket from your feature branch into develop should be seamless
What's the correct procedure to connect to a remote Perforce server that's behind a VPN? I've remoted in (Chrome remote desktop, so I don't know how Chrome handles things) and ipconfig tells me the IP address that the computer is currently connected to - I've tried using that and my normal ports for the perforce server (1666) and it doesn't work - what am I missing here?
I'm not an expert but I think to work with things on a VPN, you join the VPN with some software and then you can connect to things via their local ips like that (ipconfig shows local ip)... why put it on a VPN though? Or is it somehow beyond your control
VPNs just make it so you can access an internal network remotely
so you would connect to the perforce server using the IP of the server on the network
as you would do on the physical LAN
Hello guys, I made gitub repo which is private, and I shared it with one more person who will be colaborator, but I can't push update
I have this error
I logged out and logged again in but same error, I think best decision will be to log with ssh key but I have problems with generating and when I checked for existing keys and it writes that I have 26, lol
also for sourec control I use the desktop application for Windows
I follow this tutorial: https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/connecting-to-github-with-ssh
but ๐ฆ
You can connect to GitHub using SSH.
I have this
xfs or ext4 for a perforce linux server?
@fringe abyss https://community.perforce.com/s/article/2984
great article for sure.. doesnt answer my question. but still very good to know
also read this one https://community.perforce.com/s/article/3131
but it says at the end they have not tested ext4(i guess in answering my own question, go with what has been tested)
YMMV but I've been running p4 for 3 people for 12 months on ext4 / Ubuntu 18.04 with no problems ๐คท
Git question: I'm having an issue with a fairly large commit causing a build to crash. When I rollback one commit everything works just fine. Unfortunately the logs in the packaged build don't give me much info, so I need to comb through the commit to find what change caused the issue. What's the most efficient way to do this? I was thinking to branch from the working commit then make each change one by one to see which one breaks it, but that seems really tedious when the commit has 50+ files changed.
a really naive approach that would be slightly faster would be to auto-commit each changed file with a loop and then utilize git bisect
for file in $(git diff --name-only); do git add $file && git commit -m "$file changed"; done
you'll have to keep in mind the cases where some files' changes depend on other files' changes which get committed later in the order
Right
also, are you able to package the build with debug symbols? i assume by saying you don't have much info means you don't get a stack trace or anything like that
Well, I get a stack trace, but the error I'm getting seems to point to a large possibility of problems based on my research.
But, I am fairly new on packaging stuff.
Hey all, I'm looking for suggestions for large object handling in github. We've been using GH for a long time, but we haven't come up with a good workflow for our large assets. We hit the github LFS 1GB free limit and started using zip files for large assets after that happened. I'm thinking there's gotta be a better strategy that doesn't involve paying for github's increased storage limits... anyone have advice?
Haha alright, that probably makes the most sense. I take it that git LFS is the standard solution for this?
Yep
Its the most widely supported method for storing large files in git and includes other things that are usually necessary like file locking
If you need another provider, look at azure devops. There's no hard limit on storage, though I believe it charges if you have more than some number of users. Might also look at gitlab and bitbucket, not sure what their limits are for private repos
I've accidentally unpushed, uncommitted, unstaged and discarded two git commits of mine - is there any way to get them back
anyone know why SVN would suddenly get really slow (only from Unreal editor, not from Tortoise)?
Up to half a minute to check out a file.
Hi Light3n1ng, we use SVN as well.
This is what is in our SVN:IGNORE property:
.vs
Binaries
DerivedDataCache
Intermediate
IOS
Mac
Plugins
Release
Saved
Source
Stage
TVOS
WindowsNoEditor
build.bat
MacNoEditor
Hi enko, we have found that SVN is terribly slow in UE4 editor when working on the VPN.
It does a massive amount of SVN STATS
which is so frustrating
so you guys keep the entire project directory for sync? and ignore these dir and files
Yes, thats correct
I think I may have missed the Build folder in that list
@latent blade, though if its just checking out a file, that could be the machine that svn is running on having something use the disc
thanks a lot.....although i switched form svn to git. ill add the following to my git:ignore.
yeah we have the same basic folders for our git ignore as well.
did you do the git lfs properties as well?
I haven't got my lfs properties handy though sorry
I've got a dev who's working remotely so he needs to change computers
He's got his P4 workspace on an external drive
Can he just install P4 on his new computer, create a new workspace and point its root to the workspace directory on the extenal drive
i am using aws ..it says some stuff that it can handle large files
i have tried upoading files as bug as 100mb
itwas working..should i try it for larger files?
its been a long while since i did the lfs properties, but if you dont set them then its just storing them in git as binary files.
In the lfs properties thing, you tell it that uasset, umap files are binary and it then stores them in the lfs side of things.
this is one of the better links I had bookmarked: https://stefanperales.com/blog/unreal-engine-4-and-git-lfs/
Devops, programming and gaming
I dont get it .......even if i update this with the adress of the git repo where i am hosting it..it wont push anything to the server. rather show me a msg of changes made on push
No I wouldn't recommend that. Workspace maintains a list of file states and two different workspaces pointing to the same files is asking for trouble.
They should have 2 workspaces with 2 copies of the files if at all possible.
Or maybe they can re-use a workspace but you need to configure perforce into allowing use of the workspace on a different computer.
https://www.perforce.com/manuals/p4guide/Content/P4Guide/configuration.workspace_view.multiple_computers.html
thank you, that was it... turned off VPN - back to normal. So odd... because SVN server is local.
does anyone know if blender has a p4 plugin, it's very annoying having to go find the file im working on to check it out
can anyone please explain on linux how to setup a p4 depot on a drive other than the one p4 has been installed on?
chant dark rituals and commit blood sacrifice
you want to config your perforce root / database root
I think thats what they call it
you can run p4d with a command line arg that changes its root
if you're talking about one depot on one drive and one depot on another drive
I don't think thats possible
at least not easily possible
oof Allar.. i been following your tutorial lol
lol
alright
i want to have all depots on this 100gb drive i have
instead of the main system one which is 25gb
i plan by end of january 2021 to have a whole new way to set up source control
so i can delete that old article
aye
that old article was meant to be a meme from the start
it was meant to generate free server hosting money through referrals
and it totally did
oops.. i had already setup a DO server before that
You can always symlink the depot root folder to a different drive. Probably best to just figure out what config option can be used to relocate it though.
-r
e.g. ./p4d -d -In p4master -p 1234 -r /opt/perforce/servers/p4master
runs p4d in the background, named p4master, on port 1234, using the specified folder as root for server data
yeah Hojo thanks
i finally got to that when i go to p4 config
it asked me for the folder to setup it all up in (this is all a fresh install)[and i put it on the mnt point for the new drive + those additional paths]
im just stuck now at this unicode yes or no question
it is, just wasnt sure if i should use the default..
yay i did it.. this was kinda fun
Hey guys ๐ My very first question! I just want to mess around with the real-time caustics shown in the readme of this repo.
https://github.com/NvRTX/UnrealEngine/tree/NvRTX_Caustics
I'm aware that opening the project in Epic's official branches is not enough, meaning I would have to use this branch in order to make the caustics effect work. However, I'm not quite sure how to change a branch on my Windows. Normally, I just run Epic Games Launcher and then choose which UE4 to be opened. What am I missing here?
it sounds like you've never used GIT, so you might want to start by finding some resources such as tutorial videos or other websites to learn about what GIT is and how to use it
I have. souds like you didn't understand my question., but thanks.
guys anyone uses SVN here ?
well then, if you know how to use GIT then surely you know that you just open your preferred GIT client to the UE source repository which you've already found and presumably connected to, and follow your client's procedures to switch to the branch.
I'm tryna work on a project with my friend,I just tried SVN yesterday but I think its not a good idea to use SVN cloud server for such use case is it ?
Sorry, my question was not clear. I've never used any GIT client to use unreal. Epic Games Launcher manages ue4 versions (at least speaking of my case. make sense?
should help you out
actually the first link is kind of old and obsolete ... I shouldn't have posted it
if you've used GIT then you should generally already know how to use github to clone the engine source repo, if you haven't or if you need help with that, you should say so
thanks so much! I guess these links would be enough. I've cloned repos. just wasn't sure how to connect a repo to Epic Games Launcher...
you don't
it is possible after you've manually built a source engine to get the launcher to recognize it but normally you don't bother
good luck!
how do people typically store secret keys that might be needed on a dedicated server, but not accessible in client builds?
(in such a way that you don't have keys available in your source control)
@gaunt arrow Good question - I keep mine in two places. Firstly in my CI server (TeamCity) the aws credentials for uploading builds are stored as TeamCity secrets.
Secondly I keep AWS credentials for dedicated server file access in a local file created by TeamCity job when the dedicated server is started.
A lot of people use SVN. I've been on maybe 4 or 5 projects with it and I always have issues with it at some point, maybe its something I've done wrong. I switched to perforce because it always "just works" and seems a lot faster. Branching also isn't really a feature of SVN if you do want that at some point. SVN might be easier to setup though, but perforce also isn't that hard... you can self host either for free and keep it on your own local machine or use something like Digital Ocean/Linode/Aws to host a server in the cloud (which isn't that difficult either)
the thing is I'm in a restricted country but still I think I can use perforce on my local machine right ?
what do you think about Plastic SCM ? doesn't that provide more free stuff than Perforce ?
I've never used it. Perforce is free up to 5 users, though 1 user can have multiple workspaces... SVN is free for unlimited amount of users...
my new partner and I are trying to set up source control with Github. and are receiving an authentication error that has had us stuck for days. Hopefully yall can help. ill have him post the error now
Hello guys, I have problem with the source control, I use github desktop for source control and github web server, and I connected it in ue 4 successfully
I succeed in commiting the stuff but I can't push
I have this error
Most likely The first, yes. However it looks like you had a commit A that branched to both B and C, and now you want to rebase C on top of B, there have been changes to that uasset in both B and C. If you are happy to lose the changes from B then fine. If you want to use both sets of changes youโll have to manually merge them.
Hello everyone. I use github for source control, I can submit my files properly and I do not have a problem so far. The problem starts here:
As I import the content from Epic Store to the project, the size of the project is growing. For example, whatever is in a 2GB character pack is imported, everything from animation to texture part. Does it happen when you install 3-4 more packages of this kind, the project is 10GB. Now I want to submit and commit this project and then fetch it. But my upload speed is pretty low. Now what I want to ask is:
What do you pay attention to while storing your projects on github? Is there a different and effective method you use? Or do you upload directly as I do?
use git lfs for larger files
Has anyone used Digital Ocean's Spaces with git lfs? If so, how did you set it up?
I've tried linking the lfs endpoint to the server, but it is seems to not be working.
Just says Client error: {server url} and then that's it.
I'm trying to put it in a specific folder.
So my endpoint is as so - https://test-storage.nyc3.digitaloceanspaces.com/project-name
anyone can help with the problem?
but when I check for key that is used I can't find
I generated a lot keys and still doesn't work
anyone?
@jolly fog oof i had that error a lot using git as well
the only thing that fixed that for me was to just keep trying
this is also why I moved to other source control, and as far as possible from git.... ๐
@jolly fog Seems like you are committing directly to Git (writing objects 15.92GB). This is not recommended
You should be using Git LFS
@karmic meadow that doesn't seem like an LFS endpoint, usually its at info/lfs/
Also, we just released PBSync 0.2.21: https://github.com/ProjectBorealis/PBSync/releases/tag/0.2.21
It is a utility, like Unreal Game Sync, for syncing an Unreal Engine 4 project.
It has the following features:
- Unreal Engine 4 build download through Google Cloud Storage (via gslib integration, supports chunked, parallel downloading), or any HTTP download server (via ue4versionator, which we just released v4 of ( https://github.com/projectborealis/ue4versionator )), both of which support download resume and verification
- You can specify various bundles (different build configurations of UE4 to optimize for different workflows, like stripping UE4 game targets to save space) and exclude split symbols into a separate optional download
- Old UE4 builds are automatically cleaned up
- Highly compressed 7z archives are supported, with fast, native, multi-threaded extraction
- Automatic Unreal Engine 4 setup (prerequisites, file associations, and for programmers, project file generation)
- Game binaries syncing for artists with md5 check-summing, and optional user selected version support
- Handling of Git pull, including handling of various errors, and it's optimized to be fast
- Background maintenance of the Git repo
- Configuration of the Git LFS 2 source control plugin
- Git, Git LFS and Git Credential Manager Core auto-update (with version enforcement policy)
- Configuration of p4merge as a visual diffing tool for Unreal Engine 4 text based assets (data assets and some Blueprint graphs)
- CI tools, like version management of UE4 and game project version, as well as Discord game deployment using Dispatch
- And most importantly, a Python backend with rich, extensible API, that can be extended for your project
We mentioned it a bit in our blog post ( https://www.projectborealis.com/git-distributed-collaboration-at-scale )
Just in time for Christmas comes a culmination of many months of improvements!
Updated to Python 3.9
Updated to PyInstaller 4.1
User config now automatically creates new sections
CI user config no...
ue4versionator is a tool to download custom UE4 engine builds, extract and register them for use. - ProjectBorealis/ue4versionator
If anyone is interested in using this stack, let me know and I can prepare some more information about its usage and get it ready for general use.
Do you have any experience with setting up LFS with Digital Ocean's Space product?
No
I see that it is an object storage service, you can't use object storage directly with LFS
you have to use something like GitLab or some other LFS server to handle the object storage
Impressive tech @quaint obsidian, kudos. I'm with perforce right now, but I have it noted down in case I switch back to git.
this will be done directly without URLs, through the object storage's API
So, self host a gitlab, then set it up to direct the LFS to the object storage?
if you are tight on compute, gitlab will likely be too heavy
you can look around for S3 compatible servers here
most of them should be pretty light weight
Thanks for the nice words :) lemme know if you do, and I will see about cleaning up stuff and adding more documentation, if I haven't already by that time!
Self hosting gitlab would be too heavy? Dang.
Huh - looks like gitea claims to be lightweight.
So, I can slap that on my droplet, then set up some way to have it move the binary to the object storage.
how i fix this i got lfs install
did you migrate
what is the proper command in git bash it give errors
git lfs migrate import --everything --include="*.uasset,*.umap"
that
git lfs migrate import --fixup --everything
or git lfs migrate import --fixup --include-ref master
I would say a well functioning GitLab server needs 4 cores and 8GB RAM. It has high initial cost but scales very well. Versus the other solution that have light footprint but probably don't scale nearly as well
Gitea should be fine
where i put reference to the file location
make sure it has S3 or Space integration for LFS objects @karmic meadow
you don't. fixup uses your .gitattributes
i got this error
can you try commiting the files you currently have
click commit anyway
then migrate
.gitattributes file fix it
Yeah, my project is around 17gb and I created empty repo and then decided to move all the project into the new folder where is the new repo and try to push it but I was not ebale to do it, so how I can do it with lfs? @quaint obsidian
Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.
Install Git LFS here, it saays how to use it in getting started
58GB now :D. We use a GitHub Pro plan with a personal repo ($4/month flat), not the team plan. Our organization is entirely open source.
I installed it days ago
Don't you have to buy data packs to use that much storage on GitHub?
how to add the files
idk which are the biggerfiles
the content is bigger than 100mb
umap and uasset
yes then git lfs migrate import --everything --fixup
Attempting to learn Perforce. Anyone here familiar with removing exclusive locks?
If you manually locked the file (p4 lock) then you can use p4 unlock to unlock it.
If the files were locked via checkout (due to their file type) then they should be reverted (or submitted) to remove the lock. They can be force-unlocked with p4 unlock -fx if you're an admin, but that's not part of the normal workflow.
I wish it were that easy
On the Linux side Iโm told I need to add a client whenever I attempt any p4 commands
Yup
Had no issues with the setup
The file is exclusively locked to a user need to clear that lock
I should've looked into perforce before using git ๐คฆโโ๏ธ
if you are on a pkg-build system you can add "p4v" and "p4". The latter one is the command line client and the former one a gui to access your repo. The client is needed if you use perforce from within vscode, I figured
Does anyone know how much each additional user costs with Helix Core?
I can't seem to find that info anywhere.
it isn't public, you have to contact them for a private quote. I think if you look enough you'll find people usually claiming they're paying around $40 per user per month or a little higher
so once you get that 6th user you're looking at like a $3000-ish bill annually
[don't quote me on that]
Wow...that's kind of a big jump, lol
yeah it's basically... completely retarded 
$420 per user per year, so $35 per month. I know that Bitbucket operates a similar model - you can get 5 users for free, but once you want the 6th user, you have to pay for all 6.
^ is what we pay, anyway
Yeah - that sudden hit for the 6th user is pretty lame. Should be gradual imo.
Most offerings I've seen like that just make the first N always free
or have a lower base price once you pass N users
I'll reask/phrase this question: how do I remove an exclusive lock from an improperly deleted repo(perforce)?
Good storage for unreal project.
I don't mean HDD or SSD.
You can ask on my question.
Please?
What?
I'm so happy :) finally got all sorts of binary merging & diffing for blueprint based assets :D
using p4merge + binary merge driver in git
Wait, what? What is this blueprint merging you speak of?!
Hey folks - I don't suppose there are any Perforce Server admins/experts around are there? I'm having major trouble trying to migrate my P4 repository from Windows to Linux using p4migrate despite following the documentation to the letter every time. When restoring the checkpoint on the Linux server, I still get errors about case-handling mismatches as well as a specific line error.
Server version 51 is replaying a version 46 journal/checkpoint.
Case-handling mismatch: server uses Unix-style (-C0) but journal flags are Windows-Style (-C1)!
Tried every combination I can think of at this point, even trying to run P4 in case-insensitive handling instead, but every approach I take comes to a dead end. Appreciate any tips before I sacrifice 3+ years of changelogs to the source control Gods.
Also, as there are actually no case-issues with the checkpoint reported when using p4migrate, and since the depot files have successfully had their case updated on the Linux server - does anybody know if I can force-replay the original checkpoint file with -C0?
@muted gulch
Sorry I have never moved a server from Windows to Linux, and I never had a failure replaying a checkpoint.
Did you open a ticket with Perforce? In my experience the support has been very good - even on the free tier.
I did open a ticket yeah, but the e-mail I got back essentially said "you have no license so we'll transfer you to sales"
I'll see what they come back with, it's odd that they let you file a ticket without a valid license if all they're going to do is transfer you to sales...
that sucks, I had an issue about 9 months ago and they helped me resolve it no problem.
Yeah! I'm hoping they'll give at least a nudge of help if nobody else knows
why is source control seeing this redirector?
I've been checking through my pending to make sure that file doesn't still exist and any deletion revisions were committed - what else should I be looking for?
(and submitted list too0
@muted gulch Did you use -C1 when you rebuilt the database from the checkpoint?
@winged tiger yeah I tried that, still get the same error message
Are you migrating to the same version of Perforce?
You should migrate to the same version that your windows box is using
And then upgrade the linux box post migration
Don't try to migrate different versions of Perforce
Ah so one problem there...
Yeah
That is what that error message is talking about
Server version 51 is replaying a version 46 journal/checkpoint.
Your linux perforce is ahead of the one you are trying to restore
Which is going to cause issues
I can't get the same version of perforce on Linux, as they don't have that version for Ubuntu 20.04
Do you think I could update the windows server version instead then try?
yes, that will work
Okay, I'll try that for now!
Still confused about that very specific line error too
Also I don't suppose you know, can you delete checkpoint files?
Actually.. I'll keep them just in case ๐
yeah you can delete whatever you want. But I would wait until the migration is complete before doing that
So upgrade your windows perforce and then redo all of the migration steps, with the new journal/checkpoint generated after upgrading.
Thanks! I'll give that a shot!
No problem ๐
@winged tiger Dammit ๐ฆ Unfortunately got the same issue again. the checkpoint file is bigger now, but still getting a replay failed at line x error and the case-handling mismatch message too
Strangely the line is always around the 5,000-ish mark. Starting to wonder if the file is too big, but it's only 42mb :/
Your checkpoint file is 42mb? Or journal?
It gives the same line every time? What is that line?
the checkpoint file yeah, I haven't uploaded the journal
well, it's a different line this time - but with the last checkpoint I tried, it was line 5,012 or something. This time it's line 5,111
But there's nothing wrong with the lines themselves as far as I can tell
@pv@ 0 @db.integed@ @//Stormtide/HTGame/HTGame/Content/Vehicles/Dropship/Animations/ANIM_VehicleHangar_Close.uasset@ @//Stormtide/HTGame/HTGame/Content/Vehicles/Dropship/Animations/ANIM_VehicleHangar_Open.uasset@ 0 1 0 1 6 1190
no different to any of the other lines really
I'm just surprised I'm still getting that case mismatch error too
Why does your checkpoint have .edited on it? Are you manually editing it?
no that's just the output file from p4migrate
Are your linux and windows machines the same byte order?
i.e. are they both 64 bit or 32 bit
yeah both 64 bit
hello, first time posting here. It been some time since I had to mess with P4 and I'm finding that UE 4.25 won't hold my settings. I'm fairly sure there is an INI where I can enter in the settings manually and permanently but I can't recall which INI. Can anyone point me in the right direction?
Did you enable the p4 integration in the editor? If you did, did you click the "check out" button that appears at the top of the settings window? If you don't, unreal likes to discard your settings.
To actually answer your question though, configs are in the Config folder. Which config depends on what setting you want to change.
I appreciate the reply. Thanks. In the editor it comes up each time with "None (Source Control Disabled). When I select Perforce it populates with my settings minus the workspace. If I expand the area beneath my "available workspace" the Host area is populated with my P4 systems address and port. if I clear that section P4 then locates my proper "available workspace". P4 now works, at least until I exit, and then I have to reenter all of that again.
The settings I want to hard enter are the most basic ones: server, username, workspace
Saved/Config/Windows/SourceControlSettings.ini
[PerforceSourceControl.PerforceSourceControlSettings]
Port=
UserName=
Workspace=
[SourceControl.SourceControlSettings]
Provider=Perforce
Odd mine is completely blank
That file should autosave when you accept the p4 settings. The only reason I can think of it not saving is if the file is read-only... did you add your Saved folder to perforce?
was working properly prior the update to 4.25
I'll check the permissions
that makes logical sense
definitely make sure you didn't add anything in saved to p4, don't want to submit that stuff...
regardless, you should be able to edit that ini and put in your settings
Yes, it was read only. Good call. Setting adjusted and saved. Thanks
Hi, We're trying to use perforce with an local server, everything works except the .p4ignore file is not working properly. It's sending the binaries, intermediate, saved file for no reason. Maybe we didn't follow the right process to make the ignore file.
We just create the file and copy/paste one we found on internet.
We didn't use any command, maybe we have something to setup before
Yeah it's not very well supported - you need to run this cmd on all your clients
p4 set P4IGNORE=.p4ignore
hmm, not that I know.
What happens in p4v if you right click the workspace folder and choose 'Open Command Window here' ?
I think p4v will use environment variables if it can't find them in p4 set variables
you could try setx P4IGNORE .p4ignore from cmd window
then restart p4v
the p4ignore option is used in p4v when adding files to changelist so you should know straightaway if it's working or not
It will delete the folder in the server depot ?
directly or the admin will have to do it
So I download p4 cli, then do p4 set P4IGNORE=.p4ignore , mark all the folder that I don't want to be deleted ?
That's a fcking mess to setup x)
yes but it's an important mess ๐
this is a good one imo
https://gist.github.com/trentpolack/428594c91019401e46ecd23195956cc3
replace depot with whatever your depot is called and just use it
ok you should be able to use it straight up then
That's just copy paste ?
yep
On server or every clients ?
just once on one client (with admin rights)
this has some useful info in: https://www.perforce.com/video-tutorials/vcs/start-collaborating-unreal-engine-and-helix-core
that's better to restart from scratch ?
Because we already put the full project in the depot
it should be fine to fixup these few things
where is your server hosted?
the reason I ask, is deleting something does not free up the space (obviously I guess!)
if you want to actually free up the hard drive space on the server you'll need to use the p4 obliterate command
The server is on my friend's computer
You don't have to use the p4 cli for this, no, just set OS environment variable. It's easy on windows
But that means that there's only one P4IGNORE per computer ?
What if you have multiple workspace ?
or maybe I didn't undersand workspace
If you have multiple projects using different ignore filenames, a cleaner workflow instead is to set and use P4CONFIG
Then in each project/team you can specify the ignore file locally for that workspace
(In a config file)
So creating an .p4config file in the workspace with P4IGNORE=.p4ignore in it ?
there's no path information in P4IGNORE=.p4ignore it checks each level of the folder so you can re-use the same var for all workspaces
assuming you always call the file .p4ignore
That's look so weird coming from git
You set an p4ignore in the OS environment that check all the p4ignore in all the available workspace ?
That's right
Or you set p4config instead the same way, and in the config file you can set unique ignore, password, user names for the workspace
Do you have any documentation, or tutorial to follow ?
To be sure to not mess everything
Just Google p4config, come back if you can't figure it out
I include template config and ignore files in my depot for users to copy and modify locally
- Edit your p4config file and set:
โข P4IGNORE = the ignore file you just copied above
โข P4USER = your Perforce username (optional - Rider can use this for source control)
โข P4CLIENT = your Workspace name (optional - Rider can use this for source control)
โข P4PORT = our repo server address
- Edit your p4ignore file, ensure your config and ignore file are both entered
Just random copy pastes from my own notes - still look at official docs for more
So I download back the P4 CLI, open an cmd and set the command set P4IGNORE=.p4ignore but when i'm submitting, it still send binaries and shit folders
Do i need to restart ? or do something to check
Did you already submit those to perforce before the ignore was set up?
It will continue tracking files that are already in it. If so you might want to go into the admin program and obliterate the stuff that was supposed to be ignored
Yes they were already on the server
Ok we obliterate all the folders that should be ignored, we test that
Obliterate is generally considered a "dangerous" command but I've done this a few times just fine. It might mess up the commit logs for the changes where these files were submitted. Usually not a big deal
Is there anyone here good with Perforce that could quickly hop on a call and try to sort out problems? Happy to pay in advance.
@mushroom sause
<@&213101288538374145>
got it
hello i will need help for a collaborative connection i think i use perforce but i don't understand how other users can access the UE4 project modify it and send it back to me thanks in advance
through the magic of internet
your depot just needs to be accessible by all users over their network
Perforce works as a server client system. Each client has a copy of a branch of the server. You submit your changes as part of a Changelist to this server and then others can sync them.
I think it's a pity that there is no simple solution it's a real lack thank you for the info I'm going to inquire
domage*
It doesn't really get much simpler tbh
yes except when you don't know ^^ and there are a lot of videos that are all different you would have a reference video ? it would help me a lot
@muted gulch
Note - this video does not cover providing access to perforce server for remote users
if you are hosting locally you have to use port forwarding on your router to allow access, or you can host on a VM in the cloud
Has anyone here used Plastic Cloud before?
There seems to be no option to increase or buy extra storage space for the repo.
I haven't, but from looking at it previously I thought it was a "pay as you go" type of deal, where there's no hard limit and you're just charged for what you use
i.e. you don't increase/buy more storage, you just get charged more on your next bill when you use more
Does plastic have a ignore; like a git ignore? Trying to figure out how to do that
Perforce: Our project is 2 gigs, yet the depot is almost 30 gigs and trying to kick us out of the free AWS tier. I'm not finding the right keywords to find anything on this issue, can someone point me in the right direction?
Did you set up typemaps on the server to limit the version history of binary files? Have you checked your depot with the admin interface to see if there is stuff in there (that is now deleted) that shouldn't have been put in there and could be obliterated?
Hi there, got an issue with Source Control : I deleted a file media source asset referencing to a file containing spaces in its name. When trying to submit to source control, I've got fatal issues. The file causing the trouble is called "Animation logo Teapot-version courte.mp4" (I know that this is not a good file name ๐ ). Oh and the file does exist in the git.
I tried re-importing the file with no success (the issue still persists)
Well I succeeded in re-implementing an old version of the file. I'll keep it unused somewhere instead of deleting it.
Hi guys, how's it going? I have an issue with version control now(GitHub desktop). one of my workmates was working on a level in 4.25.3 and I use 4.25.1 DQS. We merged branches: MyBranch -> main; WorkmateBranch -> main; I tried to merge on my PC, than some conflicts came up in GitHub desktop but they were auto resolved.After the merge, the map is gone from Content browser, but present in the file system. If I switch to his branch on my PC, it works, but my changes are gone. If I copy over the level, I get a ton of BP compiler errors, bad cast nodes etc... Anyone run into this before? Thanks
\o/ Don't think its an unreal issue, since you use extern git to handle your stuff. So I can just give some tips to prevent this: Don't use merge, use rebase. If you are not sure if a merge/rebase succeeds, push your changes at least into a local branch to reroll your changes.
Don't have your engine running while pulling stuff. Check your code matches your data. Don't work on the same assets. (You can use git lfs to lock files)
it would be very strange if the map auto merged conflicts
cause that's a binary file
auto resolving would corrupt your stuff
which is the case here
I think the easiest thing is to just checkout this specific file from either your commit or your friend's commit
one of you will lose your changes, but there's no way around that
can you drop me a link for a tutorial or something?
I guess its best to not to work on the same umap files at the same time
but I still dont understand, after merge conflict, it should use the newest version, right? in this case my friends one, because that is the one we merges last
yeah that should work if you pick his version
except if his is corrupt
you should probably also be using the same engine version though
I read about umaps dont like jumping around engine versions
Not a good idea to use different engines for many reasons unless you like tracking down strange bugs on a regular basis!
Yes you should not work on anything in the Content folder at the same time.
That's the reason many people use source control that supports locking (like perforce / svn) or extensions to Git that support locking (sorry I don't know much about git).
ok thanks!
my teammates not really into version control, they dont know how to use it and scared of it, but without it everything gets messy buggy and just difficult to deal with. but if I mess up with Git just like now they might blame me. not sure how to deal with it. would you be able to carry out gamedev from empty project to packaged alpha without any sort of version control?
could you? maybe, should you ever? absolutely not
it enables such easy content sharing and gives you complete version history of everything in your depot so that if something breaks you can just revert
or if you need something that you removed 3 weeks ago in code somewhere, you can just get it back
it has so many advantages
how else would you work to share project updates
that's what I tell 'em but I have to tell them harder ๐
Sell it to them that it's great learning if they ever want to work in professional gamedev industry ๐
but how would it even work logistically to not have it
just send files over to everyone all the time?
what versioncontrol would you prefer for a team of 3 mostly BP based project
if you're working on mostly binary content like BPs definitely some centralized stuff like perforce or svn
perforce is probably best
absolutely copy pasting is just causing bullshit
Perforce is always my go-to, and it's free for <=5 users.
It's quite easy to use for the end user and less likely to screw up than git.
but it does mean you need to setup a server somewhere (cloud or local machine)
I like git a lot more, but then again, we do mostly code so no problems with binary files
if you were to use git with that you'd definitely need to setup git lfs as well
done that