@open bone
#source-control
smoky trench
open bone
stoic wren
https://allarsblog.com/2014/09/24/setup-perforce-digital/ @burnt shadow I followed this guide few months ago to setup my perforce server and it has been working fine until recently I have had some connectivity issues with it. So yesterday I went and rebooted the server. Rebooting worked back then but what you know, the p4dservice doesn't work anymore on my server. I tried to recreate it but just doenst work.
So now I have to manually start the p4d and set the root directory
burnt shadow
that is pretty shitty
did you upgrade ubuntu or something
I have absolutely zero knowledge on how to repair unix services/daemons
if I did I'd make a ton more money
burnt shadow
if you know the service is fucked I'd hack it with a cronjob trigger using @reboot
most likely not the best solution but
I haven't read this page I just googled 'how to do shit on cronjob reboot'
shove your p4d boot in there
yolo
leaves
stoic wren
I'll check it out
wild sage
Hello friends
I'm curious:
I have Git source control set up. I'm on my laptop right now and realized I forgot to push to rep on my PC yesterday evening - what would happen if I change/push stuff now? 🤔
rotund bobcat
if you push same files there might be conflicts that you'll have to resolve, other than that nothing special
just pull on your other pc again next time before you try to push your stuff
wild sage
Thanks @rotund bobcat!
carmine walrus
if you have conflicting binary files then the only real option is to accept either change
wild sage
OK, that's fine :>
uneven coral
Question. We recently developed a few scripts to compile the cpp and shaders and whatnot outside of the engine for when we have large syncs or landscape changes etc. However part of all this conversion was to stop the syncing of the .sln and .uproject file.
mainly so you dont have to check it out to run the builds.
However when using UE4 to submit to P4 it wants to checkin the .sln and .uproject file to the repository.
SHOULD those files be synced, Is there data that changes in them that the other devs should be using.
burnt shadow
.sln should not be synced
.uproject is usually synced
do you mean .csproject and .csxprojects? those should not be synced
uneven coral
yeah we pulled the sln for cpp compiling purposes.
what are the csproject and csxproject files?
also we should be syncing thr uproject then. okay.
also edit: it does not try to sync the .sln.
its trying to sync the uproject and the defaultengine.ini file.
burnt shadow
recompiling shaders should not require changing of the uproject file?
uneven coral
no not at all.
other than being the project file, i just wasnt sure what uproject contained with data.
we synced those files for the last year but we didnt use cpp until the last month. haha
wooden forge
ok so I have never personally set up assembla perforce before.
I've followed all the "setup" steps but noting I do seems to work
Currently I've managed to make a repository inside of my space and downloaded p4v
from that point on I have no idea what to do next
wooden forge
so I've managed to get p4 talking the the server but when I try to login it says slashes (/) not allowed in 'ue4jam/Spring_ue4jam_33'
despite following assembla's own instructions
echo P4HOST=ue4jam/Spring_ue4jam_33 >> p4config.txt
wooden forge
@burnt shadow you provide cloud services?
haughty ember
@vocal moat ask your question so we can figure out what you need 😃
(most people around here know Perforce "more than a little", so just ask whatever ails you my friend)
vocal moat
I can't because it's related to UDN :P But I did get an answer on UDN shortly after
I suppose the only remaining question is this
Say my workspace is mapped to stream A
And I have CL10 on stream B
But the streams are read only
So I can't integrate it
Is there any way for me to get all the diffs from CL10 to apply manually on the local copy of my files?
Something as simple as "here are all the files that were modified in this CL"
If I souble click a file in the CL browser I get a copy of it in my temp folder but if a CL has 100 files that quickly becomes unviable
P4V has "show file diffs" which is even better as it shows all the changes in one window... which then can't be exported and would have to be copy pasted one by one because you can't select multiple entries either.
Every path seems one step forwars two steps back
Apparently perforce has a patch utility... that might help.
blazing inlet
has anyone ever set up a hansoft server ? for agile management
haughty vault
I'd try hack'n'plan. Hansoft is pretty much overkill even for mid sized game teams.
probably better suited for #ue4-general or #lounge tho since it's not source control related.
jolly fog
Need some help getting my local repo onto cloudforge, im using svn and i cant create a dump
thick knot
Hello everyone!
In our project we use P4. We have access to Epic's P4. Right now we're on 4.21.1 and we want to upgrade to 4.22.1.
Our setup is something like this:
Both workspaces share the same path (use the same local folders).
[workspace for Epic's P4 depot] ------------- [workspace for our project p4 depot (assembla)]
- Download files from here
-
check out files here. Remove unchanged files -
submit files
I switched the stream in the left WS to 4.22.1 (Release - 4.22.1) and I want to do the same process (download things on the left, upload things on the right)
I don't see a way to do this and keep our modified engine files or, at least, have an automated merging process that will keep our changes or warn us about merge conflicts.
And here is my question: how do you guys handle this? I know there's a Depot to Depot integration in P4 but I heard it's super slow. Any experiences with UE4 and that? Is there a better/different approach?
Cheers!
I bet this is been asked many times here, so if you can point me to a place in this thread or to a pinned conv, that would help too
thick knot
Update: we found a thread explaining something like this. Does this make sense to you?
river orchid
how can I use Perforce with Microsoft Azure Cloud?
hybrid musk
Is it possible to use Perforce and Git together?
I want to use Perforce for all the big files and GitHub for the code
haughty ember
@hybrid musk https://www.gamasutra.com/blogs/RyanDarcey/20161021/283799/An_Unholy_Alliance_Unreal_Engine_GitHub__Perforce.php
clear gull
Guys I'm using Sourcetree + GIT to commit changes to my github. I'm wondering if it worths to migrate to Github Desktop instead of Source Tree ? any experience
lilac aspen
i'm using SmartGit for GitHub. it's not free, but well worth the money. the nightly versions have LFS support.
rotund bobcat
tbh GUI wise you can use whatever you want, just pick what pleases your workflow/style the best, it's nothing more than a more fancy command prompt anyway 😛
thick knot
Anyone here has tried to upgrade the engine using P4 Branch?
I'm trying to do it following this advice https://answers.unrealengine.com/questions/153551/how-to-upgrade-a-version-of-ue4-code-build-to-a-ne.html
and I'm at the resolve part. Doesn't matter if I try using a stream or a branch, P4 detects ALL the files as conflicts. Auto-resolve seems to be stuck and I wonder how will this behave when it finds a binary file like an .exe or a .dll.
My alternative is just replace all the files and then track down our engine modifications and re-implement them by copy/paste, but I'm looking for a smarter approach.
Any suggestions? Has anyone here tried to upgrade the engine using P4 (both Epic's and own)?
I really need suggestions/input in this
blissful apex
The resolve is generally the longest part. Just to be sure, you're integrating from your UE4 vanilla branch to your project's UE4Engine folder
For binary files you'll want to take the ones from the UE4 Vanilla (depending how you've decided to handle binary files in P4)
thick knot
Hi Lliam, thanks for the answer. I ended up doing this:
- Vanilla source for 4.22.1 in our depot
- Branch our current engine (4.21.2 + couple engine mods)
- Delete folders from the Branch and copy/paste the new ones from the Vanilla Source
- Reconcile offline work and submit all new files in the branch
- Merge Branch into main
For resolving:
- Source: automatic + locate all the modified files and manually diff them to recover our changes
- Everything else (Content, Plugins etc) accept source (the newly copy/pasted folder).
Then Re-install plugins
Unreal responded this last night to this same question:
Hi Bruno,
We generally encourage licensees to set up a local repository, with a "tracking branch" that mirrors Epic's latest release. You can then reconcile the contents of that branch against each new release without having to worry about conflicts, and merge from there into your game's development branch. This separates our changes to the engine from changes that you make to the engine.
This might be a bit trickier in your case given that there have been engine files modified on your end, but should be solvable by null merging your changes into the tracking branch.
proud gust
Hey guys, i have a question: Live edit was implematet, but its not Working for ne if my friend want to join the session they cant see it... Server is right, hammachi aktiv on all pcs ans working but it doesnt work.... Can someone help me?
hybrid musk
Has anyone here used GitLab for UE4 projects?
Been thinking about hosting an instance on our production server
quaint obsidian
I have
hybrid musk
@quaint obsidian is it good?
quaint obsidian
Yeah
hybrid musk
It has LFS right?
quaint obsidian
Yep
hybrid musk
aight
finite axle
Does gitlab support lfs by default? It talks about an enable lfs option in forums, but there is no option for that in the repo settings currently. I'm guessing it's standard now
No, it's there. My bad
haughty ember
Oh, while I am not sure I would recommend Assembla for most studios because of their ridiculous upcharge on Perforce.
They did recently create an "Indie Tier"
$100/mo for 5 users, limited to 100GB repo space.
Obviously, I still recommend Vultr (or Digital Ocean, or similar) for $10/mo and just pay for space as needed
half sluice
can someone tell me why this isn't affecting all files in saved dir?
there's no whitelisting for those files
nevermind, when i've created repo VS pushed all files according to its own default gitignore file. this file is already listed and can't be simply deleted with gitignore change :/
quaint obsidian
@half sluice git ls-files -i --exclude-from=.gitignore | xargs git rm --cached
lone owl
Hey all, I'm trying out the source control features for 4.22.1 and I'm not seeing the option to 'checkout' a file from the context menu for that file. I'm interested in using that feature with my team so we don't trip over each other as much. Testing it now on two machines, it doesn't seem like one knows what the other is doing.
anyone have any experience with this? Here are my Source Control settings in Editor Preferences. I'm using Git, btw.
lone owl
Is this because git doesn't support file locking?
lone owl
@quaint obsidian Yes! I am!
All of our .uasset, .umap, etc. files have been added to LFS
lone owl
Sweet. Thanks @quaint obsidian
hybrid musk
Hey just a quick source control question
Is it a good idea to make a second repo for all the big art assets, and then add it to the main one through a submodule?
iron bronze
if you got a large team I would advise to do so yes, you could balance the server charge that way
quasi frost
Ive got a guy here getting this error out of UGS: "Archive manifest signature does not match." As far as I know theyre the only person having an issue pulling things down with UGS. Is there some easy diagnostics I can run or file I can nuke from their local machine to get them moving?
shy zealot
Hey all, any suggestions on git vs perforce?
smoky trench
Really recommend perforce, can't emphasize that enough.
Git struggles with UE4 even with LFS. Git is passable for super small teams but i can tell you now it's not made to handle UE4 projects
quaint obsidian
We've found great success in using Git in a 70GB project in a team of 100 people working on it entirely remotely
neat grotto
@quaint obsidian That's great. What's your workflow like? How did you manage to scale git so well with a large game dev team?
quaint obsidian
We have our content team (artists, level designers) working on one content branch. Our programmers use a standard feature branch model that is merged into a dev branch. Content is merged over to dev daily automatically, and dev is merged weekly according to a QA cycle.
Once we implement unit tests, we may enable automatic merging from dev to content if possible.
Binaries for plugins and the game are in submodules, which store DLLs in LFS. These are compiled automatically by the build server and submitted.
We use a custom engine which is downloaded automatically using a git hook for ue4versionator, software we made for this purpose that parses the .uproject file for a custom version to download.
We use a fork of the latest UE4 git plugin with a few optimizations, like multithreaded multi file locking, caching support and tick optimizations.
This is what people use to lock files for modification.
The latest git plugin also checks the origin for file revisions, so even if a file is unlocked after push, one cannot modify it until they pull the new version.
We use CloudFlare and have aggressive custom caching to make sure a majority of our LFS content is on the edge. We are working on using CloudFlare workers to enhance caching for custom requests.
Finally, we use a launch script that pulls the latest, and does an autostashed rebase and then launches the uproject file on success.
Git is blazingly fast, especially with the commit graph feature and index optimizations made recently. It also has much better merge and branching tools, and also allows our devs to work extremely efficiently even if their connection is very weak.
Additionally, we don't have to worry about traditional backups due to the redundancy we can have with the flexibility of git and GitLab.
valid moat
can I submit another users changelist with p4?
molten marsh
Not remotely. That would make no sense.
valid moat
hmm so my windows drive died, I had some files checked out in p4; now I reinstalled everything and p4, but the files are checked out by my user, but a different workspace
i guess then I could copy them somewhere, revert them, then copy over and submit them?
valid moat
hmm I can try that
the last time I did that p4 deleted all the files in my hard drive in that location 😄
valid moat
so I just have to load or map to the existing workspace that its still there
that was easy, just had to edit the workspace to use the host the new machine name and done!
noble knoll
people allways forget that SVN exist
and its better than git (significantly so) for ue4 projects
@quaint obsidian slightly offtopic, what did you do on the tick optimizations?
uneven coral
@valid moat if your comfortable with p4.exe as opposed to p4v, you can actually delete the old workspace with a single command which checks in all the files
I had to do it for a member of the team who had his pc crash and had about 100 files checked out.
valid moat
@uneven coral hmm nice, i'll keep that in mind next time. I ended up using the old workspace and checked them in that way
valid moat
@noble knoll I use svn at work and it's a lot easier and simpler. but I have hundreds of commits on personal perforce work and dont want to lose them
noble knoll
people allways act like its perforce vs git
everyone allways forgets about svn
svn is actually the best for an indie team who isnt going to do branching (branching suuuuuuuuuucks in svn)
valid moat
as an artist, git? I have to type stuff to commit? 😄
uneven coral
but branching is important imo for devs who are getting close to release to have at least a Build and dev branch at minimum.
noble knoll
artists tend to like svn a lot
specially with tortoise
nothing easier than clicking the file you just changed and hitting "commit"
uneven coral
no comments needed i take it?
noble knoll
build + dev branch in svn is no issue
valid moat
yeah tortoise is awesome
noble knoll
the issue is when you want to actually do stuff like feature branches
uneven coral
we actually just installed Fisheye to tie in Perforce to Jira bug tracking so it index the notes.
noble knoll
you can do that in svn or git too
sharp fable
working with artists who were using source control for the first time perforce wasnt that hard for them to learn, for the most part it was just using the built in functionality
Also, i want to get UGS running on my perforce server.. is there a guide on how I can do that?
the ue4 docs dont really go into the process of getting it up and running unless they've added that since i last looked
quaint obsidian
@valid moat why would you need to type stuff besides the commit message?
@noble knoll We improved the async/sync handling and used switch sleeping (0) instead of 1ms
noble knoll
interesting. In PUBG we have too many tickers, and are investigating options to optimize shit a bit
one of the things we are thinking is to make sure the tickers are sorted by class, and we call an Update() function (not a virtual)
useful for things like weapons where we can have up to 300 of them
quaint obsidian
Ah, yeah, we aren't really experiencing that sort of scale in our git plugin, it's just some simple tick code cleanup specific to the plugin really
thick knot
@sharp fable we just started using UGS. We used to have our own launcher, but UGS is all we wanted. No documentation at all as far as we're concerned. Only problem we had was that it relies in the fact that you're using the "official" project structure (the project folder is within the engine etc). But we use a different structure, so we had to change the code a bit for every part where the ROOT env. var was used.
The last 2 pieces of the puzzle for us are:
- The tagging system (people can tag builds and commits to let others know if it crashes or if it's a bad build) relies on a database/server that registers said tags and does other functionality (like pop a tray icon message when a new build is available I think)
- UGS distinguish between content and code change lists. We need to figure out if there's a way for the tool to auto-trigger a build when a code change is submitted. Don't know if that functionality is available or not (I'm not the programmer looking into it)
Apart from that, is pretty straight forward to use it. There's the server part for the tagging etc, and a client that get's latest and builds (if you have Visual Studio) for you.
We use jenkins for the builds and it's pretty cool because this thing uses the build tool scripts and, one of the steps it has is Cleaning PDB's. It makes them smaller and deletes the unnecessary ones. We have moved from a 1.3GB zip file (generated by us in jenkins) to a 300-ish mb zip file that it includes PDB's for Crash reports and callstacks.
I can't recommend it enough.
@uneven coral what kind of extra advantages do you get by using Jira+Fisheye +perforce for bug reports?
We want to improve our pipeline a little bit and, one part, is set up how we're handling bug reports
reef oriole
Ah, no worries, I did put my original Git plugin in MIT license
reef oriole
then when I noticed some pull requests activity around source control in engine last year I talked with him, so all is clear between us 😃
haughty ember
He is definitely off track
reef oriole
Interesting, I would love to hear more !
haughty ember
It would be one thing if he hosted
But he's not hosting
And wants people to buy the plugin for EACH project
Which is technically impossible AFAIK
I don't think I can buy the same thing twice in MP
😃
So, if he's selling the license on his website, on top of the plugin in MP...
And not really providing anything more than your own plugin
reef oriole
To be honest, I don't really care for the publishing side of thing
as for feature, yes he provides some more :
haughty ember
The built-in Unreal plugin does not support working with remote repositories, which makes it unsuitable for any team collaboration scenario. For example, your changes will not be shared, the history of an asset will only reflect the changes you made locally and you will not be able to share or update only what you need. You will also need to use the command line to push your changes to a remote repository.
That's the only difference that he stated anyway
reef oriole
Git is not very good when the team grows, because you need to keep updating: he provides something for this
=> but this is the whole point of a real source control for gamedev
haughty ember
But you still have to host it somewhere
You consider Git real source control for gamedev?
reef oriole
Yes, he just provides a plugin, but it's the same for Game Analytics plugins
My plugin is the best source control for quick prototyping; just work as a "save" for local changes, than publish to Github when you want to share or save more securely
Then for anything more professional, you would want some centralized version control where you don't need to have all content at the latest
So Perforce, Subversion, Plastic SCM in "Gluon" mode, and now GitCentral
Also Victor, this will get me some more motivation to improve the Git Plugin in Editor 😃
haughty ember
Heh, I don't think so
This is going to piss people off
And make them more confused
I mean, I think I know quite a bit about how Source Control in its various forms
reef oriole
ah, yes, and there is indeed already 2 git plugins from me...
haughty ember
😃
@reef oriole Since you know more about his stuff and Git in general. How would you rank it against Perforce, SVN, and Plastic?
reef oriole
I have never tried it, and have not use Subversion a lot, so I don't know / can't compare...
The good thing is using LFS enable using Github / Github, which are probably a lot "better" than any other source control hosting solution
@burnt shadow is also working on a new source control thing
haughty ember
Is LFS even better nowadays?
And File Locking works just like Perforce?
I thought it was broken
in LFS
reef oriole
LFS is slow as hell if you don't use SSH if I recall correctly
or at least File Locking is slow
But I think it can only improve, and is quite well supported now, with GUI and all
haughty ember
reef oriole
I have always been mostly using https for personal stuffs
haughty ember
Man... why can't we have some real Perforce competition
At these prices
(pulled from Plastic)
reef oriole
Also interested by you complains with Plastic SCM 😉
haughty ember
It is sort of exciting that more and more VCS are coming out
Definitely need more and more options
But Perforce isn't feeling it still
Their perpetuals are still a bit out there.
quaint obsidian
Oh that reminds me, I need to stabilize and isolate my changes to your UE4GitPlugin and submit them as PRs :)
haughty ember
@reef oriole wait... You work for him? Lol
reef oriole
Ah ah, you found out, nice job!
haughty ember
There's nothing you can't find out nowadays ROFL
reef oriole
Not that I have been hiding this previously 😉
haughty ember
Uh huh lol
reef oriole
But in this particular instance, I don't want to interfere, nor do I want to be involved on his project
haughty ember
Oh, so you didn't (help) create that at all?
That's super strange in my eyes
But k. To each their own.
reef oriole
He actually worked on this for many months in 2017 before we met
reef oriole
The we talked, he liked my work, and at the same time he was hired on a starting game dev studio
Darewise
It sounded interesting to me, and I ended up hired 😃
Today we are releasing a pre-alpha closed live test of our first game, Project C
So to get back to his plugin, he worked on it on weekends, we never talked that much about it, never had to help nor to have a look at the source code, nor testing it
(Working on the tech side of a "MMO" game is already a shit ton of work... but you got me side-tracked 😉 )
@quaint obsidian I would gladly integrate your changes, thanks 😃
Though I have had a hard time working on side projects like that recently
I think the most interesting feature to add to the official integrated Git plugin would be a "push when submit" option
That would fail if you are not up to date, so perhaps with a fetch to prevent this, or a pull to try update before submit (risky)
But a least, a simple "push when submit" would be nice for me when working alone with Github as a publishing/cloud save
lofty warren
is it possible to backup only blueprints, no meshes, textures, etc?
backup gets huge with textures and meshes, and I can refetch those from the marketplace
summer lantern
just spotted a bug with UE4's perforce implementation
if you delete a uasset that isn't reference, it's deleted in the editor, and marked for delete in P4
now if you rename another asset to the name of the previous, it doesn't update P4
so you get a redirector, and an asset marked for delete
...now I've got to work out how to undo this mess
molten marsh
Might be a backup in the saved folder
dusk dove
well undoing the mess is pretty easy
reconcile in p4 then revert everything
but it's annoying bug
summer lantern
yeah, I don't need a backup - my local files are correct, the problem is in source control
reconciling might work
dusk dove
it also happens when you rename something, and then rename something else to the previous name
fucks up the p4 too
molten marsh
Ah
summer lantern
yeah, Epic's P4 plugin doesn't take this into account
dusk dove
should probably report the bug
summer lantern
hmm, can't see a way of reconciling - where's the option for this normally hidden?
dusk dove
are you using p4v? right click a folder and "reconcile offline work"
summer lantern
aha, right click on directory in depot view
yeah
I was right clicking the file
cool, hopefully that worked
looks correct, cheers @dusk dove
dusk dove
made bug report, wtf that took long to describe sensibly
proud gust
hey guys live edit is a kind of source-control right,? but how to use it I mean I can start the server and that stuff, BUT if my friends are in the same Lan(Hamachi or OpenVPN) it wont work.... they cant connect to it, cant even see the server.... does anyone use that allready and would be able to help me=
proud gust
@sharp fable and where do I have to post it/ ask for it?
dusk dove
has someone made a thing that blocks checking in files failing to follow the naming convention?
haughty ember
Is that even possible?
I mean, I'm sure it is technically possible somehow
You have Allar's linter thing
I suppose you could jimmy that up to the p4 plugin
And have some intermediary making sure that you can't upload a certain uasset type without the proper naming conventions
Might be better off having people just read the naming/code standard/conventions you've already set into place somewhere
dusk dove
yeah but every now and then someone still checks in a file with the prefix missing accidentally
mortal siren
We have a fork of UE4 github source that we're trying to make visible to other people who also have access to UE4 github source. However people with access to UE4 source aren't able to see it. I suspect it might have something to do with the fork being part of an Organization. Anyone ever resolved this issue?
dusk dove
you can give people in the epic organization read access
amber blaze
anyone else have issues moving large folders in one action? svn gets error when there's too many assets
i have to do it incrementally
left lily
I don't have that much experience with SVN but my experience with source control in general is that a lot of them cannot handle excessively large commits
e.g. >100mb
I know git has a problem with it, and while p4 doesn't, p4 doesn't do checksums very well which results in inconsistent data
quaint obsidian
git lfs handles large commits fine
left lily
no it does not
to be fair I did mean large file size, but with something like umap file sizes you are going to have issues (also because they are binary). not straight from the git source, but essentially breaking down why git doesn't work outside of smaller size files: https://docs.microsoft.com/en-us/azure/devops/repos/git/manage-large-files?view=azure-devops
neat grotto
@left lily That article indicates that using LFS can be an acceptable solution for managing large files, which is what mastercoms said.
It's well established that vanilla git handles large binaries poorly, but LFS can work just fine for a variety of teams.
That said, the article does do a pretty good job of breaking all of that down.
left lily
LFS is wonky at best unfortunately. checksum issues are common, as are arbitrary upload failures. the reason I posted the article was because they literally say you have to have a strategy for committing large files. which is essentially like saying "it's not going to work all the time so try to minimize the amount of stuff that you do with it"
My personal experience with git for projects with large binary files (e.g. games projects) have mirrored exactly what I said. I do not recommend git/git LFS for game projects.
grave ether
@left lily Do you use Perforce or what?
quaint obsidian
"they literally say you have to have a strategy"
For vanilla Git, not Git LFS.
Git LFS works just fine with large files.
We have a 75GB repo with Git LFS, and it works just fine.
I am curious as to what you mean by checksum issues and arbitrary upload failures, and what service you're using where you're getting these issues.
left lily
first github, then private server. what I mean by checksum failures is that the repository reported different md5 checksums on the same files on different machines. git LFS still had issues allowing us to upload large files, as in, randomly failing the uploads for no discernable reason (we did a lot of diagnostics to find out the cause). eventually both of these issues meant that we had to ditch git
quaint obsidian
which git server software did you go with? and how long ago was this?
valid frost
jeez RTC, haven't seen that for years
left lily
was back in 2015
quaint obsidian
ah we started using it in 2017
I've heard git lfs back then in 2015 was pretty bad
pseudo osprey
that's like 4 years ago XD
ruby elk
Hi guys. How do you manage source control?
Is blueprint diffing really a thing? Is there any way to diff Maps?
Do you usually do lock file policies when someone is working in something or you do rebase override merges?
Did you find using feature branching a pain when dealing with Blueprints and other assets?
I'm happy to hear your experiences
fickle roost
everything is uasset in the end right ? so what would be the difference with blueprint : ) @ruby elk
ruby elk
so you mean the diff tool will work with any of them?
neat grotto
As far as I know, only Blueprints can be diffed in Unreal.
It's a purpose-built tool. Right-click a Blueprint class in the content browser and select Source Control → Diff Against Depot.
left lily
yeah we're using blueprint diff and it's pretty awesome. granted still not as good as text editors in seeing changes, but it definitely helps given that blueprint files are binary
rotund bobcat
does the blueprint diffing work with git or just perforce?
dusk dove
can I make p4 reject a changelist that contains an ignored file because someone forgot to make it use the ignore file again?
haughty ember
@rotund bobcat TBH, I haven't tried diff on git, but I don't see why it would be different. It works in SVN and Perforce so that tells me that's agnostic to the actual VCS platform
quaint obsidian
It works with Git
rotund bobcat
ok nice thanks for the info
@dusk dove I don't know for p4 but for git you have pre commit hooks, seems like perforce has this similar thing though https://www.perforce.com/perforce/r12.1/manuals/cmdref/triggers.html
ocean ridge
@left lily jesus you use RTC? I'm so so sorry
I spent many man-months trying to automate access to RTC for jenkins at my last job
the only good thing I can say is that it was a different hell than CVS, which they used previously
(yes, CVS in 2014....)
left lily
I'm sorry for you too, genuinely the best version control on the market. I mean I'm sorry that you have to go back to whatever you have stock in. At least I have md5 checksums certify my files but if you feel awful about that I guess there's always the new and improved git lfs 2018-19 (r)
ocean ridge
I'm using Plastic SCM, which has been a joy compared to both git and rtc 😃
hollow badge
i know theres no best source control option, but I am looking for what would work well for my team. I have some hefty content folders and inside folders and file names get renamed a lot. I'd like to do file versioning and or just be able to access a master version that we all pull from and submit to while retaining our own copies or however you'd word that.
not really sure what would be a good way to approach that
rotund bobcat
Perforce and SVN are centralised repos, which would probably seem like the best option for you, especially if you're talking about all binary files in this situation
If it's more code or text based assets, probably git
quaint obsidian
If you want to retain your own copy, Git is decentralized and allows you to have a full local copy with branching.
mighty ibex
How do you folks manage external projects, like blender projects and even their exported FBX files, etc? Do you have a folder in your Unreal project root for stuff like blender files, and FBX files?
Mine are currently just all over the place. Would suck to lose it all.
I use git and figured I'd just, at the very least, make a Blender directory in my Unreal project root and throw everything in there
dusk dove
SourceAssets folder next to Content with everything in it, that has the same structure and naming as the files in Content as much as possible, everything checked in to source control
mighty ibex
nice, thanks
shell lantern
Anyone know what : error on line 1 at column 2: StartTag: invalid element name Means? Trying to use visual studio to run a website for a ue4 game, just out o luck
ocean ridge
@dusk dove hah. I use SourceContent :)
fringe abyss
day 30 of git desktop and git for source control, going in between 2 computers , everything works beautiful.. 100% legit
and git lfs
dusk dove
hosting it where though?
dusk dove
it doesn't use too much bandwidth?
fringe abyss
no, its very efficient
dusk dove
the 1GB repo size and lfs bandwidth limit on github is far too low for most unreal projects
dusk dove
how did you put a 10gb project on github
huh it seems like they've increased the limit from 1GB to 100GB since I last tried to use it
fringe abyss
yeah
versed kestrel
what
fringe abyss
time to decrease that first unreal project size from 160 to 100gb !
fringe abyss
*disclaimer: your files might get lost
versed kestrel
this is what it says for me - still 1gb
fringe abyss
lol ive only been using it for 30 days so take it with a grain of salt
versed kestrel
dusk dove
dusk dove
I see
versed kestrel
it's billing tab
fringe abyss
dusk dove
are you not using lfs
dusk dove
fringe abyss
i followed a tutorial
i mean
everything
has been working
inluding blender fbx's
dusk dove
how long have you been using it
fringe abyss
wait so how am i syncing all this stuff in between computers if it says 1.88mb
dusk dove
that probably excludes lfs data
fringe abyss
ahh
versed kestrel
oh huh
fringe abyss
fringe abyss
is that expensive?
dusk dove
oh wait it includes 50GB of storage too
fringe abyss
ah i see, you can buy them in packs
thats pretty cool
so i have 30 days to finish my project?
lol
30 days to make a project or all your data gets wiped game jam
magic belfry
anyone pay for perforce? How much does it cost, no hosting?
I just want the software I can host my own server
dusk dove
perforce costs something like $400/user/year so actually paying for it is insanity
it's good for the free users
magic belfry
Bleh
noble knoll
50 gb on LFS will go in no time
dusk dove
how big is the pubg repo? 🤔
noble knoll
400 gb
and thats the repo itself without history
with history? probably more than 40 terabytes
LFS is trash, and cant handle repos of "useful" scale
for reference on the 50 gb
i tried to use LFS on DWVR
which is a 6 gb repo
it took me about 2 weeks to bloat the server beyond 50 gb
and btw. I was unable to commit the entire project as 1 commit
it would crash every single git client i tried
had to split the upload into around 10 separated commits
dusk dove
didn't epic say at one point their perforce for all projects was only 10TB
buoyant basin
is it possible to pull and get changes of the assets while UE4 editor is runing?
I am using Tortoisegit
buoyant basin
yea. it broke some of my codes. is there any alternative way to not close the editor every time?
I tried to use Editor Integrated git plugin. but It seems so weird for me. I an not familiar with Check in check out , ...
quaint obsidian
@noble knoll that's because of data duplication?
noble knoll
@quaint obsidian yes
@dusk dove perforce is the best for big projects
in fact.. kinda the only choice
SVN also scales well, but you better spam the fuck out of those HDDs
in perforce it uses delta compression for the binaries
dusk dove
interesting
in the settings it only says "compressed copy per revision" so I was worried it wouldn't
quaint obsidian
yeah, that is one of the big pain points, but we have a guy on our team working on bindiff and data dedup to submit to LFS
might be in LFSv3
noble knoll
yall need to remember that LFS is a hack
and its an unoficcial plugin
so you can use a system that can deal with bin files natively (svn/perforce), or rely on a hack on a system that was not designed for it
lfs will never be good unless its somehow implemented properly into git itself
quaint obsidian
what's the advantage of native?
noble knoll
doesnt bloat to hell
and doesnt need the users to periodically clean/reset their stuff to avoid their hard disk to bloat to hell
noble knoll
that is. Git will bloat basically by definition
bigger changes are needed to make it not bloat
git is amazing for code, but its not not designed for huge binary projects
and its also too complicated for artists almost all the time
quaint obsidian
right, but i don't see why git lfs can't be a long term solution
it provides exactly what git needs, a binary file object store
noble knoll
lfs, by design, will not scale to big number of archives
this is the issue i had with my own prroject
it needs to intercept the stuff one by one to see if the file is a lfs file, and its 1 by 1
if you have a ton of binar files, it will perform terrible and/or crash your clients
quaint obsidian
whats the definition of a ton
noble knoll
few hundreds
quaint obsidian
10,000?
noble knoll
my unreal engine project, medium sized, was enough to crash all git clients with lfs, and to fail mid/commit
quaint obsidian
I'm pretty sure our project has thousands of lfs files
noble knoll
clean repo btw
quaint obsidian
there have been improvements done to git core for filtering and smudging to optimize git lfs
noble knoll
i tried a year ago more or less
using self-hosted gitlab (comes with everything setup), on a LAN buildmachine
quaint obsidian
strange
i have no idea why our project doesn't have problems then
i gotta figure out what we did
noble knoll
maybe newer versions of both git and lfs?
all i know is ive used both svn and perforce without even the slitest issue ever
but lfs has been an absolute nightmare
people allways forget svn exists
think its either git lfs or perforce
svn is a lot better than lfs
specially for art or teams that wont be branching like crazy (svn branching sucks)
dusk dove
do you use branching at all with unreal projects?
dusk dove
I wonder how it would even work since code changes usually come along with content changes too
noble knoll
badly
on pubg we do branching (kinda needed for feature development), and its kind of a nightmare
dusk dove
so while it would perfectly work for the code the content is now fucked
quaint obsidian
we use branching a lot
noble knoll
for branching to work well in an unreal project, you need to setup things so you use data-assets on nearly everything
and your BPs are either art/display only, or "code"
this is also good practise anyway (unreal recomends it)
quaint obsidian
hmm, we dont do that :/, what happens if you dont?
noble knoll
you will have huge issues merging
dusk dove
what do you mean by bps are art only or code
ocean ridge
data table life
bah site is under maintenance but this would be the link: https://docs.unrealengine.com/en-us/Gameplay/DataDriven
dusk dove
data table is horrible it makes a change to any individual thing mark the whole table dirty
ocean ridge
you could also use data assets 😃
noble knoll
@dusk dove separate your display bps from your logic bps
some bps are for setting things like particle systems or cosmetic stuff for characters
while other bps contain actual logic, like AI
quaint obsidian
why would there be merge issues if uassets are locked?
dusk dove
but then things are constantly locked from other branches 🤔
quaint obsidian
usually other people don't touch those code BPs
open wind
How do I link my project build guys? Ty
rotund bobcat
what do you mean
fiery prism
Package your game and upload it to Drive
or itch
or steam
or imdb
or w/e kids are using these days
grave ether
Looking for opinions on whether or not to add source files from external apps (like Blender, Photoshop) directly into your project folder (in a folder called something like ArtSource or External). My inclination is that I would like to have all my art assets backed up and version controlled just like everything else in my project. I suppose it could be backed up in a separate way though.
I plan to use Perforce.
molten marsh
Thats not a silly idea at all. Especially if you also want collaborators to be able to work on the source assets as well.
south stratus
hey guys, any reccomendations for a free service, for a small team, working with UE4 on a prototype?
quick roost
@south stratus the problem is you’re probably going to hit the free limit on any service because of how big asset files tend to be...both BitBucket and GitHub cap free LFS storage at 1GB, and GitHub tracks bandwidth usage also
After that GitHub costs $5/month for each extra 50GB and Bitbucket $10/month for each extra 100GB
south stratus
That is exactly the issue we are running into now =]
molten marsh
Host Perforce yourself
Like locally on your machine.
I bought an old Dell PowerEdge R610 server and put it in my garage to host my personal Perforce (which can be accessed by external users) and as a automated build server.
Best $700AUD ive spent in a long time.
If your in the US you can probably get them at half that price.
If you cant buy another machine, just use your own.
quick roost
Yeah, self-hosting can work, but the downside is you don’t have a remote backup (unless you configure that separately)
molten marsh
Turn it on when others need to use it.
Yeah well you wont get everything you need for free elsewhere.
Unfortunately.
Your only downside with that solution is no backup against hardware failure and loss of data due to hardware failure.
south stratus
yeah that was the next thing I was looking at.
Either using an old machine as a server or getting a new machine just for source control.
awesome, thanks for the tips guys!
south stratus
yeah I just submitted for that, waiting for the email to come in =]
molten marsh
You dont need to ask them
Just install perforce and you have 5 free users with no data limit
Once you add a 6th you get limited
Unless you go and order a licence
south stratus
oh
So with perforce I essentially install a 'server' on my machine?
its not a cloud based service?
molten marsh
You can.
Yes
Its not cloud based no.
Its a centralized system, unlike GitHub.
Which makes it ideal for working with Game Dev.
Also the same reason why Github is trash to work with IMO
south stratus
awesome thanks ill check it out
github seems to be more optimized for code heavy projects, not so much in combination with other assets
molten marsh
Exactly
left lily
you can also look at something like svn
yeah I can't recommend git/github
not for UE4 anyway
south stratus
alright, excuse my ignorance, but how would I get other people to login to the server? I doubt the port number is enough for other people to access the server on my local machine?
quick roost
They would need your public IP address and also if you have a router you might need to setup port forwarding on it
Assuming you have a normal ISP you probably would want to setup a free dynamic DNS like noip
Because your IP address will change every so often
south stratus
yeah thought so
quaint obsidian
@south stratus GitLab can work too :)
rotund bobcat
anyone know if there's a way to run a script after pulling from a branch
I tried the git hooks but I don't seem to be able to get them working properly with sourcetree
digital light
$5/m for 50gb are not so bad
Usually my projects gets from 30 to 50gb super fast, thinking about it honestly
frank night
host your assets elsewhere
shell cliff
Alright, so i'm new to this whole source control thing, so bear with me.
I've been using Git for my solo projects, and this is the first time i'm going to collab with someone. I heard that when you work with Git, your team should be aware to not work on the same blueprint on the same time. This should be true since i don't see any way to diff/merge blueprints. Is this true?
I just discovered that there's a built in diff/merge tool in unreal itself, but it only does it locally (As far as i am aware). Is there a way for my team to work without the discomfort of not being able to work on the same blueprint at once?
rotund bobcat
not that I know of
the bluerprint diffing is the best you can get for that afaik
if you do somehow magically find a way, please let me know 😛
shell cliff
The source control feature in UE only "pushes" the files locally right?
im thinking maybe, let's say A and B both work on the same blueprint, and A pushed first to the repo
B just need to pull the files with A's changes and replace B's work with A's changes, and then use the merge tool locally to merge the current files (with A's changes) with the previous ones (B's changes)
shy zealot
what is the command to sync to your workpspace destination in p4? is it "p4 update"?
novel barn
left lily
git has a problem with binary assets
quick roost
If you use the Git LFS extension, it fixes the binary problem mostly, and LFS does have file locking support (though I haven’t tried using it)
But one thing I wish is that LFS would let you say only store x past revisions for some files, like the light build data for levels
Which in Perforce you can set that up pretty easily
quaint obsidian
you can git lfs prune, but you cant specify some files
file locking doesn't associate with a commit itself, so you can only really use it to let someone know youre working on something that you haven't pushed yet
once its checked into the server, it is usually unlocked
so people can modify it
ue4 git plugin has a new feature that prevents this
quick roost
Hmm prune only removes from the local storage though, right? I’m more interested in being able to save the remote storage space so I don’t have to pay more when I go over
quaint obsidian
ah, well i guess the problem there is not having those objects available in history, but rewriting old commits might solve that. though, depends on your provider for if the loose LFS objects actually get deleted
and of course, you have to rewrite history, which is another problem
it would be a nice feature, but it would have to be provider specific since LFS doesn't really dictate that itself
and I guess most providers are going to be overzealous about maintaining the git history
left lily
I've had a lot of problems with LFS, but to be fair we stopped using git back in 2015 2016
umbral tundra
Are the CPP files controlled and synced through VS? For some reason I can only control things in the Content folder.
haughty ember
@umbral tundra as long as they are added to repo
umbral tundra
perforce here
molten marsh
@umbral tundra You need P4VS plugin.
molten marsh
CPP files with P4VS are hit and miss ive noticed. I usually always reconcile offline work anyway to pickup anything that got missed.
faint cairn
is there a way of automatically enforcing UTF8 encoding via Perforce?
I justed enabled the unicode stuff on the server, but I can't find if I can actually have the Server force the Charset on the Clients?
molten marsh
I dont think so. Maybe the typemap has some funky thing it can do to enforce it?
faint cairn
I assume if other users submit stuff they might still give non utf8 files then
Annoying
molten marsh
Yeah its probably one of the few things that i dislike about P4
Tooooo many projects have i come across that dont know what a typemap is or dont have a proper ignore file and have to go through and clean up permissions and all this shit.
Really annoys me lol
faint cairn
I just spend 30 minutes on changing freaking encoding
I really hope it packages for linux now >.>
smoky trench
If your repo is utf8 and the user is connecting via a different encoding method the server will reject the client when it tries to push
errant moon
is there any way to turn off locking on source control? for when you are the only person working on a project
haughty ember
@errant moon uh, sure
Wouldn't recommend, but you could just overwrite and keep it like that
instead of checking out
But are you asking if there's a way to "check out/submit" without the prompt?
Obviously, there's also just disabling source control
And Offline Reconciling at the end of the day/stretch
errant moon
@haughty ember I mainly don't want the delay of it
I usually keep source control off and only enable when I need to diff as a result
haughty ember
you have your VCS outside your office?
errant moon
yeah
haughty ember
Ah, that sucks
I keep mine on-site, or if I need to, at a server very close by:)
I would probably recommend my last suggestion
Disable Source Control completely in editor
then just reconcile
errant moon
yeah that's what I do now
I don't know what you mean by reconcile
but I'm using svn
maybe that's a perforce term?
errant moon
I just commit through my shell extension (tortoisesvn)
haughty ember
yeah, so that should be all you need to do then
errant moon
and leave it off in the editor because of the constant little delays locking things
haughty ember
that's the file explorer, right click?
errant moon
yeah
haughty ember
yeah, I've only used SVN a handful of times
But that seems like the best way to go
errant moon
I also had to modify the source of putty/plink to make it a lot faster for remote servers
it had a fixed TCP window size so the more your ping was the slower it goes.. can't believe it is that widely used and has that problem
haughty ember
You might also reconsider having the server on-site
Unless you definitely know you need it in the cloud
You can still give access to your server for remote workers later
errant moon
yeah for stuff on my own it would probably be fine on site, as long as I did automated backups
errant moon
are there any mature options for self-hosted git-LFS now?
in the past when I looked into it the big ones like github's reference implementation said it was non-production ready and there were just millions of options for other implementations and I couldn't make my brain go through and evaluate all that
haughty ember
Last I tried was a year or so ago, and others recently. Seems like Git-LFS is still poop
quaint obsidian
GitLab @errant moon
buoyant basin
is perforce server free ? if I install it on my own machine
dusk dove
for 5 users and 20 workspaces
viral storm
anyway i can see a nice commit-history graph with SVN ? like there is in Sourcetree
i'm fairly new to it, but have more experience with Sourcetree and it really helps me to be able to view such a history graph, pick the commit i want to checkout and just view edited files especially the source code for the parts done in c++
rotund bobcat
never used SVN but this existss https://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-revgraph.html
viral storm
yeah that doesn't look like what i imagined ...
but playing with it some more, through more windows and right clicking i kinda can get what i want at least for the c++ source files so that i can see the changes
fringe abyss
today github hit me with the purchase additional data pack email
still though, 5$ for 50gb extra a month, bring total to 51gb for 5$ a month
very happy with github / github desktop / ue4 pipeline
fringe abyss
yeah the pro subscription seems worth it with protected branches
and unlimited users in your private repo
but im still a FFF studio
😆
cold lotus
^^
ocean ridge
FFF implies you have a big budget as a failing studio 🤣
left lily
so you're saying that FFF is the new AAA?
jolly fog
can someone explain how to add files to the workspace in perforce? i cant seem to figure it out. spent several hours watching step-by-step guides. i dont have the same buttons they do...
golden hemlock
hi hi, I was trying to use the git lfs plugin to use it with a project that I already have on github. (mostly to be able to lock file since right now I can't).
So after install the plugin, I try to use it with my project and lock a file but it throws an error saying that is not possible to lock. So not sure if I have to do any step more to be able to use it
should a convert my project files to lfs? or install anything else?
rotund bobcat
I'm not 100% sure if github supports locking, I know bitbucket doesn't
but your file extension should be added to lfs for sure in the .gitattributes
if they're in there the locking should work I think if it is supported
carmine walrus
git by nature doesn't really work with locking
since git is a decentralized system
each repository clone is the full repository
with the full history
dusk dove
lfs locks is unbelievably slow since someone had the glorious idea that it should make a new https connection and a new request for every file name you want to lock
golden hemlock
mmmh so maybe github does not support lfs
My other choice is move to svn, but I was looking for something cheap or free. But it seems that there are not many options there
Do u know if github allows lock?
for lfs?
dusk dove
it does
broken loom
I need a personal perforce server, I want to get a computer for it, what should the min specs be
Probably just find an I3 NUC?
Would it run on a super weak tablet PC running Windows 10?
(Not bothering with perforce min specs because UE4 is a lot more demanding with source control than most stuff)
dusk dove
it runs fine on a crappy $5 vps so it probably runs on a tablet too
haughty ember
@broken loom perforce runs on a Raspberry Pi 3
2/4 core 3.0+
Most important parts are SSD and ram
Not sure what the specs for tablet PC are though. Might be able to. I/o might be a bit low.
broken loom
Ah thanks guys
noble knoll
perforce on a raspi works, ive been using it for a couple years
just get a good SD card to hold the space, its easier to use an sd card than usb HD for raspi server
it will be kinda slow, but for source control you are likely bandwidth bottlenecked anyway
fringe abyss
i have been working on my laptop on a separate branch i made for my project (which was just a copy of the project of my development branch) , and now i am back at my desktop, i have pushed all of the changes to that laptop branch to github, and i want to get them all into my development branch so that i can delete the laptop branch, what would be the correct process to do that with github desktop?
which brings a 2nd question: if i create a pull request and discuss it with myself how much therapy do i need
rotund bobcat
just merge your laptop branch into your development branch
fringe abyss
okay i did that on github desktop
but i dont see any changes, and fetching the origin doesnt bring up anything new onto the dev branch
@rotund bobcat thanks, doing it on github desktop did nothign but going to github.com and actually doing it there by putting in a pull request and then merging got it done
but cant seem to pull origin now
just going to restart my computer and then delete the lock file, still ahve the backup on laptop anyways
rotund bobcat
try restarting your github client
that helps with sourcetree when it gives me that error
fringe abyss
yup that worked, also had to go into command line and do a git stash because the input.ini had changed between projects
boreal pilot
So, I'm trying to setup Perforce on GCP and for some reason I can't access my remote server from my client. I keep getting a timeout, even though my server can connect in p4 using a loopback address. Here is my environment and config, anyone know what's wrong?
I'm using SSL and the P4SSLDIR directory has the generated the certificate and private key.
molten marsh
Make sure the firewall isnt blocking the process and also make sure that the port has been forwarded.
boreal pilot
@molten marsh Any idea how to do that on GCP? this is my first time using their compute instances
molten marsh
No clue, dont know what GCP is... Typically if P4 cant be accessed externally its because of one of those issues.
boreal pilot
Also, I've been able to successfully ping and tracert the server
Google Cloud Platform
molten marsh
Id dare say that you would need to declare the port accessable.
boreal pilot
I've set a firewall rule to allow traffic in from all IPs via tcp ports 1666 and 443 (ssl)
molten marsh
Contact their support services?
Or google it?
I cant imagine your the first person to have this issue.
boreal pilot
I've looked. But I guess I'll keep looking. Thanks for the suggestions, unfortunately still no luck.
magic belfry
whats the best way to deal with .map modifications across perforce streams? is there a good way to avoid conflicts
any best practice suggestions?
should scenes be imported from mainline?
rotund bobcat
do file locking for your maps so multiple people don't work on them at the same time
magic belfry
@rotund bobcat but does that work across streams?
That's the concern, not when people are in same stream
magic belfry
I need a perforce expert 😭
dusk dove
they are hiding
magic belfry
locked in the server room
developers dont let the perforce admins out of their cage
no communication with the outside world
burnt shadow
help computer
modest willow
hey guys, if i have a local project sized at 60gb (contains paragon assets) and would like to use perforce, they give 5gb per seat for their Premium subscription.
any idea if i can increase the storage size? and would i really need a 60gb storage for the whole project or are there compressions or something other?
molten marsh
@modest willow Perforce have a free 5 user license by default.
You have no size limit as long as you dont have more than 5 users.
modest willow
@molten marsh oh really? https://www.perforce.com/products/helix-teamhub/pricing i see 1GB Total + 1-5 users. what do they mean by 1GB?
molten marsh
Ignore their pricing plans. They dont tell you that by default you get 5 free users and unlimited data.
You just have to find a way to host it thats all.
modest willow
oh that's awesome! so all i need is to rent a server
molten marsh
Pretty much
modest willow
and host p4 on it, and that's it?
molten marsh
Yes
molten marsh
👍
modest willow
any recommendations for a hosting service that is cheap and is good for perforce?
molten marsh
🤷 I host my own Perforce server in my garage.
Get an AWS EC2 instance or something
Search around.
cold orchid
hey guys how do I initialize git LFS on my current project I know how you can do it at the beginning but now I'm caught in the middle of a project and I need to enable it https://gyazo.com/05dc7cd47696b057815741d3bd72c733
quaint obsidian
cold orchid
😃 👍
I already gotted working yesterday this is all I did I added this line in the console : git lfs track "*.uasset"
river tinsel
@quaint obsidian is it any faster than previous version ?
becuase that's the only issue i have with LFS
quaint obsidian
@river tinsel i was just linking that because it has instructions on how to migrate files.
but yes, generally, LFS is pretty fast and just bottlenecked by your internet, with one exception being file locking.
visual ingot
Hey guys.
I was trying to add source control (git lfs2, via plug-in), but upon initialization the editor crashed.
Now it doesn't let me enter anything and the initialize button is not there anymore. And if I try to press apply settings button, it says I have to initialize the project with the git first.
Am I screwed up or is there anything I can do?
tranquil walrus
This is the first time I'm trying to use the UE4 in engine source control.. is it not a total solution? Is there no way to push to the remote host from inside UE4?
molten marsh
@tranquil walrus Top left on the Source Control button in the Editor allows you to checkin changes.
tranquil walrus
Yes, but that checks them in locally, it doesn't push them to the remote host
it seems I need a plugin for that
molten marsh
What do you mean? Of course it would push them, thats what a Checkin is.
What SC solution are you using?
quaint obsidian
@visual ingot i would just init your project externally
tranquil walrus
@molten marsh I'm using github. When I click "check in", it checks them in locally. But it doesn't push it to github. At least it didn't when I pushed check in. I loaded up sourcetree and saw the commits there, but they weren't pushed to github.
molten marsh
🤷 I dont use Github for source control in UE4 so im not sure why its not doing it properly...
Not a fan of Github.
tranquil walrus
What do you use? If you don't mind my asking
molten marsh
Perforce.
tranquil walrus
Do you host your own servers to back that up to or use a third party host for it?
molten marsh
Personally i have my own Server sitting in my garage. Professionally, usually hosted on an EC2 instance.
tranquil walrus
Okay, i haven't tried that. I'll look into it. I did find someone making a plugin for the github that adds push and other git commands to the UE4 interface, so I assume it's not set up to do it inherently now.
tranquil walrus
Yeah, they have LFS support, which I've enabled and I can diff blueprints on it.
quaint obsidian
@tranquil walrus what's your output log?
thick knot
Hi! Question for licensees: can you connect to Epic's P4? I was able last week without a problem. I can't today
thick knot
Ummm I just saw that Epic is on holidays and their support is very limited. Could that be it?
blissful apex
I can connect to it OK Bruno...
lunar ridge
@tranquil walrus I use Digital Ocean to host perforce - 5 dollars for the cheapest instance. Enough for Perforce, presumably
tranquil walrus
Thanks, I'll check that out if this ends up not working out for me. Right now it's more or less doing what I want. I could always install the plugin if i want to push directly from the engine.
lunar ridge
As a developer myself I am okay to use git, but I hate to explain it to less-technical guys, so Perforce is much easier in this sense
tranquil walrus
@lunar ridge Yeah, I am probably going to have some graphics people pulling from this to work on materials/lighting/etc..
that might not be a bad idea.
is there a good tutorial/crash course on perforce I can check out somewhere?
We have 3 developers, but for the most part the other two guys are only going to be doing blueprint stuff. They're coming from Unity and have no real C++ experience, so I'm going to handle most of the stuff on that end, and we're going to be covering 3 different areas..so I'm not really envisioning any kind of difficult merges or branches or anything like that.
molten marsh
Perforce have a 5 min setup video on Youtube somewhere.
tranquil walrus
So I guess you could describe preforce as kind of active source control, where something like git is more passive. Preforce seems to always be monitoring and checking if the file is checked out
tranquil walrus
How important is the drive letter in the perforce workspace? I'm using this on G: but they might be storing the project on other drive letters
lunar ridge
As I undertstand, perforce have a workspace and a depot. Depot is a remote storage. ANd the workspace is the depot mapped to local directory. You can map depot to anywhere on your computer. Other participants should not be care.
@molten marsh correct me if I am wrong
tranquil walrus
Ah okay, and it doesn't do anything weird to the files right? I can choose my existing project as my depot and it won't erase it or anything?
lunar ridge
What I've done - I have set up a perforce server.
Then locally I set up a workspace (a folder with existing projects). Then I just added all the files from the workspace to the depot.
tranquil walrus
okay well I've already got it backed up and committed with git I guess in case it blows it up
lunar ridge
Yes, back up everything before experimenting
tranquil walrus
OKay well it didn't delete anything. Now I'm adding files, this is something i messed up once with git. Should I drag my actual project folder into here (the root) or should I drag the files that are in the root onto the server?
lunar ridge
Cannot answer right now - away from the computer with P4, but as I remember you usually use Mark for add command to add new files
tranquil walrus
Okay thanks, I'll do some googling and see if I can find an example of adding an existing project
lunar ridge
np
tranquil walrus
ugh disk i/o seems abysmal on digital ocean. I'm uploading at 4.2 MB/s and getting .09 Mb/s disk write..
tranquil walrus
I officially hate perforce.
The digital ocean droplet I had was useless. I had to force the P4 client closed. Cancel just refuse to work for 20 minutes. So apparently that leaves the files locked. After moving my droplet to another location, I got back in and ran a command that is supposed to unlock the files in this case, and it seems to succeed, but I still can't do my initial upload
tranquil walrus
1.5 hours all that work of moving it from singapore to SF, and I'm still getting .23 Mb/s disk write..
ocean ridge
solo user? or do you have a team
if solo, Plastic is perforce-like and has cloud hosting for about the price of that DO droplet
numbers change a bit if you need to add other people though (but not so drastically as crossing that 5 user threshold on perforce 🤣 )
dusk dove
meanwhile, we're hosting perforce on linode and it is beautiful fast on even the crappiest server 
tranquil walrus
I wish someone told me that before I signed up for digitalocean.
dusk dove
they have hourly billing, so 🤷 just make a new server
tranquil walrus
you've never had an issue on linode?
dusk dove
can't remember any serious issues
tranquil walrus
The 2.5 GB initial upload took somewhere around 30-40 minutes on Digital Ocean that's just.. I can't go to my team and say that's what we're going to use.
dusk dove
yeah that's a bit slow
tranquil walrus
I don't suppose there is any chance I can take my image from digital and upload it to linode is there?
dusk dove
nop
oh yeah linode also comes with daily backup service
which seems to be lacking on digital ocean
you can set up a small script that runs daily before the backup is taken to do a db checkpoint
tranquil walrus
Yeah, looks like digitalocean has once/week backup for $1
Ah..linode doesn't take paypal?
dusk dove
they do once you signed up ( )
the $5 server works fine for us with low activity and ~12GB size
the $10 one is a bit faster on the downloads
but not really worth it
tranquil walrus
Hey, at least they have a server in Japan.
closer than singapore
with better routing I believe
We might need the storage. I don't know exactly what they have in mind for assets yet. Just the basic VR plugins has us at 2.5 gb.. I think they're planning on a lot of character models and skins, so it might be better to go the 10. I got the $10 signup bonus anyway
dusk dove
you can upgrade or downgrade it at any time
but you have to get out a calculator and figure out exactly what size each partition should be and resize it manually or it refuses to work
tranquil walrus
Well I'll just get the 10..it's only $10.. it's not going to break the bank or anything. 50Gb should more than cover us I expect.
dusk dove
the extra ram is certainly going to be useful aswell for caching more files
tranquil walrus
Yes, so an ubuntu image on linode should more or less be the same as the ubuntu image on digitalocean right? I can basically follow the same tutorial for installing perforce as that?
dusk dove
yeah, make sure you're not following the ancient allar guide and using the package based install instead
tranquil walrus
perforce really doesn't handle losing it's connection well.
dusk dove
tranquil walrus
I had the client open when I killed my droplet and now the client refuses to close because it can't connect.
dusk dove
ye p4v really doesn't like getting logged out either
you'll have to kill it with task manager
tranquil walrus
That's not very attractive..but once this is setup and connected to UE4, do we need to run the p4v client at all?
dusk dove
yes, to submit your source code and sync
never sync with the editor open it breaks everything
you can submit content from the editor though and lock files as you edit assets, that part works fine
tranquil walrus
Hmm... I'm not going to be in office for a few months until I move to that city. I'm working remotely until then. That's one reason I'm setting this up. I don't know if I want to introduce them to something that finicky when I'm not going to be there if it breaks. But locking the files is a really good feature.
dusk dove
it isn't finicky as long as everyone uses it correctly :D
tranquil walrus
Well yes..but I'll have developers and graphics people using it. I'm not too worried about the devs. I just hope the graphic people won't get jammed up
dusk dove
the most common problem is forgetting to manually set the ignore file path
perforce is so that unlike git which automatically uses a gitignore file, every client must type p4 set P4IGNORE=.p4ignore once after setting up the workspace or it doesn't work
tranquil walrus
can you do that from the visual client, or you have to install the command line for that?
dusk dove
you always have the command line tools
p4v works by executing p4 commands
but no, you need to run a command for that
or make a .bat file that does it
tranquil walrus
Ah perforce doesn't have a package for 19.04 ubuntu yet
which is the latest on linode.
dusk dove
get 18.04 lts
our is still on 16.04 since there was no 18.04 package when I set it up
but the new one got to work fine too
tranquil walrus
It's already done now, I finally got the file to find the installation
obviously.. that's why I'm in the new user creation window..
I just made it through the admin client.
dusk dove
users should always be made with the admin client
make sure you got the server properly secured
tranquil walrus
Yeah, when I did it on the digitalocean install, I was able to make my first user through there.
Uploading project files now, so this is the real test.
dusk dove
there's some command I saved as being important for the setup
p4 configure set security=3
p4 configure set run.users.authorize=1
p4 configure set dm.keys.hide=2
p4 configure set dm.user.noautocreate=2
p4 configure set dm.user.resetpassword=1
p4 configure set net.parallel.max=8
p4 configure set net.parallel.threads=8
p4 configure set net.parallel.submit.threads=8
been a while tho
tranquil walrus
Thanks, I'll use those.
I notice that linode doesn't give me real time graphs.
or it doesn't seem to.
dusk dove
graphs of what?
tranquil walrus
disk io, cpu usage, etc..
dusk dove
use netdata
tranquil walrus
Okay, on digital ocean I could basically refresh and the graphs would be updated to the current minute. This one looks like it's about 15 minutes out of date.
but it seems faster so far. up to 16% already
Looks like it's on track to take about 15 minutes for 2.5 GB.. not bad. Much better than digital ocean, but still not great.
dusk dove
what upload speed do you have tho?
tranquil walrus
me? 500Mb/s
tranquil walrus
that's the smaller package here in south korea 😉
dusk dove
it just maxes out my shitty 40mbit/s
tranquil walrus
They started rolling out 10000 Mb/s this year.
dusk dove
if you want submit to go real fast make sure you enable the parallel submit in p4v options and in p4d config
tranquil walrus
that's the commands you gave me above?
dusk dove
it's been so long that I don't remember if I messed with the config more after those lol
tranquil walrus
those are only run on the server. The client doesn't have to run those?
beyond setting the .p4ignore
dusk dove
no those are server settings
the client needs to enable parallel submit tho
in p4v options
tranquil walrus
I'll wait till the upload is done before running them.
dusk dove
and then it's so smart that it doesn't show you a progress bar anymore for the upload
looks like values are all we have changed from the defaults yeah
tranquil walrus
Nice, that'll be nice to set up here. So what's the flow here for people using this? We get unreal 4.22 installed on the computer, we install the p4v and connect to the server and do an initial sync of the files. Fire up Unreal and load the project, enable source control in unreal. At that point whenever they edit something they'll "lock" it so that others can't use it while they're using it. Once they're done they can check it in and it'll be available for others right?
dusk dove
basically, yeah
tranquil walrus
So.. if the blueprints are tracked in real time. Would a user who is only editing blueprints (like a graphics person only doing materials and post processing) need to sync anything else? or would all their work automatically just be added?
dusk dove
sync = download new versions of content from the server
if you don't sync, you're working with an old version of the project
tranquil walrus
sync basically is the same as pull
dusk dove
this would be catastrophic especially with blueprints
tranquil walrus
of course. So outside of source files, the upload of the blueprints is handled by the checkin function you mentioned before?
dusk dove
yeah
tranquil walrus
Okay, well that does sound easier than git, because it sounds like the graphics team can just make the materials and things they need, check them in and they don't really have to do anything else.
dusk dove
just make sure you sync first every time you're about to open the editor and start working
tranquil walrus
that should be simple enough.
dusk dove
when you add new files outside the content folder that is managed by the editor, like new art source files, you gotta use the reconcile offline work function to make p4 realize they exist
tranquil walrus
So things like textures and meshes or wav files?
dusk dove
yeah
tranquil walrus
okay
I have those server commands in and I've got it connected.. is it possible to make a new copy of this to make sure what I put on the server is all the right files to get this working? I'd like to download a copy like I'm a new user.
tranquil walrus
Ah, that's pretty easy. Well fingers crossed I did it all right. Thanks for all the help
tranquil walrus
So just doing a little testing of the check out/in system, with both copies open. What's the refresh time on this? Can that be modified? if I check out a file in one and then refresh in the other, it'll show up. but if I don't manually refresh the file, even after a couple minutes it's still not showing up.
and is it possible to set it up so that opening a blueprint to edit it would automatically check it out?
dusk dove
yes you can modify the refresh timer in p4v
idk about the editor integration
never bothered much with it other than check out stuff and submit
when you save the blueprint it will check it out
tranquil walrus
Right, but since it's not propagating the check out right away, i envision a scenario where someone could open a blueprint someone else has open, do a bunch of work then go to save and find they can't because the other person actually checked it out
dusk dove
well, such is life sometimes
usually people don't try to edit the same asset at exactly the same time and this problem won't happen
tranquil walrus
It's already a change for them to move to UE from Unity so I'm aiming for low frustration with this.
But I found the refresh time, that should help.
dusk dove
automatically checking something out when you open it would be nonsense
so it would have to do it when you start editing I guess
I made it a habit to save the thing I am working on every few seconds just in case the editor decides to randomly crash on me
tranquil walrus
I suppose, you could just be opening it to look at it.
I could train them just to try to save right away or something.
hmm.. the refresh timer was 5 minutes, I changed it to 1, but it's been 7 minutes since I checked the blueprint in and it's still showing as checked out in the other client.
dusk dove
in p4v or the editor?
tranquil walrus
p4v
dusk dove
strange
tranquil walrus
I told it to check for server updates every 1 minute
if I manually click refresh on the p4v client it still doesn't show up in editor. I have to manually click the blueprint and choose refresh on that asset.
dusk dove
well the timer in p4v doesn't have anything to do with the editor
I don't know if the editor refreshes at all
it might when you open a different folder in the content browser and go back
tranquil walrus
oh..well.. that somewhat defeats the purpose. I was hoping it would actively show what files were checked out.
Ahha
yes that did it
well sort of.. it seems like only if you change the folder after a certain timer is up. Not sure what that is.
dusk dove
btw you have 5 users or less right
tranquil walrus
why?
dusk dove
because otherwise you need a license
tranquil walrus
ah