#source-control
1 messages Β· Page 46 of 1
The server can't handle all the deltas it has to compite and you end up with those 'remote hung up' which is just a timeout
Even cleaning or using the BFG might result in these timeouts.
@cold lotus very true m8. Also very highly worth noting once more to the team. Thanks again for the incentive to do such.
Yeah, git is meant for source code not binaries, a shame UE does not have a text format for their assets, while that would not solve the issues with your resources like textures, meshes, etc
yea, tru that. thanks again for the reminder
if you have a project on SVN, if one person enables a built-in plug-in, does everyone else have to enable it as well?
there's no code in the project so the included folders are just content, config, and the uproject file, how do plugins get synced?
I'm really lost here, I'm new to subversion and source control, and I have it working correctly from my server, except for when I enable a built in plugin, it does not get enabled for everyone else, is there no way to SVN my marketplace plugins or built in plug ins to a project for everyone else to use?
@pearl marsh since you have the project under source control, you can test this: when you disable/enable the plugin yourself, are there any changes in the files that SVN manage? If there are not, then - what does that imply? If there are, look at those changes in detail (look at the diffs, line by line) - does it seem reasonable that those changes are all it takes to enable/disable the plugin?
@pearl marsh when you download plugin from marketplace, launcher puts its folder in your engine's Plugins/Marketplace folder - this is your personal installation of engine and your personal folder of plugin ;)
you need move plugin folder to the project's Plugins folder (located in the project's root with Config and Content folders) and submit this
- change in .uproject (enabling plugin, if it's not enabled by default)
Thank you guys for your replies, yes files get changed when enable/disable plugin, SVN says only the uproject file gets updated. yet, the built in plugins do not get enabled for anyone else. Specifically, we are attempting to get multi-user editing working. Also, there is no plugins folder in my root project folder, I have not used any marketplace plugins for this project yet, sorry, that was for future speculation, but for now, I am just trying to have built in plugins get pushed to other users, and only the uproject file seems to change when enable/disable, yet the plugin enable does not make it to any other users
@pearl marsh yes, you need to update project's folder from SVN, not editor - and then run the editor
think logically - .uproject is responsible for defining what should be loaded with the editor, in case of source builds you would have to compile newly enabled plugins before starting the editor
just get used to updating entire project's folder with closed editor
- "content hot reload" is still experimental
- often you need update project binaries, configs, plugins, .uproject
I see, thank you, I understand on syncing the project folder part now. But as for working with plugins, I know custom plugins will generate a plugins folder in the root, but for the built in plugins, the only change that occurs is something inside the .uproject file, which you say is responsible for what should be loaded with the editor, so if it is a built in plug in I do not understand why the .uproject gets updated successfully, yet nobody else gets the plugin enabled or any settings pushed. Does this mean that we would need to switch to a source build of the engine to make sure plugins are consistent across all users?
@pearl marsh no, probably somebody messed up something - didn't submitted or downloaded correctly
@kind olive I set up Perforce on AWS a few months ago and it was nowhere near as difficult as I thought it would be. I'm running the free version of the P4 server and using the free AWS tier. I've come close to the limits of AWS free tier (80% or so) but even if I had a busy week the costs for usage seem very reasonable
free tier expires though right?
Nope, you can use it indefintely as I understand it, it's the "personal license"
I mean, maybe I'm wrong on the legal aspect of that but there's no shut off switch either
hmm used to be 1 year free
I think it probably just blocks you from adding additional users? I think the max is maybe 6?
AWS?
Amazon Web Services
yeah, just wasnt sure if you meant perforce.. hmm i did not know free aws forever was a thing
confusing
Oh yes, both free. AWS has a free tier as long as the E2C instance you are using stays below a certain amount of data transfer and CPU usage
Yup! That first one is the one I'm using. Two people active and an ArchViz project so the data transfer has been the part that hits the limits
Here's the video I used to get up and running, worked a treat: https://youtu.be/7eVO0qXC3nk
This is just a quick video to go over the first time setup for getting a Perforce Helix server going on an Amazon Web Services EC2 instance.
Introduction and Downloads
Amazon Web Services - https://aws.amazon.com/
Perforce Helix Core (P4D) - https://www.perforce.com/download...
Oh dang, @neon tulip I think you're are correct in that it expires after 12 months
Apparently I don't need read the fine print very well. Car salesmen love me...
That said, still a year free, it'll be interesting to see what my AWS monthly bill is when that offer ends. I suspect still reasonable π€
tbh the ec2 pricing isn't bad if you are just running a tiny instance
even the bigger instances are not too expensive
def worth the money imo for all the features AWS offers
ive even used the gpu instances for rendering and NN style transfer and its def reasonably priced for the hardware you get
just dont accidentally leave an instance running all night π€¦ββοΈ
Does anyone know how to resolve this issue with git?
Is your git initialized in the right folder?
Digital ocean is a fair deal cheaper than ec2
expensive cloud computing
I was able to clear the error by manually committing the file outside of UE4
@neon tulip Have you used Digital Ocean? I had been looking into them when I first realized I was going to need some sort of source control. I'm also a web developer so my first choice had been git but obviously that tech wasn't built for the amount of binary files a typical game project has
Yes. I use svn
DO is way way cheaper than beanstalk app for the same storage
Is around 45 dollars a month for 350gb. SVN compresses things on the server so it goes way further than that
Quick question about git lfs. I just started using it recently and my first few projects it seemed seamless. However, my new project is creating an lfs folder that it wants to commit. When I go to stage the files Sourcetree warns me that they're over 2 megs and I should add them to git lfs. Should I ignore the lfs folder and the hooks folder? My other projects don't have them. Thanks!
don't commit the lfs folder
the lfs folder and hooks should be in your .git folder right?
isn't that one ignored by default
they're in the same folder as my .git folder instead of inside it
yeaaa, not sure how/why that happened
seems like your lfs was wrongly initialized
did you just run git lfs init in your repo?
or how did you do it
used sourcetree to do it, same as the other ones (which seem to be working fine)
I would somehow try to reinit lfs properly if I were you
dno how to go about it though
otherwise, for sure add the lfs folder to your gitignore
because adding the lfs folder to lfs would just create a weird infinite loop of creating assets to point to assets or something
yea
huh, started a new repo and it worked fine for a while and now the new repo has the lfs and hooks folder outside the .git again (actually it has a copy in both). The only thing I'm doing different in this project is that I'm doing a c++ based project instead of a blueprints one, so I'm using visual studio 2017 (the course I'm following is on 4.20). I found this https://stackoverflow.com/questions/50546186/are-the-git-lfs-and-hooks-folders-supposed-to-be-outside-of-the-git-folder on the topic, so I'm wondering if visual studio is creating those folders. I do see it has git in the lower right hand corner. Not sure how to go about fixing it though. π¦
disabled git in visual studio, hopefully that fixes the issue
@quaint obsidian new computer so should be latest, just downloaded/installed 2.9.2 just in case
ah, the fix will be in 2.9.3
download the artifacts here if you want
At the core of any successful project lies planning, tools, and execution. To reduce risk and improve collaboration between developers, agile teams often implement Version Control systems to record changes to files over time, allowing them to easily recall specific versions a...
needs more git
nope, git is terrible for assets and non-programmers, as highlighted π
depends on your workflow and teamsize really
we have had no problems with artists using git
it doesnt work out of the box in ue4 though :)
(just like perforce, really)
it just requires its own set up that hasnt been documented much, but im preparing a blog about that :D
well, entire AAA project with hundreds of thousands of asset commits and p4 is still super fast with it
also easy reverting, shelving
yeah like I said depends on workflow and teamsize how bad it can be
if you're working with over like 20 people on a project, I'd definitely do p4
obviously, I have nothing against git, but since p4 is free for small teams, I really never needed git π
I do it even with 2 people π
we have over 80 people working on our project with git
but yeah, it depends what you used first
well yeah, but if you're over 5 people it's not free
also known as crazy expensive
i started with AAA, so p4 is what I've started with
not tru π
it's limited by number of workspaces, never clarified on the website though
or many number of users, really I'm not sure π
but studios usually create users like Design, Programming, QA and individual workspaces for developers (which is normal workflow)
yeah, so we can use it for 20 developers for free
minus build system or sth
but I know bigger teams which don't pay yet... so there's something shady with their numbers π
however, their licensing is plainly stupid
they should go like normal subscription plan
well we did a quote for our team and it would cost us about x7 compared to git hosting
for how many people?
at the time, you could still use p4 for free π
still I'd gladly pay for full license if given a choice - it's still super efficient with big data sets
whereas git LFS is far away from that
it's funny thing that theirs unclear wording on website keeps teams from trying it π
instead of saying "get it for free for 20 people" and then you'd almost forced to buy when team grows
the perforce GM said this though βWhat we heard at GDC is that indie game developers love Helix Core, which is currently available for free to teams of five or fewer, but they wanted similar access to our Hansoft project management product,β
so I'm guessing explicitly 5 users in the team
what the fuck...
man, I'm using it for decade in many companies
it was never restricted to 5 people
a single dev = workspace, not user
p4 user is unit for setting up access/privileges
i.e. devs sometimes have different workspace for office machine and remote PC at home
so you're saying you could have 20 people working under 1 user?
it sounds they p4 guys think they convince people to pay for licences at the start - they probably don't care if you'd use free version for years, no bonus for salesman
yep
I guess, you need at least separate user for admin account, but generally yes
and it's maybe second time I hear that indie studio would like use their Hansoft - I know only one producer in Poland that likes it π
well, that explains a lot, they don't want people using free setup π
so if you have on-site server in studio, try it, basic setup is super-simple and works like a charm
client UI is quite minimalistic, so simple
VS plugin handles auto checking out files you start editing
that's really annoying
I don't wanna check out code files
merging them is super easy
ah, but terminology will be confusing for you initially "checkout" means marking file being edited π
you won't notice checking out
plugin does that for you, when you start typing in the source file
yeah but doesn't that lock the file for other developers
so what's the point of checking them out
sometimes is useful to know that somebody works on this
it prevented me sometime from deleting file π
or messing heavily with the class layout or doing cleanup while somebody else is working on it
but technically p4 sets all submitted files to read-only, so either you need check out while starting work on it or set file to writeable (editor has option for that for assets)
ok, but let me stop promoting p4 π
actually I wonder if some of sweet p4 options (there are here since beginning of time) are possible in some Git UI clients?
in p4 it's possible to sync any folder or asset without syncing entire repo
which is awesome when you're tracking where bug appeared and which file (and file version) caused an issue
I can also review history of given asset/folder easily, like with entire repo
yes you can do that
you can cherry pick commits or files from specific commits
and log history
not with folders though I don't think
never had the need to though
and can I see history of given file first? all revision of it?
yes, you can see all the historical changes
and then pick specified revision?
yes
which client then? π
sourcetree
oh, so it's a bit better nowadays or I used it wrong sometime ago π
good to know
I do everything sourcetree + commandline
Everyone likes a different program. Choose one, test it and if you like it, stick with it.
We have everything UE4-wise with P4. And all our clients too.
And all other kind of pure code outside of UE4 hobby projects are somewhere on git.
I do have to say, setting these programs up sucks for all of them
this is example of the history thing in sourcetree
another question about git clients - is it possible to have multiple working changelists? without sending anything, even to local repo?
You mean branches?
like branches?
no, like few lists of assets I work on
Perforce can do that too iirc. Problem is, if you use UE4 with BPs (or in general Uassets), you can't merge these back in.
Hm
I don't follow
I don't git what you mean
badum tss
do you mean like having the changelist of everything that has been updated since last commit, but being able to split it up?
what would be the point ofthat
You can have multiple changelists if that's what you mean
Pretty sure that's a key feature of source control
ok, I prepared concept art
π
@rotund bobcat it allows to split work easily
even distribute on code change between few commits, so history is easier to understand
well you can just stage whatever you want per commit?
Yeah pretty sure git can do that
yeah, I'm not sure that's why asking π
you know, in p4 is something visible by default in the client π
you don't have to commit all your open changes at once if that's what you're asking
maybe it's also easy in git, but hidden
I know π
but can I have a few list of local changes visible in UI? without using commandline or other tricks?
still asking about UI clients, of course π
if I'd stage changes - how I would check what files are there and i.e. diff the files?
you can have staged and unstaged, dno if you can split further, cause I've never had the ned for it
ok π
like this deciding what goes into the commit
but you want to split the unstaged files into more lists?
exactly
could you give me a use case where that would be useful
cause tbh I don't see the point of that atm
it's my screenshot above π
Well I worked on a lot of shit the last 3 days, but you don't want one huge commit.
So I cut it into smaller lists and commit then one by one (making sure they don't rely on each other int he wrong order).
lfs sounds like a nightmare 
I also have some stuff staged for some later commit
one list for new system I'm working it
if somebody ask me for a bugfix something else, I put my "new system" to separate changelist and send bugfix code only
well, you could either make a separate commit with the bug fix
or make a branch to fix the bug
also often separate list for crap/test assets, so I would send this accidentally
or I can split a system into few lists while I'm working for days on it
and if you don't want to send stuff just put it in an ignored folder
evey list would describe it precisely what are changes
you can split your work in seperate commits as much as you want
you could have a commit per file
I can also write detailed description for commit long before clicking "send"
yeah, you can commit locally
and then push everything at once when you're done with what you're doing
sorry for a noob question then, but I can push only 3 out of 5 staged lists? not all of them?
I probably gonna hear "yes, use commandline" π
you can from the command line
π
not sure in UI
but if you really wanna stick to UI you could make a branch of of the one you want to merge to
cherry pick the 3 commits
and then push that one
good look telling that music composer or artist π
well you can add custom actions to sourcetree
so you could set it up for them
also why would you want to do that
if the 3 commits were finishing something, why not push it then
why wait till 2 commits later that you don't want to push for some reason
ok, so old differences between these systems are still alive
in p4 everything works for user without command line or specific setup (no LFS, file locking as extras) - also history view for every folder, not only file - but that's issue of lacking UI implementation or something it's difficult to query in Git (history of folder?)
and branches in p4 are hard to work since branches aren't exactly needed for everyday work
also, p4 is easier in regards of revision numbering - just plain numbers starting from 1, easier to track or tell somebody what revision they should get/check π
anyway...
if anybody would try p4, Epic's livestream explains almost everything well
just missing info about typemap, which is important to setup, to be sure UE4 binary assets would use exclusive checkout, one of biggest advantages of p4
How to setup Perforce so that you can share assets with other on your team.
and it's good they're talking about version control, it's now easy to link to their video if somebody on forums would complain "oh no, I just lost my project, cannot open assets, I hate Unreal" π
hey so I just set up a git repo for my new project, UE can see it, vs2019 can see it, but I can't figure out how to add a file to the repo, like the equvialent of git add -- I have a .md file in a folder called Docs that lives in the project root next Config, Content, and Source -- the Docs folder didn't get added by the automatic UE git process, and I'd like to add it. I can do it on the command line, but I want to know how to do it in the UI
i believe that is the case. I think UGS streamlines the p4 sync process somewhat
An overview of UnrealGameSync (UGS), an internal tool used by developers to sync their Workspace with a project's stream.
Learn how to get started using UnrealGameSync.
This page describes how to configure UGS for deployment as a studio.
to my understanding, UGS doesn't streamline basic actions
it's more like:
- allow non-programmers to download updated assets only if matching build is already available (if you're use build machine and programmers send assets together with code - for few minutes you have repo with assets updated to build that didn't came yet)
- sync compressed engine builds easier
- and easier communication, like they don't need to use internal chat to say "hey, I'm working on build fix" which probably happens very often while working on the engine
I'm as curious as @pearl marsh, if somebody outside of Epic used it?
At the core of any successful project lies planning, tools, and execution. To reduce risk and improve collaboration between developers, agile teams often implement Version Control systems to record changes to files over time, allowing them to easily recall specific versions a...
<@&213101288538374145> I reckon this is a good pin for this channel
Me and a friend are running source control there's a .vs folder in the project root which is half a gig - is this important to share or can it be generated on the remote device without sharing?
@maiden turtle never share it, it gonna generated on another machine that would need this π
generally, what you need to store in repository is the source code, configs (.ini, .uproject)
and binaries for non-programmers
Intermediate, Saved or other folders - safely put them on ignore list
@fallen geyser yeah, as we went down the list in the project - it seemed 90% of the folders needed sharing so we just blasted over the whole thing, I'll look into adding ignores, you'd suggest that for intermediate too? Hadn't realised that wasn't necessary! cheers
others programmers need source code to compile (raw material) and non-programmers needs an ready-to-use product (binaries)
.vs and intermediate - that's things stored on the local drive, so the work inside IDE and compilation would be faster, but that's need to be generated for everyone separately
Green required, yellow optional, required only if team is building the engine from source
the non highlighted gets generated
https://www.visualsvn.com/server/ you install this to make the server @loud tide
then https://osdn.net/projects/tortoisesvn/storage/1.13.1/Application/TortoiseSVN-1.13.1.28686-x64-svn-1.13.0.msi/ tortoise svn for the client (which can be on the same computer).
thanks @pearl marsh & @fallen geyser
who knows how to properly setup .p4ignore for p4 plugin in Jenkins?
my reconcile works fine if I run it from p4 client, but reconcile run from Jenkins plugin doesn't seem to respect .p4ignore at all...
have you run the silly p4 set P4IGNORE=.p4ignore command there?
it won't work without running this on every computer with a workspace
@dusk dove I did
it's just Jenkins plugin using P4Java, totally separate implementation than native C++ one, and there are some differences...
for now I just send everything from Binaries folders (workspace view narrowed for these folders only)
I checked that .target and .version files are also included in the launcher distribution of the engine, it doesn't harm to send it then...
I forked the UE engine repo and cloned it, and now I have 2 remotes on my local repo. I used git tag and notice the 4.24.1 isn't there, so I did this:
git fetch src refs/tags/4.24.1-release (src is the remote for the original UE4 engine source).
It took some time and in the end showed * tag 4.24.1-release -> FETCH_HEAD. From some reason though doing git tag still doesn't show it (and I can't checkout it either). What am I missing here?
hey, everyone. I can't quite comprehend what's the subversion con Arran is talking about at 6:15 in video @fiery prism posted. Could anyone explain why I'd have to separate my project somehow? The "game build" and "engine build". I never used ue4 with version control. All i did was git for coding simple projects.
Here's the link to what I'm talking about:
https://youtu.be/JxXydvG4mlI?t=375
At the core of any successful project lies planning, tools, and execution. To reduce risk and improve collaboration between developers, agile teams often implement Version Control systems to record changes to files over time, allowing them to easily recall specific versions a...
I kinda do have to. I'm about to start a group project
If you're not tweaking the engine then there is no need
ah, you meant doing THAT
alright then
thanks for clearing that out. I don't intend to tinker with engine files
Any case: all the files needed to debug/build the game don't need to go to non programmers. I haven't watched the video though
I just started to watch it. Figured it would be a good place to start. I'm gonna work with 4 other people. We probably won't be doing any c++ coding. Blueprints should be more than enough. I'm just trying to decide which version control tool would be most convinient for us
is it possible to compile the shaders in a project and send it through perforce for everyone else?
@pearl marsh you probably want to set up shared DDC, if you're cooperating on LAN
Overview of the Derived Data Cache and the storage of assets in formats used by UE4 and its target platforms.
there are some methods for compressing DDC and including with plugins... but that's not feasible for actual projects where your materials change all the time
Hey anyone else got troubles in using the free version of perforce? They tell you perforce is free to use for teams up to 5 people but after one month, the server simply expired on me while using it with only one user and I couldn't get it up again. From looking online I was told that this happens when your license expires.
@fallen geyser seems to have more experience with free perforce
did you make more than 5 users or 20 workspaces?
Not at all, I simply had 1 workspace with 1 user
I used it perfectly fine for about one month
and then the background perforce service would simply stop running
even if you start it up again?
Thats the problem, whenever I tried to manually start it I get an error 1067
maybe your ip changed?
apparently your license needs to be updated then
because by default at least local ip address changes on windows every so often
dno if this is the cause though, I've never used perforce
don't you have a log file somwhere that would say what was causing it to stop
still server should run if you'd manually start service on server... no matter of local IP address, p4 doesn't care
it happened in past that Perforce changed licensing rules of "free tier" with the next server version, but it wouldn't change on already installed server
@kind olive try asking on Perforce forums or simply nuking old server and reinstalling (it's possible to move data - it's basically folder for asset versions and .db files)
What?
I mean, maybe your p4 server installation got corrupted some way, maybe some of its data
I have no idea what you're talking about.
ok π
Is it an old question? I don't recall ever having issues figuring anything out with Perforce other than finding a cheap host.
ah,
Thanks! Ill check it out π
oh π
how much space would one need once source control setup, running,,,ex when it saves maps,typically mine are 122,220, or does itsomehow just save 'changes',
I SO need a huge ssd,,who doesn't ;))
https://www.youtube.com/watch?v=JxXydvG4mlI , should suffice
At the core of any successful project lies planning, tools, and execution. To reduce risk and improve collaboration between developers, agile teams often implement Version Control systems to record changes to files over time, allowing them to easily recall specific versions a...
You can use HDD for repository, this is still common as bigger studios generate lot of data and you wouldn't notice a performance impact
And p4 is probably the most efficient at storing asset versions, every file is compressed to gzip basically
Even big AAA title could end up using only few terabytes for the project repository
Or you can get cheap ADATA M. 2 SSD (not as fast as Samsung ones), but still faster than HDD and silent ;)
hi sorry got busy as usual...reading over
HM perforce seems a little pricey but Ill thinkabout it...plastic scm seems a tad more affordable,,butis itas good I dont know...damn keyboard membrane drivesme nuts...waiting on logitech tosend replacement so sorry for spacebar issues
any idea if p4 truly is the best way fwd considering price
ya m.2 my wish going fwd
my pny ssd 256 is too slow
and ya adata is very reasonable
ty for reminder
well I have no hdd reallly that has enough room so might as well spend a few bucks and geet something that makes sense for future proofing etal
ya evn 2 tb is getting within reach of most of us,,TY for letting me know about aaa,,thats reassuring
I have a question. If I have 4.22.3 on my home computer and commit it to my repo, open it on a school computer which is 4.24, make changes, come home, then pull the changes back onto my 4.22.3 version at home, now certain uassets aren't showing up? In explorer there's my TownGameMode.uasset but it's nowhere to be found in the editor...
Any help would be appreciated
if you're using git
make sure to do a git lfs pull
also, you should use the same editor version on all your pcs for the same project
Lost a day of progress doing an engine integration because of perforce weirdness, might as well warn people against it:
p4 reconcile tries to compute if an added file is 90% the same as a deleted file, and if so treats it as a move instead of a delete/add. This was falsely picking up changes in my engine update and creating bogus integrations, and is also part of why reconcile is so ridiculously slow
https://community.perforce.com/s/article/15277 describes how to turn the feature off completely. It's a stupid feature.
@wet comet
wow, that's interesting
I didn't have such issues, although reconcile was obviously slow and I had to it by subfolders π
thanks for posting that, I'll try that
are you able to say how much faster reconcile is now?
I haven't tried it yet with the option off, just read the doc
But if it has been scanning every single added and deleted file, that must be making it fairly slow
in my case scanning entire Source folder was like 15 minutes, annoying but not an issue
the problem was when I was trying to commit this, p4 seemed to stuck forever and couldn't save changelist at once - p4 was more like commiting suicide π
so I had to click reconcile separately for every Source folder and then it worked
so I wonder if your trick could with that
but I'm gonna find out at 4.25 π
Does anyone have issues with UE4.24 Crashing while checking in files?
I dont even get a crash screen. It just closes
@wet comet I turned off this thing in p4 you mentioned and, wow, reconcile is as fast as it should be now, thanks!
How can I make a Github repository with an existing project? (Without having to make the repo before the project)
you create the repo on github, and then it has instructions for using existing repo
you may have to run git init
Is there a way in Perforce to have a file be synced but still writable? I want to have save game files be able to be synced, but not write blocked so they can just be overwritten locally if so desired.
How do I use the Multi-User Browser with someone on a different network?
@rich steppe I don't know why anybody would send save game files to repository... but there are 2 ways:
-right-click on every file -> "change filetype" option and select "writable" from the list, it will add
"+w" parameter to the filetype
or you make it writable for every file of this extension, if you'd edit server's typemap
How to setup Perforce so that you can share assets with other on your team.
changing filetype from typemap won't change filetype of files already submitted to repo, you'd have do it manually
@fallen geyser Thanks, always writable works.
I need help, I discarded stashed changes and it discarded changes to files locally too, is there any way to recover?
I most likely found the lost stash
How do I return it
Although they seem to be in two "commits"
Please someone
cherry pick them
There is no built-in source control with Unreal right? Like I HAVE to use Preforce or SVN or Git right? I don't want my team to use any sort of coding outside of blueprints I'm not sure what the best way to go about is for that
There is no built in source control like there is for Unity, you should really try and get the basic perforce working, you can use the free edition, it only comes with 5 seats but multiple users can have their own workspace per account. EG we have Programmer1(workspace 1, 2, 3, ect), Artist1 (workspace 1, 2, 3, ect), Guest1((workspace 1, 2, 3, ect), SVN is very slow, and you will have to find some sort of cloud hosting, GIT is not good for when you are super deep into projects, its great for code, but not for engine binary files and such, I would really recommend the free tier of perforce with the basic AWS cloud hosting, it's decently cheap per month and its blazing fast, once you get everyone set up with git and SVN you will really hate how slow it is to do little things in the editor. Plus AWS lets you try their services free for 12 months
@fallen geyser Thank you for that screenshot, I would never even think that p4v has a damn dark theme, you changed my life
My biggest concern is making sure blueprints stay applied to whatever file they're applied. I am using SVN for another project (I myself didn't set it up) and let's say if someone else made a blueprint that makes the blue box go up, once I "download" the new edition, I have to apply the blue material to the box and I have to apply the blueprint to the box. Does Preforce make it so that if Person A applied a blueprint to something, Persons B-E will receive it as is? @pearl marsh
@devout quail it should work with every version control... that's the entire point....
Someone didn't set up your SVN properly
@pearl marsh haha, you're welcome, on of the last "black tools", they made it black just few months ago π
Yeah I guess it's not set up correctly or I might not have set it up correctly on my end. Anything I upload uploads but like the materials and blueprints do not stay applied at all (vice-versa with downloads too). I just want to make sure my team in my current project has access to all the changes
sounds like you reverting them instead of submitting, dunno, used SVN a decade ago
are sure that svn ignore rules aren't set to ignore folders with problematic assets?
Honestly have no clue. I just followed the steps I was asked to take but they seem kinda fishy. In order for me to "submit" assets, I have to place the assets in a separate Windows folder that is linked to server. So it's literally just copy asset, place asset, upload asset, refresh and make folder up-to-date, repeat process
Has anyone worked with the shiny new RoboMerge and/or BuildAgent yet? I'm looking into them but docs are scarce and for whatever reason they aren't where they should be on my version of 4.24, which is a problem in of itself
hello guys I have friend who can't setup the source tree-we have the project but when he log with the info of his acc and paste the link of the gitlab project it writes invalid url but it is now-how to fix this problem?
Hi! Has anyone worked with Microsoft Azure DevOps repos with UE4? We're currently using P4 and have a 500gb+ repository. We're looking for an alternative. Azure DevOps free unlimited space private repo seems like a nice option for us but I don't know how good Git LFS currently is. Thanks!
@wintry citrus Git LFS is not as efficient as p4 with huge binary repositories, IMHO "free unlimited space" isn't good reason to leave p4
are you sure you want to focus on free repo instead of paying a bit for i.e. Digital Ocean + P4 or setting up machine for p4 in the same physical location as your build/cook machine? π
By the way, we don't have any problem with P4 itself, it's just that the cost of maintaining the P4 server is getting too high
@fallen geyser currently we're hosting our P4 server in a dedicated server and getting the same amount of storage on digital ocean is even more expensive
Setting up the P4 server locally sadly is not an option for us. We work remotely and our third world upload bandwidth doesn't allow us doing that :/
I see, you got a bit conundrum then... ;)
(can't help about Azure DevOps, so I hide now)
Yeah, it's a tricky situation haha
you can always nuke the current repo history and reset the gigabyte counter, annoying at the beginning but you probably don't need entire history if you don't run live game π π π
@wintry citrus I would say Git LFS is a good solution but you need some additional user facing layer on top of it
the cost of the p4 server is the problem.. and not the cost of p4 itself?
@dusk dove p4 has a free license for small teams
git is working on binary stuff, but until it's ready git lfs is as best as it gets :/
unless you are windows only maybe, and can use the special microsoft git version
hello guys I have friend who can't setup the source tree-we have the project but when he log with the info of his acc and paste the link of the gitlab project it writes invalid url but it is now-how to fix this problem?
have you tried the solution to this https://community.atlassian.com/t5/Bitbucket-questions/quot-This-is-not-a-valid-source-path-URL-quot-when-connecting/qaq-p/165949
he might have installed wrongly
@devout quail to make source control work well, you should not need to copy files around manually before or after a submit or update operation. The itnention of Subversion/P4 etc is that you should have one work folder with your Unreal project; you use the UE editor to modify the files within that folder; then you use your Subversion/P4/etc client to synchronize your local changes against the changes on the server. The Subversion/P4/etc client should be able to tell you which files you have made local changes to (and which therefore need to be committed), which files others have committed changes to (and which therefore need to be fetched), and whether both of you are attempting to commit to the same file without having each others' changes (in which case the system will complain, and tell y'all to go talk to each other).
@devout quail having a separate folder for uploading/downloading makes many of the automatic features not work. You should really look into this, your lives will be better if you use the source control tools as intended.
Okay yeah so the shared drop doesn't have a uproject file it seems which is why everything isn't working the way it should. I'm gonna look into this because what you said makes a lot more sense than what is happening. I'm also gonna talk to the project in case I am the one messing things up @valid olive
Anyone knows how to solve these?
Severity Code Description Project File Line Suppression State
Error MSB3075 The command "....\Build\BatchFiles\Build.bat -Target="UE4Editor Win64 Development" -Target="ShaderCompileWorker Win64 Development -Quiet" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command. UE4 C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets 44
Folks familiar with SVN: Any idea why a folder would be marked modified (red exclamation mark) even if everything inside the folder has the green checkmark?
So, having an issue with Git and UE4. Whenever I select the git.exe file path, it only gives me this with no other options
When it should look like this
it's also not implementing a .git file in my project files, therefor I assume it's not actually installed correctly
Anyone has idea, why can't open FL_CPP_Info.h? The file is there. I try to convert project for 4.24
creating their own?
nuget
4 tries???
i mean, maybe
just maybe git is not the answer
looks like they also dont have proper source control on binary files with all of that....
and they didnt even try to use SVN or mercurial
btw i can 100% confirm this has happened to me But we faced a bigger issue than technical ease-of-use: repo size. GitLab.com was quite unstable at the time we were using it, and with larger repository sizes, it could cut out regularly, especially for new people cloning the entire repo
i had a high end self-rolled gitlab server, and after 1 week of ue4 LFS usage the thing just choked
@noble knoll sorry for being a dumbass :(
man, did you try other systems? git is known for being terrible on LFS, but SVN and mercurial and others exist
this just reads like repeatedly hitting a wall and just forcing through
it works fine now
what do you mean by source control on binaires?
can you do common source control operations on blueprints and levels?
yes
as in looking at log, diffing stuff, and being able to sync an older version for checking
i mean, if you think about it, git hasn't been properly built up for game dev, someone had to pioneer it
yes, through the ue4 git plugin
it's in LFS
all the blueprints and levels
wait, isnt that whole nuget stuff going outside of git?
oh, so its actually LFS for blueprints/levels, and those extras for stuff like binaries and maybe source assets?
then its not nearly as much as a trainwreck as i was thinking
yeah idk
were you going into this thinking it was supposed to be bad
or was it genuinely confusing what i meant by binaries
i was fairly confused by this Now, we sync project binaries through Nuget, rather than in source control.
I will put editor DLL binaries, good?
but it's not just editor
it's also project plugin binaries, though we dont have those atm, they're in the custom engine
i guess thats technically editor
that changes all, in that case its actually a pretty nice thing
i was thinking that it meant that you rolled your own hacky system for unreal engine binary files, like maps and blueprints, and was like WTF
yeah haha
i was confused why you hated it so much
and then i was ohhhh
i call it binaries, because it is in the Binaries folder
maybe specify that the binary project files are still LFS, but that executables/dll/etc are downloaded through nuget
how do you deal with tools choking on LFS repos @quaint obsidian ? I cant find the repo on github with the actual Content folder
The tweak makes it much more understandable
we haven't gotten any choking issues since switching to GitHub
and using watchman, git config large repo options, etc
oh, very interesting. I guess tools are now better now
my real question is... how do you pay the github?
i have my own unreal engine LFS repo with an open source project, and it ran out of traffic almost inmediately
(public)
I don't think I'm allowed to answer that, sorry
no prob
my real problem with the free one is that its 1 gb storage, and 1 gb bandwidth
also we don't have the Content folder available, it's private
oh i see
I thought github repos were hard capped at 10GB? is this not the case with LFS?
LFS has a different thing
LFS data packs
default LFS is 1GB bandwidth/storage
GitLab repos are hard capped at 10GB
5 dollas LFS is 50GB bandwidth/storage
50gb storage is nice. Even on the extremelly bloat-y LFS. But that bandwidth...
assuming a team of 5 people and a project of 5 gb, thats just 2 full downloads per worker a month
but yeah, basically, we made unrealgamesync for git
and optimized git for large repos with thousands of files
so it doesnt have to loop through so many files
ill go try those for myself, but try it on my own hardware
cant pay the extreme rates of github
wow this pricing is stupid
its literally better to buy perforce + amazon server with how that pricing is
I guess the price of perforce would get you 400GB bandwidth per month on github?
for each user
perforce is free up to 5 users
amazon server, high end, maybe 20 euros a month
for a repo
20-30
that would be 200 gb bandwidth on github
once you have 6 users you immediately have to buy 6 licenses
and 200 gb bandwidth is literally nothing
lmao its cheaper than github
thats the cloud-hosted one
helix cloud, hosted perforce
perforce is like 40/user/month
for the self hosted version
helix cloud also hosts git stuff make sure you're not mixing them up?
perforce can serve git repos
it answers and "emulates" a git repo
i think thats how unreal engine syncs their perforce with github
dammit they dont have an actual price on the website
its an "ask us"
if this is helix teamhub it is not perforce
only assembla hosts actual perforce
it makes no sense
the rumored pricing for perforce licenses in here is 40/user/month with a 50% discount for the first 2 years for new companies
so cost of a perforce server is 40 per user + another 30-40 for a hosted cloud server
yeah
team of 10 would be 450 per month (rounding for simplicity)
that would get you 4500 gb of bandwidth in github
pretty sure a team of 10 like that wouldnt crunch those 4 tbs of bandwidth in a week
we have about 100 people
wew
now im really interested about htf you pay that lfs
as costs are likely astronomical
only about 80 like devs
@noble knoll @dusk dove no, p4 is free up to 20 workspaces, one person in p4 = workspace
"p4 user" is a group of workspaces - commonly used to set different access levels, i.e. Programming, Art, Design
it's confusing, I know, but Perforce don't clarify that hoping companies would just simple jump onto paid licenses
one person can use multiple workspaces
build machine would use another workspace
one user is one account tho
no
with its own login
i mean i use 1 user on 3 workspaces on PUBG atm
an user has his own email login and password
workspace is a different thing
yes, but it's normal to have user Programming used by many people, the same login
why?
so then as author in every commit you just get "Programming"
as user, but you can display workspace
yeah but thats kinda hacky
that does make sense for less traffic stuff, like artists. Sharing account in there makes more sense
every operation goes on p4 goes via workspace π
that would allow to stretch the 5 users
yeah, "user" is practically a "group of people" everywhere I worked with p4 for the last decade π
from indie to AAA
well im at pubg, i doubt we care much about the money XD
I see π π π
so 1 account each is alright
on indie teams ive often seen stuff like 3 programmer accounts for 3 programmers, 1 designer account for one designer, and 1 art account for a couple artists (shared)
programmers don't separate accounts, so I don't get this too π
but obviously when someone starts small team, they create separate users π
anyway... Perforce could finally rethink they licensing, they could have a lot of money from small gamedev - instead having them used free P4 or Git
I wonder if CDP Red is even using "1 person = 1 user" nowadays π
ive thought some times if it could be possible to hack perforce as a git LFS backend
as perforce keeps deltas and its miles faster than usual lfs
it could be an interesting trick where you have a git LFS repo for your project, but its mirrored to a perforce repo on every commit
like the reverse of what unreal engine does, where they mirror perforce to git
haha, wonder if possible, entire history of single asset is just gzip
but they need to keep P4 journal to keep responsiveness with huge repos
but difficult to imagine what would happen on P4 if you'd call Git rebase
or deleting some commit - in p4 it would create another changelist and file version? (adding version instead of deleting) - or obliterate operation?
maybe you can make those work by literally deleting the files and then adding them again
I admit that sounds like voodoo magic territory to me π
sharing p4 users is very cursed those people would need to share a password then
the thing is the admin set a password for you, and nobody needs "privacy" here
actually it's common to have a single password for entire studio (except maybe p4 admin)
and it's good since you can easily submit/revert/unlock some locked files from developer being on vacation
so it's not your "personal password", just a password keeping anybody on the network from accessing the repository π
i keep the admin-generated password for my pubg account stuffs. My logic is "at any point, if someone wants to access, its possible to reset it anyway, so what even is the point"
given that the IT admin can inmediately reset password for everything, and then you can access all
anyone use multiple p4 proxies here?
I tried accessing perforce from my laptop again
I tried clone, get latest and they both only download the oldest version of assets
It downloads all of the assets, but the earliest version of them
It does not update the asset data when I submit, it just makes sure the asset EXISTS on the server
major outage on Bitbucket right now FYI. Pushing not working on the repos: https://bitbucket.status.atlassian.com/
Welcome to Atlassian Bitbucket's home for real-time and historical data on system performance.
I'm reading this book Pro Git and in the Keeping up with Upstream section it says:
https://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project#_keeping_up_with_upstream
If you see something like Pull Request does not merge cleanly, youβll want to fix your branch so that it turns green and the maintainer doesnβt have to do extra work.
You have two main options in order to do this. You can either rebase your branch on top of whatever the target branch is (normally the master branch of the repository you forked), or you can merge the target branch into your branch.
Previously in the book it said that I should never rebase anything that was made public.
So I shouldn't really use rebase in this case, because I pushed my local topic branch to my online GitHub fork. Is that correct?
doesn't matter if you are the only one on the feature branch
and even if you aren't, there's ways to work with it
How?
In the The Perils of Rebasing they showed that rebasing a branch that was already pushed can create some serious problems if other people pulled it down and worked on the branch:
https://git-scm.com/book/en/v2/Git-Branching-Rebasing#_rebase_peril
if your work is a feature branch awaiting a merge
it's damn weird to have others basing their work on your branch in the first place
also it seems git got a lot smarter about that and allows them to mostly rebase their work ok
yeah people dont use your feature branch to work on top of it
they just reset to your latest version of the branch
rebasing is way cleaner for feature branches imo
i hate seeing PRs with merge commits, they just become so messy
I'm following the instructions given in the last Inside HQ: Version Control Fundamentals setting up a Perforce server on Digital Ocean (basically @burnt shadow 's instructions), but when I go to connect using P4Admin it's coming back with a connect failed: check $P4PORT .... WSAECONNREFUSED. Can anyone help me out here?
Ooo, I think I found the problem...
Yep, for some reason Perforce didn't start up, just had to log in through putty and run p4d and all seems well
what is HQ: Version Control Fundamentals
nvm
oh no
OH NO
THE LIVE STREAM POINTED PEOPLE TO THE BLOG
OH GOD
Haha
@burnt shadow you mention daemon tools in the blog post, but the code to download it no longer works (at least for me)
now you have to delete the blog post to avoid misleading so many people
what blog are we talking about and what's wrong with it
I assume the one that explains how to set up p4 in a completely ancient way
I unlisted it once and I got a bunch of email about where did it go
The only solution it seems is to update it
Zzzzzz
replace it with a link to the official guide for package based install 
Docker is where its at this days
Thatβd constitute updating it
Kubernetes is the new docker
In 10 years weβll come full circle back to bare metal monoliths because a single node will have 256 cores
@mossy basalt launch client P4V and it will ask you connection details
you can add users/groups and set privileges though p4admin
Ok Iβll look at that now one sec
you need to show screenshot where you stuck and why it doesn't let you connect π
K
P4Admin wonβt open .-.
Took awhile
What do I do from here?
I opened it and clicked ok
Iβll get the other guy to do it because I canβt be bothered to do it
@burnt shadow we wont. Because microservices are not really about performance (;
which I'm sure will change
micro services is just a more complicated, distributed monolith
which when done right is resilient and available
once you 256 cores machine is down
all you services go down with it (;
same if the cloud provider decides to run your 100 micro services on the same physical machine and that goes down
Has any one done a git to perforce migration?
@winged tiger migration as moving entire history and users?
Yes
well, I only know that p4 can operate on Git (didn't use it myself)
but I'm surprised how easily one could sync repositories between Git and P4
Learn how to use git-p4 to set up a two-way code-sharing process between Git and Perforce code-bases so that improvements in either version can be shared
for what's worth, keep in mind that a single developer in p4 is a "workspace", and one "user" (group of privileges sharing a password) could have many workspaces
although official docs could try to convince you that you need a separate "p4 user" for every person - which you can do if want to spend a lot on P4 licenses π
Does anyone know what the difference between git fusion and helix4git is?
Or was git fusion just the old name?
git fusion is set of python scripts, translating git commands to perforce
helix4git is actually new type of perforce depot
(Graph Depot)
which works like git
while maintaing perforce scalability for large unmergable files
but
it only works from command line
any other perforce client cannot see Graph Depots
Hmm we are trying to setup a workflow that allows for coders to use git and artists to use perforce
And aren't sure which of those to use
Oh so artists can't use p4v?
the only distinction is that in helix4git you don't really have git server but perforce server posing as git (;
no
you have to create custom client
or let them use any graphical git client
Oh argh
whatever (;
But with git fusion we can work with git and perforce guis?
I'm also trying to migrate our existing git repo to perforce with history
git fusision
is translation layer
so in theory yes
in practice it gets slower with amount of users/size of repo
i will tell you
if you already have perforce
just use Task Streams
it's close enough to git branches
What workflow do you guys use for streams? Do you create one for dev/qa/prod and then also a main?
And do you create streams for artists or shared with developers?
@winged tiger
stream with game/engine code for programmers
stream with assets source for artists, separate for audio guys with their sources
stream for designers contains only binaries and game content
QA actually uses VS and writes automation tests, so QA gets the source code too
So you keep your code and content in separate streams?
@fallen geyser we did end up distributing the cache and reducing the shader compile time to 0 for the team. I am curious, however, about what you were saying about potentially reducing the compilation overhead by modifying the shaders themselves.
@winged tiger mhm, a single branch for most of the productions (unless doing hacky demo or weekly updates of the game) and streams with workspace mappings excluding parts of repository which given group of users wouldn't ever touch
- official P4 plugin for VS which automatically checks out code files while you're starting to edit it
so if you don't need sweetness of git branches all the time, P4 is simply enough for everybody
@lavish latch I'm no artist myself and this should be asked in #graphics room
I don't know your project/assets, so just generally speaking
- reducing unique materials and replacing them by materials instances (we're not that insane as other engine to use just a single Uber Shader, but also... unreal artists love to create a new shader for everything)
- reducing number of materials per actor (i.e. if given mesh uses 5 materials, it could be probably reduced to 1 or 2) - and that's also reduces number of draw calls
- inspecting your shader graphs, probably lot of them can use half or even less instructions to get job done without noticeable visual difference (it's super easy to make this graph huge)
- checking if you need as many frequently material functions as you have, and Material Parameter Collection - changing something things there can send you to the compilation town
- trying if new toys like new Material Layers (which doesn't seem to be a dead end this time?) would help in general
@fallen geyser thanks Iβll pass this along. π
@winged tiger what @fallen geyser said + I would recommend using Task Streams for indivudual features especially if they are contained in code
these are supposed to be used like git branches
@winged tiger doesn't really do 'forks' but we use streams for this purpose here
I promised to post a tutorial on using Perforce streams, so let's just dive right in. Hereβs the short synopsis of what weβll do:
Yeah, streams is the closest you'll get to forking functionality.
They work pretty well as long as you are aware of their limitations vs git
I am setting up p4d on my webserver. Should/can I use the ssl certs I use for my webserver ssl (provided by letsencrypt) or is using a self signed cert preferable?
note that this is going to be a personal server ( i will probably be the sole user) so self signed is fine for my needs. bud id like to do things "right" and I already have certbot set up
alternatively I may want to gen new letsencrypt certs for p4 altogether?
sigh, can't even fetch all from UE 4 repo without git getting stuck at 76% while resolving deltas π
how does one go about doing code reviews in a p4 workflow?
There's also Helix Swarm, which is kind of like a web version of shelve: https://www.perforce.com/products/helix-swarm
Helix Swarm is a web-based, free code review tool for Helix Core. You can use this code review software to collaborate better, do code reviews faster, and improve release quality. See how.
BEst way for code review if you want to make some setup
it make task streams
which are auto merged to dev stream upon review
anybody get the error: Mac file format detected: please convert the source file to either DOS or UNIX format when compiling with files from p4?
not sure what step i missed
(my line endings are set to 'win' on the client)
maybe the wrong char encoding?
@pseudo osprey Thanks Stof! I actually ended up figuring this out last night (I had to manually convert the line endings)
Hey people, was wondering if someone could give me a bit of advice on source control. Whenever another person pulls the UE project branch from our git repo they have to rebuild it because the "plugins are missing or were built on a different engine version" (i assume because the project is a custom engine due to plugins) , and then they get hit with an error saying that there are some .dll files missing. we are both relatively new to SC and so we have the general idea/workflow down but we keep running into this issue. I'm guessing there are certain plugin considerations we have to make that we just aren't... any help would be really appreciated π
for reference its a BP project
@frank shale it's not clear to me, are you using custom engine with blueprint project?
this kind of error is usually caused by mismatch .modules files - every module has this file and build ID in every single one of them needs to be the same (enabled module with the mismatched build ID will trigger error)
it shouldn't happen under vanilla engine, unless somebody would download a plugin compiled for different engine version or didn't submit everything from plugin Binaries folder
Question: When creating a new branch with Github Desktop, it created a new hooks directory and lfs\objects directory in my main game directory. Now those directories show up under changes. Should those directories be committed along with my changes?
@fallen geyser we are using vanilla engine with three plugins. And using the standard gitignore from ue4. So everything gets commuted and pushed but the error happens on the other end.
@frank shale difficult to say what's wrong then ;)
need to manually check if they have properly updated files: .upluging, what's in Binaries
I'd use remote desktop to compare everything with files on computer where everything works
sometimes it's just stupid mistake on ignore definition or conflicted edits
@old imp you need to update your lfs to 2.10
I've got a Perforce issue I can't figure out. How do I get Perforce to ignore the Intermediate folder?
good heavens. nevermind, I was putting the slash in the wrong spot...
@quaint obsidian thanks i want to try that.. is it git lfs update command? I've tried downloading and installing 2.10 but git lfs version is still showing 2.9
@old imp you have the bundle version of lfs that comes with git for windows, delete the git-lfs exe in the git bin folder
@quaint obsidian i tried doing that, but it still wasnt working for some reason. so i ended up uninstalling git and git-lfs. then reinstalling git (without the lfs option) followed by the git-lfs 2.10 installer. so now git lfs version is showing 2.10, but the hooks/lfs directories are still showing up in git status. do i have to run git lfs install again or another command once it has been updated to version 2.10?
hmm that's odd. It's not showing in Github Desktop anymore which is good since that's what I always use. For some the command-line git status has the hooks/lfs folders listed as green (changes to be committed) even though I have never used command-line git directly, always the Github Desktop GUI. So now it shows me deleting the hooks/lfs folders in red (changes not staged for commit). Hope I didn't screw anything up by running git lfs update because I'm not really sure what that did. @quaint obsidian
i never manually added any of those files/directories so im not sure how they appeared under changes to be committed in the first place
I guess because the Github Desktop GUI had them checked by default, it added them automatically. So maybe I should have unchecked them in Github Desktop first
@old imp can you tell me what git status says
it's probably still having the files added, since thats the change that was last staged
but now the unstaged is to remove it, which will cancel out the previous add
you should be able to do git rm hooks
I think that what you are saying sounds correct, since under "Changes to be committed" in green it shows "new file" for each one.
new file: hooks/post-checkout
new file: hooks/post-commit
new file: lfs/objects/....
And then in red under "Changes not staged for commit" in red it shows:
deleted: hooks/post-checkout
deleted: hooks/post-commit
deleted: lfs/object/...
So I should run that command git rm hooks
@quaint obsidian I ran git rm hooks -r and git rm lfs -r . I don't see any of the directories listed in git status anymore and Github Desktop doesn't show them either. I hope this means the problem is resolved now. Hopefully I didn't screw anything up in the process but I guess time will tell that! Thanks for your help. π
whats the difference between 4.25 and 4.25 plus
no one knows atm (and there's no differences between them in the github branches currently). I feel like it would be best to ask in #ue4-general
but I think you should use 4.25 for now
nobody should use 4.25 for now π
it's probably quite unstable just after making a branch
I meant when given a choice between 4.25 and 4.25-plus
was just curious as 4.25 is labeled in trello to have improvements to atmospheric sky ( which is blown out in 4.24 ) so yeah
who wanna join voice chat
<rant>
Perforce's UI is just... not great. I really should be able to see some kind of progress/status when syncing hundreds of thousands of files. Rather than just watching a percentage change every few minutes at best. Is it hung? Has Perforce crashed? Who knows.
wait you get a percentage?
for me it just shows copying <n> files to workspace (parallel sync)... and then nothing else happens until its done
@robust jacinth @dusk dove
check the 2nd option there, it will log operation on every single file - I kinda feel "left in the dark" without this logging enabled π
although there's no progress bar on single file progress, "obviously"
it the P4 connection would hang, it would show error after timeout passed... so I'm not sure if we have a real issue here π
I've used that before, but super verbose logging is a poor solution for bad UI/UX.
seems like in your case that'd make it dump hundreds of thousands of messages to the log lol
does p4 write to a log file? in that case you could make a progress UI from the output
How do I delete a changelist without deleting my entire game's project files
because I just wanna be able to save my files again
perforce keeps locking them
@cursive cosmos It sounds like you're doing something very unusual as a typical workflow.
Why would deleteing a changelist delete your game's project files?
You can't delete a changelist that has files checked out - it would make you revert.
And if the files are marked for add, then the option to revert should ask you if you also want to delete the file, or simply revert their status from "marked for add".
Hey all! Moved some files around inside of Unreal, to make things better organized, when I try to submit my changes via perforce, I get an error stating that the file path cannot be found.
I was wondering if anyone here can help me with this. I would greatly appreciate it! π
@pulsar orbit you could have one or more files on the changelist that don't exist anymore, i.e. you "marked for add" some asset, but later deleted it manually or without connection to p4 - and p4 didn't catch up
you could try submitting changelist in parts if there are many assets on that list
@fallen geyser I tried it, but still didn't work.
@fallen geyser I did however revert the changes, but still seem to be fine, but in the depot (server end), it's still outdated.
Green "O" = folder moved to "3rdParty_Assets"
Red "X" = deleted folder.
if you reverted all changes locally, just try moving assets again and submit it one more time
not worth to investigate it further (I have no clue what happened) if it would work now
obviously, you're moving assets in the editor, right?
Yeah I moved assets in the editor.
if you reverted all changes locally, just try moving assets again and submit it one more time
@fallen geyser
The weird thing is, is that the folder structure in the UE editor corresponds to the "workspace" folder structure in P4V.
So there's nothing to resubmit.
@fallen geyser Should I just "mark for delete" on the depot end? That's the only solution I can think of. Unless anyone else has other ideas.
run Reconcile on the entire content
it would find all differences between server depot and your local workspace version
there's no such a things "as marking something on the depot end", you're doing everything locally and after commit you'd see changes
Hey I'm curious and wondering if people know, does p4d know about the free "5 users and 20 workspaces" deal, or is it just expected that once the user exceeds that they pay up?
Similar to the Unity and Unreal licensing deals?
does anyone here use something other than git-lfs for their assets? alongside git of course
im getting annoyed with the size blowouts and github yelling at me to buy more storage
@gentle bay You can use Azure DevOps Repos, it has unlimited storage, including LFS. Private repos have first 5 users free, then $6/user/month.
GitLab has a 10GB storage limit, with no current way to buy extra storage (though, it is being worked on -- https://gitlab.com/gitlab-org/gitlab/issues/5634), and it has free unlimited collaborators for private repos.
BitBucket has up to 5 users free with 1GB Git LFS storage included, and has addons for 100GB Git LFS storage for $10/month. If you go beyond 5 users, you will have to pay $3/user/month, 5GB Git LFS included. BitBucket does not support LFS file locking.
Fantastic thanks for the info
can anyone who knows how to use GitHub well for game dev give me some tips on uses branches and things LIKE that to make the workflow smoother?
what we do is have a master branch, and art branch and development branch, basically on master the idea is to maintain a completely stable build at all times, e.g. at the end of every sprint or whatever
then the art branch is just a branch for artists we use, they're all on the same
and then development is where all the developer's work comes together, we make feature branches of it when working on something and then merge those into development with pull requests
we also have an automated system that then compiles binaries every time development is updated and pushes those binaries to the art branch so they're up to date as well
oh thats actually really smart
thanks guys
so then once everything is done on the "Development" branch it would then be merged with the master branch?
How does this handle binary merges? (We used git and git-lfs in the past and it was a bit of a mess - lots of broken merges)
Unless we did something really wrong ofc π
@chrome edge generally you shouldn't even try to merge binary assets
that's why I'm surprised anyone in gamedev is willing to work on multiple branches when project is obviously composed mostly out of binary assets, and in Unreal blueprints and maps are extremely important, but difficult to merge
there are exception of course, like development of already published game: early access, game-as-service
working on code itself on multiple branches is entirely different tale π
YAGER knows first hand that live service games are under constant evolution after ship, which puts heavy requirements on the initial architecture. In this presentation from Unreal Fest Europe 2019, Senior Game Engineer Joakim Ohlander explains how YAGER changed the Dreadnought...
that's entire talk from Jager, for instance they stopped doing items/weapons/things as blueprints - this would be a hell to merge - now this things defined as data table, etc.
shouldn't "git add --all" add all of the files within my repo that aren't on the ignore list?
because i for the life of me can't get my source files to be tracked
@autumn geyser maybe git add . --all? I'm not sure if it'll make a difference but that's what I always do
@chrome edge We have locking, so we don't merge binaries
will try
Yeah, we swapped to perforce to avoid merge issues. Just wondering that others keep talking about git. Works well for code, not so for uassets.
it's because perforce gets ridiculous expensive past those 20 workspaces you get for free
@chrome edge not sure why it wouldn't be good for uassets, it has locking that we use. that blocks merge issues from happening just like perforce checkout.
@quaint obsidian - do you lock with LFS? Or a provider? (I think gitlab has this concept?)
with LFS
it has a locking protocol
github, azure and gitlab implement it
clientside, files are locked in unreal using the git source control plugin
yes
Can someone help me out, i cant open my project
but my friend can on his other computer
Pls help I have zero idea what to do
The changes (shoebox) made are showing up in the master file so I have no idea what the issue is
@devout quail you had a branch that is being pull requested, and you modified files in that branch that were modified after on master
you need to remove those changes from your branch
in the future, use locking to prevent it
turns out AMD is not UE4 friendly, and the drivers sometimes dont make ue4 projects run, We fixed it by disactivating all lighting, pushing it, and later re activating them after pulling from the other side.
I just need to vent. This past week my HDD started dying. "No biggie I have my project stored on github, have several physical backups." Man was I naΓ―ve. For my professional job i use git, but simply getting my project over onto my new HDD has been a living nightmare. I had the project stored on an SSD, but my OS is on an HDD. You would think installing the correct UE4 engine version + plugins on a new HDD would be enough to run that same project from my OS on the new HDD. But no, it hard freezes my pc at 95% every time. Ok....I'll copy it over, NOPE HARD FREEZE! Well I'll try to clone it from my github. Still.....a hard freeze. Last thing i tried was running the cloned repo stored on my new HDD, while operating from my old HDD that is slowly dying, it actually loads it, but is missing so many references that it's not usable. I feel physically sick to my stomach over this. The UE4/Git workflow is a complete joke. I'm sad, angry, frustrated, and just disgusted by the whole thing.
So where are we supposed to save our large files such as 3D assests and maps?
@autumn geyser I use github and gitlfs too. I work with 2 computers (desktop and laptop) but I didnβt have that issue. I was able to clone my project onto a different type of drive too. What was the project referencing ?
how sure are you that it's freezing at 95% and not just compiling shaders for a long time for example
you should check your task manager to be sure
I don't think it's related to git in this case, and more with the HDD unfortunately. I used to use UE4 with an HDD and it was really awful.
i think 45% is for compiling shaders, not sure what 95% is.
without seeing your log, it will be hard. but keep in mind that it can get stuck on 95% for even 10 hours
@autumn geyser oh, it would be stuck for long if the new OS drive is still HDD... SSD should be only used for development, HDD performance is basically useless with software like UE4 editor
even new consoles will have SSD as standard
@night cloud
- it also happens with NVidia to encounter issue with drivers, it's nothing unusual in GPU driver world, it's just rare on Windows
- for future, you callstack (Crash Reporter window) doesn't contain symbols - just showing module names, not details: class, line of code, function name. You would get that by installing debugging symbols through launcher - it's the optional download.
with the symbols installed, the big chance we'd see "GPU hang" there and immediately suggested that's GPU driver issuue π
How does source control with a team of 2 work? Me team mate lives on the other side of the world, is this do able with perforce?
@dire frost yes, it is
you can get perforce in a cloud - people recommend Digital Ocean for instance
not sure if would get CDN for P4 repository there...
although P4 got services like proxy servers to solve issues with connecting remote servers - so while you work it would be faster than directly connecting to original server (faster locking files, querying current state of file on P4, etc.)
@dire frost subversion is also an option. it is a bit simpler to set up and administer than perforce IME
i think that dreamhost offers hosted svn instances
oh, maybe not. it looks like it may have been deprecated
Aright thanks, and I looked into cloud, stumbled upon assembla but that was a little out of our league with their 44.5$/user/month
don't do assembla
p4d is very efficient so you can just run it off a $5 vps for a small number of users
only issue will be getting enough storage, depending if this is one of those projects with 500GB art source files..
@fallen geyser the project and editor are on the ssd my os is on the hdd
Running the editor from my new hdd/os freezes the pc.
@autumn geyser UE stores cache for all the assets on C partition, buried in Users folder π
still, even OS on HDD will slow you down, especially if virtual memory is in use
π
yeah, but I have no idea how slow would opening project with HDD, haven't used HDD for Unreal like... never π
As even in my 8 year old to shins drive performance was more than acceptable.
Toshiba*
sell your bed, buy SSD π
Read post
Have multiple
Does unreal store asset caches in c drive if the editor is not on the c drive?
C:\Users\UserName\AppData\Local\UnrealEngine
if you have 100 GB free on that SSD and gonna run out of space soon, try to move everything to this SSD
and try then π
I'd rather starve then run OS on HDD again π
Iβd rather just accept that everything is capped by the cables and not that big of a difference
que?
Iβll double check after work to see if the cache storing is true.
Then move my new hdd to that partition if thatβs the case.
is there a size limit on azure devops repos?
@autumn geyser The problem might be shader compilation, you can slow it down using these settings: https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Config/BaseEngine.ini#L1161
@simple bloom no
Ok questions number 2: can I store lfs stuff on one repo and non LFS stuff on another?
using two repos? it works but it's a pain to keep them well in sync, especially when you want to do updates on both at the same time
Adding to the above statement, it's not necessary to store LFS-controlled assets in a separate repo since the assets themselves are stored outside the repo anyways.
Well I like using gituhb apps and stuff but storage cost money. So I want to store code in GitHub and lfs stuff in azure devops
I think you can still use the github desktop app with repos on azure though
Friendly reminder: For the love of gods, please put your engine and project directories side-by-side.
Or you will have a very bad time with various Epic created tools relating to Perforce.
Also, anyone have experience getting Jenkins p4 triggers to work?
I seem to be hitting the endpoint using curl without error, but nothing hapens...
I do, but I can't remember any of it
once a trigger works I'll never change it
I just remember that its all bullshit and perforce has shit tools for greping changelist nonsense
If you go to the trigger in a browser does it work? Also are you requiring authentication in Jenkins?
It does work, via the browser.
As in the manual trigger from the Jenkins page.
I wrote a Python script that's using PyCurl to send the Post data, but nothing happens.
I even went into Jenkins and enabled ALL logging for the P4 plugin - it has lines for when I do the manual trigger, but nothing for my Python script's run.
I get back a response code of 200, which is "OK". I know it's getting to Jenkins because it was asking for a crumb, which I got working, and then I switched to using an API Token.
I tried making the payload for the post request in a few different formats and nothing different seems to happen regardless, so I've begun to wonder if it's the JSON data not being quite right somehow.
Hmm, that sounds familiar as well from when I was setting it up... but I can't think what might be wrong. We're using requests in python to do the request/provide the auth.
requests.post("http://jenkins/job/CodeBuilder/buildWithParameters?token=llamapalooza&BranchParam=%s&cause=P4TriggerCL %s" % (branch, changelist), auth=P4Auth)
@robust jacinth I followed these instructions + added p4 trigger on submit, reacting on .cpp, .h and .uproject files
https://stackoverflow.com/questions/50702620/jenkins-perforce-triggered-build-on-windows/50759015
Yeah, that can also work. I was trying to do the proper, built-in thing. I ended up resorting to having it poll every 15 minutes instead of having P4 trigger the build. :/
Both of these are alternative solutions and viable, as is the SCM polling.
but polling isn't instant reaction π
No, but it shouldn't be too bad for now.
ok π
I am not a back end engineer or web dev, by any means, so debugging a plugin of a web app is... π¬
I think I'm getting a response code of 200 even if the JSON is malformed.
Something something are your headers correct?
Not much to it, my Python code looks like so:
anyway, proper P4 trigger doesn't work since Jenkins uses P4Java implementation - can't remember details, but this failed on me
so I'm using curl solution
no Python needed
changeTriggerCurl = pycurl.Curl()
changeTriggerCurl.setopt(pycurl.USERPWD, jenkinsUsername + ":" + jenkinsAPIToken)
changeTriggerCurl.setopt(pycurl.URL, jenkinsURL + jenkinsP4ChangeTriggerEndPoint)
changeTriggerCurl.setopt(pycurl.HTTPHEADER, ["Content-type:application/json"])
changeTriggerCurl.setopt(pycurl.POST, 1)
payload = {'change': int(changeList), 'p4port': "\"" + p4Port + "\""}
jsonPayload = json.dumps(payload)
changeTriggerCurl.setopt(pycurl.POSTFIELDS, jsonPayload)
changeTriggerCurl.perform()
print(changeTriggerCurl.getinfo(pycurl.RESPONSE_CODE))
changeTriggerCurl.close()```
@fallen geyser I don't follow. I'm using curl. It's just not actually triggering anything in Jenkins.
I happen to be invoking it from Python, but that's neither here nor there.
I know it's hitting the end point because it'll make me fail if I prove the wrong API Token
Or if I supply the right password, it'll demand a crumb, and I'm able to get that working, but that's old deprecated stuff compared to API tokens.
why python?
i simply got .bat as described on Stack Overflow answer
C:\curl-7.60.0-win64-mingw\bin\curl.exe -u userID:APIToken JenkinsIP/job/TestJob/build?token=buildCode
and bat called from the trigger set on p4 server
Because Batch files are unmaintainable nightmares, compared to Python or Bash.
I mean, even Microsoft created PowerShell rather than keep using Batch. :X
yeah, I know, you can use anything to call that Jenkins link, basically
Like PyCurl. π
And yes, that's doable too.
I was hoping to keep things extra secure by only exposing the necessary end point.
I see π
Arbitrary execution of builds from remote machines is... a little more worrisome.
is perforce actually better than git? is there a free host for perforce repos? perforce.com doesn't seem to mention perforce repos, instead it has something called Helix, which works with git/svn/mercurial... and i feel like i'm just misunderstanding something.
unless helix IS the front end client for perforce, and they just don't ever say that? like it's expected you already know it or something? lol
perforce is the name of the company
the version control is called helix core
they also have a product called helix cloud that is unrelated to helix core and hosts other version control systems, but not their own (why ...?)
every time i hear someone talk about source control (on unreal streams, etc), i've never heard the word "helix" lol
so is p4 the command line client for helix?
they used to call the software perforce aswell, then tried to rebrand it, but that didn't work, because now everyone uses the old name
yes
ok, that all makes sense, thanks
is there a standard host for it, like github/gitlabs?
no
or does everyone just run their own server on like a droplet or whatever?
the standard is to get a tiny server from linode or similar and run it there
ok cool
i think that's enough info for me to just google anything else i want to know lol
make sure you follow their own guide for package based install
not the ancient allar guide
(if you end up installing it)
so is it actually better than git? or am i better just sticking with what i know and moving on?
for binary files, which is gonna be most of an unreal project, yes
(unless your team is too big to make use of the free option, in which case you're looking at 40/month/user, making it way too expensive to be viable)
my team of one is not that big
for a team of one, using free git + lfs hosts might be more than enough
I use GitHub for my solo projects and itβs perfectly fine.
from what I see, getting a free Azure Teams repo is the best even π
cause they have kind of unlimited storage
Iβve heard! I just prefer GitHubβs UX and ecosystem. π
yeah but if you are a solo dev really, the UX/ecosystem feels a little meaningless π
as long as my git pull/push commands still work, I'm all set \o/
for a solo project, free git + lfs is more than enough yeah
you definitely want git + lfs for sure
@jolly fog its not
github on LFS on any unreal project will inmediately go over the limit
https://github.com/vblanco20-1/HoverGame this little thing, which is like 200-300 megabytes, is constantly spamming me with the "has reached the limit"
because if 3 people download it, BAM, over the github limit
github LFS limits are so strict and so expensive, that perforce will actually be cheaper, and thats actually BUYING perforce and renting an amazon server to run it
cheapest is to not even run perforce. Grab a SVN server on amazon or any other cloud provider, and set it up. Works quite easy and SVN itself is free, so it only costs you the 10-20 euros/month for the server itself
something i actually recomend to small studios or solo devs is to grab a Raspberry pi and guetto it as a server. One-time cost of 50 euros or something and you basically forget about it
i'm not using github
that's why I said to use Microsoft Azure Teams
@noble knoll that repo doesn't have a .gitattributes file?
@quaint obsidian could be github hiding it?
my repo still has it
I wonder... is there a way to use or access UnrealGameSync Tool without compiling the whole engine? (just using the official Unreal Released verrions, in this case 4.24)
@here
@jolly fog It's a standalone project so you definitely can
Hi @smoky trench , when I have try it, at the time to open the project says something like "you don't have this version of the engine in perforce". So, right now I'm compiling my own version of the engine and uploading it to perforce. The question that I asked was based on this, If I would be able to avoid this step (which I'm currently doing already)
I spent some time setting up UGS last week.
It seems very much set up to work assuming things are like Epic's internal conventions:
Your project's root directory is side-by-side with the Engine directory and in a Perforce stream.
If any part of that isn't true, it misbehaves somewhat.
somewhat is an understatement
also, found out today, if a file name has a # sign in it
ugs will fail syncing
perforce supports such filenames
but ugs' calls to perforce don't escape names properly so p4 interprets the # has a revision lookup command
whats best source control you guys have used for LFS?
what do you mean?
LFS is for git, so there's not really options
or do you mean which host
I got this depot MagicCode, where my project is. Another user needs acces to it but he can't see MagicCode depot. Do you know how to resolve this?
@burnt shadow if I can ask you for help, Sir
@hushed spoke check if this has proper access in p4admin
you probably see everything as admin
he has the same acces as other team members who see MagicCode
is he on the same stream? could select something different in his workspace
the problem is his "stream" tab is empty and he can't click browse or anything
@fallen geyser
yes but he can't click anything there