source-control

[ocean anvil]

it should, but in a proper UE workflow with perforce you should almost never need that though.

[open jay]

Right, this is a special scenario though, as I want to upgrade the engine

[open jay]

Indeed this is crazy fast, Thank you for it!

[fading chasm]

can some help me understand something regarding IDE/Rider + SourceControl, I get errors like these, but I have no clue what the reason is or what is emitting it... All the mentioned paths are on the repo and tracking is going fine. These seems like false positive but I want to investigate.

its been happening for many project with no apparent effect, so I wonder why

[ripe prism]

Does anyone use AWS code commit? Curious if it really is 50GB repos for free

[open jay]

Anybody know how I can push a changelist with a large number of files (~300k) to perforce? It gets stuck at 1%

[woven sluice]

the only feeble advice i would consider giving is to try using the command line... it should have said the command it's using in hte p4v log window, right click on your project workspace folder root and open cmd window here, paste the command, sacrifice a lamb inside a candle pentagram, etc

[open jay]

what does the command line do as opposed to the visual client?

[woven sluice]

not suck balls at big operations

a little bit

[open jay]

Well, I did, but nothing is moving 😅

[woven sluice]

might wanna leave that one overnight lol

[open jay]

yeah, I don't think it does anything, looking at the task manager, there is no network usage

[woven sluice]

i wouldn't expect there to be any for a while

but... i also hope i'm not leading you down a path of waiting several days for no visible progress

[open jay]

Well, I can use the -I arg to get an idea of what's happening

but I expect the same result

[open jay]

Right, so it seems that my submit always gets stuck around the same files

If I skip those, it will get stuck somewhere else, but close by nonetheless

[woven sluice]

I want to ask if you ever let it run, but I'm too scared

[long mural]

anyone familiar with p4transfee

https://github.com/perforce/p4transfer

[pulsar parcel]

What even is the usecase for that?

[long mural]

cloning the epic perforce to your local perforce

im just hitting this

        [Error]: "Too many rows scanned (over 50000000); see 'p4 help maxscanrows'."

Traceback (most recent call last):
  File "./P4Transfer.py", line 2653, in replicate```

even specifying -m 1000, it craps out

ubuntu@ns3101456:~/p4t$ ./P4Transfer.py --ignore-integrations -m 500 --ignore-errors so confused

[pulsar parcel]

You need to copy the Epic P4 CL by CL? At previous studios we just reconciled changes

[long mural]

i mean i don't need too

i just wanted to automate pulling it from epic on our machine

@pulsar parcel

[pulsar parcel]

Oh then that's not really appropriate since it's a copy of each CL, not whatever the head is

It's an automatable job with the reconcile route

[long mural]

yeah never done that, im a bit of a P4 noob

[open jay]

eventually ran it in batches ~20 k each

Got it done

[wind gazelle]

So, got a problem.
git clone/push/fetch from github via https stopped working on a machine with win7
for people who yell about outdated os - it is intended to have win7 buildtest environment, but it is not git that is compiled

$ git clone https://github.com/xxx/xxxxxx.git
Cloning into 'xxxxxx'...
fatal: An error occurred while sending the request.
fatal: Request interrupted: Failed to create secure SSL/TLS channel. <message translated>
fatal: The request was aborted: Could not create SSL/TLS secure channel.

some googling told that it is caused by tls 1.1/tls 1.2 for some reason is off for WinHTTP so it fails to connect because github turned off tls 1.0 support
did some kb checks ( KB3140245, and few others) and registry checks according to ms article gave no effect.

https://stackoverflow.com/questions/63352796/error-in-windows-7-the-request-was-aborted-could-not-create-ssl-tls-secure-cha

Simply switching to ssh is not an option (got reject, since it is company repo) but fixing git to work in such env is a req. maybe someone had to deal with same thing

auth is not a problem - all access goes via access tokens, no login/pass

edit - solved: i accidentally got a trailing whitespace in one of registry keys when making edits. kb/regedit fix above worked

[sweet agate]

Can someone give an example of what kind of branches to have on Perforce for a game with monthly or weekly updates?
I would assume there would be a Production branch for the latest version of the game released to the public,
a Development branch containing all the unreleased changes,

maybe a QA branch to merge changes from Development pending to be released

and then merge the QA branch to Production when it's ready?

[ancient oxide]

are /ExternalActors/ intended to be in version control?

[hoary wren]

Is it at all possible to unshelve files into one's workspace on a different machine if those files are exclusively checked out? I just want to download them from the server read-only so I can build that version, I don't actually need them to be editable at all.

Right now the unshelving won't work because it says "can't edit exclusive file already opened", which is understandable, except I don't need to be able to edit it, just see it.

This seems about as close as I've gotten to an answer: https://stackoverflow.com/questions/51202446/unshelve-a-file-which-is-in-exclusive-check-out-in-perforce-p4

[wide raven]

That's odd, we can definitely both checkout and unshelve things on P4V bypassing locks if we need to, without actually jumping through any hoops.

[noble reef]

What do I need to do to enable VC with Perforce for my C++ classes ?
I can't manage to connect Rider with Perforce (the connection always fails). I'm lost and the lack of resources on this matter is seriously disturbing

[open jay]

What are the steps you're taking and what's the point of failure?

[timber drum]

Hey, anyone knows how to allow users out of network access my Perforce server?
Apparently it’s something to do with port access but I have not been able to open it
Below are the tutorials and steps I’ve been taking and watching but still no luck

1. downloaded p4v and p4d along with p4 cmd
2. created workspace
3. created streaming depot along with typemap and ignore maps
4. perfoce seems to be working on local machine
5. Laptop within local network does not seem to have access
6. check my port 1666 is still closed
7. Having issue with port forwarding on my Arris router

https://youtu.be/2me3R2FFjcI?si=_lTtvfQV8Xt3YpPY
https://youtu.be/z0kB3Mdm8rU?si=rdG_tG7EN049AsR8
https://youtu.be/FQn6rOzf-Ho?si=7Ido-OHkWphg2ncb
https://youtu.be/7PRo8gK6SNM?si=kE9ss8c72ZjQtvCU
Would really appreciate the help

[wind gazelle]

do you have a white/public ip in first place? basic ping is enough to check

if you behind local router NAT you need to forward the services ports to expose them from internet

check what ip:port perforce binds service to (should not be loopback interface)

you can use nmap/nping to do more advanced pinging and port check

[timber drum]

i see

this is definitely out of my league as I'm very IT dumb

I'll try to look those things up hopefully I'll get it

so I guess should the port forwarding for NAT be enabled?

[timber drum]

yes I did setup a static one

this is the UI i'm dealing with on my router

[woven sluice]

port forwarding

however, "Laptop within local network does not seem to have access" is questionable - port forwarding is to open doors between your network and the WWW, not within your LAN

[timber drum]

I see, while figuring out the out of network user. How would I give access to my laptop as a “in network” user? I get this error when I connect to localhost:1666

[woven sluice]

localhost means this pc... does your router interface have any network map showing devices?

you want something like 192.123.456.789

[timber drum]

yes I can see the server's "my main pc" ip address and type it in followed by :1666 but I get the same error

[woven sluice]

open a cmd window and ping 123.456.789.XYZ your server IP

does that work or just time out

i mean do that from your laptop

[timber drum]

it works

I dont get the "time out error"

however when I ping my laptop's ip from my main pc, I do get time out error

[wind gazelle]

for such things nping --count 1 --dest-port 1666 some.ip.v4.addr is gold
you can run netstat -ano | find /i "1666" on a machine with service to see which port service is bound for, it must not be localhost/127.0.0.1

[timber drum]

could this be something I did in perforce server when I created a depot? or this is purely my IT issue?

[timber drum]

I run this in cmd?

[peak nacelle]

Is it enough to add this .gitattributes for Github-based LFS UE5 projects?

[trail zinc]

looks correct unless you want to use the locking functionality, then you need to add lockable to the entries

Can anyone using the UEGitPlugin give advice on how to deal with locked files? After making an edit or commiting an asset, the files remain locked, and the only way I've found to unlock is to use git lfs unlock from the command line.

[trail zinc]

My issue with not being able to revert files is because of the username I put in the field next to Uses Git LFS where it says Username to lock files on the LFS Server. I noticed that the files were being locked not by me, but by other "proper" username used on the LFS Server. Once I put that name into the field, then the system recognized they were checked out by me so I could Revert them.

[peak nacelle]

thanks

[noble reef]

Anyone knows how I can fix the following problem: Rider won't integrate Perforce as the VCS solution for my project. It keeps using the default environment variables, even though I tried to edit them in my console command AND in Rider's Perforce settings. So whenever I relaunch the IDE, I see that the connection fails cause perforce:1666 isn't available.
Maybe I missed some stuff during my configuration but right now, I'm clueless. I have no idea where it's coming from. What's weird is that last week, it was working perfectly.

And proof that it's a weird issue, the "Test Connection" button in th settings says that the connection was succesful when I'm clicking on it

So the server is fine and P4V can connect to the depot as well so it comes from Rider. BUT some tickets were created years ago about this and they're saying that they actually fixed it. So wtf?

[hoary wren]

Perforce question: we want to give a bunch of folks access to our perforce for creating assets which we'll later port into the game, but there's no point in giving them access to the game folders within the depot because they won't be doing anything with it anyway. What's the the best way to accomplish this sort of depot slicing?

I've seen permission tables recommended, alternatively I think you can use a separate stream which then is mapped to a particular folder in mainline? Or could create a separate depot altogether, but ideally we'd have it all in one place. Is that at all possible? Curious what the best practices here are.

[arctic hornet]

Well, the structure itself depends... but in general perforce roles and permissions will be your biggest friend - if it's to protect stuff server-side
From what I've usually seen studios usually just do //MyProject/Assets stream where they store raw assets (FBXs, MBs etc...)
Once you have that you assign them permissions to have write access to only to that stream (and potentially also remove access from other streams)

The other options, if it's controlled environment and you don't worry about people potentially having access to the rest of the project is to use workspace filters to limit the access...

And lastly, stream mappings are also fun... both in a way of using virtual streams or mapping a stream back
you could make the //MyProject/Assets to be a virtual stream that just limits the scope to whatever folder in your //MyProject/Main you have for raw assets... any changes pushed to the virtual streams really happen in the Main stream
Other other way round is to use the stream mapping and that is to map the //MyProject/Assets/... stream to //MyProject/Main/RawAssets/... (where RawAssets is a folder name inside the stream) and if you properly set the mapping it will be two way linkage... so you'll be to push both from Main andd Assets streams... and regardless where you push stuff from, the raw assets will get stored in the //MyProject/Assets stream

TL;DR: There's many way to do it 🙂

[hoary wren]

Appreciate it, super helpful!

As far as that "raw assets stream" approach, what type of stream would you recommend that one use for that?

[arctic hornet]

from practical standpoint all the streams are the same (i.e. Development, Mainline, Release) perforce just gives some warning regarding copy/migrate but that's it... but in general I'd use Mainline since it's not a development nor release branch based on Main stream

[hoary wren]

I see, that's great! So effectively have a mainline branch for assets that flows into another "everything ends up here" mainline stream type of situation?

[arctic hornet]

you have mainline branch for raw assets, where arists can either push stuff to directly... and you then ideally map it into mainline branch into something like /RawAssets/... and then import from within there, users can push into that folder too and it will show up in the Assets stream (you could treat it as like a window to the other stream)

the benefit of doing it that way is that all of your import paths will be relative to your workspace root

which simplifies future reimports

because you won't suddenly have assets imported from "C:\Users\MyArtist\Desktop\RandomAsset.fbx" but rather ../../RawAssets/RandomAsset.fbx

(although it's a policy that you have to maintain yourself... or write custom validators to prevent people from importing from outside of the RawAssets)

[hoary wren]

That makes a lot of sense, thank you! I haven't had to map streams to specific folders before, so I'm going to have to dig into that a bit to figure out how to do that best. Until now we were just using one depot with one mainline branch, didn't really have to know much about anything 🙂

[arctic hornet]

it's quite easy, I usually just do it in text files rather than visually but the syntax is quite easy

but you'd just do import+ RawAssets/... //MyProject/Assets/.. in the Paths section in P4V

import+ makes it a R/W assets, import is read-only one etc...

here's the doc for it https://www.perforce.com/manuals/p4guide/Content/P4Guide/streams.paths.html

[hoary wren]

Appreciate it, super helpful 🙏 Let me do some homework here and get back to y'all with more questions 🙂

[arctic hornet]

and we're using import+ rather than share simply because we awalys want to map it to the specific streams rather than duplicating the assets on merge etc...

I won't be availalbe that much as I'm flying tomorrow for vacation for a week to London but I'll try to answer your questions asap if nobody else does in the meantime 🙂

[hoary wren]

Thanks so much, you've been super generous with your time as it is 🙂

Unrelated, but does shelving files in Perforce typically imply that they will be unlocked once the shelving is done? Or are locks preserved inspite of shelving those file, the same way they would be if you were to move those from the default changelist to a numbered one?

[noble reef]

I'm struggling with something that seems simple but I can't manage to find a solution. I configured Perforce in Rider, it works and all. But I have all these files that are in folders specified in my p4ignore file in my change list AND in this "Modified without checkout". My current workspace is supposed to be clean (it is in p4v) so these "changes" shouldn't even be mentionned. It's not blocking but it's annoying to see that (call it my OCD kicking in or whatever).
How do I clean this window of all these files so I can only my actual code changes?

[arctic hornet]

Modified without checkout means that those files are locally writable but not checked out (duh) or not on the server... honestly from experience with Rider, I just use rider to auto checkout code and then submit from P4V or P4 directly rather than using perforce interface in Rider, as it can get wonky

So, just ignore the tab

[arctic hornet]

Shelved files get unlocked locally since they move from your computer to the server, once someone else unshelves it gets locked again...

This technique is used for example by Horde for preflights... I.e. you make changelist, shelve all the files and horde will take over, do it's tests and stuff and the optionally submit the files

Or it's used to transfer files between computers without pushing the code for real

[noble reef]

Isn't there a way to simply disable that read-only system in Perforce ? I don't even really need it and it's the thing that caused the majority of my problems...

[arctic hornet]

You can do that but personally I don't recommend it, as it will really make it a pain when pushing

As you'll have to manually add files you want to submit etc.

If you open your workspace settings in the advanced tab there should be a checkbox like "all write" check that and everything will ve writable by default

Although you'll have to manually go and uncheck the "read only" flag on the workspace folder in windows, because perforce will not change existing files

[noble reef]

@arctic hornet Ok thanks dude. Got a last question if you don't mind.
Why some files are ignored (not in the right way) by my p4ignore file ? I specified the .idea/ folder in my p4ignore.txt but when I build the project, some files are edited in my workspace in that directory and they show up in the pending files

[fiery pelican]

For those using Github, do you have to pay for it or have you found ways to keep file storage down? I've only ever used github for Unity projects which were always small enough to not be an issue. My current UE project is already 2.7gb, which is over the free limit I believe. I'm not concerned about using github for source control as I'm already running my own perforce server, but I would like to put it up for portfolio purposes.

oh wait, maybe there's 15gb on github now? When I was googling a few minutes ago it said 2. lol. Guess I'm gonna try to put this up there and see if it works fine.

[ocean anvil]

use azure devops if you're using git, you get 250GB of storage per project for free.

[fiery pelican]

I've never used azure devops, but I assume that's just another git based storage? Is it entirely separate from github then? As in, the projects I put there won't show on github?

[ocean anvil]

for UE you really dont want to put your projects on github in the first place.
to answer your questions though,
Pretty much its just a cloud based storage location for Git.
No it wont show up on github.

[fiery pelican]

Why not on github? I don't need it for source control, I'd like it on github for portfolio purposes. I have my own perforce server for source control. So using azure devops doesn't get me anything I need.

[arctic hornet]

I'm not sure why that would be the case, I recommend double checking p4ignore format since it's not the same as gitignore, that's the only thing I can recoomend you right now, you could then use the p4 ignores command (https://www.perforce.com/manuals/cmdref/Content/CmdRef/p4_ignores.html) which can tell you what rule is being used for specific file etc.

[ocean anvil]

If its not for source control its probably not as bad but github has alot of limitations regarding filesize and project sizes etc. so you'll quickly start reaching limits on that.
it also does not have any good way of handling UEs binary filetypes so the majority of work on a project can't be seen by any recruiter on your github page.

if you are showing off code for a gamedev project it often helps to show the code as well as what it does, so show the source code and then a clip of the gameplay where that code is used for instance which you also can't do in github.

[fiery pelican]

Oh yea ok I gotcha. yea those were the reasons I used perforce to begin with. And yea it does sound like maybe an alternative might be better due to githubs other limitations as well.

[arctic hornet]

Git in general has been built for code in the first place (because it was made for linux kernel in the first place) it is decently usable with Git LFS for larger projects or projects containing binary files... which unreal does and there's A LOT of them... I personally can highly recommend using perforce if you can rather than git, simply due to the level of integration with unreal and it's surrounding tools... But git with git LFS is decently fine for small teams

but please, never.... ever... use raw git (without lfs) for unreal projects

it's gonna choke after few tens of commits

And also, Git is the software, github is just one of many implementations, so if you're ok with git, you can use github, bitbucket, gitea, azure devops and whatnot as they all provide standardized git (+ lfs) services at varying price points

[ocean anvil]

for better info on setting up a portfolio you might want to ask in #career-chat describe your fields of expertise and ask for recommendations or examples of suggestions for a good portfolio for those.

[fiery pelican]

Thanks, might do that!

[orchid solstice]

can someone help me fix this error
when i try to pull my project from sourcecontroll

''Cloning into 'C:\Users\myuser\Documents\Unreal Projects\MyGame'... remote: Enumerating objects: 1020, done. remote: Counting objects: 100% (1020/1020), done. remote: Compressing objects: 100% (993/993), done. remote: Total 1020 (delta 0), reused 1020 (delta 0), pack-reused 0 Receiving objects: 100% (1020/1020), 126.17 KiB | 161.00 KiB/s, done. Downloading Content/Characters/Mannequin_UE4/Animations/Jog_Fwd.uasset (203 KB) Error downloading object: Content/Characters/Mannequin_UE4/Animations/Jog_Fwd.uasset (81b4dd2): Smudge error: Error downloading Content/Characters/Mannequin_UE4/Animations/Jog_Fwd.uasset (81b4dd2a00331ba3086ce06a8673667359183399c5fb1ab924060e2c84720691): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. Errors logged to 'C:\Users\myuser\Documents\Unreal Projects\MyGame.git\lfs\logs\20240626T115623.9551488.log'. Use git lfs logs last to view the log. error: external filter 'git-lfs filter-process' failed fatal: Content/Characters/Mannequin_UE4/Animations/Jog_Fwd.uasset: smudge filter lfs failed warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/' Would you like to retry cloning ?''

when i open my project then everything is in there except the importaint files like bluprints, maps, and everything else that uses .uasset extension

[woven sluice]

Sorry but... isn't it clear enough?

[hoary wren]

Actually, that's a super key detail I feel like I don't understand. When I lock a file in perforce, does it not get locked for everybody else on that stream regardless of whether that file is the default changelist, a numbered one, or a shelf? Or does shelving a changelist actually unlock the file so that suddenly others can check it out and modify it, and now I'm in trouble when I try to unshelve later?

[arctic hornet]

file is locked for everyone in that stream as long as it has the "Exclusive checkout" flag (usually binary files, but can be changed by in p4 typemap) and it is locked for everyone (as that way it avoids overwriting binary assets since they can't be merged)
and the file stays locked as long as it's in someone's changelist, be it default or numbered one (as long as it's not submitted)
files put in shelf are just stored there so anyone unshelf them locally - you could imagine it like staging environment, you haven't pushed stuff, but it's also not just on your computer... but shelved files are not locked on their own, instead they're up to grabs until someone unshelves them, at which point they become locked

And honestly, I'm not really knowledgeable about shelves as all we use them for is to relinquish the ownership of unsubmitted files to our build farm so it can run tests on our behalf

i.e.

• I make code
• I make shelf
• Build server unshelves the files
• Build server runs the tests
• Build server submits the files under my name if they passed the tests
• If the files did not pass the tests I will get notified by the build server so I can fix them
• I then unshelf the stuff, fix the stuff, re-shelf them and retry

This is how the "preflight" thing works in Unreal (or I should say Horde) in it's basic most form

otherwise we only use selfs in limited capacity when I have some WIP changes that someone else wants to continue on, so we just "swap" the files but it's immediate so nobody else locks or modifies the asses in question during the transition etc... etc...

This is probably as much info as I can give you regarding shelves 🙂 so I'd probably redirect you to perforce documentation and internet in general if you can dig out more.

[woven sluice]

"the file stays locked as long as it's in someone's changelist" is the important bit there. IIRC when you shelve a file you can specify what happens to your changelist

and perforce does NOT lock files across streams, part of why most small teams just embrace simple single-trunk development

[ocean anvil]

i mean it says pretty litterally what the problem is.

[elder robin]

did you ever solve this? been having this issue with unicode/utf8 localization related files. tried adding the extensions as text to the typemap but no go

[long mural]

No

Updated the engine and it stated working again

Never seen it again

[hoary lichen]

Hey, whenever I open my UE project that is connected to git, it automatically takes the changes from the repo although I have local changes and it kinda removes my local changes..

this happens when I'm connected to git in the editor

if I disable it, it doesn't happen

how can I have it enabled and prevent this from happening ? I couldn't find anything in the project setting..

[rustic canyon]

Make your own Branch of UE5 and move the changes to it, if you need updates from other branches just merge them into that personal branch.

such as this

[open flame]

Hey getting a weird issue with perforce and VS Code

I can't connect to source control without VS Code trying to push a bunch of files on the ignore list and freezing the entire solution

has anyone run into this issue or similar

I get errors like the above then vs code simply freezes on me

[woven sluice]

tbh a real solution might be to just stop using VSCode for unreal, unless you're one of the diehards of it... hardly anyone uses VSCode

[open flame]

what would you recomend?

[zealous lance]

VS or there's Rider if you don't mind paying

Infact a convo yesterday that I seen was mentioning how much better it is to use rider with perforce, take that with a grain of a salt #programmer-hangout message

[pulsar parcel]

Though that screenshot is VS, not VSCode

They are not the same

Just disable the P4VS option to automatically add the solution file

Cc @open flame

[wind gazelle]

how do you setup github access tokens locally for each git repo?

quick find found specifying access token in push url in local git config, is this correct?

[remote "origin"]
url = https://[e-mail]/username/reponame.git
[user]
    name = xxxxxx
    email = [e-mail]

[rustic canyon]

just use the UE5 Perforce to do the updating, unless this is the 1st push to mainline?

[pulsar parcel]

The UE5 integration has no awareness of source code so that's not particularly useful

[orchid solstice]

can you tell me i didnt understand it

[ocean anvil]

read it, left to right top to bottom, it says word for word exactly what the problem is.

[orchid solstice]

alr thanks for the help!

[woven sluice]

i don't think you understood what sswires meant somehow

[rustic canyon]

ok.

[arctic hornet]

it has awareness of source code

well, if you use the full mode and not just the simple "submit assets"

as the full perforce "editor" allows you to control changelists etc... and it's also aware of config files and source code, as it can be treated more like alternative P4V UI etc...

since especially if you're using virtualized assets or external actors P4V becomes pain since it A) can't virtualize assets and B) for external actors it can only see the hashes rather than the "file names" since those are only stored in the metadata

[rustic canyon]

This is what I've noticed myself.

[arctic hornet]

the tools are still in development and ultimate goal for epic is to integrate everything into unreal directly + also technically speaking move away from Perforce as primary source because the licensing for epic is like millions of dollars a year due to their scale and stuff

so this is also part of the "Skein" which is nowadays used in UEFN and is called Unreal Version Control... and it's using that as it's primary UI (unless you use CLI)

so these tools are bound to get better over time 🤞

[rough rampart]

anyone using Azure DevOps here? I setup a repository and use SSH key to push pull. I was able to push when I push .gitattribute to enable LFS.
After that I initialized a blank project and check in. When I tried to push, after enable the keyphase for my ssh key, this windows keep poping up :

[rough rampart]

seem like git pull is working, only git push got this error.

[teal marsh]

Anyone got a good p4ignore for a 5.4 project being build with engine source alongside? The epic one seems to include a bunch of dll/so files that don't play nice with rebuilding (e.g. in horde) because they get marked read only and so rebuilds can't rewrite them...

[arctic hornet]

That sounds more like you pushing files that are not intended to be pushed, the p4ignore provided by epic in Extras folder is plenty and I've never had any issues with that

[teal marsh]

Yeah, that's what I mean, I am indeed pushing files that aren't intended to be pushed, but I wanted the p4ignore that would stop them being pushed 😅 I didn't know about the one in Extras/Perforce, it looks different from the epic one in their docs here https://dev.epicgames.com/documentation/en-us/unreal-engine/using-perforce-as-source-control-for-unreal-engine so I'll give it a try, thank you

[reef oriole]

Hey, I recently released the version 1.11.0 fo the "Plastic SCM" revision control provider https://github.com/PlasticSCM/UEPlasticPlugin/releases/tag/1.11.0 (for reference, the one integrated by default into UE5.4 is version 1.9.0 from 7 month ago)

https://forums.unrealengine.com/t/plastic-scm-plugin-1-11-0-released-with-a-new-view-changesets-window/1919904

[cunning surge]

Is there a way to have perforce only make certain folders readonly? I have the source engine in my depot but when i build I come across issues because some files/folders cant be written to. Looking to avoid this when adding new members.

[silver token]

those files shouldn't be in perforce to begin with.

[tiny ore]

Hum, little question :

On git, i've cherry-pick a list of commits.
Some of those commits was merge.

and i've got the following message :
error: commit 41500af is a merge but no -m option was given.

I've git cherry-pick --continue, awaiting to have a conflict resolve at the end, or something like a prompt, but nothing and the local branch seem fine, any clue of what happened ? do i need to re-cherry pick those commits ?

[wind gazelle]

if you did not allow merge pick (-m) it would simply do nothing and prompt an error.
you can view the tree in git gui

https://stackoverflow.com/questions/9229301/git-cherry-pick-says-38c74d-is-a-merge-but-no-m-option-was-given

[solid sage]

So the p4ignore provided by epic also includes, for example, large numbers of .pdb files. Are those really necessary to source control? Many are under Engine/Binaries/DotNET/AutomationTool/AutomationScripts, is maybe this entry in the p4ignore outdated?

[arctic hornet]

this would mean that the whole AutomationScripts folder is being ignored

which it should be since it contains locally compiled transient data

[solid sage]

https://github.com/EpicGames/UnrealEngine/commit/e7039d3d35d2a96ce9561b432701f0e4b3f1b1fa#diff-7a6787bf2e3be44509fac31c628dcd8ab91f0a48d2fb2d7a2676d9b16feb493eL21

My point is that in 5.0 they changed the output folder of the build to /AutomationTool/AutomationScripts, so that line in the p4ignore does nothing

[arctic hornet]

well, it's entirely possible it's not been updated

I'll send you a p4ignore that I use personally

# Ignore root Visual Studio solution files. We do check in some sln files in subdirectories, so only ignore
# the ones found in the root.
/*.sln
/.p4sync.txt

# Ignore all Visual Studio temp files.
*.suo
*.opensdf
*.sdf
/Engine/DerivedDataCache/*
**/DerivedDataCache/Boot.ddc
**/DerivedDataCache/**/*.udd

# Ignore all Intermediate and Saved directories
*/Intermediate/*
*/Saved/*
# Ignore UBT's configuration.xml
Engine/Programs/UnrealBuildTool/*
*.uatbuildrecord
*.tmp

# Ignore built binaries and temporary build files
*/obj/*
*.csprojAssemblyReference.cache

# Ignore UBT's log output files
/Engine/Programs/UnrealBuildTool/*.txt

# Ignore Python cached files
*.pyc

# Ignore JetBrain's IDE folders
.idea/
!.idea/runConfigurations
.gradle/

# Ignore autogenerated files from HoloLens WMRInterop
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/packages/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/Generated Files/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/ARM64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/ARM64/*

# Ignore files added by Finder on Mac
.DS_Store

# Ignore all .code-workspace files
*.code-workspace

here it goes

never had issues regarding readonly stuff etc. with this one

but the thing is, we use the suggested way of pushing engine + we use the horde buildserver and stuff, so we don't push what we shouldn't push, so the p4ignore might be incomplete

as the way to ingest engine from Github is:

• download from github
• run Setup.bat
• push everything excluding the .git folder, without using any ignores etc...

that will give you fresh copy of the engine code as close as possible to the one epic has on UDN/Perforce

then you just work like normal and only push files that you explicitly change, simple as that

[solid sage]

hmm, I guess I can try to push it through, but this results in 300k files being submitted, including some larger pdb files that take along time to upload. Is that really the MO? I wouldn't have thought all of those locally-built files were necessary...

Like, for example, you're including /Engine/Binaries/ThirdParty/CEF3/Linyux/libcef.so? That's over 1GB and is generated by the build and is one of many such files... is it just de rigeur to include all such files then?

the gitignore for engine source ignores the whole of AutomationTool too :/ https://github.com/EpicGames/UnrealEngine/blob/release/.gitignore#L206

[arctic hornet]

most of the thirdparty binaries are precompiled for ease of use

becuase compiling those can take... hours

my p4ignore is based on the one used internally at epic

so I trust it 🙂

and I do indeed have that specific file pushed to perforce (and so does epic)

so yeah, as I've said, empty folder, sync from git, run setup.bat, push it all (excluding .git) that way you won't have any issues with other build tools etc.

[solid sage]

Welp, ok, good thing it's a sunday and I can kick it off now 😅 thanks

[solid sage]

I guess a follow up question then, is about UGS/downstream users. Let's say there's an artist on the team who just wants to be able to launch the editor and our game, import and modify some art assets, check they look good and submit. If you're using precombiled binaries and UGS, do they still locally get a stream that also includes these giant files, or do you have a development stream with some filters?

[arctic hornet]

Ugs by default doesn't download everything and especially not when using PCBs, though you have to uncheck a lot of sync files, iirc you only leave "Content" enabled

As that will only sync uassets and uproject files and skips all source code etc.

And by sync filters I mean the ones that are in UGS directly, I'm currently walking through London so I can't send screenshots

[long mural]

can you integrate from one stream depot to another

say i have //UE5/Interkaos5.4

i want it to be part of //AresGames/Main/

@arctic hornetyou might know 🙂

[arctic hornet]

Sorry for the delay, I'm currently as close to touching grass as you can while on a roof of london building

[silver token]

Have you considered an import instead? It can help to isolate the code you have consider when doing stream merge and copy operations.

[arctic hornet]

But in general, yes you can integrate between streams

[long mural]

is it good practice to integrate or import

cause i import my plugins from other streams

[arctic hornet]

Well, depends

[long mural]

but not decided on the engine

[arctic hornet]

In general I integrate engine, simply because we do have single project

But you'd probably want to use import if you have custom engine shared by multiple projects

I.e. integrate from//UE5/Raw (direct unmodified engine from epic) to //UE5/MyEngine ... and then import that to all projects if you use separate streams

That way you can keep all the engine changes shared between projects

If that's not something you want, then integrate is for you

[long mural]

yeah

[arctic hornet]

Simply said

[long mural]

import is probably what i would do

as we have 2 projects on the go

[arctic hornet]

I'm still more in favor of integrates but that's simply because I'm old fashioned 🙂

• they're really fast to do if you know the proper cli commands

But hey, in the end it's your decision

Each of them has it's own pluses and minuses

[silver token]

This is what I do at home.
At work I’m taking it further and each project gets a child stream of the company engine (epic plus mods). That way project1 that is near shipping can easily decide to stop taking updates (for a little while) but project2 can take the newest engine immediately.

It also lets projects make project specific engine changes that may not be relevant to another project.

[long mural]

hmm

seems my imports are fucked

import+ .../ //UE5/InterKaos-5.5/...
import+ AresGame/Plugins/Game/KaosOnline/... //GamePlugins/KaosOnline/Main/...
import+ AresGame/Plugins/Game/KaosAbilitySystemCore/... //GamePlugins/KaosAbilitySystemCore/Main/...
import+ AresGame/Plugins/Game/KaosGameCore/... //GamePlugins/KaosGameCore/Main/...
import+ AresGame/Plugins/Game/KaosCamera/... //GamePlugins/KaosCamera/Main/...
import+ AresGame/Plugins/Game/KaosCore/... //GamePlugins/KaosCore/Main/...```

what did i do wrong?

it started importing every depot

any idea @silver token ?

i think its the engine import

yeah it is

[silver token]

Yeah, I needed to hope onto the work perforce to see how it was setup.

share BurbankGame/...
import ... //Burbank/Mid/Engine.Main/...
import Langserve/... //Burbank/Mid/Landgserve.Main/...
import+ BurbankGame/Plugins/ThirdParty/... //Burbank/Mid/Plugins.Main/ThirdParty/...
import+ BurbankGame/Plugins/Midsummer/... //Burbank/Mid/Plugins.Main/Midsummer/...

so the only real difference I see is the .../ vs ...

[long mural]

i had that originally

and didnt work

maybe the share line

[silver token]

maybe. That one probably depends on whether you're doing a native or foreign project. This one is native
My home ones are foreign

[long mural]

yeah that works

by fixing the share line

btw, is there a way to restrict artists from seeing specific files

like source code but still have the data so UGS works?

[arctic hornet]

I tend to do that through virtual streams that are locked at specific CL

Though as I've said, I prefer to use integrates since the files now go with projects so it works fine with multibranching etc.

[silver token]

You could make a virtual stream. We have it the other way around, a virtual stream for artists that adds in the art source.

[long mural]

virtual stream of the main stream?

[silver token]

yes

[arctic hornet]

Well, there are always permissions you can set... but if you're fine with them seeing the source code, you can just disable source code sync filter in UGS and that way it won't sync the code

[long mural]

cause we want everything to always push to main

[arctic hornet]

At epic everyone sees everything so there's not many things to block it

[long mural]

i don't want them to see certain code, and i want to restrict platform specific code

[arctic hornet]

Virtual streams are just virtual

So they push to main

[silver token]

But the virtual stream wouldn't allow for project specific engine changes. A project change to the engine would go to both because the virtual isn't a "real" stream

[arctic hornet]

Permissions are your friend

[long mural]

yeah but setting permissions, UGS breaks lol

i dont mind them seeing what files are there

just not to open them

[silver token]

Here's what I hope to get our work perforce to look like. Burbank is the only real project. Starships is a speculative project to show how the streams/branches work for multiple projects.

[long mural]

must be a way to do it

[arctic hornet]

It doesn't

This is what I use and works fine

[long mural]

o

ok

[arctic hornet]

You can probably just ignore some etc.. as it'd hard to navigate remote desktop fom phone 🙂

[long mural]

yh

my server going nuts

[arctic hornet]

Looks fine to me 🙂

[teal marsh]

Hello again, so the push finished and so I kicked off another horde build and it errorred in the same way; hundreds of errors of this form; the file to which access is denied is not covered by the p4ignore and is therefore in source control and therefore Read Only in horde's workspace after it syncs and cannot be written to

[hoary wren]

Love the htop color scheme, like a little Christmas

[teal marsh]

looks like maybe I also need to update the typemap

[orchid solstice]

does anyon know how to fix this when trying to pull?

[arctic hornet]

Looks to me more like you pushed something you were not supposed to, I.e. you compiled and ran unreal before pushing to perforce and it included some intermediate files and now they're locked so it can't write to them

(This message is for Ben, I was just underground and we finally got signal)

[arctic hornet]

Well, stupid question, but have you tried the steps outlined in the message?

[orchid solstice]

yeah and nothing worked

[noble reef]

I asked already but it's the last thing bugging me with Perforce/Rider. And I fix it sometimes, I just don't remember how since I'm always trying so many things.
It's about the "Modified without checkout" tab in the Perforce window of Rider.
Why Ô why it's always filling this tab with files that haven't been changed. It happens when I edit the read-only status of the project's folder. I usually do it in the windows folder explorer.
Does this mean that EVERY time I'll do that outside Rider, it'll fill again this tab?
How am I supposed to do it then? (Like I can't efficiently regenerate VS files from Rider)

[arctic hornet]

You should never modify the read-only flag manually, ever

Transient files should never be pushed to perforce in the first place and files that are in perforce should be checked out instead using p4, p4v, rider etc...

[noble reef]

Why though ?
I found these two options in my worskpace advanced settings, it works for the moment.

[noble reef]

Well that's what I'm trying to build, a "seamless" dev environment where I can submit classes and assets from their respective softare of choice

But one software always gets in the feet of another one

My bad, forgot the screeshot...

[arctic hornet]

You will always get everything simply because you are working in one workspace.

And you should never manually disable read only flag unless you have allwrite enabled (which I don't recommend personally for multitude or reasons I don't have time to go into right now) if you have allwrite then you should ideally remove readonly flag on every file

[noble reef]

(I was talking about the two checked options)

[arctic hornet]

But your setup right now will spell disaster when working with multiple people at once

Simply because clobber will overwrite any of your local changes etc.

[noble reef]

Yeah for the moment I'm on my own (and will remain for a long time)

Hmmmmm ok, yeah that's a nightmare fuel

[arctic hornet]

And in general, perforce is built around the flow of, checkout -> edit -> submit

[noble reef]

So concerning this checkout shit then, is there a way to "fix" it cause in the end it's only a matter of that.

[arctic hornet]

Checkout does remove the readonly flag, but also marks the file as checked out on the server, if the file has exclusive checkout (lime binary files) it will not allow others to check it out... and once you check it back in it locks the file again etc.

[noble reef]

Ok ok, thanks for the clarification about that.
But then, how do I prevent this panel from filling up whenever I need to re-build/regenerate my files from the outside of Rider? (which is mandatory since there's no valid way from doing it from Rider (am I wrong about that?))

[arctic hornet]

You just ignore it, simple as that

Or you add metric ton of ignores to your p4ignore file

[noble reef]

The thing is that they show up for my next submit in the edited files, I wouldn't mind otherwise

It's also never consistent on which files are included in it

[arctic hornet]

That's most likely the byproduct of the all write imo, since I've never had files automatically added unless I added them myself

[noble reef]

It's fucking weird that it acts like that

But thanks dude

I'll continues my search on this on my own

[arctic hornet]

Or missing ignores

[noble reef]

Can you check my file if you don't mind, I dont thik it does but I'm a beginner in this

(or maybe you can share your p4ignore file if you have one for game projects 🥺 )

[long mural]

that ignore looks weird

# Ignore root Visual Studio solution files. We do check in some sln files in subdirectories, so only ignore
# the ones found in the root.
/*.sln
/.p4sync.txt

# Ignore all Visual Studio temp files.
*.suo
*.opensdf
*.sdf
/Engine/DerivedDataCache/*
**/DerivedDataCache/Boot.ddc
**/DerivedDataCache/**/*.udd

# Ignore all Intermediate and Saved directories
*/Intermediate/*
*/Saved/*
# Ignore UBT's configuration.xml
Engine/Programs/UnrealBuildTool/*
*.uatbuildrecord
*.tmp

# Ignore built binaries and temporary build files
*/obj/*
*.csprojAssemblyReference.cache

# Ignore UBT's log output.. files
/Engine/Programs/UnrealBuildTool/*.txt

# Ignore Python cached files
*.pyc

# Ignore JetBrain's IDE folders
.idea/
!.idea/runConfigurations
.gradle/

# Ignore Samples, QAGame and EngineTest
/Samples/*
/QAGame/*
/EngineTest/*


# Ignore autogenerated files from HoloLens WMRInterop
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/packages/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/Generated Files/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/ARM64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/x64/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/ARM64/*

AresGame/Binaries/*
AresGame/Intermediates/*
AresGame/DerivedDataCache/*
.vs/*

# Ignore files added by Finder on Mac
.DS_Store

# Ignore all .code-workspace files
*.code-workspace

[noble reef]

Thanks a lot 👌

[long mural]

this is built cause we also have engine with project

per Epic best practices when using source engine

[dark cosmos]

will 4.27 work on Visual Studio 2022?

[arctic hornet]

nope

well, unless you patch Unreal Build Tool

and you might face some other C++ 20 related issues

[woven sluice]

Yeah it will

Not 426 though

[dark cosmos]

Thanks.

[arctic hornet]

iirc 4.27 never got the VS 2022 patch

like, it might compile in 2019 compatibilty mode, but I don't think it would pass necessary certifications especially if you're targeting consoles

since I presume it's some sort of pre-existing project

although 4.27 would have certification issues for XBX and PS5 anyways due to old-af compatible SDKs

[dark cosmos]

I don't have any intentions of releasing anything console specific, so that works for me anyways

[woven sluice]

Yeah console dev would mess that up

[dark cosmos]

I'm just tired of UE5's crap with packaging

[woven sluice]

?

[arctic hornet]

please elaborate

[dark cosmos]

UATHelper: Packaging (Windows): LogMaterial: Display: Missing cached shadermap for UDS_FTTTF in PCD3D_SM5, Low, SM5, Game (DDC key hash: 690c7560191dee162c397c8ba6a01357b8882f01), compiling.
UATHelper: Packaging (Windows): LogMaterial: Display: Missing cached shadermap for UDS_TTTTF in PCD3D_SM5, Low, SM5, Game (DDC key hash: 8dc4f8783b573fd83f863c809355c2e6e346f502), compiling.
UATHelper: Packaging (Windows): LogMaterial: Display: Missing cached shadermap for UDS_FFFTF in PCD3D_SM5, Low, SM5, Game (DDC key hash: bb5ded1af2037a2cd3a28d491239a38c0e8ed3db), compiling.
UATHelper: Packaging (Windows): LogMaterial: Display: Missing cached shadermap for UDS_FFFFF in PCD3D_SM5, Low, SM5, Game (DDC key hash: 7e6cb816e0e4c0eddd058ec6f4ffba55724bf1cb), compiling.
UATHelper: Packaging (Windows): LogCook: Display: Cooked packages 6222 Packages Remain 680 Total 6902
UATHelper: Packaging (Windows): LogWindows: Error: begin: stack for UAT
UATHelper: Packaging (Windows): LogWindows: Error: === Critical error: ===
UATHelper: Packaging (Windows): LogWindows: Error:
UATHelper: Packaging (Windows): LogWindows: Error: Fatal error!
UATHelper: Packaging (Windows): LogWindows: Error:
UATHelper: Packaging (Windows): LogWindows: Error: Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0xffffffffffffffff
UATHelper: Packaging (Windows): LogWindows: Error:
UATHelper: Packaging (Windows): LogWindows: Error: [Callstack] 0x00007ffc8cd88f74 UnrealEditor-CoreUObject.dll!UnknownFunction []
UATHelper: Packaging (Windows): LogWindows: Error: [Callstack] 0x00007ffc8ce1380f UnrealEditor-

never had this happen in UE4

[arctic hornet]

uhhh, hard to say what's going on, please install debuggign symbols if you're using Epic Games Launcher edition

as there's possibility of corrupted asset, but as it's "UnknownFunction" in the core of the engine it's hard to say without symbols which would provide function names etc.

[dark cosmos]

Alright.

[dark cosmos]

Seems to be compiling just fine, it asks if you want to automatically upgrade the project .net to 4.28 or whatever or downgrade the editor version

so i chose upgrade

[arctic hornet]

there shouldnt be a need to compile anything... just install debugging symbols from epic game store - if you are using version from egs that is

and if you are not, then you should already have debugging symbols (hence why I kinda expect youto use the EGS version)

[orchid solstice]

someone please?

[dark cosmos]

I'm installing the debugging symbols and compiling a copy of 4.27 just incase.

[arctic hornet]

👍

[dark cosmos]

looks like it's out of Flinker Fluid 🤷‍♂️

[arctic hornet]

ok, so, this issue either a corrupted blueprint referencing either invalid or non-existent asset... or just a blueprint referencing corrupted asset

without adding some breakpoints to get the value of ObjectPathName in BlueprintSupport.cpp:1712 it's hard to see which asset is borked

but it's getting to a territory which is out of scope for this channel + I'm about to sleep...

[rustic canyon]

I added a few personal files to the ignore list as we kept Overwriting each other's builds and we had to rebuild projects each time we merged with the mainline.

[teal marsh]

Ok, I'm trying this now... I get a bunch of errors of the form:

 The file named 'Engine\Source\ThirdParty\DotNetZip\Examples\C#\ZLIB\test_flush_sync.cs' contains wildcards [@#%*].
    Can't add filenames with wildcards [@#%*] in them.

What do you do about these?

[arctic hornet]

you could do -i iirc which ignores the wildcards

or, you could just ignore this specific files, since you're highly unlikely to run this specific test suite

[teal marsh]

okidoki, thanks... Do you happen to have an example p4 typemap to hand by the way?

[arctic hornet]

there's one somewhere in Epic's documentation and I'm using that one as there's nothing special about it

[teal marsh]

ok, I have the same then

[arctic hornet]

just regular defaults + .uasset and .umap .uexp and .ubulk being binary files iirc

[old bear]

Hey ya'll I'm collaborating with someone for the first time on my project and was just curious what people would recommend to use for source control. I'm a programmer and the person will be doing level design. I have a fairly big project 60 GBs total as I have quite a few asset packs. Was looking at Git at first but it seems their file size might be way too small and I'm not sure if Git LFS is like super slow for level designers or anything 🙂 any help is very much appreciated

[arctic hornet]

I personally recommend perforce (helix core) it's fully free until t users, although you have to self-host (I.e. get a server/VPS) unreal works almost seamlessly with perforce since it's also used by Epic Games directly... if you'd use "source build" (I.e. taking engine source code from github rather than using pre-built from Epic Games Store) you could also expand your workflow with things like Horde and Unreal Game Sync which makes stuff much more convenient to use in general, but these two tools only work with perforce... and... "source build" and not the Epic Games store one

But yeah, perforce itself is relatively simple to set up, might be little hard to wrap your head around at the beginning if you're used to git, but it works great (and it's also used by any large game studio I know, so you'll know how to use industry standard tool if you ever decide to join corporate)

[old bear]

Ohhh okay, I'll use Perforce then! Probably set up a server with google cloud then?

Yeah for collaboration stuff I used to use git a lot (I used to/still do make Minecraft mods so there its kinda the dominant thing)

[arctic hornet]

Yeah, git is really useful if your code is mostly... well... code 🙂 meanwhile unreal is really binary heavy

And git (even with lfs) tends to bog down

And regarding providers... any cloud or even your local VPS provider will work, I recommend using Linux VM, with SSD storage... and installing perforce on that

[vague whale]

hello, i am looking to move my p4 server, i've got a new vm with a different provider and i want to move certain repos, then close the old server, i can't find good info on it, can anyone point me in the right direction?

[arctic hornet]

There is the easy way and then there is the hard way 🙂

[vague whale]

i like the easy way 😄

[arctic hornet]

Once my laptop starts up I'll expand on that 🙂

[vague whale]

thanking you kindly!

[arctic hornet]

ok, laptop's up... so yeah... the easy way is... to just copy all the files from whereever your p4 server is pointing to, to the new server using programs like rsync or even just scp/sftp etc... and then just launch the server in new location

that way it will be 1:1 of your old server including users, changelists etc...

the "hard" way is to use software like P4Transfer, where you spin up new server and then just "replicate" all the changes one by one using that

[vague whale]

oh i only wanted to move select depots as i have a lot of cruft, i could delete the repos i guess

[arctic hornet]

and then there's the "mid" way, where you use processes for backup and restore.... where on the old server you make a checkpoint, copy that checkpoint to new server and then restore it on the new server, and then copy over the bulk data

[arctic hornet]

ah, the thing with perforce is that it has 2 storages, one for metadata (the database) and the bulk storage (versioned files themself)

so you could just copy over the whole database, and then only copy over the bulk data of whatever you want

and after you're done copying you could obliterate the depots/streams you did not copy over files for, thus nuking them

p4 stores the bulk data in humanly readable file structure which represents your stream structure, so it's easy to only copy what you need

Perforce server requirements (recommended by epic, for best compatibility):

• Non-Unicode
• Case-Insensitive

Execute commands to set basic security settings:

p4 configure set security=3
p4 configure set dm.user.setinitialpasswd=0
p4 configure set dm.user.resetpassword=1
p4 configure set dm.user.noautocreate=2
p4 configure set dm.info.hide=1
p4 configure set run.users.authorize=1
p4 configure set dm.keys.hide=2
p4 configure set defaultChangeType=restricted
p4 configure set dm.password.minlength=8
p4 configure set net.autotune=0
p4 configure set net.parallel.max=48
p4 configure set net.parallel.batch=32
p4 configure set net.parallel.threads=8
p4 configure set filesys.bufsize=2M
p4 configure set net.tcpsize=2M
p4 configure set net.parallel.submit.threads=8
p4 configure set net.parallel.shelve.threads=8
p4 configure set minClient=2024.1
p4 configure set minClientMessage="Please upgrade to 2024.1 or higher"

feel free to tweak the threads for parallel submit and shelve based on your VM

and, here's the typemap, again, as recommended by Epic games:

    # Perforce File Type Mapping Specifications.
    #
    #  TypeMap:             a list of filetype mappings; one per line.
    #                       Each line has two elements:
    #
    #                       Filetype: The filetype to use on 'p4 add'.
    #
    #                       Path:     File pattern which will use this filetype.
    #
    # See 'p4 help typemap' for more information.
 
    TypeMap:
                    binary+w //....exe
                    binary+w //....dll
                    binary+w //....lib
                    binary+w //....app
                    binary+w //....dylib
                    binary+w //....stub
                    binary+w //....ipa
                    binary //....bmp
                    text //....ini
                    text //....config
                    text //....cpp
                    text //....h
                    text //....c
                    text //....cs
                    text //....m
                    text //....mm
                    text //....py
                    binary+l //....uasset
                    binary+l //....umap
                    binary+l //....upk
                    binary+l //....udk
                    binary+l //....ubulk

This should be enough for you combined with some internet googling regarding "how to setup perforce server" etc... to make yourself a perforce server

[vague whale]

oh i was stupid an enabled unicode

[arctic hornet]

This was mostly for @old bear 🙂

[old bear]

Oh thank you!

[arctic hornet]

then you're pretty much stuck with it, unicode-ness can't be changed afterwards iirc...

well, you could write a program that will manually transfer stuff from one server to the other while disregarding the unicode-ness but that's the "hard" way I originally described

[vague whale]

right i will just delete and start again

[ocean anvil]

i would add the minimum version required command as well.

(adjust the version to appropriate client version ofcourse)

p4 configure set minClient=2019.1
p4 configure set minClientMessage="Please upgrade to 2019.1 or higher"```
https://www.perforce.com/manuals/p4sag/Content/P4SAG/superuser.basic.minimum_version.html

[arctic hornet]

Good to know, thanks, will add it to my notes for next time 😄

[vague whale]

thank you adding to my config notes!

just a quick question, is the latest ubuntu LTS supported? should i use 20.04?

ok it seems like yes 🙂

[teal marsh]

Ok, I:

Obliterated my stream to start from scratch
got the engine source
Ran setup.bat
Added all
Submitted
Ran generate project files
Added all
Submitted
Opened ue5.sln in visual studio 2022
Ran Development Editor on UE5 project

And got:

C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(4890,5): error MSB3021: Unable to copy file "obj\Development\net6.0\EpicGames.UHT.xml" to "bin\Development\net6.0\EpicGames.UHT.xml". Access to the path 'bin\Development\net6.0\EpicGames.UHT.xml' is denied.

Which is somewhat back to where I was before

[arctic hornet]

Ah, yeah, your issue is the second "Add all"

Because during project regeneration it builds Unreal Header tool etc...

So, stuff until first submit was correct

You then have to regenerate the project and only add stuff from the project folder that is not Intermediate, Saved, Build or Binaries folder

(More or less just a source and content folders)

[teal marsh]

Forgive my naivete but isn't that the point of a robust p4ignore? That I should be able to add all and have p4ignore elide the stuff made during project generation etc.?

Or is it just accepted that we only mark for add/reconcile offline work on the project folder specifically?

[arctic hornet]

It is indeed a responsibility of robust p4ignore, so you most likely have stuff missing from it etc.. I'm leaving hotel in 15 minutes and need ro finish packing as I'm flying back home today, so I won't be of much more help than this

[shadow flint]

Does anyone have a solution to working on Foliage in Unreal with Perforce? Multiple users?

[arctic hornet]

There is sadly none, regardless source control used... foliage tool is softly deprecated in favor of PCGs and hasn't been updated much since UE3 days

And foliage tool even barely works with world partition as is for example.

So yeah, as with any source control, I'd recommend OFPA (one file per actor) and PCGs rather than foliage tool

[teal marsh]

I just don\t see what in this p4ignore would stop Engine/Source/Programs/UnrealBuildTool/bin/... from being marked for add

[arctic hornet]

As landmass and foliage tools predate all the modern approaches for multiuser editing and thus kinda work/not work well

[arctic hornet]

Well, it's entirely possible that such folder is missing from there as this is a file provided and used by epic themselves, so who knows

[teal marsh]

Sorry, I just thought you said this was the file you were using with Horde? So is it somehow the case that it's not including that folder for you, or are you just avoiding reconciling Engine, or is it something else, like your p4ignore has something this message doesn't?

[arctic hornet]

Yes, I'm using it with horde etc... but it'd been originally provided by epic... and horse's buildgraphs only do what you tell it to, which usually includes stuff like "Build executable to staging directory and then zip all files from there" and unless the repository is used for incremental builds it will get fully cleaned up after that operation is done

[teal marsh]

Yeah, still implies reconcile offline work or mark for add from your local workspace would break the setup though, right?

[arctic hornet]

possible, althouh in the seven + years we've been running this p4ignore we've never had single incident like this happen

as people usually only run reconcile in edge cases or on specific folders... and I've almost never had to manually mark anything for add and if I had I knew what I was doing... and our artists only do stuff in unreal, so they did not have to do this either

hence probably a reason why we never noticed (or anyone else)

since people who can break it like this usually know what they're doing, and people that don't, won't do this kind of add/reconcile

[teal marsh]

Hm, how do you handle engine upgrades?

[arctic hornet]

we have //UE5/Main ingest branch which contains clean version from UDN
we then make temporary dev stream //ProjectName/Dev-Release-EngineReleaseNumberHere
we merge in the project files from //ProjectName/Main
we merge in the //UE5/Main
we auto-resolve without merging
we manually resolve any other changes, stomping any binary changes

[teal marsh]

hm, so files that are new then end up added to mainline without having gone through an add/reconcile on the overall engine folder, right...

If it's indeed the case that the p4ignore doesn't ignore files that break the project if they get submitted, I would challenge the statement "people who can break it like this usually know what they're doing"... you said yourself a robust p4ignore would allow the add to work... I'm over here playing by the rules, so to speak, right? Just saying "oh, but never reconcile offline work or add on any of the /Engine hierarchy in case you include some files not covered by p4ignore that break everything"... that's... bizarre, to me

[arctic hornet]

I'm not saying the p4ignore that I provided is the best or fully accompassing, all I'm saying is that it's being used internally at Epic and we've used it since UE 4 days too and never had such issue... so all I'm saying is... it's enough for day-to-day work where people know what they're doing

if you need full "idiot-proof" protection, then you'll have to expand your p4ignore to cover those cases I guess

[pulsar parcel]

this is the tried and tested way, and the way Epic presented at Unreal Fest

hopefully Ari's talk is uploaded soon

[arctic hornet]

Ari's talk has been uploaded since before the start

https://dev.epicgames.com/community/learning/tutorials/8JYW/setting-up-an-unreal-engine-studio-the-epic-way

[pulsar parcel]

yeah, this is the slides

[arctic hornet]

it's not a video, but it's an article containing the whole presentation

but otherwise it's everything

[pulsar parcel]

Ari did a great job of presenting the info though

[teal marsh]

I'd still challenge terms like "idiot" or "people who know what they're doing". Relying on a robust p4ignore is very much by the book development, IMO... Those geniuses who are clearly more knowledgeable than me, should just... write a robust p4ignore, right? Then it fixes it for them, who are so smart enough as to know never to add/reconcile inside /Engine/..., and for us idiots who assume the p4ignore will ignore files that need to remain local...

[arctic hornet]

sorry, english is not my main langauge, I should've used "fool-proof"

[arctic hornet]

it was fun talk indeed, I'm glad I could've contributed to the presentation personally too 🙂

[pulsar parcel]

figuring out how to deploy Horde must be a lesson in pain though

[arctic hornet]

these days it's easy, it was much harder in 2021

[teal marsh]

I'd counter with maybe "conform to the intended use pattern of perforce VC..." 😅

[pulsar parcel]

Our studio is currently in the process of switching to P4 from Git. Getting everyone on board with the idea has been like pulling teeth

[arctic hornet]

if you have any questions feel free to ask even in DMs, as I've been running full stack that even folks from epic said is "Epic but on small scale"

[pulsar parcel]

At least I got a chance to speak to Ari and he linked me to a previous talk he did that explains why you shouldn't use Git

[arctic hornet]

as we're running everything from horde, perforce, jira, confluence, robomerge, ugs etc... etc... 🙂

[pulsar parcel]

Yeah I'm used to that, minus Horde

Mostly used Jenkins, current studio uses TC

[arctic hornet]

I mean, anything works as long as it works 🙂

horde is just kinda a first class citizen in UE ecosystem

especially as it tends to replace a lot of deprecated stuff like UGS metadata server (and one day it plans to include Jupiter/DDC too afaik)

[pulsar parcel]

well yeah I can imagine it's better to just use build graphs directly

[arctic hornet]

but hey, that's more of a discussion #automation than here 😉

[pulsar parcel]

at previous studios, we just used the standalone metadata server, even used a forked port of it to .NET Core

easier than doing full Horde

but in the talk, it's nice to see Horde to replace things with AutoSDK and precompiled binaries so they don't have to live on the P4 server anymore

[arctic hornet]

I don't remember autosdk being able to live on horde (and I've sat on the presentation ), might check that 😄

as we use AutoSDK from perforce directly for now... and honestly same is true for PCBs since right now UGS still has some permission related issues that are scheduled to be fixed after the company wide vacation is over

[pulsar parcel]

might have misremembered, but I seem to vaguely remember that also being a part of UGS and being able to install required SDKs easily

[arctic hornet]

AutoSDK is a thing, but iirc it can "only" load data from either perforce, google drive, s3 and UNC network share

[pulsar parcel]

ah it was probably S3/network then

[arctic hornet]

horde can also use it and so can turnkey

[long mural]

hey, as this is p4 question:

    D:\UE5_MAIN\Engine\Source\ThirdParty\Intel\TBB\IntelTBB-2019u8\lib\Win64\vc14\tbb.lib - ignored file can't be added.
    no files opened

i have no p4ignore

[arctic hornet]

we've had some talks about it yesterday in #automation you can find turnkey config and example of the autosdk structure etc there

[arctic hornet]

what does the p4 ignores xxx command tell you? as that might help us to narrow it down

where xxx is the path to the file

[wind gazelle]

is it possible to have git remote pointing to a file path? like [remotes] shared = file:///X/Repositories/Repo.git

[arctic hornet]

there is surprisingly many ignores

[long mural]

i have no p4ignore file

in that dir

[arctic hornet]

the easiest way is to run p4 add -I to ignore the ignore

[long mural]

its clean UE5 from epic p4

[arctic hornet]

hum hum, weird, I'm about to head to the airport, so if you're fine with waiting for later tonight or tomorrow, we can go through it from scratch as it might be easier than me just sitting on a phone at the hotel lobby ngl

[long mural]

P4CLIENT=Daniel_Laptop_UE5 (set)
P4EDITOR=C:\WINDOWS\system32\Notepad.exe (set)
P4IGNORE=.p4ignore;.p4ignore.txt;.gitignore (set)```

alright it was the .gitignore in here

from another project

but where is it pulling that from

[arctic hornet]

not sure, there's plenty of .gitignore files there, but iirc they're only applied to child folders relative to their location

[long mural]

yeah i subcontracted for a while to a company and they had gitignore (don't ask why)

so my ignore line was borked

[arctic hornet]

and I also generally discourage use of gitignores with p4ignores because of the slightly different syntax 😄

[ocean anvil]

but why gitignore in ur p4ignore?

[arctic hornet]

because it can cause weirdness such as this

[long mural]

read what i said @ocean anvil

wasn't my choice lol

[arctic hornet]

some companies are mess, as always 🙃

[ocean anvil]

ohhhh wait they... wth

[long mural]

really should have made them a p4ignore, but wasn't in my contract haha

[ocean anvil]

i thought you meant they used git so you had to use gitignore.... i see what you mean now.

[long mural]

nah gitignore does kinda work as seen here

but it has some difference

[ocean anvil]

technically there would have to be a gitignore file in one of these folders then.

[long mural]

yeah

it was

now to fix the engine

ffs, all cause of the dumb gitiignore thing

[ocean anvil]

huh, i guess that subcontractor just used git in the past and instead of properly switching over to p4's workflow they just changed their git files xD

[long mural]

reckon a ton of files got missed

thats exactly it i think

[ocean anvil]

considering you also have git attribs

[long mural]

this is epic engine

it comes with all this

cause epic pushes to github

i assume they want to restrict certain binaries from github

[ocean anvil]

ohhhhh nvm then 😅 i never used the source engine so ignore me on that

[arctic hornet]

Well those gitignores are there because those are third-party libraries that usually come from github etc.

And epic does as little changes as possible, if any... other than adding build scripts for UE etc.

[long mural]

yh

[long mural]

got myself into a pickle

and i don't know how it actually happened

User name: Daniel
Client name: UE5Merge
Client host: ns3101456
Client root: /home/ubuntu/UE5Merge
Client stream: //UE5/Epic-Pristine
Current directory: /home/ubuntu/UE5Merge``` this is my current client setup

i did p4 reconcile

ended up with like

        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_saddle_edgeonly_level4.obj     # edit
        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_triangle_edgecorner_level0.obj # edit
        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_triangle_edgecorner_level1.obj # edit
        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_triangle_edgecorner_level2.obj # edit
        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_triangle_edgecorner_level3.obj # edit
        //UE5/InterKaos-5.5/Engine/Source/ThirdParty/OpenSubdiv/OpenSubdiv-3.6.0/regression/hbr_regression/baseline/loop_triangle_edgecorner_level4.obj # edit```

in my changelist and i can not revert nor delete it

//UE5/Epic-Pristine/Engine/Source/Runtime/MassEntity/Public/MassExecutor.h#1 - has been moved, not reverted```

[vague whale]

urg moving depots is difficult

there has got to be a way to move just a single depot,

p4 backup depot some_file, scp it, p4 restore some_file

[arctic hornet]

Not really, you backup the metadata, restore those on another server and copy over the bulk data, that's how backup procedure is

Perforce is quite old piece of software, so it's kinda archaic in certain cases

[long mural]

anyone seen this

Translation of file content failed near line 1 file /home/ubuntu/UE5Merge/Engine/Source/ThirdParty/nvTextureTools/nvTextureTools-2.0.6/src/project/vc8/nvcompress/nvcompress.rc
Translation of file content failed near line 1 file /home/ubuntu/UE5Merge/Engine/Source/ThirdParty/nvTextureTools/nvTextureTools-2.0.6/src/project/vc8/nvddsinfo/nvddsinfo.rc
Translation of file content failed near line 1 file /home/ubuntu/UE5Merge/Engine/Source/ThirdParty/nvTextureTools/nvTextureTools-2.0.6/src/project/vc8/nvdecompress/nvdecompress.rc
Translation of file content failed near line 1 file /home/ubuntu/UE5Merge/Engine/Source/ThirdParty/nvTextureTools/nvTextureTools-2.0.6/src/project/vc8/nvimgdiff/nvimgdiff.rc
Translation of file content failed near line 1 file /home/ubuntu/UE5Merge/Engine/Source/ThirdParty/nvTextureTools/nvTextureTools-2.0.6/src/project/vc8/nvzoom/nvzoom.rc```

[arctic hornet]

ah, yeah... type detection gone rogue

you'll have to manually find these files in p4v and change their type from utf8,utf16 or whatever to ascii or text... don't remember what exactly

if you're ingesting from UDN then check for stuff on their perforce server

that's all I can suggest

[long mural]

yh

will take a look

i just reverted them for now

utf16+l

mine is just tf16

[arctic hornet]

it should be text+l

at least one of them

and at least per my database based on UDN 🙂

[long mural]

this what i see on epics

my p4

my typemap fucked?

[arctic hornet]

hum, they changed it recently then, since my database is from May and all of those files are text+l

[long mural]

how did you sync

p4transfer?

[arctic hornet]

custom software

[long mural]

oh

here im doing it manually by hand

lol

[arctic hornet]

but this database is just a scrape of UDN

I built map of every file and it's type (mostly for this exact purpose of type fixing)

[long mural]

i tried to use p4transfer, but failed

[arctic hornet]

it excludes NDA'd platforms like PS5 and XSX since I don't have access to those (because I have UDN access under my own name and I don't have NDAs with Sony... and the studio I work for doesn't have UDN access but has NDAs )

Feel free to search in this database, it's just sqlite DB

[long mural]

oh i have all the platforms

how did you get UDN under your own name?

paid for it?

[arctic hornet]

paid for it, indeed

and it's currently not possible anymore iirc

[long mural]

:/

[arctic hornet]

as they changed policies few months ago

[long mural]

when i last enquired the price was ridiculous

[arctic hornet]

you now need at least 100 seats where one is 1.5k

a year

but in the past you could've gotten just a single seat, for the same yearly price

[long mural]

so 150k a year?

[arctic hornet]

and compared to the usefulness it provides it's "nothing" (for one seat)

[long mural]

thats mental

[arctic hornet]

yeah, it is

[long mural]

i wouldnt mind if UDN was actually sometimes helpful

i have waited months for help

[arctic hornet]

I've had quite fast response, at least when I asked

but I mean, I've got fast turnaround on everything, if I do pull request it's usually merged in less than 8 working hours

[long mural]

its normally min of 1 week

before a UDN reply

[arctic hornet]

which isn't UDN related at all, since that's still done through github etc.

[long mural]

i have PR's still just sitting there

they don't care about them most of the time

even a simple PR like this

https://github.com/EpicGames/UnrealEngine/pull/12035

which fixes an annoying buf

bug

https://github.com/EpicGames/UnrealEngine/pull/11368 like this one, to make GameplayTargeting better

[arctic hornet]

https://github.com/EpicGames/UnrealEngine/pull/12025 meanwhile, this is my last PR, submitted at June 19th 9:03 pm (GMT+2) and merged at June 20th 1:12 am (GMT +2)

though this one was a tiny one

[long mural]

yeah when your CCing people

it normally is quicker lol

[arctic hornet]

I mean, that's why I do that

[long mural]

lol

[arctic hornet]

if I know who's responsible for that specific component I CC them and it gets done quite fast

another alternative way is to open UDN post about "merging this" and it'll get triaged and sent to responsible people faster than them randomly going through PRs one day

just leaving it hang usually results in it being unmerged forever

either way, back to the topic at hand... I'll be able to send you my migrator tool thingy tomorrow, it's almost midnight and I've still got ton of unpacking to do since I jsut came home 2 hours ago

it's a C# (.net core) app that's built for merging from UDN stream A to Your stream B every so often, it does all the reconcile magic etc. so everything is preserved, files that are removed on UDN get removed locally etc. etc... if you're interested

and more or less you just specify this + servers and it does it's magic... the ZIPs are there for the restricted platforms since I have to fetch them annoyingly through organization portal

[long mural]

btw

the incremental builds

do they get pushed to p4 still for UGS?

[arctic hornet]

if you mean PCBs (precompiled binaries) then it depends on what buildgraph you call

[long mural]

yeah tool sounds good, manual is annoying me a bit

[arctic hornet]

epic does provide both in the buildgraph iirc...

[long mural]

well i assume its just doing the standard p4 push

[arctic hornet]

PCBs in perforce are:

• build
• strip PDBs
• zip it
• shove it to p4

as you most likely know

and iirc for horde it just skips the zip + p4 push and shoves it into the horde storage

but iirc there's outstanding PR/bug (even on UDN) regarding OIDC permissions in UGS which prevents fetching PCBs from Horde

but don't quote me on that since I'm tracking like 60 PRs right now so I could be confused

[long mural]

yeah

                {
                        "id": "incremental-build",
                        "name": "Incremental Build",
                        "description": "Performs an incremental editor compile, and uploads the produced binaries to Horde for syncing with UnrealGameSync.",
                        "initialAgentType": "Win64",
                        "showUgsAlerts": true,
                        "showUgsBadges": true,
                        "arguments": [
                                "-Script=Engine/Build/Graph/Examples/BuildEditorAndTools.xml",
                                "-Target=Submit To Perforce For UGS",
                                "-set:UProjectPath=$(ProjectPath)",
                                "-set:EditorTarget=$(EditorTarget)",
                                "-set:PreferredAgent=IncrementalWin64;CompileWin64;Win64",
                                "-set:ArchiveStream=//$(Project)/Editor-Binaries"
                        ],
                        "workflowId": "ares-main-workflow",
                        "schedule": {
                                "enabled": true,
                                "requireSubmittedChange": true,
                                "commits": [
                                        "code"
                                ],
                                "patterns": [
                                        {
                                                "interval": "1m"
                                        }
                                ]
                        }
                },``` im just using this

for incremental

but... where is it actually pushing too, what depot on P4?

oh

Ares/Editor-Binaries

[arctic hornet]

horde tends to create it's own workspaces if that's what you mean.... and the ArchiveStream is where it's pushing to

whic hin this case would be //ProjectName/Editor-Binaries

and it pushes like this

[long mural]

[//Ares/Main/AresGame/Ares.uproject]
ZippedBinariesPath = //Ares/Editor-Binaries/++AresGame+Main-AresEditor.zip

is what i have in UnrealGameSync.ini

[arctic hornet]

and to let UGS know about where the PCBs are, you have to make UnrealGameSync.ini in your ProjectName/Build folder

and yeah... it fetches from there, using some black magic sauce so it knows which PCBs to use etc.

[long mural]

yeah

[arctic hornet]

I still recommend this way simple because... it works...

but with horde (and UGS connected to Horde) you could also use Horde artifacts for this purpose, I've just never tried it + the bug I've mentioned etc. prevented me from using them i nthe past

so it's just easier to use PCBs with P4 for now

at leats im o

[long mural]

🤞 Horde does a build

i had a fucked up engine lol

[arctic hornet]

fun...

[arctic hornet]

but oh well, I'm going to bed now, so just ping me tomorrow when you're available so I can get you up to speed with the migrator tool etc.

[long mural]

yeah will do if i remember lol

[arctic hornet]

🤞 it's gonna look like this soon:

[long mural]

doubt it haha, one of the plugins is now borked

[vague whale]

morning, 🙂 i thought i would try and use p4 zip/unzip to migrate a repo (sadly the easy way won't work i have too much rubbish on the source server)

i can see the files on the server when i look at the depot but when i try and sync them it says it can't find them

did i miss something

ah the unzip hasn't loaded the files

but i have a 100g .zip file so hmm

[long mural]

tried p4transfer?

https://github.com/perforce/p4transfer

@vague whale

[vague whale]

no i didn't but i am sure i did it like this last time

well its unziped all the directories by no files

[vague whale]

maybe the easiest thing is just create new blank depot and forget the history, not like i need the history

[arctic hornet]

the official backup prodecure from perforce team is as follows:

• create checkpoint (using the p4 checkpoint command)
• backup all your bulk data

If we extrapolate it to migration it would be like this:

• create checkpoint
• restore from checkpoint on the new server (it restores all the metadata, history etc...)
• using rsync or something else, just copy over the bulk data to the new server
• done

p4transfer (which I recommended originally when you asked) does go change by change and syncs the data one by one etc... so it's not 1:1 copy, but it's as close as you can get.... meanwhile doing backup and restore the way above will make 1:1 carbon copy including all the IDs, times and whatnot

there's nothing much I can recommend that this, as the migration process cannot be more straightforward than this... well... excluding some hypothetical built-in solution for "p4 migrate" etc...

[vague whale]

ah i thought p4transfer was the hard way 🙂

its strange as i managed to migrate depots a year ago with no problem

well it was easier to just push up the code from my box, i suck

[arctic hornet]

I mean, it's "hard" in a sense that it takes a long time etc... meanwhile just copying stuff over and restoring from checkpoint can be done in 5 minutes and then you just wait for rsync or whatever to be done which can range from few minutes to few hours/days depending on the amount of data

I consider that the easy way

especially as p4transfer is a "copy" rather than "migrate" etc... so ti's not 1:1

[vague whale]

i read it as p4transfer is hard so avoid 😉

i transfered last year and it was "easy"

and i hadn't even heard of p4transfer

[arctic hornet]

iirc the only limitation of p4transfer is that it can't copy from unicode to non-unicode server and vice versa, but I might've imagined that or they might've fixed it since, last time I checked was couple of years ago

[vague whale]

i probably should purge my history off every 3 months, i don't think i will need it

[arctic hornet]

never say never

[vague whale]

ok i can't think of any reason why i will need it

[arctic hornet]

I've never purged my history in the 15 years and 800k CLs on my private server... and boi was I glad few times where I had to go and touch some ancient files

[vague whale]

i think i will snapshot the code for a release build

[restive mica]

Super odd, works on everyone else's machine

[ornate summit]

Hello, my team is using unreal 4.27.2, but now we finally decided to upgrade to 5.3.2, in terms of project migration, I already know what to do, but are there any additional considerations in terms of Git/github?

[frigid helm]

Hi, is there a way to have unreal ignore certain folders from the Default Uncontrolled Changelist? sort of like a p4ignore for the engine (the folder is already under p4ignore, but that changelist is bloated with stuff I dont want to see)

[arctic hornet]

Uncontrolled changelist is just a feature that's agnostic to the source control and it's just unreal keeping track about some things, you can either safely ignore it... or iirc. There's a checkbox somewhere in the settings to disable Uncontrolled changelists altogether, don't remember exactly where but if you search through project and editor settings for "Uncontrolled" you'll most likely find it

[arctic hornet]

Same requirements for version control apply to both ue 4 and ue 5 if that's what you're asking

[proven violet]

Can anyone give advice on fixing conflicting bps?

[pulsar parcel]

They're binary. You pick one or the other, and then manually reperform the change from the one you didn't pick.

[proven violet]

I pick one then it wont open and crashes the editor

my partner did some changes to character and added sockets to the skeleton and messed stuff up. Could I duplicate them and rename so doesnt overwrite mine and copy his code into my character?

[frigid helm]

The thing is I want to use the feature. It's just that I have a single folder I want to keep that has like 80,000 files in it, and those are always in the uncontrolled change list making the feature useless due to pollution

[arctic hornet]

Ah, I presume you're using engine from epic games launcher rather than the source build?

[frigid helm]

Nah I do have a source build

[arctic hornet]

hmm, weird, usually 80k+ files happen when you don't have all assets pushed to perforce/git etc... which tends to be the case with EGS version because engine is outside the project folder

*version controlled folder

[frigid helm]

I should clarify, these aren't random files I don't know about, I know what they are and they are intentionally there, I just don't want those files in version control

[arctic hornet]

epic recommended way of structuring is like this and if you pushed your whole engine and all the assets etc.. to version control it shouldn't be there

[arctic hornet]

ah... nevermind then, ignore my posts 🙃

but yeah, I'm not aware of any "filters" for that

as uncontrolled changelists are a way for unreal to keep track of everything that is not controled by version control

[frigid helm]

Fair enough, maybe I can modify source to just hardcode ignore that path when searching, just a very hacky thing I'd rather not do though

The feature itself is neat, definitely want to keep it, just would like to also have local only files I experiment with that I never commit to p4 and don't pollute the change list

[arctic hornet]

just make p4 changelists and never submit them you can delete them afterwards etc.

yes, it will make a gap in the numbering, but that's how it works... and gaps can happen for multiple reasons

the benefit of it being in regular p4 changelsits is the fact that you can shelve it and unshelve it later, or on another pc etc.

[frigid helm]

Guess that's an option I can try actually! Thanks 🙂

[arctic hornet]

I hope it works for you... but yeah, if you create them, they will get assigned a number, it will be tracked by the perforce etc... but until you submit them they will be only on your computer (you could imagine it sortof like staging) - though one file can only be in one changelist at once... and once you submit that changelist it will get new nuber so it's latest as of time you pushed it etc... etc... 🙂

[frigid helm]

Yea all that's fine, I don't mind numbering gaps, as long as the files stay local

[arctic hornet]

local only until you hit submit 🙂

or until you shelve it, as at that moment the files are uploaded to the server, but are in separate temporary storage and not pushed as numbered changelist... though I presume you know how shelves work in perforce so no need for me to explain

[valid vine]

Ok - I am going to start using p4, I have to. I have a growing fear every day my git lfs project will need me to revert and I just have a bad feeling about git with lfs binary data and how it stores .uassets with only references to the real binary assets so sometimes when you revert you still have no assets.

Long story short can someone please TLDR how perforce is different? (i dont mean file locking or feature wise), I mean its not centralized or hosted is it? I need something like azure to host my p4 server right?

So what is the typical run down or process for setting one of these up? I am so unfathomably lost

[valid vine]

I suppose it does not have to be azure right? Just a VM (or even remote local machine but I dont have that option) that can host a p4 server and I can ssh in and control administratively.

Can I have some advice on how to even go about all of this. I feel like I am drowning for something so simple in concept

• Host files on a server
• Locally clone and modify files
• Push changes and sync changes when needed

Thats it but holy shit

[woven sluice]

perforce has a very steep learning curve to administrate, overall it's more complex than git by quite a bit, all you can do is keep watching videos about it and stumble forward for a bit (and asking questions to clarify your understanding). but yes, you can run a perforce server on anything more substantial than a raspberry pi (you can't run it on a raspberry pi, though you used to be able to)

if i was working solo i probably would run a local perforce server and set up some sort of weekly offsite rotating backup

working in a small team i rent a linux VPS near the rest of the team and run it on that instead so they get better service

having said all this i don't know if i would be so afraid of git LFS... i barely use it but i mean it is pretty tried and tested. if you're otherwise happy with it IDK if i'd switch or not. unless you want to just get some experience with perforce too so you know them both

[valid vine]

Yeah I've tried detaching the local head to a specific commit (just so I could launch and check something in editor) it had all my assets replaced by just strings in the .uasset that points to the real ID or whatever in the LFS commit.

Long story short I also do Intend to learn all of this. Can I ask for some pointer or places you'd recommend starting? For example I was looking into perforce and that honestly didnt seem to scare me (although I didnt dive too deep in admin stuff, just how to setup streams and workspaces, sync and merge etc) but it did make me realize I know nothing about cloud computing or VPS or servers in general.

I will start there I think, after I learn the basics of creating a usable hosting server the rest will make a lot more sense to me. How did you start with all of that stuff and am I correct in assuming its usually ssh into the vps to administrate it?

I appreciate the reply too, this seems to be such a hard topic to self learn in its kinda crazy

[woven sluice]

depends if you want to get into linux or stay windows... learning how to administrate a linux server from scratch is another beast

and yeah, the cheapest hosting option is typically command line only linux

so... attractive lol

[valid vine]

linux since its so popular I think

the os shouldnt play too much into something like a p4 server right? its all just setup and let it exist afterwards

[woven sluice]

i use gandi.net for my VPS but i think there are slightly cheaper and better options out nowadays (i used to shill gandi but i don't anymore because they sold out and the new owners have done a couple of meh things)

have to learn how to SSH in and operate it via PuTTY

all the different linux distros are similar but different, speaking as someone who very much does not care to deal with linux, it is infuriating lol

but yeah, it will work with any major distro

[valid vine]

Do they come with the OS installed or do I need to do all of that? its just like a clean fresh computer to begin right?

[woven sluice]

typically a VPS will start with the OS cloned and the machine up and running

[valid vine]

sweet

Do you ever run into issues with bandwidth or permanent storage for it all?

[woven sluice]

bandwidth no - every supplier is different, many offer unlimited bandwidth
VPS's have expandable disks - just keep an eye on disk usage each month and expand it + pay more as needed

[valid vine]

looking into these now

[valid vine]

Okay, I know these might seem like silly simple questions but you are making me feel a lot more peace at mind

[woven sluice]

i want to say something like "linode" was popular in here a year or so ago but search around for different VPS providers and opinions of them before you decide to try one

[valid vine]

Yeah, I think I want to try one for free if some random provider offers that, just for learning, then I need to work out how to transfer files between..

(this is just to learn the basics before trying to setup a p4 server)

[woven sluice]

typically they bill you X per month but it's actually charged hourly so once you pick a provider you want to try you just go and start messing around. spawn a VPS, log into it, break it, delete it, make another one

and you don't need high end specs for a few people working. single core, low RAM will work fine

[valid vine]

well gandi seems cheap and you have experience so I might just try that for a mess around (or even for real after if I learn it)

[woven sluice]

so far my only beef with gandi is i can't seem to get any more than 100 Mb/s download from my server but for my team with our 20 GB project it's fine so i don't care

[valid vine]

I'm excited dude! Thank you for your time

Hmm, It claims hosting in france which makes me assume you are near that or the UK, I dont think that would be a great solution for my geo location. I will continue to look, I have heard good things about digitalocean, do you have any opinion on them?

omfg - your profile picture... I am dumb

[woven sluice]

nope i'm in canada. my team is all in germany, portugal and the like though 😄

everyone talks about digitalocean, it's probably good

(i also wanted to make sure our server was completely safe from a potential evil meese uprising, so not in canada)

[valid vine]

Alright, wish me luck!

[quaint obsidian]

unless your LFS filter is not properly configured, this shouldn't happen, but i get your feeling about it

[teal bone]

Honestly it was really easy to set up a server and get people set up to use it. There's a really simple starter guide on their website. Didn't cover advanced features but, honestly, nobody in here is going to be starting a 2000 user server with 100 streams... unless they already know how to do it.

[long mural]

https://contabo.com/en/vps/cloud-vps-3/?image=ubuntu.267&qty=1&contract=1&storage-type=vps-3-1-2-tb-ssd we use this for our source control

1.2tb SSD (not nvme)

get 32tb bandwidth on a 600mpbs port

[arctic hornet]

perforce does scale really well and once you're done with the initial config you don't have to do anything at all

the most cumbersome thing imo is the permission UI in p4admin

[teal bone]

That is pretty bad.

It's like straight outta the 90s.

[arctic hornet]

I mean, it's been the same since I started using p4 15 years ago

so I'm pretty sure it's been the same since the beginning

but in general all the techniques you'd use for "small" server also work and scale for large server

[teal bone]

Yeah.

If they were to revamp their UI (from scratch) it'd be a really lovely bit of software.

[arctic hornet]

I've talked to few senior devs from perforce relatively recently, and there are some ideas to do it... mostly to get rid of QT dependency (because of it they can't make the p4v source code available)

so there are some ideas to rewrite it without that dependency and release it as open-source... but as of now there's no official go-ahead just ideas floating around

because p4api has matured a lot and they could easily build it on top of that library, so it would mean that anything p4v can do can also be done with p4api and p4 executable (which is the case even now... but yeah, the client would be open-source so you could modify it more deeply etc.)

[teal bone]

The one thing that really annoys me about the client, and maybe I just can't find the option, is that there's no timestamps in the log window.

[arctic hornet]

you mean this?

[teal bone]

Maybe?

[arctic hornet]

it doesn't give date, but it gives time, which is usually plenty

[teal bone]

Yeah my p4v log looks nothing like that.

[arctic hornet]

so yeah, right click on that and "View Timestamp"

I also switched to monospace font long time ago (Consolas in this case) otherwise no customization done to it

[teal bone]

I don't just mean the timestamp. 😛

Okay, so it looks a little like that.

[teal bone]

Thanks. I never thought to right click on the text. I tried looking through the options and right clicking the tab...

I may be special.

[arctic hornet]

happens 🙂

[teal bone]

I always seem to look in the wrong place in the client to find settings.

[frigid onyx]

[EDIT: Solved #source-control message]

Hi, I have a git-lfs problem with Unreal and would like some help/suggestions.

Recently we added the "ProjectBorealis/UEGitPlugin" to the project but it didn't quite behave as expected. We weren't able to checkin already submitted files and they remain locked which defeats the whole purpose of it. So, we decided to revert the plugin and continue normally as we were doing.

Reverting the plugin didn't remove git-lfs files from the server and kept the files locked. After a lot of googling, the unlocking was done using git-lfs unlock --id XXXX" where ids are listed using git-lfs locks. I also re-wrote the histroy using BFG to remove all traces for the .gitattributes and the plugin but the files are still on the git-lfs whenever I list the files using git-lfs ls-files -a I can see them still avaliable.

Following some random stackoverflow and other blogs, I did what they say but the files are always there.

The main problem is that the project isn't working anymore, even after a clean pull (deleting everything and getting latest)

The .uasset files that were locked before are downloaded as 1Kb in size and they corrupted (not visible in unreal) but in the GitHub website, they are with their correct sizes and downloading them manually fix the issue.

Oh, and I read we can't remove git-lfs from the server and this has to be requested from the github support which I opened ticket for.

So, is there a way to un-corrupt these files or download them correctly without lfs instead of manually doing it for every file?

Other than this, should we switch to perforce instead? I read that perforce is free upto 5 people (and 20 workspace) but hosting is not. What hosting service would you recommend?

[valid vine]

why is that so cheap?

[valid vine]

do me a favor, read this and then check your uasset and then tell me its not that, I pray for you brother #source-control message

[frigid onyx]

sounds like it 😦 no recovery from it?

[valid vine]

no way, @woven sluice I predicted the future, just not mine 😦 rip this man

[valid vine]

Join me on the path to learning cloud computing, sever hosting and perforce! I am having a blast

Installed ubuntu locally on a VM just so I can be comfortable

[frigid onyx]

I was planning to do that but I didn't want to get forced to mid project

Where are you hosting btw?

[valid vine]

was going to do digital ocean (still am right now because they have really cheap testing prices) but I am thinking about this one since its crazy cheap #source-control message

[valid vine]

in terms of recovery, id try do a hard rest of your local head

my issue happened when I did a local detach on a commit that was really early

[frigid onyx]

That's a bit expensive for me. Especially that the project size is around 1.2 GB and we are 3 people so I will look for other options. Thanks 🙏

[frigid onyx]

I did this but they are always down to the 1kb files

[valid vine]

rip, this lady laura helped me with it but they arent here anymore and deleted all history so I cant help

[frigid onyx]

No worries, you already did help! I will update here what we did later in case someone faced the same thing in the future

[arctic hornet]

If you need any perforce related help... I can help 🙂 been running perforce personally for 15+ years and I can help with quite a lot of the things necessary for proper deployment, especially reagarding proper setup so unreal (and it's surrounding tools) work as painlessly as they can

(and I've also helped to set up more perforce servers than I can count over the years)

[valid vine]

wow thats awesome! I will for sure take you up on that, thank you @arctic hornet

[arctic hornet]

If you need something just ping me as I'm quite busy otherwise and might miss your messages otherwise 🙂 I'll get to it when I'm available...

[valid vine]

will do, thanks again

[frigid onyx]

Everything makes kind of small sense now 😅
Let's start with the 1kb files. These were there because they are just pointers to the LFS storage. That's why they were corrupted in Unreal. They are 1kb when downloaded and in github browser they are full size with a note saying "Stored in Git LFS"

So, if the new clean pull has only the pointers, you need to fill up these pointers by first git lfs fetch -all then git lfs pull to fill up these pointers with their data.

Then, to remove them from LFS, the solution is as follows:

• Use git lfs ls-files to list the local pointers in the project
• Move them outside the project or just exclude them from the repo using git rm --cached <filename> (There was a link that automated this process for all files [1])
• PUSH! Since these files are tracked by GitLFS, they will be deleted from LFS remotely
• Stop tracking files in .gitattribute (by removing the lines containing "filter=lfs")
• PUSH! This will let LFS know that it doesn't need to track anymore
• Re-add the files that were removed and finally PUSH again

Finally, open a support ticket to ask for removal of LFS

Useful links:
[1] Remove and Add pointers: https://github.com/git-lfs/git-lfs/issues/3026#issuecomment-451598434
[2] Support ticket: https://stackoverflow.com/a/76944147

[valid vine]

Ah big brain, very nice! Glad you got it working, I still think I am done with git for gamedev

[frigid onyx]

true, it's good only for small solo projects

[outer kiln]

never had any luck with it don't recall why atm it was two monts ago,svn issues over versions,so perforce only option really, atm anyway

[wind gazelle]

just in case if someone else get a problem git connections failing with SEC_E_ILLEGAL_MESSAGE (i've battled with it for a week) my solution was
(although im using regular gitforwindows and not github desktop)
https://github.com/desktop/desktop/issues/12743

git config --global  http.sslBackend openssl
git config --global http.schannelCheckRevoke false

[proven violet]

can fixing redirectors make the engine crash on another persons pc?

[formal path]

Hello, i am working on a project for a few months and push it to github. I just got a laptop for mobility but when i clone my project there all the code that already works on my desktop will throw errors everywhere.

I am using the github’s recommended .gitignore file. What could be the issue?

[silver token]

I would expect the engine to be pretty good about not doing that, but your project code might not be as safe.
But it's possible.

[silver token]

You'll have to post the actual compiler errors to get any sort of useful advice.

[formal path]

they are usually nullptr errors.

[proven violet]

just trying to figure out why project crashes when friend opens but doesnt for me

[silver token]

Oh, my mistake. Still, there should be log messages or a callstack that you can post.
There's no one fix to all nullptr errors.

[proven violet]

i noticed when i went to update redirectors it mentions it might mess something up in source control

[silver token]

Things can get messed up if your friend was adding a new reference to the old asset. Depends on what source control you're using and how long you waited between creating and deleting the redirector.

[formal path]

I have been trying for a couple hours now to get rid of the nullptr and make sure everything is working safely. But eventually i realized it’s more than nullptr errors.

Gamemode won’t spawn default pawns.
Even if i write a logic to spawn them PlayerController->Possess won’t possess.

The parts of the project that i never touched (engine code) doesn’t world as expected.

What could cause this?

[silver token]

The only thing that comes to mind (and would be relevant for this channel) would be files on your original machine that you haven’t actually pushed. It happens more frequently than you might think.
If you’ve made sure you have all the same files on both machines, you may need to ask in one of the other channels.

[formal path]

i am not sure. i am clonning a second on my desktop to try if it works

[fringe socket]

I'm trying to set up a project using git in such a way that folks who aren't changing cpp code won't have to recompile changes from other people. I think this is doable by commiting the contents of the Binaries folder whenever making code changes, is that accurate?
When trying to test this I'm also running into issues pushing due to the size of the binaries, is git lfs the fix for that?

[arctic hornet]

git-lfs is a need in this case, although if your team is up to 5 people large I'd recommend using perforce instead since it handles stuff much better (+ all the tooling from epic is built around perforce)

I personally don't recommend pushing binaries to the tree directly as it will get messy real fast, the way epic does it using Unreal Game Sync (perforce only) is to have precompiled binaries zipped in separate stream and then once you sync to given version it automatically downloads and extracts those binaries, making stuff much cleaner... but as you're using git there's probably no other option for you right now than to slap it into the tree directly

[fringe socket]

Yeah, I've got >5 people and much more git experience than perforce experience, although it sounds like that might be a good thing to learn. Does the messiness of pushing binaries directly come from things mysteriously breaking if anyone doesn't push binaries when changing code, or are there other issues?

[arctic hornet]

mostly just the need to keep the binaries updated and in general keeping track about stuff

• if you're going to have multiple branches you need to make sure that after merging you rebuild your binaries etc.

and debug binaries can also get quite large

for example just the binaries for my game with debug symbols and stuff are ~25 GB

and add additional 600 MB for the game specific binaries

• ideally so you don't get weird compile errors or have to recompile whole editor etc... you want to also submit the Intermediate folders and keep them in sync which is A LOT of gigs

Although... this is only the case if you've got engine source code directly in git, if you are using engine from Epic Games store, stuff gets quite a bit easier and smaller by a lot

as you only have to deal with the GameName/Binaries and binaries for each plugin

[fringe socket]

Yeah, I was going to ask about that. I'm not compiling from source right now, so hopefully it should get that huge. I hadn't thought about merging requiring a recompile, that's a bit stickier

[arctic hornet]

i.e. like this (this is perforce screenshot, but stuff also applies to git itself) - this project uses engine and editor from epic games store

[arctic hornet]

so yeah, if you're going to stick to mostly one branch (which I'd recommend especially with git... and use epic games store engine) you'd be fine with pushing binaries to git directly

although if using git (and git-lfs) I'd recommend setting up the git lfs source control so you can get file locking etc. because without it you're going to be in a hell of merge conflicts 😄

[fringe socket]

Haha yeah, I think that'll be my project for the afternoon. Thanks for the info, that's really helpful

[proven violet]

For some reason my branches the character is corrupted for my friend but for me it's fine? Is this fixable?

[woven sluice]

you would never submit the Intermediate folders, that'd be crazy, there's nothing useful in there

[at] regoradin, i don't think i would ever recommend pushing binaries to git directly... unless you have a way to automatically purge old revisions of just those files like you can do in perforce easily... it's just going to balloon into so much disk usage over time

[zealous lance]

Ooh I didn’t know perforce could automatically purge old binaries

[valid vine]

what distro do you guys normally use to host your servers? Im at a toss between ubuntu or Alpine because I hear thats the dominant server OS. Does it really matter for source control hosting

[arctic hornet]

I personally host a lot of stuff on CentOS, but Ubuntu server is generally decent option that is widely supported, both from the standpoint of percompiled binaries and tutorial availability, for example with Alpine you'd have to manually install perforce as officially perforce provides apt and yum "installers" and then raw tar.gz file

[woven sluice]

agree with klukule. I used Ubuntu myself, as it seemed like it's very well rounded. Red Hat or CentOS seemed like they might be aimed more at corporate style environments? i don't really know though, i don't pretend to be a linux expert at all

[woven sluice]

when anyone on our team pushes a code file change (or also wwise file change), we get a discord message "build in progress", CI attempts to build and submit the binaries, we get a discord message "build success/fail" ... artists just don't run a pull while a build is running (or a build is failing), pretty easy. nuisance to set up but we're still avoiding a source engine / UGS as long as we can 😄

[long mural]

my swarm is broken and i cant work out why

If you have changed your configuration, delete your Swarm config cache to rebuild it, see Swarm config cache file delete. If this API does not work, you will need to manually remove the config cache file with the command:

rm -f /opt/perforce/swarm/data/cache/module-config-cache.php
Be aware, if your Helix Server is running at security level 3 or above, you must use a valid long lived ticket instead of a password Swarm configuration file password.

If your ticket has expired, obtain a new ticket by using the following command on your Swarm instance:

p4 login -p
If this is a new Swarm install, please see the links below and ensure you have configured Swarm correctly. Install and upgrade Swarm documentation; in particular:

Runtime dependencies
Installation
PHP configuration
Swarm configuration
If this does not solve your Swarm issue, contact Support Perforce Support portal.```

[arctic hornet]

CentOS is a fork of RHEL (ReadHat Enterprise Linux - well, these days it's little more muddy, but this was generally true in the past) - which as the name suggests is generally used for heavy duty production use where you want stability etc... since RHEL packages are updated less often than ubuntu and are generally more stable + the ABI (Application Binary Interface - simply said system libraries) are also more stable

Meanwhile Ubuntu, which is a fork of Debian is more user focused and user-friendly as it originally started as "easy to use linux distro compared to the others" - for example it was one of the first if not first, that had easy to use setup you're familiar with today, i.e. "next, next, next, auto settings, next, next, install" rather than "here's a /dev/sda device, I have to make / partition, /swap partition, manually configure everything etc...") ... and because it's more user friendly, it's also easier to find more-user friendly tutorials for it...

Thought that all was like 10 years ago, so some of this info could be stale, but in general, Debian and it's forks (including Ubuntu) generally are more user-friendly and have more "getting started" tutorials written for them that do not require advanced knowledge of linux

[arctic hornet]

it's been quite some time since I had to debug swarm last time, but iirc check the apache error logs and/or swarm also has separate log file iirc... those should point you in general direction

but, in general, unless you borked something:

• make sure your ticket is cross-machine
• make sure your ticket is not expired and your "swarm" user is in correct groups where ticket doesn't expire and that swarm has enough permissions

[valid vine]

alright well thats awesome because I was a little curious of the install process but with apt support it couldnt be easier right? Doing installs for things that require a site or GUI for installation worries me if I ever need to deal with that but I guess most things linux based for servers need cl interface support due to nature of headless.

Thanks for the breakdown and history too!

[arctic hornet]

most cloud providers come with pre-made images for both centos and ubuntu... and installation of both is super easy these days, as you just provide username, password, timezone, let it do it's magic with disk and done... gone are the nightmares of the past

but honestly, ubuntu server (GUI-Less version, made for server use - you mostly access through SSH) is what I'd recommend to get started

[zealous lance]

Is your hosting machine for SC also dealing with CI and discord API status updates? That's a really cool idea

[woven sluice]

Linux VPS runs perforce + overly complicated discord bot, the CI builder is a virtual machine in my house

The discord bot is a c# app that talks back and forth

[haughty yoke]

For a reasonable sized open worl game (roughly skyrim world sized) what amout of drive space would be best for backups

Not actual dev just older build backups and basic source control

Primarily be a disk drive as those kinda cost less

[arctic hornet]

Honestly, depends... because disk size obviously grows with the number of revisions to each file, amount of metadata, whether you also store raw assets for things etc... etc... 🙂

For example the project Titan art jam which is 8x8km piece of world full assets (although no audio, and little to no animations, no raw assets stored etc.) After 10 weeks was about 120 GB of total used storage on the perforce server, and now I don't remember whether the synced workspace had 30 or 60 gigs.

Since titan was art jam there was a lot of inefficiencies etc...

But TLDR a 1TB of dast SSD based storage should be more than enough, unless you're going AAA dev, at which point I'd rather get few more TBs just in case

[valid vine]

Could you run me through the general process @arctic hornet if you arent too busy?

I think I am ready to spin up a droplet an try to setup perforce. Do I start by downloading the client first or doing the server first?

also how does one go about downloading the client lol

[ocean anvil]

u need both, but probably best to do the server first on either a second machine or a cloud host

[valid vine]

I see here that digital ocean has an image with perforce but its on rocky 8, if I were to just not use an image but just a fresh OS is it too much more effort to just install it with apt an do whatever setup needed?

[valid vine]

Yeah cloud host is the plan, I am doing it right as we speak if you dont mind walking me through it?

[ocean anvil]

if u're handy with linux its not that much more effort.

its a couple of commands to install etc. all the actual set up remains the same

[valid vine]

I'm not terrible, if I could find some docs i will be able to handle it easy, ill go look now actually

[ocean anvil]

best to look up some tutorials/guides on it tbh, they'll tell you more than i can in a reasonable amount of time plus you get visual examples

[valid vine]

https://www.perforce.com/manuals/p4sag/Content/P4SAG/install.linux.packages.install.html

[ocean anvil]

i also don't run my perforce server in the cloud or on linux so i'm probably not the best to walk you through that process step by step

[valid vine]

Yeah, I am not too worried. I think the scariest part is spinning up an VPS for the first time and deleting it and being sure I know it wont charge then I will have peace of mind lol

[ocean anvil]

the AWS dashboard is pretty decent at tracking and showing you your usage/incurred costs iirc. though its been a couple years since i used it.

idk about digital ocean

[valid vine]

Yeah I am using DO since its a lot cheaper and I have heard good things

[arctic hornet]

Hey there, if you're still interested I can guide you through the basic setup 🙂

[valid vine]

yeah I still am getting around to it. Assume I have nothing lol like no admin or p4 acc (there is no sign up?)

I am about to spin up a server in a minute but just doing something

[arctic hornet]

Ok... I'll send you some of my personal notes that I'm prepping for future publication, they're rough but they'll get you up and running regarding the Linux server... then we'll get on the side of uploading content etc... but that will depend on your setup (I.e. whether you just want a project in perforce, or if you also want other stuff like engine source code etc.. etc...)

I'm going to make me a coffee and something for breakfast (though it's 12:30 pm ) as I just woke up 🙃

[valid vine]

haha its like that sometimes 😆

I really appreciate that and would be grateful to be able to use it. Yes I have an engine install but thats through git and less of an issue for me as I dont really do too much in that repo, but I would love some tips or even just like if I knew I could later add something like that and not worry about it to begin with

[arctic hornet]

while I'm making breakfast here's a questionnaire that will guide some decisions later down the line:

• Engine source code in perforce or from Epic Games Launcher (will guide how we set up the streams and depots, so you can later easil update the engine version etc...)
• Planning to use Horde or something like that? (will not affect us right now, but later down the line I'd show you how to offload SSL to a broker and have internal non-ssl endpoint which can be used by horde agents so you don't have to deal with .p4trust files and other ssl magic)
• Planning to give a shot to Unreal Game Sync? (if yes, we will have to include engine source code in the project no matter what as UGS expects it)

[valid vine]

No intentions of UGS. I think I will just worry about my project, its kind of a lot all at once I am ready to start slow 😅

I currently just SSH'd into my first server and sudo apt update/upgraded everything, now I am going to try follow p4 guide on apt install

[arctic hornet]

well, I'm going to slow you down a little, because default configuration for perforce is... well... not exactly fortunate... it tends to deploy case-sensitive and unicode server, meanwhile for unreal you want case-insensitive and non-unicode server 😄

[valid vine]

okay good to know, so how far do you think I should go

[arctic hornet]

😄

it's an export of my private notes, follow that and you should be golden... well, it goes as far as spinning up a server and basic security + perf config

once you're done with that we can start doing all the magic regarding streams, unreal engine itself etc....

[valid vine]

life saver

I'll get to reading now

[arctic hornet]

all teh steps here are perfectly doable directly through ssh 🙂

although if you're not familiar with editting files in VI, you can skip the typemap for now and we'll get to it later once you can do it in notepad on windows 🙂

[valid vine]

do you mind if I first do an empty stream with just a hello world program to begin with? I ask because the server I picked was the cheapest shittst one just for first testing, i dont intend to use it for VC

[arctic hornet]

yeah, sure that's absolutely fine, I'll still give you the exact pointers like we'd do if we were setting up whole unreal project, so you can just replicate it later down the line

[valid vine]

lol okay pls dont flame but already at step one I am having issues I think

I am trying to run wget -qO - https://package.perforce.com/perforce.pubkey | gpg --dearmor | sudo tee /usr/share/keyrings/perforce.gpg

as per the link in step one says

the terminal is returning some cryptic garbage when running it

[arctic hornet]

you wanna hop into VC? might be faster debugging that way... though not sure how Voice channels work here on the server

[valid vine]

ls -lah returns normal

[arctic hornet]

oh ,yeah, that's normal

[valid vine]

oh it is?

[arctic hornet]

don't ask me why, but some "smart" people thought it was better than the old way of using apt-key command which didn't do this, but got deprecated

it's just a by product of the fact that you download the key, do some gpg magic on it and then write it to the keyring 🙃

[valid vine]

okay great to know ahah I will continue forward 😅 Any more issues an I might consider taking you up on that VC if you dont mind

[arctic hornet]

I don't mind 🙂

and, if your apt update command after doing the step 1 fetches stuff just fine then it worked without issues

(i.e. if the step 3 on the perforce site - step 1 in my guide runs fine, you're golden)

[valid vine]

Add Perforce's repository to your APT configuration

Create a file called /etc/apt/sources.list.d/perforce.list with the following line:

deb [signed-by=/usr/share/keyrings/perforce.gpg] https://package.perforce.com/apt/{os}  {distro}  release

Where {os} is replaced with ubuntu, and {distro} is replaced by either precise, trusty, xenial, bionic, focal, or jammy. Supported versions can be found at https://package.perforce.com/apt/ubuntu/dists/

ubuntu I know because thats just the OS but idk the servers distro

[arctic hornet]

what version of ubuntu are you on?

22.04 or 24.04?

22.04 is jammy, 24.04 is noble

and 20.04 is focal 🙂

[valid vine]

ok noble then, thank you again for the 100th time ahha

[ocean anvil]

u should really add minimal client version to your basic security set up.

[valid vine]

ok vc if you dont mind lol any channel ahah

[arctic hornet]

forgot to update the configs after last time 🙃

[arctic hornet]

I guess time to slack 🙂 gonna be in Slackers in a sec

[ocean anvil]

https://www.perforce.com/manuals/p4sag/Content/P4SAG/superuser.basic.minimum_version.html

https://www.cvedetails.com/vulnerability-list/vendor_id-5860/product_id-164609/Perforce-Helix-Core.html

[arctic hornet]

Thanks

[long mural]

Was apache upgrade breaking things. :/

All fixed now

[arctic hornet]

ah, yeah, I tend to forget that swarm is written in PHP

[valid vine]

@arctic hornet is the man! ty again

[valid vine]

What is everyones choice for backup protection? I see that VPS's offer backups but I have also heard they are a ripoff and you could do it better with a simple cloud storage plan (like one drive or whatever, kinda irrelevant) and then use something like Cron to schedule daily cloud uploads

IE something simple like - https://dev.to/itsbetma/automated-backups-with-cron-and-rclone-3do4

[neat grotto]

I just use my provider's automatic backup feature. It's a couple more dollars per month. I think it's worth the time saved.

[woven sluice]

our indie backup is "F it, the project is regularly updated on 4 or 5 machines around the globe, if the perforce server gets hit by a plane we'll just scrap our history and remake it from whoever committed last" [our actual indie backup is: TODO: implement backup]

[valid vine]

Yeah its just putting all my eggs in one basket that I might be afraid of lol

[valid vine]

not wrong, I will have it locally too

[arctic hornet]

Had this approach in the past and then one day the raid controller on the machine died and we lost 10k+ commits... since then we've got second machine that is a forwarding replica and just mirrors the main server just in case.

But in general, if your cloud provider has backup solution for the block volumes, use that, simple as that

Perforce has 2 (technically 3,if you include logs) types of data that can be backed up separately.

First is that archives (I.e. this bulk versioned data) - if you loose this part Perforce will still show history, you just won't be able to download anything historic

Second is the database which is the b-tree and metadata in general - if you loose this it's gone gone 🙂

So just run daily backups on both if possible and you're golden (for on-premisse hosting it would work by rsyncing the archives to another location, and then creating a checkpoint and backing that one)

[valid vine]

Is block storage for something like source control fine just HDD storage? or do you think the NVMe will matter?

The price difference is pretty substantial so I think I am going to wait for a response before I decide

I feel like HDD would be fine

[proven violet]

should I never fix redirectors when using source control with other users or should I fix them when I move something etc.?

[valid vine]

@arctic hornet 👀 (sorry I am just waiting to mount drive before setting up p4 for real, if I could just get your thoughts on the storage question above)

[arctic hornet]

well, for small team (and solo dev) HDD is sufficient enough, although some I/O heavy operations like initial engine submit or large merges might take little bit more time

but for larger team I recommend going SSD because the performance is much better overall

[valid vine]

okay but for a single person or two, there shouldnt really be a difference besides once off big submits like the initial one or whatever, in general I wouldnt notice?

its just I can basically 3-4x my storage by going HDD over NVMe

[arctic hornet]

It should work just fine 🤞 I've moved away from HDDs like 8 years ago so I don't have any relevant data about hdd performance with unreal

[valid vine]

better than nothing, thanks.

Have a wonderful day

[proven violet]

my question really is it safe to fix redirectors when working with a team and if so when wouldn't it be safe?

[valid vine]

@pulsar parcel would know

[pulsar parcel]

bit of a random ping

I wouldn't really say it's ever unsafe, but the commandlet will fail if it resaves anything that is checked out (if exclusive checkout is on)

in the past, we ran it semi-regularly over the weekend

[proven violet]

i just learned about the command but we added a bunch of assets to our project and I want to organize everything we just added so I'm just wondering if that would cause issues with github and how to prevent issues ig

So I'm guessing it will be fine if I just fix any redirectors when I'm done and then commit and push

[arctic hornet]

In general it's safe to fix redirectors, but you have to make sure that no assets referencing the redirectors are being checked out by others (or in case of git, you just have to pray :D) if you fix and push everything there should be no issue with anything, unless you're referencing the old location from C++ code (for example hardcoded path inside constructor etc...) or from INI files as those locations don't get updated

[proven violet]

okay thanks, no c++ so good there.

so yeah as long as they dont push any changes before I push mine its good then

[arctic hornet]

but as I've said, make sure to submit EVERYTHING that got changed after fixing redirectors, that's the most important part

if you miss one asset, unreal will most likely be fine, but that asset will have broken references which will either crash your editor/game or just get set to empty/none the next time you save the asset etc 🙂

so as long as you submit everything you're fine

[proven violet]

shouldnt it automatically submit everything that changes?

[arctic hornet]

it should, but I'm just saying to make sure 🙂

sometimes stuff can get weird

[proven violet]

ah ok ok

[arctic hornet]

better be safe, then suddenly notice half your rocks are missing in a level because you forgot to submit it

the worst thing about this is... it will look fine on your computer, but it will look wrong for others since you've got correctly fixed assets locally 🙃

[proven violet]

we dont really have any real levels at this point just testing levels. One of the things I might reorganize is a marketplace level.

[arctic hornet]

just giving an example 🙂 as it happened to our team couple of times and was annoying to fix - well, mostly annoying manual work

[proven violet]

So when I commit it will show redirectors in the list, or in this case shouldnt if im fixing them all up etc

[arctic hornet]

redirector is just a file, similar to the lnk files on windows... but instead of it containing the asset itself it's like "Hey, the princess is in another castle... Castle address: XXXX"

so fixing redirectors will cause few files to be removed, unless you fix the redirectors while keeping them in place

[long mural]

we still use HDD backups, we have 3 point of data, Machine, Local backup and Remote Backup. Our backups are all HDD mechanical drives never had any ssues with our projects for backups. Our machine hosting the source control, runs on NVMe ofc

still dont trust SSDs fully with backups

[valid vine]

Trying to setup perforce right now and I keep getting this error

The filesystem 'P4ROOT' has only 1.9G free, but the server configuration requires at least 2G available. when I try to run the p4.sh install script with all the arguments, providing the -r flag to the location of my block storage

sudo /opt/perforce/sbin/configure-helix-p4d.sh -n -p ssl:0.0.0.0:1666 -r /dev/mnt/blockstorage/p4/master -u super -P "SomePassword" --case 1 master

i know how it looks “you’re out of space” but im not though

unless i am passing the pass wrong?

I was, it didnt need /dev

[mystic lagoon]

Why won't this stuff show in the GitLab? I commited to main and pubished to origin.

[valid vine]

I am having trouble now signing into the p4admin client, I keep getting

[teal bone]

Is your server running?

[mystic lagoon]

?

Did I do it right?

[arctic hornet]

are you putting the ssl: there too etc... but otherwise what Daekesh said 🙂

[mystic lagoon]

Enumerating objects: 25591, done.
Counting objects: 100% (25591/25591), done.
Delta compression using up to 12 threads
Compressing objects: 100% (18577/18577), done.
Writing objects: 100% (25590/25590), 46.56 GiB | 23.43 MiB/s, done.
Total 25590 (delta 6769), reused 25357 (delta 6536)
error: RPC failed; HTTP 413 curl 22 The requested URL returned error: 413
fatal: the remote end hung up unexpectedly
fatal: the remote end hung up unexpectedly
Everything up-to-date

When I try to publish to origin.

[arctic hornet]

are you self-hosting your git server?

if so the issue is with your web server (the proxy) the files you're trying to submit are too large (hence the HTTP status code 413 Content Too Large)

and if you are not, then you probably need to switch from HTTP to SSH

[mystic lagoon]

How do I work around?

[arctic hornet]

depends on if you're self-hosting or using some provider... if you're self-hosting it then depends on what web-server you are using etc... if you want to use http based upload that is... in generla I do recommend using ssh for large files... but even better I recommend setting up and using git-lfs for binary files because otherwise git will be really slow after some time

but as there is not a specific answer I do recommend using google as you can find your answer there more easily based on all the variables

[mystic lagoon]

How do I check if git-lfs is congifured?

[arctic hornet]

I've personally never used git-lfs so I don't know as I use perforce for all my version control needs, as I've said, try googling, you'll find your answer 🙂

or maybe someone else here who's got git experience can help.

[valid vine]

I suspect I just did something wrong in the setup, it was a fresh VPS so I just nuked it then (I just got up) and I will try again 🦾

Everything did seem setup though and p4 info was showing it like normal but anyway, onto attempt 2

[valid vine]

Nope,I still am getting the issue.. fuck

I am getting this when running p4 -p 1666 info

root@PerforceServer:~# p4 -p 1666 info
Perforce client error:
        Failed client connect, server using SSL.
Client must add SSL protocol prefix to P4PORT.

[valid vine]

Actually p4 -p ssl:1666 info works (from the server) but I still cannot connect on admin client

woiwowowowoow

sudo ufw allow 1666/tcp
sudo ufw reload

This solved my issue... This should have been opened in the setup script right?

Just making sure, this is typical and safe yeah? 22 is encrypted ssh and tcp is also fine (although both ipv6 and 4 might be unneeded)

sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
1666/tcp                   ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
1666/tcp (v6)              ALLOW       Anywhere (v6)

[arctic hornet]

This is fine, although I usually tend to hide ssh behind internal network and just vpn in first or something like that, but for you this is absolutely fine

[valid vine]

Can I define a folder inside of type map? Basically I want the -S functionality of limiting how many revisions I store of a given directory folder (a build folder in this case)

[arctic hornet]

you can, the typemap follows the usual perforce schema... so //depot/... is valid or even //depot/....txt (notice the 4 dots... three of them are the recursive wildcard and then it's the .txt extension) etc.

and in typemap you can either specify full type, for example binary+x (executable binary)

or you can just specify the flags, i.e. +x which means it would make anything fitting the filter executable but keep the base type the same

[valid vine]

okay cool, so I would do +s5 //Builds/... if i wanted to maintain just the 5 latest commits for that entire directory and binary+xs5 //Builds/... for the last 5 commits in the directory but only executable binaries?

[arctic hornet]

well, I'd probably just add the +s5 //Builds/... as that would limit it to last 5 for every single file in that stream/depot whatever...

and let the other rules decide whether it's text file, binary executable etc..

although I've personally never done these kinds of tricks in the typemap (I use the typemap from the PDF I've sent here before)

so not sure if it'll work, but it should

[valid vine]

yeah perfect, I was thinking the same thing with just the entire directory being marked 👍

ive got everything in your pdf here already

I replaced it with all that and havent needed to add any yet so you had it covered

[valid vine]

I should mark it with w too right like +ws5 otherwise its read only? is that right or am I mistaken

I am reading the docs now but still a little unsure as to when I should mark it w

for example all yours do not have a w flag so I guess I dont need it to write to it? so whats it for

ooh so I can checkout a file and so can someone else, thats not the same as adding the l flag which is exclusive locking, the w is if I dont want file checkouts but still want to store revisions?

should I mark source like this or is there a different way to configure auto checkout

(sorry about all the questions guys)