Creating a Perforce VCS Plugin for Godot

[stark kindle]

I'm looking in to creating a P4 plugin for Godot as it's what I wanted to use for a project but P4's behavior is directly at odds with how Godot needs to work. For those reading unfamiliar with P4, all files are locked by default and must be "Check Out" in P4 to be editable.

The behavior I would need for the plugin to be actually usable would be:

• Godot auto checking out files that it's trying to save/overwrite
• Files that are deleted or added in Godot need to be marked for "delete/add" in Perforce

My first thought was to look at the existing official Git Plugin but I'm honestly not clear on where to start. My understanding is that I need to implement this:
https://docs.godotengine.org/en/stable/classes/class_editorvcsinterface.html

I have the p4 documentation (here https://www.perforce.com/downloads/helix-core-c/c-api), so I think I have everything I need to start making it but I'm not sure actually sure where I would start 😅

For example in the Git Plug in repo (https://github.com/godotengine/godot-git-plugin), I could be mistaken but none of the c++ files seem to implement the EditorVCSInterface? Hopefully someone can provide me with some direction, thanks!

[pure ember]

i can't guide you here at all but i'd like to point out another possible approch you might not have discovered yet: extending the git plugin with perforce compatibility using git-p4 https://git-scm.com/docs/git-p4
theoretically the dlls of the git plugin should have support for this already seems libgit2 doesn't implement this

[stark kindle]

I had heard of that, the main issue is that I would rather have it be native p4 so that there don't need to be a git server and a p4 server. Larger orgs may have infrastructure built around a specific VCS (i.e. P4) and if Godot can't support the VCS with all of the IT built around it Godot becomes a non-option which is really the problem I'm aiming to solve.

Although maybe if as part of the extension it could establish it's own local git server? That might be a bit more than it's worth when a more targeted p4 integration doesn't seem impossible 🤔

[pure ember]

git does not require a "server"
to create a valid "remote" all one has to do is git init --bare . in some directory - the path to which becomes a valid remote supporting all git features

native p4 seems better for sure tho

[peak verge]

Godot auto checking out files that it's trying to save/overwrite
What should it do if the checkout fails? For a reasonable UI I think it would need to detect the read-only status earlier and warn when someone starts making changes, instead of postponing until they try to save their work.

[stark kindle]

Right sorry I should make that more clear. In no way should the VCS plugin interfere with a person's workflow; it shoudl checkout whatever it needs to checkout and no pop up Godot's warning about a file being locked (the problem I run in to now).

My hope is I can get it working similarly to Unity's Perforce integration where it's able to check out whatever it needs to when it needs to, then you can go to P4V and organize the files as needed.

[peak verge]

Could there be concurrent users, thus potentially merge conflicts that appear while Godot is open? If you only need 1 concurrent user, a script could check out project.godot, use that file as the mutex preventing other merge conflicts, and then launch Godot.

[stark kindle]

Yes there can be version conflicts; Perforce has its own merge tool that Godot could use but generally in my workflow a resolve doesn't happen automatically. Or at the very least, the file is flagged as out of date, and when you try to submit in Perforce you're told "hey you need to get latest and resolve merge conflicts"

[full lodge]

@vale wolf and I got a GDextension to compile with the P4API. If you'd like to be added as a contributor I can add you on github, we're still in the early stages and it's in a really jank state right now but it can connect to a server from within godot. @stark kindle

[stark kindle]

WHAT I've had to put to the side because of work (and also imma be real the p4 api was enormously intimidating)

[full lodge]

Yeah getting the API to compile was pretty gross, but we're at a point now where we just need to extend the EditorVCSInterface functions correctly

I'm guessing some changes will need to be made to EditorVCS eventually as most of the functionality is based around git it seems

[vale wolf]

We'd definitely be eager to have some extra hands on the project if you're interested. We want to do everything we can to get this to the point of being an official integration

[stark kindle]

Oh you know what, I was talking with someone months ago about this but I went left the state for a few months for a work trip and never got back to them

@rapid fossil not sure if you ended up making progress, but it seems like these guys have something going!

I can see if I can poke one of the p4 specialists I know if he would be able to offer at least some knowledge on how we integrated p4 in our internal engine

[stark kindle]

He's directing me towards some internal ccp files to study that I will go over this weekend
He sent me this code snippet and said it's the quivalent of p4 edit path/to/file
Which is very promising haha

const char* args[] = { path_to_the_file };
p4.SetArgv(_countof(args), args);
p4.Run("edit", &context);

[vale wolf]

Yea that's a great resource thank you! Any guidance on any of the internal p4 stuff would be much appreciated. Documentation has definitely been a tad scarce haha

[vale wolf]

Should hopefully be able to use this as template for at least a few of the functions

[stark kindle]

He specifically directed me to this article:
https://www.perforce.com/manuals/p4api/Content/P4API/chapter.methods.html

Do you have a public git repo? I'd definitely like to try and get it up and running

At the very least have the ability to auto check out files

[vale wolf]

Repo's not public just yet, we were hoping to get a few things cleaned up before we did that. I'll see if we can get something put together that you could poke at though

[stark kindle]

No rush! Keep me posted I'll keep tabs on this thread :)

[vale wolf]

Sounds good! We'll definitely keep you in the loop

[full lodge]

Yeah one fun quirk of that ClientUser object that p4.Run returns is that you have to override the info functions if you want to print to the godot console

Or do error handling

[stark kindle]

Mmm fun, womder how much more of a lift it would be to add to the "checked out", "matked for add", etc icons to Godots file manager
More of a nice-to-have

[vale wolf]

Right right, having the icons definitely adds a lot so we wanted to try and figure that out too

That part may require a pull request so we'll have to look into it. We wanted to see how far we could get without modifying the engine so more people could start using something that at least sorta worked, even if it wasn't super clean. But a pull request is probably inevitable in the long run

[stark kindle]

Perhaps it could be something separate in the GDExtension tab?
Just a list of file paths checked out, marked for add, marked for delete better than nothing haha

[vale wolf]

Yea, not too sure how extensible the built in godot menus are without modifications, so we'll cross that bridge when we get there. But for now, I think I reuse the git menus to show stuff. I'm thinking we could reutilize the "staged commits" menu to show modified files, etc

I'll be working on the UI side of things this evening so I'll hopefully be able to have something that is somewhat functional that you could download pretty soon

[stark kindle]

Not trying to be nosy or anything, but just wanted to check in :)

[vale wolf]

Haha no worries! Things are moving along. Right now we've got it to where you can login using the VCS interface input fields (yay no more hard coding p4 login credentials lol) so that's a step in the right direction. Currently there's this weird bug on the GDExtension side that's causing a lot of problems, so we're gonna try and work our way through that and then we should have something pretty solid we can send your way

@full lodge was traveling for work this past weekend, but he's back now so we should be able to deliver something this week hopefully 🤞

[stark kindle]

Oh yeah, not publishing your p4 creds on accident would be good yes haha

[rapid fossil]

@vale wolf @stark kindle Hey, thanks for the ping on this, had missed it for a while - I'd be interested in following this and helping out if I can, as Perforce support is something that comes up regularly for Godot. I had started on something similar using the P4 lib with GDExtension 🙂

Never got really far as I was busy in the last few months however. If you can invite me to a repo or something, I'm happy to see about contributing, potentially?

[vale wolf]

Hey there! Thanks for reaching out. Yea we'd love to have the help! And yea totally, having dedicated p4 support in godot would add a lot of value in our opinion so we really want to get this project of the ground

We're still working through the early stuff, verifying p4 connections, etc but we should be getting pretty close to having something we can share with you guys so can start contributing

[stark kindle]

100% agree, the studio I work at has network security infrastructure build around p4 and not git, so any project that can't use p4 is a non-starter which makes it difficult for me to push for adoption

[vale wolf]

Absolutely, I know p4 is a hard requirement for a lot of studios so it's going to be pretty crucial for any wise spread godot adoption

[rapid fossil]

Awesome, feel free to always reach out/ping me/DM me about this or any updates, super stoked to help make P4 support happen 🙏

[vale wolf]

Of course! We'll be sure to keep updating everyone when progress is made

[muted sun]

@full lodge @stark kindle @vale wolf
I saw you discussed Perforce with Godot. A while ago. Can I ask how are you hosting P4?

[stark kindle]

Probably more of a question for my IT department; p4 is a bit too much for my personal projects imo haha

[full lodge]

I'm hosting from a linux blade server I borrowed from an old job

Going to eventually get something that's less power demanding though

[muted sun]

Ah okay 😄
I am the techie in a indie team of four. I know Git, but they are more P4 heavy. So I was hoping to find a easy to maintain and cheap way of hosting reliably somewhere 😄

I could take an old laptop or whatever and I probably have harddrives somewhere.
But I am really bad at the network part. How would everyone else reach inside my network?
Also it would need to always be running in my appartment and so on.

[full lodge]

If you have access to your router settings you just need to open port 1666, the setup is easy otherwise

[muted sun]

Okay. And how about security? Maybe I'm paranoid though.

And ofc I need some kind of static IP to my apartment.

[full lodge]

Yeah you might be out of luck trying to open a connection to your apartment but P4 supports SSL

[muted sun]

You lost me there. What do you mean "open a connection to your apartment"? Wasn't that the 1666 portforward?

[full lodge]

You might have a router between yours and the public that's managed by the apartment

You can verify the port is actually open with https://www.yougetsignal.com/tools/open-ports/ if the service is running

[muted sun]

Thanks!

[stark kindle]

Also as just an FYI, the current workaround to use P4 with Godot is to checkout the entire repository and then submit what you need to

[vale wolf]

Hey @here! We got something pretty solid put together! https://github.com/kitchensynk/godot-perforce-plugin

It's defiently still a bit jank for now, but after getting it in the project it will be able to track file adds and push/pull

[stark kindle]

I'm on EST and will look at this tomorrow but I'll try to get it up and running then!
Fantastic work, excited to see what we can do to take it to the next level :)

[vale wolf]

Sounds great! Feel free to message with any questions

also, you guys are welcome to submit a collab request as well if you want

[stark kindle]

Separate repo is being set up for me, will start working on compiling later this evening

[vale wolf]

Cool cool, let us know if you need anything!

[stark kindle]

Ah is the demo project not intended to be plug and play yet? Are these dependencies I need from a clean compile?

[stark kindle]

Ah rats also getting stuck trying to build openSSL, don't think I set up perl properly? Not finding a configure function...

[full lodge]

Ah yeah sorry we didn't want to throw out binaries yet

we used strawberry perl

[stark kindle]

Oh hmm I tried active, I'll give that a shot next

[muted sun]

Will look into this as well. I want to contribute if I can 🙂

[full lodge]

Also if anything isn't clear on the setup feel free to ask questions, trying to make sure our docs are legible 😅

[stark kindle]

I tried installing Strawberry perl to build openSSL, but it still says I don't have a command for "Configure"

[stark kindle]

Alright I have concluded that the build guide in the OpenSSL zip is dense and confusing, so instead I'm going to try following this guide at the end of my normal hours to try and build lol
https://developers.lseg.com/en/article-catalog/article/how-to-build-openssl--zlib--and-curl-libraries-on-windows

[full lodge]

Hopefully that guide helps, although the only differences between that guide and the install.w64 are instructions to add perl and NASM to your windows path

[vale wolf]

If that guide proves to be more helpful, we can always link to that in the readme as additional information for future compiles

[stark kindle]

That might be what I was missing? Perl is showing up as a command, but just not configure

[full lodge]

Ohhhh important thing you might be missing

You need to run from the MSVC command prompt

[stark kindle]

Was doing that, still didnt have the configure command

Actually do I need to download the year of the MS SDK is says in the read me or would the most recent version work?

[full lodge]

I used the latest version I think

Looks like other folks have this issue if they are running the command from the wrong directory

"Configure" refers to a file in the OpenSSL zip

[muted sun]

Do you guys know of a good guide for setting up a depot for godot?
Like ignore, typemap and so on.

[stark kindle]

Not I'm aware of, I had a list at some point but lost track of it

You can take the default git ignore, then convert it to p4's ignore syntax
I believe that's what I did last time

[muted sun]

Yeah seems to be working fine. Not sure if there are other stuff to be done to make it work better.
I know Unreal Engine want uassets to be added to Typemap

[vale wolf]

Yea I think with godot being mostly text files p4 can handle things a little better, but not sure what needs to be typemapped or not

[muted sun]

I was thinking of creating a how to guide. Like how to setup perforce, good way to setup your depot/workspace and ofc how to work with p4 from within Godot.

Documentation always takes more time than I expect 🫠

Maybe there should be a channel for version control? 🤔

[stark kindle]

If there was it would mostly get clogged by git questions haha

[vale wolf]

Lmao you're probably right

[full lodge]

Honestly the only typemapped file I can think of would be the project file

For exclusive checkout

[muted sun]

Shit you're right 😅

[muted sun]

To be honest I'm not sure what typemap means.
As I understand it it's for files that do not need check-in history. You're never interested in an old version of it.

[full lodge]

Perforce can handle files in all sorts of different ways

The typemap defines how each file type is handled

IE If one user has a file open, perforce can lock that file for all other users

[stark kindle]

Just posting here again; have been busy at work will try to find time to build the SSL libraries this week 🙏

[vale wolf]

All good! Appreciate the update, best of luck!

[full lodge]

Merged in some updates today, not sure we can get much farther without modifying godot

or adding a custom file browser in godot somehow

[rapid fossil]

Would be super helpful if we could create a list of what is missing from the Godot side for this to be easy/possible, happy to champion possible needed changes on the Godot/GDextension side

[full lodge]

So I've sortof done that with the repo readme, but ideally we need

• EditorVCSInterface to support custom login window fields
• Custom right click options in the file browser (Mainly for checkout)
• Custom ChangeType enum, to better align with P4 changetypes

Some nice to haves:

• Overwriting the names of EditorVCSInterface buttons (push/pull/fetch -> sync/submit) and (branch -> stream)
• Icon support in the file window, similar to Unity

[rapid fossil]

Thanks, I'll review those and look at the code but these seem mostly reasonable at first glance 🙂 And extending EditorVCSInterface to be more adaptable makes a ton of sense, its very Git-focused right now, just because that was the first and only implementation

[full lodge]

Sweet, sounds good.

I'll let you know if we can think of anything else, going to take a stab at inline diffing tonight

[rapid fossil]

great work so far by the way (:

[full lodge]

Thank you much 🙂

[vale wolf]

Appreciate it!

[stark kindle]

Stuck building again

Was following this guide and got stuck on step 3 it was going so smoothly up until then too ;-;

huh maybe I have a bad version?
I needed openssl-1.0.2u right?

[full lodge]

openssl-1.0.2t

is what we used

unless the command you ran says something different

[stark kindle]

Okay yes I might have downloaded the wrong openssl version then

Oh... I looked at the download page again and just noticed all of them basically have different letters lol

What umm... is the difference? haha

[vale wolf]

Honestly no clue lol, but p4 will definitely get grumpy about compiling if it doesn't have the correct version

[full lodge]

Yeah we spent wayyyy longer than we should have trying to compile with the wrong version as well

So many linker errors o(╥﹏╥)o

[stark kindle]

wait I might have finally done it LOL (sorry work has been super busy!)
I think openssl to actually compile
So the article I linked before was able to get perl and nasm set up, but then it's installation instructions were wrong and I went to the windows install file you mention on the git

[vale wolf]

Hey congrats! Glad to hear you made some progress! And no sweat, things have been busy our way as well so it's all good

[stark kindle]

Happy New Year!
I'm back because some stuff has shifted at work again and I am very interested in Godot but again same problem.
Complete non-starter without perforce.
Going to take a stab at getting this to compile over the weekend 🙏

[vale wolf]

Happy New Year! Long time no see. It's been super hectic our direction so we haven't had much time to focus on this project unfortunately, but definitely let us know how things go! Hopefully we'll be able to commit a little more time towards it in the near future. Looking forward to hearing any progress!

[stark kindle]

Yes sorry my focus shifted to some things at work, but things have cooled off, and I have my fingers crossed on making something happen 🙏

[stark kindle]

Alright officially attempt #2, I don't have the tabs I had open before but here's hoping haha

[stark kindle]

Huh I swear I ran in to this before but the nmake script has some syntax error than nmake doesn't like

For my future reference:
cd in to the folder containing the openssl 1.0.2t download
Launch Visual Studio command prompt
Run perl -v and nasm -v to make sure they're installed

From there follow the instructions in the openssl build guide included in the folder specifically these instructions

perl Configure VC-WIN64A --prefix=c:\some\openssl\dir
ms\do_win64a
nmake -f ms\ntdll.mak
cd out32dll
..\ms\test

I modified the first command to a different path, I'm hitting errors on teh 3rd command but I believe that's because I dont' have the windows platfomr sdk installed

hopefully lol

bruh I just noticed I've been using 32-bit visual studio

OMG IT WORKED

WHY WAS IT LAUNCHING THE 32 BIT VERSION

you love to see it

[vale wolf]

Woooo! That's awesome news

[stark kindle]

Alright I am finally on this step
I will try to pick it up tomorrow after work 🙏

[stark kindle]

Alright I'm hung up on this part
I downloaded the cpp for Godot 4.1, which file here is the is the cpp bindings?

ope wait

i found the folder nvm

ok building the cpp libraries, if I'm at this step in the gdextension guide am I good to try to build the perforce plugin?
https://docs.godotengine.org/en/stable/tutorials/scripting/gdextension/gdextension_cpp_example.html#creating-a-simple-plugin

[full lodge]

Oh hey welcome back, for some reason I stopped getting notis for this thread

[full lodge]

Yeah I think you should be good

[stark kindle]

Do I just run scons on the project in the git?

[full lodge]

I think so, you may need to specify a platform as well

[stark kindle]

I'm getting an error

oh wait I need to move stuff in to the projects cpp folder derp

actually... why did it try to access that folder in the project

I just need to cd in to the project folder and run scons to the build the plug in right?

I'm not able to run scons on the project folder in the git

It errors every time

Is there something else I need to take from the godot cpp sample project and move to the perforce extension project?

Tried moving the sconstruct script to the folder it created but no luck with that :/

OHHHHHHHH the paths are hardcoded

fixing this throws a recursion error

oh derp I just realised I'm pointing it at itself, but not I'm not sure what I'm supposed to point it at

OHHHHHHH

I needed to drop the whole godot-cpp sample project in to the godot-p4 project lol

Probably should have that on the readme haha

gahh it's so close to compiling i can feel it

shoot ok I really don't know what this error is trying to tell me

WAIT
I put lib files in both places... one of those had to be dlls

nope that didn't do it :/

all of these are in the right spot

@vale wolf sorry for the ping but I don't suppose you might be able to tease apart what I'm missing?

yup stuck here

chatgpt's best guess

gotta head out for the night, but i'm pretty stuck :/

[full lodge]

It's been awhile since I've worked on this but you should be able to clone godot-cpp with the project

do you have the right version of godot-cpp?

The right version is added as a submodule in the repo

[stark kindle]

Ohh I had the old version before that was added I believe, I'll try the updated repo 🙏

[stark kindle]

New error trying to compile the included godot-cpp

oh if I don't specify the custom api in the tutorial it works

new error trying to compile the perforce extension

oh I forgot to move the p4 files over

hopefully that's it doesn't seem related but who knows haha

new errors!

   Creating library demo\addons\godot-perforce-plugin\win64\p4_plugin.windows.template_debug.x86_64.lib and object demo\addons\godot-perforce-plugin\win64\p4_plugin.windows.template_debug.x86_64.exp
libclient.lib(clientrcvfiles.obj) : error LNK2019: unresolved external symbol __std_init_once_link_alternate_names_and_abort referenced in function "void __cdecl std::call_once<class <lambda_0e64988457575224279dcb6d61e61bdd> >(struct std::once_flag &,class <lambda_0e64988457575224279dcb6d61e61bdd> &&)" (??$call_once@V<lambda_0e64988457575224279dcb6d61e61bdd>@@$$V@std@@YAXAEAUonce_flag@0@$$QEAV<lambda_0e64988457575224279dcb6d61e61bdd>@@@Z)
libsupp.lib(sysinfo.obj) : error LNK2019: unresolved external symbol __std_find_trivial_1 referenced in function "char const * __cdecl std::_Find_unchecked<char const *,unsigned char>(char const *,char const * const,unsigned char const &)" (??$_Find_unchecked@PEBDE@std@@YAPEBDPEBDQEBDAEBE@Z)
libp4api.lib(clientscript.obj) : error LNK2001: unresolved external symbol __std_find_trivial_1
demo\addons\godot-perforce-plugin\win64\p4_plugin.windows.template_debug.x86_64.dll : fatal error LNK1120: 2 unresolved externals
scons: *** [demo\addons\godot-perforce-plugin\win64\p4_plugin.windows.template_debug.x86_64.dll] Error 1120
scons: building terminated because of errors.

Just to clarify, the docs specify p4 api 2023.1 which is what I have but I noticed the p4api link only provides 2023.2, going to try the newer version when I get the time

Nope errors :/

Am I supposed to do something with the p4api or just download it and move the files over 😅

Just to confirm again I have compiled OpenSSL 1.0.2t and the version of p4 I'm using requires that specific version

Tried rebuilding godot-cpp, same errors T_T

[full lodge]

You can download earlier versions from the menu on the right

[stark kindle]

this is the version I have rn

I'm assuming I need to match that last number too?

[full lodge]

That shouldnt matter

[stark kindle]

kk

[full lodge]

You might need to copy the build godot-cpp library files over to the perforce build directory? I'm not sure

It's been awhile since we've looked at this, I might have time sometime next week to rebuild myself

[stark kindle]

oh you may be right, I'll give that a shot!

[stark kindle]

that would be great if don't end up figuring this out, thank you so much for the help so far!

[full lodge]

Yeah sorry we dont have all the answers at the moment we ran into many similar issues getting this set up

For reference, this is all built in a similar way to the git plugin for godot

[stark kindle]

Ah I have not attempted to build tjat before haha

[full lodge]

Oh just in case it fills in some holes in our documentation

[full lodge]

Some quick googling leads me to believe this might be a VS version issue?

[stark kindle]

Oh do I need something more recent than 2019?

(sorry was out to lunch, and I have some meetings coming up)

[full lodge]

Yeah you need 2022

and no worries

The P4 API has a version number on it, we'll have to call that out on the docs

[stark kindle]

Ah downloading and installing VS2022 now!

[stark kindle]

omg wait it's going and those don't look like errors

either that or it's a lot of errors lol

👀

ok I'm going to try it after work! thank you so much!

Assuming the compilation succeeded I'll make a pr with changes to the readme to help clarify the steps for people haha

Hopefully I thought bubbled enough in this chat for me to be able to remember what my pain points were :p

[full lodge]

This is super useful, thanks for putting up with building this

[stark kindle]

No prob! Glad to have something potentially working!
Hopefully if it picks up steam around the office they'll be incentive to update it to latest Godot (if it doesn't already work) and add more support for things like marking files the way p4 does but in Godot's file browser so it's visible from there without having to check the perforce window

[full lodge]

Sick yeah that would be great

[vale wolf]

Hey congrats! Glad everything compiled! I know that was a process haha, so that's awesome to see things are moving along

[stark kindle]

Huh, how would I input server information?

oh wait I see in your example how you set it up nvm

[stark kindle]

Have been given a stream, going to try it today 🙏

[stark kindle]

ope

rookie mistake

I checked in the .godot file

that's all local stuff

[stark kindle]

I can add .tmp files to my p4ignore yeah?

I mean... it's in the name lol

ah ok, it doesn't remove things it looks like

oh wait nvm it does?

oh those

tmp files are still there

oh huh... they aren't in gitignore either

@rapid fossil as someone closer to the core Godot team do you happen to know what the purpose of the .tmp files are? if they aren't in the gitignore I assume they're actually needed and should be submitted?

wait... all 3 of these tmp files are exactly the same thing

all of them are this

[stark kindle]

oh https://github.com/godotengine/godot/issues/82270
it's a thing then
I'll look in to this haha

[full lodge]

Going off instinct I would ignore those

although I have almost 0 godot experience 😅

[rapid fossil]

No idea, I'll try to find out, but I assume they should not be submitted. (The issue you found looks like the right thing, though.)

Should probably indeed be ignored

[stark kindle]

Yup added tmp to my p4 ignore
Going to try and get someone else to help me test stuff haha

[full lodge]

Any issues so far?

[stark kindle]

Haven't really been using it unfortunately, stuffs moving pretty fast rn sorry

[full lodge]

All good just curious

[stark kindle]

Busy here at the studio, I'll do what I can when I have the headspace in my freetime