#php

1 messages · Page 2 of 1

radiant flume
#

As long as it fits the language's dev conventions, that's what matters

green otter
#

Maybe. Let's see how it evolves over time. I'll keep this in mind though, and bring it up with folks to see if it's possible to move it to /

green otter
radiant flume
#

I just wanted to avoid a worst of both worlds where 1) it's an extra directory layer for dagger users, and 2) it doesn't actually match php dev conventions

green otter
#

@radiant flume majority of all PHP users will not be making reuseable modules .. but instead just Dagger Module/Functions for their existing webapp... inside the same repo

So like
/ci/src/Build.php
/ci/src/UnitTests.php

Kind of thing.

summer sequoia
#

The src directory diferentiates the user module code from the sdk directory containing the auto generated sdk and the vendor directory containing and dependencies the user needs for their module

#

composer might also choke on trying to point a PSR-4 namespace to the directory which contains the vendor directory as well; not sure, never tried it

#

that template is designed to look like something a PHP dev would expect a project to look like

serene light
#

And as has sort of been stated already, conventionally, PHP projects tend to keep php files in a src/ directory

green otter
#

Guys, I'm trying to dagger init on an existing PHP project ..

# 
mkdir ci && cd ci

dagger init --sdk=github.com/carnage/dagger/sdk/php/runtime@add-php-runtime --name=gt
✔ connect 0.8s
  ✔ starting engine 0.5s
  ✔ starting session 0.1s
✘ ModuleSource.resolveFromCaller: ModuleSource! 0.4s
! failed to collect local module source deps: failed to get git module sdk: failed to load sdk module github.com/carnage/dagger/sdk/php/runtime@add-php-runtime: select: module name and SDK must be set

Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to get git module sdk: failed to load sdk module github.com/carnage/dagger/sdk/php/runtime@add-php-runtime: select: module name and SDK must be set
#

module name and SDK must be set
How is it not set? 🤔

serene light
#

It's a bit awkward because of all the git branches atm... but when initialising it, it seems you need to use the exact branch

summer sequoia
#

I've just resolved that by merging your branch, my branch and toms' branch into a single linear history

#

it should work with the add-php-runtime branch now

serene light
#

ah okay, I will have to fetch the latest changes

green otter
#

ummm .. I am using the original branch we've been using

#

add-php-runtime should be correct

serene light
green otter
# 
dagger init --sdk=github.com/carnage/dagger/sdk/php/runtime@add-php-runtime --name=gt

So this should work, that's a legit branch name

but it broke for me? 🤔
select: module name and SDK must be set

Can you see what's wrong @summer sequoia @serene light ?

summer sequoia
#

Might have broken it when I did stuff to merge the branches together

#

@green otter dagger init --sdk=github.com/carnage/dagger/sdk/php/runtime@charjr-add-php-runtime gt works, using my branch with @shy goblet 's change cherry-picked in didn't :/

shy goblet
#

Hmmm

#

Wait, do the TomChv fork works with it?

#

But not the carnage branch?

summer sequoia
#

let me try

#

hmm, that did work :/

#

ohhh

#

didn't remove the /runtime from the git url

#

@green otter this works: dagger init --sdk=github.com/carnage/dagger/sdk/php@add-php-runtime gt

green otter
#

Hahaha. My bad. Didn't see it 🙈

inner walrus
#

This channel is so energizing!! 🚀 🚀 🚀

faint current
#

any ETA for module support?

summer sequoia
#

hopefully a basic implementaiton next week

#

though you can start playing with it now using: dagger init --sdk=github.com/carnage/dagger/sdk/php@add-php-runtime mymodule

#

just be warned that when it's merged, you'll need to fix your module's dagger.json to point to the real repository

#

that command sometimes fails the first time, but rerunning works fine - I'm trying to debug that currently

green otter
#

@faint current give us a week

radiant flume
#

I just want to say I'm amazed by the activity in this channel, you guys are relentless!

summer sequoia
#

@green otter @serene light I've Rebased the add-php-runtime branch to squash every sequential commit by the same person, so we've not got 100s of commits on the branch any more; updated your workgin copies as appropriate but that branch now contains the latest code.

green otter
#

Thanks

green otter
#

Anyone about? I'm trying some stuff and have general dagger Qs

#

@inner walrus hey! 🙂

inner walrus
green otter
#

I wanna copy code from my localhost into the dagger container ...

        $dir = (new Host())->directory('./laravel-app');
        $container = $this->client->container()->from('jakzal/phpqa:latest')
         ->withDirectory('/tools/app', $dir)
         ->withExec(['phpqa', '', ''])
#

What is the $dir line? I'm looking at the source code here (generated PHP SDK code) and trying to figure it out

inner walrus
#

With functions, you don’t have access to the host, so have to pass a Directory in to your function as a function parameter

green otter
#

aye

#

All these code examples online are taking source from the CLI (--source) and don't actually show you how to do it with code

#

ok this makes sense why I'm stuck 🙈

inner walrus
#

Yep, add a param, source is popular, of type Directory and then add to Container with withMountedDirectory if you won’t be exporting/publishing that Container, but just doing something in it. Use withDirectory if you need that source to be copied in “permanently” and show up in a published/pushed image

green otter
#

so if it's temporary code mounting .. just to do Linting and stuff

#

am I using Mounted or non-mounted ?

inner walrus
#

Mounted is fine for that

green otter
#

ok

inner walrus
#

It’s not a two-way bind mount, just an ephemeral snapshot

green otter
#

I just want docker -v tbh 🙂

#

docker run -v src:dest

inner walrus
#

lol. Yep. We are much more sandboxed, but there are proposals to allow that sort of thing to be possible

green otter
#

hmmm .. I added a new function to my local dagger module ... dagger functions isn't picking it up 🙈

inner walrus
#

One other gotcha is, if you want something like a Directory or Container or File artifact to make it back to your actual host (laptop, CI runner, etc) then it needs to be a return value of a function.

green otter
#

oh, we gotta re-init the module, I guess? so dagger re-parses it all

#

Can i dagger init, without specifying the remote git URL? just local dir

#

Dagger has to re-run the code gen .. somehow 🤔

inner walrus
#

Any other export side effect you do with the API will put things on the “host” it can see, which is the runtime container (PHP, TS, Go, Python)

inner walrus
green otter
#

ok

#

that ran .. but it didn't find my new function :/

inner walrus
#

So if you run dagger functions you don’t see it? 🤔

green otter
#

indeed. it's not showing up

#

It's caching it ..

inner walrus
#

Huh. I wonder if the PHP SDK needs to implement something to rerun codegen. I know we changed that a while back

green otter
#

I just moved my Example.php file (dagger functions file) out to /tmp/ and re-ran dagger develop && dagger functions and it still finds my 2 functions ..

#

so it's caching

inner walrus
#

Might need a cache buster somewhere. Worth looking in typescript or python sdk for such a thing. 🤔 edit: I don't see anything about that in other SDKs at a glance

green otter
#

ok

#

HEHE 😄

#

so there's a diference between

 dagger init --sdk=github.com/carnage/dagger/sdk/php@add-php-runtime mymodule

and

 dagger init --sdk=github.com/carnage/dagger/sdk/php@add-php-runtime --source=./mymodule
#

the latter brings down a '/src/ DIR and other bits .. and now I realise in our SDK golang stuff, it looks in ./src/ for the stuff to codegen

#

so the latter works

inner walrus
#

Ah, yes, the source there is part of init and sets the source in dagger.json so Dagger knows where to look for the module implementation, even though the module root is where the dagger.json is in "."

#

The default lately being ./dagger/ relative to the dagger.json

green otter
#

right, cool. Noted 🙂

inner walrus
# 
mymodule ➤ dagger init --sdk typescript
Initialized module mymodule in .
mymodule ➤ tree -L 1
.
├── LICENSE
├── dagger/
└── dagger.json

2 directories, 2 files
mymodule ➤ cat dagger.json
{
  "name": "mymodule",
  "sdk": "typescript",
  "source": "dagger",
  "engineVersion": "v0.11.7"
}
#

the most typical values for source there are . and dagger, today

# 
mymodule ➤ dagger init --sdk typescript --source .
Initialized module mymodule in .
mymodule ➤ tree -L 1
.
├── LICENSE
├── dagger.json
├── package.json
├── sdk/
├── src/
└── tsconfig.json

3 directories, 4 files
mymodule ➤ cat dagger.json
{
  "name": "mymodule",
  "sdk": "typescript",
  "engineVersion": "v0.11.7"
}
#

default if you don't specify source is dagger today.

green otter
#

understood

#

it's all running 😎 sexy

# 
dagger call user-list --svc=tcp://localhost:3306

@inner walrus does this have to be a dagger service? or just any Daemon running on that TCP port?

#

It's a host service btw ☝️

inner walrus
#

your userList func has to have an arg called svc of type Service (dagger service), but the tcp://localhost:3306 service can be anything on your host, yes.

green otter
#

Yup, got it. Ty

inner walrus
#

PHP 8 typically declares functions in camelCase, right? like function userList(Service $svc) {}

green otter
#

that's right.

inner walrus
#

going to go look at code now after guess 😆

green otter
#

@inner walrus yes .. we have our PHP PSR standards (like PEP8) for all code styling rules ... and yes in PSR we use camel case for methods/functions

#

I'm able to run

  1. unit tests
  2. linting checks
  3. static analysis checks

On the laravel framework
🙂

#

Why is dagger truncating the line length? can we fix this? It's pre-maturely wrapping lines

image.png
inner walrus
#

can you check it in Traces?

green otter
#

huh?

inner walrus
#

it's free.

#

trying to figure out if it's the TUI or something else

green otter
#

I tested it out .. but not currently using it.

#

So yea, the terminal is wrapping the lines .. and I want it to go full width 🙂

inner walrus
#

I'm working on some things with long lines...but I guess that's the Dagger output and not log output...hmmm

image.png
green otter
#

Yea

#

It's the STDOUT output that it's truncating, I think, coz that's the PHP runtime saying "exception found .. here is the stderr output"

inner walrus
green otter
#

Sure ... I'll sync with @summer sequoia to see if there's anything we're doing about it .. before raising an issue. but I will 🙂

inner walrus
green otter
#

I'm gonna publish my first daggerverse module now 🙂

#

my browser is just loading and loading ..

#

Every other website loads .. but that one isn't

#

@inner walrus is there a URL on how to publish your repo into the Daggerverse? (that's not on daggerverse.dev, as that's not loading for me) - so I can learn

inner walrus
#

Not sure why slow! Sorry about that. I let our infra folks know.

#

Could you try dagger publish from the root of your git repo.

#

If it has a git remote called origin, it should work no prob.

green otter
#

ok

#

@inner walrus I'm running dagger publish now

# 
Error: git repository is not clean; run with --force to ignore

On branch main
Your branch is up-to-date with 'origin/main'.

nothing to commit, working tree clean

?

#

i have nothing to commit btw ..

radiant flume
#

@green otter dagger publish with no arguments is best avoided. Run it against a remote url, and run it from outside a local git repo for peace of mind

green otter
#

Ok. It didn't work anyway. Daggerverse is down.

radiant flume
summer sequoia
#

We now have a fork of the graphql library so the composer stuff is sorted

tidal geyser
#

Hey Paul,

Let's go through a few steps and figure out why https://daggerverse.dev. is slow for you.

First of all, let's see where the TCP slowness is coming from. I find mtr best for this: https://www.tecmint.com/mtr-a-network-diagnostic-tool-for-linux/

The first screenshot is what that looks like for me: sudo mtr -brwc 10 -Z 1 daggerverse.dev (note: I am on a Mac & need to use sudo).

Based on the above, I get about 90ms of latency to the TCP endpoint that currently services daggerverse.dev. I am based in Milton Keynes. What does your result look like?

The second thing would be to check the HTTP connectivity from your end to daggerverse.dev, and understand the different segments. I find httpstat useful for this: https://github.com/davecheney/httpstat

See the second screenshot for what this looks like for me: httpstat https://daggerverse.dev. It confirms that TCP is 90ms away, and shows me how much time is spent in TLS, server processing & then the final transfer. All in all, I would expect this to complete within 1s in the UK. What do you get?

Lastly, https://worldpagespeed.fly.dev/ will show you how fast https://daggerverse.dev loads around the world. It uses chromeheadless so the total time is the full page load, including all the assets. Third screenshot is what I get.

Also, if you are in doubt whether daggerverse.dev is operational, this is the place to look: https://status.dagger.io/

Let us know what you find out!

image.png image.png image.png
#

FWIW, we monitor daggerverse.dev from all major continents. This is what the daggerverse.dev HTTP latecy looked like for the last 24h from all of Europe (there are multiple probes across all major European countries):

image.png
green otter
green otter
#

@tidal geyser I still can't access it 😦 timing out. I'm running mtr now to check

#

There is no thing in my hosts file, to make it not work

tidal geyser
#

We have recently migrated the infra for Daggerverse, so the only thing that I suspect now is - you guessed it - DNS.

If you run this locally in a terminal, what do you get? dig daggerverse.dev +trace

This is what you should see:

image.png
#

These are the important IPs:

❯ dig daggerverse.dev +short
107.20.229.45
18.211.147.51
inner walrus
#

same for me

;; ANSWER SECTION:
daggerverse.dev.    60    IN    A    18.211.147.51
daggerverse.dev.    60    IN    A    107.20.229.45
#

@tidal geyser one thing I've noticed is that I can't reach daggerverse.dev from my grandmother's apartment wifi because it says it can't establish a secure connection because of our certs and the fact that the ISP uses a particular firewall.

cobalt pelican
#

Hey guys, how do you define the description of a function argument in PHP? And what if you have extra metadata like "ignore" patterns for a Directory argument?

summer sequoia
#

extra metadata would be extra params

serene light
# summer sequoia ```#[DaggerFunction('Search a directory for lines matching a pattern')] pub...

On the same topic:
Currently all arguments are considered to be a DaggerArgument i.e. something that can be provided by the user.
Currently the #[DaggerArgument] attribute is optional and only used for metadata.

Suggestion:
If we want an "ignore" pattern... I think for consistency:

  • Only functions with the #[DaggerFunction] attribute are considered to be a DaggerFunction i.e. something that can be called by the user. (this is the currently implemented behaviour)
  • Only arguments with the #[DaggerArgument] attribute are considered to be a DaggerArgument i.e. something that can be provided by the user (this is the suggested change)
summer sequoia
summer sequoia
#

Just rebased on the latest dagger main branch & updated the engine and I'm now hitting this error when init'ing a module

# 
Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to get git module sdk: failed to get sdk source for github.com/carnage/dagger/sdk/php@add-php-runtime: select: failed to resolve git src to commit: failed to update submodules for https://github.com/carnage/dagger: git error: exit status 128
stderr:
Submodule 'engine/telemetry/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'engine/telemetry/opentelemetry-proto'
fatal: No url found for submodule path 'telemetry/opentelemetry-proto' in .gitmodules
#

@shy goblet your PR added some telemetry stuff to the go.mod file; is that causing this issue?

shy goblet
#

Which PR?

#

Oh you mean the one I did on php?

#

Hmm it was working before right? That's a strange error

#

Maybe try to clean up the go.mod? I'm not aware of that kind of issue

summer sequoia
#

hmm, removed from go.mod and now get

# 
Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to get git module sdk: failed to get sdk source for github.com/carnage/dagger/sdk/php@add-php-runtime: select: failed to resolve git src to commit: failed to update submodules for https://github.com/carnage/dagger: git error: exit status 128
stderr:
fatal: No url found for submodule path 'telemetry/opentelemetry-proto' in .gitmodules
serene light
shy goblet
serene light
shy goblet
serene light
shy goblet
#

Hmm okay I need to check

shy goblet
#

Ok that super weird, because I'm hitting an issue completely different when I try from my branch:

dagger init --sdk=github.com/TomChv/dagger/sdk/php@feat/local-php-runtime --name=test

Fatal error: Uncaught TypeError: GraphQL\Utils\BuildClientSchema::build(): Argument #1 ($introspectionQuery) must be of type array, null given, called in /codegen/src/Command/CodegenCommand.php on line 54 and defined in /codegen/vendor/webonyx/graphql-php/src/Utils/BuildClientSchema.php:95
Stack trace:
#0 /codegen/src/Command/CodegenCommand.php(54): GraphQL\Utils\BuildClientSchema::build(NULL)
#1 /codegen/vendor/symfony/console/Command/Command.php(279): Dagger\Command\CodegenCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#2 /codegen/vendor/symfony/console/Application.php(1029): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 /codegen/vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(Dagger\Command\CodegenCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#4 /codegen/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /codegen/codegen(22): Symfony\Component\Console\Application->run()
#6 {main}
  thrown in /codegen/vendor/webonyx/graphql-php/src/Utils/BuildClientSchema.php on line 95
#

And it looks like a php error

#

Same on your branch

dagger init --sdk=github.com/carnage/dagger/sdk/php@add-php-runtime --name=test 
Full trace at https://dagger.cloud/Quartz/traces/17c330e5caecda6730b3b69abf88f795

✘ ModuleSource.asModule: Module! 19.2s
! failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to call sdk module codegen: select: call function "Codegen": process "/runtime" did not complete successfully: exit code: 2

Error: failed to generate code: input: moduleSource.withContextDirectory.withName.withSDK.withSourceSubpath.asModule resolve: failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to call sdk module codegen: select: call function "Codegen": process "/runtime" did not complete successfully: exit code: 2

Stdout:
marshal: json: error calling MarshalJSON for type *dagger.GeneratedCode: input: container.from.withMountedDirectory.withoutEntrypoint.withWorkdir.withExec.withExec.withExec.withExec.withNewFile.withExec.directory resolve: process "./codegen dagger:codegen --schema-file schema.json" did not complete successfully: exit code: 255

Stdout:
Fatal error: Uncaught TypeError: GraphQL\Utils\BuildClientSchema::build(): Argument #1 ($introspectionQuery) must be of type array, null given, called in /codegen/src/Command/CodegenCommand.php on line 54 and defined in /codegen/vendor/webonyx/graphql-php/src/Utils/BuildClientSchema.php:95
Stack trace:
#0 /codegen/src/Command/CodegenCommand.php(54): GraphQL\Utils\BuildClientSchema::build(NULL)
#1 /codegen/vendor/symfony/console/Command/Command.php(279): Dagger\Command\CodegenCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#2 /codegen/vendor/symfony/console/Application.php(1029): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 /codegen/vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(Dagger\Command\CodegenCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#4 /codegen/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /codegen/codegen(24): Symfony\Component\Console\Application->run()
#6 {main}
  thrown in /codegen/vendor/webonyx/graphql-php/src/Utils/BuildClientSchema.php on line 95

I do not see any go error, am I missing something?

summer sequoia
#

which dagger engine version are you on?

#

I had that error on 0.11.0 but it went away on the latest

shy goblet
#

v0.11.8

summer sequoia
#

that error confuses me; as it's basically got a schema file, passed in from the runtime and json decoded it. So to get null, the file is either empty or contains invalid json

#

Ahhh got a lead on it. A change last week swiched the json passed from a string to a file

#

hmm, I've pushed a fix, however I'm still geting the go error before it gets to that point :/

#

Is there a quick way to update the dagger engine to the latest version?

cobalt pelican
summer sequoia
#

hmm, seems the go issue was specific to 11.6/11.7 it's gone on 11.8

summer sequoia
#

@cobalt pelican What is the expected behaviour of the SDK if a user's module throws an error? As obviously, it can't set a result properly on the function call as the type won't match

cobalt pelican
#

That's what already happens with unhandled exceptions. Python tries to print a more user friendly error (without the traceback) when it can. Traceback is included when debug level logging is enabled or if it's an unhandled exception.

#

Basically there's a custom exception that the SDK re-raises for errors that it catches in several places (e.g. FatalError).

green otter
#

I had a good call with @summer sequoia today .. we were able to take an existing app "dagger init --source=." onto it, and start coding some dagger functions onto it..

We even took an existing dagger module that Carnage made (from github, not daggerverse) and start interacting with that as well.

It's starting to take shape! on the fundamentals at least..

#

@cobalt pelican @astral hollow @radiant flume I'd say we're hitting ready for merge point now .. it can still be tagged as experimental, but I'm feeling confident atm that we have a solid base to make future PRs upon (and smaller ones too, not 1 months of work 😉 )

What do you guys think?

cobalt pelican
green otter
#

Thanks

green otter
#

Pasting this here so @serene light can see it too. /cc @summer sequoia

        try {
            $container = $container
                ->withExec(['./vendor/bin/phpunit']);
            return $container->stdout();

        } catch(\Throwable $e) {
            return $e->getErrors()[0]['extensions']['stdout'];
        }

This is how I had to find the internal exception thrown, from the ->withExec() call that returned an exit code of 2 when there were a few unit tests failing.

Yes the unit test suite failed, but the execution of this command was successful, from the Dagger/SDK/GraphQL perspective, so we have to doing try/catch of the output .. and return that .. instead of it bubbling up to the top.

This is probably #1 priorty to fix, for the workshop which is next week
😅

serene light
summer sequoia
#

Unit tests for a project; using the SDK to make a ci pipeline

green otter
#

@serene light any tests .. run a bash script that does "exit 1" and it will replicate

serene light
#

Oh, I see! Apologies I misconstrued as the unit tests on the php sdk itself

summer sequoia
#

@green otter do you have time for a call on this?

green otter
#

@summer sequoia yes. How abt 3pm?

summer sequoia
#

That could work; though I might be going to the vet at some point

#

let me check

summer sequoia
#

👍 for 3PM //cc @serene light

summer sequoia
green otter
# 
         complete successfully: exit code: 1                                    

 [ERROR] #0 /src/dagger/vendor/carnage/php-graphql-client/src/Client.php(160):  
         GraphQL\Results->__construct(Object(GuzzleHttp\Psr7\Response), false)  
         #1 /src/dagger/vendor/carnage/php-graphql-client/src/Client.php(115):  
         GraphQL\Client->runRawQuery('query {\nphp {\nw...', false,             
         Object(stdClass))                                                      
         #2 /src/dagger/sdk/src/Client/AbstractClient.php(40):                  
         GraphQL\Client->runQuery(Object(GraphQL\Query))                        
         #3 /src/dagger/sdk/src/Client/AbstractObject.php(26):
# 
    #[DaggerFunction('phpunit')]
    public function phpunit(
        #[DaggerArgument('Path to your project')]
        Directory $source,
    ): string {
       return $this->client->container()
           ->from('php:8.3-cli-alpine')
           ->withMountedDirectory('/tmp/app', $source)
           ->withWorkdir('/tmp/app')
           ->withExec(['./vendor/bin/phpunit'])
           ->stdout();
    }
# 
dagger call phpunit --source=.
# 
        try {
            $container = $container
                ->withExec(['./vendor/bin/phpunit']);
            return $container->stdout();

        } catch(\Throwable $e) {
            return $e->getErrors()[0]['extensions']['stdout'];
        }
cobalt pelican
#

Hey guys, what's the canonical PR to review? Is it Paul php runtime? Is it up to date?

summer sequoia
# cobalt pelican Hey guys, what's the canonical PR to review? Is it [Paul php runtime](https://gi...

It'll be this one https://github.com/dagger/dagger/pull/7493 but I need to check it's got the latest commits on it. seems to be up to date

GitHub
[WIP] Add php runtime by carnage · Pull Request #7493 · dagger/dagg...

Very minimal attempt at a PHP runtime for module support.
To the best of my knowledge, the go part of this PR is complete & working.
Outstanding is to write the PHP code to

Create a module ent...

green otter
#

@serene light @summer sequoia any movement on that non-caught Exception issue on withExec() / stdout() ? 🙂

As per our call yesterday ... it is making the DX gods very sad 😢

serene light
summer sequoia
#

Yeah, it needs to be robust regardless of what ends up being thrown as it's a generic exception.

green otter
#

Okay. Got it @serene light @summer sequoia

summer sequoia
#

It doesn't always contain the same data

serene light
# summer sequoia It doesn't always contain the same data

If it fails to make a container i.e. ->from('php:8.3-cli-alpine') then getErrorsDetails()['extensions'] is not set

If it fails to execute the command i.e. ->withExec(['./vendor/bin/phpasta']) then getErrorsDetails()['extensions'] is set

array (
 '_type' => 'EXEC_ERROR',
 'cmd' => 
 array (
   0 => 'docker-php-entrypoint',
   1 => './vendor/bin/phppasta',
 ),
 'exitCode' => 127,
 'stderr' => '/usr/local/bin/docker-php-entrypoint: exec: line 9: ./vendor/bin/phppasta: not found',
 'stdout' => '',
)

If it phpunit fails a test then

 array (
  '_type' => 'EXEC_ERROR',
  'cmd' => 
  array (
    0 => 'docker-php-entrypoint',
    1 => './vendor/bin/phpunit',
  ),
  'exitCode' => 1,
  'stderr' => '',
  'stdout' => 'PHPUnit 10.5.24 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.8
Configuration: /tmp/app/phpunit.xml

.........................................................F...SS  63 / 112 ( 56%)
.................................................               112 / 112 (100%)

Time: 00:00.072, Memory: 10.00 MB

There was 1 failure:

1) GraphQL\\Tests\\QueryTest::testThatFails
This test fails

/tmp/app/tests/QueryTest.php:26

FAILURES!
Tests: 112, Assertions: 139, Failures: 1, Skipped: 2.',
)
cobalt pelican
#

The "extensions" in errors is only set on an exec error at the moment.

#

It allows creating a custom exception in SDKs.

serene light
#

@summer sequoia ^

serene light
cobalt pelican
#

You can look at the "path" in a GraphQL error to see which API call raised the error.

#

So if it errors in "from" here: $dag->container->from('php:8.3-cli-alpine')->withExec(['./vendor/bin/phpasta']), then you should see ["container", "from"] in the error's "path".

#

I think ideally you store the data you get in the GraphQL error in a general custom exception, in a way that's decoded from JSON. This way you don't lose that info when debugging. Should already be done by your GraphQL client library. So a sub-exception for ExecError reads those properties to customize the error a bit further, especially allowing to catch errors coming from executing the withExec instead of something else. At the moment, the custom error types like EXEC_ERROR, aren't in codegen, so it's created manually.

summer sequoia
#

Yeah I think that's the conclusion we were coming to earlier

serene light
#

@green otter referring to this branch: "sdk": "github.com/carnage/dagger/sdk/php@charjr-add-php-runtime"

It now gets the proper stdout from a failed phpunit run:

imhotek@imhotek2:~/git/php-graphql-client$ dagger call phpunit --source=.
PHPUnit 10.5.24 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.8
Configuration: /tmp/app/phpunit.xml

.........................................................F...SS  63 / 112 ( 56%)
.................................................               112 / 112 (100%)

Time: 00:00.069, Memory: 10.00 MB

There was 1 failure:

1) GraphQL\Tests\QueryTest::testThatFails
This test fails

/tmp/app/tests/QueryTest.php:25

FAILURES!
Tests: 112, Assertions: 139, Failures: 1, Skipped: 2.

But it doesn't currently return the correct exit code.

imhotek@imhotek2:~/git/php-graphql-client$ echo $?
0
#

I know how to get the correct exit code... but when I try having the EntrypointCommand return anything other than 0 I get this:

imhotek@imhotek2:~/git/php-graphql-client$ dagger call phpunit --source=.
resolve: process "docker-php-entrypoint ./vendor/bin/phpunit" did not complete successfully: exit code: 1

imhotek@imhotek2:~/git/php-graphql-client$ echo $?
127
cobalt pelican
#

If I'm reading this correctly, the exit code in a withExec doesn't currently propagate to the CLI. Actually never thought about that and no one asked for it, but it's something we could do.

#

Needs an issue though.

green otter
#

@cobalt pelican well if you have a CI pipeline .. and you run a command "run unit tests" .. the pipeline should ofcourse stop and fail because of a non-zero exit code ...

#

Niceone @serene light for that.. at can live without the exit code for now .. until it gets fixed..

serene light
#

@green otter I've given it a quick test with the phpunit command... but if you happen to notice anything awry when planning your workshop please let me know 🙂

green otter
#

@serene light we need this merged into Chris' main branch 🙂

serene light
cobalt pelican
green otter
#

@cobalt pelican that's good and works for me 🙂

#

@cobalt pelican above .. @serene light mentioned he is having a challenge with the SDK runtime ...

When we return exit 1 from our PHP entrypoint command, back to the main.go runtime we get this output ..

$ dagger call phpunit --source=.
resolve: process "docker-php-entrypoint ./vendor/bin/phpunit" did not complete successfully: exit code: 1

It no longer accepts our custom output .. it overrides/changes it with that ..

Is that right @serene light ?

serene light
#

@green otter @summer sequoia and I solved this just now, no longer a problem @cobalt pelican, thank you

#

@green otter You can point your module to the main branch add-php-runtime now

#

It's merged

green otter
#

ooh.. shiny

#

if I have an existing dagger.json in a repo .. and I wanna get the NEW version of our SDK code ... what's the best way to do this?

#

It currently looks like

{
  "name": "laravel-app",
  "sdk": "github.com/carnage/dagger/sdk/php@add-php-runtime",
  "dependencies": [
    {
      "name": "php",
      "source": "github.com/carnage/dagger-php-module@22aa90a967f4eef40bba1ddce096ad04e8888b33"
    }
  ],
  "source": "dagger",
  "engineVersion": "v0.11.8"
}
cobalt pelican
#

If latest changes are in the repo referenced in "sdk", it should be automatic.

green otter
#

noted. so it pulls every time.

#

I'm giving it a spin now.

#

@serene light it's working 😎

# 
Error: response from query: input: laravelFramework.phpunit resolve: call function "phpunit": process "/src/dagger/dagger" did not complete successfully: exit code: 255

Stdout:
An error occurred inside PHPUnit.

Message:  Undefined constant PDO::MYSQL_ATTR_SSL_CA
Location: /tmp/app/tests/Database/DatabaseMariaDbSchemaStateTest.php:66

#0 /tmp/app/vendor/phpunit/phpunit/src/Metadata/Api/DataProvider.php(158): Database\DatabaseMariaDbSchemaStateTest::provider()
#1 /tmp/app/vendor/phpunit/phpunit/src/Metadata/Api/DataProvider.php(59): PHPUnit\Metadata\Api\DataProvider->dataProvidedByMethods('Database\\Databa...', 'provider', Object(PHPUnit\Metadata\MetadataCollection))
#2 /tmp/app/vendor/phpunit/phpunit/src/Framework/TestBuilder.php(41): PHPUnit\Metadata\Api\DataProvider->providedData('Database\\Databa...', 'testConnectionS...')
#3 /tmp/app/vendor/phpunit/phpunit/src/Framework/TestSuite.php(508): PHPUnit\Framework\TestBuilder->build(Object(ReflectionClass), 'testConnectionS...', Array)
#

@cobalt pelican so we're not getting real-time output .. it's buffering it up until the end

image.png
summer sequoia
#

probably needs something such that we can stream the response from stdOut called after withExec

#

not sure that's possible currently

cobalt pelican
green otter
#

it's kind of underwhemling when we run 10,000 unit tests ... and see no output until the end 😛

cobalt pelican
#

Yeah, up to you guys when you want to add OTel.

green otter
cobalt pelican
#

It needs to be done in PHP using https://opentelemetry.io/docs/languages/php/getting-started/. I can help with what's needed when you get to it. You'll basically want the SDK to make some automatic OTel instrumentation based on the existence of OTEL_ env vars, and propagate the context. For the live progress you'll need a custom span processor.

green otter
#

Understood.

#

This will give the real-time STDOUT ?

#

we wanna see this stuff ..or a CLI progress bar

image.png
cobalt pelican
#

You can try a small repro in a Go module to see how it'll look like.

green otter
#

This is a multi-stage build question

#

This is the Dockerfile syntax
COPY --from=composer /usr/bin/composer /usr/bin/composer

# 
$composer = $this->client->container()->from('composer:2');
cobalt pelican
# 
$ctr->withFile("/usr/bin/composer", $this->client->container()->from('composer:2')->file("/usr/bin/composer"));
green otter
#

ty - testing

#

@cobalt pelican how can I make sure a step is cached? For example ..

composer install which is the same as npm install

I don't wanna run it twice.

# 
->withExec(['composer', 'install']);
cobalt pelican
#

First you should add a cache volume for composer's cache. That'll avoid re-downloading packages. As for composer install, you can add another cache volume for vendor, in this case with a more specific cache key.

#

In general, to avoid cache invalidation you need to make sure inputs don't change.

green otter
#

Ok.. Do you have a good recommended URL (from the cookbook?) I can follow to try and add this

cobalt pelican
#

Also, for the cache volume for composer's cache, you need to know if composer allows concurrent writes to that cache or not.

green otter
#

It does .. because composer installs packages in parallel.

cobalt pelican
green otter
#

Actually .. that's for bringing in host files .. I don't have any host files .. it exists only inside the container

# 
      .withMountedCache(
        "/src/node_modules",
        dag.cacheVolume("node-21-myapp-myenv"),
      )
#

This is my code:

            ->withMountedDirectory('/tmp/app', $source)
            ->withWorkdir('/tmp/app')
#

I'm trying this

->withMountedCache('/tmp/app/vendor', $this->client->cacheVolume('composer-volume'))
#

composer installs to ./vendor

cobalt pelican
#

Doesn't need to be an absolute url, if you put it after workdir.

green otter
#

cool

#

composer.json should be the invalidation file - if that changes then it should re-run - how to achive cache invalidation?

cobalt pelican
#

To invalidate the cache, it's just the opposite. Make sure you use an input that's guaranteed to be different. One simple example of forcing that is using a cache buster env var, like in https://docs.dagger.io/cookbook#invalidate-cache. In your case, composer.json does what you need already. Just needs to be added first.

green otter
#

I mean ->withMountedDirectory() is called first .. so ?

# 
        return $container
            ->withMountedDirectory('/tmp/app', $source)
            ->withWorkdir('/tmp/app')
            ->withMountedCache('/tmp/app/vendor', $this->client->cacheVolume('composer-volume'))
            ->withFile('/usr/bin/composer', $composer->file('/usr/bin/composer'))
            ->withEnvVariable('COMPOSER_ALLOW_SUPERUSER', '1');
cobalt pelican
#

It's a good idea to copy composer.json (and lock) first to install dependencies, then bring in the rest of the code. Otherwise any changes in code will invalidate the cache for installing the dependencies.

green otter
#

In composer that won't work as there can be refenreces in composer.json which point to the codebase .. such as 'plugins/hooks' .. so the codebase needs to come in

cobalt pelican
#

Can't you include only enough to make the install step work?

green otter
#

Every project will be different, so we can't recommend that as a solution, since some people will have code-base references, and some won't.

cobalt pelican
#

Are you making a reusable/general module for php projects?

green otter
#

No, I'm making workshop examples for people to learn from .. and take away to their projcets (whatever it looks like)

cobalt pelican
#

In that case you can afford to do what's right for your composer.json. In this case, you can follow the same best practices as in Dockerfiles it's the same cache invalidation.

green otter
#

ok.

#

I'll do this

COPY composer.json
RUN composer install

COPY . /var/www/html   <-- copy all the source code
#

let's build that in daggerland.

cobalt pelican
#

Isn't there a lock file too?

green otter
#

Yea there is .. but you can change your .json file and composer.json will pick up that change (diff vs the .lock file) and run the 'require' command for you

#

require = new packages

#

lazy mode 🙂

#

How to copy just 1 file from the host $source DIR?

#

basically ..

dagger call testing --source=.

From this call, it will need to bring in 'composer.json' from that source DIR
and then do composer install
and then mount the entire directory
and then run the test suite

cobalt pelican
# 
$source->file("composer.json")
green otter
#

Like this?

$container->withFile($source->file('composer.json'))
cobalt pelican
#

Almost, first arg to withFile is the path. You can leave it empty ('') to use the same file name.

green otter
#

optional is no longer a thing .. it's mandatory in the PHP generated stuff ... so mayb graphql has been changed

#

thanks - testing

# 
->withFile('composer.json', $source->file('composer.json'))
cobalt pelican
green otter
#

ah blank string .. got it

#

🙂

#

Yea so this isn't gonna work .. bcoz the composer.json has application things in it

image.png
#

At least we tried .. in any case .. we're trying to invalidate the composer cache ..

cobalt pelican
#

Then just add what's needed, you can add more dirs and files, instead of the whole dir.

green otter
#

we need everything 😄 it's coupled

cobalt pelican
#

Yeah, that's a caveat to be aware of because any change in code invalidates the compose install step. However, with the cache volume, even though that step re-runs, it should be faster since it doesn't need to refetch everything.

green otter
#

Understood. We will just allow the entire app's codebase to invalidate it .. like you said.

It was a good exercise to try tho .. and I will hopefully have a simple example where I can showcase this simple composer.json / composer.lock invalidation ... on a simpler example

cobalt pelican
#

Another thing to be aware of is that caching works on a best effort basis. It's not guaranteed on following runs. For example, if you're low on disk space things get pruned automatically.

green otter
#

@cobalt pelican what do you think? 🙂

     #[DaggerFunction('phpunit')]
     public function phpunit(
         #[DaggerArgument('Path to your project')]
         Directory $source,
     ): string {

        return $this->base($source)
            ->withExec(['composer', 'install'])
            ->withExec(['./vendor/bin/phpunit'])
            ->stdout();
     }
     
     private function base(Directory $source): Container
     {
      
        $composer = $this->client->container()->from('composer:2');

        $container = $this->client->php()->cli('8.1');
        foreach(['pdo_mysql', 'gd', 'gmp', 'intl', 'exif', 'zip'] as $ext) {
            $container = $this->client->php()->withExtension($container, $ext);
        }

        return $container
            ->withMountedDirectory('/tmp/app', $source)
            ->withWorkdir('/tmp/app')
            ->withMountedCache('/tmp/app/vendor', $this->client->cacheVolume('composer-volume'))
            ->withFile('/usr/bin/composer', $composer->file('/usr/bin/composer'))
            ->withEnvVariable('COMPOSER_ALLOW_SUPERUSER', '1');
     }
#

base() gives the runtime for the application environment (extensions ..etc)

cobalt pelican
#

You're missing the cache volume for composer's cache. That's most important.

green otter
# 
            ->withMountedCache('/tmp/app/vendor', $this->client->cacheVolume('composer-volume'))
cobalt pelican
#

As for the vender cache volume, you need a more specific cache key as you don't want another module to reuse the same vendor dir.

green otter
#

Okay.

cobalt pelican
#

And the cache key for composer's cache dir needs to be specific to what it's compatible. So for example, php's version or the tools' major version, to avoid incompatibilities in different modules. That depends on how compatible composer's cache is between which versions. Maybe it's different between alpine and ubuntu so the base image should probably be added as well. Depends.

green otter
# 
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano']);

This is very cumbersome (annoying)..

Can I do the bash -c trick and make it smoother string?

#

@cobalt pelican 😄 I just learned I can pass an argument to ->terminal() by default it was 'sh' and now I'm able to get 'bash' 😄

->terminal(['bash']);
#

How to make bash the default shell, after installing bash from apt-get? (general linux Q)

            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash'])
#

I tried this
->withExec(['chsh', '-s', '/bin/bash', 'root']);

and then ->terminal() but it's still defaulting to 'sh'

#

interesting!!
I checked this

root:x:0:0:root:/root:/bin/bash

but when i do ->terminal() I still get 'sh' 🤔

cobalt pelican
green otter
#

Yea, when I do ->terminal() I'm getting 'sh' shell dropped

cobalt pelican
#

There's a withTermalDefault.... something.

green otter
# 
        $container = $this->client->php()->cli('8.1')
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash'])
            ->withExec(['chsh', '-s', '/bin/bash', 'root'])
            ->terminal();
#

LOL wait - wrong container and stuff.

#

That code wasn't running on this dagger call 😅

cobalt pelican
# 
        $container = $this->client->php()->cli('8.1')
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash'])
            ->terminal(["bash"]);

or

        $container = $this->client->php()->cli('8.1')
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash'])
            ->withDefaultTerminalCmd(['bash'])
            ->terminal();
green otter
#

uhhh

# 
     #[DaggerFunction('phpunit-debug')]
     public function phpunitDebug(
         #[DaggerArgument('Path to your project')]
         Directory $source,
     ): Terminal {
        return $this->base($source)
            ->terminal();
     }
     
     private function base(Directory $source): Container
     {
      
        $composer = $this->client->container()->from('composer:2');

        $container = $this->client->php()->cli('8.1')
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash']);
        
        foreach(['pdo_mysql', 'gd', 'gmp', 'intl', 'exif', 'zip'] as $ext) {
            $container = $this->client->php()->withExtension($container, $ext);
        }

        return $container
            ->withMountedDirectory('/tmp/app', $source)
            ->withWorkdir('/tmp/app')
            ->withMountedCache('/tmp/app/vendor', $this->client->cacheVolume('composer-volume'))
            ->withFile('/usr/bin/composer', $composer->file('/usr/bin/composer'))
            ->withEnvVariable('COMPOSER_ALLOW_SUPERUSER', '1');
     }
#

I'm running ->terminal() and I don't get a shell at all, I get this

# 
✔ Koel.phpunitDebug(
    source: ✔ ModuleSource.resolveDirectoryFromCaller(path: "."): Directory! 0.0s
  ): Terminal! 3.5s
✔ Void.websocketEndpoint: String! 0.0s

Error: exited with code 1

Output:
cobalt pelican
green otter
#

I should be able to run ->terminal() in the code .. no problem ..

#

I'm doing it on another container, and it's working fine.

     #[DaggerFunction('phpstan')]
     public function phpstan_debug(
         #[DaggerArgument('phpstan')]
         Directory $source,
     ): Terminal {
        return $this->client->container()
            ->from('jakzal/phpqa:latest')
            ->withExec(['apt-get', 'update'])
            ->withExec(['apt-get', 'install', '-y', 'nano', 'bash'])
            ->withMountedDirectory('/tmp/app', $source)
            ->withWorkdir('/tmp/app')
            ->terminal();
     }

This works fine ..

cobalt pelican
#

Wait, are you using a build from main or a released version of dagger?

green otter
#

that image starts from jakzal/phpqa and ->terminal() works fine ..

but when I start from php:debain on the base() function it doesn't work 🤔

cobalt pelican
#

Which version are you on?

green otter
#

0.11.8

cobalt pelican
green otter
#

uhh - I've ran ->terminal() on some ubuntu image ... and it works
I run ->terminal() on another ubuntu image .. and it doesn't work tho

cobalt pelican
#

Do you get an error?

green otter
#

no

#

empty output

cobalt pelican
#

I'm surprised to see Void.terminal, what's the code you have now?

#

Use withoutEntrypoint to make sure the base image doesn't have something prepended to the command.

green otter
#

ok I figured it out

#

this is alpine, and I'm trying to do apt-get on it ... silly mistake .. but.. the issue here is

#

@cobalt pelican here

image.png
#

So the ->withExec() calls were failing .. but silently failing .. and when I tried to run ->terimnal() it somehow didn't handle the failures and gave empty output

#

ok so.. if I call ->stdout() on the container .. i see the error 'apt-get command not found' ... but if I call ->terminal() I get no output and an exit code 1

cobalt pelican
#

That's a known issue but I thought it was fixed. Probably fixed in main with the new changes.

green otter
#

I'm pulling 0.11.9 now

cobalt pelican
#

It's not in 0.11.9 though, the new changes are going to be in 0.12.

green otter
#

How can I get the latest main?
I'm installing using

curl -L https://dl.dagger.io/dagger/install.sh | BIN_DIR=$HOME/.local/bin sh
cobalt pelican
#

We don't have nightlies, but you can build it yourself if you have a clone of the dagger repo. I also think there's a module somewhere that does it too, but don't know where. If you have the clone it's easy enough though.

green otter
#

Okay. I tested with 0.11.9 and it doesn't fix this known issue. At least I know about it now. It was a hidden gotcha

obsidian forge
#

you can do DAGGER_COMMIT=... in an env var

#

just put the commit from main that you want to download

green otter
#

@obsidian forge got a full CLI command I can run here? 🙂 please. (with the env var and all)

cobalt pelican
#

That's just the CLI right @obsidian forge? How does it handle the corresponding engine?

obsidian forge
#

curl -L https://dl.dagger.io/dagger/install.sh | DAGGER_COMMIT= f39e3883718808d5e026cb7fe2c0bac72ed8dd37 BIN_DIR=$HOME/.local/bin sh

cobalt pelican
#

Oh, don't we have a published engine in every push to main?

obsidian forge
#

i've started daily driving off of main to try and find more random things when using dagger day-to-day

green otter
#

I've created this dagger function called base-test which does ->stdout() and will show me any errors .. so when ->terminal() fails (with this known empty output) then I can go to base-test and see exactly what's wrong... Until we're on 11.0 🙂

#

@cobalt pelican how can I see the actual output of the ->withExec() in real time? on my CLI?

For example I'm doing apt-get install and I want to watch it .. right now Dagger CLI is just showing the timer clock going up

cobalt pelican
green otter
#

ok fair enough .. and that'll shuttle across the STDOUT in real time 🙂

cobalt pelican
# green otter

If you expose base as a function then you can do either dagger call base stdout or dagger call base terminal. No need for the extra debug functions.

green otter
#

@cobalt pelican good point! 🙂 I'll consider that.

#

Is this safe?

     private function cmd(string $cmd): array
     {
        return ["/bin/sh", "-c", $cmd];
     }
#

writing this out by hand is terrible DX

           ->withExec(['apt-get', 'install', '-y', 'nano', 'bash']);
cobalt pelican
#

Yeah, that's fine. Some users do it.

green otter
#

I wanna do
$this->cmd('apt-get install -y nano bash')

#

cool. I might even include it in the PHP SDK out of the box ..

#

@summer sequoia what's your thoughts on adding this utility function into our PHP SDK... so when we do dagger init we have it?

Maybe we can include it as a standalone function, and import it at the top of the file?

use function Dagger\Utility\dagger_cmd

cobalt pelican
#

I suggest trying to keep parity with other SDKs to avoid drift.

green otter
#

@cobalt pelican that's a good point .. so what do the other SDK's do in this case? 🙂

cobalt pelican
#

Recommendation is to embrace the array syntax because it makes it really clear what the arguments to commands are, and you don't need a shell to execute so there's less overhead. Some people add a simple function like you did for convenience when they don't care, but that's in userland. You need to know what's the available shell btw.

green otter
#

Understood, thanks for sharing. Insightful

#

I'm gonna pause for now .. Gotta get back to CEO duties but today has been great progress .. and lots of knowledge handover ..

Thanks @cobalt pelican you've been awesome 🤝 💪

cobalt pelican
#

Dockerfile supports both, whether you use array syntax or not but as I understand it, it's a source of complication in that code base as there's a need to handle various edge cases. Also, not viable to accomplish in withExec because of the type safety. And lastly, best to keep SDK-side utilities to a minimum to avoid maintenance burden on all SDKs in order to avoid drift. There's a general purpose with that PHP could implement though (for reference: https://archive.docs.dagger.io/0.9/592101/custom-callbacks).

cobalt pelican
green otter
#

I'll share my workshop samples soon... for peer review .. as the next thing I'm going to back asking you + the team is "what things should be in the dagger worksho" for people to start adding Dagger into their existing projects

Examples are

  1. Taking existing Dockerfiles, and importing them into your Dagger file, and doing docker build that way

So moving away from "building reudable modules" to "replace your current app's CI/CD shit, with Dagger sexyness" is my angle ..

Have a think and we can catch up soon .. and each scenario we think is important to teach the community, I'm make sure to have PHP samples made for this workshop .. and I think it'll be helpful for moving forward.

Cheers 👍

summer sequoia
green otter
#

Workshop Topics - Thread

green otter
#

I had a great call today with @obsidian forge - answering many of my open dagger core engine questions, and talking about developer flows and thus workshop things 🙂

green otter
#

Hey @cobalt pelican 🙂

#

You around for some sync on workshop topics ? 🙂

cobalt pelican
cobalt pelican
green otter
#

@cobalt pelican all the different topics, and things to cover.

After running phpunit, and adding a withServiceBinding() for redis/mysql...

then what?

It's a 4 hour workshop, so I need plenty of materials to take them throguh

green otter
#

@cobalt pelican I took that Laravel 11 web page and have a functional system using it 🙂

It's time to start adding Dagger into it 🙂

summer sequoia
#

@green otter how dod the workshop go?

green otter
#

@summer sequoia it went well! I learned a lot, about how people learn and interpret what Dagger is

summer sequoia
#

Any pain points in the Dagger SDK we could look at addressing?

green otter
#

Yes. I'm back home on Thursday so we can have a sync then

green otter
#

@summer sequoia the number one thing right now is no real-time STDOUT output. Helder says this is OpenTelemetary related, so I think this should be our next immediate focus

#

@cobalt pelican @grim stump @shy goblet we should discuss on merging the current PHP SDK now .. since the PR now contains 1+ month of work.

What are good next steps? Could we have a repo such as github.com/dagger/php-sdk and then we (dagger team + me + carnage/charjr) can continue to develop on that repo, and manage future PRs from the community

LMK your thoughts

shy goblet
#

Hey! Congrats! Can I get the link of the PR so I can review it properly

green otter
cobalt pelican
summer sequoia
#

John has been adding support for arrays and objects over the past few days

#

is there a php library for the open telematry or do we need to write one?

cobalt pelican
green otter
#

@summer sequoia can you update the PR to "ready for review" as it is 🙂

#

We can add OTel after we get this existing PR merged (either into dagger/dagger, or into dagger/php-sdk)

#

Which direction do you guys want to take @cobalt pelican ? (dagger/dagger, vs dagger/php-sdk repo)

summer sequoia
#

Done

cobalt pelican
green otter
serene light
#

dagger functions

echo               Echo the value to standard output
echo-simple-list   Echo the value to standard output
grep-dir           Search a directory for lines matching a pattern

Skipped 1 function(s) with unsupported types: echo-nested-list

Does dagger support nested lists? i.e. [[String!]]

#

Also is there somewhere in the documentation that I can refer to for what types are/aren't supported?

cobalt pelican
#

But should still be supported when used from another module, so it's good if the SDK can support it.

serene light
serene light
#

To support enums I need to upgrade the sdk to version 0.12.0 but, when running dagger develop on my module I encounter this issue:

Error: failed to get module SDK: input: moduleSource.resolveFromCaller resolve: failed to collect local module source deps: failed to get git module sdk: failed to load sdk module github.com/carnage/dagger/sdk/php@charjr-add-php-runtime: select: failed to initialize module: failed to call module "php-sdk" to get functions: call constructor: process "go build -o /runtime ." did not complete successfully: exit code: 1

Stderr:
# php-sdk
./main.go:20:17: undefined: Directory
./main.go:22:17: undefined: Container
./main.go:28:16: undefined: Directory
./main.go:46:60: undefined: ModuleSource
./main.go:46:93: undefined: File
./main.go:46:101: undefined: GeneratedCode
./main.go:58:64: undefined: ModuleSource
./main.go:58:97: undefined: File
./main.go:58:105: undefined: Container
./main.go:125:66: undefined: ModuleSource
./main.go:125:66: too many errors

It seems that the sdk/php/runtime/main.go is unable to find the definitions that should normally be generated.

This is only an issue on 0.12.0 I can successfully call dagger/develop on 0.11.9.

I've noticed 0.12.0 does introduce breaking changes, but I'm having trouble pinpointing what change I'm butting heads with.

Any help would be greatly appreciated.

cobalt pelican
serene light
obsidian forge
#

that can happen if the go.mod doesn't match up your imports

#

or if it's happening randomly in ci, it means that you're hitting an absolutely miserable flake 😛

serene light
#

Right, I apologise, I'm quite new to Go

cobalt pelican
#

go.mod says php-sdk so it should be php-sdk/internal/dagger.

serene light
#

Ahah! Thank you @obsidian forge @cobalt pelican!

I've learned a bit more of go and you've solved my issue! Now I should be able to get Enums in 🙂

obsidian forge
#

yay 🎉

#

honestly, some of the go stuff around packages is very tricky to understand, it's designed in a ... particular way

serene light
green otter
#

Niceone @serene light

serene light
#

I'm trying to make the changes to the sdk/php/dagger.json mentioned in the PR:
https://github.com/dagger/dagger/pull/7493#discussion_r1657470819
https://github.com/dagger/dagger/pull/7493#pullrequestreview-2186514247

I'm noticing that, no matter what I do, I can't break the module. I literally changed the sdk/php/dagger.json to this:

{
  "name": "php-sdk",
  "sdk": "go",
  "include": [
  ],
  "exclude": [
      "."
  ],
  "source": "runtime",
  "engineVersion": "v0.12.0"
}

my natural assumption would be this would exclude everything, and it should blow up in my face. But it worked as normal, as if it had all the files in the container.

GitHub
[WIP] Add php runtime by carnage · Pull Request #7493 · dagger/dagg...

Very minimal attempt at a PHP runtime for module support.
To the best of my knowledge, the go part of this PR is complete & working.
Outstanding is to write the PHP code to

Create a module ent...

#

My first guess is that, I've missed a vital step in sdk/php/runtime/main.go that reads it. But I haven't had any luck discovering it. Any help would be greatly appreciated.

cobalt pelican
#

runtime/main.go doesn't read that. It's the engine when loading the runtime module. So that happens before loading those files. I'd just dagger call the runtime manually and in the cli, terminal into the source dir and see what files were loaded under /src.

#

@serene light Like this:

Screenshot_2024-07-19_at_13.15.56.png
#

@serene light, note that if you increase verbosity in the TUI, you can see the complete set of include/exclude patterns.

Screenshot_2024-07-19_at_13.17.22.png
#

Notice also that only src is included if I only add that. I had to exclude "vendor" because I think that's always being included by default by the Go SDK.

Screenshot_2024-07-19_at_13.25.03.png
#

@serene light, basically, just continue to add only what you need in include and avoid exclude. Easier to manage.

serene light
#

I've literally put in the following:

{
    "name": "php-sdk",
    "sdk": "go",
    "include": [
        "dagger.json"
    ],
    "exclude": [
        "src/",
        "vendor/"
    ],
    "source": "runtime",
    "engineVersion": "v0.12.0"

}

then with dagger call source-dir terminal -> ls
I get the following:

dagger.json

So I've got basically nothing in there at all.

But then I initialise a new module based on this, and I can still call echo, grep-dir, no problems

#

What should be "breaking" when I don't include the right files?

green otter
#

@serene light i got your DM - let's have a sync today

green otter
#

@serene light @summer sequoia is there a time today that works for you guys, for a sync, on this topic?

#

@cobalt pelican question - why are you suggesting singleton .. is this actually so it re-uses a connection, i.e: for performance reasons? or is there another motivation about moving to Singleton .. (assuming I've interpreted everything correctly)

cobalt pelican
#

Is there an example of doing something similar in a popular framework, like for example using a database connection?

serene light
#

@green otter We'll be having a sync at 2pm - 2:30pm, so you should be able to join then

green otter
cobalt pelican
green otter
#

my bad - i read it wrong

#

multi-tasking

cobalt pelican
#

The point of this is to just not require passing that client instance around in function parameters, but the DX should be as idiomatic as possible for users, that's why I'm interested in knowing how popular PHP frameworks/libraries do something similar to this.

green otter
#

@cobalt pelican I understand. I'll sync with @serene light and agree on the right implementation here. Thanks for your initative, we'll work on it more 🙂 to improve DX

summer sequoia
green otter
#

@serene light @summer sequoia send me a calendar invitation, please, so it's blocked out.

summer sequoia
green otter
#

composer.json


{
    "autoload": {
        "files": ["functions.php"]
    }
}
#

dump-autoload

#

autoload-dump

#

@cobalt pelican so we're moving towards a nice and simple dag() function which returns the singleton instance 🙂

This will make the PHP SDK user experinece the same as Golang

See the bottom right (creation of the function), and on the top left the function is imported, and then you can do dag() in your dagger module

image.png
green otter
#

@serene light @summer sequoia I'm out of meetings now - are there other topics we should discuss? I'm available again, now

serene light
#

I think we're fine at the moment, I'm just cleaning up what we discussed earlier, that should clear up every suggestion from the PR

green otter
#

@serene light thanks a lot. QoL improvement for sure dag()

serene light
#

@cobalt pelican We've got the SDK working to the best of our current knowledge and rebased to latest. Is there any pressing issues we need to resolve for this to get merged? If so I'm free to talk through them, see if we can get it over the line 🚀

cobalt pelican
serene light
cobalt pelican
#

I’ve marked resolved what I can atm, see the remaining ones while I try it out.

serene light
#

I've removed the comment, I have got objects working fully in a separate branch.

The current plan is to get this merged, then follow up with a second PR supporting custom objects.

EDIT: The plan can change to include it in this PR if that is preferable, but it does make quite a few additions, so it's probably easier as a second PR

summer sequoia
#

custom object support should come in a followup. I've also realised we've got a bug in the existing implementation - it'll blow up if you return a custom object which doesn't have a #[DaggerObject()] attribute on it.

serene light
#

Ah. Then yes, we'll keep that to a follow-up PR while we squash the bug.

cobalt pelican
#

What value have you been using on --sdk=?

serene light
#

I've been putting this in the dagger.json of modules:

"sdk": "github.com/carnage/dagger/sdk/php@add-php-runtime"

#

I've not been using the --sdk option itself, but I assume it takes the same value? @summer sequoia

summer sequoia
#

yes

cobalt pelican
summer sequoia
cobalt pelican
#

@serene light, I did a review round for now while I take a deeper look at the patterns so you can make progress while I do that.

#

Uh 😕

Screenshot_2024-07-25_at_14.12.19.png
#

Oh, I think when the source is a git remote those excludes don't apply. \cc @worthy ice @ocean creek

serene light
#

Sorry, for the delay, something came up I had to deal with. I'm back now and will look into the review

green otter
#

@cobalt pelican I've never ran the SDK as a module.

We are doing

dagger init --sdk=githubUrl .

cobalt pelican
green otter
#

I didn't think it was a module.. but a runtime SDK with "generated" code instead

#

Anyway, I'm sure in time I'll understand why an SDK is a module

cobalt pelican
#

The SDK runtimes are built as normal modules. Go modules in this case.

#

The Go runtime is the only one that's not implemented as a module because someone needs to be the bootstrapper.

serene light
#

@green otter
From the sdk/php/dagger.json

{
    "name": "php-sdk",
    "sdk": "go",
    ...
}

The SDK is actually just a module built from the Go SDK, or at least, that's my understanding of it

green otter
#

Okay, I get it.

#

Thanks

serene light
#

@green otter One of the review suggestions is that we rename the dagger script to entrypoint.php. I think I've done it...

In sdk/php/main.go I changed this:

const (
    ...
    RuntimeExecutablePath = "entrypoint.php" // was "dagger"
    ...
)

And when finding the src directory I changed it

    private function searchDirectory(string $dir): ?string
    {
        return file_exists("$dir/entrypoint.php") && is_dir("$dir/src") ? // was "$dir/dagger"
            "$dir/src" :
            null;
    }

Can you think of any other places we used the name?

#

I wasn't involved in its development quite as early as you were, so I'd just like to double check if that sounds about right

green otter
#

I agree with this change. It's the right thing to do.

Implicit vs Explicit

serene light
#

Definitely

#

It seems to work with what I changed in my branch

summer sequoia
#

it will break any existing modules though as it's generated by dagger init

cobalt pelican
#

It hasn't been merged yet and you already worry about existing modules 😅

green otter
#

Merge it! Nobody is using the PHP module yet

radiant flume
#

Hi guys. I have a question. If I run dagger init --sdk=php in an existing PHP project, how well will the module source code co-exist with the existing project?

summer sequoia
#

pretty well, by default it'll stick everything in a dagger sub directory

#

only a dagger.json ends up in the root

summer sequoia
green otter
#

Hey @radiant flume

It will work like the other SDK's, and it will drop all the dagger files into standard dagger folder structure

dagger.json
dagger/
  ... sub dirs with SDK code, and your module code.

Does this answer your question? do you want to know?

PS: we can't do --sdk=php yet, as it's not been merged by the dagger core team, so we're using @summer sequoia's github fork, meanwhile 👍

summer sequoia
#

we wont be able to do sdk=php once merged either -> needs to wait until php becomes an officially supported sdk or that there is agreement on how communiy skds are aliased

green otter
#

I'm aware.

green otter
green otter
#

@cobalt pelican what's outstanding for merging the current version of PHP SDK? we have new features we want to add, but are keen to have the current PR merged.

Please let me know 🙂

cobalt pelican
green otter
#

@cobalt pelican right. I thought @serene light and @summer sequoia said they had fixed everything.

cobalt pelican
#

I see changes pushed 1h ago. Is it done?

serene light
#

@cobalt pelican I was looking at what I needed to change to rename the "codegen/" directory to "sdk/" but overall it should be done. I'll have a quick check

cobalt pelican
serene light
cobalt pelican
serene light
summer sequoia
#

I think the pr allows changes by maintainers

cobalt pelican
#

I know, I just meant your consent.

#

Don't want to cross wires.

serene light
#

I'm fine with it, I was still working it out, so if you know what to do I'm sure you'd get it done a lot sooner

#

I am done for today, so I wont be committing anything until tomorrow

cobalt pelican
#

For the next few hours I'll be afk, but I'll see if I can finish it up quickly.

serene light
#

Referring to this discussion: https://github.com/dagger/dagger/pull/7493#discussion_r1697254329

generated/ (codegen should be able to run with this being empty so it generates it from a clean slate)

We completely agree with this change. But it seems the PHP SDK, in its current state, depends on an already generated Client, in order to generate the code. (We're guessing this is why the generated/directory is committed to the dagger git repository as well) .

This is the error I get if I try to call dagger functions without the generated/ directory included:

! failed to initialize module: failed to call module "potato" to get functions: call constructor: process "/src/dagger/entrypoint.php" did not complete successfully: exit code: 255
  ┃                                                                                                 
  ┃ Fatal error: Missing code generated dagger client in /src/dagger/sdk/src/Dagger.php on line 25  

Error: input: module.withSource.initialize resolve: failed to initialize module: failed to call module "potato" to get functions: call constructor: process "/src/dagger/entrypoint.php" did not complete successfully: exit code: 255

Stdout:
Fatal error: Missing code generated dagger client in /src/dagger/sdk/src/Dagger.php on line 25
GitHub
Add php runtime by carnage · Pull Request #7493 · dagger/dagger

Very minimal attempt at a PHP runtime for module support.
To the best of my knowledge, the go part of this PR is complete & working.
Outstanding is to write the PHP code to

Create a module ent...

cobalt pelican
serene light
# cobalt pelican Thanks for letting me know! I made quite a few changes yesterday but there's one...

Ah fair enough! I did push one change but it's relatively small. I changed the New function to be similar to what you wanted

func New(
    // Directory with the PHP SDK source code.
    // +optional
    sdkSourceDir *dagger.Directory,
) *PhpSdk {
    if sdkSourceDir == nil {
        // TODO: Replace with a *default path from context* when
        // https://github.com/dagger/dagger/pull/7744 becomes availble.
        sdkSourceDir = dag.Directory().WithDirectory(
            "/",
            dag.CurrentModule().Source().Directory(".."),
            dagger.DirectoryWithDirectoryOpts{
                // NB: these patterns should match those in `dagger.json`. When `--sdk` points to a git remote
                // the files aren't filtered using `dagger.json` include/exclude patterns since the whole repo is cloned.
                // It's still useful to have the same patterns in `dagger.json` though, to avoid the unnecessary uploads
                // when loading the SDK from a local path.
                Include: []string{
                    "composer.json",
                    EntryPoint,
                    "generated",
                    "init-template.sh",
                    "LICENSE",
                    "README.md",
                    "src",
                    "template",
                },
                Exclude: []string{
                    "vendor",
                },
            },
        )
    }

I hadn't gotten round to moving the files though. I will leave that alone for now.

EDIT: I also rebased the branch to keep up with main

cobalt pelican
#

That's ok, I'm going to force push my changes now (rebased on top of main).

#

@serene light, I just pushed. Please test it out and check if everything's ok. It seems the sign-offs in your commits aren't there though.

green otter
#

Things are getting juicy 🧃

cobalt pelican
#

Have you guys thought about showing off the PHP SDK in our community call? \cc @tropic whale

serene light
cobalt pelican
serene light
#

Why should the variable sdk be called m? I noticed it in Python's sdk but it made more sense to me to be called sdk since it was the sdk performing the action.

Is this a Go convention I'm unaware of?

I only ask because it might help me understand it a bit better.

cobalt pelican
summer sequoia
#

how do you add signoffs for existing commits? we've got 3 people who contributed to this branch so coordinating signoffs will be troublesome

cobalt pelican
#

You'd have to change the history and edit those commit messages. Didn't notice the DCO before, I can just push through the DCO this time, so we can get this merged.

#

@serene light please let me know if everything looks good on your end.

serene light
#

@cobalt pelican I've done some quick tests and it all seems good! Thank you for the help

cobalt pelican
serene light
#

I think we're all good, I will make sure to sign off the custom objects one after I check it's rebased and working

cobalt pelican
#

Merged 🥳

serene light
#

Woo! That's brilliant, hopefully, many smaller PRs from now on 🙂

summer sequoia
#

Party!

obsidian forge
#

woooooo 🎉

#

hype hype ❤️

cobalt pelican
#

@serene light, I changed the location of the codegen script but didn't consider to adjust the generate task in CI as well. I'll send a fix shortly.

obsidian forge
#

out of curiosity ... this is gonna be in the next release yup?

#

could someone add a changelog entry if so?

cobalt pelican
obsidian forge
#

ahhh i see, it's not bundled

cobalt pelican
#

Still, need a change log for it.

#

I'll include it in the generate fix.

obsidian forge
#

@cobalt pelican wdyt about the idea of having aliases for "sdk" - e.g. php could alias to the full github.com link

obsidian forge
cobalt pelican
#

But we need a prefix instead of just php.

serene light
#

👀

obsidian forge
#

let's thread 🧵

cobalt pelican
obsidian forge
#

@tidal geyser can we get an "SDK PHP" team in github? then we can add it to CODEOWNERS like we have for elixir/etc

#

(would also be nice to have "SDK Go/Python/Typescript as well)

tidal geyser
#

(would also be nice to have "SDK Go/

serene light
green otter
green otter
#

it's party time - get your daggers out 🗡️

#

we sign our commits in blood

summer sequoia
#

Next steps are

  • Custom object support
  • migrate ci pipeline for the sdk to a dagger module
  • add open telemerty for realtime feedback
#

I've got a speaking slot at PHP Leeds in September and plan to show off our work here

cobalt pelican
#

Anyone willing to demo the PHP sdk tomorrow at the Community Call?

summer sequoia
#

What time is it?

cobalt pelican
green otter
#

@cobalt pelican I can't make tha time, sorry. I was planning/intending to do some nice demos, but I'll be travelling on that day/time so I can't.

Happy to do another week. If Carnage wants to do the demo next week Thu, then fine by me

serene light
#

Next steps are

summer sequoia
#

@Helder Correia I can't make tha time,

serene light
#

I'm trying to make a sdk/php/dev module capable of running the sdk/php tests: https://github.com/dagger/dagger/pull/8078/files

  • I can run unit tests currently with : ~/git/dagger/sdk/php/dev $ dagger call unit-test --source=../ stdout
    • I had to add a source argument as it is currently the easiest way I could think of getting the sdk/php directory inside the container.
  • The integration tests fail. My guess being that they are not connecting to dagger properly. Could anyone point me in the right direction for getting these integration tests to connect to dagger from inside a module?
GitHub
feat: Add PHP SDK dev module for running tests by charjr · Pull Req...

Add a module under sdk/php/dev, using the PHP SDK, to test the PHP SDK.

Fix previous refactoring oversight left in existing unit test.

Implement unit-test function to run unit tests only
Impl...

summer sequoia
cobalt pelican
serene light
cobalt pelican
serene light
serene light
#

Sorry, just to clarify, I do know PHPUnit is at least successfully running. I can run the unit tests, as they do not depend on Dagger\Client or any other generated/ classes. They failed when they were meant to and passed when they were meant to.

With the Integration tests, they depend on Dagger\Client and their failures are identical to if you run them locally through a simple $ ./vendor/bin/phpunit --group=integration. In other words, they are failing in the same way as they would if you do not run them through docker-compose

summer sequoia
#

docker-compose presumably adds some setup/additional services eg a dagger engine then - replicate that in the dagger module?

cobalt pelican
#

You don’t need to manually add the engine to the dev module.

summer sequoia
#

what about for containers spawned by the dev module?

serene light
#

the docker-compose file doesn't actually work atm, I'm seeing if I can fix it.

#

Well, seeing if I can fix it quickly, since we're planning to do away with it right now (just so I can run the tests do pass in the right env)

serene light
#

@cobalt pelican Would it be best if I limit this first PR to simply making a Dev module that successfully performs unit tests?

Then I can get linting, static analysis and integration tests in follow-ups

cobalt pelican
serene light
cobalt pelican
#

Ah, I see. I'll help you. Taking a looksy.

#

@serene light, are all your changes pushed to the PR?

serene light
#

All the changes I think I needed for the PR at least.

The integration test command looked identical to the unit tests one currently though

#

I will put it back in.

#

Okay it's back in

cobalt pelican
#

Was the module initialized correctly? I don't see the composer.json and entrypoint.php files.

serene light
#

It is, sorry I'll include those... I figured they would be generated when called. Am I misunderstanding?

cobalt pelican
#

What shouldn't be committed to git is what the runtime adds to WithVCSIgnoredPaths, which in your case is sdk and vendor. Everything else should be committed.

#

The template should only be applied when the module is being created for the first time.

#

For now you can just check if the file exists and create it if not.

#

But shouldn't overwrite an existing file.

#

You may add entrypoint.php to WithVCSGeneratedPaths.

#

For composer.json you could add PHP dependencies for calling a library in your module's function, for example.

#

The lock is important for reproducibility.

#

Just call dagger develop -m dev (from sdk/php). That should update the module from codegen.

serene light
#

Okay, that all makes sense. I'll make those changes now

cobalt pelican
#

Currently, the php functions in src/ are being overwritten by the template.

#

But that's because you removed composer.json.

#

Since init-template.sh is using that to check if it's a new module or not.

serene light
#

Right I see, obviously it was working locally since I had the files... but simply hadn't commited them.

cobalt pelican
#

I see

#

There should also be .gitignore and .gitattributes files. It's not committed.

serene light
#

Added them just now

cobalt pelican
#

A dagger develop -m dev in a clean workdir shouldn't create new changes to commit, assuming you ran it soon before with the same dagger version.

serene light
#

Yep okay, seems to be identical

#

Thank you, I get it now

cobalt pelican
#

Btw, there's a weird extra space in your indentation after the annotations ("public function" lines) 🙂

#

@serene light does the unit-tests function return successfully for you?

#

There's 33 errors when I run it. All seem related to autoloading, basically not finding some classes.

serene light
#

It is running successfully for me.

cobalt pelican
#

Then you must have something in your host that I don't have.

#

Try removing your sdk/php/vendor directory

#

The php:8.3-cli-alpine image doesn't include composer, right?

#

Don't you have to get composer and composer install before running phpunit?

serene light
#

🤦‍♂️

cobalt pelican
#

Btw, you can create a function base that doesn't have the [#DaggerFunction] annotation, so you can just call from unitTests something like: return $this->base()->withExec(['phpunit', ...]).

#

Basically an internal function, not exposed to dagger.

serene light
#

That's a good idea, I was planning to do something like that if it turned out the logic was duplicated among each function

#

I will call it base()

cobalt pelican
#

At some point you could create a reusable php module that'll provide a base container with composer installed so it's much easier for any PHP module to get a PHP container quickly. Then you could do:

dag()->php()->withSource($source)->container()->withExec(["phpunit", ...])
#

The way to hook that in is, let's say you put that module in sdk/php/modules/php, then dagger install -m dev modules/php (or in another git repo).

#

That'll add an entry for the php module in your dev/dagger.json and the client bindings for it will automatically be generated so you'll have that dag()->php().

serene light
#

Alright, I have added a base() function and the unit tests now run without having vendor/ already installed.

cobalt pelican
#

You should add vendor and sdk to the excludes in dagger.json.

#

That would have caught the issue sooner for you.

#

Actually, scratch that, your issue comes from the Directory --source argument. That would be fixed by adding a view.

#

But that'll be replaced soon with context directories.

#

You'll need to add support for it in the PHP SDK btw.

#

@serene light, does the PHP SDK have support for a module constructor?

#

Doesn't seem like it.

serene light
#

I don't think we do yet

cobalt pelican
#

That's easy to add and adds great value.

serene light
#

So should I still add vendor and sdk to dagger.json excludes?

summer sequoia
#

I wasn't sure what a module constructor did so had left it out so far

cobalt pelican
summer sequoia
#

Needs an update to the latest sdk

#

I also want the functions to return an object not a container so you can do things like dag()->php()->withExtension('xdebug')->withComposer()->container()

cobalt pelican
summer sequoia
#

oh, so it's only for the module base object, not for sub objects?

cobalt pelican
#

All you have to do is similar to the way you already register functions for an object:

                $func = dag()->function(
                    $daggerFunction->name,
                    $this->getTypeDef($daggerFunction->returnType)
                );

But you only need this for the main object, the name should be empty (""), and you shouldn't need the [#DaggerFunction] annotation, assuming you can easily detect if the code defines the constructor or not.

#

That includes registering the constructor's arguments, but that should be the same as for any other function.

summer sequoia
#

we could default to using the object's construct method but that might be needed in future to pass in dependencies

cobalt pelican
#

Then on invocation, you need to check if the function name is "" and if so, just invoke that constructor with those arguments to get the object.

cobalt pelican
#

I'm talking about function __construct() { }.

serene light
#

I think the idea is, if we need to inject dependencies into objects, and those dependencies don't/shouldn't need to be provided by the user, then usually they're injected in function __construct() {}

#

We may be able to use the __construct method still, but it's just something to consider

cobalt pelican
#

Yeah, but in that case you'll probably be able to know programmatically which of the constructor args are used for dependency injection, in which case you can exclude those args from being registered to Dagger.

serene light
#

True, it's definitely something we'd want to add.

Obviously the list is stacking up 😅

cobalt pelican
#

It's mostly like any other function, just needs small adjustments. This is just so you have an idea of the effort here, but totally up to you when to get to it.

serene light
#

We'll focus on this Dev module first. Once we have that running on CI, we should be able to tackle the rest of the list with more confidence 🙂

#

But first, lunch 🍲

cobalt pelican
#

Of course 🙂 Just thought about the constructor because you could move the --source arg to there, so there's no need to add it to every function. Meaning also that you could include the composer install step earlier, and the result shared by all functions.

#

For now, I suggest you accept the $source arg in base and do those steps there.

#

For when you come back, I think what you need for the integration tests is to add the experimentalPriviledgedNesting option to the withExec for running phpunit. That'll give you the env vars in that exec for connecting to the engine, which Dagger::connect() should already be able to pick up on.

#

Hmm, you guys may need to make a breaking change in codegen. Optional arguments should be named arguments. Looks to me like they're all positional atm, is that right?

summer sequoia
cobalt pelican
# summer sequoia what do you mean here? named arguments in PHP only applies to calling a function...

I know:

return $this->base($source)->withExec(['phpunit', '--group=integration'], experimentalPriviledgedNesting: true)

What I mean is that codegen doesn't seem to define them as named arguments:

    public function withExec(
        array $args,
        ?bool $skipEntrypoint = true,
        ?bool $useEntrypoint = false,
        ?string $stdin = '',
        ?string $redirectStdout = '',
        ?string $redirectStderr = '',
        ?bool $experimentalPrivilegedNesting = false,
        ?bool $insecureRootCapabilities = false,
    ): Container
#

Don't you need to provide all args until that option currently?

summer sequoia
cobalt pelican
#

Oh, it works like that currently?

summer sequoia
#

yeah

#

thats how php named arguments work

cobalt pelican
summer sequoia
#

the rational for adding it to PHP was to allow skipping over a bunch of other properties in exactly the way you defined

#

any property with a default in the function defintion can be excluded from a call

cobalt pelican
summer sequoia
#

see example 17 on that page

cobalt pelican
#

It's not the same as the way withExec is defined.

summer sequoia
cobalt pelican
#

Ok, looking into the other sections I can see it, like example #9.

#

Defined like $style = "Greek" but used like style: "natural".

#

Cool! 🙂

summer sequoia
# 
htmlspecialchars(
    string $string,
    int $flags = ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401,
    ?string $encoding = null,
    bool $double_encode = true
): string
#

that's the defintion for htmlspecialchars which is used in a number of the examples

serene light
#

There is no step to explicitly define a function as accepting named arguments. Every function can accept named arguments I believe

cobalt pelican
#

Oh I think I misread it! In example #14 (Named argument syntax), I saw:

myFunction(paramName: $value);

But somehow read it as:

function myFunction(paramName: $value) { }
#

Like that had to be in the definition.

serene light
#

Yeah that one is a bit confusing, but basically any function at all that you define, whatever the argument's name is, you can pass it as a named argument

# 
function myFunction(string $thisIsTheName, int $thisIsADifferentOne)

That's it, now I can refer to them by name

cobalt pelican
#

Pretty cool 🙂

summer sequoia
#

I wanted there to be some way of explicitly opting a param of function into being usable as a named parameter as auto opting everything in has some interesting edge cases which are not ideal but the PHP community decided to go ahead with everything everywhere being namable

cobalt pelican
#

In Python it was all opt-in in as well, but a few years ago they added the ability to explicitly split the arguments into required positional and/or required named.

#

I only added the required named to optionals, but decided to allow named for positional args.

cobalt pelican
#

Getting back to the issue at hand, it seems your issue is in the provisioning tests, right? Failing the checksum when downloading a dagger version.

#

You can skip the provision tests for this PR, and solve them later. This way you focus on the dev module and replace the docker compose completely.

serene light
#

I think I've got them running thanks to [your help](#php message)

There's only one test that fails and that is for something we want to deprecate: the CliDownloader

#

It was used before the runtime was created, in order to download dagger directly and use for cli

#

And then set all the environment variables manually

#

Whereas now the runtime sets that up for you just by creating a new module

summer sequoia
#

Is there a good place to keep track of the outstanding work needed on the PHP SDK?

serene light
#

Should we start creating a issues for it on the main repository?

summer sequoia
#

That's one potential option

cobalt pelican
serene light
#

Brilliant, I will get on that shortly and list the follow ups from the original runtime PR

cobalt pelican
#

Ok, let's hook that into CI 🙂

summer sequoia
cobalt pelican
#

Oh, I ask you to make a simple change first. Rename the module from dev to php-sdk-dev please.

#

So it creates a dag.PhpSdkDev() instead of dag.Dev().

serene light
#

Ah, I did see that naming convention in the python one, I didn't realise that was what it was for

serene light
summer sequoia
#

composer.json

#

though that doesn't affect anything

serene light
#

Made the change anyway, thank you

cobalt pelican
#

I'm going to push a few changes to your PR

serene light
cobalt pelican
#

Yay 🎉

Screenshot_2024-08-02_at_14.22.31.png
#

Now you need to fix those checks 🙂

#

@serene light, from the root of our repo, try this:

dagger call -m dev --source=.:default sdk php test

And also:

dagger call -m dev --source=.:default sdk php lint

To generate client bindings:

dagger call -m dev --source=.:default sdk php generate export --path=.
#

Need to pull my changes

serene light
#

bigbrain Learning: What does --source=.:default do?

cobalt pelican
serene light
#

Right, so it avoids looking at anything in that list

cobalt pelican
#

Yes, those files are excluded from what will be sent to your dev module as $source.

#

The only one matching will be sdk/php/dev/sdk.

serene light
#

Right, but with anything in the :default filter excluded?

cobalt pelican
#

But also helps cut down on time when running the command locally because it'll avoid uploading those things in case you have them (e.g., a node_modules).

serene light
#

I was able to run dagger call -m dev --source=.:default sdk php test but not dagger call -m dev --source=.:default sdk php test stdout

cobalt pelican
serene light
#

It did run though (it took about 4 minutes)

cobalt pelican
#

It's not redirecting to yours. See the implementation in your PR, in /dev/php_sdk.php.

serene light
#

Right, I see.. so it only returns nil or error?

cobalt pelican
#

Only sdk php test is hooked into your dev module. Would be great to move the rest there.

#

Yes, if no error, it'll return nil.

#

Output from the test will be available from telemetry. TUI/cloud.

#

But it only matters if it errors, so the output should show up in your screen.

serene light
cobalt pelican
#

Aka codegen. Basically dag()->..... Those are client bindings to functions in the server. With runtimes you started getting more "kinds" of codegen, not just for client bindings. So sometimes I prefer being explicit here.

serene light
#

Right so by performing that command I'm generating bindings to call functions from the php sdk dev module?

#

I see, it's generated Client, Container Directory and Function_ ... but with the correct linting.

cobalt pelican
#

The sdk php generate function runs the SDK's codegen: generated.

#

Only sdk php test is connected to sdk/php/dev.

serene light
#

When I wanted to generate the files before, I was calling dagger develop when inside sdk/php, what is the difference?

#

What difference was made to generate the code with correct linting this time, that wasn't done before?

cobalt pelican
#

I didn't add sdk php generate now, it's been there since the beginning of the PHP SDK. That's how we run codegen (to update bindings) for all SDKs when we make changes to the API. It's included in:

dagger call -m dev --source=.:default sdk all generate export --path=.
summer sequoia
#

Is that needed now we have modules?

#

Feels like an inferior solution

cobalt pelican
#

Yes, but the Go code that makes it possible should just call a function in sdk/php/dev for that. We didn't have it before.

cobalt pelican
summer sequoia
#

well, if it's exported like that you only get the base dagger functions

#

so you cant use modules directly

cobalt pelican
summer sequoia
#

or am I misunderstanding what the generate is doing?

cobalt pelican
# summer sequoia so you cant use modules directly

You can use either. I can generate python bindings from the root of the repo with our /dev module (in Go) that puts all of it together. Or I can run the /sdk/python/dev module (in Python) if I'm in the SDK's directory. In the case of Python, the Go module just calls out to the Python module. It's what I'm saying you guys should do.

cobalt pelican
#

Modules have their own codegen and include bindings to their dependencies.

#

These are two different things.

serene light
#

I think I'm following, but I've been taking in a lot over the last hour or two (thank you for all the help! @cobalt pelican ) so I'm going to be afk while it sinks in.

I added the linting changes. The PHP tests failed in the PR but they are passing locally, I'm not sure why as I can't seem to access the trace on github :/

cobalt pelican
# 
There was 1 failure:
1) Dagger\Tests\Unit\Service\FindsDaggerObjectsTest::itFindsDaggerObjects with data set "test fixtures" ([Dagger\ValueObject\DaggerObject Object (...), Dagger\ValueObject\DaggerObject Object (...)], '/src/sdk/php/tests/Unit/Servi...ixture')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Dagger\ValueObject\DaggerObject Object (
         'daggerFunctions' => Array (
-        )
-        'name' => 'Dagger\Tests\Unit\Fixture\NoDaggerFunctions'
-    )
-    1 => Dagger\ValueObject\DaggerObject Object (
-        'daggerFunctions' => Array (
             'returnBool' => Dagger\ValueObject\DaggerFunction Object (...)
             'returnInt' => Dagger\ValueObject\DaggerFunction Object (...)
             'returnString' => Dagger\ValueObject\DaggerFunction Object (...)
@@ @@
             'explicitlyOptionalFile' => Dagger\ValueObject\DaggerFunction Object (...)
         )
         'name' => 'Dagger\Tests\Unit\Fixture\DaggerObjectWithDaggerFunctions'
+    )
+    1 => Dagger\ValueObject\DaggerObject Object (
+        'daggerFunctions' => Array (
+        )
+        'name' => 'Dagger\Tests\Unit\Fixture\NoDaggerFunctions'
     )
 )
/src/sdk/php/tests/Unit/Service/FindsDaggerObjectsTest.php:25
FAILURES!
Tests: 236, Assertions: 241, Failures: 1.
serene light
#

I made a fresh clone of my branch, still passes. But I can see from what you sent that all it's failing on is getting the array of objects in a different order.

#

So I'll see if I can make that test a bit more robust so it isn't bothered what order they come in, because that is not important to the test.

summer sequoia
#

yeah, it was running fine on my local after pulling your branch as well

serene light
summer sequoia
#

hmmm, that's new

serene light
#

It claims to do exactly what we're after.

serene light
serene light
#

I've updated The PhpSdkDev PR

I unified the test functions into one with an optional --group argument.

I think this PR is ready to be reviewed (and maybe merged)

Then I'm ready to add PHPCS for linting.

GitHub
feat: Add PHP SDK dev module for running tests by charjr · Pull Req...

Add a module under sdk/php/dev, using the PHP SDK, to test the PHP SDK.

Fix previous refactoring oversight left in existing unit test.

Implement tests function to run all tests.

required --so...

green otter
#

nicely done @serene light

serene light
#

feat: Add PHP SDK dev module for running...

serene light
#

@cobalt pelican I've added a PR for linting in the PhpSdkDev. It looks like a lot of changes but it's mostly just automated fixes from the linter.

I also copied the way the python linting command works in the Dev module so it should perform the check in CI too

https://github.com/dagger/dagger/pull/8112

The only files with significant changes are:

  • dev/sdk_php.go ,
  • sdk/php/dev/src/PhpSdkDev.php
  • composer.json

Everything else is linting

GitHub
PHP: Use phpcs for PhpSdkDev Linting by charjr · Pull Request #8112...

Add PHP Code Sniffer

Configured to lint against standards: PSR1 and PSR12

Can be called through the PhpSdkDev.Lint
Added to to Dev module in the same way as Python's Linting works.

cobalt pelican
serene light
# cobalt pelican Just a minor issue and it's good to go.

https://github.com/dagger/dagger/pull/8112#pullrequestreview-2221860777

The codegen logic is still fairly unfamiliar to me, I'm looking at it now, but I'm not sure what I need to do to resolve this feedback.

Do I need to add a simple .WithExec() Or actually dig into the codegen a bit?

GitHub
PHP: Use phpcs for PhpSdkDev Linting by charjr · Pull Request #8112...

Add PHP Code Sniffer

Configured to lint against standards: PSR1 and PSR12

Can be called through the PhpSdkDev.Lint
Added to to Dev module in the same way as Python's Linting works.

cobalt pelican
serene light
#

That part, I assume I want to replace the dagger.Source with t.Base

#

I meant how do I fix the potential linting error that the codegen could cause when the engine gets bumped up a version

#

Would I want to run PHPCBF, the automated fixer for lint errors, on the generated dir? Or do I need to dig into the codegen and fix the error wherever it is generated?

cobalt pelican
cobalt pelican
#

Or to at least make sure that codegen produces an output that doesn't create linting errors, if that's possible.

serene light
#

By delegate codegen to the PhpSdkDev do you mean to replace t.PhpBase() with dag.PhpSdkDev.Base()?

cobalt pelican
#

See Python for example.

shy goblet
serene light
#

feat: compute file digest by TomChv · Pu...

inner walrus
#

Do you use a JetBrains IDE? Guessing maybe the best one for PHP?

summer sequoia
slim thistle
#

hi @summer sequoia, thx for the demo! I tried to replicate your commands but it failed:

$ dagger version
dagger v0.12.4 (registry.dagger.io/engine:v0.12.4) linux/amd64
$ dagger init --sdk=github.com/dagger/dagger/sdk/php demo

✔ connect 1.1s
✔ moduleSource(refString: "demo"): ModuleSource! 0.0s
✔ ModuleSource.kind: ModuleSourceKind! 0.0s
✔ ModuleSource.resolveContextPathFromCaller: String! 0.0s
✔ ModuleSource.withName(name: "demo"): ModuleSource! 0.0s
✔ ModuleSource.withSDK(sdk: "github.com/dagger/dagger/sdk/php"): ModuleSource! 0.0s
✔ ModuleSource.withSourceSubpath(path: "."): ModuleSource! 0.0s
✘ ModuleSource.resolveFromCaller: ModuleSource! 9.7s
! failed to collect local module source deps: failed to get git module sdk: failed to get sdk source for github.com/dagger/dagger/sdk/php: select: determine default branch: failed to run git: exit status 128
Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to get git module sdk: failed to get sdk source for github.com/dagger/dagger/sdk/php: select: determine default branch: failed to run git: exit status 128

any idea what is wrong or what should I do?

inner walrus
#

dagger version dagger v0.12.4 (registry.dagger.io/engine:v0.12.4) darwin/arm64

#

@slim thistle most sources suggest it looks like a git problem.
Wondering if you have a firewall or something that is blocking the anonymous git clone.

slim thistle
#

Ok... I reboot and retried, it works 😅

serene light
#

I've made a format function for PhpSdkDev.

The only gripe I have is it relies on shoving a small shell script in just to circumvent PHPCBF's unconventional exit codes.

   #[DaggerFunction('Format the source directory')]
    public function format(Directory $source): Directory
    {
        $scriptPath = '/usr/bin/format';
        $script = <<<SH
            #!/bin/sh
            phpcbf
            if [ $? != 3 ]
            then
                exit 0
            fi
            SH;

        $original = $this->base($source)->directory(self::SDK_ROOT);

        $formatted = $this->base($source)
            ->withNewFile($scriptPath, $script)
            ->withExec(['chmod', '+x', $scriptPath])
            ->withExec([$scriptPath])
            ->directory(self::SDK_ROOT);

        return $original->diff($formatted);
    }

If an exit code of 0 is not received by the withExec call, then execution stops there and it wont return the diff.

Is there a better way to handle this that I'm not taking advantage of?

GitHub
Disable report for phpcbf · Issue #1818 · squizlabs/PHP_CodeSniffer

Is there a way to disable reporting for phpcbf? Something like: phpcbf -q --report=none .?

GitHub
Add DaggerFunction `format` to PhpSdkDev by charjr · Pull Request #...

Closes #8124

obsidian forge
#

locally, i ran dagger call --source=.:default sdk php generate -o ., and it looks like it's out of date

#

however, when i pr this in, it looks like ci fails?

#

cc @sudden shuttle

serene light
obsidian forge
serene light
#

Yeah, were they caused by the generate function?

obsidian forge
#

well apparently

obsidian forge
serene light
#

What is it passing locally?

obsidian forge
#

this job

serene light
#

Right apologies, should have looked.

So to be clear, for my sake as I'm still quite new to Dagger: you're calling something like dagger call -m dev sdk php lint?

obsidian forge
#

yeah, i'm calling dagger -m . call --source=.:default sdk php lint

#

i can't quite tell, but it seems like the codegen is doing different locally vs in ci? i don't really know how that's possible 😢

serene light
obsidian forge
serene light
#

I forked your branch and called dagger -m . call --source=.:default sdk php lint

I'm got the following:

Stdout:
invoke: input: dirdiff.assertEqual resolve: call function "AssertEqual": process "/runtime" did not complete successfully: exit code: 2

Stdout:
invoke: input: container.import.withMountedTemp.withMountedTemp.withWorkdir.withMountedDirectory.withMountedDirectory.withWorkdir.withExec.sync resolve: process "diff -r a/sdk/php/generated b/sdk/php/generated" did not complete successfully: exit code: 1

Stdout:
diff -r a/sdk/php/generated/Client.php b/sdk/php/generated/Client.php
149,150c149
<     ): GitRepository
<     {
---

... and so on.
serene light
obsidian forge
#

😕

#

what

#

that's so strange, i wonder what's happening

serene light
#

However I did not call generate first or anything. So maybe my local branch doesn't match yours

obsidian forge
#

no i'm working from a clean state

#

hmm, i wonder if it might have something to do with sdk/php/vendor/?

#

removing that now, trying again

serene light
#

lint shouldn't touch vendor/ at all

obsidian forge
#

yeah, cool @sudden shuttle that did it

obsidian forge
#

when before it didn't

#

could it have had some old deps in it?

serene light
#

That is surprising! It's passing on CI by removing vendor locally?

#

I'm glad it worked, but I am curious why

obsidian forge
#

the pr i opened was just to demonstrate the issue - what was happening was that running lint locally was failing, because generate was making changes to the generated code

this seemed to be because i had a vendor/ directory, which was somehow influencing the codegen (i guess from an old dep in there?)

serene light
serene light
# obsidian forge the pr i opened was just to demonstrate the issue - what was happening was that ...

@obsidian forge In case it happens again, we've figured out why this issue is occurring.

Too Long Didn't Read: If you encounter issues again, delete sdk/php/dev/composer.lock and try again.

Too Short Want To Read: The sdk/php/dev/composer.lock takes the sdk/php directory, but it doesn't realise when sdk/php has changed because it's just a local path. This is mostly an issue during development as the dependencies of the sdk change rapidly, but we may develop a composer plugin to solve that in the future. For now, it's on the ever increasing back log.

cc @summer sequoia if you can double-check I've got the facts straight here, thank you 🙂

sudden shuttle
green otter
#

@summer sequoia @serene light ping

serene light
green otter
#

@summer sequoia @serene light can we arrange a call for next week sometime? Such as Wed or Thu to discuss outstanding things with PHP SDK?

Such as, blockers, open discussions, release plan ..etc

You don't have to prepare anything in advance, just some topics.

summer sequoia
#

there are a few issues in github regarding things that are still to do; interfaces, constructors and telemetry are the main ones i think

serene light
#

@Carnage @charjr can we arrange a call

#

@cobalt pelican Quick question: When it comes to enums what should the value look like when returned?

Lets say I'm looking at the NetworkProtocol enum

enum NetworkProtocol: string
{
    case TCP = 'TCP';
    case UDP = 'UDP';
}

Should I get back "TCP" or TCP ?

EDIT: More precisely, should I get back the case's name or value?

If for instance we had a case

case Name = 'Value'

What should I be seeing as a result?

cobalt pelican
serene light
ocean creek
green otter
#

@ocean creek hey 🙂 👋

#

@zealous star hey! 🙂 let us know once you've given your opinion on these changes 🙂 /cc @tidal geyser

serene light
#

@shy goblet First; Thank you for getting context directories in, they will certainly make calls simpler. I had a couple of questions about supporting them in PHP.

  1. The PHP SDK piggybacks off the Go SDK, I'm assuming your work makes them immediately accessible to the SDK with the use of the annotations in Win's PR. If I'm wrong, please let me know. I'd love to understand it more.

  2. What, if anything, needs to be done to support context directories for modules built using the PHP SDK?

GitHub
Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

zealous star
ocean creek
#

@green otter Hi 👋 Will looking suggestions on that PR tonight.

green otter
#

Cool

ocean creek
#

@shy goblet @cobalt pelican I have a question about +ignore on https://github.com/dagger/dagger/pull/8369.

I try with with-dev dagger call -m github.com/wingyplus/dagger/sdk/php/runtime@cd3a3e724aa5a2ce259fe6b217f0efe5abaaabcb source-dir (it is the latest commit on that PR) and found that ignore does't work as I expected because it returns:

_type: Directory
digest: sha256:4efcbbc3978405e30f078b41ff5acb6a0d1f6084906a42d3cf32d4377190a414
entries:
    - .changes
    - .changie.yaml
    - .gitattributes
    - .gitignore
    - .php-cs-fixer.dist.php
    - CHANGELOG.md
    - LICENSE
    - README.md
    - composer.json
    - composer.lock
    - dev
    - docker
    - docker-compose.yml
    - generated
    - phpcs.xml
    - phpunit.xml.dist
    - psalm.xml
    - runtime
    - scripts
    - src
    - tests

But if I change defaultPath to ".." I got:

_type: Directory
digest: sha256:87f768c76fbf1901d1b036461fc9207d82f7eccfc26d892540ef276e332aafbb
entries:
    - LICENSE
    - README.md
    - composer.json
    - composer.lock
    - generated
    - scripts
    - src
GitHub
Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

#

I'm unsure what I am missing on the ignore configuration. But why does it act differently?

cobalt pelican
#

The value for defaultPath is relative to the directory where dagger.json is.

#

Why don't you use it locally?

with-dev dagger call -m runtime source-dir
shy goblet
ocean creek
#

From https://github.com/dagger/dagger/pull/8369/files#diff-0586053a20f05d1973e9fafdb90d0e09153f7943df4f7983b78f88e0338a5beeR28-R29. I change the defaultPath to /sdk/php, If I call against local runtime directory, I got:

_type: Directory
digest: sha256:87f768c76fbf1901d1b036461fc9207d82f7eccfc26d892540ef276e332aafbb
entries:
    - LICENSE
    - README.md
    - composer.json
    - composer.lock
    - generated
    - scripts
    - src

That looks very similar to defaultPath .. but when I call on module remote path, I got all files unlike the local one. So I'm unsure I set the ignore configuration correctly because the behaviour is not the same.

GitHub
feat(sdk/php): use context directory to get sdk source dir by wingy...
cobalt pelican
ocean creek
cobalt pelican
#

@ocean creek, I know I'm the one that asked you to move dagger.json back into runtime but I'm having second thoughts. The problem is we want to avoid having to have <...>/runtime at the end of the git URL for --sdk, but we also want to avoid having to -m dev every time while developing .We can't have both though.

#

One possible idea is that the runtime module is the module of sdk/php. Basically, make sure that the SDK's dir (i.e., sdk/<sdk>) is that module and can be used directly with dagger init --sdk=. Then, for the dev module, it's possible to merge it in the "runtime", the problem is that it would have to be in Go, meaning that you lose the dogfooding opportunity of having the dev module in the SDK's own language.

So I'm torn between these:

  1. Not having end users point an SDK to <...>/runtime
  2. Not having maintainers type -m dev while developing
  3. Keeping the dev module in non-Go SDKs in their own language
radiant flume
#

@cobalt pelican what about the reverse, developing the whole SDK in its own language, including the runtime? You mentioned performance reasons for using Go, maybe those are fixable? There might also be a bootstrap problem? But I guess that might apply to the dev module as well (can't test the php module because the php module is broken etc)

cobalt pelican
#

That's not feasible anytime soon. The problem is bootstrapping. Last year right around this time we were battling with this same issue. We got around it by giving up the idea of having a normal Go module defining the runtime container for Go Modules. What you have now is that Go's runtime is defined with lower level core code, and hardcoded in core. It's not using a sdk/go/runtime or anything. And so Go is managed in a more lower level and core native way, but it allows using Go modules to define other runtime modules. It's possible someone has a fresh new idea though, I'd be interested in what Justin can think of.

#

However, I think it's a good idea to keep the runtime module in Go. The more hops you have, the longer it'll take to initialize, not to mention probably a slower language. Unless we pre-build these runtimes or something, and even then what does that mean when it's a non-compiled language?

tropic whale
green otter
#

@tropic whale don't merge that work please. It's duplicating the other work I'm already doing, and Vikram and levelaz are working in isolation from Carnage, Charjr, and myself .. and it's the first I've heard of this work going on ..

My 4 hour PHP dagger workshop already has all the PHP sample codes that reflect the SDK, daggerizing existing PHP apps.

The plan here between Carnage and I is to start to lift and shift my existing work across, to the dagger docs.

Carnage, Charjr and I have a team call scheduled for this coming Friday to go over the Dagger PHP state of play, what's missing over all.

I'll raise the question of Docs on that meeting and we can circle back with a good plan.

Is this okay?

tropic whale
green otter
#

Expect a report/notes from me on Friday, on outstanding items and when/how we plan to have them solved by.

#

/cc @summer sequoia @serene light

tropic whale
# green otter Expect a report/notes from me on Friday, on outstanding items and when/how we pl...

awesome. Just keep in mind that we are trying to keep all of our "integration" pages in the same format. Thanks again for all your contributions!

You can find the integration page template here: https://github.com/dagger/dagger/blob/main/docs/current_docs/integrations/_integration.mdx.template

GitHub
dagger/docs/current_docs/integrations/_integration.mdx.template at ...

An engine to run your pipelines in containers. Contribute to dagger/dagger development by creating an account on GitHub.

green otter
zealous venture
#

@tropic whale @green otter instead of above, any objections to merging the current version asis and then iterating on it? We can ofc add more examples as they arrive and the PHP SDK becomes more stable?.

green otter
summer sequoia
#

The readme that is generated by dagger init --sdk=php is fairly comprehensive on how to use the PHP SDK

radiant flume
#

Looking at the contents of the various sdk/X/dev modules I may have a suggestion

#

which we can discuss later, I just want to write it down before I forget 🙂

#

context: I am refactoring our CI and my last obstacle is integrating the SDKs...

ocean creek
ocean creek
#

@cobalt pelican what do you think of revert move dagger.json on that PR? So we can push context directory setup forward without any blocking.

summer sequoia
#

Are there any plans to allow for extending dagger objects (especially core ones) in modules? I'm thinking I'd like to extend Container to make a PHPContainer, which along with all the existing container functions also has some additional php specific things, like composer commands added as additional functions. As is I can create a dagger object which has all of those things, but it can't be passed to other functions which want a Dagger Container object without unwrapping it first

shy goblet
ocean creek
#

Nice!!!

#

Will try your PR in around 1 hour

ocean creek
# 
wingyplus@WINGYMOMO ~/s/g/d/d/s/php (php-context-dir)> with-dev dagger call -m github.com/wingyplus/dagger/sdk/php/runtime@cd3a3e724aa5a2ce259fe6b217f0efe5abaaabcb source-dir
✔ connect 0.3s
✔ initialize 5.6s
✔ prepare 0.0s
✔ phpSdk: PhpSdk! 0.6s
✔ PhpSdk.sourceDir: Directory! 0.0s
✔ Directory.digest: String! 0.1s
✔ Directory.entries: [String!]! 0.1s

Setup tracing at https://dagger.cloud/traces/setup. To hide: export NOTHANKS=1

_type: Directory
digest: sha256:87f768c76fbf1901d1b036461fc9207d82f7eccfc26d892540ef276e332aafbb
entries:
    - LICENSE
    - README.md
    - composer.json
    - composer.lock
    - generated
    - scripts
    - src

🎉

cobalt pelican
ocean creek
ocean creek
serene light
serene light
#

I'm having a confusing issue with setting up a constructor:

We've decided, if the PHP magic method __construct is exposed as a DaggerFunction then it is the constructor for the module.

I have this module:

    #[DaggerFunction]
    public function __construct(
        private Directory $source,
    ) {
    }

    #[DaggerFunction('Retrieve a base PHP CLI Container')]
    public function php(): PhpContainer
    {
        return new PhpContainer($this->source);
    }

and call

dagger call --source=. php

This works fine.

Then I try running a DaggerFunction on my PhpContainer called test

dagger call --source=. php test

This blows up:

[ERROR] Method DaggerModule\MyModule::() does not exist

Is this an error anyone recognises? Or is this a quirk of me having not gotten it quite working?

I've put up a draft PR if anyone has the time to help 🙂
https://github.com/dagger/dagger/pull/8437

GitHub
feat: Support constructors in PHP SDK by charjr · Pull Request #843...
cobalt pelican
shy goblet
green otter
#

@shy goblet 🤘

#

@ocean creek 🤘 😄

ocean creek
obsidian forge
#

👋

#

just noticed that the sdk_php.go Test function doesn't spin up a dev engine 🤔

#

which means it's querying against the main engine - so if new apis are added, which @cloud trench has hit, then we're testing against the wrong engine

#

does it make sense to alter the test function to take a service? and then have something like the installer function?

radiant flume
#

@obsidian forge that's one thing I caught in my CI cleanup. Would love to enroll you & @cobalt pelican so we can get that done together, it massively simplifies our CI in general, and will clarify the contract between core and SDK pipelines, in a much needed way (it's a mess at the moment)

#

among other things we have drift between sdk dev modules (not used in CI with the exception of python) and core sdk_xxx.go functions which reimplement the same stuff, slightly differently, in Go, and are called in CI (but perhaps not used by SDK devs upstream?)

obsidian forge
#

think an interface would be good for this - we could have an Installer interface that has an Install method. then we implement that on the DaggerDev top-level, and pass that into the sdk modules

summer sequoia
green otter
serene light
#

I may be mistaken on the issue, but that's what it is sounding like to me.

green otter
#

FYI @zealous venture and @tropic whale ..etc

We had our PHP group call today (@carnage, @serene light) We spent the 1hr only discussing the documentation stuff, and he have conclusions drawn on, and positive improvements to make.

@serene light is prepping some functional examples, and give me over the weekend to write up the meeting minutes and conclusions.

I'm focusing on some dagger tutorials stuff with @pure lodge atm, so prioritizing and will circle back.

Cheers 😁

zealous venture
serene light
zealous venture
#

Yes, that should be fine. Thanks

hidden cairn
#

Hi, everyone.

green otter
#

Welcome @hidden cairn

#

Nice ElePHPant photo

#

Yes @zealous venture give me at some point today to write an update.

Charjr (john) is going to do the work and PR to you, but my message will make more sense in context.

Speak soon. Thanks

hidden cairn
#

Nice to meet you, @green otter .

green otter
#

@serene light change the code so it doesn't git clone a repo, but mounts the source code.

Update the command to do --source=.

#

withDirectory()

serene light
#

We can't do --source=., default paths havent been merged yet

green otter
#

Can you take my phpstan code too? Add that as another test.

Lemme get you the code from my workshop.

serene light
green otter
#

@serene light you're not understanding me 🙂 .. we can already bring code into the container. This was possible from day 1.. You're thinking of something else.

#

Add the Directory argument and call it $source

serene light
#

Are we making examples expecting them to supply their own project, and not with an example project?

#

If that's the case we can probably go back to the original suggestion of just reusing code from PhpSdkDev.php

#

the bitnami:laravel container means we don't need certain lines like: ->withEnvVariable('COMPOSER_ALLOW_SUPERUSER', '1') but as it turns out that's because that docker image already does it and uses root as the user

#

But if we're using the code from PhpSdkDev then I already have functional examples of running PHPUnit, PHPCS, PHPCBF and PHPStan from any generic php container

#

I understand if we make any complex examples in the future of setting up nginx then bitnami:laravel would be handy. But for running basic CLI tools it might be more convenient simply to use a framework-agnostic container since we run a composer install anyway

#

And obviously saves time since I can just reuse code

green otter
#

@serene light hey! ping my name when you reply, so I see it 🙂

serene light
#

@green otter I've reused the PhpSdkDev but reworked it to avoid that allow_superuser stuff

green otter
#

@serene light

  1. The example approach is for you to let them add Dagger to their existing project. They will have a project already, and we want to get Dagger on to it.

So that's why you're doing ->withDirectory()

  1. Stick with the bitnami laravel image. It's strategically being used here. Add on the phpcs and phpstan calls onto this example, and that's fit for purpose.

In the future more things will come, that we want to use the laravel image for

serene light
#

@green otter Playing Devil's advocate a little: Why not both?

#

We could have the framework-agnostic tests and static analysis etc.

#

then framework-specific example for the framework-specific publishing and what not

#

If they're used as separate examples, built from separate containers.

#

The only downside I suppose is based on having to have 2 different build function examples

green otter
#

For now, let's just stick to the plan 🙂 please

There's good reasons for it, but we can talk about it more over a call, soon.

serene light
#

Fair enough 👍

green otter
#

I'm going to DM you a link to some of my workshop code.

zealous venture
green otter
#

Hey @zealous venture

Here is the update from Friday's call and stuff. Sorry for the delay in replying.

Integrations PHP page - current examples on the site doesn't reflect real life of a PHP developer.

  • The changes in your current PR are to remove the apache webserver from the docker image, and to bring in Slim Framework.

  • Having a docker image without a webserver isn't reflecting real life in PHP, so we need the webserver back in there, but not apache as nobody uses apache anymore.

  • Using Slim isn't a good choice either as it's super niche and not reflective of the real life of PHP developer, and they can't relate.

I asked myself what is the purpose of this page? What should user experience be?

The answer is it should outline examples of things in the day and life of a PHP developer, so that is what Chris, John, and I discussed on the call, that we will bring to the table some real-life-like samples and scenarios of a PHP developer, that they can relate to and immediately pick up, an start using Dagger on their PHP projects.

Lots of this content is already on my Dagger PHP workshop, so we can quickly lift them across.

Note: We're going to add the "PHP" tab here, too, on this page only, and have a note that it's "experimental" and not official PHP SDK, yet, but at least the PHP people can read PHP code (upon page load), but they can switch to the other languages on the tabs, too.

As for your question about your current PR, which strips out Apache.. We will come up with new page content for PHP, and file a PR for it, you can expect this later today 🙂

zealous venture
#

Thanks for the info, that helps. One comment: regarding adding a PHP tab to the page, we decided some time ago not to document experimental SDKs in code listings. So I would skip that atm and only have the code in the officially supported languages.

green otter
#

@zealous venture I understand, and I understand why that decision is made across the entire dagger.io website

I'm not suggesting you revert on that decision, but given we're on a language-specific page, then we should have it here, as the exception to the rule, so that it makes sense.

If I land on a PHP page and don't see PHP code, then it will feel confusing, coz they are PHP developers, not Golang developers.

Outside of this 1 page, then these factors don't exist

obsidian forge
#

@serene light your pr here https://github.com/dagger/dagger/pull/8478
reminded me of something i hit about a couple years ago in docker - turns out all the docker build tools used to produce 3-space indented json everywhere, and there were critical tooling pieces that relied on it...

#

insane indent levels, always good fun 😄

serene light
serene light
#

Has anything changed between 0.12 and 0.13 regarding AbstractScalars and enums?

I'm running into these errors:

Error: get module name: input: failed to get schema: failed to get schema for module "test-everything": failed to create function "testBuiltinAbstractScalar": failed to find mod type for function "testBuiltinAbstractScalar" return type: "Json!"
obsidian forge
#

I don't think so? do you have a snippet of code?
abstract scalars aren't a dagger thing as far as i'm aware though, so not entirely sure

serene light
#

I don't fully understand how the automatically generated code works yet, but the PHP SDK generates an AbstractScalar interface.

abstract readonly class AbstractScalar implements Stringable
{
    public function __construct(private string $value)
    {
    }

    public function getValue(): string
    {
        return $this->value;
    }

    public function __toString(): string
    {
        return $this->value;
    }

    public static function from(string $value): static
    {
        return new static($value);
    }
}

Which is implemented by things like IDs, Platform and Json

/**
 * The platform config OS and architecture in a Container.
 *
 * The format is [os]/[platform]/[version] (e.g., "darwin/arm64/v7", "windows/amd64", "linux/arm64").
 */
readonly class Platform extends Client\AbstractScalar
{
}
#

It's basically a fancy wrapper around a string

#

Basically, I had enums and abstract scalars working as far as I was aware, but now they only work in limited instances and I've been struggling to fix it

#

Sorry forgot to make them replies @obsidian forge

#

Either way, thank you for the input. It is probably a change I made, I'll just have to keep digging.

shy goblet
#

Hey @serene light @green otter, hope you guys are doing well!
There's an issue with the PHP SDK generator on that PR: https://github.com/dagger/dagger/pull/8427#pullrequestreview-2324715836

Could you have a look so we can merge it, it's the last thing that hold the PR 😄
It's quite a strange issue though.

/cc @cloud trench

GitHub
feat(engine): support expanding variables in more functions by raja...

This adds expanding environment variables in WithNewFile. ref: #7951. If this looks OK, I will add similar changes to other API's in this same PR.

serene light
#

@obsidian forge In reference to #1288075868073234494 message

Pointing to the specific commit hash would be quite annoying for development. As dagger moves fast 🚀
I end up rebasing my PRs incredibly frequently and would have to update this every time. It would be a pain and also a mistake waiting to happen.

Are there any alternatives that might work?

obsidian forge
#

is the dev module always using the latest and greatest features? i think the options probably are:

  • leave it as implicit/explicit main - if we ever make backwards incompat changes to the php sdk, old engines will fail to build - this is pretty bad
  • leave it as implicit current engine tag - this means, that like the python/typescript dev modules, you can't use features that have been merged to main and haven't been released
  • pin to a specific commit on main (but how to make sure it doesn't lag too far behind, etc, is tricky)
#

personally, i'd be happiest with the 2nd option, but means that for example, you wouldn't be able to use constructors in the php dev module until v0.13.4 is released

#

(which is no worse than the other dev sdks)

serene light
#

That is a bit awkward as it sounds like we wouldn't be able to test new SDK features until after they've been merged for a while.

Do other SDKs have a way around this? Or do new feature tests have a delay?

#

@obsidian forge

obsidian forge
#

for tests imo, it doesn't matter hugely - but for the dev module (which is imported into the core ci), it does

#

fyi @cobalt pelican

serene light
#

@cobalt pelican @obsidian forge I think the best solution sounds like we fix it at 0.13.4 once that's released. That way we wont break anything currently, and then bump it to the latest version during any PR

cobalt pelican
#

Yeah, for Python's dev module (or any CI module) I need to keep it compatible between dev and stable. You need to make sure the engineVersion in dagger.json points to a released version of dagger, and not a dev build.

#

It's just because our CI runs with both, in order to catch possible regressions in a change, earlier.

serene light
cobalt pelican
#

I just need to double check on the isOptional thing, but I need to take care of something else now. Will get back to it after.

serene light
# cobalt pelican I just need to double check on the isOptional thing, but I need to take care of ...

Ah that's this bit

    public function isOptional(): bool
    {
        return $this->type->nullable
            || $this->default !== null
            || $this->defaultPath !== null;
    }

Basically it's optional if the type has been specified as nullable OR has a default value OR has a defaultPath

GitHub
dagger/sdk/php/src/ValueObject/Argument.php at d4e95da36319131a531b...

An engine to run your pipelines in containers. Contribute to dagger/dagger development by creating an account on GitHub.

#

Then I tested the function here:

    #[Test]
    #[DataProvider('providePropertiesThatMakeArgumentsOptional')]
    public function itMayBeOptional(
        bool $expected,
        Type $type,
        ?Json $default,
        ?string $defaultPath,
    ): void {
        $sut = new Argument('sut', '', $type, $default, $defaultPath);

        self::assertSame($expected, $sut->isOptional());
    }
GitHub
dagger/sdk/php/tests/Unit/ValueObject/ArgumentTest.php at d4e95da36...

An engine to run your pipelines in containers. Contribute to dagger/dagger development by creating an account on GitHub.

#

And no worries, take your time. Thank you very much for taking the time to review it today 🙂

cobalt pelican
#

That's the thing, isOptional isn't necessarily true if there's a default value.

#

isOptional actually means "does it take a null"?

#

So an argument on the language side that has a default, but that default being a complex type that can't be registered to dagger, you need to force it with withOptional even though it doesn't take a null. But at runtime the SDK needs to detect this so that if the engine sends a null, the SDK actually needs to omit it so the default value gets triggered.

Today, the engine never sends a null, but that could change so it's good to prepare for it.

serene light
#

I'll read the context more thoroughly shortly, when I have the time. Thank you for your input

cobalt pelican
#

isOptional should really only be forced when the function has a default value but can't be reported to dagger (e.g., complex type). I think when it's a defaultPath it's the engine that should force that in, rather than continue to make SDKs do it.

serene light
zealous venture
#

Hi. We've been having a few discussions on docs internally so I'd like to provide a quick update related to the PHP page.

#
  1. PHP "integration" page in docs - We're pushing a big docs change later today and we would like to include this page in it. So we're planning to merge the page in its current state, but ofc it's totally fine for you to send a PR with new examples etc once they're ready. https://github.com/dagger/dagger/pull/8321
#
  1. PHP code tab - Yes to this with some qualifications. We're going to try it as an experiment only on this PHP page as you suggested earlier. We will however need to highlight that the PHP SDK is experimental, both in the tab label eg. PHP (Experimental) and as a callout box in the tab content. We're also assuming that as maintainers of the PHP SDK, you'll take care of keeping the example code in the PHP tab sync with future changes in the SDK.
#

Thank you for all your hard work on the SDK and for all the feedback as well!

serene light
#

Thank you

serene light
#

👋 Hi @cobalt pelican We've noticed a bug with enum support for the PHP SDK.

I've been working on it here.

Looking at our implementation, it seems identical to the typescript implementation:

  // Register all enums defined by this modules
  Object.values(module.enums).forEach((enum_) => {
    let typeDef = dag.typeDef().withEnum(enum_.name, {
      description: enum_.description,
    })

    Object.values(enum_.values).forEach((value) => {
      typeDef = typeDef.withEnumValue(value.value, {
        description: value.description,
      })
    })

    mod = mod.withEnum(typeDef)
  })

        foreach ($daggerObjects as $daggerObject) {
            if ($daggerObject instanceof Dagger\ValueObject\DaggerEnum) {
                $enumTypeDef = dag()
                    ->typeDef()
                    ->withEnum(
                        $daggerObject->getNormalisedName(),
                        $daggerObject->getDescription(),
                    );

                foreach ($daggerObject->getCases() as $case => $description) {
                    $enumTypeDef->withEnumValue($case, $description);
                }

                $daggerModule = $daggerModule->withEnum($enumTypeDef);
                continue;
            }

My implementation works for built in enums. But it is failing on user-defined enums with this error:
Error: invalid argument "hello" for "--arg" flag: value should be one of

It's just empty after that... so presumably my implementation is failing to register the values properly.

GitHub
Fix: Php enum support by charjr · Pull Request #8608 · dagger/dagger

PHP SDK support for enums appears to have regressed at some point.
Draft PR till it so I can get feedback.

serene light
#

As a separate issue, some of the scalar types work and some don't.

For instance, I can pass Platform as an argument and return it.

I cannot pass Json as an argument nor can I return it.

worthy ice
#

Something very odd is happening with php codegen locally for me (running dagger call sdk php generate export --path .). When I run locally on main with no changes I get this diff: https://github.com/sipsma/dagger/commit/7f15a0020fc89ff912a56af3ed4b01580492f02b

Consists of moving some curly braces from one line to the line below it.

But in our CI that never happens and checks pass.

It's a problem because I have to hand edit the generated files or else CI in my PRs fails after regenning SDKs.

Any idea what could be happening? Seems extremely odd it's different locally and in CI (I'm on arm64 linux if that somehow matters due to a platform specfic container image at a different version from x86 or something like that)

cc @cobalt pelican @obsidian forge in case this rings any bells

GitHub
php: regenerate sdk to fix out-of-band formatting changes · sipsma/...

I get this weird diff when running php codegen locally, but it doesn't
happen in CI?

Signed-off-by: Erik Sipsma

worthy ice
#

Ah... on a hunch I just ran git clean -d -f -x ., saw that sdk/php/vendor got removed, and now when I run codegen that diff goes away...

So I guess a stale vendor dir broke it somehow? Either way, not a huge deal, just a big gotcha laying around to be aware of

sudden shuttle
serene light
#

@cobalt pelican @obsidian forge I'm noticing the php integration tests, that were passing on 0.13.3 are failing on 0.13.4.

These tests specifically

Has something changed that might break them? None of the breaking changes look related to the issue:

🔥 Breaking Changes

Changed behavior of Git to default keep the .git directory by @obsidian forge in #8318
This can be disabled with tree's new discardGitDir option. Modules with
older engineVersions will keep the old behavior.
Deprecated git's keepGitDir argument by @obsidian forge in #8318

Looking at the tests, they seem to assert that basic functionality works as expected, making a file, directory or container.
I haven't yet learned the nitty gritty part of how the PHP SDK integrates with dagger, so any insight would be appreciated.

GitHub
Blaming dagger/sdk/php/tests/Integration/ClientTest.php at v0.13.4 ...

An engine to run your pipelines in containers. Contribute to dagger/dagger development by creating an account on GitHub.