php

[radiant flume]

@green otter 👋

I created this channel for the occasion 🙂

cc @tropic whale @zealous venture

[green otter]

What an occasion 😎

🐘

[radiant flume]

We might have a PHP SDK in the works 🙂

[green otter]

I have a few questions on just getting setup and started, but once I have dagger running things for me then we're flying!

Good to meet you Miranda / Vikram 🙂

[radiant flume]

Good people to ask about the SDK tooling and dev process:

• @tidal geyser on the infra & tooling side
• @cobalt pelican who wrote the Python SDK and super involved in overall SDK architecture
• @ocean creek, @steep orchid, @trail sparrowwho contributed experimental SDKs

[zealous venture]

Nice to meet a fellow PHP-er @green otter 🙂

It would be great to have a native PHP SDK, it would make https://docs.dagger.io/128409/build-test-publish-php so much cleaner 🙂

[green otter]

yeah ... few steps ahead already of what Dagger + PHP collaboration could/should look like.

Just need to get Dagger running first and foremost, then I can show you guys what I'm thinking

I'll likely be reaching out to you @cobalt pelican 😛

[green otter]

I'm picking up this article @zealous venture https://docs.dagger.io/128409/build-test-publish-php/

composer require gmostafa/php-graphql-client --with-all-dependencies

done, now I have a graphql lib

this PHP pipline class is now in place.

It wants 2x env vars DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN

where can I find these values? prior to this step I've just installed the Dagger LI

I found this from a Java SDK webpage online
If the environment variables DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN are defined, the SDK will use these variables to initialize a connection with the Dagger engine. However, if they are not set, the SDK should automatically download the corresponding Dagger CLI for the SDK version and the host machine’s architecture.

I'm without an SDK right now, so wondering of the approach here 🙂

that SDK code just flat out throws Exception if they're not provided

[zealous venture]

trying to repro

[green otter]

Step 1) I completed this guide - I have the CLI tool installed https://docs.dagger.io/quickstart/729236/cli
Step 2) instead of SDK I'm looking at this page https://docs.dagger.io/128409/build-test-publish-php/
Step 3) I'm wondering where to find the ENV vars needed to talk to Dagger CLI over HTTP protocol

On this URL: https://docs.dagger.io/api/254103/build-custom-client/

I found this:

It was SUPER USEFUL that we had a "php" tab here ....

by running dagger run then it provided the .php file with the these ENV vars 🙂

question resolved ✅

It's running, it's alive!

[zealous venture]

@green otter sorry, but I'm unable to repro here. The code works as is on my system (no exception is thrown as you mentioned above).
Yes, dagger run will set the DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN env vars, they are then read from the env in the pipeline code.

[green otter]

@zealous venture thanks for replying - i fixed it already, basically the solution was NOT to try and find my own ENV vars, but to run dagger run and it generates them for me 🙂 all is good #learningCurve

[zealous venture]

Glad you were able to get it working!

[green otter]

Is /mnt/ a special Dagger thing? where it puts volumes/mounts?

    // get host working directory
    $sourceQuery = <<<QUERY
    query {
      host {
        directory (path: ".", exclude: ["vendor", "ci"]) {
          id
        }
      }
    }
    QUERY;
    $sourceDir = $this->executeQuery($sourceQuery);

    // add application source code
    // set file permissions
    // set environment variables
    $appQuery = <<<QUERY
    query {
      container (id: "$runtime") {
        withDirectory(path: "/mnt", directory: "$sourceDir") {
          withWorkdir(path: "/mnt") {
            withExec(args: ["cp", "-R", ".", "/var/www"]) {
                id
          }
        }
      }
    }
    QUERY;
    $app = $this->executeQuery($appQuery);

[radiant flume]

not a special dagger thing

[green otter]

Understood, it's just this person's example then. Strange to me to put app code inside /mnt/ that's all 🙂

[zealous venture]

Yes...however it's an intermediate location, the next step is to copy the app code to /var/www (more typical)

[green otter]

@zealous venture indeed! I'm wondering why we don't just copy code from HOST -> /var/www/

Is there a special Dagger reason why you'd want to do withDirectory(), to /mnt/ and then cp -R over to /var/www/

        withDirectory(path: "/mnt", directory: "$sourceDir") {
          withWorkdir(path: "/mnt") {
            withExec(args: ["cp", "-R", ".", "/var/www"]) {
              id
            }
          }
        }

If no magic reason then we could get rid of it?

[zealous venture]

I cant remember why I did it that way, but what you suggested should work fine. I'll open a PR to update it.

[green otter]

@zealous venture the real question is... do you intentionally need a staging DIR for Dagger? or was it just your preference to setup /mnt/ first (intermediary)

[zealous venture]

Just preference (afair) to have the host dir in /mnt

https://github.com/dagger/dagger/pull/5886

[green otter]

@zealous venture think I found a bug in your PR - you forgot to put the "cp -R" command back in ?

            withExec(args: ["chown", "-R", "www-data:www-data", "/var/www"]) {
              withExec(args: ["chmod", "-R", "777", "/var/www/storage"]) {
                withExec(args: ["chmod", "+x", "/var/www/docker-entrypoint.sh"]) {
                  id
                }
              }

Secondly, if you have

        withDirectory(path: "/var/www", directory: "$sourceDir") {
          withWorkdir(path: "/var/www") {

Do you really need withWorkdir? given you've already set withDirectory() - are they different things and thus we need both ?

[zealous venture]

1. It shouldn't be necessary to copy it, withDirectory will return the container with the directory written to the path
2. Yes, withWorkdir can be removed here because it's set anyway in the next step, I've updated the PR - thanks for the feedback!

[green otter]

SUCCESS! got my own baby pipeline running, with 1 unit test

[green otter]

So .. day #0 on Dagger.

I now have a partially working SDK

Here is the code:

  $output = $this->dagger->container()
      ->from('php:8.2-apache-buster')
      ->withExec("php -v")
      ->stdout();

It now generates this Dagger GraphQL

query {
  container() {
    from(address: "php:8.2-apache-buster") {
      withExec(args: ["php", "-v"]) {
        stdout
      }
    }
  }
}

🥳

[dense pewter]

That's awesome @green otter !!! I can certainly envision the vast PHP community adopting this SDK in the near future

[green otter]

I spent my evening going through a ~4500 line python and nodejs generated client API SDK's to understand how they work 😛

Now I know kung fu.

https://tenor.com/view/neo-neo-kung-fu-i-know-kung-fu-kung-fu-matrix-gif-21409172

[grim stump]

This is dope, really inspiring to see you knock this out in less than a day. Your enthusiasm motivates me 😄

[radiant flume]

@green otter are you handrolling for now, or is this generated? Do you need any help on the codegen side?

An important note, is that Python SDK rolls its own codegen tool, whereas Go and NodeJS share the same codegen implementation. I think @cobalt pelican and @hollow hare recommend following the example of Go and NodeJS (using the common codegen tool). Is that right guys?

[cobalt pelican]

I'd love to converge on using Go eventually, but it needs serious refactoring (see https://github.com/dagger/dagger/issues/5226).

[green otter]

Hey @radiant flume yes, I am hand rolling right now. I'm doing PoC stage currently, familairizing myself with your nested graphql DSL, and anticipating how to reverse engineer your graphqlspec into a SDK generated client.

Focusing on the critical path, such as withUser, withExec, withWorkdir, from(), container() and host()/directory().

I've got the fluent interface working nicely, with immutability. Exactly like how the NodeJS one works.

I came up with a nice DX improvement.

->withExec() now support strings
->withExec('cat /some/file') will become ->withExec('cat', '/some/file'), underneath, so you can just feed it normal string commands and it'll do the array translation underneath

(unless there's a reason why splitting on '<space>' won't work and you gotta write out the arrays, like the Python SDK works.)

[radiant flume]

Makes sense. I think once you generate the SDK, you will have less flexibility to tweak the signature of individual functions. withExec takes an array in GraphQL, so it takes an array in all generated SDKs.

[green otter]

I was wondering something like.

<?php
function withExec(array|string $args): string
{
  // Convert string command to array
  if(is_string($args) { $args = explode(' ', $args); }
}

Maybe not possible, we'll see I guess

[ocean creek]

Need to consider sh -c 'echo helloworld', the echo helloworld should not get split because it is the value of argument -c

[green otter]

Yea you're right. I knew it wasn't that simple 🙂

[green otter]

I've never used GraphQL before, so also had to learn it a bit on Day 0

I've just reached out on my LI/Twitter/FB for graphql libraries that generate PHP client from the spec.

To do some research before we move to next steps, which is going to be finding something and customizing it for Dagger's needs.

[green otter]

I've found this.

😄 looks good.

It takes the queries and generates PHP code.

Looks like it will be a good fit here!

[green otter]

I've taken a few days break to catch up on booking travel/accom for the upcoming conferences in October.

I'm returning back to Dagger soon, I'm sleeping on ideas and reflecting on searching I've done.

Next steps for me is reviewing and comparing these SDK Gen clients, and playing with them.

Hey @cobalt pelican ..etc

Can you link me to the code that currently does the NodeJS SDK codegen? Want to see how it runs, and start to now anticipate things from the 'gen' side and not only 'client' side.

[cobalt pelican]

Hey 🙂 Right now it's inside sdk/go/codegen.

Oh wait... recently changed to cmd/codegen.

https://github.com/dagger/dagger/tree/572bd3de9340dea4121627163f4f7b58c818beb4/cmd/codegen/generator/nodejs

[green otter]

Ty @cobalt pelican

[mossy cosmos]

Hello 🤩

[grim stump]

Hello 🤩

[green otter]

Meet Manu (emmanuel) from Spain. ☝️

He is from the IPC conference and is very enthusiastic to try the PHP SDK once it's ready for release/testing.

🙂

[radiant flume]

Exciting stuff!

[fierce olive]

Hello, I pushed the first version of an experimental SDK for PHP here : https://github.com/the-diamond-dogs/dagger-php-sdk

[zealous venture]

How do you import it into a PHP project? Will there be a Composer package?

[fierce olive]

There's one already but not official : https://packagist.org/packages/the-diamond-dogs/dagger-php-sdk
I will send a pull request to the dagger repository to include the sdk.
I guess we will have to do a git read only subtree split because packagist can only take repository with a composer.json located in the root folder.

[radiant flume]

1. Thank you for the contribution, and excellent work all around!

2. We have some experience with the “package manager makes annoying arbitrary rules on how your git repo should be laid out” problem, with the Go SDK. We are happy to share our experience with you, and we can work out the best solution together.

Thank you again!

[fierce olive]

Pull request is here : https://github.com/dagger/dagger/pull/6165
@radiant flume I did mention the packaging issue in the PR.

[fierce olive]

How can I add php-cs-fixer, phpunit and psalm to the dagger repository Github workflow ? I have zero experience with Github CI

[cobalt pelican]

We can help with that. Ideally that would be encoded in a container. Our CI uses the Go SDK and every SDK has a file with dagger pipelines in https://github.com/dagger/dagger/tree/main/internal/mage/sdk. With the new modules support (in Go, Python and NodeJS in flight), the SDKs will move their CI pipelines to a module that can be written in their own language. As a first step I suggest you try creating pipelines in PHP for lint and test. That'll make it easier for someone to either port that to Go or run the PHP pipeline using Go. This way it's still unified in CI and easier for others to help manage.

You may notice that we test/lint/build dagger with dagger, in every SDK. 🙂

It helps with dogfooding, which is also true for this new PHP SDK if the pipeline is written in PHP.

[fierce olive]

Ok, I will try to do this. Do I need to put the PHP pipelines in a specific subdirectory in the sdk ?

[cobalt pelican]

Up to you, but it may help with organization. Perhaps in a ./ci subdir.

[tropic whale]

@fierce olive I'd love to have you share all the work that you've done for this during a community call! I'll DM you.

<@&1166745944444899449> fans, good PR to check out 🙂 https://github.com/dagger/dagger/pull/6165

[vague atlas]

Hello all. Thanks for the invitation, I feel history in the making here!

[fierce olive]

@cobalt pelican I have a working pipeline in PHP to lint with php-cs-fixer the sdk itself. Can I post a snippet here or maybe a gist ?

[cobalt pelican]

Yeah, of course!

[fierce olive]

@cobalt pelican With my latest commits , do you think my pull request could be accepted ? I'd like the composer package to be publish to start using it in other project.

[cobalt pelican]

Yeah, I think we're close to it. I'll take a look!

We'll still need to integrate into our CI before being able to publish.

[fierce olive]

Yes and maybe create a read only subtree for packagist or find another way to publish the package to the composer registry

[cobalt pelican]

@fierce olive, when will you be available to do a bit of work on the SDK? We plan on cutting a release today. I'm going to push changes to your PR to add CI integration. I need to rebase so the force push will require you to reset locally. Don't know if you have any changes locally.

[cobalt pelican]

@fierce olive, just merged 🙂 Please follow-up in new PRs for improvements, thanks!

[fierce olive]

@cobalt pelican Hello, sorry I wasn't available. I will look into the comments and make a new PR.
The problem with the \Dagger\Dagger namespace is because I wanted to put the generated code in a specific folder. Maybe I can create a stub class called \Dagger\Client which is going to extend Dagger\Dagger\Client ?

[cobalt pelican]

No, that won't help with Dagger\Container. As a first step you can remove Dagger from several classes like DaggerClient.

I think you should just:

{
    "autoload": {
        "psr-4": { "Dagger\\": ["src/", "generated/"] }
    }
}

[fierce olive]

I forgot that you can do this ahha

I will make the change. On my local repository I have a cli to lint the sdk using the sdk. I will make a separate PR so that we can discuss it

[cobalt pelican]

@fierce olive btw, there was an issue with the DCO. Your commits have your personal email while GH expected to see the users.noreply.github.com one.

[fierce olive]

Yes, sorry if it's not correct. There's both. Author is my github email and committer is my name and email

[fierce olive]

Should I reverse ?

[cobalt pelican]

I've manually passed for now, but you should check your settings and our DCO policy.

[fierce olive]

Ok I thought that I needed to add my email but DCO policy just need my name right ?

[cobalt pelican]

Basically the sign-off email needs to match the commit author.

Do you have vigilant mode enabled?

[fierce olive]

I don't think so but I sign my commit for other projects but outside github

[cobalt pelican]

I do. Since I force pushed a rebase, it changed your commits so I became the commiter and you the author. Do you have your personal one in your GH settings?

Also, do you have the Keep my email addresses private on?

See Summary in https://github.com/dagger/dagger/pull/6165/checks?sha=48f9edb07db09e6f9db6b59138ef2de57513eac1

[fierce olive]

My local git config is wrong, I think I just need to change my email for the github provided one

[cobalt pelican]

I'm going to try to cut a first release now.

[cobalt pelican]

@fierce olive, try it please: https://packagist.org/packages/dagger/dagger

[fierce olive]

It's working 🙂

[fierce olive]

Really cool

[fierce olive]

@cobalt pelican sorry had to force push, didn't know that Github repo sync create a merge 😅

[cobalt pelican]

Force pushing is actually how we do it 💪

[fierce olive]

Perfect ahah 🤠

[cobalt pelican]

We need to put the codegen in a pipeline to automate this 🙂

[zealous venture]

@fierce olive I'm trying out the PHP SDK but running into some issues.

<?php
// dagger.php
require __DIR__ . '/vendor/autoload.php';

$client = Dagger::connect();
$output = $client->pipeline('test')
    ->container()
    ->from('alpine')
    ->withExec(['apk', 'add', 'curl'])
    ->withExec(['curl', 'https://dagger.io'])
    ->stdout();

echo substr($output, 0, 300);

dagger:~/public/examples/php/myapp/ci$ composer require dagger/dagger

dagger:~/public/examples/php/myapp/ci$ php dagger.php 
PHP Fatal error:  Uncaught Error: Class "Dagger" not found in /public/examples/php/myapp/ci/dagger.php:3
Stack trace:
#0 {main}
  thrown in /public/examples/php/myapp/ci/dagger.php on line 3

Any idea why the class is not found?

[cobalt pelican]

I think you need use \Dagger\Dagger;.

[zealous venture]

That worked, thanks!

I did a quick test by porting our Laravel guide using the new PHP SDK. It works well!

[fierce olive]

@zealous venture Nice! The \Dagger\Dagger namespace will be just \Dagger in next release I think.

[zealous venture]

Code attached for reference

[zealous venture]

I was about to open a PR to update the example in the README to use \Dagger\Dagger, should I wait?

After running through this example, I have a couple of questions as well

1. Is there a way to log output eg. the GQL queries being generated? It would be useful for debugging.

[fierce olive]

Maybe you can use the sdk from main branch by manually adding the path to your composer json. That way you can test with the new namespace

[zealous venture]

2. When I tried to composer require dagger\dagger in a fresh Laravel project, I ran into some dep conflicts

$ composer require dagger/dagger
./composer.json has been updated
Running composer update dagger/dagger
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - gmostafa/php-graphql-client v1.13 requires psr/http-message ^1.0 -> found psr/http-message[1.0, 1.0.1, 1.1] but the package is fixed to 2.0 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
    - dagger/dagger v0.9.5 requires gmostafa/php-graphql-client ^1.13 -> satisfiable by gmostafa/php-graphql-client[v1.13].
    - Root composer.json requires dagger/dagger * -> satisfiable by dagger/dagger[v0.9.5].

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.
You can also try re-running composer require with an explicit version constraint, e.g. "composer require dagger/dagger:*" to figure out if any version is installable, or "composer require dagger/dagger:^2.1" if you know which you need.

So then I needed to use -W, which downgraded the dep

$ composer require dagger/dagger -W
./composer.json has been updated
Running composer update dagger/dagger --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Lock file operations: 2 installs, 1 update, 0 removals
  - Locking dagger/dagger (v0.9.5)
  - Locking gmostafa/php-graphql-client (v1.13)
  - Downgrading psr/http-message (2.0 => 1.1)
Writing lock file

[fierce olive]

ouch that's not good. I guess the graphql client will need a PR 😅

[zealous venture]

The other option, which also worked, was to create a ci/ sub-dir and then run composer require dagger/dagger in that sub-dir. That worked without issues, because it was a standalone install of the dagger/dagger package. However I then needed to run the script from the parent dir as php ci/dagger.php.

Just noting both approaches here for others in case it comes up 🙂

[fierce olive]

I guess downgrading http-message can lead to some weird bug but idk. The ci subfolder seems cleaner yes

It's not possible to log the gql queries right now but its a good idea. Does the python or js sdk have this ?

[zealous venture]

Regarding debugging, I found the easiest way was to use dagger run php ci/dagger.php

[zealous venture]

No, I don't think they have it. It was just a thought.

[fierce olive]

Yes with dagger run you get some logs. To get the log from dagger process into your PHP code you'd need to create the connection manually by creating an instance of ProcessSessionConnection which implements LoggerAwareInterface

[fierce olive]

@cobalt pelican Hello, any issue with publishing the 0.9.6 version on packagist ?

[cobalt pelican]

No, it's not hooked in CI yet. You need a PR with updated codegen first. And we need to add the pipeline for it before adding Publish as part of our release process.

Problem is that after v0.9.6 was released there's already changes in the API so you can't add those in as part of v0.9.6. Thus we need a release branch to do this. We may cut another release this week so an alternative is to add the codegen to CI so we can proceed to add to our release process.

You said you already built a pipeline for PHP, right?

[fierce olive]

I see. Yes I already started building a symfony command to lint the sdk. But I wonder how the CI is going to launch it since you need php and composer ...

[cobalt pelican]

You need to replace Dockerfile/docker compose with a dagger pipeline.

For now it needs to be in Go, but if you do it in PHP we can help port it to Go as it's much easier than figuring it out ourselves.

[fierce olive]

Ok. I will take some time to create a PR with the pipeline in PHP so that it can be ported to go.

[cobalt pelican]

I've added codegen to CI, can you please check the generated files? 👉 https://github.com/dagger/dagger/pull/6446

[fierce olive]

wow nice!

[cobalt pelican]

https://packagist.org/packages/dagger/dagger#v0.9.7

[green otter]

Good

[fierce olive]

FYI #1204482091610280058 message

Should we integrate this inside the SDK ?

[green otter]

Hey @fierce olive can you clearly articulate what you're trying to achieve? Thereafter I can discuss/review technical implementations for it

Another question is, why are you looking to connect to the WS shell, can you explain this to me ?

@fierce olive I've read some more .. it would be good to see a working example/use case for this .. do you have that?

[fierce olive]

@green otter My use case is running a very long migration script. I need to see the stdout in real time in my pipeline when I test this script.

[green otter]

@fierce olive in PHP we have a few different options which are "best practise" .. it depends what this very long running script is used for

What is your use case, for example is it tailing logs ? or something else?

oh, nvm you said migration i missed that

I recommend not running the migration script as PID 1, due to risk of OOM/Segfault ..

Putting events onto an event bus (lightweight) and then processing things on that bus is the right way to perform a migration, with PHP stuff.

Each event will be picked up off the bus and then it will OUTPUT its results onto a log file ...

Then the dagger command would be tailing the log file (or streaming the file descriptor)

So yes, tailing a log file or FD from dagger is fine ... but running the mirgation isn't the right practise in PHP community.

Does this help?

[fierce olive]

yes it's the output of a symfony consumer that i'm trying to test at the functional level.

[green otter]

@fierce olive okay that is perfect, great to hear it's an event worker. Give me a few hours.

What is your timezone ?

[fierce olive]

CET , I don't have discord on my phone , I'll get email notification only

[green otter]

Want to download it? I'm going to have some Qs for you today.

I wanted to ask last night but you were 💤 😴 so I waited.

I'm going to be pushing the SDK to its limits today and want to cross check a few things with you, as I do it.

[fierce olive]

I'm available today until 2pm CET or tomorrow morning. My phone can't handle discord. I only use it on desktop

[green otter]

Understood.

Do you run a blackberry ? Lol

[fierce olive]

No i use a pixel on GrapheneOS without google play services. I only install foss apps on it. 🤠

[green otter]

Team standups 9am. (UK) and then I'll be with you for the next few hours.

[green otter]

@fierce olive I'm back now, and ready to talk about more topics

[fierce olive]

did you try the sdk yet?

[green otter]

@zewro

[fierce olive]

yes

[green otter]

@fierce olive I've been using my own SDK, actually.

I was liasing with the dagger team last year, and I built the initial PHP SDK. I didn't finish it as I got busy with organizing PHPUK Conference (in 2 days time), and now you've pushed your own SDK copy, since.

The talk(s) I did last year were examples using my SDK.

I'm giving a talk in 2 days at PHPUK confeence, and I have decided to commit to use your SDK instead.

That's the context.

Today I am going to be testing and trialing your SDK, to ensure the use-cases I will be featuring, that it can already do these things.

If it can't do these things, then you and I will have to quickly build them, in time for the deadline.

All in all, exciting times 💪

[fierce olive]

Wow, ok didn't know that you started to work on the SDK at the time 🤦‍♂️

[green otter]

Nobody told you, and nobobdy told me you were working on a new SDK 🤦‍♂️ I discovered it like 2 weeks ago, so we both did a lot of duplicate work, and our architectural designs are a bit different too.

I'm not too precious on our design differences, don't worry. I'm a team player

[fierce olive]

In terms of architecture I didn't do much on my own. It's mostly a transposition of the java sdk 😄

I guess we can change and break some things , I don't think a lot of people use the SDK.

[green otter]

@fierce olive I'm ready to collaborate and pair on things now. Are you free now for a quick sync? I see you've sent me a friend request.

Maybe you're currently busy with other things atm, so just understanding your schedule.

Thanks 🤝

@fierce olive I got your DM, I'm going to post the use cases here and we'll do async comms, given your meetings.

With the current copy of this SDK, can we:

1. take an existing docker image for MySQL, and Redis
2. do service binding on them to a PHP image, so they're in the same "docker network", and spin up all 3x containers, ready to communicate amongst each other
3. do DB migrations (PHP app doing TCP comms to MySQL, to run the PHP app's DB migrations)
4. expose the port to localhost:8080 (for the webapp), to access it via browser (for a demo)

Did you test/try this already?

[fierce olive]

At the moment I can confirm that 1 ,3,4 is working. Didn't try the service binding for container to container communication but should work.

The only issue I had is caching. If you do DB migration and execute the pipeline multiple times it will do the migration only once. But this is not SDK specific

[green otter]

This is because of the "layers", yes?

[fierce olive]

Yes

[green otter]

Has someone on the Dagger team communicated a way to say "don't cache these steps" ? out of curiosity

[fierce olive]

there's a sync method

On various objects to force execution

[green otter]

Understood. Do you have your demo code you tried for 1,3,4 .. to hand, for me to take and drive it forward?

if not I'll have to build it myself, it's just about avoiding more duplication at this point 🙂

[fierce olive]

Not right now. You can easily reuse snippet from other sdk and transpose to php. That's what I did 🙂

[green otter]

I'll give it a crack, and if I get stuck I'll ping you here 🙂 thanks, have a productive day ahead

[fierce olive]

This is how I create a base container for a given php version and install dev tools inside @green otter

[green otter]

@fierce olive just seen this, been hacking away! Thanks

You can't do

implode(' ', [' ........'])

I already tried this, the dagger team made me aware of the edge cases in shell/bash that'll break it.

It's one of the very first things I solved/proposed in my impl.

It might work in your example, but won't work completely.

I don't remember what the edge case is .. just sharing with you stuff I've already figured out

[fierce olive]

Oh I didn't know. This is with bash -c. It's used in the nodejs example : https://github.com/dagger/dagger/blob/main/examples/sdk/nodejs/pulumi/index.mjs#L27

[green otter]

@fierce olive noted.

[green otter]

@fierce olive found a bug, it's probably my fault but at the same time it shouldn't be broken for me, as the user

withExposedPort() is the point in which it breaks. Saying

has no such argument: "e
┃ erimentalSkipHealthcheck

[fierce olive]

@green otter , this arg is recent from the last dagger release.

[green otter]

I've fixed it by commenting out some code.

What do we do here?

[fierce olive]

Maybe a mismatch between your dagger engine and the lib ?

[green otter]

Let's see

• Engine: d54b05870ef7 (version v0.9.7)

[fierce olive]

Yep, the last lib on packagist is 0.9.10. This is one missing feature from the sdk. We need to implement the version check

[green otter]

So I should re-install Dagger engine 0.9.10 ? right? The website documentation told me specifically to install 0.9.7

So if I bring crowds of people to try this out, they're gonna hit the same problem as I just did.

I'm gonna upgrade my dagger engine now.

Thanks for the opinion @fierce olive it helped me a lot

[fierce olive]

The doc might be outdated then. If you run your script without using dagger run it should download the cli version matching the lib version and launch the graphql server as a process.

[green otter]

I'm gonna reinstall latest dagger engine and keep doing dagger run php ..

[green otter]

@fierce olive done, it works.

Next, how do I do this equivelat?

docker exec -ti <thing> sh

I did dagger run, the PHP container and the MariaDB container are up and running and I wanna shell in.

What's the standard practise here?

[fierce olive]

well this is pretty much what I'm trying to do with my consumer script. There's dagger shell but I think it's meant to be used with dagger modules (not yet implemented for php because there's some go code to implement to make it possible). What I'm doing is using the container.terminal function (https://docs.dagger.io/api/reference/#Container-terminal) and hack my way to connect to the web socket endpoint. (#1204482091610280058 message)

[green otter]

@fierce olive I thought that was the same thing. It's why I asked.

@fierce olive what's the quickest way to have us shell into the running PHP container?

[fierce olive]

I have no idea. I guess it's not really meant for this because CI is not interactive I suppose ? 🤔

[cobalt pelican]

Since PHP doesn't have module support yet, and it seems like you want to connect while it's running, you can try putting an SSH server in the container and connecting that way (or something similar).

[fierce olive]

Ah yes good idea or maybe the famous insecure nc -l -p 2200 -e /bin/bash.

@cobalt pelican should the SDKs implement connection for the terminal/shell endpoint or maybe some function to ease connection to the websocket or is it strictly reserved to dagger shell / modules ?

[green otter]

thanks for the advice - this isn't a huge deal for my demos - i was just curious

[cobalt pelican]

It's not strictly reserved to modules technically but would create an "imbalance" to have it only on PHP. We want to focus on supporting modules instead. That's how you get cross-language interop, and become truly a part of the larger ecosystem: https://daggerverse.dev

[green otter]

@zero you about today?

I'm going deep on our SDK.

I have ideas for symfony console commands for making "snippets" for the devs.

To make their DX awesome

[fierce olive]

Hello! I'm at a client this afternoon but can answer if you need anything. What kind of snippets ?

[green otter]

Give me a database.

Give em a cache

Give me a PHP CLI setup

Give me a PHP web app setup for my CI pipeline

Generate dagger files for them

We can use Twig

[fierce olive]

This is funny had the same idea while developing a custom cli for a client.

Tried with frankenphp docker image + dagger to get lightweight ephemeral backend to connect against a mobile e2e test suite

I think dagger can be a great way to scaffold apps in a sandboxed/reproducible env and customize everything using the same language

[green otter]

@fierce olive today I'm heading to PHPUK conf.

[fierce olive]

nice 🙂 I wish i could attend the conference

[green otter]

We did it ! 🙌

[high kiln]

👋

It was a fantastic talk Paul

[radiant flume]

Twitter was abuzz 🙂 Nicely done

[finite pilot]

Great talk @green otter and nice to (briefly) meet you afterwards. You have got me intrigued

[midnight parrot]

👋🏻

[green otter]

@fierce olive we have x2 graphql deps in composer.json

Why do we have 2 diff ones and not the same one?

I'm guessing there was a good reason

[fierce olive]

well there's the query builder used in the generated code and the spec implemetation used in the graphql introspection and codegen

[green otter]

my question is why did you use two different ones? what's wrong with one of them?

[fierce olive]

I used the graphql query builder because I don't think the graphql spec implementation package has a query builder

And I used the graphql spec lib because it's easy to manipulate types from the graphql schema with it

[summer sequoia]

Quick Q: when I run something like: $client->container()->from('php:8.3-fpm-alpine')->stdout() How can I tag that container and put it into my local docker so I can run it?

[radiant flume]

Here you go 😁 https://docs.dagger.io/252029/load-images-local-docker-engine/

[radiant flume]

The examples don't have php because it's not an officially supported sdk (yet!) but it's all the same API under the hood, so you should be in familiar territory.

[cobalt pelican]

More information in https://github.com/dagger/dagger/issues/6271

[summer sequoia]

hmm, so dagger isn't using the local docker at all 🤔

[radiant flume]

correct, but it can integrate with it

[grand yacht]

👋 finally got round to joining on here after the talk on friday

[radiant flume]

Welcome 🙂

[green otter]

👋 hey Matt @grand yacht

[summer sequoia]

I've been playing around with some design ideas for creating higher level interfaces for dagger wondered if anyone has any feedback

https://gist.github.com/carnage/6e82fe24987100e3b0fc7eab8f1767a7

couple of different ways of designing the DX for it

I'm leaning towards option 2

which could be turned into a plugin system in future using a PSR-11 container and other supporting stuff

[green otter]

Hey Chris! (@summer sequoia)

This is taking my Builder idea to a new level, with specifying the runtime - which I like.

Can you please update your GIST to show usage, which is how we can evaluate the DX.

Ping me here or send me a SMS message when you've updated it, so I can re-take a look 🙂e

[summer sequoia]

give me a few

@green otter updated

I think I prefer the builder varieties and also the version where all the parameters are provided at the same time - easier to make sure you've got a valid set

[green otter]

So let's name this a "runtime builder" not a "container builder"

Just like my code examples I gave, during my talk, it gets the base runtime, and gives you the Dagger instance back, so you can just keep doing stuff on it DIRECTLY.

DIRECTLY = because we don't want to just wrap the entire Dagger SDK, that's not good apporach, so it's a "dip in and dip out" approach here is the right balance.

[summer sequoia]

also example 2 cuts down the duplication of version, base and variant in method names and enum names

yeah the current example just returns a dagger container instance which you can do containery things to

[green otter]

2 is the best, chris, DX wise.
3 is more extensible in the future, since we're not using arguments but we're using the Query Builder pattern.

[summer sequoia]

but then I was wondering if it'd be useful to have specific PHP container objects and Mysql container objects so you could then do stuff like ->withComposer or ->withUser()

[green otter]

I get it, but that's actually just re-inventing the wheel of adding a layer on top of existing Dagger, which we want to avoid right now.

[summer sequoia]

yeah, I'm not super keen on extending auto generated code either

Option for now is trust the dev so if you've got an addComposer method trust them to give you a PHP container and not a mysql container

gives the flexibility for them to provide their own base image that way as well

[green otter]

I'm thinking ... Give me 5

I commented on your GIST, but here it is in all its glory

$runtime = PHPRuntime::create($dagger)
  ->version('8.3')
  ->os(OS::Alpine)
  ->addOption(Options::ZTS)
  ->addOption(Options::FPM)
  ->get();

[summer sequoia]

zts and fpm are mutually exclusive in the current images from what i can see

[green otter]

can you rephrase that? 🙂 laymans terms

[summer sequoia]

you can pick one and only one

zts, fpm, cli or apache

[green otter]

but we ship ZTS and non-ZTS copies of everything, Chris. So you can have ZTS CLI and non-ZTS CLI

[summer sequoia]

https://hub.docker.com/_/php/tags?page=1&name=8.3.0 << there don't appear to be tagged versions of that though

I was assuming zts would be a boolean option to start with

[green otter]

I looked, but anyway just trust me that's how it works. Maybe you're right at some level, but for the sake of our SDK API, we can bake in some things like "You can't have ZTS and FPM enabled at the same time, pick one" for example.

[summer sequoia]

Might be worth taking it up with the docker people and figure out why they treat zts as a separate thing

when as you say, it should be an option for each other variant

[green otter]

Now here comes the next question of my Builder idea.

Apt-Packages
PHP Extensions

Do we bake this into the Runtime builder? My opinion yes, and that's where I've been heading with this idea.

However, that's a bit tricky and there are no separation of concerns here.

Because installing packages is a verrry contextual thing, such as alpine or dpkg or apt stuff .. as well as compiling from source on some things .. so

I'm proposing we split it into a separate class, and you pass IN a container, and get back OUT a container too.

Such as:

UPDATE:

The more I think about this the more I want to add it into the original RuntimeBuilder class, as it has all the CONTEXT we need about that OS architecture.

[summer sequoia]

well, another thing I'm playing with is the php extension installer

https://github.com/mlocati/docker-php-extension-installer

that has all the apk & apt dependencies for building pretty much all php extensions baked into a shell script

[cobalt pelican]

I encourage you guys to get acquainted with Dagger modules: http://docs.dagger.io/zenith/. It would be great for PHP to have modules support.

[green otter]

@summer sequoia I'm already using this in my Builder demo .. did you watch the talk on youtube already?

[summer sequoia]

I watched it live :p

[green otter]

It's on the list. we're first getting more familiar with stuff before taking that on. Also i'm getting more people involved first, so there's more of us to take on that challenge.

Be prepared to have a call or 2 with me/us to really get into it, at the time 😉

[summer sequoia]

We'd still need sdk support to call the new modules though, right?

[green otter]

@summer sequoia why are you bringing up that extension? it is already in my PoC and it "just works" .. so Q: what extra stuff is there to consider?

[summer sequoia]

is your stuff on github somewhere I can look at the code?

[green otter]

It's on my laptop, beside me, i'll publish it soon. 🙂

[summer sequoia]

right now I'm just playing with some ideas and seeing what looks useful

[green otter]

$builder->aptPackages([....])
   ->extenions([ ....., ..., .. ]

The PoC code

which does the php-extension-installer thingy call

@summer sequoia let's add extension support to our PHPRuntime class, I'll update GIST

[summer sequoia]

I think this is starting to lean towards option 3 for the interface though

[green otter]

I agree. I am too, now that the scope of this runtime builder class is opening up to more things.

@summer sequoia so this is what my Builder PoC would look like, on top of our current class, and Option 3

https://gist.github.com/carnage/6e82fe24987100e3b0fc7eab8f1767a7?permalink_comment_id=4919732#gistcomment-4919732

        $runtime = PHPRuntime::create($dagger)
          ->version('8.3')
          ->os(OS::Alpine)
          ->addOption(Options::ZTS)
          ->addOption(Options::FPM)
          ->addPackages(['curl', 'nano', 'git', 'unzip')
          ->addExtensions(['pdo', 'pdo_mysql'])
          ->get();

[cobalt pelican]

Using PHP, yes.

[cobalt pelican]

I'm mentioning it now because I saw quite a few things at a glance from this convo, that modules actually solve. 🙂

[green otter]

@cobalt pelican example(s) ? pls

[summer sequoia]

As I understand it, a module would extend dagger's API so that anyone could call our Get PHP Runtime code to get a php container

but it would be a new graphQL endpoint which the sdk wouldn't know about

[green otter]

it'll be shipped with the SDK anyway, so they'll have it.

Having a module will allow people from other languages to call our runtime builder.

[cobalt pelican]

With modules you'll have an interface, with CLI over functions defined in code. You're able to do $dag->php('8.3') return a PHP container. Same for a DB container, etc. And it's cross language, so it'll be able to be used in modules written in Python or Go, for example.

[summer sequoia]

how does the skd know about installed modules though?

I've not seen anything on discovery

[green otter]

i.e: IDE Intellisense, on what methods are available to call, based on available dagger modules ☝️

[cobalt pelican]

As part of supporting modules in PHP, the codegen needs to be integrated in. So when you execute a module, the PHP codegen kicks in and generates the client with all the dependencies. When you do dagger develop, that codegened client is saved locally so your IDE can pick up the definitions as well.

[green otter]

Understood, good answer @cobalt pelican

[summer sequoia]

oh that's interesting

so we'd drop the composer package then?

[cobalt pelican]

Yes, you'd just make modules and publish in https://daggerverse.dev.

[green otter]

@cobalt pelican if you can prepare a document or list of URLs as to "why PHP SDK can't call the Dagger Module system" and "what's required in order to make PHP SDK talk to Zenith (module manager codename), then we can get started on it.

[green otter]

no. we still have dagger/dagger composer package. the SDK is till the entrypoint for everyone. The Module system just means the SDK (.php code user) can use other modules, written in other languages, right from the PHP SDK.

[cobalt pelican]

No, it's not just that.

[summer sequoia]

but if adding module support means adding codegen to dagger itself; you'd not need to install the composer package as it'd be codegen'd for you from your local dagger

[cobalt pelican]

Anyone could use a PHP module, even with only the dagger CLI in one's computer. Or other modules written in another language.

[summer sequoia]

If I understood correctly?

[green otter]

@cobalt pelican I get you.

I'm meaning from the perspective of a PHP Developer, they'd still pull in the the dagger SDK package, to get stuff done. Which was @summer sequoia's query.

composer require dagger/dagger

[cobalt pelican]

Modules run in containers. That runtime container is defined in a special module called a "runtime module". So PHP would have one such module to create the necessary environment for PHP code to run. As part of this, in order to say "sdk": "php" in a dagger.json file (i.e., shorthand rather than a full git repo ref), we actually bundle the SDKs in the engine container itself. So the SDK version you get is the one that corresponds to the dagger CLI you're using. There's no more need to install from composer, to be clear.

This is Python's runtime module: https://github.com/dagger/dagger/tree/main/sdk/python/runtime

[green otter]

@cobalt pelican okay so the PHP SDK will ship with dagger then, in that case.

[cobalt pelican]

Yes

[summer sequoia]

but you'd still need it locally for ide support

[cobalt pelican]

But that's done automatically by dagger develop.

[green otter]

PHP Devs will still need code-level access to the PHP SDK. As they'll have custom code sitting on top of the SDK.

How would you get the .php code, inside your current project (i.e: in /vendor/) same as in node_modules dir.

[cobalt pelican]

Because it simply triggers the Codegen function in the runtime module.

Python and Typescript have the same requirements. They both support modules, next to Go.

[green otter]

Okay.

[cobalt pelican]

We're (Dagger) working hard to make modules ready for a technical preview release pretty soon. Shortly after, it will be officially released. So we still need to document how to add support for it in an SDK. I'm just mentioning this because I think you should at least try using it (as a user) and see what modules can do, so it informs on what you want to do next with the SDK.

Potentially saving duplicated effort.

[green otter]

In PHP you can't just codegen files btw. It has to be loaded into our "package manager" which isn't just a package manager it's a Code Autoloader too, so it has to be all integrated.

You can load in classes into it using $autoload::addClassMap() and give it a directory of files that exist on the HDD, that weren't pulled in using Composer.

I'll pick this up with you again @cobalt pelican when you feel we're ready to add the PHP module to Zenith

Cool?

[cobalt pelican]

That's not a problem. The SDK that get's generated in your host should not be synced to the runtime container. The container will itself generate the files already. So there's no absolute paths problem. The local files are just for the IDE.

[green otter]

Noted about duplication. There are no current dagger modules that can support what we're building here, as it's bespoke to PHP ecosystem.

[summer sequoia]

logic is going to be the same so extracting it to a module shouldnt be too hard

[cobalt pelican]

What I meant is that you may be working on a solution for wrapping the SDK that modules actually solve and most people will move into the modules ecosystem anyway. 🙂

[cobalt pelican]

Until then, I can guide you step by step. The most difficult part is to find a DX for PHP to define functions and have the SDK be able to introspect those functions (name, arguments and their types, docstrings, return type, etc) in order to report to the API what those are, and be able to execute a specific function, in code, based on a few pieces of information (name of parent object, state of parent object in JSON, name of function to runs, name/value pairs for the function's inputs).

[cobalt pelican]

In terms of pipelines, sure. What I'm cautioning about is working on a tool/wrapper that "competes" with modules. To be clear, it's OK to do it! This is about value for your time, nothing else. 🙂

[summer sequoia]

yeah that makes sense; just trying to get my head around how modules would work along side PHP

~/git/blue$ dagger develop
✔ ModuleSource.resolveFromCaller: ModuleSource! 0.1s

Error: dagger develop on a module without an SDK requires either --sdk or --source

[cobalt pelican]

If you're new to it, you can try following the Quickstart to get a better sense of how it works as a user.

[green otter]

Thanks @cobalt pelican what we're doing isn't going to compete with existing Modules, as no other module can possibly exist for this.

In the future we'll make it a module 🙂

[cobalt pelican]

We're actively working on the Zenith docs.

[cobalt pelican]

no other module can possibly exist for this
How so?

[green otter]

Show me a module that builds PHP runtime docker image tags, subject to CLI or FPM or ZTS options.

[cobalt pelican]

That can be done in any language with Dagger. Doesn't have to be in PHP.

All languages are just DX on top of an API. It all translates to the same thing.

[green otter]

$runtime = PHPRuntime::create($dagger)
          ->version('8.3')
          ->os(OS::Alpine)
          ->addOption(Options::ZTS)
          ->addOption(Options::FPM)
          ->addPackages(['curl', 'nano', 'git', 'unzip')
          ->addExtensions(['pdo', 'pdo_mysql'])
          ->get();

Chris, we'll have to ensapsulate the php redis situation. This means on-the-fly we'll add php-dev dependencies, and add withExec() lines for the phpize and make commands

https://github.com/phpredis/phpredis/blob/develop/INSTALL.md#installation-from-sources

So we can have an addExtension('redis') method .. so if we find the redis key in the list, then we'll have to go over it, and add more execlines

Know what I mean?

Chris = @summer sequoia

This is all because our php-extension-installer doesn't support redis. It's always done via the clone method.

[summer sequoia]

The php extension installer I linked to does support redis

[cobalt pelican]

I think I haven't been clear enough. When I said "compete with modules", I wasn't talking about any one module that does something. I'm talking about the whole abstraction ecosystem from code to API/CLI on top of the pipelines you know and use today.

[green otter]

i see. useful! already done then.

what about blackfire extension? maybe that's still manual too ?

[summer sequoia]

It might be a useful goal long term to port that extension installer to PHP using dagger, but as MVP we just use it

[green otter]

newrelic

[summer sequoia]

blackfire is supported, newrelic isn't

https://github.com/mlocati/docker-php-extension-installer?tab=readme-ov-file#supported-php-extensions

[green otter]

awesome table!

I guess my point still stands, either we can bake in the stuff for newrelic .. or we just let them do ->withExec() lines themselves

[summer sequoia]

I think newrelic is niche enough that it can be done manually; there's probably a good reason the extension installer doesn't support it

[green otter]

Fair point.

@summer sequoia so there's nothing at the moment that my existing PoC doesn't cover. I can add the x2 Enums though, and we can test it out, and then I'll request adding it to our PHP SDK so everyone gets it via composer (still currently in discussions)

[summer sequoia]

Can you push up your code?

[green otter]

Sure!

[summer sequoia]

The thing I started with was building up a tag for the php docker image as I always forget the format for tags and have to look it up

not sure that was in your talk example - i think you used a single string

[green otter]

You're right, that's why I said "add the enums"

Here is the code: https://gist.github.com/dragoonis/c6c192d49c9df6746706ec603011d4a7

So the remaining part is the part that builds the docker image.

Maybe you can add 1 function that takes args and builds a docker image tag, that aligns with the convention on Docker Hub?

$tag = $version . '-' . (hasZts ? 'zts' : '') . (hasFpm ? 'fpm' : '');

Basically, what do you think ?

[summer sequoia]

I think it'll need a bit more smarts than that but that's the basic format

[green otter]

conceptually speaking 🙂 ☝️

[summer sequoia]

give this a look: https://github.com/carnage/sai/blob/main/example.php

Might adjust how it works so that everything is stored on the object and the container is only created (and returned) when you call fpm/cli/zts/apache

the package stuff might need some additional smarts; for now it just switches apt/apk based on the selected os

[green otter]

Sai? Lol

[summer sequoia]

:p

short, stabby, similar to a dagger

[green otter]

Chris, mlocati tool already ships with ALL the docker images anyway.. so why are you copying it manually?

[summer sequoia]

it doesn't? https://github.com/docker-library/php/blob/master/8.1/alpine3.18/zts/Dockerfile

it probably /should/

[green otter]

I'm quite sure it does ship on Alpine.

[summer sequoia]

well, it's not in that dockerfile

[green otter]

That's a surprise to me! Fair point.

[summer sequoia]

https://github.com/docker-library/php/issues/1498

[green otter]

@summer sequoia so .. your idea of variant isn't good. As it is only 1 value set.

We need to make it just OPTIONS or something.

You can have both apache and cli or apache and FPM

Rather than just 1 value

[summer sequoia]

not from the available image tags on docker hub

[green otter]

Ok

[summer sequoia]

cli is always included regardless of the image type

zts seems to only have a cli variant - it's not available as apache or fpm

i think apache includes fpm as well

so the apache image is apache+cli+fpm but is tagged on docker hub as apache

right, I'm heading off to play with some swords instead - let me know if you've got any other feedback on Sai

[green otter]

No feedback, I'm going to turn this into a formal PR, it will pave the way for future things we have in the works.

Thanks for idea 💡 on the image tag builder

🤝

[summer sequoia]

Might it be better to keep it out of the official SDK until we get module support?

[green otter]

No, we have more to come. CLI utils for making PHP snippets.

[radiant flume]

@summer sequoia @green otter @cobalt pelican may I suggest we bring the discussion of developing a module-aware PHP SDK here 🙂

[summer sequoia]

sure 🙂

[green otter]

@radiant flume you may! here to talk

sorry for the spam

[summer sequoia]

we were discussing it on Monday in here already ^

[green otter]

Q: how can i do withExec() that is NOT to be cached, what is the flag/function for it ?

For example I did a php -v and it didn't give me the output, it just said "CACHED"

Even if it does cache it, I still want to get the STDOUT from it.

[summer sequoia]

I'm not sure what the difference is between the current PHP SDK and one which supports the zenith stuff; is it just this runtime thing?

[green otter]

@summer sequoia zenith is just going to WRAP our current SDK and "bake it inside"

[summer sequoia]

Looking at the python example, the only file with any significan code in it seems to be this one: https://github.com/dagger/dagger/blob/main/sdk/python/runtime/main.go

[green otter]

@summer sequoia yea, see my above msg ☝️

It's just wrapping PHP runtime and pulling in our SDK and off we go.

Right now we're using the SDK directly via dagger run php and that will transition us not DIRECTLY doing composer require but instead it'll be baked inside.

I'll do the deep dive with Helger next week and hopefully get it all done in that call.

(this is all based on my current understanding of zenith and how it works) 🙂

[cobalt pelican]

No, the runtime is just the environment to run the PHP code in.

From the SDK's POV, the difference is in declaring the "TypeDef" API to declare functions, and then being able to invoke them. This is all done with API calls. See this Go example of registering a module: https://github.com/helderco/daggerverse/blob/a4ea781801e4c40452a19d850765f54d4513be52/go-python-codegen/dagger/dagger.gen.go#L664-L677.

It's easier to see here because Python and TypeScript introspect the code at runtime, so these calls are dynamic, while Go does it at codegen time. Go still executes these calls at runtime, just introspects the code in codegen.

Whenever you do a dagger call <function>, Dagger uses the module's SDK to run the proper runtime container which does an exec /runtime (where /runtime is an entrypoint executable from the PHP SDK). In this entrypoint PHP needs to call the API to query for information about which function to execute. See: https://github.com/helderco/daggerverse/blob/a4ea781801e4c40452a19d850765f54d4513be52/go-python-codegen/dagger/dagger.gen.go#L573-L587

[green otter]

PHP can give you both options really.

Our static analysis tool, mixed with our built-in AST means we can inspect the source code up front in any way we want, and check types and all that (just like what a compiled language does)

When it runs then you can also inspect the AST and use Reflection to see what's going on, at runtime

We've got great options here, really.

[cobalt pelican]

Great!

One part of the process is to find a nice DX for PHP, for users to declare their functions.

Go has an implicit system, just because there's no other way. Python and Typescript use decorators to explicitly define what gets exposed to the API or not.

[green otter]

PHP has Attributes, which you can use as Decorators. We took C#'s syntax for it. It's AOP-like 🙂 if you need decorators, we've got that covered too

[cobalt pelican]

Oh yeah.

Something like:

#[Object]
class MyModule {
  #[Function]
  public function build() {
    // my php pipeline here
  }
}

Called with dagger call build.

[green otter]

Seems reasonable. I can advise on a great DX once we're deep into the setup

Just like in Short Circuit movie "Need more input!" - feel free to keep feeding me more relevant stuff ..

[cobalt pelican]

The typing system for modules follows some constraints from GraphQL. Ultimately the purpose is to extend the GraphQL API where the above would allow this query:

query {
    myModule {
        build
    }
}

So functions are attached to objects, and there's a main object that's added to GraphQL's "Query" object as a field.

[green otter]

I get it now .. so the code Decorators are going to "build up" graphQL queries, then.

[cobalt pelican]

In other words, the above produces this schema:

type Query {
  myModule: MyModule!
}

type MyModule {
  build: String
}

How's PHP typing system?

It's important to be able to declare return types and types for arguments.

[green otter]

PHP's typing system is f**kin amazing

We don't have Generics, but we have a workaround 😉 our static analysis tool does the "compile time" checks, and finds the inconsistencies in the generics

[cobalt pelican]

Yeah, as long as DX is nice and you can get that info at code introspection time, that's all good.

[green otter]

We have excellent tools in this space, that tell you everything you wanna know about a piece of code. from the Annotations it has, to its types.

[cobalt pelican]

Good news is that's the hard part 🙂

[green otter]

16.5k github stars - PHP-Parser is a great tool for the job

https://github.com/nikic/PHP-Parser?tab=readme-ov-file#features

It'll tell you anything about a piece of code, either on disk or in memory.

[cobalt pelican]

Varies from language to language. Not just on difficulty but also on being able to delegate it in some common way to the server.

The server does try hard to handle as many decisions as possible, compared to earlier iterations.

[green otter]

Our main static analysis tool (12.4k stars) is actually probably better for the introspection

https://github.com/phpstan/phpstan

Once I know deeper what we wanna introspect, I'll be able to point us in the right direction 🙂

[cobalt pelican]

There's libraries that make a CLI based on functions in code. That can be a source of inspiration to find a good DX.

What's the most popular in PHP?

[green otter]

They're for slightly different use cases you see, so once I know deeper usecases I can say: "Use X"

There's also a very good Reflection API built right into the language, that we can use and might not even need these extra libs.

So: "it depends" 🙂

Here is an example of our Reflection API - to pull out the Attributes used on a class/function - https://www.php.net/manual/en/language.attributes.reflection.php

[cobalt pelican]

I just meant inspiration for the DX, not reusing the code internally.

[green otter]

There's a few ways, it's all about context. globally, in the community, it's inconsistent, but contextually it's consistent .. so again "it depends" 😅

My DX decisions are all about 1) familiarity 2) adoption

[cobalt pelican]

Yeah, familiarity and adoption was exactly why I suggested it, in case people are used to doing this in a certain way.

[grand yacht]

obviously quite a bit behind & trying to get up to speed by messing around with dagger & reading this (very insightful 🙂 ) conversation, assuming Dagger would only support on PHP 8+ then? Or would this just mean the OS running dagger would need to have 8+ on but you in theory could then pass a different php run time into it?

That's referring to the above about baking it in, not the current composer require

[cobalt pelican]

With modules you don’t need to have php installed locally. Even in the runtime container, it can support different versions.

[grand yacht]

Oh cool! The above syntax on attributes was only added at php 8 was what triggered my question.

[summer sequoia]

I don't think we'd target lower than PHP 8 though as even 8.1 is now out of general support

8.2 seems reasonable, but nothing against 8.3 since the version will only affect new stuff people are writing anyway

[green otter]

Hi @grand yacht

We'll be publishing dedicated documentation for PHP devs, when it comes to Modules, and the likes, so it will make sense to you. Stay tuned for that.

As for your Q about PHP versions. Basically the Module system will run "inside dagger" and it won't actually run on your local machine.

Yes you will execute things from your local machine, but it won't use your local machine's runtime. Thus versions won't matter too much.

There are a few "issues" I foresee, and aim to solve them next week when pairing with Helder on module runtime building.

Good question! Feel free to ask another 🙂

I will add this to our FAQ section, as it's a common question for sure

[grand yacht]

Hi both cheers make sense, I think one of the instant ideas I taken from the talk last week was being able to speed up classic CI pipelines on legacy codebases than run on php 7 for example. Which was what I was asking about but they'd essentially be seperate right as you pass the version in i.e. how you passed in 8.2 & 8.3 on the demo at PHP UK. As Dagger wouldn't be locked to the version of your project, it will be at whatever version is baked into dagger to understand attrbiutes etc.

[green otter]

@grand yacht correct. You can already do this with the current SDK.

[grand yacht]

Cool cheers, going to try and find time to create some of the scenarios I envisioned using dagger this weekend & dive deeper into it

[green otter]

Feel free to ping me once you've got a few scenarios down. We can have a call and I'll walk you through any bits. (Knowledge handover).

Just like @tidal geyser has done with me. Passing it on 😀

@grand yacht 👆

[slim thistle]

hi, I was trying to use the php sdk to test how to start services, here is my code:

#!/usr/bin/env -S dagger run php
<?php declare(strict_types=1);
require 'vendor/autoload.php';

use Dagger\Dagger;
use Dagger\NetworkProtocol;
use Dagger\PortForward;

$dag = Dagger::connect();
$service = $dag->pipeline('test')
    ->container()
    ->from('nginx')
    ->withExposedPort(80)
    ->asService()
;

$tunnel = $dag
    ->host()
    ->tunnel($service, [new PortForward(
        frontend: 8042,
        backend: 80,
        protocol: NetworkProtocol::TCP,
    )])
    ->start()
;

but here is the stacktrace:

PHP Fatal error:  Uncaught Error: Object of class Dagger\NetworkProtocol could not be converted to string in vendor/dagger/dagger/src/Client/AbstractInputObject.php
:35
Stack trace:
#0 vendor/gmostafa/php-graphql-client/src/Util/StringLiteralFormatter.php(39): Dagger\Client\AbstractInputObject->__toString()
#1 vendor/gmostafa/php-graphql-client/src/Util/StringLiteralFormatter.php(71): GraphQL\Util\StringLiteralFormatter::formatValueForRHS()
#2 vendor/gmostafa/php-graphql-client/src/Query.php(222): GraphQL\Util\StringLiteralFormatter::formatArrayForGQLQuery()
#3 vendor/gmostafa/php-graphql-client/src/Query.php(249): GraphQL\Query->constructArguments()
#4 vendor/gmostafa/php-graphql-client/src/FieldTrait.php(64): GraphQL\Query->__toString()
#5 vendor/gmostafa/php-graphql-client/src/Query.php(238): GraphQL\Query->constructSelectionSet()
#6 vendor/gmostafa/php-graphql-client/src/FieldTrait.php(64): GraphQL\Query->__toString()
#7 vendor/gmostafa/php-graphql-client/src/Query.php(238): GraphQL\Query->constructSelectionSet()
#8 vendor/gmostafa/php-graphql-client/src/Client.php(115): GraphQL\Query->__toString()
#9 vendor/dagger/dagger/src/Client/AbstractClient.php(40): GraphQL\Client->runQuery()
#10 vendor/dagger/dagger/src/Client/AbstractObject.php(20): Dagger\Client\AbstractClient->queryLeaf()
#11 vendor/dagger/dagger/generated/Service.php(70): Dagger\Client\AbstractObject->queryLeaf()
#12 test.php(20): Dagger\Service->start()
#13 {main}
  thrown in vendor/dagger/dagger/src/Client/AbstractInputObject.php on line 35

is there a bug or did I do something wrong ? 🤔

[summer sequoia]

That looks like a bug to me

I'm assuming that NetworkProtocol is an enum and the graph ql client is attempting to convert it to a string unsuccesfully

the enum needs converting somewhere.

[slim thistle]

that what I though, I will try to propose something soon

[fierce olive]

Yes I think you need to do NetworkProtocol::TCP->value need improvements

[green otter]

Hey @slim thistle I didn't see your message until now.

[slim thistle]

hi 🙂 sorry I did nothing on github because I was off
if you have the time, don't hesitate to do it

[green otter]

@fierce olive could patch it

[zealous venture]

I have added a PR with an example of using Dagger Functions with a PHP app, lmk any feedback https://github.com/dagger/dagger/pull/7027

[summer sequoia]

I added a suggestion to your PR

[summer sequoia]

Did we get anywhere with module support for PHP?

[zealous venture]

Thank you! I replied in the PR

[summer sequoia]

I think your answer brings us back to my second message: where are we with module support for the PHP sdk?

[fierce olive]

I'd love to help bring dagger function to the PHP SDK. Started learning some Golang but I don't have time for now

[cobalt pelican]

You can have help for making the runtime module in Go but that's a small part of it. Most of the work needs to be done in PHP.

[summer sequoia]

I'm planning to get the work done on it starting next week when one of my staff is free

Do you have any docs on exactly what needs to be done to get it working?

I've studied the runtimes for python and typescript: all they seem to do is return a container for running things in

and a few utility bits

[cobalt pelican]

Yes, that's all it needs to to. That and running codegen.

[summer sequoia]

Didn't seem very complex which was why I was wondering why it hadn't been done yet

[cobalt pelican]

Just someone with time to do the work 🙂 Just doing this isn't enough for supporting modules though. So even if you do it and don't work on the rest, you won't have anything that works.

And it's not a required first step even. You can go a long way in PHP before working on the runtime module.

[summer sequoia]

yeah; it's the "what is the rest" that needs doing that I'm unclear on

I've got a junior dev with time between other projects who I was going to get to work on this, but I need to know what this is :p

[cobalt pelican]

I'm going to document that soon. Major things to work out in PHP is:

• Decide on DX: how does it look like to author a Dagger Function in PHP?
• Work out the introspection: you need to be able to introspect the user's code and know which functions are available, with arguments and return type included
• Work out the serialization: you need to be able to call a function based on it's name (string), inputs (JSON), parent object (string) and parent object's state (JSON); and return the result in JSON.
• Build out the SDK (entrypoint): the runtime module will need an entrypoint (executable) which can be in PHP, as part of the SDK, which will do all the introspection, serialization, establish a dagger connection and invoke the function. Python and Typescript just import a "main" function inside the SDK to do all of this, in a sub-package for modules.

[summer sequoia]

DX should be fairly straightforward: looking at the python stuff there seems to be two decorators for object and function; we could mirror that in PHP with attributes

We also have very solid reflection capabilities so that should be doable as well

[cobalt pelican]

The current SDKs are doing that a bit differently:

• The Go SDK is doing it during codegen, so it's generating the code that will make the API calls that use this introspection. This is more performant.
• The Python SDK is doing the introspection at runtime (when the runtime container executes the entrypoint). This is easier to develop with, if you have good reflection at runtime.
• The TypeScript SDK is doing it at runtime, but uses a separate process for the introspection, and uses the result in the entrypoint (also at runtime). Separate process required because that information isn't available at runtime.

[summer sequoia]

the parent objects state might be a tricky one

I think PHP would do it the same as the python one

[cobalt pelican]

How do you instantiate an object in PHP from JSON? Basically need to instantiate that object in order to call a method from it (i.e., the function to execute). But... you can start simple and not support state until later. You can start with making a function work that has no arguments and only returns a primitive like string or boolean.

So basically you only require the function's name as a string.

[summer sequoia]

you can't do that natively: there are a couple of options; packages that serialise/deserialise to/from json

[cobalt pelican]

That's ok. Python uses a library, not because it's necessary but because it makes it easier to maintain.

[summer sequoia]

the sticking point is some things can't be serialised very easily/at all eg sockets and anon functions/classes

[cobalt pelican]

That's also ok. We're limited by what can be used in GraphQL so Dagger doesn't support everything. Python has lambdas (like anonymous functions) but they're not supported in Dagger too.

[summer sequoia]

how does dagger make the dagger client available to python user modules?

[cobalt pelican]

In the end what this PHP function will do, is add a GraphQL field for it in the API. Modules are for expanding the API schema.

[summer sequoia]

My understanding is that they are good for code reuse?

So the community could create a PHPUnit module which runs tests and produces code coverage artifacts etc then anyone can use that module for tests

[cobalt pelican]

So, when a Python module depends on a PHP module, Dagger loads the PHP module, thus expanding the API schema. When the Python module loads, in the codegen phase, it generates the client from the API, not knowing or caring where it came from. Could be core or a module in any supported language.

[cobalt pelican]

Yes.

[summer sequoia]

Will be quite valuable for anyone who has a PHP backend and JS/typescript frontend

Can the PHP SDK currently call python modules?

[cobalt pelican]

Exactly. Can be used even if you're a Go engineer that needs to create a CI pipeline for a PHP app.

It can, but it won't have the API additions from the module in PHP's generated client. But it can use a GraphQL query to do it.

We have plans to fix that (for all SDKs).

Design for this feature isn't done yet though. Meaning we need to invest some cycles into designing a solution for this.

[summer sequoia]

Is there an example python module somewhere?

[cobalt pelican]

This one is basic: https://github.com/helderco/daggerverse/tree/main/hello-world

[summer sequoia]

Digging into it more; I think that the runtime is probably the first bit to do, as you need that to test everything else

at the very minimum, once the runtime is done; you should be able to call dagger dev --sdk php and have it codegen you an empty module

[cobalt pelican]

Not strictly necessary since you can mock the api calls while developing and you can get pretty far with that, but yes, to make it work with the dagger CLI you'll need the runtime container.

[slim thistle]

hi, any idea why the CI in my PR is failing?
https://github.com/dagger/dagger/pull/7285

[tepid hornet]

Hey @slim thistle. I see that one of the failures is the TypeScript sdk. We've been having a bit of flakiness in those tests for a bit now. I haven't checked if its an actual test that is broken in a non deterministic way or if its something else. Maybe @cobalt pelican knows about it? I'm not sure, sorry for the ping

[cobalt pelican]

Yeah, we've been having a lot of flakes in CI. We run CI on latest release so we need a new release (v0.11.3) for CI to pick up on fixes that have already been merged. Until then, if the error seems unrelated to your changes, then feel free to ignore.

[slim thistle]

does phpunit run well on the main branch? on my machine I have Errors: 4, Warnings: 5

[green otter]

I'm back 🙂

Been out of the game for a little while

[radiant flume]

https://tenor.com/view/i-have-returned-general-kirigan-ben-barnes-shadow-and-bone-im-back-gif-12199656944440281441

[green otter]

I'm reading up.

@cobalt pelican

How do you instantiate an object in PHP from JSON? Basically need to instantiate that object in order to call a method from it (i.e., the function to execute)

Can you show me an example of this JSON, please? I need to visualize it, to see how we can execute things in the PHP runtime

[cobalt pelican]

With the help of AI... 😁

Example class:

class Book {
    public $title;
    public $author;

    public function __construct($title, $author) {
        $this->title = $title;
        $this->author = $author;
    }

    public function describe($format) {
        if ($format == 'short') {
            return "Book: " . $this->title;
        } else {
            return "Book: " . $this->title . ", Author: " . $this->author;
        }
    }
}

After JSON being parsed, just focus on these data structures:

// Name of the class
$parentName = 'Book';

// Initial state of the instance
$parent = array(
    'title' => '1984',
    'author' => 'George Orwell'
); 

// Name of the method to call
$name = 'describe';

// Inputs to the method call
$inputArgs = array(
    'format' => 'long'
);

// Result of the method call, parsed to JSON
$returnValue = '"Book: 1984, Author: George Orwell"'

You get the above after making these API calls: https://docs.dagger.io/api/reference/#definition-FunctionCall (after JSON parsing and putting input args in an array). I chose those var names to match the separate API fields.

The essential is you need to be able to:

• Create an instance of class $parentName
• Initialize it's attributes with the state in $parent
• Then execute the method from this instance in $name, with arguments in $inputArgs
• Parse result into JSON in $returnValue

Here's an example (Python) of getting and preparing these values: https://github.com/dagger/dagger/blob/b8409fb11a163168860259f26fa27cb9e2e7b48b/sdk/python/src/dagger/mod/_module.py#L263-L276.

[serene light]

@cobalt pelican What steps would be involved in setting up a new runtime for PHP? I've created a new directory "runtime" with "dagger.json" in it but it does not seem to be picked up by the engine

[cobalt pelican]

You run it with dagger init --sdk=<path to dir with runtime module> ...

[serene light]

Running the command:
../dagger/bin/dagger init --sdk=php potato
Outputs

✔ connect 0.5s
✘ ModuleSource.resolveFromCaller: ModuleSource! 0.0s
! failed to collect local module source deps: failed to collect local sdk: missing config file /home/imhotek/git/daggertest/potato/php/dagger.json

Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to collect local sdk: missing config file /home/imhotek/git/daggertest/potato/php/dagger.json

But if I try running it with the relative path to it:
../dagger/bin/dagger init --sdk=../dagger/php potato
I get this:

✔ connect 0.5s
✘ ModuleSource.resolveFromCaller: ModuleSource! 0.0s
! failed to collect local module source deps: failed to collect local sdk: local module dep source path "../dagger/php" escapes context "/home/imhotek/git/daggertest/potato"

Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to collect local sdk: local module dep source path "../dagger/php" escapes context "/home/imhotek/git/daggertest/potato"

It seems to either get upset from not finding the config, because it's checking my PWD or it gets upset because I "escape context" to get to the runtime

[cobalt pelican]

You can't use --sdk=php yet, so you need to use a path (git or local) instead.

[cobalt pelican]

It needs to be in the same repo.

Let's say you're developing in the dagger repo, in sdk/php:

# create runtime module
dagger init --sdk=go --name=php-sdk --source=runtime runtime

# edit code and test it
dagger init --sdk=./runtime modules/potato

[faint current]

hello, on above same topic let say i ahve an empty repo and i want to install dagger with php sdk which will be the steps for it?

[cobalt pelican]

Like second command above, but full git address:

dagger init --sdk=https://github.com/dagger/dagger/sdk/php/runtime

Anyone has an example of running a dagger pipeline with PHP? For #1238191490207842325 message. Ping @green otter

[serene light]

@faint current I found this example while researching, it may be outdated at this point. I haven't had a change to look through it yet but it is a PHP tutorial https://archive.docs.dagger.io/0.9/128409/build-test-publish-php/

[faint current]

thanks, let me take a look

[serene light]

I'm trying to get an MVP SDK for PHP working.

I've created a runtime using the first command you suggested:

# create runtime module
dagger init --sdk=go --name=php-sdk --source=runtime runtime

That worked, thank you!

Currently the runtime directory looks like this:

internal/
dagger.gen.go
dagger.json
.gitattributes
.gitignore
main.go

I've been editing the main.go file, using the Python and Typescript SDKs as references.
But I am struggling to test it.

~/git/dagger/sdk/php$ dagger init --sdk=./runtime modules/potato -vvv
✔ connect 1.2s
  ✔ exec docker start dagger-engine-d3f2fbf13f75a975 0.0s
  ┃ dagger-engine-d3f2fbf13f75a975                                                                           
  ✔ starting engine 0.1s
  ✔ starting session 0.2s
  ┃ OK!                                                                                                      
✔ moduleSource(refString: "modules/potato"): ModuleSource! 0.0s
✔ ModuleSource.kind: ModuleSourceKind! 0.0s
✔ ModuleSource.resolveContextPathFromCaller: String! 0.0s
✔ ModuleSource.withName(name: "potato"): ModuleSource! 0.0s
✔ ModuleSource.withSDK(sdk: "./runtime"): ModuleSource! 0.0s
✔ ModuleSource.withSourceSubpath(path: "dagger"): ModuleSource! 0.0s
✘ ModuleSource.resolveFromCaller: ModuleSource! 0.0s
! failed to collect local module source deps: failed to collect local sdk: missing config file /home/imhotek/git/dagger/sdk/php/modules/potato/runtime/dagger.json

I've tried replacing the sdk flag with random rubbish and I get the same error. So I don't think it's picking up anything from the runtime, so I feel like I must be missing something.

[cobalt pelican]

There may have been a regression. It seems like it's trying to load the sdk's path relative to the new module's path. cc @obsidian forge @worthy ice

There's actually a test that should cover this: https://github.com/dagger/dagger/blob/e678cdd17bee86faea9837e10dd0c8612994aa7c/core/integration/testdata/modules/python/git-dep/dagger.json#L3. Let me try with ../.

[serene light]

That was it, thank you!
I had to run
~/git/dagger/sdk/php$ dagger init --sdk=../../runtime modules/potato -vvv
It's getting into the runtime, pulling the correct container, then failing on the Codegen function (which is what I expect at the moment)

Error: failed to generate code: input: moduleSource.withContextDirectory.withName.withSDK.withSourceSubpath.asModule resolve: failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to call sdk module codegen: select: call function "Codegen": process "/runtime" did not complete successfully: exit code: 2

Stdout:
marshal: json: error calling MarshalJSON for type *dagger.GeneratedCode: input: container.from.directory resolve: lstat /src: no such file or directory

I'll see what I can do with this

[cobalt pelican]

~/git/dagger/sdk/php$ dagger init --sdk=../../runtime modules/potato -vvv

Yeah, that's bunkers! Shouldn't be looking for the sdk inside the module to run. 😄

[summer sequoia]

So a quick Q. After making changes to the prototype runtime module; what are the steps required to get the dagger engine to pick up those changes? Go requires compiling, right? or does the engine take care of that automatically?

[cobalt pelican]

Nothing special, it should pick up on the changes automatically.

[serene light]

We're trying to create the PHP runtime, at the moment we just want it to create a hardcoded module.

We're getting this issue with the proxySpan which seems unrelated to what we have written so far. It doesn't look like the Python or Typescript runtimes use this either

Error: failed to generate code: input: moduleSource.withName.withSDK.withSourceSubpath.resolveFromCaller resolve: failed to collect local module source deps: failed to load local sdk module source: select: failed to initialize module: failed to call module "php-sdk" to get functions: call constructor: process "go build -o /runtime ." did not complete successfully: exit code: 1


Stderr:
# github.com/dagger/dagger/sdk/php/runtime/internal/telemetry
internal/telemetry/proxy.go:50:14: cannot use proxySpan{…} (value of type proxySpan) as "go.opentelemetry.io/otel/trace".Span value in return statement: proxySpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
internal/telemetry/proxy.go:59:20: cannot use proxySpan{} (value of type proxySpan) as "go.opentelemetry.io/otel/trace".Span value in variable declaration: proxySpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)

[cobalt pelican]

Context here: https://github.com/dagger/dagger/issues/7314

[serene light]

Brilliant, thank you!

[summer sequoia]

Container.withExec(args: ["/codegen/sdk/php/install-composer.sh"]): Container! 0.0s << how can I get the output from running this command; it's not working for some reason

[summer sequoia]

@cobalt pelican I'm making some progress on the PHP SDK, however my current sticking point is working out how to mount the whole sdk/php directory into my container - I've tried replicating the way the typescript module works but that fails; other options I've tried seem to only mount the php/sdk/runtime directory (or in some instance the sdk directory but still only the runtime dir is present in the php directory)

My current wondering is if calling the SDK module as ../../runtime is ever going to work, as there seems to be some special setup for sdks in the core/schema/sdk.go file

[cobalt pelican]

It's better to look at the Elixir module here, since that's not bundled like the others, same as PHP needs to do: https://github.com/dagger/dagger/pull/6967

There's two problems here.

First there's a bug that if you use a relative path on --sdk, it's being resolved in relation to --source. So if you try to init, with --source on a subdir, you need to undo it in --sdk:

-dagger init --sdk=sdk/php/runtime --source=demo demo
+dagger init --sdk=../sdk/php/runtime --source=demo demo

When not developing, end users should use the git remote URL instead: dagger init --sdk=https://github.com/dagger/dagger.git/sdk/php/runtime@v0.11.5 --source=demo demo (for example).

The other issue is that we need to find a better solution for getting the SDK's sources in the runtime. As you can see with Elixir, it's hardcoded for getting the sources from main (see code). So if you use a URL in --sdk for version v0.11.5 like above, you'll get newer SDK files, from main, which can potentially be incompatible. The quick fix here is to change the URL in --sdk to point to main, but that may still be incompatible with the dagger CLI version that you're running. Ideally they'd all be the same version.

There's a few ideas to help fix this, like with https://github.com/dagger/dagger/issues/7199 or https://github.com/dagger/dagger/issues/7432, the runtime module would be able to access the parent directory.

[summer sequoia]

The option I was considering was making the runtime the base directory and putting all the php code in a sub dir of runtime

I just tried using the hardcoded git repo to get the code but weirdly it's not pulled all the files in

      ✔ exec ls -al /codegen 0.1s
      ┃ drwxr-xr-x    2 root     root          4096 May 28 13:52 generated                                                                                                                                                          
      ┃ -rw-r--r--    1 root     root           709 May 28 13:52 phpunit.xml.dist                                                                                                                                                   
      ┃ -rw-r--r--    1 root     root           522 May 28 13:52 psalm.xml                                                                                                                                                          
      ┃ drwxr-xr-x    7 root     root          4096 May 28 13:52 src                                                                                                                                                                
      ┃ drwxr-xr-x    5 root     root          4096 May 28 13:52 tests  

should contain all of: https://github.com/dagger/dagger/tree/main/sdk/php

[cobalt pelican]

Something simple in the meantime would be to put the entire sdk in the runtime module's sources. Basically put dagger.json in sdk/php and point source to runtime, but then the path to the runtime module is just sdk/php. Make sure to add the proper exclude patterns in dagger.json for the files that don't matter in the runtime.

[cobalt pelican]

I think we were saying the same thing at the same time 😄

[cobalt pelican]

I'd need to look at some code, but basically: cd sdk/php && dagger init --sdk=go --source=., which should put the dagger.json at the root of sdk/php. Then, edit dagger.json to add "exclude" and "include" patterns appropriately. You can then add a function in the module to list what files are actually uploaded.

[summer sequoia]

https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/runtime/main.go#L34

[cobalt pelican]

That looks ok, how are you testing what's inside?

[summer sequoia]

rm -rf modules/potato && dagger init --sdk=../../runtime modules/potato -vvv

I'm confused as to why it's only got half the sdk dir in there though

[cobalt pelican]

I'll have to look at the codegen part of the module. You can make a function to debug the contents of sdkSourceDir directly.

[summer sequoia]

I've currently got this:

    ctr = ctr.WithMountedDirectory("/codegen", sdk.SourceDir).
        //WithEntrypoint([]string{"/bin/sh", "-c"}).
        WithoutEntrypoint().
        //WithWorkdir("/codegen/sdk/php").
        WithExec([]string{
            "ls", "-al", "/codegen",
        })
    ctr.Stdout(ctx)

but the output is what I posted above. a few files eg composer.json are missing from the directory but they are clearly there in git

[cobalt pelican]

You can run the module directly and use the cli to inspect the directory directly. Let me try it.

[cobalt pelican]

Try this:

dagger -m "github.com/carnage/dagger/sdk/php/runtime@add-php-runtime" call source-dir glob --pattern '**/*'

This will more accurately show you what's in that dir.

[summer sequoia]

hmm so why is ls not picking it up?

[cobalt pelican]

My guess is you're getting truncated output. It's better to debug via dedicated functions and execute them directly.

Let's say you add this:

func (sdk *PhpSdk) Test() *Container {
    return sdk.Container.WithWorkdir("/work").WithMountedDirectory("", sdk.SourceDir)
}

Then you can terminal in and look around:

dagger -m runtime call test terminal
/work # ls -lha
total 180K
drwxr-xr-x    1 root     root         388 May 28 14:31 .
drwxr-xr-x    1 root     root          58 May 28 14:41 ..
drwxr-xr-x    1 root     root         358 May 28 14:31 .changes
-rw-r--r--    1 root     root        1.2K May 28 14:31 .changie.yaml
-rw-r--r--    1 root     root          32 May 28 14:31 .gitattributes
-rw-r--r--    1 root     root         115 May 28 14:31 .gitignore
-rw-r--r--    1 root     root         306 May 28 14:31 .php-cs-fixer.dist.php
-rw-r--r--    1 root     root        8.9K May 28 14:31 CHANGELOG.md
-rw-r--r--    1 root     root       10.5K May 28 14:31 LICENSE
-rw-r--r--    1 root     root        1.0K May 28 14:31 README.md
-rwxr-xr-x    1 root     root         621 May 28 14:31 codegen
-rw-r--r--    1 root     root         701 May 28 14:31 composer.json
-rw-r--r--    1 root     root      114.7K May 28 14:31 composer.lock
drwxr-xr-x    1 root     root          28 May 28 14:31 docker
-rw-r--r--    1 root     root        1.3K May 28 14:31 docker-compose.yml
drwxr-xr-x    1 root     root        2.4K May 28 14:31 generated
-rw-r--r--    1 root     root         709 May 28 14:31 phpunit.xml.dist
-rw-r--r--    1 root     root         522 May 28 14:31 psalm.xml
drwxr-xr-x    1 root     root         122 May 28 14:31 src
drwxr-xr-x    1 root     root          80 May 28 14:31 tests
/work #

Or just run a command directly from the cli:

dagger -m runtime call test with-exec --args=ls,-lha stdout
total 180K
drwxr-xr-x    1 root     root         388 May 28 14:31 .
drwxr-xr-x    1 root     root          50 May 28 14:43 ..
drwxr-xr-x    1 root     root         358 May 28 14:31 .changes
-rw-r--r--    1 root     root        1.2K May 28 14:31 .changie.yaml
-rw-r--r--    1 root     root          32 May 28 14:31 .gitattributes
-rw-r--r--    1 root     root         115 May 28 14:31 .gitignore
-rw-r--r--    1 root     root         306 May 28 14:31 .php-cs-fixer.dist.php
-rw-r--r--    1 root     root        8.9K May 28 14:31 CHANGELOG.md
-rw-r--r--    1 root     root       10.5K May 28 14:31 LICENSE
-rw-r--r--    1 root     root        1.0K May 28 14:31 README.md
-rwxr-xr-x    1 root     root         621 May 28 14:31 codegen
-rw-r--r--    1 root     root         701 May 28 14:31 composer.json
-rw-r--r--    1 root     root      114.7K May 28 14:31 composer.lock
drwxr-xr-x    1 root     root          28 May 28 14:31 docker
-rw-r--r--    1 root     root        1.3K May 28 14:31 docker-compose.yml
drwxr-xr-x    1 root     root        2.4K May 28 14:31 generated
-rw-r--r--    1 root     root         709 May 28 14:31 phpunit.xml.dist
-rw-r--r--    1 root     root         522 May 28 14:31 psalm.xml
drwxr-xr-x    1 root     root         122 May 28 14:31 src
drwxr-xr-x    1 root     root          80 May 28 14:31 tests

[summer sequoia]

right; this should get me where I need to be to get the codegen running I think

[summer sequoia]

I've now got to the point at which the problems are with the PHP code 😄

[cobalt pelican]

That's good progress 🙂

[summer sequoia]

yeah, it's trying to spawn a process to connect to dagger via the unix socket I think

that's obviously failing due to the container not having that socket

I think the solution is to remove that connection code and read the schema from the container's filesystem

[cobalt pelican]

Yeah, the PHP code needs to be able to connect directly with DAGGER_SESSION_TOKEN and DAGGER_SESSION_PORT if defined. Those will be available both on dagger run and dagger call.

Notice that's tcp, not sure why there's a unix socket there.

[summer sequoia]

I think it's expecting to be able to use the local docker setup

hmmm, it does have code to use those env vars

are they automatically added to every container run in dagger or do they need to be explicitly passed?

[cobalt pelican]

The docker setup I think it was a dev environment setup through docker compose, but it wasn't necessary and should be replaced with just the dagger cli.

[cobalt pelican]

No, you don't have to worry about that at all. The SDK just needs to connect to those if available, and not do any more provisioning like calling dagger session or anything.

[summer sequoia]

hmm, need to debug why it's failing to connect then

[cobalt pelican]

Doesn't it connect with dagger run php script.php? Making sure that no dagger session is being spun, of course.

[summer sequoia]

looks like it's first attempting to connect using 127.0.0.1:{DAGGER_SESSION_PORT}

but that env var isn't set inside the container

so it seems to be falling back to using the dagger cli

which fails as it's not in the container either

[cobalt pelican]

Hmmm, how are you running it?

[summer sequoia]

right now

dagger -m "runtime" call codegen -vv

WithExec([]string{"env"})

PHPIZE_DEPS=autoconf            dpkg-dev dpkg           file            g++             gcc             libc-dev                make            pkgconf                 re2c
PHP_INI_DIR=/usr/local/etc/php
PHP_CFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
PHP_CPPFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
PHP_LDFLAGS=-Wl,-O1 -pie
GPG_KEYS=1198C0117593497A5EC5C199286AF1F9897469DC C28D937575603EB4ABB725861C0779DC5C0A9DE4 AFD8691FDAEDF03BDF6E460563F15A9B715376CA
PHP_VERSION=8.3.7
PHP_URL=https://www.php.net/distributions/php-8.3.7.tar.xz
PHP_ASC_URL=https://www.php.net/distributions/php-8.3.7.tar.xz.asc
PHP_SHA256=d53433c1ca6b2c8741afa7c524272e6806c1e895e5912a058494fea89988570a
FTP_PROXY={"parentClientIDs":["7iw8ig74s4abwofrs1qcn4sxz","sgnu6yjt2dop655nwuer17rxi"],"serverID":"zoy8rj4gg715hjztzmfmr878o"}
OTEL_TRACES_EXPORTER=otlp
OTEL_TRACE_PARENT=00-24d63c551ebdf923028cf16a4521ead5-e418ab2c7499c7e3-01
TRACEPARENT=00-24d63c551ebdf923028cf16a4521ead5-e418ab2c7499c7e3-01
_DAGGER_PARENT_CLIENT_IDS=7iw8ig74s4abwofrs1qcn4sxz sgnu6yjt2dop655nwuer17rxi
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:40971
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://127.0.0.1:40971
OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://127.0.0.1:40971
OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://127.0.0.1:40971
HOME=/root

[summer sequoia]

Code is an absolute STATE but it has working codegen: https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/runtime/main.go

[summer sequoia]

Dropped a PR in for the php runtime module; https://github.com/dagger/dagger/pull/7493 not ready for merge yet but there for visibility/feedback

[green otter]

@cobalt pelican I'm on a call with @summer sequoia

We're going through his Golang code (and php edits) line by line, so that I understand everything, and can assist @summer sequoia in taking things forward.

@cobalt pelican @tidal geyser @radiant flume 👋 🙂

So in a Dagger Module, we need to make some API calls, (at dagger call time), to grab the Module Name / Module Args.

Can you link me to some javascript examples of how it's doing this, so we can make those calls in PHP land?

[cobalt pelican]

Sorry, I’m on my phone 🙂 the module name is only needed to generate the right class name in the default template (PascalCased module name).

[green otter]

Hey @cobalt pelican so I found this, and have been reverse engineering it.
https://github.com/dagger/dagger/blob/main/sdk/typescript/entrypoint/entrypoint.ts#L26-L50

Next question.

What is result, as in once we actually CALL the js/php code, and run the operation

Is it STDOUT we return back to dagger engine, after it invokes our entrypoint??

[cobalt pelican]

You must return the result in an api call, not stdout.

[green otter]

I know, I meant when we call Dagger Engine API, with the result .. what am I returning? 🙂

returning stdout?

do you have an example sample return value for me to look at?

[cobalt pelican]

That entrypoint is called two times. First for loading the module, in which case parentName is empty, you have to return a ModuleID. The second time, parent name is not empty and you have to return the serialized result of executing the function. JSON

[green otter]

Okay, we're pushing the result, in JSON format, back to dagger API - got that part ..

Serialized result .. can you give me an example of a serialized result in python/js/golang?

[cobalt pelican]

Even if it’s a ModuleID it should be in JSON. I’m on my phone so it’s hard for me to get examples but start with a string. If the string is "hello" then the json is "\"hello\"".

Have you registered the function, with correct return type? It’s not enough to send a result. Dagger needs to know the types before that.

[green otter]

sorry, my notifications are off 🙈

Okay, upon first call (no parent) I can return a string with the ModuleID, that's no bother

$result = serialize($moduleId)

Maybe when you're not at your phone ... can you tell me what a result example looks like on the second call ?

Let's say I'm running a linter QA checker, and the output of that linter QA checker is STDOUT stuff..

Maybe this is enough?

$buffer = start_output_buffer()
$linterLibrary->run($fnArgs..etc)
$output = end_output_buffer()

return serialize($output)

[cobalt pelican]

Yeah. The function in user code would return $output and the entrypoint in the sdk serializes that before passing to returnValue API call.

[green otter]

Understood.

Since we can't return actual serialized PHP code, since dagger doesn't know what to do with that 🙂 so returning serialized string (stdout) is probably the way forward .. right ?

[cobalt pelican]

In Python: https://github.com/dagger/dagger/blob/71b8135ca153bc91fac70f81684f926b0d4870e5/sdk/python/src/dagger/mod/_module.py#L212

No you’re not supposed to return the contents of the function. The sdk needs to execute the user’s function and serialize that return, whatever it is.

It’s not the php code that needs to be serialized.

function test() { return "hello" }

That’s a user function.

[green otter]

It’s not the php code that needs to be serialized.

Yup, this is what I'm trying to say too. It's just the result of it

[cobalt pelican]

Yep

[green otter]

What do you think about this logic?
PHP entrypoint code - it's pseudocode, just to show how it works

list($moduleName, $functionName, $functionArgs) = getStuffFromDagger()
list($classToCall, $functionToCall, $argsToCall) = $introspectionThingy->introspect($moduleNane, $functionName, $functionArgs)

$buffer = start_output_buffer()

$$classToCall->$functionToCall(...$argsToCall)

$output = end_output_buffer($buffer)
$json = json_encode($output)

$dagger->setReturnValue($json)

exit 0

🙂

[cobalt pelican]

The output buffer thing is out of place. That’s something that maybe a user function would use but not the sdk

[green otter]

This is our current php runtime module entrypoint - https://github.com/dagger/dagger/pull/7493/files#diff-6798aa21136fec549e40af53bd517bd3343f6b5bc54536ffc5ad0cd298124349

it gets called from main.go here - https://github.com/dagger/dagger/pull/7493/files#diff-0586053a20f05d1973e9fafdb90d0e09153f7943df4f7983b78f88e0338a5beeR143

This dagger PHP entrypoint will invoke our PHP script (which doesn't exist yet btw, this is why I'm asking Qs)
https://github.com/dagger/dagger/pull/7493/files#diff-6798aa21136fec549e40af53bd517bd3343f6b5bc54536ffc5ad0cd298124349R23

[cobalt pelican]

Yeah that looks ok.

[green otter]

So that PHP script is what will
a) get the fnCall, fnArgs ..etc from Dagger API
b) run introspection
c) run the actual code
d) supress the output from STDOUT, and catch it in a buffer
e) JSON it up
f) return the JSON back to Dagger Engine

Sound about right?

you said the STDOUT catching is out of place, but I don't see why, yet, since that dagger PHP entrypoint script IS the script that'll actually call the end code needed

[cobalt pelican]

Yeah but the php entrypoint should be able to find the function wherever it is (Python uses a decorator to store the callable in a map), then just execute it and get the return. No need to look into stdout from the function.

It’s not that the php entrypoint reads a source file and parses the code or anything.

They run as if from the same script.

Just need to import the user code as if a library.

If it’s still not clear can you give an example to clarify how you see stdout needing to play a part? It’s not clear to me why you think that. 🙂

[green otter]

@cobalt pelican yea I get you, it's just my articulation and your undertanding of PHP stuff 🙂

I'm not reading source code, I'm just invoking a class->function() call 🙂

PHP reflection will return something in which we can call (like the typescript one which uses Reflect)

Once we call it, that code will try to output its result to STDOUT, but we're not gonna let it, we're gonna capture it and then return that back to dagger, in an API call

it's kinda like in bash

OUTPUT=$(some function call)
JSON=json_encode($OUTPUT)
sendToDagger($JSON)

[cobalt pelican]

Why is that code outputting to stdout if your function uses a return for the result, can you clarify that part?

[green otter]

The code isn't our code, it's gonna be a third party library that does something, like Linting, Code Sniffer checks (like PEP8)

Those existing libraries, we will be pulling in, to then call them, to do the work

[cobalt pelican]

But it’s the users responsibility to return a value and in the case of those libraries to buffer the output and return as the result. Not the sdk’s responsibility.

[green otter]

Okay

We will split the responsibility then

Cool ?

[cobalt pelican]

What do you mean? 🙂

[green otter]

This

list($parentName, $moduleName, $functionName, $functionArgs) = getStuffFromDagger()

// Module registration
if($parentName == '') {
  $json = json_encode(getModuleId());

// Module invocation
} else {
  list($classToCall, $functionToCall, $argsToCall) = $introspectionThingy->introspect($moduleNane, $functionName, $functionArgs)

  $result = $$classToCall->$functionToCall(...$argsToCall)
  $json = json_encode($result)
}

$dagger->setReturnValue($json)

updated ☝️

[cobalt pelican]

Yeah, just need to branch off on parentName there. For registration vs invocation.

[green otter]

updating

how about now?

[cobalt pelican]

Yeah that’s it on a high level 🙂

[green otter]

@cobalt pelican good. passing this over to @summer sequoia now. 🙂

[cobalt pelican]

Just one clarifying note. The introspection part is more necessary during module registration. During that phase, you'll create a $mod = $dag->module() and continue adding withObject(name)->withFunction(name)... in the end you'll get the id by $mod->id(). Actually introspection should run before anything else because you'll need that info both to register the functions, but also to know how to get them in order to call.

[green otter]

Okay, I was wondering how do do this

This is reflection in PHP
https://rakibdevs.medium.com/exploring-the-power-of-reflectionclass-in-php-a26d5ce533f3
https://www.php.net/manual/en/class.reflectionclass.php

Things like this:

$foo = $reflectionClass->newInstanceArgs($dependencies);
$foo->bar->hello(); // Output: Hello from Bar!

[cobalt pelican]

In Python, there's no explicit introspection step actually. Simply by importing the user's code means the decorators in that code will run. The decorators then just register the decorated callable in a registry object. By the time you want to register, I just need to look into the registry to get the list of objects. Keyed by their names. The same to invoke. During registration I introspect the callable's types. During invocation I just call the callable.

[green otter]

Okay, understood.

We will do the same in PHP, we will use Annotations (our decorators) 🙂 to scan all this, at Registration time, and then it can be called at call-time

Deal?

[cobalt pelican]

The annotations should be run automatically just by importing, right? No need for an explicit "scan"?

Ok, so reflection already is loaded from a string. So if the user's code is in a certain namespace, then you just need to look for a list of class names in that namespace.

Which the Annotation can help with.

[green otter]

Something like that, yea

PHP has a global "autoloader" (Composer PSR-4) so all classes are in memory at runtime boot time 😉

We don't need to scan or load anything tbh.

PHP is more efficient than JS or Python scanning, in this manner

but maybe some stuff needs "figured out" - I'm unsure still atm

[cobalt pelican]

Yeah, same with python. Typescript is different because according to the maintainer of that SDK, TypeScript doesn't have introspection at runtime, you have to run a separate process to get that info. So Python is the closest to PHP here.

[green otter]

$mod = $dag->module()

Is this a dagger API call? to get the unique module ID?

or do we generate it by-hand in PHP?

I'm wondering how we generate or get $mod->id 🤔

[cobalt pelican]

Go is also different because it's doing the introspection at codegen time. That means by the time the runtime executes the entrypoint, there's no introspection necessary. The code for making the registration and invocation have already been generated (not dynamic). But that also means that any change in functions' signatures can invalidate the codegen step.

[cobalt pelican]

You need to the dagger's API for that.

[green otter]

Cool, that's easy. We can make an API call.

[cobalt pelican]

It's lazy, so you'll basically use the PHP dagger client to build a query, and that will only make the API request at the end, on $mod->id().

[green otter]

To get the ID ..

We only need to make $dag->module() upon Module Registration, right?

Or are we making it upon the Module Execution (function call), too?

[cobalt pelican]

https://docs.dagger.io/api/reference/#query-module
https://docs.dagger.io/api/reference/#Module-withObject
https://docs.dagger.io/api/reference/#ObjectTypeDef-functions

[cobalt pelican]

Just registration.

[green otter]

awesome! 😄

Understood

[cobalt pelican]

That Module data structure that you'll send to the API is what the engine will use to expand the API schema. You're basically defining new GraphQL types, fields, and their resolvers.

[green otter]

Can you link me to the internal python/js code for $dag->module(), please? 🙂

[cobalt pelican]

So that's what happens on load. The engine gets all the files from the module, loads to corresponding runtime module, asks that runtime to give all the type definitions, expands the API schema, and creates a resolver for the new fields in that extended schema to execute the runtime with passed in input, and return back the result.

[cobalt pelican]

Yes, one moment.

This may help understand the setup here a bit more: Python has a Module class (not the same from the API), and the decorators are actually methods from that class -> https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/__init__.py#L7-L11.

So dag.module() is done in the constructor: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L79

Registration vs invocation is decided here: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L193-L197

On registration, all objects registered during import time (via the decorators) are looped and added to self.mod: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L239-L247

Actually added here: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L256

[green otter]

/cc @summer sequoia

[cobalt pelican]

But you need to do a few $dag->typeDef() (https://docs.dagger.io/api/reference/#query-typeDef) to build type definitions.

The functions are registered here: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_resolver.py#L124-L147

That returns a TypeDef, that can be used with $mod = $dag->module()->with_object(typedef)

For a simple function that only returns a string (no arguments):

$mod = $dag->module();
$stringType = $dag->typeDef()->withKind(Dagger\TypeDefKind::STRING_KIND);
$func = $dag->function("myFunction", $stringType);
$obj = $dag->typeDef().withObject("MyModule")->withFunction($func);
$mod = $mod->withObject($obj);
// $mod->id()

[green otter]

Why id $mod->id() here? 🙂 assuing this is module invocation, not module registration

[cobalt pelican]

That code is only for registration. During invocation you don't call these apis.

[green otter]

ok got it. so that's module registration .. ☝️

[cobalt pelican]

No, it's registration.

[green otter]

typo 🙂

[cobalt pelican]

On invocation you need:

$call = $dag->currentFunctionCall();
$parentName = $call->parentName();       // should already have this... it's the name of the class
$name = $call->name();                   // name of the method in parent object
$parent = json_decode($call->parent());  // map of properties and values for the parent object instance
$inputArgs = $call->inputArgs();         // list of FunctionArg types for each of the function arguments
$inputs = []
foreach($inputArgs as $arg) {
   $inputs[$arg->name()] = json_decode($arg->value());
}

Then you deserialize the json on the parent and the input arguments, instantiate the object, and call the method with those inputs. In the end just:

$call->returnValue($result);

Example on the first part: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L263-L272
Result sent here: https://github.com/dagger/dagger/blob/15d88222ef97b4a3b152e3d17916cafdf29dfbb1/sdk/python/src/dagger/mod/_module.py#L224

[green otter]

🔥 this is awesome 🙂

foreach($inputArgs as $arg)

You're nearly a PHP software engineer @cobalt pelican 😛

[cobalt pelican]

Haven't used it since the early days of 8.0 😄

[summer sequoia]

I have code I wrote recently that loads up classes with a given annotation on them

That'll be a start point

Then reflect to get the function definitions

[green otter]

@summer sequoia ping me for a sync when you're happy with it, as a PoC

[summer sequoia]

https://github.com/atto-php/hydrator/blob/main/src/HydratorProvider.php#L46

I'm about to sleep :p

[green otter]

Not today, I mean in the near future 🙂

thanks for your time tonight @cobalt pelican ! speak soon

[green otter]

Notes from call with @inner walrus and @summer sequoia

• issue with sdkSourceDir var being nil each time - https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/runtime/main.go#L25-L36
• consider adding 'php' to this switch - https://github.com/carnage/dagger/blob/add-php-runtime/core/schema/sdk.go#L86-L95

[green otter]

@cobalt pelican its done - Built a dagger module, with a function with no args 😎

$mod = $dag->module();
$stringType = $dag->typeDef()->withKind(Dagger\TypeDefKind::STRING_KIND);
$func = $dag->function("myFunction", $stringType);
$obj = $dag->typeDef().withObject("MyModule")->withFunction($func);
$mod = $mod->withObject($obj);
// $mod->id()

I built this. basically 😄

@summer sequoia 😎

[summer sequoia]

All pushed?

[green otter]

ALl pushed Chris, to my branch 🙂

We have a Dagger Module ID 😎

[summer sequoia]

Ah hard coded one.

let me see if I can add some stuff from reflection now I'm back at my laptop

[inner walrus]

I sat in on the PHP SDK call with @green otter and @summer sequoia. Paul is going to use it in a workshop https://websummercamp.com/workshop/building-cutting-edge-ci-cd-pipelines-using-dagger-and-php at a conference in a few weeks so if we can help them get it over the line, it would be amazing. @shy goblet and @cobalt pelican

cc @tropic whale

https://tenor.com/view/charlie-always-sunny-in-philadelphia-gif-8902231

This is what the session was like. In a really good/fun way.

[summer sequoia]

Got a bit futher pulling out info using reflection

stuck with this error now: Error: query module objects: returned error 400 Bad Request: failed to get schema for module "test3": failed to create function "echo": failed to find mod type for function "echo" return type: "Test3DaggerContainer!"

now, my question is; how do I correctly tell dagger that the return type/arg type is a Container? eg a built in object.

[inner walrus]

Does this help?
https://github.com/dagger/dagger/blob/57e004e212c50fadd203e8432d858312c9fe588e/sdk/typescript/introspector/test/testdata/state/expected.json

Seems like

"returnType": {
            "kind": "OBJECT_KIND",
            "name": "Container"
          }
        },

[summer sequoia]

maybe, let me try

I /think/ that worked

looks like it's got onto trying to call the module now

error is: Error: main object not found

[inner walrus]

https://github.com/dagger/dagger/blob/main/cmd/dagger/functions.go#L425

https://github.com/dagger/dagger/blob/main/cmd/dagger/module.go#L868

[summer sequoia]

hmmm, looks like it expects a dagger object to exist which matches the dagger module name.

That will take a bit of thinking about

as currently the class exists inside a namespace

I'm going to call it a night and see what can be done in the morning; tonnes of refactoring to do on that entrypoint code anyway

[inner walrus]

Thanks @summer sequoia ! Awesome stuff.

Indeed that's the way.

@object_type
class Nostalgia:
    """
    Nostalgia functions
    """

{
  "name": "nostalgia",
  "sdk": "python",
  "source": "dagger",
  "engineVersion": "v0.11.6"
}

[summer sequoia]

yeah, it's a touch more complex in php as generally every class exists in a namespace and the class must be in a file matching it's name

[inner walrus]

no idea if correct

Since each module will be sandboxed in its runtime and not loaded into other PHP programs, might be able to use a global namepace (?) and either ignore or adhere to filename<>class name match

[summer sequoia]

It's the autoloading that enforces those constraints.

Just needs a smart solution.

I have a few ideas

One option is just a convention that every dagger module has the same hard coded namespace; then it just gets filtered from the class name

[shy goblet]

@summer sequoia You could also look for another pattern to implement the modules support.
Go isn't following the class system but instead uses functions exposed on a struct.

I'm not sure there's such concept in PhP but you can adapt the implemention based on the feature of the language.
To implement the module support, you basically need a data structure that:

• can store variables
• expose functions
• can be defined as public/private

For most language, this is the class concept but maybe PhP has other ways to achieve that? 😮

[green otter]

Good to meet you @shy goblet 🙂 thanks for the help. Let's plan a call for next week sometime, to keep moving forward ?

---

@summer sequoia do you have all my code from last night ?

I wanted to start simple with StringKind and then add args to that dagger function (with their types) and get that working.

[summer sequoia]

Yes I took it and extended it to add proper types; that all works

Current issue is that it expects the main class to be the same as the module name so that needs a bit of refinement

[shy goblet]

Sure! Let me know what time

[green otter]

So does the grepDir() function work ? @summer sequoia

What about that Directory class argument.

[green otter]

How about next Tuesday at 5pm UK time?

[shy goblet]

Sounds good to me, I send you my email in private

[summer sequoia]

yes it does. See the convo a bit further up where we were working out how to do that bit

[green otter]

@summer sequoia hope you can adjust your schedule to suit, and join us

Wasn't sure which part yous were talking about.

Thought you were talking about return types, not about Argument types

[summer sequoia]

I'm unlikely to be able to do a Tuesday

Yeah, but once return types were working, arg types were trivial

[green otter]

Did you get the echo() function working? "Container" class return type

@summer sequoia

We need like a consumer module repository, which can call this "Dagger Example.php" we are building

To test it end to end.

Can you add me to that repo, pls, chris?

[shy goblet]

Could you add me to the repo too please? My GH is TomChv https://github.com/TomChv

[summer sequoia]

Call a module function and print the result.

If the last argument is either a Container, Directory, or File, the pipeline
will be evaluated (the result of calling `sync`) without presenting any output.
Providing the `--output` option (shorthand: `-o`) is equivalent to calling
`export` instead. To print a property of these core objects, continue chaining
by appending it to the end of the command (for example, `stdout`, `entries`, or
`contents`).

Usage:
  dagger call [flags] [FUNCTION]...
  dagger call [flags] [command]

Examples:
dagger call test
dagger call build -o ./bin/myapp
dagger call lint stdout

Flags:
      --json            Present result as JSON
  -m, --mod string      Path to dagger.json config file for the module or a directory containing that file. Either local path (e.g. "/path/to/some/dir") or a github repo (e.g. "github.com/dagger/dagger/path/to/some/subdir")
  -o, --output string   Path in the host to save the result to

Function Commands:
  echo          
  paul          

Global Flags:
  -d, --debug             show debug logs and full verbosity
      --progress string   progress output format (auto, plain, tty) (default "auto")
  -s, --silent            disable terminal UI and progress output
  -v, --verbose count     increase verbosity (use -vv or -vvv for more)

Use "dagger call [command] --help" for more information about a command.

[summer sequoia]

I added you last night, same fork we were using on the call

[summer sequoia]

invite sent

[green otter]

That's a fork of the dagger repo itself, not the consuming downstream library that does

$module->grepDir()

Like we are a real downstream person just calling these DaggerFunctions from our Laravel/Symfony project

[summer sequoia]

Oh I don't have a repo for that yet

and I'm not sure that's how you'd use it in the wild

[green otter]

Yea, cool, that's like next step I'm saying ..

We won't know everything and Tom can help us there, too 😀 and even work in parallel ( you and me )

Producer + Consumer

[summer sequoia]

I think the next step is going to be some hefty refactoring breakfast

[green otter]

😎👌

[summer sequoia]

doing help on echo seems to work as well; recognises the return type as a container and offers you up the commands you can call on a container

notices that it's got an argument and offers that as a flag

Usage:
  dagger call echo [flags]
  dagger call echo [flags] [command]

Flags:
      --value string

Function Commands:
  as-service                    Turn the container into a Service.

My current understanding is if you were doing a dagger pipeline for a symfony project

You'd do dagger init mybuild

getting yourself a local module

then inside that module you'd add dependencies to other modules eg phpunit & phpstan

then you'd call them from within your module

[shy goblet]

Yes but all this part is handled by Dagger, the only thing you need is a generator that use the GQL schema to generate the corresponding code

I think you already have a php generator so it shouldn't be too hard, you just have to call it from your module runtime then

[green otter]

No no, as an end user of Dagger, you don't need your own module Chris.

You just "call one" like in github actions we have the "checkout" module or the "aws-eks" module/action there.

[summer sequoia]

yeah, so when I say call; I mean doing $daggerClient->call('PHPUnit', $directory, $configFile);

well, you could also do it from a shell script doing dagger call phpunit --config=phpunit.xml etc

[green otter]

We are aligned.

[summer sequoia]

I'd personally do it from a private php module as that keeps everything in php

[green otter]

@shy goblet got a URL to some JS/python code that is calling a Dagger Function ? Maybe something like Jest, or PyTest or JUnit

This is at PROJECT level

[summer sequoia]

I'm not sure how youd do, on the command line a script to build your docker container, then pass the code dir to php stan and phpunit modules; but I could do that with ease in php

[shy goblet]

Every examples in our docs is calling Dagger function, I'm not sure if I understood correctly

[green otter]

Okay. Well I was reading lots before Zenith release so thought it's changing a bit

Pre-made DF for something shiny. Like AWS ECS or just a CI test run.

[shy goblet]

I've looked at my steps for the support of module in the TS SDK, here's a summary of what must be done:

• [ ] Create a module Runtime for PHP (you've done it there: https://github.com/dagger/dagger/pull/7493) so it's nice

  • The runtime should:
    • Have a Codegen method that will add the generated code to the user source
    • Have a Runtime func that will call the entrypoint to your module's code.

• [ ] Create an entrypoint (example in TS: https://github.com/dagger/dagger/blob/main/sdk/typescript/entrypoint/entrypoint.ts)

The entrypoint has 2 purposes:

1. Parse the user's source code and extends the Dagger API schema, this happens when the function call is empty, it's the first thing executed when calling a module (https://github.com/dagger/dagger/blob/main/sdk/typescript/entrypoint/register.ts)
2. Execute the actual user's function (https://github.com/dagger/dagger/blob/main/sdk/typescript/entrypoint/invoke.ts)

All the context around the execution (object called, function, arguments, current state) can be retrieved with API call
https://github.com/dagger/dagger/blob/57e004e212c50fadd203e8432d858312c9fe588e/sdk/typescript/entrypoint/entrypoint.ts#L28-L30
https://github.com/dagger/dagger/blob/57e004e212c50fadd203e8432d858312c9fe588e/sdk/typescript/entrypoint/entrypoint.ts#L48-L50

As a first step, I would try to have a piece of code that can take some PHP sources files, parse them and returns a data structure that you can use to extends the schema for Dagger.
This part does not require any runtime, it's super simple to test and will help you finding the right structure for a php module.
After that, you can start to link it to the runtime and work on the execution

Can one of you tell me what's the current state of the PHP SDK? Do you have any introspector or something to parse PHP?
If not, I think you should prioritise it

You can use the TS one's as reference: https://github.com/dagger/dagger/tree/main/sdk/typescript/introspector

[summer sequoia]

That's what we built last night

the next step is refactoring that

and then writing the code to execute the function based on the parameters from dagger

[shy goblet]

Okay, so the registration works right?

[summer sequoia]

yes

[shy goblet]

Do you have an example of a PHP module?

[summer sequoia]

if you do dagger init --sdk=github.com/carnage/dagger/sdk/php/runtime@add-php-runtime test you'll get one

[shy goblet]

Ok I'm trying rn haha

The init works, dagger functions isn't tho

In ParserAbstract.php line 359:
                                                          
  Syntax error, unexpected '-', expecting '{' on line 14  

It should even if the execution isn't done yet

[summer sequoia]

bexio@imhotek-PCx0Dx:~/git/dag$ dagger functions -m test5
Name   Description
echo   -
paul   -

what did you call your module?

[shy goblet]

test

Oh you mean the name? I just used the command you sent above

Oh okay, I forget the test it seems that if you omit it, the generation works but not dagger functions, might be the same for other sdk tho

[summer sequoia]

what did it generate in src?

[shy goblet]

A Test-PHP module

[summer sequoia]

ah

as i guessed - we need to do some cleanup of modulename => class name conversion

[shy goblet]

Repro:

dagger init --sdk=github.com/carnage/dagger/sdk/php/runtime@add-php-runtime     
13:14:46 WRN no LICENSE file found; generating one for you, feel free to change or remove license=Apache-2.0
Initialized module test-php in .

➜  test-php dagger functions
✘ initialize 2.1s
! input: module.withSource.initialize resolve: failed to initialize module: failed to call module "test-php" to get functions: call constructor: process "/src/dagger/dagger" did not complete successfully: exit code: 1

I tried with the TS sdk, it works fine

[summer sequoia]

is test-ts a valid function name in ts though?

you didn't provide a dir, so I'm guessing it defaults to the current dir which is test-php

test-php isn't a valid classname in php so it chokes after generating that clas

[shy goblet]

Ohhh I see

[green otter]

Sorry. My internet is slow. Didn't see you replying Chris

[serene light]

Managed to call an echo function without arguments from a module https://github.com/carnage/dagger/tree/charjr-add-php-runtime

next is with arguments

[shy goblet]

For some specific data structure like dagger types, you’ll need to load them from their ID.
You should have a bunch of function like LoadContainterFromID, you’ll need to do that for any object type

[serene light]

I'm performing a grepDir function, using the phpsdk and getting this response:
Error: response from query: input: potato.grepDir resolve: failed to unmarshal result: invalid character '\n' in string literal
I'm not sure where to go with that one

(I am passing in basic arguments now using this branch) https://github.com/carnage/dagger/tree/charjr-add-php-runtime

[green otter]

@summer sequoia @serene light I've read all of the code on that branch, and I'm up to date with our latest implementation of things 🙂 nicely done guys!

[summer sequoia]

It's so close to basic functionality

[inner walrus]

💪

[green otter]

I didn't quite understand all the json encode/decode parts .. but in time I'm sure I will

[summer sequoia]

Tbh, I'm not sure why they were needed exactly; but that was the format the data was in so 🤷

Ideally the generated code would have taken care of that for us

[serene light]

Yeah we seem to be getting the data in as json, so it required decoding before we could convert it to the appropriate type

If it's possible to decode it beforehand we can fix it at a later date, but from inside the entrypoint, that's what we needed to do to get it working

[green otter]

I am giving a 4 hr workshop at a PHP conference in 3 weeks.. so the goal is just to "make it work" and later we can refactor, because the end users don't care how it is implemented.

@serene light @summer sequoia

[summer sequoia]

yeah, that was my thinking; make it work, make it good

[shy goblet]

The output for the Dagger API is standardized as following:

• If it's a custom object (from your inside module): you return the fields of this data structure in JSON
• If it's an external object or native (like Container), you call its id method and return the ID
• If it's a scalar value (string, int, bool, scalar, enum..) you return the raw value
• If it's void, you return null

Not that a custom object may have all of the types above inside it so you need to recursively format each field/subfields

[serene light]

Any idea why this query should be failing?

[INFO] query {
currentFunctionCall {
returnValue(value: "ChV4e...1ODUyMmMwMjE=")
}
}

Error: input: module.withSource.initialize resolve: failed to initialize module: failed to call module "potato" to get functions: failed to unmarshal result: invalid character 'C' looking for beginning of value

(I've squished the returnValue)

[shy goblet]

did you resolved the ID of an object?

and ID is a scalar, not a string, I'm not sure you can do it that way, I usually don't use the raw GQL client

What are you trying to do exactly

[serene light]

Currently the FunctionCall class has a returnValue method that MUST take a Dagger\Json object (which in turn, MUST take a string value)

So to get the module, using the generated FunctionCall class, we have to cast ID to a string

does this seem correct?

[cobalt pelican]

Any idea why this query should be

[green otter]

@serene light @summer sequoia what is the latest branch that we should be working from?

John's or Chris' ?

[summer sequoia]

John's is most up to date.

[green otter]

I'm on a call with @shy goblet and I'm asking him how to fix our "feedback loop" issue, where dagger isn't passing us the SourceDir, and thus we're having to push every time

[summer sequoia]

I think the only thing that's blocking an MVP of this is a bug in the graph ql library we're using

[green otter]

Articulate it to us? Write a wee ticket on the issue, people can help out.

[summer sequoia]

Well, we need to encode all results as json

If the result is a string and contains a " it gets JSON encoded as \"

The client then encodes the " as " resulting in the value \\" going into the query

[green otter]

I'm on a call with Helder and Tom now, Chris, so ask your questions now btw 🙂

[summer sequoia]

Which is incorrect

Can't join ATM

[green otter]

@summer sequoia 😐 oh dear, can this be fixed?

[summer sequoia]

It's in the graph ql library we're using - it should probably be doing addslasshes but that might not be correct in all cases

First time I've touched graph ql so I don't know what the rules are

[summer sequoia]

If you can fix the library; then the last thing to do imo is to throw proper exceptions on things we don't fully support yet like interfaces, custom objects and parent state/constructors to lock it down early so we don't just end up with confusing error messages

[radiant flume]

The rules of graphql itself are very simple, the spec is a lightweight document

If the result is a string and contains a " it gets JSON encoded as "

I can't think of any valid reason for doing this in the graphql spec. If the result is a graphql string, it should be encoded as a json string.

[summer sequoia]

Dagger wants the return value of any function to be a string containing a JSON encoded value; so you have a JSON string inside a graphql string

And the PHP graphql library doesn't handle escaping properly

[radiant flume]

Which interface are you talking about here? clients querying the engine over GraphQL; or the engine calling the php runtime over custom rpc/json files?

[cobalt pelican]

He´s talking about this: https://pkg.go.dev/dagger.io/dagger#FunctionCall.ReturnValue. It's part of modules support in an SDK. The PHP library for building GraphQL queries is making assumptions about strings, that it shouldn't be making.

[radiant flume]

Got it, thanks. So back to this 🙂 #php message

@cobalt pelican welcome back? 🙂

[cobalt pelican]

Can you link to the code in the library's repo that's handling this?

[cobalt pelican]

How would you render a JSON document where one of the values is itself an encoded JSON document? GraphQL expects strings to be represented in double quotes (""), so any quotes inside need to be escaped. If your library supports input variables, that would be the simplest way to handle this.

I think you're using this one, so it does support variables: https://github.com/mghoneimy/php-graphql-client?tab=readme-ov-file#query-with-variables

There's also a RawObject that allows inserting something into the query as is (without manipulation) which can be a simpler solution still.

[summer sequoia]

JSON requires " in strings to be escaped. Graph ql does as well; the library fails to handle " that have already been escaped

The required solution is to escape both " and \

[cobalt pelican]

Yeah, but you can skip that logic with RawObject.

For example, in https://github.com/carnage/dagger/blob/charjr-add-php-runtime/sdk/php/src/Client/QueryBuilder.php, check for a DaggerJson and wrap the value in new RawObject(...).

[summer sequoia]

If you do that; the resulting string won't decode as valid JSON

[summer sequoia]

https://github.com/mghoneimy/php-graphql-client/pull/87/files << someone fixed the bug already

[cobalt pelican]

I'm just saying you can control what gets rendered in the query for a field, without worrying about extra handling by the library.

[summer sequoia]

Yeah, i think it'd need a lot of changes in the codegen to o that fully though

How complete does this need to be to be mergable?

[cobalt pelican]

It's ok to make progressive improvements with multiple PRs.

[summer sequoia]

I'm thinking we just about have an MVP which can handle: running codegen and getting the module runtime going; handle module registration and invocation; allow for defining functions and objects in PHP. For modules, you can't define arrays, custom objects or interfaces yet and it doesn't support all built in dagger objects but the basic types are there: Container, Directory, File, string, int, bool.

Code is a MESS but it works

[cobalt pelican]

Yeah, MVP sounds good.

Curious about the arrays, isn't it easy to add support for that?

[summer sequoia]

relatively

PHP has no way to define array of X natively

so it needs additional code

My other libraries that do introspection type stuff use a SubType attribute to define the array item type

[serene light]

This branch: https://github.com/carnage/dagger/tree/charjr-add-php-runtime
Commit: 0f8c838

Input: dagger call -m potato grepDir --directory=./ --pattern=".json"

Output:

./src/Codegen/SchemaGenerator.php:        return json_encode($this->schemaArray, JSON_PRETTY_PRINT);
./src/Command/SchemaGeneratorCommand.php:        file_put_contents('schema.json', $generator->getJson());
./src/Command/EntrypointCommand.php:        $parentName = json_decode($currentFunctionCall->parent()->getValue(), true);
./src/Command/EntrypointCommand.php:            $currentFunctionCall->returnValue(new DaggerJson(json_encode((string) $result)));
./src/Command/CodegenCommand.php:            'Path to the schema json file',
./src/Command/CodegenCommand.php:            $schemaArray = json_decode($fileContents, true);
./src/Connection/ProcessSessionConnection.php:                    // @TODO Rewrite when PHP 8.3 json_validate is available
./src/Connection/ProcessSessionConnection.php:                        json_decode(trim($line));
./src/Connection/ProcessSessionConnection.php:                        return JSON_ERROR_NONE === json_last_error();
./src/Connection/ProcessSessionConnection.php:                    $sessionInformation = json_decode(array_shift($validLines));
./generated/ModuleSource.php:     * The path relative to context of the root of the module source, which contains dagger.json. It also contains the module implementation source code, but that may or may not being a subdir of this root.
./init-template.sh:if ! [ -f composer.json ]; then

TLDR; It's working for echo and grepDir without errors

downside: it's depending on a fork of the php-graphql-client https://github.com/mghoneimy/php-graphql-client/pull/87/files mentioned by @summer sequoia

• The upstream for that fork seems to have been abandoned for 3 years.
• While depending on the fork
  • Minimum stability in composer.json had to be dropped to "dev"
  • We had to add openssh and curl to the alpine container
• @green otter we may have to consider making our own fork to maintain

[green otter]

@serene light just fork and maintain it. As we don't have time to fix the official repo.

We have 3 weeks until workshop time.

We will fix it properly, later.

Just add a custom composer package on packagist and load it up.

Also.look at what helder said about RawQuery

[summer sequoia]

It's working now using the fork of the person who made the PR to fix the bug; the suggestion is that "we" should make a proper fork of that library to maintain alongside the PHP SDK

[shy goblet]

Hey guys, just to notice you!

I'll implement the bundling by tomorrow so you can improve the feedback loops.

[green otter]

So does that mean we don't have any errors now? And you can call grepDir() from a downstream PHP project?

[summer sequoia]

yes

[cobalt pelican]

Have you started on that yet? I'm wondering about another structure. Basically putting dagger.json in sdk/php with "source": "./runtime". This way you can access the SDK's files from runtime/main.go with dag.CurrentModule().Source().WithoutDirectory("runtime"). And "exclude" patterns in dagger.json would suffice for the rest.

[shy goblet]

How, that's smart

I was about to start, I'm pushing a final commit on enum and hopefully CI will be green

But with your idea, they can keep their current runtime, just calling dag.currentModule from the New

[cobalt pelican]

Yes, and the path in --sdk would point to sdk/php rather than sdk/php/runtime which I think it's better anyway (more sense for end users, not just shorter).

[shy goblet]

Yep, it's a great idea!
So do I need to update anything? They just have to move the dagger.json & update New, sounds pretty straight forward since they already made the runtime

[cobalt pelican]

Can you give a helping hand there, and see if it works alright?

[shy goblet]

Alright! Finishing my commit and I'm on it

[shy goblet]

Since like I hit some issues trying to making this work.
1 - The php runtime dir must be in a children dir to work, this is not that annoying, just a limitation
2 - Some paths are different with this, I'm hitting the following issue:

Stdout:
marshal: json: error calling MarshalJSON for type *dagger.GeneratedCode: input: container.from.withMountedDirectory.withoutEntrypoint.withWorkdir.withExec.withExec.withExec.withExec.withNewFile.withExec.file resolve: process "./install-composer.sh" did not complete successfully: exit code: 2

Stderr:
[dumb-init] ./install-composer.sh: No such file or directory

[shy goblet]

By setting the source to ./runtime, we do not have access to the php sdk dir, I'm trying to use ../ to refer to the parent dir but it's not working
That would be cool if we could access the parent (but that's what contextual module are made for if I understood correctly)

EDIT: I may have found a workaround, still trying

[shy goblet]

Ok it works:

dagger init --sdk=dagger/sdk/php --source=test
19:53:14 WRN no LICENSE file found; generating one for you, feel free to change or remove license=Apache-2.0
Initialized module dagger in .

However, it must be done from a parent dir so it can include dagger/sdk/php

If that's good for you @cobalt pelican, I can push the changes

The idea is basically:

diff --git a/sdk/php/runtime/main.go b/sdk/php/runtime/main.go
index 352fd6953..063ab6b4a 100644
--- a/sdk/php/runtime/main.go
+++ b/sdk/php/runtime/main.go
@@ -28,10 +28,11 @@ func New(
        sdkSourceDir *Directory,
 ) *PhpSdk {
        if sdkSourceDir == nil {
-               sdkSourceDir = dag.Git("https://github.com/carnage/dagger.git").
-                       Branch("charjr-add-php-runtime").
-                       Tree().
-                       Directory("sdk/php")
+               // Go back to ../ to get the root of the PHP SDK
+               // This is a bit of a hack, you need to call it from a parent directory but it works.
+               // Example from ../../ (the directory above the dagger git repo)
+               // dagger init --sdk=dagger/sdk/php --source=test
+               sdkSourceDir = dag.CurrentModule().Source().Directory("../").WithoutDirectory("runtime")
        }

And this inside sdk/php instead of sdk/php/runtime

{
  "name": "php-sdk",
  "sdk": "go",
  "include": [
    "."
  ],
  "source": "runtime",
  "engineVersion": "v0.11.6"
}

[green otter]

Seems juicy I can't wait to try it out

@summer sequoia can you link me to this downstream project ?

[green otter]

@summer sequoia can you link me to it pls? As well as add me to the repo so I can play about

[summer sequoia]

https://github.com/mghoneimy/php-graphql-client

[summer sequoia]

same repo as before

[green otter]

@summer sequoia misunderstanding.

Where is the repo that calls this dagger function grepDir()

[summer sequoia]

That's an example function created by the template

[green otter]

Where in that repo do we have .php code that calls dagger functions?

I.e: mount in the application code. And then run a command on it.

[summer sequoia]

That's what the runtime does

[green otter]

I don't think we are aligned

I had a call with Chris. We are now aligned.

The webapp repo doesn't exist.

[green otter]

@cobalt pelican @shy goblet

Do you have an example of a web app project that will invoke an existing dagger module?

I want to see the end users experience of using dagger to build their web webapp's CI pipeline

[shy goblet]

@zealous venture did some cookbook with that if I remember well

You can also check the guides in archived doc, you should find some!

[zealous venture]

The previous example has been updated to use functions: https://docs.dagger.io/integrations/php

[green otter]

@zealous venture good.

@zealous venture @shy goblet @cobalt pelican I'm having a 12:00pm (UK time) sync with @summer sequoia .. you're welcome to join us if you're around and interested.

Thanks

[cobalt pelican]

Where?

[green otter]

https://meet.google.com/pbp-qpzu-dae

[shy goblet]

@cobalt pelican Did you see my proposal on the php sdk local runtime?

[cobalt pelican]

Thought you already submitted it 🙂

Go ahead, if it works.

[green otter]

documing this issue

[shy goblet]

No I was waiting for your approval

[green otter]

@todo - update the class name generator to convert from X to "PSR-12" StudlyCaps (camelcase) to get rid of hyphens

[shy goblet]

@green otter https://github.com/carnage/dagger/pull/2 😄

[green otter]

looking

[shy goblet]

I thought I could directly pushed, but I actually couldn't so I pushed to my fork and opened a PR

[green otter]

@todo - Chris' getting some kind of error on the first pass, and then it works on the second. Show Helder an example, later, to replicate and understand it.

@shy goblet we just made some fixes .. we pushed to add-php-runtime not charjr-add-php-runtime 😄

So can you re-point your PR to not John's branch?

add-php-runtime is now the latest branch

[shy goblet]

Alright

Done

I let you review/approve and merge

[cobalt pelican]

Fyi, https://github.com/dagger/dagger/issues/6749

[cobalt pelican]

Can you share the link to that here?

[summer sequoia]

Why does it need reverting before merge?

[shy goblet]

here @cobalt pelican

[shy goblet]

I meant, before merging to Dagger

Because it's using local source for the runtime, but when your SDK is ready, you want to point to your branch, as you did before

[cobalt pelican]

No need to revert. That would be for the bundling solution.

[shy goblet]

Because I removed the install from git in favor of the local

[cobalt pelican]

Yes, that's the proper fix. Will be improved by https://github.com/dagger/dagger/issues/7647.

[shy goblet]

Nice

[summer sequoia]

Yeah, moving the dagger.json file means it should be able to find that dir from the git URL given for SDK

So seems mergable as is

[shy goblet]

I see

[summer sequoia]

Unless I'm missing something

Could suggest the elixir team do the same as I copied the git approach from them

[cobalt pelican]

Yeah, he's aware of the suggestion, but see how the implementation works for you @ocean creek.

@shy goblet, have you tried init outside the dagger repo, with the git remote? You can try the URL from your PR.

[shy goblet]

I only tried from local, which was the main purpose but I should

I'm on something rn though, I'll try later

[ocean creek]

For Elixir, will open a PR in 1-2 hours ✌️

[shy goblet]

@cobalt pelican It works:

dagger init --sdk=github.com/TomChv/dagger/sdk/php@feat/local-php-runtime --name=test
16:43:53 WRN no LICENSE file found; generating one for you, feel free to change or remove license=Apache-2.0
Initialized module test in .
➜  test-php dagger functions
Name       Description
echo       Echo the value to standard output
grep-dir   Search a directory for lines matching a pattern

/cc @green otter

[green otter]

weee

[shy goblet]

I let you merge the PR whenever you want

So you can point it to your active branch

[green otter]

merged /cc @summer sequoia https://github.com/carnage/dagger/pull/2

[shy goblet]

So now it should be github.com/carnage/dagger/sdk/php@add-php-runtime

And you can also point to the local if you're developping, pretty good you get both!

[green otter]

testing

/cc @cobalt pelican @summer sequoia

this is that "composer" issue again, too, on github.com auth .. did you guys discuss a fix?

exit code = 1

[shy goblet]

Yeah, after rerunning it, it works

I don't know why it fails the first time

[green otter]

Chris and Helder were discussing it on our call earlier. I'm sure they'll figure it out 🙂

@summer sequoia @serene light where is this Example.php btw? which repo is it in? I can't find it.

[serene light]

@green otter It should be sdk/php/template/src/Example.php

[green otter]

ok

FFS! I'm on main branch 😅

I'm making my own external repo, to play with this.

[serene light]

@green otter We discussed a fix, I think for now it is to simply create a private package but I there was discussion on what fix needed to happen... however @summer sequoia probably understands what needs to be done better than I did

[green otter]

So .. this is the template structure

Question

• Is this a pre-created format that something else is expecting?

Yes/No:

1. Does dagger executable need to be in this directory root?
2. Does composer.json need to be in this directory root?
3. Do I need a src folder here with my dagger module inside of it? where is the src/Example.php stuff defined?

Thanks

3. I found PSR-4

    "psr-4": {
      "DaggerModule\\": "src/"
    }

So that's the path defined there

[serene light]

1. and 3. So the dagger executable should be adjacent to the src directory, as far as I'm aware, that's the main requirement. I don't know of any other requirement to the structure.
2. I'm not sure

I didn't make the template myself, so I can't say for sure. But I did refactor some logic which finds your DaggerObjects by searching for the directory containing a dagger executable AND a src directory https://github.com/carnage/dagger/blob/charjr-add-php-runtime/sdk/php/src/Service/FindsSrcDirectory.php

[cobalt pelican]

What's that dagger executable?

[green otter]

It's our entrypoint 🙂

https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/template/dagger

I dunno why it's in the downstream dagger module tho - this should be in our "SDK" since it handles all the codegen and module registration

Maybe I missed something

[green otter]

nevermind, this is out of date this file - I'm deleting it.

entrypoint gets bundled in already, this is just a dangling file

[radiant flume]

@green otter @serene light if I understand correctly, currently the PHP sdk creates a src directory, and expects module source code to go there. Is that right? If so, is it possible to use . instead? The reason is that Dagger already has a concept of module source directory, which is the equivalent of src already (kind of like a dagger_src/ directory). So using a src adds an extra layer of directories. In Python there is a src directory because (I hear) it's a Python convention, tooling won't work if it doesn't exist. But in Go there is no such convention, and a result the source code of Go modules is easier to navigate. If the PHP SDK could be more like the Go SDK in that regard, it would be a plus. But I'm not familar with PHP conventions.

[green otter]

src is optional (AFAICT) .. however for our class autoloading, we specify the NameSpace of the code, against a directory

see: https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/template/composer.json#L24****

and the namespace on our dagger module - https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/template/src/Example.php#L5

We don't put .php files in the repo / root, because it's messy as fuck, and instead we bundle it into a nice clean directory of its own .. away from the Makefiles, and other things.

Conventionally, we treat tools and code separately ..

Hope it makes sense?

@serene light @summer sequoia this isn't helpful 😛

dagger call grep-dir --directory=/mnt --pattern=a

I hope this is the correct syntax, against https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/template/src/Example.php#L29-L34

but if it's not then we need to be helpful to the user .. an internal graphQL thing blew up and doesn't help (me) the end user

Update:
Ok so I changed it from /mnt to . and it worked

from
dagger call grep-dir --directory=/mnt --pattern=a
to
dagger call grep-dir --directory=. --pattern=dagger 

So whatever its issue with /mnt was, it blew up and we don't know.

[radiant flume]

That makes sense, I don't mean the root of the repo, but the root of module source directory, which is configurable by the user. For example, in our repo github.com/dagger/dagger, there is a Dagger module, configured to have its source code in ./ci. So if I want to see the source code of the module, I open github.com/dagger/dagger/ci/main.go. If it were written in PHP, ideally I would open github.com/dagger/dagger/ci/index.php. Instead of github.com/dagger/dagger/ci/src/index.php (extra src which adds no value, because ci is the src dir)

[green otter]

Yes @radiant flume it works EXACTLY like that.. we could rename /src/ here to /ci/ and it'll be exactly like that... it's purely configurable

So PHP works like Go 🙂

[radiant flume]

Just to confirm, who is "we" here, the end user writing a module in PHP? Or the authors of the SDK?

[green otter]

the end user writing a module.

[radiant flume]

OK got it! So earlier when that dagger directory was created by dagger init, the solution would be to call dagger init --source=src in that particular repo?

[green otter]

wait .. actually we baked it into the SDK - it's not forced by the PHP conventions, we "the SDK authors" decided that.

https://github.com/carnage/dagger/blob/add-php-runtime/sdk/php/runtime/main.go#L14

 dag.GeneratedCode(ctr.Directory(ModSourceDirPath)).

Conventionally... EVERY PHP repository contains its source code in /src/ tho .. it's the defacto way to find a repo's code, separating it from configuration files and stuff .. so tbh we kind of wanna keep the Module's git repo lookling like

/src/DaggerModule.php

[radiant flume]

OK then it's like python

ie if we rewrite our CI to php, the main source file would be at github.com/dagger/dagger/ci/src/index.php