#Nested exec

1 messages · Page 1 of 1 (latest)

rigid vector
#

.

#

Things that would need to be namespaced:

  • Host fs access import + export (should be the container's fs)
  • Interacting with services (should only see services you start, not the parents)
  • Cache volumes (addressed by human friendly names)
  • Secrets?
  • OCI registry credentials if they are handled as a special case (?)
tight trout
#

Just to get the temperature on this one: It's not a hypothetical feature in a remote future, I need it right now to test the Python SDK 😄

So if we think it's not a good idea / needs design discussions I can find another path forward

#

I think maybe the safest would be @plush chasm's suggestion -- do it but hide it and disable by default

tight trout
plush chasm
#

Yeah if we make it only available in a development build, I'm good with it

rigid vector
#

Well if you need to implement something now, seems like a good idea to start the design discussion now; even if we make tradeoffs for implementation speed, I prefer to know about the tradeoffs 🙂

tight trout
#

Of course

plush chasm
#

All the namespacing stuff needs to happen both for extensions and for multitenant anyways, so once we solve it there then we could consider also enabling DAGGER_HOST access for exec ops in non-dev builds

tight trout
#

Just wanted to know if I should start looking for an alternative

rigid vector
#

So: allow nesting, but hide it, and worry about namespacing later?

#

How about we add: configurable by an exec option, disabled by default

#

That way no risk of someone somewhere accidentally exec-ing a tool that instrospects for dagger api access and does evil things (long shot right now, but things could change quickly)

tight trout
rigid vector
#

emphasis on accidentally. @tight trout can enable it for tests; he knows the risks and is careful what he execs

#

I think it's ok to make it public if it's disabled by default, and there's a scary warning in the docs

#

we can enable it by default later when it's safe

tight trout
#

(Just thinking that after designing it -- we might remove the exec option because it's always on, since it's so secure there are no downsides)

plush chasm
#

It's almost the equivalent of --privileged (not quite, but comparable) so it needs a scary name

rigid vector
plush chasm
#

Can we just put this behind a buildtag for now?

rigid vector
#

no opinion on that

tight trout
#

but let me check

plush chasm
#

Could there be a const like enableDaggerSock that's false by default and true if you're doing a dev build. Then our code can check that bool

tight trout
#

I'm going to try with the most hidden way I find, then we can make it more/less public? Changing how to enable it should be the easy part

tight trout
#

"10 tricks functional programmers don't want you to know"

plush chasm
#

I think it's probably extra paranoid to do it that way... but it just makes me nervous to give users that sort of option right now

#

even if it's tucked away

rigid vector
#

dagger.ExecYouWillGetHackedIfYouUseThisOpt

plush chasm
tight trout
#

dagger.ExecYouWillGetHackedIfYouUseThisOpt{
SignedBy: "..."
}

#

and we can check the signature against the user's github ssh public key

#

i'm sure they'll never forget

rigid vector
#

I think the following is OK:

  • Disabled by default
  • Option name conveys the risk AND experimental nature, and is long for extra dissuassive power: dagger.ExecExperimentalPrivilegedOpt
  • Clear and scary warning in the docs. "Do not use this option unless you trust the command being executed. The command being executed WILL BE GRANTED FULL ACCESS TO YOUR HOST FILESYSTEM"
  • Clear and scary warning in the go doc (same text) (this is a nice to have)
  • We all agree that in the future it's possible and desirable to remove privileged mode in favor of always-on, fully sandboxed nesting
  • If anything goes wrong, we never had this conversation
tight trout
#

I was midway reading and was about to suggest "if anything goes wrong, it's Solomon's fault"

rigid vector
#

lol

#

compromise edit

tight trout
#

ok i'm going to try

rigid vector
#

But the field description from gql schema makes it into the go doc comment no?

tight trout
#

Yep yep

rigid vector
#

oh, but it's not a field, it's a single argument

tight trout
#

yeah

#

tucked under exec options

rigid vector
#

ok - well I think the name + regular docs is enough 🙂

tight trout
#

ugh no hold on

#

if it's an exec flag (rather than an engine option), code must be aware of it

#

so it means all our tests: go, python, node -- have to use this Exec opt

#

it gets trickier when testing public facing bits (examples, docs) -- we don't want to put that flag in there

rigid vector
#

Yeah but we only need the flag when exec-ing dagger itselfcode that imports the dagger SDK no?

tight trout
#

Oh right! Yes

rigid vector
#

mmmm...

tight trout
#

it will work ONCE, not recursively

#

but we don't do it recursively anyway

#

so all good

#

well -- there's just the meta case of writing tests for this 😄 in which case it'll be 2 layers deep. But I can just not do tests

#

(mage test --> go test container --> the actual test)

rigid vector
#

If the flag stops working, we'll either 1) see our tests fail, or 2) get a complaint from malware developers asking for their money back

#

If a feature doesn't have a test, but is used by all the other tests, is that considered a test?

tight trout
#

tests are running 🙂

# 
#21 36.69 PASS
#21 36.69 ok    dagger.io/dagger/internal/querybuilder  0.006s
#

dagger in dagger!

#

tests are failing!

#

ugh 😄

plush chasm
tight trout
#

Yeah, WorkDir based tests are failing. I don't see a way around. FS stuff

#

And .Connect() tests are silently passing ... but they're not testing the same thing

rigid vector
#

Makes sense, we need to wire the host fs to container

plush chasm
#

I just went on a big thought tangent wondering if we could model extensions not as schemas loaded into our server but on the client codegen level: codegen would still model them as an api but internally would just be submitted as exec ops with this flag enabled. I don't why but that's intriguing. Stashing away for later

#

I guess we could just sync the whole host into the container 😅

rigid vector
plush chasm
#

If we load host workdir -> test container then when tests run they should have the same workdir... so yeah I can't think either where that would break

rigid vector
#

does the inner engine connect successfully to the buildkit container?

tight trout
tight trout
#

fakextensions. i like it, don't know why either 😄

tight trout
plush chasm
# tight trout fakextensions. i like it, don't know why either 😄
  • It moves complexity from the server-side to the codegen-side, but we could still present users the same facade of schemas, chainability, etc. as we wanted to before
  • Avoids the annoying stuff around having to load extensions and dynamically change schema (and multitenant-ing that)
  • Allows us to use gqlgen in our server instead of the abandoned graphql-go
  • Answers questions around how to make the automatic mounting of extension input directories not a huge hack (it's just the mount api of exec op)

Only downside I can think of is that you wouldn't be able to easily do alpine{build(...)} from raw graphql. Which is meaningful, but not necessarily devastating. Essentially, our core graphql api becomes our assembly language, extensions are just compiled to that assembly language rather than becoming new instructions in the assembly language...

#

I don't have time for this right now 😅 but seems like an idea worth considering more!

tight trout
#

How does this sound?

        Exec(dagger.ContainerExecOpts{
            Args: []string{"go", "test", "-v", "./..."},
            // go test requires running dagger in dagger
            ExperimentalPrivilegedNesting: true,
        }).

doc description is this:

    Provide dagger access to the executed command
    Do not use this option unless you trust the command being executed
    The command being executed WILL BE GRANTED FULL ACCESS TO YOUR HOST FILESYSTEM
rigid vector
#

IMO the “nesting” in the opt name is redundant

#

“privileged” evokes docker —privileged which is exactly the right analogy (nesting + danger)

tight trout
# rigid vector “privileged” evokes docker `—privileged` which is exactly the right analogy (nes...

I thought privilege was too much of a blanket flag. Do I get cgroup privileged access? Is it about AppArmor? I wanted to convey what the flag was doing and avoiding having a "catch all" privileged like docker

Also -- it's confusing with docker's privileged flag which does something completely different

So I thought the feature is "nesting", then "privileged" is a prefix (just like "experimental") -- took your advice of making it long and scary

WDYT?

rigid vector
#

yeah you’re right

#

also the longer the better

tight trout
#

@rigid vector re your question earlier today: how do I re-generate the API? now it's mage sdk:go:generate. And if you want to re-generate all SDKs, it's mage sdk:all:generate 🙂

#

same with everything sdk:all:test, sdk:python:lint, ...

#

new SDKs have to implement an interface and they get consistently supported by our CI pipelines. The joys of CI as code

rigid vector
#

that’s amazing

#

I love mage but every time I see a dagger pipeline run by mage I can’t help but feel weird. It makes sense intellectually to focus on the backend and embrace the wrapping, but it still feels weird

#

I don’t know what to do with that gut feeling yet

tight trout
#

well, we can always build our own CLI or go run -- mage works as a library too

rigid vector
#

oh I didn’t know that!

tight trout
#

their CLI is just a convenience to avoid having to go build or go run

rigid vector
#

that would actually solve one aspect of the problem: lack of a standard loading model in the go sdk

#

we could standardize on go run without sacrificing use of mage

#

which means in turn we can standardize CI integration guides, at least for each sdk

#

it would still feel weird to not have a unique standard CI integration shared by all SDKs… but baby steps 🙂

tight trout
#

As a user tonight, I was kinda happy to not have a standard thing shared by everyone ... being pure Go allowed me to do some neat stuff which wouldn't translate to other languages

#

it's empowering

rigid vector
#

Yeah I love that too

tight trout
# 
type SDK interface {
    Lint(context.Context) error
    Test(context.Context) error
    Generate(context.Context) error
}

var availableSDKs = []SDK{
    &Go{},
}
#

this is how we do our pipelines 🙂

rigid vector
#

No intention of taking that away (we probably couldn’t even if we tried, it’s too popular already 😀)

tight trout
#

helder just needs to write a Python type that implements the SDK interface

#

and everything is taken care of

rigid vector
#

I think a standard way is CLI-based and additive (and optional)

rigid vector
tight trout
#

our CI is entirely in Go

rigid vector
#

python type -> go type for doing python things?

#

I get it now - just had to re-read a few times 😅

tight trout
#

SDK interface, lint+test+generate -- every SDK in our repo has to implement it, doing whatever they want

#

but then it's hooked up into everything automatically

#

sdk:all:generate all include python generation out of the box

rigid vector
tight trout
#

I just want to type mage <whatever> or make <whatever> 😄

#

don't want to cd around and autocomplete paths to a file somewhere

tight trout