#`cli.Run`

1 messages · Page 1 of 1 (latest)

misty flicker
hollow briar
#

would be nice if there was a nice wrapper around all of the cli.#Run docker arguments tho instead of having to build up a string/array for the command... so it worked like the more traditional docker.#Run

#

i guess if i knew enough Cue i could probably build my own action to wrap it

misty flicker
#

between #""" and """#, you could add as many lines as you want here

misty flicker
misty flicker
hollow briar
#

well basically, docker.#Run has params for mounts and such, it'd be nice to have something that wrapped cli.#Run that didn't take an arbitrary "docker run" command but actually constructed the docker run string itself based on the higher level params that docker.#Run uses

#

if that makes any sense lol

#

in my case, it looks like i have to build an extensive command string with "sh -c" that in turns launches docker run that also runs a complicated "sh -c" string as it's own command thus making the whole thing quite messy ... i mean i guess i could wrap things in bash scripts but i was hoping to avoid that

misty flicker
#

An example, there are other ways to do it

hollow briar
#

that looks intriguing

misty flicker
hollow briar
#

inception ?

#

you'll have to excuse me, i only just began my second coffee this morning 😉

misty flicker
# hollow briar inception ?

Sorry, a docker run sh -c running a docker run sh -c, that you mentioned above. No worries, I sometimes have my own references, that no one else understand 🤣

hollow briar
#

hehe ... i'm trying to achieve bi-directional volume mounts so that whatever the docker container builds can be accessed outside of dagger

#

and that's when someone suggested i stop using docker.#Run and use cli.#Run instead

misty flicker
hollow briar
#

right - so i'm playing with it now, is there any way for me to see the docker output of the cli.#Run command ?

misty flicker
silent ruin
#

.build/requirements.txt locally outside of dagger
@hollow briar is this the important part to access outside of Dagger?

hollow briar
#

yeah i was hoping to avoid having to export in this particular scenario due to the fact that there will be a lot of data built and i'd like to be able to transfer the result of data for testing purposes

hollow briar
#

the more i discuss these things here, the more it seems i'm trying to "work against" the nature of dagger.io by having realtime bidirectional mounts and that i should just use import/exports as necessary instead

#

and i should simply "trust" dagger.io to do the right things speedily 😉

#

fyi i'm trying to reproduce my fairly large Makefile+bashscripts system in a dagger fashion

silent ruin
#

I was wondering what would be the simplest scenario we could all get working together.

I'm thinking if we have a builder docker image (or its Dockerfile), plus whatever is needed to build an actual wheel.

hollow briar
#

so here's what i'm ultimately trying to accomplish boiled down...

  1. create a new python-based image with developer deps installed and python poetry
  2. launch new container using the built image that generates requirements and then downloads/builds all necessary python wheels
  3. create my real image that is bare bones python and sets up a virtualenv using the wheels artifacts from #2 (so it doesn't have to go out to the network)
#

i'm basically also trying to prevent my Dockerfile's from having to reach out to the network

#

my standard way of handling the artifacts was the basically store them locally via standard docker mounts (bi-directional)

silent ruin
#

i'm basically also trying to prevent my Dockerfile's from having to reach out to the network
For performance? Air gapped?

hollow briar
#

yes and yes

#

i don't want a Dockerfile to have to worry that a pypi registry where wheels are stored is offline, having problems, missing the latest version i need

#

etc

#

lots of tiny reasons i guess

silent ruin
#

Or use your Dockerfile if you can share it 😄

hollow briar
#

well i could do #1 manually and simply reference the docker tag i created myself

silent ruin
#

Right. So we can play along, I mean.

hollow briar
#

oh i could show you the Dockerfile's ... they're a WIP of course because i'm trying to adapt them to dagger and such

#

that's the Dockerfile for step 1

#

i'm not doing any cleanup afterwards or anything as i'll deal with optimizations after

silent ruin
#

I'll throw some things in a repo. What will I need to build a minimal wheel 🙂

hollow briar
#

looks

silent ruin
#

I just updated

#

I split your #PythonImageBuild out into separate file

hollow briar
#

i haven't worked much with github pull requests and all that jazz so just let me know what you need 😉

#

there would be two dockerfile's ... one to setup the built wheels, and the second to construct the actual application btw (at least for my workflow)

#

and yes, i know i could use build stages, but the thing is, the first stage (the one that builds the wheels and such) ... would be nice to be reusable

silent ruin
#

Nice. Yes, you could build with Dockerfiles or use Dagger itself to build the images. So the Dockerfile we have is the build deps, etc, right?

I'm going to push it all up to my repo. Just copied your content over.
I renamed Dockerfile to Dockerfile.build, like in your Dagger gist.

hollow briar
#

yep yep

#

i'm using raw Dockerfiles and passing in build arguments to keep things configurable in an effort to allow other members on my team the ability to modify the Dockerfile workflow without having to learn dagger.io

#

if/when the rest of the team becomes more familiar with dagger.io i may convert the Dockerfile's to Dagger build steps

silent ruin
#

Cool. So, makeBuilder works 🙂 Cache speeds things up on subsequent runs.

dagger do makeBuilder --no-cache
[✔] _base                                                                   0.1s
[✔] actions.makeBuilder                                                    25.8s

dagger do makeBuilder
[✔] _base                                                                   0.1s
[✔] actions.makeBuilder                                                     0.1s
hollow briar
#

btw i had no idea of naming conventions of actions so feel free to clear things up 😉

silent ruin
#

looking good to me 🙂

#

So, now to buildWheels...

#

Do I need another Dockerfile?

hollow briar
#

yes... for the actual running of the final app

#

it'll look similar to this...

ARG PYTHON_EXACT_VERSION
FROM public.ecr.aws/docker/library/python:${PYTHON_EXACT_VERSION}-slim-bullseye

RUN \
    python3 -m venv /app

COPY app.py /app/src/app.py

CMD ["/app/bin/python", "/app/src/app.py"]
#

but in that Dockerfile we'll also need to copy over all of the wheels (built from Dagger buildWheels action which is currently commented) and install into the virtualenv

#

normally i'd use bi-directional mounts to share those wheel build artifacts to speed up the build process

#

rather than export/import/copy/whatever

#

in a larger project, there would be thousands of wheels

silent ruin
#

So you would create a venv for each one of the thousands of wheels?

#

Maybe I'm getting ahead of myself....can I build a wheel now with the python code I have?

#

Guessing I need to at least run the poetry command to get the requirements.txt

hollow briar
#

oh sorry... venv is basically just a localized Python environment so when you install dependencies (such as wheels) it doesn't infest the root python

#

so one venv for the entire project

#

yeah... poetry export --dev --without-hashes --format=requirements.txt generates requirements.txt ... requirements.txt is basically a dependency list of all wheels that are needed

#

what i don't have in any of the samples i've given you so far is the step that acutally builds the third-party wheels... after generating requirements.txt the step would be pip wheel -w someoutdir -r requirements.txt

#

i hadn't gotten that far yet because i was making sure all my previous steps were operating at peak efficiency 😉

#

so the command for buildWheels action would be something like this... mkdir -p .build .build/wheels && poetry export --dev --without-hashes --format=requirements.txt > .build/requirements.txt && pip wheel -w .build/wheels -r .build/requirements.txt

#

btw a nice to have would be some sort of #RunSteps construct where you could add individual lines to run but in turn generates the full string with adequate "&&" to run the whole thing

#

i hate writing a long sh -c "step1 && step2 && step3" ... etc manually

#

maybe have docker.#Run take an optional shell param which is an array of strings and generates the appropriate sh -c string

#

just spitballing atm

silent ruin
#

just pushed again

#

try this

dagger do buildWheels --log-format plain and you should see the requirements

hollow briar
#

fyi, not sure who's responsible for docs, but took me a lot of digging to figure out i needed to use --log-format plain to see anything useful 😉

#

but yeah that shows the requirements.txt

#

fyi, could you do me a favour and change the tag name to something less noticably a part of my company's info? (ie that 519507978765 is our company AWS account id... it's not enough to invade our system but still indicates us)

silent ruin
#

will do!

#

just in your fork now. I destroyed and re-created mine

hollow briar
#

i'm gonna delete my fork 😉

#

btw, this is the version of main.cue i just reworked a bit...

package rentalsapi

import (
    "dagger.io/dagger"
    "dagger.io/dagger/core"
    "universe.dagger.io/docker"
    //"universe.dagger.io/docker/cli"
)

dagger.#Plan & {
    _base: core.#Source & {
        path: "."
    }
    actions: {
        makeBuilder: #PythonImageBuild & {
            pyVersion: "3.11-rc"
            dockerfile: path: "Dockerfile.build"
            tag: "app:build-py3.11-rc"
        }
        buildWheels: docker.#Build & {
            steps: [
                makeBuilder,
                docker.#Run & {
                    mounts: project: {
                        dest:     "/app/src"
                        contents: _base.output
                    }
                    always:  true
                    workdir: "/app/src"
                    command: {
                        name: "sh"
                        args: ["-c", "mkdir -p .build && poetry export --dev --without-hashes --format=requirements.txt > .build/requirements.txt && pip wheel -w /app/build/wheels -r .build/requirements.txt"]
                    }
                },
            ]
        }
    }
}
#

the previous version you were working was specific to the company project i was working on, but that new version i just pasted is more appropriate to the new app we're building based on app.py

silent ruin
#

The docker.#Build construct connects the outputs and inputs of steps that yield images without you having to manually wire

step1: ...
step2: docker.#Run & {
  input: step1.output
hollow briar
#

oh, nice

#

btw, in python i would do this to get the complicated sh command string...

my_shell_cmd = ["mkdir -p .build", "poetry export ..."]
args = ["-c"] + "&&".join(my_shell_cmd)

to keep things a bit cleaner... not sure if there's an equiv in Cue

#

i haaate long "command1 && command2 && command3" strings lol

silent ruin
#

I think we can do something like that...just a sec

hollow briar
#

yep, looks about right

urban wedge
#

@hollow briar You seem to be describing a multi-stage build. That's what I do in Python with Poetry.

#

It's easy to do in Dagger, you don't need to export the wheels to your machine, just pass the dagger.#FS to the run stage.

hollow briar
urban wedge
#

As for the long command, you can use a multi-line string and set -euo pipefail.

urban wedge
hollow briar
#

right

urban wedge
#

You can write your build into a reusable action (e.g, #BuildWheels).

silent ruin
#

like splitting out build.cue from main.cue.

urban wedge
#

Got it.

#

Just a quick sketch:

#BuildWheels: {
    input: docker.#Image
    source: dagger.#FS
    _run: bash.#Run & {
        "input": input
        mounts: project: {
            dest:     "/app/src"
            contents: source
        }
        always:  true
        workdir: "/app/src"
        script: contents: """
                poetry export --dev --without-hashes --format=requirements.txt > /requirements.txt
                pip wheel -w /wheels -r /requirements.txt
                """
        export: directories: "/wheels"
    }
    output: _run.export.directories."/wheels"
}
#

Then

        buildWheels: #BuildWheels & {
            input: makeBuilder.output
            source: _base.output
        }
#

and buildWheels.output will have your wheels, so you can docker.#Copy into the run image.

hollow briar
#

interesting

#

that bash script is running outside of docker correct? i was running it inside docker container because i'm trying to reduce dependencies on the outside environment

urban wedge
#

It's running inside a container.

#

The image of that container is what's in input.

hollow briar
#

ohh, i hadn't realized it worked that way

urban wedge
hollow briar
#

i've been reading the docs constantly over the past couple of days, it's crazy some of the stuff you miss

#

yeah i knew everything run inside a container because that was the nature of Dagger, but by default it runs in some sort of private Dagger container, no ?

urban wedge
#

Someone's working on a "How it all works" doc currently 🙂

hollow briar
#

lol

urban wedge
#

You mean because you don't see it in your local docker daemon?

#

Think about a Dockerfile, that bash.#Run or docker.#Run is the same a RUN instruction.

hollow briar
#

no, i understand why that doesn't happen (sort of) ... but what i meant was that if you run a bash script and don't specify an image input, what container/image does it run ?

urban wedge
#

It's the same, they're handled by buildkit.

hollow briar
#

oh, hm

urban wedge
hollow briar
#

oh, haha

urban wedge
#

Some actions have a default image, you can do it as well. bash.#Run and docker.#Run don't. They're too generalized. So you need to specify it.

hollow briar
#

so now i'm starting to wonder why docker.#Run is even relevant

urban wedge
#

python.#Run has a default.

urban wedge
hollow briar
#

yeah i think my head is not wrapping all the way around this lol

urban wedge
#

bash.#Run is a convenience on top of docker.#Run.

hollow briar
#

oh right, yeah, that makes sense

urban wedge
#

So if you want to run some python code, it's easier. Same as bash.

hollow briar
#

neat

#

so any time i see myself wanting to run docker.#Run ... chances are there's a higher level abstraction that will make things easier such as bash.#Run or python.#Run or whatever

urban wedge
#

Yes, possibly 🙂

#

docker.#Run is the lowest of the high level abstractions 🙂

hollow briar
#

lol

#

well, the fact that bash.#Run lets me run a series of commands without concontaneating some large string is enough for me 😉

#

there should be a disclaimer in the docker.#Run docs ... "if you are reading this, there's a good chance you might want to check on bash.#Run which specifically lets you run a bash script this way"

urban wedge
#

Yes, but that works with docker.#Run as well, here's the difference:

// docker.#Run
command: {
    name: "bash"
    flags: "-c": """
        set -euo pipefail
        <command1>
        <command2>
    }
}

// bash.#Run
script: contents: """
    <command1>
    <command2>
    """
urban wedge
hollow briar
#

oh gotcha, i wasn't certain how the flags argument worked for docker.#Run

urban wedge
hollow briar
#

nice

urban wedge
hollow briar
#

np

urban wedge
#

Basically, the flags map gets flattened between command: name and command: args.

#

So that produces the following command: bash --norc -e -o pipefail <script> [args]

hollow briar
#

heh, nice ... i've found in general the cue files themselves are actually documentation enough, until i look up something like dagger.#FS source which just confused me lol

urban wedge
#

Yes, the pkg/dagger.io folder has the core stuff. Some things there come from Go or are interpreted by it. It's universe.dagger.io that has stuff that could be written by anyone. We just collect a few of them.

hollow briar
#

i guess that means i'd understand some things easier at a glance if i knew Go then

#

i keep meaning to learn golang, but who has the time lol

urban wedge
#

No, not necessarily. It just helps to know where the boundaries are, but if you want to stay clear of that just know that this happens.

hollow briar
#

probably be good to document what those boundaries are

urban wedge
#

It's actually simple, let me just give you the run down and see if it makes sense: When using types (https://docs.dagger.io/1234/dagger-types-reference/), they hold internal ids that you can't access in CUE, that have meaning in Go. So these types will be interpreted and filled by Go.

hollow briar
#

yep i asaw a few @dagger references that threw me for a loop

urban wedge
#

So it's anything you see with $dagger: ... that is interpreted by Go, or @dagger(generated).

hollow briar
#

fyi, this line which i see quite often... i still only barely grasp what it's for... client: filesystem: ".": read: contents: dagger.#FS which is why i was looking for docs on dagger.#FS and got stumped lol

hollow briar
urban wedge
hollow briar
#

i know what it ultimately gets used for, but the individual fields... like i see it takes a filesystem param, but where is the filesystem param documented?

urban wedge
hollow briar
#

does the filesystem param only take a string?

#

i can't help but feel if i knew Go syntax better it would be a little clearer to me

urban wedge
hollow briar
#

heh, honestly, i've read all of these docs before (including that source cue file) and at the time it didn't "connect" for me, but given the more basic understanding i've acquired today even in this discussion thread, things seem to be connecting better in my head

urban wedge
#

It helps us to know what could be improved so that first timers have an easier understanding.

hollow briar
#

it certainly helps to have a more fundamental understanding of how Cue data structures are defined and used... which is starting to happen for me 😉

urban wedge
#

Yes, definitely. When I started using dagger I made sure to learn CUE first. When you try to learn both at the same time I think it's harder to know one from the other.

hollow briar
#

the single line Cue data structure definitions threw me for quite a loop in the beginning

hollow briar
#

is Cue used on any other big projects generally used that i might be aware of ?

#

i have to admit that this was the first barrier for me to even consider using Dagger, was that i had to learn yet another configuration language lol

urban wedge
#

Not sure. But some use it extensively. We have team members that used it for thousands of cloud formation configs before joining dagger. I heard of it from kubevela.io, but only learned cue with dagger.

hollow briar
#

gotcha

#

wonders how long it'll take before he ends up adding Cue to his resume lol

urban wedge
silent ruin
urban wedge
#

Now looking back, on one hand it allows us much flexibility and power. Once you get it, it unlocks a great deal. On the other, it adds to the learning curve for first timers.

hollow briar
#

yeah, right now my brain is going, "how could i best express the workload that @silent ruin is doing in that github repo in a simple yaml format that gets translated into cue" lol

urban wedge
hollow briar
#

yaml-first, but then doing raw cue to do truly powerful things later on, to lessen the initial learning curve 😉

silent ruin
# 
    makeBuilder: #PythonImageBuild & {
      source:    _base.output
      pyVersion: "3.11-rc"
      dockerfile: path: "Dockerfile.build"
      tag: "app:build-py3.11-rc"
    }
    buildWheels: #BuildWheels & {
      input:  makeBuilder.output
      source: _base.output
    }
urban wedge
hollow briar
#

it'd be nice to see some common naming patterns for the toplevel cue files too... like to me, main.cue and build.cue doesn't really make sense to me... i'd rename main.cue to plan.cue and build.cue to actions.cue until actions.cue grew to large and then made actions be a dir with sub-cue files

#

well not necessarily actions.cue ... but whatever toplevel unit name

#

anyhow, i think now i'm just nit-picking

#

also, it threw me for a loop when i accidentally discovered that dagger just picked up whatever cue files are in the current directory, no need to have special names or whatever

#

reminds me of terraform files actually

silent ruin
#

totally. ideally, we'll make these Actions we've been working on packages that anyone can use by adding them to universe.dagger.io. Then anyone can import them into their Dagger plans 🙂

hollow briar
#

right

silent ruin
urban wedge
#

You don't need to put in universe to reuse. Any git repo you have access to can be used.

#

You can provide your own for others to use, or privately in your company.

hollow briar
#

interesting

#

curious, what is the trailing underscore in export: directories: "/wheels": _ for @silent ruin ?

urban wedge
#

It means, match whatever's the type for it. Instead of re-declaring it again.. you'd have to look what it is and do export: directories: "/wheels": dagger.#FS instead.

hollow briar
#

oh ok, that makes sense

urban wedge
#

It's creating a new struct from a template, you just need to create a new value/struct from the template, and inherit the rest of the structure.

silent ruin
#

@hollow briar I pushed a new version that doesn't export the wheels as a filesystem, but rather provides the whole image with the wheels on it. Not sure what you need exactly, but it's fun to see it working and starting the Quart app 😄

# 
10:25AM INF actions.app._run._exec | #11 1.283  * Serving Quart app 'app'
10:25AM INF actions.app._run._exec | #11 1.283  * Environment: production
10:25AM INF actions.app._run._exec | #11 1.283  * Please use an ASGI server (e.g. Hypercorn) directly in production
10:25AM INF actions.app._run._exec | #11 1.283  * Debug mode: False
10:25AM INF actions.app._run._exec | #11 1.283  * Running on http://0.0.0.0:5000 (CTRL + C to quit)
10:25AM INF actions.app._run._exec | #11 1.283 [2022-05-30 17:25:44,435] Running on http://0.0.0.0:5000 (CTRL + C to quit)
hollow briar
#

oh nice

#

i think that works fine for the case in point ... for me, the main advantage to having the wheel files themselves as an exported build artifact is so i can use more conventional means to cache them in case i need to start from scratch or clear my dagger cache or whatever and not have to rebuild those wheels again

#

btw, the last step that does the pip install thing is not using the wheels you already built which kind of defeats the purpose 😉

silent ruin
#

yeah, figured that, just changed it 😄

script: contents: """
                    /app/bin/python -m pip install --upgrade pip
                    for wheel in /wheels/*.whl; do /app/bin/pip install $wheel; done
                    /app/bin/python /app/src/app.py
#

so you'd take the wheels in a filesystem (directory) and use them, but you'd also dump them to the client machine, for example?

hollow briar
#

i think the cleanest way would be to do...

pip install -r requirements.txt -f /wheels

where requirements.txt would have come from the step that does the poetry export

hollow briar
#

and the main reason for wanting those wheels kept is due to the fact that building them can take forever if they include C extensions, particularly if you're building them on an x86 host using a arm64 emulator

#

so we really want to avoid running that stage unless pyproject.toml changes (which in turn changes requirements.txt which means newer wheels should be used)

#

one last thing tho, since you're having everything based on the buildWheels image... that means all of the build dependency junk is left in the image (making it quite large) ... so we basically want to mimick a multi stage build so that those deps are left out

#

so we want to run a second #PythonImageBuild action that makes a clean image and then get the wheels from the first image built into this new image

silent ruin
#

got it. I pushed up latest.

becoming clearer and clearer 😄

hollow briar
#

lol, you'll be a Python project image builder expert after this 😉

#

so basically a lot of the things i'm asking about (like preserving wheels separately) may seem non-sensical from a standard perspective, but there's a method to my madness based on years of building python images 😉

#

now i'm trying to take those experiences and implement with Dagger

silent ruin
#

I have to be afkb for a bit, but I'll check back a bit later.

I work a lot with Dagger users conducting Pilots related to a work (or personal) project, so don't hesitate to ping me (or whoever is around) 🙂

hollow briar
#

sounds good to me... before you go tho, could you give me a hint on how to start a new image that copies the wheels from the first image? 🙂

#

seems like i should be doing something special with dagger.#FS but it so far eludes me lol

silent ruin
#

Yes, indeed. For one thing, we could just pull a new image with the tag you specified and then copy (or ephemerally mount) the FS in and run our commands on it.

hollow briar
#

right

silent ruin
#

Just pushed another version. Check it out 🙂

#

it assumes a local registry on 5042 to push the finished image to

docker run -d -p 5042:5000 --restart=always --name localregistry registry:2
hollow briar
#

neat 😉

hollow briar
# 
# build image and setup in local docker
dagger do saveLocal
# run the app using local docker
docker run -it --rm pythonapp:1
silent ruin
#

Like the imageID output!

Field    Value
imageID  "sha256:26916eb26b4609076082374d77b403432c7cfe4747e0b0504902f868b8aa0d39"

docker run -it --rm  -p 5050:5000 sha256:26916eb26b4609076082374d77b403432c7cfe4747e0b0504902f868b8aa0d39

curl localhost:5050
hello
hollow briar
#

yeah... i've forked your repo again and doing some more cleanup based on what i know of python image building 🙂

silent ruin
#

awesome. I love how the caching kicks in

unknown.png
hollow briar
#

yeah, i would consider the next step to save those wheels into your local/client filesystem ... and then when building them again next time, check the client/local filesystem first (perhaps #Mount ?)

#

just added a runTest action that runs some useless tests for python

silent ruin
#

client: filesystem: "./.build": write: contents: actions.buildWheels.output

#

put that at the top and now I've got 60 wheels and a requirements.txt in my local .build 😄

hollow briar
#

nice!

midnight dove
#

Sorry if I’m late to the party, but i recommend looking at the yarn package for a good example of how to deal with “one-way” mounts when you need to get something out of your build process (in the case of yarn: the build). Basically the script copies just the output artifacts

silent ruin
#

hmmm...would it make sense to use a something in the requirements.txt as a cache key?

#

hash of the file, for example...

hollow briar
#

@silent ruin i'm not sure how you'd do it... but as for not rebuilding when necessary ... poetry export --dev --without-hashes --format=requirements.txt > /app/build/requirements.txt should only get run iff poetry.lock changed (and not by checking last modified, should test a checksum) ... and then of course pip wheel -w /app/build/wheels -r /app/build/requirements.txt should only be run if requirements.txt changed by testing checksum

#

it seems to me that #buildWheels _run action should be broken out into separate steps... one that bootstraps (every line before poetry export) and then a step that runs the poetry export and then a step that runs the pip wheel

#

i had that on my shortlist but got distracted (breaking out the steps ... not the checksum/hash checking)

#

i've pushed up some more small tweaks to my fork btw

silent ruin
#

Awesome. That logic is gold.
Seems super do-able. What doesn't work (I tried it out) was to just write out the wheels to the client and then mount that when building. Acts like a new build every time. Def need to let the system know we're using a cachedir.

there is a good lib to help out

import "crypto/sha256"

hollow briar
#

right... well i guess i'll begin by attempting to break out _run into multiple steps

#

seems like checksum testing should have a higher level primitive in dagger.io proper tho 😉

silent ruin
#

I'll ask a colleague about caching tomorrow. 👍

hollow briar
#

if i'm inside the steps of a docekr.#Build ... is there a way for step2 to reference fields from step1 ?