#Linking cache-able steps

1 messages · Page 1 of 1 (latest)

cosmic quiver
#

I have the following use case where I obtain an oAuth token to call an API.

getToken()
callApi()

I run into this situation where the getToken() step is cached but the callApi() re-runs but is using an expired token. How do I bust the getToken() step cache based on the running of callApi()? I dont' want getToken() to run if callApi() isn't run. How do I link those two steps cache?

reef steeple
#

Hey @cosmic quiver 👋
The best way would be to logically link them in the engine's dag. What I mean is - getToken() probably returns something to the client? Why not feed it directly into callApi() instead (or in addition to the client, if you need it on the client for other reasons).

For example, if the token here is from the stdout of getToken(), in that pipeline you can have:

func getToken() ... {
  // getToken pipeline here
  ...
  WithExec([]string{"the command that outputs the token"},
    dagger.ContainerWithExecOpts{
      RedirectStdout: "/token",
    }
  ).
  ...
}

and then in the callApi() pipeline:

func callApi() ... {
  // callApi pipeline here
  ...
  WithMountedFile('/token', tokenPipeline.File('/token')).
  ...
}

By making callApi() dependent on getToken(), you also no longer need to call getToken() directly, unless you also need other information from that pipeline of course.

Does this map at all to the logic in your pipeline? If not I can give another way if you let me know more about the pipeline 🙂

cosmic quiver
#

Hey Kyle, that does sort of map to my logic. However, the token I get is embedded in a json response. So I am doing curl mycommand > authResponse.json then using that auth response within Go to extract the token with a json.Unmarshal. I can see why this makes it disconnected from the API call. I could technically pass in that authResponse.json as a file to the API call step (it would be superfluous without usage) to tell the DAG that they are related right? There is probably a better way 🙂

That RedirctStdout is a cool way to do what I did within the curl command. I didn't know about that.

reef steeple