#Circular dependency in engine & sdk

1 messages · Page 1 of 1 (latest)

rough knot
#

I started looking at the engine-dev toolchain in the hope of simplifying it. It is truly a monolith within the monolith...

#

In the process I have re-discovered a very old circular dependency between 1) engine toolchain and 2) sdk toolchain. It goes like this:

A) engine build bundles contents of official SDKs, bundled in a particular way. This requires per-SDK logic (typescriptSDKContent(), goSDKContent(), and so on).

B) SDK test depends on building & running an engine.

#

SDK bundling functions belong in their respective SDK's toolchains. But we can't move them there & call them from engine-dev, because they already depend on engine-dev!

rough knot
#

<@&946480760016207902>

rough knot
#

I think we can solve this by decoupling test execution from test dependencies. This is directly connected to what @ruby nest is already working on with toolchain overlays

#
  • engine builds the engine
  • go-sdk, python-sdk, typescript-sdk etc. build their respective bundles for engine build
  • go, junit, pytest etc. execute our test suites (crucially: this requires an overlay to inject test dependencies from other toolchains)

This way the dependency graph becomes:

e2e (overlay) -> go:test -> engine:container -> go-sdk:engine-bundle, python-sdk:engine-bundle

#

and tada, no more circular dependency

#

EDIT: actually it's more complicated than that: each SDK has 2 different dependencies on engine-dev: 1) for testing, 2) for generating their binding.

I don't know if every SDK's generator functions could be cleanly moved to a reusable toolchain. It could work for the Go SDK (it uses go generate under the hood). But not sure about the other SDKs...