#github-feed
1 messages ยท Page 25 of 1
This change "collapses" together log "groups" containing hidden fields
for the TTY logger.
e.g. actions.foo._a and actions.foo_b will show up as a combined actions.foo.
Caveats: Since we don't know in advance how many tasks are in a group, the state will change back and forth.
For each task in a group, the state will transition from computing to complete (e.g. blue to green), then back to computing and so on.
The transition is fast enough not to be visibile.
Signed-off-by: Andrea Luz...
The default behavior of docker run when no --entrypoint is provided, and ENTRYPOINT[] has not been overridden is to use /bin/sh -c. This allows CMD to be easily processed through the shell. Overridding entrypoint allows for bootstrapping scripts to do setup, perhaps export some necessary env vars, then exec "$@" as one example.
The current implementation does not seem to allow providing a direct entrypoint to use besides the one in image config. From docker.#Run._exec:
...
Having reviewed some use cases with @shykes we determined that this ability would ease development and reduce boilerplate in many instances. The primary change is the ability to pass a #Secret as an env var to exec:
engine.#Exec & {
env: TEST: inputs.secrets.testSecret.contents
}
Signed-off-by: Richard Jones
Overview
This issue tracks the redesign of the dagger CLI for the 0.2 release aka โEuropaโ.
Status
This is an incomplete draft. Fire at will.
Design proposal
$ dagger help
Dagger is a modern replacement for Make, from the creators of Docker.
Usage: dagger [options] command ...
Commands:
help Show help information
do Execute an action
undo Undo an action
diagnostics Show diagnostics information
Most of the magic ...
Change the API for docker.#Run.export.files and docker.#Run.export.directories, so that the contents field is not needed.
The API is not changed for declaring exports:
action: run: docker.#Run & {
export: {
files: "/foo.txt": _
directories: "/bar": _
}
}
The API is changed for referencing exports. Note the absence of contents field:
action: run: _
foo: run.export.files."/foo.txt"
bar: run.export.directories."/bar"
This would ...
When I run make web, I get the following errors:

This is what I get in a browser:

If I bump docusaurus to ^2.0.0-beta.15, I get the following errors:

A...
Overview
As the Europa release aproaches, we have one last opportunity to revisit terminology and other superficial aspects of the new developer experience, such as field names and schema layout.
Proposed changes
These changes are proposed as a starting point for discussion. Fire at will.
Project, instead of Plan or DAG
We haven't yet chosen between #DAG and #Plan. I propose retiring both, in favor of #Project.
dagger,#DAGis repetitive- CLI works bette...
Bump up docusaurus to the latest stable version.
Bump down mdx-js/react causing issue with docusaurus. mdx-js/react is now ESM only but we still using "require" for remarkCodeImport. When Docusaurus will be able to use ESM syntax, mdx-js/react will be bump up.
Bats implementation in Europa. Fixes #1556
Signed-off-by: guillaume
I found some typos as I was acquainting myself with the codebase.
It ain't much, but it's honest work.
Bumps github.com/docker/distribution from 2.7.1+incompatible to 2.8.0+incompatible.
Release notes
Sourced from github.com/docker/distribution's releases.
v2.8.0
registry 2.8.0
Welcome to the v2.8.0 release of registry!
The 2.8.0 registry release has been a long time overdue.
This is the first step towards the last 2.x release.
No further active development will continue on 2.x branch.
Security vulnerability patches to 2.x might be considered, but
al...
Bumps @mdx-js/react from 1.6.22 to 2.0.0.
Release notes
Sourced from @โmdx-js/react's releases.
2.0.0
Welcome to MDX 2! See the of the website for everything:
MDX 2: https://mdxjs.com/blog/v2/
Migrating: https://mdxjs.com/migrating/v2/
@mdx-js/mdx
4a48f1f4 Fix custom elements (#1911)
@mdx-js/react
9ca9d404 Fix unnecessary top-level context changes (#1924)
@mdx-js/loader
e0b697ab 9d5501b2 Add improved webpack cache (#1912, #19...
what_isdidn't have an idoperator-manualid was clashing withciuse-case
This is expected to be a long-running branch, deployed to a preview environment. We will be adding content until we are ready to flip the switch - current pages are placeholders for now.
Resolves #1327 (when merged)
NOTE: this PR is spun out of #1520 to get the most disruptive change out of the way and avoid a rebase storm.
cmdis renamed tocommandfor readabilityscriptis removed. Feature moves up the stack (bash.#Run,
python.#Run)
Add a failing test (commented out) that reproduces the issue with image customization. See #1457.
Signed-off-by: jffarge
Problem
When I call docker.#Run with a missing image, it silently skips the underlying task, instead of returning an error. This caused me to miss a bug in my code, because the tests that should have caught it were silently ignored instead...
NOTE: this is probably not specific to docker.#Run. I didn't have time to investigate.
Solution
Incomplete tasks should cause an error, or at least a very visible warning.
https://github.com/containerd/nerdctl is an api compatible docker client.
When I did the example today I got to the point where the dagger cli made a shell call to docker and since I donโt have the docker client installed it failed.
Itโs maybe just me but I prefer to use the open source nerdctl instead of the closed source docker client. There are a few other reasons too.
if your curious here is the golang cli repos that allow a developer to use nerdctl with containerd.
https://g...
- Remove mandatory 'cache' field
- New 'name' field to customize cache IDs instead
- Simplify and expand tests
Universe tests are inconsistent in their style and file layout. This PR aims to 1) enforce a consistent style across all tests, 2) make all tests simpler to read, write and run.
Overview
This issue tracks the design of the Cue definition package.
Prerequisites: cue needs to be at the latest version
Status
In progress, will need your opinion on the subject
Design proposal
Two options:
- The
#Gostyle: one#Cuedefinition and others wrapping some options (#Fmt,#Get,#Cmd) - The bats style: one
#Batsdefinition accepting an array of string as cmd inputs
Various changes to API cosmetics. This includes breaking changes that are wide in scope, but superficial (rename package or definition, and you're good to go)
- Initially
docker.#Runhas animagefield - Later we wanted to allow using
docker.#Runas a step todocker.#Build. This requires aninputfield - We decided that
imagewas better, butinputwas necessary, so we kept both, thanks to a nice disjunction by @helderco - After some more mileage: the convenience is not worth the added complexity, especially since disjunctions are often the source of CUE issues.
This PR removes the image field and replasces it with just input.
Problem
When two outputs: files structs have the same dest path there is a race condition and/or partial file truncation issue that means the plan is not idempotent. Repeatedly calling dagger --europa up followed by cat ./test in the example below will give different outputs. The order of the contents strings as well as --no-cache options seem to possibly affect the outputs as well. The export order file then file2 versus file2 then file seems to be consistent across ...
This continues the work session with @gerhard, @jlongtine and @talentedmrjones . I couldn't find the commit from that discussion, so I re-did it from memory, and added more :)
There are a few bugs but it's a net improvement from current main, so I'm fast-tracking merge.
- Use unique IDs for all existing doc pages
- Add the must-have pages for the Europa release
- First pass at the local CI page
https://charm.sh/ looks interesting, no idea if it will work. Capturing it as a thought.
I don't see anything related to ASCII diagrams on charm.sh website; maybe I'm missing some context.
Maybe this would help:
ascii-graphs
Parse ASCII art diagrams of graphs (nodes and edges)
https://github.com/mdr/ascii-graphs
Or just plain old Grpahviz, compiled to JS/WASM: http://viz-js.com/ (not maintained).
Or the more modern, and actively developed MermaidJS: https://mermaid-js.github.io/
Problem
I'm trying to run dagger in Gitpod. There is a Docker engine, but it does not allow me to run privileged containers:
$ docker run --privileged --rm -i -t alpine /bin/sh
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "cgroup" to rootfs at "/sys/fs/cgroup" caused: operation not permitted: unknown.
...
Work in progress
- implements build of dagger binary (linux only for now)
Next:
- Port other things from the Makefile (cue fmt, golint)
- Experiment with conditional actions using params
- Support multi-arch for dagger binary (detect host platform?)
Related to #1522 #1549
Overview
When using dagger, I often wish it embedded its own cue subcommand. I would still have the regular cue command installed, and used it frequently, but dagger cue would be a great complement, for several reasons:
- Lower friction for beginners. Many new Dagger users have never used CUE. By shipping
cuewithindagger, we reduce the number of steps needed to get started. - A curated subset. The
cuecommand is very dense, and one can easily get lost in all its option...
We want doc updates to flow continuously, not just on a new tag.
You are right, this does not exist today. Context & link to original Changelog Slack thread:
<img width="398" alt="image" src="https://user-images.githubusercontent.com/3342/153389598-9d96de3f-3073-49ae-ae07-cf8ba43b6bc8.png">
https://github.com/mdr/ascii-graphs comes close, but it's not Go.
We would need something in Go so that dagger CLI could generate something along the lines of https://github.com/thechangelog/changelog.com/blob/0b5e1a7999a8179a5d892a1974e2175a9acf37fa/2021/dagger/prod_image/main.cue#L45-L63
More context: https://github.com/thechangelog/changelog.com/pull/395
As it doesn't exist atm [Gographviz creates an image from it, it's not ascii (https://github.com/goccy/go-graphviz)], one solution would be to rely on that: https://github.com/ggerganov/dot-to-ascii
If we trigger a deploy, then we want the deploy to actually run.
I am assuming that this was in place when there was a single GitHub Actions workflow that would trigger this all the time, even when nothing changed in docs/**. Since https://github.com/dagger/dagger/pull/1498, we no longer have this problem.
After merging https://github.com/dagger/dagger/pull/1591, I was caught by this behaviour which to me seemed surprising. This change should make docs deploy via Netlify behave in a m...
Nice, I'll mention this in the original Slack thread.
When using the gh cli, e.g.
or seeing the checks inline, e.g.
the existing names were poor.
This change makes gh cli checks show the following:
Problem
After manually testing dagger on Windows (#1540). Everything works fine unless the output which breaks with the escape sequence
Solution
Change the espace sequence in function of the host
Bumps postgresql from 9.4-1200-jdbc41 to 42.2.25.
Commits
See full diff in compare view
Dependabot wil...
In order to replace certain tasks implemented with make or other scripts that runs on your host machine, it's a must-have to have access to the client's OS and Platform.
For instance: #1589 - It implements basic actions ported from the Makefile. Since all actions commands run within linux containers, every binary produced will be ELF, not usable from the Mac where I run dagger up. It would be an easy fix if the action had access to the client platform and os so it can cross compile.
E...
Overview
I propose a new API for securely accessing the client machine from a Dagger plan. This is informed by:
- My own usage of Europa
- Feedback from @samalba on the DX of
inputsandoutputs - User issues with output schema: #1581
- Need to access information about client platform and arch information: #1595
- Thinking about the implications of #1558
Proposed design
This is a work in progress. Fire at will.
package dagger
#Plan: {
// Access client...
Overview
I propose formalizing the concept of a Dagger action, and exposing it to developers in a new Action API.
Using the Action API, developers can:
- Customize how the DAG is presented to the user at runtime (show actual actions instead of raw CUE tree or low-level tasks)
- Synchronize actions: #940, #1249 , etc.
- Make error messages more useful. The hierarchy of actions act as a "stack frame" with less noise than raw CUE errors
Proposed design
This is a work in ...
Currently the dagger Github Action can install dagger binary:
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
install-only: true
But running dagger up in Action will fail for any buildkit operations. For example:
patchGrafana: {=
#up: [
op.#Load & {
from: kubernetes.#Kubectl & {
version: "v1.23.3"
}
},
...
op.#Exec & {
always: true
args: [
"/bin/bash",
"--noprofile",
"--no...
The [dot-to-ascii][] is an online webapp. It's a PHP script that receives input from client, invoke a Perl script, and returns the result to the client. I think this is too heavy, fragile, and insecure a tool to be used and relied upon by dagger cli.
I feel [go-graphviz][] has a good chance of implementing ascii output, in addition to the many formats it already does. And it would be the appropriate project for a CLI to rely upon; local execution removes security, fragility, and load conc...
The Perl script that dot-to-ascii relies on, seems to be a long-standing package Graph::Easy, widely used, widely recommended, and well-maintained.
Since the daggers CLI expects buildkit to be present, would it be an option to run that in a container and grab the output. It'd still be local execution, with the other attributes of stability and security, but it'd be heavier compared using a Golang package.
Definition are now split in their own file.
#Imageis a simple base image to set up a go container#Containeris a standalone environment to execute go command, any go command
should use this definition.
It sharesdocker.#Runfields to easily maintains this definition.#Buildand#Testare high level definition that use#Containerto
execute common go operation in a CI.
Those definitions are tested with a simple greeting package.
Signed-off-by: Vasek - Tom C
The link is visible only by an exhaustive list of allowed user.
Signed-off-by: jffarge
There are a few more things to do before we can finish this, see the comments in the doc. The most important item is to switch the Dagger config to Europa. That is actually one of the next steps.
Sharing this early so that we start collaborating. Pushing to a branch in the dagger org so that we can work on it together (personal repositories make it more difficult).
cc @grouville
Bumps go.opentelemetry.io/otel/sdk from 1.3.0 to 1.4.0.
Changelog
Sourced from go.opentelemetry.io/otel/sdk's changelog.
[1.4.0] - 2022-02-11
Added
Use OTEL_EXPORTER_ZIPKIN_ENDPOINT environment variable to specify zipkin collector endpoint. (#2490)
Log the configuration of TracerProviders, and Tracers for debugging.
To enable use a logger with Verbosity (V level) >=1. (#2500)
Added support to configure the batch span-processor with e...
Bumps go.opentelemetry.io/otel/exporters/jaeger from 1.3.0 to 1.4.0.
Changelog
Sourced from go.opentelemetry.io/otel/exporters/jaeger's changelog.
[1.4.0] - 2022-02-11
Added
Use OTEL_EXPORTER_ZIPKIN_ENDPOINT environment variable to specify zipkin collector endpoint. (#2490)
Log the configuration of TracerProviders, and Tracers for debugging.
To enable use a logger with Verbosity (V level) >=1. (#2500)
Added support to configure the b...
Bumps go.opentelemetry.io/otel from 1.3.0 to 1.4.0.
Changelog
Sourced from go.opentelemetry.io/otel's changelog.
[1.4.0] - 2022-02-11
Added
Use OTEL_EXPORTER_ZIPKIN_ENDPOINT environment variable to specify zipkin collector endpoint. (#2490)
Log the configuration of TracerProviders, and Tracers for debugging.
To enable use a logger with Verbosity (V level) >=1. (#2500)
Added support to configure the batch span-processor with environme...
Bumps go.opentelemetry.io/otel/trace from 1.3.0 to 1.4.0.
Changelog
Sourced from go.opentelemetry.io/otel/trace's changelog.
[1.4.0] - 2022-02-11
Added
Use OTEL_EXPORTER_ZIPKIN_ENDPOINT environment variable to specify zipkin collector endpoint. (#2490)
Log the configuration of TracerProviders, and Tracers for debugging.
To enable use a logger with Verbosity (V level) >=1. (#2500)
Added support to configure the batch span-processor wi...
Bumps amplitude-js from 8.16.1 to 8.17.0.
Release notes
Sourced from amplitude-js's releases.
v8.17.0
8.17.0 (2022-02-10)
Features
Support seamless integration with amplitude experiment SDK (#457) (af8f9d1)
Changelog
Sourced from amplitude-js's changelog.
8.17.0 (2022-02-10)
Features
Support seamless integration with amplitude experiment SDK (#457) (af8f9d1)
Commits
c144956 chore(release): 8.17.0 [skip ci]
af8f9d1 feat: Suppor...
@gurjeet @grouville I looked at Graph::Easy a bit last night. I'm super impressed with how well it does. The unicode output in particular looks very good. Generating the DAG at each step of evaluation should be relatively straightforward. And we might be able to utilize buildkit to run Graph::Easy in a container. Although part of me just wants to figure out how to extract the dot -> ascii/unicode bits of Graph::Easy and rewrite them in golang. That'd be awesome. :)
If at some point you need some hands, I'll be glad to help ๐
WIP: This requires #Secret as env vars (tested locally with uncommitted changes). This will need to be merged after #1557.
The goal of this package is to provides 2 low-level primitives
#Build: an alpine-based image with the aws cli installed#Run: a wrapper todocker.#Runthat pre-configured credentials and config for anything using an AWS SDK
To use:
aws.#Run & {
credentials: aws.#Credentials & {
accessKeyId: inputs.secrets.AWS_ACCESS_KEY_ID.contents
...
Error
2.328 error An unexpected error occurred: "https://registry.yarnpkg.com/jest-diff/-/jest-diff-24.9.0.tgz: ENOENT: no such file or directory, open '/cache/yarn/v6/npm-jest-diff-24.9.0-931b7d0d5778a1baf7452cb816e325e3724055da-integrity/node_modules/jest-diff/.yarn-tarball.tgz'".
Context
The Yarn package embeds caching via
mounts: {
"yarn cache": {
dest: "/cache/yarn"
contents: engine.#CacheDir & {
// FIXME: are there character...
I've seen that there's support for Maven, but lots of Java users (and probably the same amount if you consider Android development) use Gradle, and I think that one should be supported too.
Bumps ajv from 6.12.0 to 6.12.6.
Release notes
Sourced from ajv's releases.
v6.12.6
Fix performance issue of "url" format.
v6.12.5
Fix uri scheme validation (@โChALkeR).
Fix boolean schemas with strictKeywords option (#1270)
v6.12.4
Fix: coercion of one-item arrays to scalar that should fail validation (failing example).
v6.12.3
Pass schema object to processCode function
Option for strictNumbers (@โissacgerges, #1128)
Fixed vulnerability rel...
Bumps follow-redirects from 1.14.7 to 1.14.8.
Commits
3d81dc3 Release version 1.14.8 of the npm package.
62e546a Drop confidential headers across schemes.
See full diff in compare view
Bumps follow-redirects from 1.13.0 to 1.14.8.
Commits
3d81dc3 Release version 1.14.8 of the npm package.
62e546a Drop confidential headers across schemes.
2ede36d Release version 1.14.7 of the npm package.
8b347cb Drop Cookie header across domains.
6f5029a Release version 1.14.6 of the npm package.
af706be Ignore null headers.
d01ab7a Release version 1.14.5 of the npm package.
40052ea Make compatible with Node 17.
86f7572 Fix: clear in...
This is how Graph::Easy renders Go Package Imports | Graphviz

Rather impressive. Especially given the medium.
Here's a URL dump of interesting/potentially useful things:
I've been paying attention to engine.#CacheDir because of the work that has been done on #1336, and I noticed that Dockerfile cache mounts have more options available than our current spec provides.
versus
RUN --mount=type=cacheThis mou...
Adds --no-vendor to dagger up to prevent rewriting contents of cue.mod
Closes #1328
Signed-off-by: Richard Jones
@gedw99 This ought to be possible. In the Dagger docs, we show how to use a custom buildkit: Dagger Operator Manual - Custom buildkit setup.
I've attempted to use colima and nerdctl to run buildkit (using nerdctl run --net=host -d --restart always -v dagger-buildkitd:/var/lib/buildkit --name dagger-buildkitd --privileged moby/buildkit:v0.9.3), but I've had trouble running against it. I ran out of time to do more de...
instead of the closed source docker client
Only Docker Desktop is closed source. Everything else is open source. For Docker client: https://github.com/docker/cli
Yes the docker client is 100% open-source! In spite of what jealous competitors will tell you. If you could please correct that point in your original post @gedw99 that would be very appreciated. Supporting nerdctl is a great idea in its own right and there is no need to drag down other products unfairly.
@shykes fixed - sorry about misinformation. I got it from wrong.
Thank you very much. And no worries, I have been in your situation many times :)
Bumps cypress from 9.4.1 to 9.5.0.
Release notes
Sourced from cypress's releases.
9.5.0
Released 2/15/2022
Features:
Enhancements were made to the error experience in both run mode and open mode
to improve readability and provide meaningful stack traces. Addressed in
#20124.
Updated the cy.request() log message to hide the origin when it matched the
browser origin to make debugging easier. This reduces the length of the log
message that could be par...
Overview
It would be useful to allow injecting data into a command's standard input, and capture its standard output and error streams.
Unlike an earlier proposal, data would be buffered and not streamed.
Proposed design:
// dagger.io/dagger/engine
package engine
#Exec: {
stdin?: string
stdout?: string
stdout?: string
...
}
// universe.dagger.io/docker
package docker
#Run: {
stdin?: string
stdout?: string
stdout?: string
...
}
Splitting out from #1579. This is a small change that everyone seems to agree with, so no need to wait on the rest.
This implements universe/aws/cli.#Command as a declarative wrapper to the aws CLI tool. By specifying the global options, service name and command, and credentials, #Command will automatically format the correct shell syntax and execute via /bin/sh storing the output in /output.txt. Furthermore when setting unmarshal: true this definition will automatically json.Unmarshal the result into a CUE struct that can be referenced by result.path.to.arbitrary.field
To use:
After dagger init, it will generate a cue.mod with default dagger packages. How can I add a new dagger package of my own? I saw a usr/ directory. Is it the right place?
Ever since I started working on #1535, I wanted to be able to also unset some things.
I tried a first pass at the time that didn't work out, but decided to not delay because of it. I want to try again ๐
For me this has come up for one reason. Sometimes I use a base image that's mostly fine, but I wish they hadn't set up a few VOLUME, EXPOSE or ENV instructions.
I've wondered about a docker.#Unset, but it's not clear to me how you would chose what to delete precisely.
What ...
I think the test field in healthcheck should be abstracted into a new cmd field and the time intervals could also be humanized.
Currently the test field is a single list where the first element has special meaning:
#HealthCheck: {
test?: [...string]
interval?: int
timeout?: int
startPeriod?: int
retries?: int
}
The options are:
[]: inherit healthcheck["NONE"]: disable healthcheck["CMD", args...]: exec arguments directl...
Currently, you can describe comand in docker.#Run with
#Run: {
command?: {
// Name of the command to execute
// Examples: "ls", "/bin/bash"
name: string
// Positional arguments to the command
// Examples: ["/tmp"]
args: [...string]
// Command-line flags represented in a civilized form
// Example: {"-l": true, "-c": "echo hello world"}
flags: [string]: (string | true)
}
_exec: engine.#Exec & {
args: [command.name] + command._flatFlags + c...
Bumps github.com/containerd/containerd from 1.6.0-rc.2 to 1.6.0.
Release notes
Sourced from github.com/containerd/containerd's releases.
containerd 1.6.0
Welcome to the v1.6.0 release of containerd!
The seventh major release of containerd includes many improvements and added
support to increase overall compatibility and stability.
Highlights
Runtime
Add runtime label to metrics (#5744)
Cleanup task delete logic in v2 shim (#5813)
Add support for ...
Bumps go.opentelemetry.io/otel from 1.4.0 to 1.4.1.
Release notes
Sourced from go.opentelemetry.io/otel's releases.
Release v1.4.1
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Changelog
Sourced from go.opentelemetry.io/otel's changelog.
[1.4.1] - 2022-02-16
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Commits
065ba75 Releas...
Bumps go.opentelemetry.io/otel/trace from 1.4.0 to 1.4.1.
Release notes
Sourced from go.opentelemetry.io/otel/trace's releases.
Release v1.4.1
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Changelog
Sourced from go.opentelemetry.io/otel/trace's changelog.
[1.4.1] - 2022-02-16
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Commi...
Bumps go.opentelemetry.io/otel/sdk from 1.4.0 to 1.4.1.
Release notes
Sourced from go.opentelemetry.io/otel/sdk's releases.
Release v1.4.1
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Changelog
Sourced from go.opentelemetry.io/otel/sdk's changelog.
[1.4.1] - 2022-02-16
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Commits
06...
Bumps go.opentelemetry.io/otel/exporters/jaeger from 1.4.0 to 1.4.1.
Release notes
Sourced from go.opentelemetry.io/otel/exporters/jaeger's releases.
Release v1.4.1
Fixed
Fix race condition in reading the dropped spans number for the BatchSpanProcessor. (#2615)
Changelog
Sourced from go.opentelemetry.io/otel/exporters/jaeger's changelog.
[1.4.1] - 2022-02-16
Fixed
Fix race condition in reading the dropped spans number for the Batc...
Having forked this, the repository name doesn't tell me that these are dagger examples:
While the fork info helps in the GitHub web UI, it doesn't locally:
I think that we should rename the repository from:
- dagger/examples
+ dagger/dagger-examples
I know that this will have a knock-on effect on a few tests and docs (possibly other things), but I would prefer that we did this before launch.
WDYT?
It should be quick & straightforward to get this running locally.
Related to https://github.com/dagger/dagger/pull/1586
Signed-off-by: Helder Correia
We should have great docs generated for our Europa types and Universe packages.
We need to figure out how to structure that generation. We can probably use AST parsing (for required + optional fields, comments, and maybe attributes) to generate solid docs from our base types.
Given the following config:
package todoapp
import (
"dagger.io/dagger"
"universe.dagger.io/yarn"
)
dagger.#Plan & {
inputs: directories: app: path: "./"
actions: {
build: yarn.#Build & {
source: inputs.directories.app.contents
}
// TODO: This is expected to fail, but it currently doesn't run.
test: #AssertFile & {
input: build.output
path: "test"
contents: "output\n"
}
// Each environment will have a specific deploy impleme...
Bumps sass from 1.49.7 to 1.49.8.
Release notes
Sourced from sass's releases.
Dart Sass 1.49.8
To install Sass 1.49.8, download one of the packages below and add it to your PATH, or see the Sass website for full installation instructions.
Changes
Fixed a bug where some plain CSS imports would not be emitted.
JS API
Fix a bug where inspecting the Sass module in the Node.js console crashed on Node 17.
Embedded Sass
Fix a bug where source map URLs we...
Bumps url-parse from 1.5.3 to 1.5.7.
Commits
8b3f5f2 1.5.7
ef45a13 [fix] Readd the empty userinfo to url.href (#226)
88df234 [doc] Add soft deprecation notice
78e9f2f [security] Fix nits
e6fa434 [security] Add credits for incorrect handling of userinfo vulnerability
4c9fa23 1.5.6
7b0b8a6 Merge pull request #223 from unshiftio/fix/at-sign-handling-in-userinfo
e4a5807 1.5.5
193b44b [minor] Simplify whitespace regex
319851b [fix] Remove CR, HT, and LF
...
Repro:
(umask 0000 && bats plan.bats)
Will work.
vs.
(umask 0077 && bats plan.bats)
Will not work (plan/outputs/files default permissions will fail)
Also, when the test fails, the generated file is not removed (different problem, that should be fixed regardless).
The tests clearly expect things to work as one would assume intuitively they would: โset the perm to exactly the one that was specified - and not the current behavior which is: set the per...
Described first in #1640 : the cleanup logic does not seem right currently (generated files are not destroyed when asserts are failing, leading to situations where things are broken and the user has to go manually cleanup leftovers).
This make things more resilient, by unconditionally removing possible leftovers before (is there a better way to do that with bats? like a per-test teardown that would always execute?)
PTAL @aluzzardi
Currently: https://github.com/dagger/dagger/blob/main/pkg/alpha.dagger.io/helpers.bash#L77 assumes that if ~/.kube/config exists, then there must be a running kube.
I was bitten by this (leftover file, no running kube, test suite failing in a very confusing way).
Is there a better way to verify that the cluster exist rather than this?
Maybe something as simple as:
k=$(grep "127.0.0.1" ~/.kube/config); curl "${k#*server: }"; echo $?
This fixes the broken test introduced by a breaking change in universe.dagger.io/docker, plus some minor improvements based on my PR review comments.
Hi :)
I don't know if it's an expected behavior but while developing a module on universe, I encountered a weird issue. I was trying to pass an environment variable MAIN_PLAYBOOK (see the attached screenshot) which was at the beginning properly evaluated during bash execution but after some modifications the variable was no longer evaluated.
I saw that the variable was actually passed in the exec command since when running echo "/home/ansible/$MAIN_PLAYBOOK" > /logs.txt, the right ...
Bumps cypress-localstorage-commands from 1.6.1 to 1.7.0.
Release notes
Sourced from cypress-localstorage-commands's releases.
Snapshots with name
Added
feat(#376): Support multiple snapshots allowing to define a name in save, restore and clear commands.
chore: Add command to check types. Run it in tests workflow
chore: Add eslint plugins
Removed
docs: Remove broken dependencies badge
Changed
chore: Remove NodeJs v15 from te...
By default the definition os.#Container cache the command by using always field.
// If true, the command is never cached.
// (false by default).
always: true | *false
This naming seems pretty confusing. It has been named this way to avoid confusion with cache mounts which are different.
We went for something literal aka โalways runโ.
We should rename it more explicitly. cmdCache for instance ?
A working prototype of dagger do --help which lists actions available in the current plan, and displays them with their associated comments:
โ do git:(dagger-do-help) โ ~/Code/dagger/dagger/cmd/dagger/dagger do --help ./actions.cue
2:02PM DBG system | loading plan args=[
"./actions.cue"
]
2:02PM DBG system | vendoring packages mod=/Users/richard/Code/dagger/dagger/tests
core-integration Run core integration tests
dagger-debug Build a debug vers...
Bumps github.com/Microsoft/go-winio from 0.5.1 to 0.5.2.
Release notes
Sourced from github.com/Microsoft/go-winio's releases.
v0.5.2
What's Changed
Allow guid package to be used on non-Windows GOOS targets by @โdcormier in microsoft/go-winio#169
Fix TestLookupEmptyNameFails by @โbitgestalt in microsoft/go-winio#225
Replace github.com/pkg/errors with stdlib errors by @โbitgestalt in microsoft/go-winio#227
Fix 'OpenVirtualDiskParameters' BOOL fields b...
Overview
This issue tracks final changes to our CUE APIs before the 0.2 release. This includes:
universe.dagger.iodagger.io
Realistically, Universe will not be finished or even fully stabilized, before the release. But we should try to pick as many low-hanging fruits as possible, to minimize future pain.
Changes
To propose a change, or ask about its status, please post a comment. If you're not sure if the change is relevant to this issue, post it anyway.
- It...
Allow Fullscreen for Get Started Video
Discussed in https://github.com/dagger/dagger/discussions/1546
I think we should add a very simple Github issue template. Therefore, we can gather minimum information if we need to reproduce bugs.
This PR aims to implement an universe module in order to run ansible playbooks with dagger.
There are two steps in order to run a playbook:
- Call the
#Ansiblestruct which installs ansible in the required version and load the specified config file - Call the
#Playbookstruct which runs the playbook with the specified options
Questions
- I didn't find another way to load a default config variable than storing it in a long string variable. Maybe storing it in a gist and cu...
Problem
In Dagger 0.1, sops is integrated for easy secrets management out of the box. But in 0.2 (Europa), that feature is removed in favor of a pluggable approach with batteries not included.
This is a good tradeoff, but it makes it harder for beginners to use sops with Dagger.
Solution
Add a guide to the documentation which explains how to integrate sops with Dagger, in a way roughly equivalent to the deprecated dagger secret feature in 0.1.
Tentative title: *Mana...
One of Dagger's flagship features is the ability to run on almost any CI environment. This includes Tekton, a popular option in the Kubernetes community. There should be a documentation page explaining how to run Dagger on Tekton, similar to the page explaining how to run it on Github Actions.
Overview
Upcoming release 0.2 Europa introduces a new docker package, which plays a central role in our developer experience: it is the recommended package for building, running and shipping Docker-compatible containers as part of a pipeline.
The developer experience is already good, but if we see opportunities to make it great, now is the best time to discuss them. After 0.2 and public launch, breaking changes will still be possible but they will be more costly, because there wi...
Problem
When running the standard integration tests for universe.dagger.io/yarn, on main, the test fails with a cryptic buildkit/docker error. As far as I know, this test is not failing in CI. There is no recent change to this package that seems relevant.
Steps to reproduce
1. Check current commit:
$ git show
commit 1135398beec4965f095ca70917908a06d804ff22 (HEAD -> main, origin/main, origin/HEAD)
Merge: 47a8bbc3 f6cee7f3
Author: Solomon Hykes
Date: Wed Fe...
In CUE commands, we can interact with local system to do operations like prompting for user input, file read/writes: https://cuetorials.com/patterns/scripts-and-tasks/
Example:
task: ask: cli.Ask({
prompt: "Approve it?"
repsonse: bool
})
That would be useful to orchestrate local workflow like environment promotion from dev to prod.
Currently action dependency are achieved via explicit data referencing. Sometimes this is too complicated to implement. For example, I just want to run kubectl wait before kubectl apply action. I can easily implement them in CUE, but composing outputs would increase learning curve and hurt developer experience.
It would be great to use cue tag to achieve that:
kubeWait: {...}
kubeApply: {...} @dependsOn(kubeWait)
This will be useful short term to workaround explicit dependencies and longer term for execution control.
Several different tests under https://github.com/dagger/dagger/tree/main/tests/plan/outputs/ ( listed in https://github.com/dagger/dagger/blob/main/tests/plan.bats ) are manipulating the same files in the same directories.
This seems to be causing random failures when tests are ran concurrently (bats --jobs 4).
This PR aims at fixing that (and also fixes triggers for the CI to run when bats files are being changed).
cc @aluzzardi @helderco
Bumps sass from 1.49.8 to 1.49.9.
Release notes
Sourced from sass's releases.
Dart Sass 1.49.9
To install Sass 1.49.9, download one of the packages below and add it to your PATH, or see the Sass website for full installation instructions.
Changes
Embedded Sass
Fixed a bug where the legacy API could crash when passed an empty importer list.
See the full changelog for changes in earlier releases.
Changelog
Sourced from sass's changelog.
1.49.9
Embe...
Bumps @docusaurus/core from 2.0.0-beta.15 to 2.0.0-beta.16.
Release notes
Sourced from @โdocusaurus/core's releases.
2.0.0-beta.16 (2022-02-25)
:rocket: New Feature
docusaurus-logger, docusaurus-module-type-aliases, docusaurus-plugin-debug, docusaurus-plugin-pwa, docusaurus-theme-classic, docusaurus-theme-search-algolia, docusaurus-types, docusaurus
#6243 feat(core): brand new swizzle CLI experience (@โJosh-Cena)
cr...
Bumps prismjs from 1.25.0 to 1.27.0.
Release notes
Sourced from prismjs's releases.
v1.27.0
Release 1.27.0
v1.26.0
Release 1.26.0
Changelog
Sourced from prismjs's changelog.
1.27.0 (2022-02-17)
New components
UO Razor Script (#3309) 3f8cc5a0
Updated components
AutoIt
Allow hyphen in directive (#3308) bcb2e2c8
EditorConfig
Change alias of section from keyword to selector (#3305) e46501b9
Ini
Swap out header for section (#3304) deb3a97f
Mong...
Bumps url-parse from 1.5.3 to 1.5.10.
Commits
8cd4c6c 1.5.10
ce7a01f [fix] Improve handling of empty port
0071490 [doc] Update JSDoc comment
a7044e3 [minor] Use more descriptive variable name
d547792 [security] Add credits for CVE-2022-0691
ad23357 1.5.9
0e3fb54 [fix] Strip all control characters from the beginning of the URL
61864a8 [security] Add credits for CVE-2022-0686
bb0104d 1.5.8
d5c6479 [fix] Handle the case where the port is specified but ...
Closes #1597
This is still a work in progress, I'm just sharing current progress to ease early review. I'll squash these commits before ready to merge.
Outstading tasks:
- [ ] Fix error when reading a
dagger.#FS - [ ] Sockets
- [ ] Platform
- [ ] Tests
- [ ] Cleanup
- [ ] Documentation
Outstanding questions:
- [ ] Read include/exclude patterns?
- [ ] Write permissions when using
dagger.#FS? - [ ] Read/write conflict
- [ ] No more trimSpace in secrets, is it an issue...
Bumps github.com/moby/buildkit from 0.10.0-rc1 to 0.10.0-rc2.
Release notes
Sourced from github.com/moby/buildkit's releases.
v0.10.0-rc2
Welcome to the 0.10.0-rc2 release of buildkit!
This is a pre-release of buildkit
Please try out the release binaries and report any issues at
https://github.com/moby/buildkit/issues.
Notable changes
Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache i...
What is the problem?
Running yarn.#Build & yarn.#Run in parallel always results in yarn package corruption:
5:13PM INF actions.test.container._exec | computing
5:13PM INF actions.build.container._exec | computing
5:13PM INF actions.build.container._exec | #9 1.958 yarn install v1.22.17
5:13PM INF actions.build.container._exec | #9 2.751 [1/4] Resolving packages...
5:13PM INF actions.build.container._exec | #9 6.398 [2/4] Fetching packages...
5:13PM INF actions.build.c...
Bumps cypress from 9.5.0 to 9.5.1.
Release notes
Sourced from cypress's releases.
9.5.1
Released 2/28/2022
Bugfixes:
Added a ready event to the Plugins process to ensure all plugins have
successfully started before the configuration is loaded and the tests start.
Fixed in #19792.
This fixes an issue where the plugin is never executed and leaves Cypress
stuck in the initialization phase, where the user can only view the loading
spinner.
This enables...
The default ignore gets applied and then we have a similar behaviour to
what https://github.com/dagger/dagger/pull/1591 was attempting to fix.
We hope that this will fix it for good.
If multiple yarn.#Run commands run in parallel, they will corrupt each other's yarn cache mount. Because we extract yarn install into a separate step, LLB will dedup the yarn install step and only run it once, regardless how many yarn.#Run commands run in parallel.
Our logger wasn't entirely compatible with Windows escape sequences when not in TTY mode. This fix changes the stderr to a Windows compatible fd that automatically takes that into account.
Signed-off-by: guillaume
Bumps github.com/containerd/containerd from 1.6.0 to 1.6.1.
Release notes
Sourced from github.com/containerd/containerd's releases.
containerd 1.6.1
Welcome to the v1.6.1 release of containerd!
The first patch release for containerd 1.6 includes a fix for
CVE-2022-23648
and other issues.
Notable Updates
Use fs.RootPath when mounting volumes (GHSA-crp2-qrr5-8pq7)
Return init pid when clean dead shim in runc.v1/v2 shims (#6572)
See the changelog f...
This pull request ensures that we are collected anonymized telemetry from dagger command runs
We started writing a new doc page - From local dev to CI/CD platform (action item for self: add link to PR) - and would like to know which CI/CD platform you would find most useful for us to document next.
Since everyone that is using Dagger today already has a GitHub account (one of the early access requirements), starting with GitHub Actions seemed like a most reasonable first step.
Drop a comment below with the CI/CD platform that you use. We intend to document the one with the m...
We invite users to help us prioritise via https://github.com/dagger/dagger/discussions/1677
So that it's easy for anyone to jump to the new docs that we are currently working on, and intend to replace the existing docs with.
While I would have preferred to link to the local dev page, it's still stuck in the PR state, currently blocked on another PR: https://github.com/dagger/dagger/pull/1586
Also added a link to the pre-Europa docs, so that it's easy to go back.
While at it, drop "Sidebar" from the name of sidebars, and replace tutorial with a more descriptive name.
Part...
This is meant to act as catch-all for pages that have been written ad-hoc, but don't currently fit anywhere.
While it was initially mentioned that this should be called Knowledgebase, this name didn't seem to accurately describe the intent. The email & message default DRAFTS seemed better, so I just went with it.
At Epitech, there was courses on Jenkins, I'm not sure it's relevant but some infrastructures are still using it so maybe we can do something with Dagger
Signed-off-by: Helder Correia
Why
Secret inputs are going to be replaced by the new Client API in #1668, and with it the common trimSpace option.
What
Added new task to trim leading and trailing space characters from any secret, no matter how they were loaded.
Bumps @docusaurus/core from 2.0.0-beta.15 to 2.0.0-beta.17.
Release notes
Sourced from @โdocusaurus/core's releases.
2.0.0-beta.17 (2022-03-03)
:rocket: New Feature
docusaurus-plugin-content-blog, docusaurus-theme-classic
#6783 feat: allow blog authors email (@โJosh-Cena)
:boom: Breaking Change
docusaurus-theme-classic, docusaurus-theme-common
#6771 refactor(theme-classic): replace color mode toggle with button; ...
Just the first step, to be continued first thing tomorrow.
Part of #1327.
Dagger orb (CircleCI's lego brick) here: https://github.com/logicalhq/dagger-orb
Not yet listed as a public orb in the Orb Registry: https://circleci.com/developer/orbs
Comments from discord #dev:dagger project thread:
@shykes says: dagger project init is basically a wrapper for cue mod init. Perhaps it has options to add templates, scaffolding, whatever. All optional features we can add later.
As a basic start, dagger project init will mimic behavior of cue mod init
- create a
cue.modfolder - fill module name with
[module]argument
GitHub knows which are the big CI tools on their platform (or competitive) and has transition guides from each one to GitHub Actions. So Azure Pipelines (from Azure DevOps), CircleCI, TravisCI, Jenkins, and GitLab CI/CD all make sense.
https://docs.github.com/en/actions/migrating-to-github-actions
Closes #1684
Signed-off-by: Richard Jones
[dagger/dagger] Pull request opened: #1687 Temporarily remove Use Cases from the Europa docs sidebar
The current plan is to add them post 0.2.0 shipping, for now the focus
is on Getting Started & Core Concepts.
Part of #1327.
When dagger runs for the first time against a local Docker Engine, it shows the following WARN messages:
WARN[0002] commandConn.CloseWrite: commandconn: failed to wait: signal: terminated
WARN[0002] commandConn.CloseRead: commandconn: failed to wait: signal: terminated
WARN[0002] commandConn.CloseWrite: commandconn: failed to wait: signal: terminated
This is a bad first experience, and we should improve our first impression.
Point-in-time capture of the latest `dagger...
This PR changes the behavior of dagger do to use a --plan flag to denote the location of the plan and deprecates the global flag --project
Signed-off-by: Richard Jones
There is no if statement now and fewer TODO comments.
The initial intent was to have the simplest build, test & deploy loop, and this is a step in that direction.
The chances of 8080 being in use and the command failing due to this port clash are higher than if we pick a less common port by default.
We are still figuring out caching, specifically Buildkit go client caching, while packages is something that users can start creating right away.
And by extension, Jenkins X that uses Tekton inside.
TeamCity by JetBrains. Could create a Kotlin script wrapper for Dagger.
This is a catch-all for now, will refine first thing tomorrow.
Add dev & ghPages pipelines to todoapp example directly. Keeping everything in a single repository makes it a lot easier to work with it all. GitHub Pages is temporary, we want to deploy to Netlify from GitHub Actions. We are not using the yarn package so that we can revert https://github.com/dagger/dagger/pull/1673 without breaking this implementation. We should also modify the docs to use the example app from this location, i...
There is a separate repository holding examples of Dagger configurations: https://github.com/dagger/examples. This repository is consistently out of date, because it holds several examples, its ownership is unclear, and its upstream dependencies are moving fast which requires constant updates.
At this stage it's better to move the examples we care about into the main dagger repository (this one), and remove the external dagger/examples repository (after creating a secure backup of its cont...
dagger project update is now used to vendor the following:
cue.mod/pkg/dagger.iocue.mod/pkg/universe.dagger.io
dagger project update [package] no replaces dagger mod get
Signed-off-by: Richard Jones
[dagger/dagger] Issue opened: #1699 \`dagger do\`: suggest \`dagger update\` on missing dependencies
When running dagger do fails because of missing dependencies, the user should be presented with a clear error message that suggests that perhaps there are missing dependencies, and perhaps running dagger update would solve the problem.
Move cache flags to 'dagger do' as it will only become relevant for this dagger command
Signed-off-by: guillaume
These were replaced by the new Client API, and shouldn't be used after 0.2.beta1 as to avoid confusion.
This is a follow up from https://github.com/dagger/dagger/pull/1668#discussion_r819527346.
client: env: [string] is the only task using path based task lookup now, but since all environment variables are always loaded by the runtime, it's not worth it to have a path based matcher just so each variable is a different task.
We should have just one client: env task that returns at once every referenced variable.
Ref https://github.com/h8r-dev/cuelib/issues/2
We have used a couple of cue APIs which we found very useful in our projects.
I'm creating this issue to list them. Would like to know workaround and solutions to support them after migrating to Europa.
dagger
- dagger.#Input/Output
- dagger.#Secret
- dagger.#Artifact
op
- op.#Load
- op.#Exec
- op.#Export
- op.#FetchContainer
- op.#Mkdir
- op.#WriteFile
- op.#Subdir
kubernetes
- kubernetes.#Kubectl
- kubernetes.#Res...
After #1668, the old inputs/outputs need to be removed to avoid confusion over.
This is a work in progress.
Signed-off-by: Helder Correia
What would you rather have?
- Keep a top level
paramsin the plan as a convention; - Just put your params anywhere you like, inside
actions.
Option 1
#Plan: {
// Receive runtime parameters
- inputs: params: [name=string]: _
+ params: [name=string]: _
actions: { ... }
}
dagger.#Plan & {
params: app: string
actions: { ... }
}
Option 2
#Plan: {
// Receive runtime parameters
- inputs: params: [name=string]: _
ac...
Signed-off-by: Joel Longtine
Signed-off-by: Helder Correia
Problem
Currently, dagger do (and its 0.1 predecessor dagger up) does not check that a plan is valid before running it. It delegates this task to the CUE loader, which in turn requires the developer to explicitly apply a top-level definition, like so:
import (
"dagger.io/dagger"
)
dagger.#Plan & {
// Plan configuration goes here
...
}
This is valid CUE, and we should support it. But it does not solve several problems caused by the absence of an explicit...
Bumps nanoid from 3.1.9 to 3.1.31.
Changelog
Sourced from nanoid's changelog.
3.1.31
Fixed collision vulnerability on object in size (by Artyom Arutyunyan).
3.1.30
Reduced size for project with brotli compression (by Anton Khlynovskiy).
3.1.29
Reduced npm package size.
3.1.28
Reduced npm package size.
3.1.27
Cleaned dependencies from development tools.
3.1.26
Improved performance (by Eitan Har-Shoshanim).
Reduced npm package size.
3.1.25
Fixed br...
Bumps url-parse from 1.5.3 to 1.5.10.
Commits
8cd4c6c 1.5.10
ce7a01f [fix] Improve handling of empty port
0071490 [doc] Update JSDoc comment
a7044e3 [minor] Use more descriptive variable name
d547792 [security] Add credits for CVE-2022-0691
ad23357 1.5.9
0e3fb54 [fix] Strip all control characters from the beginning of the URL
61864a8 [security] Add credits for CVE-2022-0686
bb0104d 1.5.8
d5c6479 [fix] Handle the case where the port is specified but ...
Bumps follow-redirects from 1.13.0 to 1.14.9.
Commits
13136e9 Release version 1.14.9 of the npm package.
2ec9b0b Keep headers when upgrading from HTTP to HTTPS.
5fc74dd Reduce nesting.
3d81dc3 Release version 1.14.8 of the npm package.
62e546a Drop confidential headers across schemes.
2ede36d Release version 1.14.7 of the npm package.
8b347cb Drop Cookie header across domains.
6f5029a Release version 1.14.6 of the npm package.
af706be ...
Flagging this for post-europa.
CUE snippets used to be external to markdown files, like so:
```cue file=./tests/cloudformation/template.cue title="todoapp/cloudformation/template.cue"
``
This allows:
- CUE linter cleaning up the files / validating them
- Adding automated tests using the snippets, to make sure the docs work
I think I've seen a couple of embedded CUE snippets in the latest Europa markdowns (/cc @gerhard @helderco)
Context
Currently, dagger project init only create an empty cue.mod.
I think it would be great to also generate template file to facilitate initialisation.
V1.0 - Simple initialisation
To begin with, dagger project init could generate a simple main.cue file with a Hello World
// main.cue
package main
import (
"dagger.io/dagger"
"universe.dagger.io/bash"
"universe.dagger.io/alpine"
)
dagger.#Plan & {
actions: {
// Pull an...
The first thing I see when visiting our github repo are scary-looking security errors. We should clean those up before opening the repo; and ideally before the next release since it will bring a lot of repeat visitors.
The file is entirely commented and I keep getting e-mails from GH on
each PR complaining about ".github/workflows/test-docs.yml: No jobs were
run"
Deprecate dagger up + hid it from the help message.
Preview:
Working on https://github.com/dagger/dagger/pull/1589, I had an old version of the dagger.io library which triggered a weird cue error on core types.
After running dagger project update, it fixed my issue.
The issue is that nobody will know when to run it and why. Ideally we want version checking of the library to automatically warn users when the library is outdated and that they should run project update.
Until we implement a proper version checking, I would automatically run...
dagger up has been replaced by dagger do (and deprecated by #1720).
We need to remove it altogether which requires migrating some tests.
Assigning to you @talentedmrjones since you're the owner of do
/cc @samalba who reported the issue
Signed-off-by: Andrea Luzzardi
- Move chan sync code immediately after operation
- Fix git hashing when not in a git repo
I met this Error when I run my plan by executing dagger up without europa.
The output is:
panic: runtime error: slice bounds out of range [1:0]
goroutine 2752 [running]:
go.dagger.io/dagger/cmd/dagger/logger.(*TTYOutput).printGroup(0xc00014e190, 0xc01083ed20, 0x79, 0xffffffffffffffff)
/home/dagger/cmd/dagger/logger/tty.go:321 +0x60f
go.dagger.io/dagger/cmd/dagger/logger.(*TTYOutput).print(0xc00014e190)
/home/dagger/cmd/dagger/logger/tty.go:203 +0x4a5
go.dagger.io/dagger/c...
Reading all environment variables at once in client: env makes it possible to abandon the new path based task lookup introduced in #1668. This was the only remaining task using it.
Suggested by @aluzzardi
Signed-off-by: Helder Correia
The starting point is a managed K8s
- an alternative to GKE plain would be welcome
- use the K8s cluster for running the app(s), managed services otherwise
gcr.io for container images so that github packages can be used for JAR packages
Ideally, GitHub Actions will be enough to seed everything else
One pipeline to build & publish all the artefacts for a single service (app)
- use the branch name for the artefact version
- https://github.com/salaboy/from-monolith-to-k8s/blob/mas...
Signed-off-by: Vasek - Tom C
We want to be able to download the latest published version for a specific Major.Minor. In the Dagger GitHub Action, we want to be able to restrict the versions to e.g. latest 0.2.
cc @crazy-max @grouville
We assume that this will be out by then.
The latest version is v1 (v1.2.1 specifically). Since we cannot undo the versioning, and go back to v0.2, we capture the breaking change in with properties, and the new version default value (0.2/latest after https://github.com/dagger/dagger/pull/1733) and go to the next version that we can: v2
cc @crazy-max @grouville
Currently tests define many different action fields for running and verifying test results. Further, most if not all tests were written before dagger do and thus need multiple verification actions to be grouped under a single action.
I propose that for testing we standardize on actions: test: as a common field name. For a single operation this allows the command line to be dagger do test ๐
For tests that require multiple actions, they can still be grouped under test: in which...
Currently the paths listed as fields in client: filesystem: are relative to cwd which means you must execute dagger do in the right directory. This will certainly lead to confusion and frustration when developers either are not aware of this requirement, or perhaps even more frustating when a plan specifies client filesystem paths in multiple cue files across subfolders.
When the client: filesystem: paths are relative to the cue file they are defined in, it frees the developer to ...
Also refactored tests to use dagger do
Signed-off-by: Richard Jones
Bumps github.com/moby/buildkit from 0.10.0-rc2 to 0.10.0.
Commits
068cf68 Merge pull request #2684 from crazy-max/fix-buildinfo-deps-sources
49aa39c buildinfo: merge build sources for deps
bffd728 Merge pull request #2712 from ktock/fixcachecleanup
c7f2e40 Apply ensurePruneAll to other tests as well
bbf149b Merge pull request #2710 from sipsma/fix-diffapply-overwrite
53722cc diffapply: make overwritten dirs opaque.
fdecd0a Merge pull request #2708 from si...
There are a few outstanding tasks, but they can be finished tomorrow.
This is just the beginning of many refinements, so it's all good ๐
Part of #1327
In `dagger project {init, update}, something like this (needs testing -- haven't tried it):
pkg/** linguist-vendored
Should inform that pkg contains vendored code. Helps keeping PRs at bay.
/cc @talentedmrjones @shykes @crazy-max
jeremyadams@Jeremys-MBP-2 dagger % cmd/dagger/dagger
A programmable deployment system
Usage:
dagger [command]
Available Commands:
do Execute a dagger action.
help Help about any command
project Manage a Dagger project
version Print dagger version
Signed-off-by: Jeremy Adams
Fixes #1626
Signed-off-by: Helder Correia
Related to #1715, this moves contributed docs from #1668 outside of markdown to allow linting.
There's still tests missing, to be added later. We need a new bats in docs/tests.
Even though most of these snippets are incomplete to keep them shorter, the multi-stage build in Building container has been tested manually with attached data files.
Signed-off-by: Helder Correia
So that we get auto-formatting and syntax checking in our code editor. The only snippets which have not been extracted are either terminal output, or file fragments (e.g. CUE) which are not valid standalone files.
Resolves https://github.com/dagger/dagger/issues/1715
While at it, do a few fly-by improvements:
- beta.1 -> beta.2
- add CUE & BuildKit links
- up -> do
If someone tries to run dagger do with a pre-Europa configuration, we should catch that and let them know it isn't compatible any longer.
Bumps github.com/docker/buildx from 0.6.2 to 0.8.0.
Release notes
Sourced from github.com/docker/buildx's releases.
v0.8.0
Welcome to the 0.8.0 release of buildx!
Please try out the release binaries and report any issues at
https://github.com/docker/buildx/issues.
Notable Changes
Build command now accepts --build-context flag to define additional named build contextes for your builds. For example, these can be used expose additional local directories to...
Bumps google.golang.org/grpc from 1.44.0 to 1.45.0.
Release notes
Sourced from google.golang.org/grpc's releases.
Release 1.45.0
Bug Fixes
xds/clusterresolver: pass cluster name to DNS child policy to be used in creds handshake (#5119)
reflection: support dynamic messages (#5180)
Special Thanks: @โcodebutler
Performance Improvements
wrr: improve randomWRR performance (#5067)
Special Thanks: @โhuangchong94
Behavior Changes
server: convert context...
Bumps github.com/spf13/cobra from 1.3.0 to 1.4.0.
Release notes
Sourced from github.com/spf13/cobra's releases.
v1.4.0
Winter 2022 Release โ๏ธ
Another season, another release!
Goodbye viper! ๐ ๐
The core Cobra library no longer requires Viper and all of its indirect dependencies. This means that Cobra's dependency tree has been drastically thinned! The Viper dependency was included because of the cobra CLI generation tool. This tool has migrated to spf13/cob...
Signed-off-by: Andrea Luzzardi
Also replace all occurences of Cue with CUE.
This is a high priority item. We want to help users migrating their plan from 0.1 to 0.2. The first step is to give them a clear entry point. This should be a page in our documentation. We can iterate on this page over time, to include more information, redirect them to the right discussion thread, issue or discord channel, etc. But the page needs to exist first.
Closes #1747
Closes #1745
Signed-off-by: Richard Jones
@gerhard a couple comments and put them in a google Doc, because it's easier to comment there on specific parts of text.
https://docs.google.com/document/d/1XolFtIzUMQQNOE6NNNO7JDmxECbj1uwjCcXv1SWVfeo/edit
It's linguist-generated=true per the documentation.
Signed-off-by: Andrea Luzzardi
When calling dagger do --help against an invalid plan, the error "failed to load plan" is printed, without details. This makes it difficult to fix the issue.
Label PRs as stale after 14 days, close them 7 days after.
Problem
Dagger Universe includes an integration test suite that requires external infrastructure to run. This infrastructure is maintained by Dagger employees, and for security and legal reasons, non-employees cannot access it. Yet, it is managed using the same open-source repository. This involves the use of sops-encrypted credentials, and automation scripts which are useless to non-employees.
Solution
Move all content related to Dagger's proprietary test infrastructure, out o...
Signed-off-by: Erik Sipsma
This has some tests, but I also tested it in my WIP branch exploring apko integration. See the commit message on that apko branch for more details on the status there: https://github.com/sipsma/dagger/commit/f9f0687c9a2a0944a9941c9e5d562dfe1267829c
In preparation of releasing 0.2.0, we need to change the auto-releaser to bump patch versions rather than prerel.
Closes #1699
Signed-off-by: Richard Jones
We are getting ready to release dagger 0.2, codename Europa. This release has one focus: improve the developer experience. It features a redesigned CLI and powerful new APIs. The result is a completely new way to develop CI/CD pipelines: more productive, more portableโฆ and way more fun!
However, there is a downside: unfortunately, this release is not backwards compatible: plans written for dagger 0.1 will not run on version 0.2. Automatic migration is not supported, but manual migratio...
- Temporary page that just points to the GitHub Discussion
- Goal: create a stable link we can share, update with proper content
later - Wasn't sure how to categorize this so I've renamed
DRAFTSinto
Knowledge Base
Doc tested and written with rc1 version, on Windows
Signed-off-by: guillaume
make integration (and probably others) do call yarn test, which in turns invoke bats.
It seems that yarn leaves processes behind (bats and dagger), causing (confusing) issues for developers.
Suggestion: if there is no important reason for it, just remove the yarn layer of delegation and call bats directly instead.
cc @grouville @aluzzardi
Different versions of parallel may not be working / compatible with bats.
Ideas to work-around it:
- sanity test parallel version and fallback to
--jobs 1if it does not fit the bill - make the number of parallelized jobs easy to control (environment) so that one can disable parallel on integration tests if need be
cc @grouville @aluzzardi
Problem
When listing available actions with dagger do --help, the formatting is messy.
- Multi-line comments are printed as is
- No "Available Options" section header
- Misaligned descriptions
- Easy to miss actions at the end
Example, running from the root of dagger repo:
$ dagger version
dagger devel (7b4b5915) linux/amd64
$ dagger do --help
Execute a dagger action.
Usage:
dagger do [OPTIONS] ACTION [SUBACTION...] [flags]
Flags:
--cach...
Problem
When running dagger do -p /PATH/TO/PLAN:
- If the workdir is inside the plan's project, loading succeeds
- If the workdir is outside the plan's project, loading fails
This is confusing because Dagger has all the information needed to load the plan without an error, but returns an error instead.
Solution
When workdir is outside the plan's project, configure the cue loader to use the project directory as its "loader working directory", instead of the curr...
Added a few files to test the affected example.
Signed-off-by: Helder Correia
[dagger/dagger] Issue opened: #1781 client: filesystem: write doesn't work with string interpolation
Context
@Alvise88 is porting his config to Dagger Europa.
His use case is: read a file from the host and modify it using dagger.
Issue
The first implementation doesn't seem to work because no action is being referenced. That way, dagger do doesn't seem to run.
package main
import (
"dagger.io/dagger"
)
dagger.#Plan & {
client: filesystem: "test.txt": {
read: contents: string
write: contents: read.contents
}
}
But, it's not the subject of this issu...
Jenkins - most enterprises use it.. multi man-decades of tooling are around it.
- Only print first line of comment
- Add "Available Options" section header
Fixes #1777
Signed-off-by: Andrea Luzzardi
Thanks for catching this one. I am deleting my WIP now.
<img width="953" alt="image" src="https://user-images.githubusercontent.com/3342/157941080-fd1c55b7-3071-4501-af50-147351120a3d.png">
I need this before I can really build anything useful.
There should be a documentation page talking about how Dagger works under the hood. An architecture diagram, core technology, etc.
For example this diagram is from https://sigstore.dev/how-it-works:
- Remove universe.dagger.io/cue.mod from
.gitignore - Re-add symlink
- Remove test-docs.yml workflow
With no buildkitd already started:
time /home/dmp/debug-ga/actions-runner/_work/dagger/dagger-0.2.0/cmd/dagger/dagger-debug do -p ./tasks/pull/pull.cue pull
... starts buildkitd, but doesn't wait for it to be ready, ending with:
1:04PM FTL unable to create client: buildkit failed to respond
real 0m12.964s
user 0m0.436s
sys 0m0.212s
For some reason, there is a (very impactful) performance issue on one of my machines.
What I'm doing on that machine:
time dagger do -p ./tasks/pull/pull.cue pull
Sometimes takes about 45+ seconds - sometimes just hangs there for minutes (and never finishes - meanwhile, buildkitd does give up).
I'm not sure what / where to look at.
Some relevant details:
Hardware: nuc (Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz), 16GB Ram
Kernel: 4.9.0-9-amd64
OS: Debi...
export DAGGER_BINARY="$(pwd)/../cmd/dagger/dagger-debug"
while true; do
time $DAGGER_BINARY do -p ./tasks/pull/pull.cue pull
done
Will report data race about 1 in 10 times.
==================
WARNING: DATA RACE
Write at 0x00c0004d83c0 by goroutine 78:
go.dagger.io/dagger/cmd/dagger/logger.(*Logs).Add()
/home/dmp/debug-ga/actions-runner/_work/dagger/dagger-0.2.0/cmd/dagger/logger/tty.go:68 +0xba7
go.dagger.io/dagger/cmd/dagger/logger.(*TTYOutput).Writ...
Problem
Some actions don't require any core tasks
Minor issue I spotted while reading the docs for the first time, apologies for the nitpick!
The reference to docker.Copy does not exist and should be docker.#Copy instead. This also leads to a confusing error message (possibly related to #493) as it isn't found while CUE is compiled, and instead results in the following runtime error:
[โ] actions.deps ...
Expected Behavior
When a user is not authenticated to github and click on the latest newsletter links (ex: https://docs.dagger.io/1202/plan/?utm_source=Dagger+Early+Access&utm_campaign=c3541b50ed-EMAIL_CAMPAIGN_2021_08_17_04_28_COPY_01&utm_medium=email&utm_term=0_05e3a05bcc-c3541b50ed-453229090), he should be redirected to docs authentication screen.
 windows/amd64
I'm attempting to initialize an existing repo with dagger. dagger project init appear to hang. ctrl+x doesn't terminate the process.
I tried cleaning up docker resources. It still hangs.
docker stop dagger-buildkitd && docker rm dagger-buildkitd && docker volume rm dagger-buildkitd
I do not have problems with the todoapp example, but the behavior seems to occur on any of our projects.
Perhaps a naive question. Given 1) the difficulties encountered with this cache export feature, 2) this is one of the easy ones (users will want to export cache to many various targets, most of which are not supported by buildkit at all)...
Perhaps we should start discussing an alternative design that is more sustainable long term (while we continue working on fixing this).
My observations:
- Buildkit local exporter is very mature, and works everywhere.
- The local exporter produces mostly...
We now have a bot auto-closing PRs, which is useful but can cause a bad experience for contributors that never get a human response, then their PR is just closed with no further feedback or discussion.
Removing the bot won't solve this problem. Instead, here's one incremental fix that could help: make it a priority to have every pull requests assigned to someone.
- First pass: pull requests are assigned by humans (eg. @aluzzardi or other appointed maintainers do a first pass at triage. It d...
This duplicates the existing page and uses a new ID so that clicking on it in one sidebar (pre-Europa) will not update the sidebar. We have
learned about this behaviour via https://github.com/dagger/dagger/pull/1758
Also links to the source of truth cue files, in the dagger/dagger repo.
While at it, move this page from Knowledge Base, which is a generic category, to Core Concepts. The URL didn't change so that we don't break existing links. FWIW /1213/dagger-cue would have been my choice.
Initially contributed as a change part of https://github.com/dagger/dagger/pull/1758
Should be merged after https://github.com/dagger/dagger/pull/1799
Related to https://github.com/dagger/dagger/pull/1768
When building images with dagger, I get a warning when running them:
WARNING: The requested image's platform (unknown) does not match the detected host platform (linux/amd64) and no specific platform was requested
It still runs though. I'm using Docker Desktop on macOS.
Example inspect from a Dockerfile build in dagger:
[
{
"Comment": "buildkit.exporter.image.v0",
"Architecture": "",
"Os": "linux",
}
]
This is the same...
Are there any plans to implement the same for bitbucket pipelines?
Having spent a significant amount of my last few weeks on https://github.com/dagger/dagger/issues/1327, I thought that it would help to share my learnings, current state and the steps that I see us taking next.
While this is the continuity that I imagine, now is a good time to share your perspectives and drive the improvements that you would like to see in https://docs.dagger.io as we open up Dagger to the world.
Learnings worth sharing
It served me well to have a clear user in mi...
Changes
Add an example of usage to :
- Test a go project
- Build a binary
- Build and push project as docker image
In v0.1 , you can list all the fields that needs to input with type and description:
$ dagger input list
Name Type Description
token string Token to access Github
But in v0.2, you can not list the inputs in client:
client: {
filesystem: {
".": read: {...}
build: write: contents: actions.build.contents.output
}
env: {
APP_NAME: string
NETLIFY_TEAM: string
NETLIFY_TOKEN: dagger.#Secret
}
}
Here, th...
Bumps cypress from 9.5.1 to 9.5.2.
Release notes
Sourced from cypress's releases.
9.5.2
Released 3/14/2022
Bugfixes:
Fixed an issue with .type() where click events could be fired on the
incorrect target element because the target focus changed within a key-down
event handler callback. Fixed in
#20525.
Fixed a regression in 9.5.0 where ANSI
colors were not removed from the FireFox warning message about the
chromeWebSecurity configuration option havin...
This would interest me a lot, and help when it comes to pushing dagger at work!
Fixes #1555
Signed-off-by: Helder Correia
Bumps go.opentelemetry.io/otel/exporters/jaeger from 1.4.1 to 1.5.0.
Release notes
Sourced from go.opentelemetry.io/otel/exporters/jaeger's releases.
Release v1.5.0
Added
Log the Exporters configuration in the TracerProviders message. (#2578)
Added support to configure the span limits with environment variables.
The following environment variables are supported. (#2606, #2637)
OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT
OTEL_SPAN_ATTRIBU...
Bumps go.opentelemetry.io/otel/trace from 1.4.1 to 1.5.0.
Release notes
Sourced from go.opentelemetry.io/otel/trace's releases.
Release v1.5.0
Added
Log the Exporters configuration in the TracerProviders message. (#2578)
Added support to configure the span limits with environment variables.
The following environment variables are supported. (#2606, #2637)
OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT
OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT
OTEL_SP...
Bumps go.opentelemetry.io/otel from 1.4.1 to 1.5.0.
Release notes
Sourced from go.opentelemetry.io/otel's releases.
Release v1.5.0
Added
Log the Exporters configuration in the TracerProviders message. (#2578)
Added support to configure the span limits with environment variables.
The following environment variables are supported. (#2606, #2637)
OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT
OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT
OTEL_SPAN_EVENT_COU...
Bumps github.com/stretchr/testify from 1.7.0 to 1.7.1.
Commits
083ff1c Fixed didPanic to now detect panic(nil).
1e36bfe Use cross Go version compatible build tag syntax
e798dc2 Add docs on 1.17 build tags
83198c2 assert: guard CanConvert call in backward compatible wrapper
087b655 assert: allow comparing time.Time
7bcf74e fix msgAndArgs forwarding
c29de71 add tests for correct msgAndArgs forwarding
f87e2b2 Update builds
ab6dc32 fix linting errors in /a...
Bumps go.opentelemetry.io/otel/sdk from 1.4.1 to 1.5.0.
Release notes
Sourced from go.opentelemetry.io/otel/sdk's releases.
Release v1.5.0
Added
Log the Exporters configuration in the TracerProviders message. (#2578)
Added support to configure the span limits with environment variables.
The following environment variables are supported. (#2606, #2637)
OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT
OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT
OTEL_SPAN_E...
Bumps @mdx-js/react from 1.6.22 to 2.1.0.
Release notes
Sourced from @โmdx-js/react's releases.
2.1.0
Core
aff6de4f minor add support for passing options to remark-rehype
by @โstefanprobst in mdx-js/mdx#1935
5d4355e4 patch replace got w/ node-fetch
by @โwooorm in mdx-js/mdx#1978
656a4ae5 patch remove use of URL from url
by @โzfben in mdx-js/mdx#1976
3f739a34 patch add missing dependency
by @โbensmithett in mdx-js/mdx#1936
Site
288...
This is the first thing that someone new to Dagger tried to do. I wanted to capture it in an example so that we can reference in the future.
I am sure that we can improve on this, it's just a small in a direction that I have seen a new member take.
Signed-off-by: Joel Longtine
I wanted to try different versions of dagger on Linux, and this was the easiest way of doing that:
curl -L https://dl.dagger.io/dagger/install.sh -O
# make this change, and then:
DAGGER_VERSION=0.2.0 sh install.sh
# ...
./bin/dagger version
dagger 0.2.0 (e499297e) linux/amd64
DAGGER_VERSION=0.2.1 sh install.sh
# ...
./bin/dagger version
dagger 0.2.1 (69b4845d) linux/amd64
This change enables anyone to do the following inst...
Part of https://github.com/dagger/dagger/issues/1796, we have learned with @nabsul that on Windows 11, following the Windows tutorial from https://docs.dagger.io/, dagger do build fails with:

I tried to reproduce this on Linux and I am not getting the above failure:
root@h11:/tmp# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Descripti...
Signed-off-by: Andrea Luzzardi
This is WIP, I just wanted to capture a thought that keeps coming back
I think that our most active & trusted collaborators (this applies to everyone from the community) could benefit from auto-merging pull requests. I am bringing this up again because I think it deserves us to think about.
I imagine a future world where collaborators can earn our trust and get the privilege to auto-merge after n days.
If all checks pass, and there are no objections from anyone else (thi...
Here is my full CUE code
package main
import (
"dagger.io/dagger"
"universe.dagger.io/bash"
"universe.dagger.io/docker"
"universe.dagger.io/alpine"
)
#Kubectl: {
version: string | *"v1.23.5"
image: docker.#Build & {
steps: [
alpine.#Build & {
packages: {
bash: {}
curl: {}
}
},
bash.#Run & {
workdir: "/src"
script: contents: #"""
curl -LO https://dl.k8s.io/release/\#(version)/bin/linux/amd64/kubectl
chmo...
Bumps github.com/docker/buildx from 0.8.0 to 0.8.1.
Release notes
Sourced from github.com/docker/buildx's releases.
v0.8.1
Notable changes
Fix possible panic on handling build context scanning errors #1005
Allow . on compose target names in buildx bake for backward compatibility #1018
Commits
5fac64c Merge pull request #1018 from tonistiigi/v0.8-compose-target-dot
24ad37a bake: allow dot in target names for compose
1066518 Merge pull request #1005 f...
Problem
All core actions are currently available under dagger.io/dagger. For example dagger.#Exec, dagger.#Pull etc.
From observing new developers, I am noticing that they tend to go to these core actions first. This makes sense because dagger.io/dagger is the primary entrypoint for the Dagger API. However, often these core actions are not what developers should be using: they are relatively low-level, and a higher-level package might be a better fit in most cases.
Examp...
Golint is replaced with revive since the former has been archived and
replaced with the latter.
Signed-off-by: Marcos Lilljedahl
Resolves #1802
Signed-off-by: Vasek - Tom C
The dagger-buildkitd container persists on restart and consumes resources, even when I'm not using dagger.
Could the restart policy be set to no to conserve resources?
While I was using the search engine, I've been redirected to a page in the 0.1 documentation. This is very confusing.
Maybe we could display a banner :
This is documentation for Dagger 0.1, which is no longer actively maintained.
For up-to-date documentation, see the latest version (0.2).
what do you think @gerhard ?
When using yarn with the alpine image, Netlify dependencies installation will fail. Using npm instead seems to fix this issue.
Signed-off-by: jffarge
Because docs.dagger.io is behind Github authentication, we had to crawl the entire website with a packaged docker image. We use the following command:
docker run -it --env-file=.env -e "CONFIG=$(cat docsearch.config.json | jq -r tostring)" algolia/docsearch-scraper
The crawler index all documentation pages (includes 0.1). We just want Europa documentation to be indexed.
This commit adds node_modules to dagger run commands so cached builds have
similar speed times as running commands locally directly through yarn
Signed-off-by: Marcos Lilljedahl
Jenkins is still prevalent in my customer projects, so that would definitely be an important target.
Only include the Europa docs
cd website
./rebuild_algolia_docs_search_index.sh
jq is /Users/gerhard/.nix-profile/bin/jq
docker is /Users/gerhard/.nix-profile/bin/docker
> DocSearch: https://deploy-preview-1833--devel-docs-dagger-io.netlify.app/1202/plan/ 16 records)
> DocSearch: https://deploy-preview-1833--devel-docs-dagger-io.netlify.app/1214/migrate-from-dagger-0.1/ 5 records)
> DocSearch: https://deploy-preview-1833--devel-docs-dagger-io.netlify.app/in...
Following up from this comment https://github.com/dagger/dagger/pull/1833#issuecomment-1076463604 I'm opening this issue to track this specific behavior separately
To validate this, I created a Dockerfile that mimics the current build behavior for the todoapp:
Dockerfile
FROM alpine:latest
RUN apk add --no-cache bash git yarn
COPY . /src
WORKDIR /src
RUN --mount=type=cache,target=/cache/yarn,id=todoapp-yarn-cache \
--mount=type=cache,target=/src/node_modules \...
This is a quick fix to attempt to automatically select the user's OS tab
on the pages that requires so which potentially avoids users following
incorrect docs
Signed-off-by: Marcos Lilljedahl
Argo is mostly a CD platform, isn't it? It already relies on pre-existing CI solutions like GA, Circle, Tekton, etc for the CI part. Not sure what could be done in Dagger as an Argo direct integration :thinking:
My personal experience tells me to take a look at Jenkins / Jenkins X (and Tekton in consequence). Most of the times I've seen Jenkins pipeline using docker plugins, they were super difficult to follow and configure. I have a feeling that there's a big opportunity there to showcase a simpler and more efficient solution that integrates with those tools like Dagger.
Resolve #1551 and #1020.
We are never returning the result of solved operations so Buildkit could not cache the layer.
This commit implements a simple system to forward operations' result to the main build to cache it.
Signed-off-by: Vasek - Tom C
Bumps node-forge from 1.2.1 to 1.3.0.
Changelog
Sourced from node-forge's changelog.
1.3.0 - 2022-03-17
Security
Three RSA PKCS#1 v1.5 signature verification issues were reported by Moosa
Yahyazadeh (moosa-yahyazadeh@uiowa.edu).
HIGH: Leniency in checking digestAlgorithm structure can lead to
signature forgery.
The code is lenient in checking the digest algorithm structure. This can
allow a crafted structure that steals padding bytes and uses unch...
Fixes #1304
This is mostly done, there's just a few more things to iron out. Publishing an "rc" now to unblock @grouville.
Signed-off-by: Helder Correia
Problem
When porting Particubes' config with @gerhard, we realized that the current docker.#Run implementation doesn't permit the connection with a server running inside a dagger plan, on Windows nor on Mac.
Repro
Given that config:
package main
import (
"dagger.io/dagger"
"universe.dagger.io/docker"
)
dagger.#Plan & {
actions: {
_http: docker.#Pull & {
source: "strm/helloworld-http"
...
Visiting https://docs.dagger.io/1201/ci-environment, we see a yaml file for github actions.
What are we supposed to do at this part of the documentation? It's copyable.
We just got the local dagger do build work. Are we supposed to do the next step on github actions directly here?
From what I understand, it's just a way how to integrate with other CI, but this is just documentation, not part of the tutorial we did right before. It confused me.
in https://docs.dagger.io/1202/plan, we present the main content of the file, the dagger.Plan.
Later, we discuss the import, but we didn't see that before, so we don't know what to do about it.
Iย guess maybe after the first step in the getting started executing locally, we should explain that now we are gonna explain more of the concept of dagger before continuing the original tutorial?
In the cue fragment, we have references to bash.#Run. Letting the user know that this comes from another package as well could avoid some implicit idea of some builtin commands.
Problem
Uninitialized projects hang on Windows, not on Mac
Context
While trying to export the improvement of the todoapp from @marcosnils', I copied the todoapp (and only this one) to another directory. Running dagger do build inside this directory made it hang indefinitely in Windows, and errors on Mac:
Mac
4:21PM FTL failed to load plan: import failed: cannot find package "dagger.io/dagger":
/tmp/todoapp/netlify.cue:4:2
: running `dagger project update` may res...
Bumps go.opentelemetry.io/otel/sdk from 1.5.0 to 1.6.0.
Changelog
Sourced from go.opentelemetry.io/otel/sdk's changelog.
[1.6.0/0.28.0] - 2022-03-23
โ ๏ธ Notice โ ๏ธ
This update is a breaking change of the unstable Metrics API.
Code instrumented with the go.opentelemetry.io/otel/metric will need to be modified.
Added
Add metrics exponential histogram support.
New mapping functions have been made available in sdk/metric/aggregator/exponentia...
Bumps go.opentelemetry.io/otel/trace from 1.5.0 to 1.6.0.
Changelog
Sourced from go.opentelemetry.io/otel/trace's changelog.
[1.6.0/0.28.0] - 2022-03-23
โ ๏ธ Notice โ ๏ธ
This update is a breaking change of the unstable Metrics API.
Code instrumented with the go.opentelemetry.io/otel/metric will need to be modified.
Added
Add metrics exponential histogram support.
New mapping functions have been made available in sdk/metric/aggregator/expone...
Bumps go.opentelemetry.io/otel/exporters/jaeger from 1.5.0 to 1.6.0.
Changelog
Sourced from go.opentelemetry.io/otel/exporters/jaeger's changelog.
[1.6.0/0.28.0] - 2022-03-23
โ ๏ธ Notice โ ๏ธ
This update is a breaking change of the unstable Metrics API.
Code instrumented with the go.opentelemetry.io/otel/metric will need to be modified.
Added
Add metrics exponential histogram support.
New mapping functions have been made available in sdk/me...
Bumps go.opentelemetry.io/otel from 1.5.0 to 1.6.0.
Changelog
Sourced from go.opentelemetry.io/otel's changelog.
[1.6.0/0.28.0] - 2022-03-23
โ ๏ธ Notice โ ๏ธ
This update is a breaking change of the unstable Metrics API.
Code instrumented with the go.opentelemetry.io/otel/metric will need to be modified.
Added
Add metrics exponential histogram support.
New mapping functions have been made available in sdk/metric/aggregator/exponential/mappin...
This fixes a bug that I introduced in https://github.com/dagger/dagger/pull/1819
Mistakes that get fixed before anyone notices never happened.
Focus on the learnings, quick fixes and let resilient systems emerge ๐ช
We currently have 0.2.0 hard-coded in a bunch of files in ./docs.
We should bump it whenever we release a new version.
I know one other Go-based project which does it nicely - ๐ @brancz.
- https://github.com/parca-dev/parca/blob/7dc4a2456d95b124459fbc7661c9d9b9099f126d/.github/workflows/release.yml#L92-L94
- https://github.com/parca-dev/parca.dev/blame/da283aec395887e0bfff05d126897ef1e6b72323/docs/kubernetes.mdx#L89
Agent & server versions on the website are always in sync. I...
@shykes, WDYT?
I meant to put it in universe.dagger.io/docker/client but docker.#Load needs it (#1842) and it would create a circular import.
It's also flexible in the way you can compose these.
The default allows for local socket, http/https or ssh:
run: docker.#Client & {
host: "ssh://root@93.184.216.34"
ssh: {
key: client.filesystem."/home/user/.ssh/id_rsa".read.contents
knownHosts: client.filesystem."/home/user/.ssh/known_hosts".r...
I wanted to address the decision to 1) make universe.dagger.io/docker a pure buildkit client, instead of a docker engine client; 2) move the traditional docker engine to universe.dagger.io/docker/client; and 3) encouraging the use of the former over the latter.
I know that it is not an intuitive decision: it's definitely a tradeoff. But I think it's the right one when you look at the current state and trajectory of the Docker platform. In short: the engine is now the legacy API, and buil...
This was discovered while trying to make the todoapp example config clearer:
diff --git a/pkg/universe.dagger.io/examples/todoapp/netlify.cue b/pkg/universe.dagger.io/examples/todoapp/netlify.cue
index 0c5cda96..d52be0c5 100644
--- a/pkg/universe.dagger.io/examples/todoapp/netlify.cue
+++ b/pkg/universe.dagger.io/examples/todoapp/netlify.cue
@@ -19,7 +19,7 @@ dagger.#Plan & {
}
client: {
filesystem: {
- ".": read: {
+ ...
DON'T MERGE until:
- #1842
This just allows expanding ~/ into the user home dir to make paths shorter and more portable:
-client: filesystem: "/home/user/.ssh/id_rsa": read: contents: dagger.#Secret
+client: filesystem: "~/.ssh/id_rsa": read: contents: dagger.#Secret
Signed-off-by: Helder Correia
In Go 1.18, we now have direct information to the version of the software thanks to VCS metadata
https://tip.golang.org/doc/go1.18#go-version
We should use that instead of using our own Makefile/vcs hacks.
DON'T MERGE (I'll rebase after #1842)
Bumps github.com/containerd/containerd from 1.6.1 to 1.6.2.
Release notes
Sourced from github.com/containerd/containerd's releases.
containerd 1.6.2
Welcome to the v1.6.2 release of containerd!
The second patch release for containerd 1.6 includes a fix for
CVE-2022-24769.
Notable Updates
Fix the inheritable capability defaults (GHSA-c9cp-9c75-9v8c)
See the changelog for complete list of changes
Please try out the release binaries and report any ...
Bumps @docusaurus/core from 2.0.0-beta.17 to 2.0.0-beta.18.
Release notes
Sourced from @โdocusaurus/core's releases.
2.0.0-beta.18 (2022-03-25)
:rocket: New Feature
docusaurus-mdx-loader, docusaurus-theme-classic
#6990 feat: lazy-load external images + ability to customize image display (@โslorber)
docusaurus-module-type-aliases, docusaurus-plugin-content-docs, docusaurus-theme-classic, docusaurus-theme-common, docu...
Bumps minimist from 1.2.5 to 1.2.6.
Commits
7efb22a 1.2.6
ef88b93 security notice for additional prototype pollution issue
c2b9819 isConstructorOrProto adapted from PR
bc8ecee test from prototype pollution PR
See full diff in compare view
[](https://docs...
Fix #1825
Signed-off-by: Helder Correia
Issue
When running dagger with the --with option in a plan using the dagger.#DecodeSecret action, structural cycle errors gets triggered, even if the referenced key is not being used
dagger -p ./debug.cue do --with 'actions: ref:"d"' good --log-format plain
5:52PM INF client.commands.so | computing
5:52PM INF client.commands.so | completed duration=0s
5:52PM INF actions.good | computing
5:52PM INF actions.good | completed duration=0s
5:52PM FTL system | failed to execu...
Whenever we need a llb.Local we wrap it in a llb.Copy because of a caching issue:
// FIXME: Remove the `Copy` and use `Local` directly.
//
// Copy'ing is a costly operation which should be unnecessary.
// However, using llb.Local directly breaks caching sometimes for unknown reasons.
st := llb.Scratch().File(
llb.Copy(
llb.Local(
path,
opts...,
),
"/",
"/",
),
withCustomName(v, "Embed %s [copy]", path),
)
We should find out if ...
Shorten one of the code samples, keeping only the part that speaks to the topic in the docs.
Problem
dagger.#Service is a core type referencing a network service. We chose an intentionnaly broad name to convey the support for multiple backends: unix socket, tcp, udp, websocket, http2, etc. With the benefit of insight, "service" is confusing because it is too broad.
Solution
Introduce a new core type: dagger.#Socket, a reference to a network socket endpoint. It is functionally equivalent to dagger.#Service, which it deprecates.
"Socket" is a better name because...
Problem
-
To connect to a unix socket on the client, I use
client: filesystem: "example.sock": read: dagger.#Socket(See #1870). That is confusing because reading from a socket is not the same as connecting to one. -
To listen on a unix socket on the client, there is no defined interface.
-
We don't yet support connecting to, and listening on, a tcp or udp port. But when we do, we will need a similar interface. Using "read" would be even more confusing there.
Solution
...
Problem
Dagger 0.1 had a command called dagger doc, which was similar to go doc. Given the names of a CUE package and definition, it would print information about their API.
Unfortunately, this feature was not compatible with Dagger 0.2, because it relied on concepts such as @dagger(input) and @dagger(output) which no longer exist. This is also why we currently don't have an API reference in our docs: the generator has not been ported to 0.2.
As a result, there is no easy ...
Overview
Dagger Universe is a catalog of CUE packages, carefully curated for safety and quality, that all Dagger developers can use to build awesome CICD pipelines without reinventing the wheel.
It is crucial to the success of Dagger that Universe can combine four properties:
- Growth. We need a lot of packages, and the capacity to publish more and more at an accelerated growth rate
- Quality. Packages must work well, be well documented, bugs need to be fixed quickly, etc.
- ...
We could automate updating versions but are we ok with that script committing back to the repo?
I prefer option 1, couldn't cuelang be used to bind supported versions ?
Dagger/Cue could register a package behaving like https://github.com/hashicorp/go-version calling this package semver.
I guess it is possible to do something like this:
import semver
#Kubectl: {
version: string & semver.NewConstraint(">= 1.19, < 1.24")
}
I don't know if it's feasible
I was discussing this earlier with @samalba: there is an overkill workaround.
If we were to use #Dockerfile with embedding, weโd be kept up to date by dependabot (automatically opening PRs against Dockerfiles when upstream gets updated).
Same principle applies to other package managers, e.g. if instead of doing a yarn install of Netlify at a specific version we embed a package.json, then the dependency gets updated automatically.
This applies to all kind of things (e.g. pip install a...
Option 1: To answer โyesโ means higher quality and safety, but considerably more maintenance work. If we fail to scale the maintenance work, we can hurt freshness, growth, or both.
I think universe packages have to pin their dependencies to upstream software
OK, thanks for clarifying! I agree itโs the most attractive option. But the additional work is not negligeable, so it requires careful planning and excellent execution to pull off.
I prefer option 1, couldn't cuelang be used to bind supported versions ?
To clarify, what are you referring to by โoption 1โ?
Option 1: To answer โyesโ means higher quality and safety, but considerably more maintenance work. If we fail to scale the maintenance work, we can hurt freshness, growth, or both.
I think universe packages have to pin their dependencies to upstream software
Another thing that occurred to me reading this is universe <-> engine API versions. We hopefully won't have a need to update engine APIs very often, but it seems pretty likely it will happen at least occasionally. So we should probably keep that in mind for the future, at the very least. More of a 1.0 thing, likely.
This is an open discussion topic, rather than a clear call to action. @aluzzardi in your opinion, what actions should we take now (in time for 0.2-beta.1) versus later? I suggest opening a separate issue for that, and moving this one out of the milestone.
I'm converting this to a discussion, since there is no clear path to closing it.
I wanted to address the decision to 1) make universe.dagger.io/docker a pure buildkit client, instead of a docker engine client; 2) move the traditional docker engine client to an obscure sub-package; 3) encourage the use of the former over the latter.
I know that it is not an intuitive decision: it's definitely a tradeoff. But I think it's the right one when you look at the current state and trajectory of the Docker platform.
In short: *docker engine is now the legacy API, and buildk...
I really like the idea and totally agree with you. The biggest blocker atm is: https://github.com/dagger/dagger/issues/1844, for which the absence of a docker.#Client package will force users to use the bash.#Run package with the docker-cli binary installed, meaning that people will be free / feel the need for such package pretty soon
Do we want to kind of control this behavior ? Shall @helderco make it his own package that people can download to keep the docker.#Client clean ?
...
Thanks for the context @shykes, makes perfect sense to me!
Just to record, we decided on universe.dagger.io/docker/cli.
I really like the subpackage name
legacyas an alternative toclient. It conveys the intention more clearly (see #1856) and is more accurate since technically both packages are clients - just not to the same API.
_Originally posted by @shykes in https://github.com/dagger/dagger/pull/1855#issuecomment-1078742175_ (emphasis mine)
Makes sense, I have just one small reservation on legacy. The Particubes use-case that @grouville is working on uses #Load. My concern is having feat...
Perhaps a naive question. Given 1) the difficulties encountered with this cache export feature, 2) this is one of the easy ones (users will want to export cache to many various targets, most of which are not supported by buildkit at all)...
Perhaps we should start discussing an alternative design that is more sustainable long term (while we continue working on fixing this).
My observations:
- Buildkit local exporter is very mature, and works everywhere.
- The local exporter produces mostly...
paging @grouville @aluzzardi @gerhard @TomChv
Just a few reaction
Buildkit local exporter is very mature, and works everywhere
Currently local cache doesn't work on Dagger, I'm still working on it.
Otherwise I agree with your proposal, I think with time we will have two solutions :
- Add new storage to buildkit to match our needs (we are more dependant to buildkit)
- Find a generic way that works in any cases, any environments
One of my concern is when we will need to integrate Gitlab runner, or Tekton or anything, it...
While I have not read all the content above, I wanted to mention about the good progress that @TomChv made with @crazy-max today: https://github.com/dagger/dagger/pull/1731#issuecomment-1068271318
@TomChv made a lot of progress and the fix is near. The problem was with the entire cache system, not just GHA.
There are several issues with buildkit caching, including the fact that its memory is short (only the last build, it's not cumulative).
I think the long term solution is to revamp it, either by contributing upstream a more modular approach (had a great conversation recently where someone mentioned the idea of having external cache backends in containers, similar to how fronten...
Did a bit of research using our own ci plan as a test.
Local cache takes about ~30 seconds after do completed to export ~800MB of data, every time.
$ time dagger do build
dagger do build 3.56s user 1.81s system 104% cpu 5.118 total
$ time dagger do build --cache-to type=local,dest=cache1
dagger do build --cache-to type=local,dest=cache1 15.17s user 6.45s system 69% cpu 31.202 total
$ time dagger do build --cache-to type=local,dest=cache2
dagger do build --...
Local cache takes about ~30 seconds after
docompleted to export ~800MB of data, every time.Comparatively, an export to registry only has a few seconds of overhead on the first run (NOTE: local registry) but has NO overhead the second run (since no bits need exporting):
warning: thinking out loud
That certainly makes always-local cache a less attractive strategy. Perhaps we should consider always-registry? That does have its own problems: is there always a suitable reg...
It would be great if Dagger could support building images with Cloud Native Buildpacks. More specifically, I'd like to integrate with Pack, which is both a Command Line Interface and a Go library.
Apart from executing the pack binary from CUE, either directly or via a custom image, there seems to be 2 ways to support this that I can see.
1. Have pack support a ...
The buildpacks/pack#768 issue above mentions some findings around export.
It is the Lifecycle and not the Platform (i.e., Pack) that makes the export. Which means that pack exports to the local daemon by default, unless using the --publish option to write to a registry. The exporting is actually done inside the builder contain...
@TomChv yeah, but better than just the registry is if we can have pack send to the local daemon (it's what it does by default). We can use docker.#Load after that if needed.
#PackBuild {
...
publish: bool
...
}
Are you thinking on using the Go library? I can always use Exec to the cli but this way there's no need to install pack.
Maybe a silly question but I want to be sure :
If we can create a standalone package that exposes a definition in cue to build & push to a remote registry, that would be a nice workaround until we support buildpacks artifact export?
E.g
package buildpacks
import (
"alpha.dagger.io/dagger"
)
// Build with buildpacks and push the result to a remote registry
#RemoteImage: {
source: dagger.#Artifact
ref: string
creds: ...
#up: [
...
For the moment I'm not sure, I have to verify what is necessary to do to use the Go library, I'll update you when I have more information.
Now I think it's fine to keep it in CUE and call the pack cli in a container (rootless.... no need for binding host socket, like kpack).
As mentioned by @kelseyhightower :) A very successful feature of Kubernetes is the ability to extend it with custom policies out-of-band from actual configurations. So an admin can inject a policy once, and everything will have to run through it.
There are 2 parallel tracks available to implement this:
- Cue itself is well suited for policy injection. I defer to @mpvl @myitcv @verdverm for details and examples.
- Runtime hooks. This is what Kubernetes does. It could be webhooks; or i...
@mpvl has a nice write up here: https://github.com/cuelang/cue/discussions/818 Using CUE for policy definition and admission control
This is WIP, I just wanted to capture a thought that keeps coming back
I think that our most active & trusted collaborators (this applies to everyone from the community) could benefit from auto-merging pull requests. I am bringing this up again because I think it deserves us to think about.
I imagine a future world where collaborators can earn our trust and get the privilege to auto-merge after n days.
If all checks pass, and there are no objections from anyone else (thi...
This reminds me of Linux where different subsystem maintainers continuously publish sourcetrees that they consider mergeable. There is no auto-merging but the humans involved build trust and gradually lower the testing & review bar based on past results. In some cases Linus typically merges the whole tree at release time without any review.
I think the same learning & trust building is already happening with our current vanilla system, you always need a manual merge but over time the maint...
An example of โstrengtheningโ the system while achieving some of the same results: remove merge privileges for all humans, have a bot automatically merge not based on identity of the contributor, but on review requirements and ownership rules encoded in software. This gives us more granular control over quality vs. speed, contributor impatience vs. maintainer angst, etc.
I think we should do this ASAP.
Purpose
After we created the serverless package with @grouville to deploy lambda functions to AWS.
It would be really nice to create a new package to deploy containerized applications to the cloud.
First, we should create a high-level abstraction on top of AWS App runner.
Requirement
- Deploy application from git repository or image from ECR
- App configuration (port, runtime, environment)
- Config deployment (scaling, res...
Relaying a pain point with Terraform... that will fail validation late in the process for items which require fetching data from the cloud provider API.
- I'm working on an image_template
- I built an image with packer having a disk size of 250G
- I used TF to create a template using this image and same disk size
- I then decided I wanted a smaller disk
- I TF planned, looked correct, then TF applied
- TF deleted the existing template (expected, it is a replace)
- TF failed on...
So after I was able to deploy to Civo Kubernetes (which is K3s based) There was s thought of integration and I am ver well onboard with this.
Thank you for the feedback, @saiyam1814
We'll look into Civo. Would you be interesting in contributing the support by any chance?
Oh Yes why not? I dpn't know cue tbh but I would want to contribute in any way, even if it required learn by doing I am up for it.
So civo has a civo cli that is currently used to manage all resources within civo, what are the next steps for me in order to have civo dagger integration ?
cc - @shykes
https://github.com/accurics/terrascan
Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
500+ Policies for security best practices
Scanning of Terraform (HCL2)
Scanning of Kubernetes (JSON/YAML), Helm v3, and Kustomize v3
Support for AWS, Azure, GCP, Kubernetes and GitHub
Idea to scan dagger application before deploying
we are planning to add cue support (https://github.com/accurics/terra...
@sangam14 Do you think this should be implemented as a package, e.g. terrascan.#Scan, so it can be used in a pipeline (e.g. scan kube yamls with terrascan, then apply them)?
Or were you envisioning a different integration?
as package make more robust within pipeline import "alpha.dagger.io/terrascan " . dagger cue package already support yaml .
This is a proposal for one particular solution to the problem of state persistence in Dagger.
Problem
Sometimes, a component in the DAG needs to remember what they did earlier - usually so that they can undo it. This feature is called โpersistent stateโ or โstateโ for short. For example, a component which creates Amazon S3 buckets needs persistent state to remember which buckets they created, in case the user wants to destroy them later.
One obvious use of persistent state is the ...
This is an interesting/difficult problem. I totally agree re the need. It's something that was coming up for me as I was thinking about dagger down: clearing state as the DAG gets unwound.
I like the idea of using CUE as the interface for this. Should simplify the interface and make it clearer. Also, the ability to define an arbitrary struct to be stored as state would be awesome.
This is still relevant in Dagger 0.2, although the design for a solution might be different now.
Issue
When referencing a non existing client filesystem, dagger do doesn't break and makes the action disappear/unusable
Repro
package main
import (
"dagger.io/dagger"
"universe.dagger.io/docker/cli"
)
dagger.#Plan & {
client: {}
actions: {
run: cli.#Run & {
host: "ssh://root@X.X.X.X"
ssh: {
key: client.filesystem."/Users/home/.ssh/id_rsa".read.contents // inexistent in the DAG, but exists on host
knownHosts: client.filesystem."/U...
Issue
When trying to store some secrets in an encrypted sops file, I realized that an action referencing the output of a #DecodeSecretbecomes unusable
Repro
Working config
package main
import (
"dagger.io/dagger"
"universe.dagger.io/docker/cli"
)
dagger.#Plan & {
client: {
filesystem: {
"/Users/home/.ssh/id_rsa": read: contents: dagger.#Secret
"/Users/home/.ssh/known_hosts": read: contents: dagger.#Secret
}
}
actions: {
run: cl...
Signed-off-by: Helder Correia
Signed-off-by: Helder Correia
Fixes #1870
Signed-off-by: Helder Correia
in https://docs.dagger.io/1203/client/#running-commands, we talk about we can use stdin + stderr, but we don't link to any other docs. And it seems we have none.
I believe the todoapp repo is in this repository to keep it synchronized with the main code.
But for newcomer, wouldn't it be nice to have a dagger/todoapp-example repo so people could fork at will and have already a github actions set up for use?
Just change netlify creds var and you're good to go.
Also, new users might not be aware/proficient with CI, yet, and don't know how to configure github actions, CI env var. So by making it the easiest, it would be maybe a magical "AHA" moment.
Issue
When trying to mount an SSH key securely inside my container, I realized that a cat command would not hide multiline env variables properly
Repro:
First, export a multiline env var
export FOO='foo
bar
baz
zoo'
Repro config
package main
import (
"dagger.io/dagger"
"universe.dagger.io/alpine"
"universe.dagger.io/docker"
)
dagger.#Plan & {
client: env: FOO: dagger.#Secret
actions: {
_dockerCLI: alpine.#Build & {
packages: {
...
Otherwise we need to uncomment the platform value.
The GOOS can be set just-in-time:
GOOS=darwin dagger do build
Not sure how well this works on Windows as I don't have a host to test on.
Deprecation warning
Fixes #1871
Signed-off-by: Helder Correia
All dagger packages should be ready to go for the getting started tutorial. This commit addresses that.
We should make this part of the CI. That was my first step, but then I've hit a bigger rabbit hole than anticipated: https://github.com/dagger/dagger/pull/1893
This needs to happen now, before v0.2.3 gets cut, leaving the CI part for later.
Fixes https://github.com/dagger/dagger/issues/1820
If you checkout this PR on a clean Windows host, does dagger do build work as ex...
We should hold this merge until we are ready to cut the release, otherwise links & some instructions will be broken until we release v0.2.3.
Even though I would have preferred to have docs/update-version.sh in Dagger, this is the quickest thing for now.
Bumps go.opentelemetry.io/otel/exporters/jaeger from 1.6.0 to 1.6.1.
Release notes
Sourced from go.opentelemetry.io/otel/exporters/jaeger's releases.
Release v1.6.1
Fixed
The go.opentelemetry.io/otel/schema/* packages now use the correct schema URL for their SchemaURL constant. Instead of using "https://opentelemetry.io/schemas/v<version>" they now use the correct URL without a v prefix, "https://opentelemetry.io/sc...
Bumps go.opentelemetry.io/otel/trace from 1.6.0 to 1.6.1.
Release notes
Sourced from go.opentelemetry.io/otel/trace's releases.
Release v1.6.1
Fixed
The go.opentelemetry.io/otel/schema/* packages now use the correct schema URL for their SchemaURL constant. Instead of using "https://opentelemetry.io/schemas/v<version>" they now use the correct URL without a v prefix, "https://opentelemetry.io/schemas/<version>&...
Bumps go.opentelemetry.io/otel/sdk from 1.6.0 to 1.6.1.
Release notes
Sourced from go.opentelemetry.io/otel/sdk's releases.
Release v1.6.1
Fixed
The go.opentelemetry.io/otel/schema/* packages now use the correct schema URL for their SchemaURL constant. Instead of using "https://opentelemetry.io/schemas/v<version>" they now use the correct URL without a v prefix, "https://opentelemetry.io/schemas/<version>"...
Bumps go.opentelemetry.io/otel from 1.6.0 to 1.6.1.
Release notes
Sourced from go.opentelemetry.io/otel's releases.
Release v1.6.1
Fixed
The go.opentelemetry.io/otel/schema/* packages now use the correct schema URL for their SchemaURL constant. Instead of using "https://opentelemetry.io/schemas/v<version>" they now use the correct URL without a v prefix, "https://opentelemetry.io/schemas/<version>". (#274...
This was mixing layer 4 & 7 concerns. It's clearer now.
Discovered via https://github.com/dagger/dagger/pull/1896
I think that requiring two packages to be imported instead of a single one doesn't sit right:
Could we simplify this so that only a single package needs to be imported for everything else to work?
Follow-up to https://github.com/dagger/dagger/pull/1866 cc @helderco @shykes
- move Dagger 0.1 references to the top
- move Dagger CUE API under Core Concepts, after What is CUE?
- move Go on Docker Swarm under Use Cases
A few minor title changes:
- Dagger CUE API (0.2+) -> Dagger CUE API
- What is Cue? -> What is CUE?
A simpler What is CUE? Europa doc page
- shorter sentences
- more lists
- less content (because less is more when it comes to docs)
Part of #1758 (the last part)
Should be merged after #1904
Longer term, we can (and IMO should) host a binary-on-demand service eg. get.dagger.io/binary/dagger-$GITREF.tgz that builds the requested version just-in-time for download.
_Originally posted by @shykes in https://github.com/dagger/dagger-for-github/issues/35#issuecomment-1071971420_
Reminder to self: before I merge this, add an issue that reminds me to add a helloworld page to the docs.
_Originally posted by @gerhard in https://github.com/dagger/dagger/issues/1817#issuecomment-1075511612_
I noticed the install docs mentions v0.2.0 on one side and v0.2.3 right after. Ideally this should not be hardcoded or pointing to the latest one automatically.
And later:
cc @gerhard
Fixes #1848
I did not add build flags for windows or if on runtime.GOOS because filepath.VolumeName is already cross-platforms (returns "" when not supported).
We can only use dagger using the binary for now. Is there any way we can call dagger via Go API?
Problem
When I follow "getting started" from the website, I dive straight into the first part of a multi-part, hands-on tutorial. Hands-on is good, but the transition is abrupt, especially for a first-time visitor who only learned about Dagger 30 seconds ago. The fact that the title does not use the words "getting started" amplifies this. Personally, I briefly thought I had followed the wrong link.
Solution
When I follow a "getting started" link, I should l...
We are working on migrating to dagger 0.2, but docker.#Run has some problem: https://github.com/dagger/dagger/issues/1466, I can only use dagger.#Exec for this time.
If you're replacing with core.#Exec then you must have a different issue. The one you pointed to is about having nested docker.#Builds. If you do have a nesting depth of 3 (docker.#Build inside a docker.#Build), you can unblock by replacing it with explicity fields:
-build: docker.#Build & {
- steps: [
- docker.#Pull & { ... }
- docker.#Run & { ... }
- }
-}
+build: {
+ pull: docker.#Pull & { ... }
+ run: docker.#Run & { input: pull.output,...
For now, we just have the documentation giving us a github actions workflow to copy.
The documentation should use as much of the already existing tooling in this repo for reproducing the todoapp in the users' own fork.
I think we should have the closest common denominator between those two files to make sure they are tested AND documented
I'm trying to deploy the todoapp from my dagger fork. The error is not explicit.
github actions log for todoapp
4:02PM INF actions.build.run._exec | #20 12.08
4:02PM INF actions.build.run._exec | #20 12.08 The build folder is ready to be deployed.
4:02PM INF actions.build.run._exec | #20 12.08
4:02PM INF actions.build.run._exec | #20 12.08 Find out more about deployment here:
4:02PM INF actions.build.run._exec | #20 12.08
4:02PM INF actions.build.run._exec | #20 12.08 bit.l...
Signed-off-by: Andrea Luzzardi
Ignore ci:, website:, and dependabot commits
An important feature of Dagger is Universe: a curated collection of reusable actions, which any Dagger developer can add into their plan.
So far we have actions for Docker, Netlify, AWS, Yarn, just to name a few.
Anyone can develop their own actions, but it's important to expand the collection available in Universe, so that de...
One common request is to add actions for kubernetes:
- Generate a Kubernetes config
- Validate a config
- Apply a config with
kubectl apply - Support native authentication to popular kubernetes hosting providers (AWS, Azure, Google Cloud, Alicloud, etc)
- Perhaps integrate with popular companion tools, such as kustomize?
The netlify package uses the official netlify CLI for certain things, but not for others (it uses a curl-based wrapper for those). Since the package was written, the netlify CLI has expanded its features, and we can now remove the curl-based wrappers, simplifying the package in the process.
All new projects are created as "locked" by default so this allow their
deployment. Additionally, a moreh helpful error message is added when
NETLIFY_ACCOUNT cannot be found
Fixes #1918
Including the version that we install explicitly.
Bumps cypress from 9.5.2 to 9.5.3.
Release notes
Sourced from cypress's releases.
9.5.3
Released 3/28/2022
Bugfixes:
Corrected a logging issue with cy.intercept() where the response status code
and response body were sometimes missing from the log details in open mode.
Fixed #18757.
Fixed an issue in cy.readFile() where the command would not retry or provide
a meaningful error message if null encoding was used and the file was not
found. Fixed #2068...
To avoid "noisy" commits, let's prefix commit for website changes with: website: XXX
Add this to CONTRIBUTING.md
Fixes #832
@gerhard can you help me put this page at the bottom of the "Core Concepts"?
Used standard types & low-level primitives for the headings too.
Moved the types defined in core out of the standard types table, so that all definitions in core are grouped under their own section.
The docs page 1213 called "Dagger CUE API" should be split up into more clear pages, and removed.
New pages should be:
-
Core concepts / "The Action API"
- Incorporates content from the 0.2 release announcement blog post
- Also incorporates content from "Cue API": https://docs.dagger.io/1213/api#core-actions
- No need to rewrite from scratch. Copy-paste + light edits is enough.
-
Core concepts / "Core types"
- Explain Dagger core types
- Incorporate https://docs.dagge...
ID 1216 was clashing with docker-cli-load doc.
The filename follows the convention of the other use cases.
Remove from docs sidebar until we do the quick fixes.
this would help me solved this problem, I just though concat the context manually is not elegant enough. in 0.1 API I use op.exec, It's perfect.
Checking at https://docs.dagger.io/1215/what-is-cue/ seems like when code blocks overflow, a global horizontal scrollbar [1] also appears which leaves an empty white space at the left of the screen [2]
[1]

[2]

This happens both in Brave and Firefox browsers in l...
Starting with CI/CD in your local dev is too abrupt, and there is a lot of great content on that blog post page that we should promote more ๐
cc @shykes
Signed-off-by: Helder Correia
Updates the Go on Docker Swarm use case to europa config. Diagrams are also included for better UX
contributors: @gerhard
Signed-off-by: guillaume
Any cue file created in the docs triggered these tests. It shouldn't.
Contributor: @dolanor
Signed-off-by: guillaume
Minor changes to the DefaultVersion definition in the go package.
- Change the default go version (1.16 -> 1.18)
- Hid the
#_DefaultVersionfield - Changed the bats test name (bash -> go)
What is the issue?
The homepage has this text:
Wire actions together using CUE, a powerful configuration language developed at Google. CUE has all the features you wish YAML had: comments, string interpolation, templating, static type checking, a complete package system, and more. And best of all, it can natively import and export YAML and JSON, for maximum compatibility with your existing tooling.
YAML definitely supports comments.
Log output
...
From this comment on Hacker News:
gregmac 4 minutes ago | next [โ]
Your Windows instructions/process needs work.
curl https://dl.dagger.io/dagger/install.ps1 -OutFile install.ps1
This uses thecurlalias, which is reallyInvoke-WebRequest. It also makes the incorre...
@marcosnils I'm not sure if that's what @eddumelendez meant, but we use Argo Workflows to create our CI pipelines. It could be useful to have Dagger integrated with it so we could test them locally. Argo Workflows + Argo Events + Argo CD is basically a CI/CD platform by itself. For reference: https://argoproj.github.io/argo-workflows/
This project seems promising, but regarding https://docs.dagger.io/1201/ci-environment and CI pipelines
With multiple stages and jobs, wouldn't you always have to execute each dagger action independently to be able to use parallelism and autoscaling of pipeline agents/workers? So in fact you are always tied to writing a pipeline file (for those platforms that support Pipelines as Code) and call dagger actions from that?
This commit enables PR's to run inegration tests by relying on keys with
only the necessary permissions to run on CI workloads
Hey, here's how I managed to get dagger to run on Gitlab-CI

Hey, I've just opened a PR with a sample gitlab ci configuration: https://github.com/dagger/dagger/pull/1949
Thank you @adrienbrault for contributing this ๐๐ป
What are you trying to do?
Install dagger by following the Linux flow as specified on https://docs.dagger.io:
/usr/local $ curl -L https://dl.dagger.io/dagger/install.sh | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7649 100 7649 0 0 111k 0 --:--:-- --:--:-- --:--:-- 109k
sh debug downloading files into /tmp/tmp.r6tBDVdRC5
sh debug http_dow...
What are you trying to do?
In pre-europa days there is a kubernetes package that supports #Kubectl. More than that, we have implemented #Helm ourselves. We would like to contribute them to upstream.
Why is this important to you?
No response
How are you currently working around this?
No response
What are you trying to do?
Rather than exporting to Jaeger directly, why not output OTLP over gRPC/HTTP in order to support trace data export in a more 'pure' format that can go to the OTel Collector or directly to a tracing system that supports OTLP ingest?
Why is this important to you?
No response
How are you currently working around this?
I did a local rip/replace of the Jaeger exporter with "go.opentelemetry.io/otel/exporters/otlp/otlptrace" and it seems to work fine ...
What is the issue?
I get black text on black background in the blog section which makes it hard to read :)
for example:
Browser:
Chrome 100.0.4896.60 (Official Build) (64-bit...
What about GitLab CI with self-hosted runners without privilege escalation capabilities?
My point being, is there a way to run Dagger in GitLab CI without exposing the runner Docker daemon?
What is the issue?
I have written some of my own CUE Modules, which may go out of memory when run multiple times over a long period of time.
Log output

Steps to reproduce
No response
Dagger version
dagger devel (b32c8732) linux/amd64
OS version
Linux VM-0-6-ubuntu 4.15.0-159-generic #167-Ubuntu SMP Tue Sep 21 08:55:05 UTC 2021 x86_64 x86_6...
What is the issue?
The documentation code blocks use the Courier New font, which is relatively old. There aren't any major problems on my retina display, but codes look aliased on my external non-retina display. To help you understand better, I took a few screenshots:
On non-retina display

On non-retina display + manually editing CSS...
@adrienbrault Thanks, Iโll take a look at this and maybe give it a try with Dagger :slightly_smiling_face:
On Windows, we run start build/index.html in our Command Prompt terminal and see the following app preview:
->
On Windows, we run start _build/index.html in our Command Prompt terminal and see the following app preview:
build/index.html -> _build/index.html
On Windows, we run start build/index.html in our Command Prompt terminal and see the following app preview:
->
On Windows, we run start _build/index.html in our Command Prompt terminal and see the following app preview:
build/index.html -> _build/index.html
Signed-off-by: Dingrui Yang abserari@gmail.com
What is the issue?
I use nerdctl with buildkit and containerd (Rancher Desktop) to normally build, tag and push my "docker" images (OCI compatible). If I want to use the docker Dagger step as mentioned in the "quick start" here, I get the error that it couldn't connect to the docker daemon. I was told to create this issue by helderco#4068 in Discord.
Log output
`11:09AM ERR failed to run docker: exit status 1 output=Client:
Context: default
11:09...
After compare the exist solutions for Github Action and Gitlab CI (currently others are not developed) in https://docs.dagger.io/1201/ci-environment, I just realized, the concept that dagger can run on all cicd, can be easily replaced by a Makefile
when change from dagger do build to make do build, it works on all pipelines.
LoL
What are you trying to do?
No response
Why is this important to you?
users want to known the difference between dagger and other ci/cd tools
How are you currently working around this?
No response
As the homebrew formula also supports Linux, I suggest the following updates in the installation procedure.
See https://github.com/dagger/homebrew-tap/blob/main/dagger.rb#L29.
Taking into account your argument regarding Azure Pipelines, dagger should find a good use case for teams that want to slowly transition from ADO to Github.
The sample GitHub action in the documentation did not work due to go packages not being installed before the deploy script.
This is a working sample and suits the documentation better, imo.
You can see the trial and errors here: https://github.com/ekinbarut/dagger_test/actions
Signed-off-by: jffarge
it was highlighting the wrong keywords (test, do)
What is the issue?
Doing a standard go.#Build fails with the following logs:
This is because of go defaulting this flag to true and dagger not necessarily send the .git in the context (for relatively good reason).
Log output
4:04PM INF actions.tknlocal.container._exec | #5 0.325 go: missing Git command. See https://golang.org/s/gogetcmd
4:04PM INF actions.tknlocal.container._exec | #5 0.325 error obtaining VCS status: exec: "git": executable file not found in $PA...
Resolve #1551 and #1020.
We are never returning the result of solved operations so Buildkit could not
cache the layer.
This commit implements a simple system to forward operations' result to the
main build to cache it.
Copy of #1839 but required with the public migration
Signed-off-by: Vasek - Tom C
Yes, we're on Azure DevOps Pipelines. Would like to see this.
What is the issue?
see errors below, using the "getting started" instructions fails on OSX
Log output
โ git clone https://github.com/dagger/dagger
Cloning into 'dagger'...
remote: Enumerating objects: 22909, done.
remote: Counting objects: 100% (622/622), done.
remote: Compressing objects: 100% (196/196), done.
remote: Total 22909 (delta 530), reused 434 (delta 425), pack-reused 22287
Receiving objects: 100% (22909/22909), 10.26 MiB | 7.65 MiB/s, done.
Resolving deltas...
What is the issue?
Hey there. I am trying to deploy to Netlify using the tutorial (Integrating with your CI environment). I am testing GH Actions but I am receiving the error message below. I am unclear what's not finding when communicating to Netlify (?)
I am not super familiar with Netlify but I have created the NETLIFY_TOKEN secrets and used massimo as a NETLIFY_TEAM (because that's my team name on Netlify). I am using the free tier, FYI.
Log output
.......
........
got it, thx! Wasn't aware about workflows. It looks like a potential opportunity for dagger as well.
Is that a "no" for using dagger without exposing the Docker daemon?
I would suppose that most Kubernetes based CI systems will recommend not exposing the Docker daemon.
I would extend this request further to add actions for helm.
What do you think?
Add an action for terraform:
- init Backend remote state
- plan
- apply
- workspaces
- ...
while dependabot automatically bump up docusaurus core, the package preset-classic is ignore.
Signed-off-by: jffarge
Wdyt about notification/notify module for handling authorization and sending messages via the most popular channels/apps:
- slack
- mattermost
- teams
etc.
I'm surprised at how DroneCI still hasn't made the list.
This adds git to the default go.#Image image so that 1.18 "just
works".
Fixes https://github.com/dagger/dagger/issues/1965
Signed-off-by: Vincent Demeester
@ravilach wrote a whole Kubernetes deployment tutorial based on Dagger 0.1: https://shipa.io/miscellaneous/kubernetes-dagger-deployment-with-shipa/
Let's help him port that configuration to 0.2 !
See also: https://twitter.com/ravilach/status/1509579332758482947
cc @grouville
Now that we are public, many more people are joining our Discord channel. The automated "welcome!" messages are starting to be overwhelming. I propose that we remove them.
Howdy Dagger Buds! I took the example straight from the Dagger v0.1 docs. I had a little trouble trying to figure out how to apply some Kubernetes manifest in the latest and greatest. The v0.1 docs were a great help [and hidden in the blog, I use EKS so had to wire in the AWS client]. Excited to see where the project goes :-)
GoCD . It's kind of cd tool but relies on docker agents for builds. So I assume shell based integration will work.
My team is also on ADO, would love to see this as well
After discussion with @grouville, use h2 title instead of h3 + fix code css
Signed-off-by: jffarge
Bumps @mdx-js/react from 1.6.22 to 2.1.1.
Release notes
Sourced from @โmdx-js/react's releases.
2.1.1
e79fc2be 0df684bc Fix to improve _missingMdxReference
by @โvlad-zhukov in mdx-js/mdx#1986, mdx-js/mdx#1988
Full Changelog: https://github.com/mdx-js/mdx/compare/2.1.0...2.1.1
2.1.0
Core
aff6de4f minor add support for passing options to remark-rehype
by @โstefanprobst in mdx-js/mdx#1935
5d4355e4 patch replace got w/ node-fetch
by ...
Bumps sass from 1.49.9 to 1.49.10.
Release notes
Sourced from sass's releases.
Dart Sass 1.49.10
To install Sass 1.49.10, download one of the packages below and add it to your PATH, or see the Sass website for full installation instructions.
Changes
Quiet deps mode now silences compiler warnings in mixins and functions that are defined in dependencies even if they're invoked from application stylesheets.
In expanded mode, Sass will now emit color...
What is the issue?
When using a file instead of a directory as the input of dagger.#FS, dagger hangs.
Log output
N/A
Steps to reproduce
Create a dagger with the following plan file and run dagger do test to reproduce.
package main
import (
"dagger.io/dagger"
)
dagger.#Plan & {
client: filesystem: "somefile.txt": read: contents: string
actions: test: {
}
}
Dagger version
0.2.4
OS version
Linux tp 5.13.0-37-generic #42~20.04....
CODEOWNERS is a tool to encode PR approval rules.
When a PR is opened, at least one code owner is required to approve it
before being merged.
It does not:
- Limit reviewers: Everyone is welcome and encouraged to review any PR.
But at least one CODEOWNER must approve before merging. - Limit contributions or ownership: Every maintainer is responsible for
the entire project. CODEOWNERs are there to review PRs for
consistency.
By default, any maintainer can approve any PR. There's a...
Is there an official dagger docker image? I can work on creating the Codefresh pipeline with the existing todoapp example
Errors out in the PreRun phase of the clientfilesystemreader task since
otherwise, dagger execution would hang
Fixes #1977
Signed-off-by: Marcos Lilljedahl
Companies like Daimler (Mercedes-Benz), BMW, Volkswagen) and many others big would like to see this integrated with Azure DevOps Pipelines.
it is listed, 6 hours before yours by me :-)
IMO, this discussion should be locked. Lots of duplicate comments, and all mostly all the ones listed here have been mentioned.
https://landscape.cd.foundation/card-mode?category=ci-pipeline-orchestration
Problem
Dagger 0.1 had a documentation page comparing Dagger to other software. This page was removed when the docs were rewritten from scratch from Dagger 0.2. This was not necessary as the page applies to Dagger 0.2 without modification.
Several people have asked how Dagger compared to other software, confirming the value of that page.
Solution
Include the original page, unmodified, in the Dagger 0.2 navbar.
It removes the curl dependency, so it simplifies the install on Windows.
Fixes #1946
Signed-off-by: Andrea Luzzardi
- copy: support for include/exclude
- universe: go: do not hardcode default platform
- universe: go: improve cache management
- client filesystem: remove .dagger from default exclusion
- task: source: fix vertex name
- universe: go: add golangci-lint support
- ci: improve dogfood configuration
Signed-off-by: Andrea Luzzardi
Signed-off-by: kgb33
Closes #1981
I made a copy of introduction/1002-vs.md
mirroring the copies of learn/1005-what_is_cue.md and core-concepts/1215-what-is-cue.md

