#GitHub - kubernetes-client/javascript: J...

1 messages · Page 1 of 1 (latest)

scarlet badge
#

Theoretically, you could. But, it wouldn't really make any sense to do IMHO.

wraith siren
#

kubectl makes sense. if you are writing lots of functions in pipelines with the js functions of the k8s client I think it does make sense

#

I think it depends how are you using k8s client. I think there is a helm wrapper or at least I saw on daggerverse so it depends what you're trying to do

scarlet badge
#

@wraith siren - Why not just write an application to manage the k8s cluster with the client? Why have Dagger in the middle building containers to run code, which can just be ran? To me, it is an added level of abstraction, which is absolutely unnecessary (for k8s management of longer running clusters). Or, are we speaking of temporary clusters for like running tests? 🤔

wraith siren
#

I think it really depends what's trying to be achieved.

#

We automate creation of clusters with Terraform

#

We automate deployments, with Flux/GitOps and Helm Charts

#

I can see usage of dagger with K8s in the sense of running and wrapping helm,kubeconfig.

#

If they already have pipelines with javascript clients and theyre doing all kinds of things, I think it makes sense to move it to dagger. If it's the right way to really be administering k8s? That's another question. If they have things already in pipelines that's hard to understand or run or develop properly on CI, i think it makes sense to move it

#

Think of apache airflow, people use that to run pipelines too, but they write dags to control the flow and execute things. IT's not a built application that 'just runs'

#

I know its not a 1:1 to dagger, but they both have "pipelines" as a concept. And ive seen apache airflow used alooooooooot for deployment things and management of services. even spinnig up containers etc

dull cedar
#

so in case if we want to run k8 clusters for our inhouse internal devops and test purpose use then is it advisable to use js client and dagger?

wraith siren
#

I wouldnt say I advise that approach no

#

If you want to automate clusters, I would find an appropriate tool for that. For example eks terraform providers and I would alse gitops to manage the estate of the cluster, like ArgoCD, FluxCD etc.

#

I finished the CKA course, and i think even after completing that, as someone who would manage a cluster, I think it depends what I am trying to do. I'd be mostly running kubectl commands based on what I needed to do. IF some for some reason, a team wanted several services in K8s, I would use helm, flux/gitops in order to set that up. I dont think i would use dagger for that.

If I wanted to adhoc, spin up some services/apps for a team, I would helm chart them if possible, helm is a nice way to pack, customise whatever you want right?

#

I dont think dagger replaces anything in the K8s ecosystem right now specifically. But if you have pipelines that trigger helm deployments, etc, you could daggerise those pipelines. If you're using a Kubernetes SDK, like a Js one, or the .NET client, any K8s client/SDK - I think it depends what youre trying to do specifically if you think wrapping up those things into portable dagger functions would benefit everyone, because of pipeline pains

#

I would say dont replace charts, or bootstrapping clusters with dagger - but maybe others have some opinions too.

scarlet badge
#

Before I'd recommend anything as a solution to your problem, I'd also ask about what you are developing and how long do the clusters need to be active? In other words, are you developing applications to be used as tools inside k8s (like the tools Patrick was mentioning), or are you using k8s as the means to building a development and delivery platform? Big difference! 🙂

#

And, either way, I don't believe Dagger is the right fit. But, to definitely know that, we need to understand the true use case.

dull cedar
#

@scarlet badge we want to build k8 cluster with microk8 to run our devcontainers, dagger pipelines and to run llms locally on k8 cluster, that's it and that cluster will be with us for many years and we need to update and upgrade also in future, please guide us. thanks

scarlet badge
#

@dull cedar - So, you wish to use Dagger to help with your microk8s local setups?

dull cedar
#

@scarlet badge Yes we dont want to write yaml again and we have done study that we can interact directly with k8 api layer and so by that way we can drop yml, can't we?

dull cedar
#

can't we do something like this
Code 1/2

import { connect } from "@dagger.io/dagger"
import axios from "axios"
import * as fs from "fs"

// Function to read the Kubernetes config file
function readKubeConfig() {
  const homeDir = process.env.HOME || process.env.USERPROFILE
  const kubeConfigPath = `${homeDir}/.kube/config`
  return JSON.parse(fs.readFileSync(kubeConfigPath, 'utf8'))
}

// Function to get the API server URL and authentication token
function getKubeConfig() {
  const config = readKubeConfig()
  const cluster = config.clusters.find((c: any) => c.name === config["current-context"])
  const user = config.users.find((u: any) => u.name === config["current-context"])

  return {
    apiServer: cluster.cluster.server,
    token: user.user.token
  }
}

async function main() {
  // Initialize Dagger Client
  const client = connect()

  // Get Kubernetes configuration
  const { apiServer, token } = getKubeConfig()

  // Configure axios for Kubernetes API requests
  const k8sApi = axios.create({
    baseURL: apiServer,
    headers: {
      'Authorization': `Bearer ${token}`,
      'Content-Type': 'application/json'
    }
  })

  // Define a simple deployment
  const deployment = {
    apiVersion: "apps/v1",
    kind: "Deployment",
    metadata: {
      name: "example-deployment"
    },
    spec: {
      replicas: 3,
      selector: {
        matchLabels: {
          app: "example"
        }
      },
      template: {
        metadata: {
          labels: {
            app: "example"
          }
        },
        spec: {
          containers: [{
            name: "example-container",
            image: "nginx:latest",
            ports: [{
              containerPort: 80
            }]
          }]
        }
      }
    }
  }
#

Code 2/2

  try {
    // Use Dagger to orchestrate the deployment creation
    await client.container()
      .withExec(["sh", "-c", `
        kubectl create deployment example-deployment --image=nginx:latest --replicas=3 || true
        kubectl expose deployment example-deployment --port=80 --type=LoadBalancer
      `])
      .exitCode()

    console.log("Deployment created using Dagger")

    // Create the deployment using Kubernetes API
    const createResponse = await k8sApi.post('/apis/apps/v1/namespaces/default/deployments', deployment)
    console.log("Deployment created using Kubernetes API:", createResponse.data.metadata.name)

    // List all deployments
    const listResponse = await k8sApi.get('/apis/apps/v1/namespaces/default/deployments')
    console.log("Deployments:", listResponse.data.items.map((item: any) => item.metadata.name))
  } catch (error) {
    console.error("Error:", error)
  } finally {
    await client.close()
  }
}

main().catch(console.error)
#

because by this wan we can build small small task functions which can be called in dagger and so we dont need to manage larger yml file problems also

scarlet badge
#

So, you want to use Dagger to create a container to make kubectl calls to build out deployments in microk8s locally? Why not just make the kubectl calls locally (programatically) via Node's own exe method? Does your code above work? Or did you piece it together for an example?

#

Sounds like you need something more like Tilt. The "config" in Tilt is a form of Python though. In the end, you can't get around "config-like" code (YAML, JSON or otherwise), as that is what k8s needs to do its work. Anything you come up with is just an abstraction of that, and thus might and probably will be prone to leaks.

dull cedar
#

@scarlet badge but let's say we have enough senior engineers who can manage api code for K8 and they can understand dealing with api, and if we dont want to orchestrate k8 in declarative manner to use power of programming language and dagger then are we on right path to use direct k8 api with dagger?

dull cedar
scarlet badge
#

Do you wish to use Dagger to actually run k8s locally too, so k8s in a container? Because otherwise, I don't see how it would be helpful. Dagger is a tool to orchestrate (aka workflow) containerized workloads. The goal of Dagger is to run these workloads (because they are in containers) anywhere. So, if your goal is to have a Daggerized workflow to run dev environments, that sort of breaks the goal of Dagger, because what you are building won't work in your production cluster.

#

Have you looked into cloud development environments? Tools that allow your dev team to work in dev clusters, instead of them having to orchestrate cluster builds locally?

dull cedar
#

no we dont want to run k8 cluster in container, but after setting up cluster we want to orchestrate other things with dagger like creating pod or services, etc

#

we dont want to use cloud dev enrionments because in that we can not give 100 GB RAM, 2 ai cpu like hailo, or gpu access to run llm locally for dev purpose etc

scarlet badge
#

we want to orchestrate other things with dagger like creating pod or services, etc

You don't need Dagger for this or rather, that's not what Dagger was built to actually do. You can do it. It just doesn't make real sense IMHO. If the cluster is already running, and you can use the API client directly to run k8s commands programatically to create the dev environment.

Another solution to "program" the internal setup of the cluster for your purposes would be Terraform and the k8s provider.

I'd also imagine you have specialized workstations for your development. 100GB RAM isn't "normal" and neither is having Hailo available either. 🙂

I would still bet you all would need a CDE, or a way to program "inside the cluster". And, I'd even go so far as to say, you should be creating a mini-data center (locally) for your dev work, so you don't have to give each dev a huge workstation. That local k8s cluster would have the extra connected AI systems and a lot of processing power and RAM. 🙂

dull cedar
scarlet badge
#

It's really up to you. With the client, you are going to have to basically create your own CLI, which is what kubectl is. So, you are reinventing a wheel. If you wish to run multiple kubectl commands, like to orchestrate a whole dev setup in k8s (i.e. creating pods with certain images, creating services, etc.), you could also just use Node's exec function and call the kubectl commands programatically.

#

Or, you could create helm charts.

#

Or use Tilt.

#

Or, have a smaller set up to only install ArgoCD and run a deployment from a particular Github repo.
The list can go on.

dull cedar
#

ok got it but if we dont want to go in hell of yml then client library is the best option right?

#

and by using client library for k8 cluster orchestration we may not required helm also, right?

#

can we use dagger to spin up and spin down dev containers in k8 custer and then developers will be able to connect that dev containers via ssh and they will be able to work with that dev container in vs code through ssh/remote feature of vs code, is it good idea or anything we are missing or can be improved?

scarlet badge
#

ok got it but if we dont want to go in hell of yml then client library is the best option right?

I think you really need to change your attitude about YAML, or rather in working with configuration to work with k8s, in general. I'm not a fan of the declarative nature of running k8s clusters either, but it is how k8s works. You can work with the API client to program a workflow to create your development setup for sure. But, the knowledge you need in terms of configuration (in YAML, in JSON in JavaScript or whatever) is still needed. The problem isn't YAML. The problem is knowing for sure what configuration affects what outcomes in a cluster. You'll run into the same exact problems in JavaScript as you have in YAML.

and by using client library for k8 cluster orchestration we may not required help also, right?

I have no idea what you are asking here. Sorry.

can we use dagger to spin up and spin down dev containers in k8 custer(?)

I believe in theory, yes. But, it's an unnecessary and even wasteful level of abstraction, if you are using the API client.

and then developers will be able to connect that dev containers via ssh and they will be able to work with that dev container in vs code through ssh/remote feature of vs code, is it good idea or anything we are missing or can be improved?

In theory yes, but you don't need Dagger to get the dev container running.

dull cedar
#

it was typo for this question
here is correct question, sorry

and by using client library for k8 cluster orchestration we may not required helm also, right?

#

ok got it when clint can spin up containers then why we should add dagger, agree got it, thanks

scarlet badge
#

and by using client library for k8 cluster orchestration we may not required helm also, right?

Also, theoretically, yes. But, you'd be reinventing the Helm wheel with your usage of the API client.

#

Also, just a word of advice. If you wish to show respect to someone trying to help you, you should try to write as best you can i.e. proper grammar and sentence structure and comprehendable sentences.

dull cedar
scarlet badge
#

No trouble. Just can be a bit annoying. 🙂

dull cedar
scarlet badge
#

Yes. Absolutely. Also there is a ton of Helm charts in the wild, as it is the defacto standard for application/ service setup inside a cluster. And with all these "examples" out there, you can learn a lot.

dull cedar
#

hmm right, agree. Thanks for help till to the conclusion.

scarlet badge
#

👍🏻 Good luck. Btw, when you have your dev environment running and need to get the dev process going in terms of CI, that is when you can use Dagger to automate testing, building etc.

dull cedar
#

thank a lot for wish. And yes our one of milestone is using open sourced llm in dev container by running it as service on k8 cluster (k8 native service or through dagger) and injecting it through dagger into dev container for developers use...may be i am sounding crazy

scarlet badge
#

Dagger is either standalone or it is inside a cluster as a service. You shouldn't be using Dagger to interface with a cluster. I'm not sure where you are getting this impression, but it is incorrect.

dull cedar
#

haha okay thanks for clarity

#

but services running in k8 cluster can't support each other? or by doing everything with dagger start to end to support this vision, like starting dev container, llm service etc? just thought

scarlet badge
#

but services running in k8 cluster can't support each other?

I don't know what you are asking.

or by doing everything with dagger start to end ma support this vision

Dagger isn't a tool to do anything in a cluster in terms of k8s management per se. You can do it, but it makes zero sense. There are many, many, many other better tools for this job. Again, you seem to have some sort of incorrect impression about what Dagger is meant to be or do.

dull cedar
#

alright got it. thanks a lot for eveything and your time.

scarlet badge
#

👍🏻

#

It's more about IaC, but k8saC is just a level below it ( or above it, depending on how you see abstractions).

#

And some of those tools can also configure k8s cluster resources, like Terraform.

#

But, Helm is the standard.

dull cedar
#

ook got it, will study this now. thanks again

scarlet badge
#

Check out the repo here @dull cedar. It might help you better picture where Dagger should be positioned and used in a dev k8s setup. If you notice, he is using Flux to "reconcile" i.e. deploy applications into the cluster. He is only using Dagger in the CI steps to get the changes made.

#

And this is just one of many ways Dagger could be used. The key thing being, he isn't using Dagger to manipulate the cluster directly. He has other tools to do that.

dull cedar
#

First of all, thank you so much for helping me till last dot to connect. @scarlet badge
Sure I will check this and we'll try to understand how we can leverage dagger as much as we can while dealing with k8 cluster.

#

Can I have repo link?

scarlet badge