#tools-and-devops

1 messages · Page 18 of 1

rapid sparrow
#

Pick yourself ^_^

lyric hemlock
#

Ok

rough mauve
#

Hi there. Currently researching Synology NAS. Does the DSM OS have any out-of-the-box automated image backup system?

I’d like to dedicate a partition in my drives for daily backups up to the last week or so.

remote sand
#

any way to compare package versions across multiple distros? any website that does that or something

#

packages in general, not necessarily python

astral apex
#

Every distro has their own package format, so they’d all look completely different

deep estuary
astral apex
#

But in general they’d still be pulling from the same Git repo for the source, so the actual code would be garunteed to be exactly the same

#

… unless they added their own patch I guess

deep estuary
astral apex
deep estuary
#

lmfao what

#

where

#

i see none!!

astral apex
#

Okay now that I clicked on your second link I’m not getting any

#

But after I clicked on the first one, every click I made, I had to close a full page add before I could continue

deep estuary
#

lmfao mental

#

it's nice though you can get loads of details on packages

remote sand
#

nice ty!

crystal atlas
#

do anyone know about best online AI/ML master course in india ..... ple suggest me

torn sparrow
#

Guys, how can I check if multiple packages (with the specified versions) are compatible? I have been trying to brute force it with just trying and see if they are, but I've been waiting for 3 hrs now for it to finish pip install -r requirements.txt.

worthy robin
#

So im trying to run this bit of powershell https://paste.pythondiscord.com/JERA in CIBW_BEFORE_BUILD_WINDOWS: in a ghactions script, but it just errors with { was unexpected at this time.

# 
Running before_build...
  + PowerShell -Command "&{ if (![System.IO.File]::Exists('./run-check')) {
    git clone https://github.com/microsoft/vcpkg.git;
    cd vcpkg;
    bootstrap-vcpkg.bat;
    cd ..;
    ./vcpkg/vcpkg.exe install pkgconfig;
    echo 'INSTALL ASSIMP';
    ./vcpkg/vcpkg.exe install assimp:x64-windows-static;
    echo 'INSTALL GLM';
    ./vcpkg/vcpkg.exe install glm:x64-windows-static;
    echo 'INSTALL SDL';
    ./vcpkg/vcpkg.exe install sdl2:x64-windows-static;
    New-Item -Path './run-check' -ItemType 'File';
  } }"
  Missing closing '}' in statement block or type definition.
      + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
      + FullyQualifiedErrorId : MissingEndCurlyBrace
``` I changed the command a little, but it still seems to be throwing errors
remote sand
#

what could i use on linux to annotate my screen? am videoconferencing my code

alpine horizon
#

I have two GH workflows: one runs on push to master, and the second is schedule for everyday
is there a way that I can trigger one to run from tihe other workflow?

#

and if so, can I trigger just a single job from the workflow?

rapid sparrow
rapid sparrow
# alpine horizon and if so, can I trigger just a single job from the workflow?

https://github.com/darklab8/fl-darkstat/blob/master/.github/workflows/deploy.yml

name: CI

on:
  push:
    branches:
    - "master"
    - "dev"

jobs:
  github-pages:
    name: Github Pages
    runs-on: ubuntu-22.04
    permissions:
      pages: write
      id-token: write
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - name: Check out source repository
        uses: actions/checkout@v2

      - uses: darklab8/fl-configs/.github/actions/checkout-freelancer@master
        with:
          freelancer-mod: "discovery"
          freelancer-folder: ${{ github.workspace }}/fl-data
          ssh-key-base64-discovery: ${{ secrets.ID_RSA_FILES_FREELANCER_DISCOVERY }}
     # other CI code.

You have two choices.
You can retrigger only specific CI step. In this case u need to put your reusable code logic into "action"

#

Alternatively if it is entire CI job like

#

just put it into separate file executable onto Workflow Call

jobs:
  ci-run:
    name: Run the CI suite
    uses: Organization/RepoName/.github/workflows/docker-pytest.yml@master
#

Workflow example with arguments
.github/workflows/docker-pytest.yml

name: Download the build image and run the tests against it

on:
  workflow_call:
    inputs:
      docker-compose:
        description: 'Path of the docker-compose.yml file'
        default: 'docker-compose.yml'
        type: string
    secrets:
      DOCKERHUB_USERNAME:
        required: true
      DOCKERHUB_TOKEN:
        required: true

jobs:
  unit-test:
    name: Run pytest
    runs-on: [self-hosted, linux, x64]
    permissions:
      id-token: write
      checks: write
      contents: write
      issues: read
      pull-requests: write
    steps:
      - uses: RepoName/actions/get-name@v5
        id: name
        if: ${{ !inputs.service }}
      - name: Check out source repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
     # the rest of CI code
#

you can reuse it then and call both as Job inside primary CI, and as everyday cron jobbed one

#

Both options can work. Having reusable Actions or reusable Workflow

alpine horizon
#

there is a Python script that is executed in the workflow that I want to keep its source private.
is there a way to do this? perhaps I can put it in a private repo along with the workflow that triggers it, and call that workflow from a public repository (of the same organizatio/user) ?

rapid sparrow
#

the regular means to obfuscate code essentially. utilize obfuscation/compilation/or write in language that is actually compilable

rapid sparrow
#

=> having workflow calling code logic at remote server 😄

#

at the cost of keeping your server power, it will remain 100% secret

alpine horizon
#

yeah I might actually do it in Rust and upload the binaries, but I will have to change some stuff to make it work

#

are you sure there is no other way to call a private workflow from a public repo of the same user?

rapid sparrow
# alpine horizon is there some cheap cloud computing you recommend?

Hetzner is ultra cheap quality i use for normal servers. Single server powers me everything, minecraft, bots, microk8s cluster.
Use it for reliable solution that will not go over limits of your payments. Regular hourly payment strategy.
https://www.hetzner.com/

Otherwise if you are feeling brave and wish it to be extra cheap, with thinking it will not be called more than free tier provides
https://aws.amazon.com/lambda/pricing/

The monthly request price is $0.20 per one million requests and the free tier provides 1 million requests per month.
you could make it running technically for free through AWS lambda calls.
But don't joke with not setting limits/alerts for pricing consumption 😄 otherwise u can receive payment charge more than your home costs.
Rate limiting is heavily encouraged for public services to set, in order to avoid surprises.

Dedicated Server, Cloud, Storage & Hosting

Your partner for system-wide hosting, from cloud to dedicated servers. We deploy the latest tech at the best price in minutes.

Amazon Web Services, Inc.
Serverless Computing – AWS Lambda Pricing – Amazon Web Services

With AWS Lambda, you pay only for what you use. You are charged based on the number of requests for your functions and the time your code executes.

alpine horizon
#

that already happened to me with AWS

#

I need to open a new account to try it again

#

I've been seeing hetzner ads all over YT lately...

rapid sparrow
#

They are great. I like them ^_^ German quality
Terraform compatibility, API access
Quick creation and destruction
Ultra cheap prices (especially for arm64)

alpine horizon
#

other than AWS, what other cloud providers have a decent free tier? (I wont be using more than 20 mins of CPU time a month)

rapid sparrow
#

like, 4 times more hardware power than you could get from Digital Ocean for same price 😄

rapid sparrow
#

Digital Ocean too

#

probably Azure

alpine horizon
#

I hate GCP

#

will try DO

rapid sparrow
#

than more complex provider => than longer their free tier as honey trap 🙂

#

DO is simple

#

but free tier for month or 2-3 months, not remembering exactly

alpine horizon
#

nvm

rapid sparrow
#

otherwise yeah... AWS lambdas or their equivalent in other providers is potential free power. free with great caution.

rapid sparrow
#

but otherwise it is more friendly to modern frontend js devs

#

some room is getting opening to us mortals (in other languages living) than more mature web wasm becomes though

alpine horizon
rapid sparrow
#

u can run locally if u wish.

#

Self hosted runners use for yourself (we use them for organization to have Github for private repos for free ^_^)

#

optionally there is nektos/act program to debug locally

alpine horizon
#

I will think about it, but so far the simplest and most free solution is save the code in a private repo, and then add a workflow that compiles and obfuscates it, and uploads the binaries to the public repo

#

I was thinking about it, how would you go about doing that?
was thinking of storing a "secret string" inside the code, and when the public repo calls the binary, it passes the token as an argument (which is stored a GH secret)

rapid sparrow
#

well, forget that i said that

#

it sounds like against TOS

#

and immoral, with against culture of FOSS

alpine horizon
#

I highly doubt it is

rapid sparrow
#

ergh, immoral people never doubt their dishonest ways 😄 they think they do things right

alpine horizon
rapid sparrow
#

trust me, it is immoral.

#

u mentioned yourself that it will be closed source. Therefore it is not open source

alpine horizon
alpine horizon
rapid sparrow
#

i don't like the direction of our conversation. in my opinon we go against TOS waters, so go on without me to that

rapid sparrow
alpine horizon
rapid sparrow
#

then it is your private service that remains private

#

abusing public free hardpower for that is kind of bad. potentially will lead to your account termination if that will be discovered in large amount

#

public free Github Runners were made free for open source

#

yours is not

#

therefore you will be abusing it

#

simple as that

deep estuary
#

🤨

rapid sparrow
#

fairly technically...

#

...there is an ultimate solution to this problem

rapid sparrow
alpine horizon
#

by your logic having a GH actions workflow in a private repo is against TOS, because "public free Github Runners were made free for open source" and nobody benefits from it

#

bruh you need to calm down with this mod-behaviour

alpine horizon
deep estuary
#

let's not turn this into a useless argument

deep estuary
placid glen
#

could someone explain to me how am I supposed to handle users, connections, and elevated rights while automating stuff through ssh ?

#

I'm a bit lost: apparently I shouldn't run commands as root or sudoer (and most tools don't handle it anyway), but I feel like most tasks needs elevated rights

#

for example starting a container, or copying a config file or secrets

static solar
#

send me a site to do beginner programs and exercices

placid glen
static solar
#

thanks

static solar
#

any other one?

placid glen
#

bro, use google

#

leetcode

#

codewars

fleet dew
#

Hey guys!

I’d like to introduce you to my indie product Apitally (https://apitally.io), a simple API monitoring & analytics tool for Python projects.

Apitally provides insights into API traffic, errors, and performance, for the whole API, each endpoint and individual API consumers. It also monitors API uptime, alerting users when their API is down.

The big monitoring platforms (Datadog etc.) can be a bit overwhelming & expensive, particularly for simpler use cases. So Apitally’s key differentiators are simplicity & affordability, with the goal to make it as easy as possible for users to understand usage and performance of their APIs.

Apitally directly integrates with various web frameworks (FastAPI, Django, Flask, Litestar) through middleware, which captures request & response metadata (never anything sensitive!) and asynchronously ships it to Apitally’s servers in regular intervals.

Please let me know what you think!

Screenshot_2024-06-11_at_9.27.53_PM.png
astral apex
#

!rule ad

rancid schoonerBOT
#
Rules

6. Do not post unapproved advertising.

placid glen
#

Hi everyone.
About automating stuff through ssh and best practice in terms of security, here's where I am so far:

  • have a dedicated user for automation, with no passwd (but with ssh pubkey check)
  • give precises rights to that user using permissions and the sudoers file
  • don't store anything secrets in the docker compose files. Secrets needs to be handle in another way (for example using docker secrets)

Here's my questions:

  • how to deploy things that are inherently secrets ? For exemple, https certificates, or config files with secrets in it
  • how can I allow a user to use docker container create or docker container stop, but not say docker secrets or docker exec ?
  • related to the previous question: Do I need to create scripts with rx rights for the automation user to use
rapid sparrow
# placid glen Hi everyone. About automating stuff through ssh and best practice in terms of s...

Fair warning. Using Docker Swarm is discrouraged and that's not the thing where main community support. Kubernetes is recommended default for container orchestration.

how to deploy things that are inherently secrets ? For exemple, https certificates, or config files with secrets in it
Okay, lets imagine it was me who was doing that.

  1. I would store your cert https_certificates, and secret values in AWS Secret Manager/SSM manager or for self hosted Hashicorp Vault is an option. For the purpose of not storing secrets in git repo, and having programmatic access to them. Configurational not secret fiiles will not be stored in it! They are fine to keep in git repository directly.
  2. i would connected my Docker access over SSH port forwarding
provider "docker" {
  host     = "ssh://user@remote-host:22"
  ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null"]
}
  1. Applied necessary secrets/configs/services creations with terraform. It automatically would patch existing things or remove outdated no longer needed stuff
    https://registry.terraform.io/providers/kreuzwerker/docker/latest/docs/resources/secret
    The end of story
Terraform Registry
#

how can I allow a user to use docker container create or docker container stop, but not say docker secrets or docker exec ?
https://docs.docker.com/engine/swarm/configs/ default docker secret feature. works only for swarm services

#

related to the previous question: Do I need to create scripts with rx rights for the automation user to use
API documentation does not hints it being needed

placid glen
rapid sparrow
placid glen
#

I don't think I do

#

I'm deploying docker compose scripts, and use docker commands

rapid sparrow
# placid glen I don't think I do

You do. Docker Secrets/Config/Servces are Docker Swarm (Container orchestration) only things
that is already beyond regular docker

placid glen
#

oh ok

#

this rabbit hole never ends 😄

rapid sparrow
#

You are entering entire WORMHOLE of problems tbh

#

Container scheduling in self hosted way... urgh.

placid glen
# rapid sparrow ^_^

I'm seriously considering just using portainer to deploy stuff and forget about repeatability

#

I mean, not automating shit, and just do everything manually

rapid sparrow
#

next devs to support it will be screwed

placid glen
#

it's personal it's not really an issue

rapid sparrow
#

ah, then do anything u wish 😅

placid glen
#

also it's not a bug, it's a feature in job security

rapid sparrow
#

personally i use for personal usage just Docker Terraform and nothing else

#

no swarm

#

comfortable enough

placid glen
#

still with AWS ?

rapid sparrow
#

nah, Hetzner VM

#

ARm64 is dirty cheap there

placid glen
#

so through ssh ?

rapid sparrow
#

i run microk8s in addition, to experiment if it will be worth it, may be may be. its ecosystem offers really easy to configure monitoring. Somewhat hesitant but very tempting

placid glen
#

and without docker compose

rapid sparrow
#

in general it runs on terraform though without compose

placid glen
#

does that mean

#

you're creating a custom docker image for each app ?

rapid sparrow
#

minecraft server is not yet refactored to this approach... but i will... because i am afraid i will not be able to relaunch it in a year if i will not do it (very fragile amount of dependencies inside this image, that are potentially not rebuildable in a year)

placid glen
#

it's slowly starting to make sence

#

a whole repos with lots of infra project ?

#

don't mind if I do :3

rapid sparrow
# placid glen a whole repos with lots of infra project ?

... i am influenced in how we do stuff at work
https://github.com/darklab8/infra single repo to create cluster + i use it for reusable infra code
Dedicated projects like discord bot import my reusable code from the repo and apply their own specifics to run apps
https://github.com/darklab8/fl-darkbot/blob/master/tf/production/main.tf
Except at work we publish under git tag reusable modules and lock to specific version. for my pet projects i am lazy to do it and prefering using always latest code

module "ecr" {
  source = "git@github.com:Company/tf.git//modules/ecr?ref=v0.102"
  name   = "prometheusian"
}

reusage at work properly.

me lazily at pet projects

module "server" {
  source = "../../../infra/tf/modules/hetzner_server/data"
  name   = "node-arm"
}
placid glen
#

so basically, from the ground up:

  • I create my custom image, with the right config, the right everything
  • I upload it
  • I use terraform just to pull and deploy said image
  • and voilà
rapid sparrow
placid glen
#

yes I've seen that, but I still need to figure everything out about that

#

the docker I've seen example used redis to manage secrets, and I don't know redis

rapid sparrow
#

😅 my pet projects are written with a switching interchangability between Docker running and Kubernetes running

#

in case my kubernetes will fail again and i will be lazy to maintain it... i fallback to good old docker (because i have real users depending on them, i need somehow to run them)

#

Terraform code is flexible enough to allow me reusing infra code for both solutions

placid glen
#

btw, I means your automation user has the right to run kubernetis/docker, right ?

rapid sparrow
#

yes. i just use root users at servers for connection over ssh. i am lazy person

provider "docker" {
  host     = "ssh://root@${module.server.ipv4_address}:22"
  ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", "-i", "~/.ssh/id_rsa.darklab"]
}
placid glen
#

if that user is hacked what prevents the hacker to simply connect to a container and steal the secrets from the env ?

rapid sparrow
placid glen
#

true

rapid sparrow
#

kind of tough to do.

placid glen
#

I guess it leaves only the physicial attack, but yes

rapid sparrow
placid glen
#

I see

rapid sparrow
# placid glen I see

modern approach to security for such usage cases usually running at most Docker containers in locked file system with non root users

#

mitigating damage to single container only compromised

placid glen
#

make sence

#

hence the rootless docker runner I've seen earlier

rapid sparrow
#

Tbh i learned my lesson i think how to make minecraft docker-compose running easily via terraform

#

i never should have written docker-compose in the first place

#

i should have written terraform from the start

#

and added easy approach to debug it locally (simple enough to do)

#

that would be a good way to keep everything terraformed in a simpliest way with no code duplications or translating effort

remote glen
#

@placid glen sorry if I'm late to the party and not addressing all of your question, but the gist I'm getting is that you want to deploy a container with some secrets and maybe some scaling. IMHO Google Cloud Run is pretty easy and reliable and even "free" most likely if it is an app "just for you."

no swarm or compose or anything. just whatever container you have working, push it up there, and it works. secret management would be like using env variables which are set one way on your computer but then in Cloud Run defined separately (so to use one database locally but another when on Cloud Run as an example)

placid glen
remote glen
harsh panther
# 
          # Set commit status using GitHub API
          curl -X POST \
            -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
            -H "Accept: application/vnd.github+json" \
            -H "X-GitHub-Api-Version: 2022-11-28" \
            https://api.github.com/repos/${{ github.repository_owner}}/${{ github.repository }}/statuses/${{github.sha}} \
            -d "{\"state\": \"${state}\", \"description\": \"${description}\", \"context\": \"Testing the API\"}"

why this keep failing

# 
Run state="success"
  state="success"
  description=$(if [ "success" == "success" ]; then echo "All tests passed successfully!"; else echo "Tests failed. Check logs for details."; fi)
  
  # Set commit status using GitHub API
  curl -X POST \
    -H "Authorization: ***" \
    -H "Accept: application/vnd.github+json" \
    -H "X-GitHub-Api-Version: 2022-11-28" \
    https://api.github.com/repos/Anubhav1603/Anubhav1603/Alpha-Demons-API/statuses/158834f6b6fb2d9d24f512b8178b426b349acc7c \
    -d "{\"state\": \"${state}\", \"description\": \"${description}\", \"context\": \"Testing the API\"}"
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.18/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.18/x64/lib
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   202  100   103  100    99    501    481 --:--:-- --:--:-- --:--:--   985
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest",
  "status": "404"
}
thorny shell
#

make sure that the -d flag to curl means what you think it means -- it might expect data to be in key-value format

harsh panther
# 
url -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST --data "{\"content\": \"Tests failed on branch '${{ github.ref_name }}'.\\n\\n$(grep -E 'FAILED' pytest_output.log | sed -E 's/FAILED ([^ ]+)::([^ ]+).*/\1\\\n\2/')\"}" "${{ secrets.DISCORD_WEB_HOOK }}"

It's so annoying the tests which fails get printed in terminal well of GitHub actions but I am getting constant error in above command from discord for format of json isn't correct ,don't know why

gentle solstice
thorny shell
tall meadow
#

Does anyone know how to have two separate sets of ruff linting rules in the same pyproject.toml? I'd like to have one light check for the CI and one comprehensive ruleset for local linting

thorny shell
#

off the top of my head, that sounds like something you should control via the command line, not the toml file

tall meadow
#

yea you're right, I just realized I can do --select ALL or any specific rule

#

it overrides the toml config

heavy knot
#

I want to do the following:
get the server running while I am offline
get the server better security
change the http/url to be more user friendly
whichever one you feel comfortable helping with is ok : ]

image.png
indigo zenith
heavy knot
indigo zenith
harsh panther
# 
            organized_failed_tests=$(echo "${failed_tests}" | awk '
            {
              if ($1 ~ /\.py$/) {
                if (file != $1) {
                  if (file != "") printf "\n"
                  file = $1
                  printf "%s\r\n", file
                  if ($2 != "") printf "%s\n", $2
                }
              } else {
                if ($1 != "") printf "%s\n", $1
              }
            }
            ' | sed '/^\s*$/d')

this is what i am getting

tests/test_example.py
test_read_example
tests/test_root.py
test_read_main
test_read_mains

this is what i want

tests/test_example.py
test_read_example

tests/test_root.py
test_read_main
test_read_mains

for some reason new line isnt getting added

gentle solstice
#

What's the input?

thorny shell
#

my rule about shell scripts is: if they grow to more than (say) three lines, I replace them with a python program 🙂

stable plover
#

Hello I want to create a tool to convert PDF files with tables with no borders to excel using python how can I do that?

gentle solstice
#

Maybe I should get this built into the code-cli

trail star
#

Hi guys, I've been trying to make a pyhton tool with which i can save all currently opened opera-gx tabs as shortcuts on my pc. i tried asking chatgpt but the code doesn't work (even after a lot of back-and-forth and changing the code) can someone help me? This is the code i am currently trying to run:

from selenium.webdriver.chrome.service import Service

opera_driver_path = 'C:\\cool pictures\\python programs\\operadriver_win64\\operadriver.exe'

opera_path = 'C:\\Users\\tobia\\AppData\\Local\\Programs\\Opera GX\\launcher.exe'

options = webdriver.ChromeOptions()
options.binary_location = opera_path

service = Service(opera_driver_path)
driver = webdriver.Chrome(service=service, options=options)

urls = []
for handle in driver.window_handles:
    driver.switch_to.window(handle)
    urls.append(driver.current_url)

for url in urls:
    print(url)```

However if i run this code i get the following error:

```PS C:\Users\tobia> & C:/Users/tobia/AppData/Local/Programs/Python/Python311/python.exe "c:/cool pictures/python programs/tab-saving-automation.py"
Traceback (most recent call last):
  File "C:\Users\tobia\AppData\Local\Programs\Python\Python311\Lib\site-packages\selenium\webdriver\common\driver_finder.py", line 64, in _binary_paths
    raise ValueError(f"The path is not a valid file: {path}")
ValueError: The path is not a valid file: C:\cool pictures\python programs\operadriver_win64\operadriver.exe```
grave canyon
#

Does anyone know any ai tools to edit videos and text to speech

drowsy frigate
#

anyone has Linux resources

indigo zenith
sturdy seal
#

So google was very dissapointing in this case because it always talks about development, but not what comes after. Our setup probably isn't conventional which complicates things.
So right now, we have a linux machine we call the "Business Box" where all of the COBOL and Python programs reside. All of the Pythons are written in 3.9, and we can't upgrade because one of the modules breaks at 3.10 due to the connector or something. It came up that people use virtual environments (my school skipped over these) for developing programs, but it was my understanding that once a project is finished, we're supposed to move it from dev to the overall production environment.

#

The question I asked in the wrong channel was about getting clarification on the use of virtual environments in production rather than a single production environment.

#

Forgot to mention that users access these programs through a separate menu system (COBOL) through Putty sessions.

rapid sparrow
#

you created your dev venv and have it activated
stuff is installed to develop. You write manually primary dependencies into requirements.txt. Optionally they can be having specified minimal necessary version or required one

#

you do pip freeze > constraints.txt to lock all installed libraries to exact versions

#

when other devs develop from other machine (or u deploy to prod) => you are supposed to install dependencies as pip install -r requirements -c constraints.txt (for activated venv lets say)

#

this makes dependencies reproducable

#

because u will have installed exact dependencies as your dev environment

#

we at work utilize pipenv that handles this manual dependency management for us into automatic

#

other people utilize poetry commonly today

#

those utilities manage Dependency file and its Lock file for you

#

you will be required just to point what to install and what to upgrade

#

in my minimalistic projects i am lazy and just use pip install -r requirements.txt -c constraints.txt, because it can work without any extra stuff installed for that

rapid sparrow
#

for Linux there is one instruction, for Windows another one

#

for linux i just use

  • python3 -m venv .venv (optionally i use python3.10 or some other exact python version to create specific venv version) (it works as long as i have installed apt install -y python3-venv or some other python specific version venv, like python3.10-venv
  • source .venv/bin/activate activates it for current shell. You can use it without activation for programmatic purposes. .venv/bin/python3 your_program.py will make venv used without activation
gentle solstice
rapid sparrow
sturdy seal
#

Thank you for the pointer. I'm confused about constraints.txt, though. Doesn't requirements.txt set the version?

#

Because when I use them, with the exception of the trouble module, I edit it from == to >= so I get the newest ones.

#

Otherwise I'm unable to upgrade the individual packages. (Packages? Modules? Libraries?)

rapid sparrow
#

that allows keeping dependencies in rqeuirements.txt only main ones, without specific version or only with more than specific version and etc. helping easier keep track what are main deps and removing them when necessary

#

in this strategy requirements.txt is human written. constraints.txt is autogenerated with pip freeze or other tool (preferably your CI validates that u keep it up to date)

sturdy seal
#

I see, so the requirements would only name the library, but wouldn't have the == version?

rapid sparrow
sturdy seal
sturdy seal
#

@rapid sparrow Do you ever run into size issues when having several venvs?

rapid sparrow
sturdy seal
#

And then that same size in production?

#

Or do you move from the dev machine to production machine

rapid sparrow
# sturdy seal And then that same size in production?

size in production can varry from hundreds of megobytes to 2 GBs i think (2-4 gbs is probably fatest service we have)
but that's Docker image packaged with all dependencies for its running (OS level included, compilers and etc), and we never used technics to downsize final layer
There was never need for that. It is very small size to care for us

sturdy seal
#

I see. We don't use docker here. The plan is to develop the stuff and then reclone it to the production server and build the venv there, then remove it from the dev server until we need to edit or test or whatever.

rapid sparrow
#

Container scheduling clusters pull Docker image for its running in AWS ECS and Kubernetes clusters

sturdy seal
#

Ah. Our stuff all runs on a linux machine with putty sessions for each user.

rapid sparrow
#

Sounds very weird stuff 😅

sturdy seal
#

Probably is. But it's my first job after college, so I don't really have anything to compare to, experience wise.

molten cradle
#

Hey

#

Anyone knows about ai scraping, flutter and jungle

modest depot
#

can someone help me

astral apex
#

With?

modest depot
astral apex
fringe aspen
alpine horizon
#

if I have two GH actions workflows: test_and_update_examples.yml and publish_pypi.yml, how do I make it so it publishes only if the tests don't fail?

rapid sparrow
#

things like that to test workflow

on:
  pull_request: {}
  workflow_call:
#

and to publish beginning:

jobs:
  ci-run:
    name: Run the CI suite
    uses: Company/RepoName/.github/workflows/ci.yml@master
    secrets: inherit
limber grail
# 

FROM  python:3.11.4-bookworm

WORKDIR /app

COPY requirements.txt requirements.txt 

RUN pip3 install -r requirements.txt 

COPY . .

CMD [ "python3", "manage.py", "runserver", "0.0.0.0:8000"]
``` I keep getting this error ``` ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt" did not complete successfully: exit code: 1
#

I created multiple images and deployed to productions before. Just forgot what is is all about.

indigo zenith
limber grail
#

Yeah. just wondering which dependency that is 'cause I did pip freeze > requirments.txt to get the same depednecies that I used in the app.

rapid sparrow
indigo zenith
plucky sparrow
#

Do you guys run docker on wsl on windows 11

gentle solstice
#

I run podman on wsl

indigo zenith
plucky sparrow
#

Thanks

rapid sparrow
#

otherwise straight using from Linux

dim eagle
#

anyone know if there's a github action to do:

poetry export --without-hashes --without-urls | awk '{ print $1 }' FS=';' > requirements.txt

on poetry.lock updates?

I don't know how to write github actions this is why i Make this thread today.

gentle solstice
# 
jobs:
  export:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v4
        with:
          python-version: "3.12"
      - run: pip install poetry
      - run: poetry export --without-hashes --without-urls -o requirements.txt
      - uses: EndBug/add-and-commit@v9
        with:
          message: Export requirements.txt
dim eagle
# gentle solstice ```yml jobs: export: runs-on: ubuntu-latest steps: - uses: actio... 
name: Update requirements.txt

on:
  push:
    paths:
      - poetry.lock

jobs:
  update-requirements:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'  # Specify the Python version you need

    - name: Install Poetry
      run: |
        curl -sSL https://install.python-poetry.org | python3 -
        echo "::add-path::$HOME/.local/bin"

    - name: Install dependencies
      run: poetry install

    - name: Export dependencies to requirements.txt
      run: poetry export --without-hashes --without-urls | awk '{ print $1 }' FS=';' > requirements.txt

    - name: Show generated requirements.txt
      run: cat requirements.txt

someone sent me this

gentle solstice
#

This doesn't do the commit

#

Use the above link to do it

dim eagle
#

It doesn't?

#

how would I modify this to do the add commit?

gentle solstice
#

Why do you need the awk?

dim eagle
#

I found the output before

# 
aiodns==3.2.0 
aiofiles==0.8.0 
aiogifs @ git+https://github.com/moanie/aiogifs.git@730fdb7bca18842db93b71cf0c50b93f9cc3c31a 
aiohttp==3.9.5 
aiosignal==1.3.1 
aiosqlite==0.17.0 
discord-games @ git+https://github.com/Tom-the-Bomb/Discord-Games@585e0dddb24aefbcc16569a2f7db44d1d194107a 
discord-py @ git+https://github.com/Rapptz/discord.py.git@3a8063602919813e91d19101f9b5a0e9e7438d4b 
discord-py-paginators @ git+https://github.com/Soheab/discord-py-paginators@3ed81f3bbd6637c91625cddb9568b5f4d21f832a 
discord-py[speed,voice] @ git+https://github.com/Rapptz/discord.py.git@3a8063602919813e91d19101f9b5a0e9e7438d4b
#

it outputs nicely like this

gentle solstice
#

What does it print without it?

#

Why are all your deps from git?

dim eagle
#

my deps are somewhat from github
but mostly static from pypi

#

anyway awk is needed to do the formatting

gentle solstice
#

It shouldn't. poetry export should create a file that can be consumed by pip

dim eagle
#

the format I received stops hashes and funky other junky

gentle solstice
#

That's what --without-hashes is for

dim eagle
#

I am also trying to avoid python_version junk too

gentle solstice
#

the ; in the requirements creates constraints for each dependency, like platform support.

#

Then just target a single version in tool.poetry.dependencies.python

#

or target latest

dim eagle
gentle solstice
#

Add this to the end of your steps ```yml
- uses: EndBug/add-and-commit@v9
with:
message: Export requirements.txt

dim eagle
# 
name: Update requirements.txt

on:
  push:
    paths:
      - poetry.lock

jobs:
  update-requirements:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'  # Specify the Python version you need

    - name: Install Poetry
      run: |
        curl -sSL https://install.python-poetry.org | python3 -
        echo "::add-path::$HOME/.local/bin"

    - name: Install dependencies
      run: poetry install

    - name: Export dependencies to requirements.txt
      run: poetry export --without-hashes --without-urls | awk '{ print $1 }' FS=';' > requirements.txt

    - name: Show generated requirements.txt
      run: cat requirements.txt

    - uses: EndBug/add-and-commit@v9
      with:
        message: Export requirements.txt
#

Like that?

gentle solstice
#

yes. Add a name if you like

dim eagle
#

I think it is fine as is

#

I just named the actual file

#

auto-poetry

#

Time to test it with an updated poetry.lock

dim eagle
gentle solstice
#

it failed to install poetry

dim eagle
#

because of some env thing right?

gentle solstice
#

Replace this ```yml
- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "::add-path::$HOME/.local/bin"

With this ```yml
    - name: Install Poetry
      uses: snok/install-poetry@v1
      with:
        version: 1.5.1 # insert your poetry version here
dim eagle
#

rooThink What's the action do?

gentle solstice
#

You can probably skip installing the package

#

So you don't need to do poetry install

dim eagle
#

I see

gentle solstice
#

You sent an empty commit

#

an empty commit does not include poetry.lock

#

@dim eagle

dim eagle
#

I see

#

I can try seeing if it works when I update one of the packages it relies on

gentle solstice
#

Do git refactor and a force push

dim eagle
#

What's a git refactor?

gentle solstice
#

rebase I mean

dim eagle
#

Idk really know what a rebase is

gentle solstice
#

you can use git rebase -i HEAD^^^ to rewrite the history.

#

it will generate a new commit, which you'll have to git push -f

#

You should do it in a new branch btw

dim eagle
#

?

gentle solstice
#

I guess you could also just revert the original commit then redo it

dim eagle
gentle solstice
#

looks like it works

dim eagle
# gentle solstice looks like it works

I do worry about:

Warning: poetry-plugin-export will not be installed by default in a future version of Poetry.
In order to avoid a breaking change and make your automation forward-compatible, please install poetry-plugin-export explicitly. See https://python-poetry.org/docs/plugins/#using-plugins for details on how to install a plugin.
To disable this warning run 'poetry config warnings.export false'.

Plugins | Documentation | Poetry - Python dependency management an...

Plugins Poetry supports using and building plugins if you wish to alter or expand Poetry’s functionality with your own.
For example if your environment poses special requirements on the behaviour of Poetry which do not apply to the majority of its users or if you wish to accomplish something with Poetry in a way that is not desired by most users...

gentle solstice
#

snok/install-poetry#53

signal cloakBOT
gentle solstice
#

So just run poetry plugin add export I think

#

try it locally

dim eagle
# 
poetry self  add export                                                                                                                                                              [11:32:51]
Using version ^0.2.1 for export

Updating dependencies
Resolving dependencies... (1.7s)
Resolving dependencies... (1.9s)
list index out of range
gentle solstice
#

It may be poetry-plugin-export

dim eagle
# 
poetry self update                                                                                                                                                                   [11:33:03]
Updating Poetry version ...

Using version ^1.8.3 for poetry

Updating dependencies
Resolving dependencies... (1.5s)

list index out of range
#

huh

# 
poetry plugin add export                                                                                                                                                             [11:34:28]

The command "plugin" does not exist.

Did you mean this?
    self show plugins
# 
poetry self add poetry-plugin-export                                                                                                                                                 [11:35:02]
Using version ^1.8.0 for poetry-plugin-export

Updating dependencies
Resolving dependencies... (1.7s)

list index out of range
#

I am so confused

gentle solstice
dim eagle
#

Tho

obtuse saddle
limber grail
#

I commented out everything and still getting this error ```py
ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt" did not complete successfully: exit code: 1
...

The docker build test command isn't recognize for some reason.

limber grail
limber grail
#

I think I found the issue ... it is the " RUN pip3 install -r requirements.txt" line

limber grail
#

@rapid sparrow I fixed the issue 🙂 . I simply restarted my computer removed the "requirments.txt" and re-run pip freeze > requirements.txt` Again. I think I fogot to active my virtual enviroment that I created using poetry and created the first requirments.txt file without activing it .... this created a requirments.txt file using the libraries and things i installed on my base OS.

rapid sparrow
dim eagle
#

How would I Fix this?

#

it's supposed to update the pypi package whenever I make a new release

#

but it failed

visual oxide
dim eagle
#

and that the ubuntu version was outdated

#

do you see anything wrong with the config rn?

visual oxide
rancid schoonerBOT
#

.github/workflows/publish-to-test-pypi.yml line 9

runs-on: ubuntu-18.04```
dim eagle
#

oh

dim eagle
visual oxide
dim eagle
#

alr

dim eagle
deep estuary
#

emojis in a workflow name 👀

dim eagle
#

hm?

astral apex
dim eagle
astral apex
#

Latest doesn't exist

#

GitHub is waiting for you to create it

deep estuary
#

swap to ubuntu-latest

astral apex
rancid schoonerBOT
#

.github/workflows/publish-to-test-pypi.yml line 9

runs-on: ubuntu-latest```
deep estuary
#

ah

dim eagle
astral apex
astral apex
astral apex
#

Now you just need to make a new release

dim eagle
#

Ah

#

I did make a new release

#

Not sure why it didn't work

astral apex
#

oh I see

astral apex
#

You only run your workflow if there's three dots

image.png
#

You can see that it didn't run for that tag

image.png
dim eagle
#

Oh

#

Oops

#

Let me do 0.1.5

#

Is there a remind me bot?

#

About to eat my birthday dinner on my bday

hazy echo
#

Pytest for testing ci/cd scripts?

Sooo, I have a few Dockerfiles
https://gitlab.com/highly-unlikely/tahoe-dx-primer/-/tree/main/Docker?ref_type=heads
and eventually might string some of them together with docker-compose.
I'm thinking that it would be nice to have some tests, for example, did the images build? Are they they recent?
Is there a method for testing CI/CD?
Do I just run tests against the resulting artifacts?

astral apex
rancid schoonerBOT
#
Command Help

!remind [mentions]... <expiration> [content]
Can also use: reminder, reminders, remindme

Commands for managing your reminders.

The expiration duration of !remind new supports the following symbols for each unit of time: - years: Y, y, year, years - months: m, month, months - weeks: w, W, week, weeks - days: d, D, day, days - hours: H, h, hour, hours - minutes: M, minute, minutes - seconds: S, s, second, seconds

For example, to set a reminder that expires in 3 days and 1 minute, you can do !remind new 3d1M Do something.

Subcommands:

!remind delete [ids]...
Delete up to (and including) 5 of your active reminders.
!remind edit
Commands for modifying your current reminders.
!remind list
View a paginated embed of all reminders for your user.
!remind new [mentions]... <expiration> [content]
Set yourself a simple reminder.

astral apex
astral apex
hazy echo
dim eagle
#

It seems to be working rn

#

I do have a few warnings

brisk kettle
#

!warn 274879044577918976 do not post crackware

rancid schoonerBOT
#

:incoming_envelope: :ok_hand: applied warning to @candid scaffold.

astral apex
# dim eagle I do have a few warnings

This should fix those warnings: JDsProjects/asuna_api#4

Though before you merge it -- do you actually support Python 3.4? If you do, I would double check it actually works on that first. IIRC some of this was only added in like 3.7.

signal cloakBOT
dim eagle
#

Most people would be 3.8 and up

#

Would mind adding that to the pr?

willow pagoda
#

yea, setuptools 61.0.0 (when pyproject.toml was introduced) only supports 3.7+, and i believe that got raised to 3.8+ in 68.1.0

gentle solstice
#

So if you put this in your pyproject.toml, will it imply >=61.0?

[build-system]
requires = ["setuptools"]
build_backend = "setuptools.build_meta"
astral apex
dim eagle
#

Hmm

modern phoenix
#

question before: is it allowed here to ask about non pep-compliant things here? having problems with installing some hellish "addon" …

#

(like i installed and doesn't work and/or i don't know something…)

gentle solstice
#

What is it?

modern phoenix
#

thought of boosting a little calculations so decided to install… just that doesn't work…

gentle solstice
#

Show the code you're using?

modern phoenix
# 
# coding: cursed-for

for (i=0; i<512; i=+16):
    print(i)
#

top line literally copied from readme

gentle solstice
#

# coding: cursed_for

modern phoenix
#

ye

gentle solstice
#

Underscore, not dash

modern phoenix
#

um.

#

ok

gentle solstice
#

You used a dash

modern phoenix
#

still. doesn't work

#

(force-installed both normally and on root btw)

gentle solstice
#

Just tried myself. It works.

# 
% cat main.py && .venv/bin/python main.py 
# coding: cursed_for

for(i = 0; i < 10; i+=2):
    print(i)

0
2
4
6
8
modern phoenix
#

i do just ./file.py …

gentle solstice
#

Do you have a shebang?

modern phoenix
#

sheat

#

forgot

#

okay. this fixed it 🤣 am dumb lol

gentle solstice
#

no shebang means it runs in bash

modern phoenix
#

forgot, yep 🤦‍♀️

#

(now dealing with non-incremented values but sort-of works i think…)

#

fixed even that. lots of LOC saved :D

astral apex
dim eagle
#

merged

astral apex
#

🎉

dim eagle
#

Pushing a new release

#

Trusted Publishers allows publishing packages to PyPI from automated environments like GitHub Actions without needing to use username/password combinations or API tokens to authenticate with PyPI. Read more: https://docs.pypi.org/trusted-publishers

#

hm?

astral apex
# dim eagle Pushing a new release

One thing that's confusing me -- What exactly are you trying to do with your version?
You bumped your project to 0.3.2 but then created a release called 0.1.6.

dim eagle
#

I see

astral apex
dim eagle
#

Ah

#

I wonder how I fill this in

astral apex
rancid schoonerBOT
#

.github/workflows/python-pypi-publish.yaml line 62

password: ${{ secrets.PYPI_API_TOKEN }}```
dim eagle
#

is it fine if i get rid of the api token already there in the github settings too?

astral apex
#

TL;DR: filling out the form allows GitHub to request a temporary password when your Workflow runs, so you don't need to add one

astral apex
dim eagle
#

Deleted from settings

#

so

#

I need to fill in the info

#

anything else?

image.png
astral apex
rancid schoonerBOT
#

.github/workflows/python-pypi-publish.yaml lines 46 to 47

environment:
  name: pypi```
astral apex
astral apex
dim eagle
#

That good?

image.png
astral apex
#

Looks good

dim eagle
#

deleted the old api token

#

on pypi

astral apex
#

👍

dim eagle
# astral apex 👍

(node:1932) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use node --trace-deprecation ... to show where the warning was created) ???

#

builds that it seems fine

astral apex
dim eagle
#

oh.

astral apex
#

Oh look at that it already found a couple things to update

#

Anyways... I know I kinda dumped a lot on you.
Does all that get you what you needed?

hazy echo
dim eagle
#

with indent=4

stuck hound
#

will git cherrypick create a new commit hash or will it use the same hash?

rapid sparrow
stuck hound
#

i have:
0: initial commit
1(HEAD): #1
2: #b
3: #c
i want to skip #b and merge/cherrypick #c

stuck hound
crimson spruce
#

Cherry-picking creates a new commit, usually (?).
I think the only time it wouldn't is if the commit already has the current HEAD as a parent.

rapid sparrow
#

it visualizes result in real time, showing how commits will be

stuck hound
crimson spruce
stuck hound
stuck hound
crimson spruce
#

Like is it a linear history?

stuck hound
stuck hound
#

i have simplified the scenario. if you can answer this i can figure out what to do

crimson spruce
#

In that case, you can indeed cherry-pick commit 3. It will create a new commit, but that's unavoidable if 3's parent is 2, since a commit's identity includes the parent.

stuck hound
#

maybe i should rebase -i and drop the #2 commit lemon_thinking

crimson spruce
#

That's the other option, yeah, but then you first have to go to commit 3. The result will be the same.

stuck hound
#

i just don't want create a new commit as it will mess up my commit history

crimson spruce
#

That is impossible, sorry. Changing a commit's parents will always create a new commit.

stuck hound
#

remote / origin has:

0: initial commit
1: #a
2: #b
3: #c (HEAD)

i have on my local repo:

0: initial commit
1: #a (HEAD)

i just want #c without #b on my local repo

crimson spruce
#

Pull and rebase then.

#

(or git fetch and git cherry-pick origin/main)

stuck hound
crimson spruce
#

As I said, it will create a new commit hash and there's no way around that.

stuck hound
#

so if it creates a new hash.. and next time i push to remote.. it will mess up remote's history

edgy basin
#

is there any way to make pylsp faster

#

pylsp takes like 10 secs to parse the file

crimson spruce
#

But if you don't want to mess with the remote: Yes, then you can't push to it anymore.

limber grail
#

I created a React app with vite. I am trying to to decroize the app using a Dockerfile. But I keep getting this error js ERROR: failed to solve: process "/bin/sh -c npm run build" did not complete successfully: exit code: 127
Here is my instructions

FROM node:22.4.0-alpine
WORKDIR /app
ENV PATH="./node_modules/.bin:$PATH"
COPY . .
RUN npm run build 
CMD ["npm", "run", "dev"]
thorny shell
#

I tried that and saw 0.257 npm error enoent Could not read package.json: Error: ENOENT: no such file or directory, open '/app/package.json' which is likely relevant 🙂

rapid sparrow
#

as well as adding... at this step copying only package.json + lock file and running npm i command to install stuff

#

and only after that copying the rest of files

#

that will ensure correct caching + a way docker is intended being used

coral holly
#

Is there a way you can install dependencies in heroku?

rapid sparrow
#

if u need system/OS level extra dependencies... u a probably looking at deployment with docker containers

gentle solstice
#

It supports requirements.txt or Pipfile

gaunt barn
#

why hello there

gaunt barn
#

def main():
message()
response()
def message():
input("Message: ")
def response():
you_say_something = input("Response: ")
if you_say_something == (""):
print("womp womp")
else: print("Yay!")
main()

#

memes!

thorny shell
#

...

gaunt barn
#

?

thorny shell
#

!

gaunt barn
#

!=

limber grail
soft sail
#

Hi guys, I'm studying webscraping and trying to webscrap a pokedex website:
If I write code like this:

print(pokename)```

This is the output:

<div class="pokedex-pokemon-pagination-title">   
<div>
      Bulbasaur
      <span class="pokemon-number">Nº 0001</span>
</div>
</div>

If I write code like this

pokename = soup.find('div', class_='pokedex-pokemon-pagination-title')
for pokenames in pokename:
print(pokenames.text)

This is the output:
#

How can I remove this spaces?

twilit forum
#

So this is general python and not really soup or tools/devops. But:

Just do .strip() on a string to remove leading or trailing whitespace.

jagged yoke
#

can I ask about ruff here?

thorny shell
#

sure

jagged yoke
#

oh right, I'll ask there then

#

cheers!

echo monolith
#

Hello I am a starter, and have learned Python a bit, can anyone help me guide a bit please? seriously passionate guy here seeking guidance! Thanks in advance

twilit forum
#

Just ask away 👍

limber grail
#

Thanks @sonic moss . But I prefer to have Nginx, Gunicorn as part of my stack.

deep estuary
#

nginx my beloved

astral apex
#

I tried it

#

It’s DSL was confusing
It kept throwing errors in my face

#

NGINX just works

#

NGINX is great
It’s always just done exactly what I needed it to

paper moss
#

what do you guys recommend me to convert a pdf to text using python?

rapid sparrow
#

the thing u wish is basically quite complex one

#

as PDF can contain text

#

and can be image

#

if it is image=> then u need Computer Vision OCR like pytesseract

soft oasis
#

whole lot of shit id understand but soon I will

rapid sparrow
paper moss
#

i'm using pdftotext

#

but this error occur in the server: poppler/error: Failed to parse XRef entry [11].poppler/error: Top-level pages object is wrong type (null)

#

and my program stops without raising any exception

#

how can I prevent it to stop?

indigo zenith
indigo zenith
paper moss
indigo zenith
#

You could do a standard try/except if you don't want to try one of those other libraries instead

paper moss
indigo zenith
paper moss
#

and as im using a centos server upgrading poppler may be a pain in the ass

indigo zenith
paper moss
placid glen
#

what could explain the load average on my server going through the roof when I'm not looking and stops when I connect to the server through ssh ?

indigo zenith
indigo zenith
# placid glen what could explain the load average on my server going through the roof when I'...

You need to monitor and find out what process it is: https://askubuntu.com/questions/1225073/how-can-i-see-what-process-spiked-cpu-usage-and-froze-the-system-a-few-seconds-a

Ask Ubuntu
How can I see what process spiked CPU usage and froze the system a ...

My Ubuntu 18.04 randomly freezes for a few seconds. I can move the mouse cursor (sometimes), but otherwise the OS is unresponsive and I can't switch to any other application.
When it resumes, I can...

placid glen
#

😢

#

perfctl

#

got p0wned most likely, idk what to do

paper moss
amber bough
#

Hi, i think this is where i should be asking. Trying to install pyaudio but getting this message

indigo zenith
amber bough
#

ah, seems like pyaudio isnt really up to date

thorny shell
empty laurel
#

Help pls

image.png
thorny shell
#

try again, but this time use py -3 -m pip install pyinstaller

willow pagoda
# empty laurel Help pls

microsoft store python doesnt add scripts to PATH, right? you probably have to run it using python3 -m PyInstaller ... instead

flat cairn
#

Hello guys I wanna make theorem prover language can some one help me in developing that ?

mystic void
flat cairn
mystic void
flat cairn
mystic void
heavy knot
#

Much easier way to test your program for efficiency.

py -m profile -s time program_tester.py > "Efficiency Results.txt"
#

Not sure why others keep saying to use the timeit package.

#

Huh doesn't seem to want to print it all to file.

#

Oh, forgot to take care of my last input, nevermind.

proud pendant
#

does anyone know how to use multiple commands with git bisect properly on windows? like with sh -c

i tried:

git bisect run cmd /c "command_1 && command_2"

but this doesnt run the commands, it just starts another empty command prompt and sits there

#

also is there a way to test how git bisect run will run the command, i.e. just running one iteration then stopping?

edgy basin
#

petition for rust pylsp

mint cipher
#

I'm building my Django app in docker-compose but it needs the VPN network connection I have running on the host. How do I share this VPN connection to the docker-compose.yml file?

steel quail
#

not sure this is the right channel to ask:

I'm wondering if there exists a documentation tool (like sphinx or docusaurus) that would allow hyperlinking the document with source files or even if possible, class/method declarations, and up to a pycharm plugin, those pages would be rendered in pycharm and I could click on hyperlink to rapidly go to parts of the code.

not sure the explanation is clear nor if someone made that kind of things already, but I'd be interested if ever one knows a bit

gentle solstice
mint cipher
gentle solstice
#

Are you on windows with docker via wsl2?

#

Don't forget to add this to /etc/wsl.conf

[network]
generateResolveConf = false
heavy knot
#

The package appears to not be named intuitively, what version do I use?

image.png
brazen forge
heavy knot
#

that worked, however new problem? I am porting 8000 so how is the user disallowed?

image.png
brazen forge
#

did you try what the error suggests?

heavy knot
#

I am trying a lot of things one sec

heavy knot
#

got it!

image.png
#

Now I want my elastic IP to be set up with the server now that I can get it to work with the project

#

can elastic IP be used for domains that can be propogated? Is that even a thing?

gentle solstice
#

You should use a aws gateway server

heavy knot
#

new problem, I am trying to get two servers to run by using redus and daphne but I can not find the proper packages

image.png
#

why don't these work?:
redis-server
daphne -p 8001 thought_sync.asgi:application

gentle solstice
#

Try using valkey instead.

#

for redis, you'd need a third party repo.

#

Though I believe the suggested approach is to use a managed server.

unkempt zodiac
#

Does Kubernetes on Windows run on a Linux VM like how Docker on Windows runs on WSL?

rapid sparrow
#

for dev env people are commonly prefering using Kind today https://kind.sigs.k8s.io/ for quickly creatable and desstructable envs. it operates by running kubernetes in a docker container. 0 VM usage

rapid sparrow
#

if u will use Kubernetes through Desktop program created for Windows, then it will create ANOTHER VM in addition, where it is running (So at one point of time u can be having 2 VMs, WSL2 + Desktop VM). 2 VM usages.

unkempt zodiac
#

So, to summarize, tools like Minikube included with the k8s toolset on Windows will create a Linux VM to run anyway, so I should use WSL2 instead of trying to run it natively on Windows?

rapid sparrow
#

erghh....

#

it has different running engines though according to documentation

#

Docker is default but using VMs is possible too

rapid sparrow
#

and if u use inside WSL2, it will potentially just reuse current Linux kernel and run via docker containers

rapid sparrow
#

i like using Kubuntu LTS versions. Comfortable GUI to former windows people. High community support and everything work related is easy to install

unkempt zodiac
#

yeah, I'll use WSL2 entirely. I already know enough Linux commands. But, using a barebone Linux is still not preferable to me since not many games support linux.

rapid sparrow
rare laurel
#

Why can't my django connect to the database, my database is in my virtual machine, I can use other tools to connect but the startup project shows Error to connect to the local database

django==4.2 mysql==8.0.2

image.png image.png image.png
indigo zenith
heavy knot
#

"RuntimeError: Numpy is not available" i did pip install numpy still didnt fix

thorny shell
heavy knot
#

how does one use redis on AWS? The Django commands do not work as usual?

image.png
indigo zenith
heavy knot
#

yes, that does appear to be the issue

image.png
#

now I am trying to connect my cache

#

does endpoint has to be some number from the actual cache?

image.png
indigo zenith
heavy knot
#

I am not following a guide, but thank you for letting me know

#

wait why in step 2 does it say something different?

image.png
rare laurel
mighty surge
#

whats a good python code visualizer beside python tutor

stable cloak
#

It's more of a beginner focused IDE/editor

#

But I love it

#

Has anyone played around with uv? Wasn't sure if anyone had any opinions on it yet. https://github.com/astral-sh/uv

I'm hoping to use uv with something like PDM. Does seem like they're working on it, at least. https://github.com/pdm-project/pdm/issues/2693

GitHub
GitHub - astral-sh/uv: An extremely fast Python package installer a...

An extremely fast Python package installer and resolver, written in Rust. - astral-sh/uv

GitHub
Support uv as the resolver and installer · Issue #2693 · pdm-projec...

Is your feature/enhancement proposal related to a problem? Please describe. uv https://github.com/astral-sh/uv is extremely fast to install pip libs, it would be nice if pdm (install/sync) can opti...

stable cloak
#

I forgot that rye exists. Ignore me

heavy knot
#

how do I download modules, pip install channels did not work?

image.png
indigo zenith
heavy knot
#

daphne appears not to work. how do i install channels? python install channels? sudo install channels?

image.png
indigo zenith
heavy knot
#

AL2023?

#

Which commands from the daphne packages do I use based on this information?

indigo zenith
heavy knot
#

i have tried docker but decided not to go with it for various reasons

#

curious problem:

image.png
#

how is debug-toolbar already installed but it still is not downloaded in the venv?

rapid sparrow
rapid sparrow
#

Venv is depended on the absolute path where it was created

#

If u move it to anywhere... It will break

#

Also usage of venv inside docker is in general pointless and not needed in 99.99% usage cases. Just install to global inside docker

heavy knot
#

Agree with the venv since it's an isolated thing already and normally runs one main software, 90% of cases maybe...
But i guess you normally still want to set user permissions @rapid sparrow ?

rapid sparrow
heavy knot
#

no i mean within dockerfile @rapid sparrow

rough mauve
#

Trying to set up automount on my Linux server, facing some issues, would like help on the fstab line item I wrote:

root@SakiUser:~# ls
Saki  snap

//123.456.78.90/Saki /Saki cifs credentials=/root/.smbServer,uid=0 0 0

root@SakiUser:~# sudo mount -a
Couldn't chdir to /Saki: No such file or directory
thorny shell
#

have you done sudo mkdir /Saki yet? If not, try that

rough mauve
thorny shell
#

uh

#

it won't

rough mauve
#

oh alright

#

I just ran that command

#

Should I try to mount again?

thorny shell
#

yes

rough mauve
#

Got an error:

mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)
thorny shell
#

🤷

#

do what it says; I'm out of ideas

rough mauve
#

Alright

fair umbra
#

can someone please help me with develoyment for django-react web app(no database) on AWS.
AWS S3 for frontend code and AWS EC2 for backend code.

heavy knot
rapid sparrow
# heavy knot How would I verify this?

docker run --rm -it imagename bash for debian based docker images, then ls
docker run --rm -it imagename sh for alpine based images
docker compose run --rm service_name bash for compose (docker compose down ensure to run in advance, so u would not enter already existing ones)

#

docker exec -it container_id_or_name bash to enter already existing ones

#

.dockerignore file is useful for prevent copying not intended stuff

heavy knot
rapid sparrow
rapid sparrow
rapid sparrow
#

i would recommend learning docker and deploying as a start just with docker-compose i guess

heavy knot
#

thanks but can we learn the AWS "baremetal way" first?

rapid sparrow
heavy knot
#

Also I want to connect this elastic ip to the server, is there anything I should change first?

image.png
rapid sparrow
#

🤷‍♂️ if it will work, then okay, if not, then u can be having issues like no public IP at server
and optionally your AWS Security group can be not permitting traffic to server from specified IP range and ports (kind of cloud level firewall)

#

there is an array of its own possible failures if u will have custom networking configured 😅

rapid sparrow
# heavy knot thanks but can we learn the AWS "baremetal way" first?

Since u are at AWS baremetal way, i will recommend you as soon as possible learning actually dealing with AWS through opentofu
https://www.oreilly.com/library/view/terraform-up-and/9781098116736/
AWS is too bloated to manage manually. Infrastructure as a code for the win to control it
I am of opinion simple baremetal without iac should be used in Hetzner or DigitalOcean, but not with AWS.
AWS requires IaC for sanity

O’Reilly Online Learning
Terraform: Up and Running, 3rd Edition
heavy knot
#

ok I just want to know what is not completely insecure and also what I should make the private ip, I want it to say "ww..mywebsite.org" for example

image.png
rapid sparrow
#

u will need AWS Route53 or some other external service for DNS records then. 😉

#

and then u can be needing Certificates...

#

...using AWS certificates that it provides... or installing self hosted ones through letsencrypt

#

pairing it with elastic ip sure, good idea

heavy knot
#

I have bought a Domain using NameCheap, now which part do I add to the private ip address?

indigo zenith
heavy knot
#

does this involve adding a redirect domain?

image.png
#

where do I add certificates, is that through NameCheap or AWS?

indigo zenith
heavy knot
#

do we care about shell versus not shell access? how do i start?

image.png
rapid sparrow
edgy basin
heavy knot
#

now how do I use let's encrypt?

image.png
indigo zenith
#

Debian or CentOS is probably close enough to Amazon Linux

heavy knot
#

which Debian lol?

image.png
#

is this technically a web hosting product?

image.png
indigo zenith
indigo zenith
heavy knot
indigo zenith
heavy knot
heavy knot
indigo zenith
# heavy knot

That's still your OS. Did you install Nginx or Apache, or how does your web app work?

heavy knot
indigo zenith
# heavy knot the web app is Django REST api

You need to set up a web server. If you were using Docker you could try this: https://medium.com/@akshatgadodia/deploying-a-django-application-with-docker-nginx-and-certbot-eaf576463f19

Otherwise you'll need to set up something similar in a manual way

Medium
Deploying a Django Application with Docker, Nginx, and Certbot

Deploying a Django application with Docker, Nginx, and Certbot is a robust and secure way to make your application available on the…

heavy knot
#

will this be set up in docker or deployed in the AWS terminal?

indigo zenith
heavy knot
#

Here is my docker pseudo-code, where do I run it?
`

Dockerfile

Use an official Python runtime as a parent image

FROM python:3.11.4-slim-bullseye AS base

Set environment variables

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

Set the working directory in the container

WORKDIR /app

Install system dependencies

RUN apt-get update && apt-get install -y
gcc
default-libmysqlclient-dev
pkg-config
curl

Install Python dependencies

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

Remove unnecessary packages and clean up

RUN apt-get remove -y gcc pkg-config &&
rm -rf /var/lib/apt/lists/*

Copy the application code into the container

COPY . .

Expose port 8000

EXPOSE 8000

docker-compose.yml

version: '3'
services:
backend:
build: .
command: sh -c "python manage.py makemigrations && python manage.py migrate --noinput && python manage.py collectstatic --noinput && gunicorn --config gunicorn_config.py your_project.wsgi:application"
restart: always
expose:
- "8000"
volumes:
- ./:/app
- /var/www/static/:/var/www/static/
- /var/certbot/conf:/etc/letsencrypt/:ro
env_file:
- .env

nginx:
image: nginx:alpine
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- /var/www/static/:/var/www/static/
- ./nginx/conf.d/:/etc/nginx/conf.d/
depends_on:
- backend

certbot:
image: certbot/certbot
volumes:
- /var/certbot/conf:/etc/letsencrypt
- /var/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
depends_on:
- nginx
`

rapid sparrow
# heavy knot will this be set up in docker or deployed in the AWS terminal?

when you do things with docker you have at minimum two basic options for beginners level

  1. you just clone git repo to host. docker compose build build at host, and docker compose up -d launch in background. Building at host option and running at same host right away
  2. more advanced choice with docker, docker build your image, docker push to docker hub (or AWS docker registry) and then at host you only run docker run / docker-compose up command. This choice has benefit of ability to rollback to previous version as docker registry saves them. and also running same app version a year later without rebuilding it. This option allows exact copy running at multiple servers easily.
heavy knot
indigo zenith
rapid sparrow
#
  1. there is a choice in between. how to augment first choice: You can set env variable DOCKER_HOST=ssh://root@darklab:22 with ssh access to your host.
    then docker compose build and up u will do at your dev machine, will be actually acomplished at target host 🙂
    I use this approach with Terraform Docker provider for pet project somewhat. (plus saving to docker registry)
indigo zenith
rapid sparrow
# heavy knot thanks, lets do option 2, where can i find the notation in AWS to set this up?

u can create AMI image with already installed docker https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html (like this one)
It will be already installed there then. probably easiest choice

Amazon ECS-optimized Linux AMIs - Amazon Elastic Container Service

Amazon ECS provides Linux Amazon ECS-optimized AMIs that are preconfigured with the requirements and recommendations to run your container workloads.

heavy knot
rapid sparrow
# heavy knot

you are supposed to have Dockerfile in your project folder with instruction how to build your project. then building happens like... docker build --tag test .

heavy knot
rapid sparrow
# heavy knot Thank you for the recommendations, could you order them in the order I should do...

  1. get docker to your dev machine.
    If linux then install as https://docs.docker.com/engine/install/
    if windows, then install wsl2 and install docker with same instruction as above inside wsl2 linux

  2. create Dockerfile for your project, install gunicorn there to run your Django project?
    docker build --tag test . it, and then try to run docker run -it -p 8000:8000 test
    if u will have correct Dockerfile, your project will be launchable locally and visible at 8000 port.
    u need to have stuff like inside like

FROM python:3.10.5-slim-buster as base

# This flag is important to output python logs correctly in docker!
ENV PYTHONUNBUFFERED 1
# Flag to optimize container size a bit by removing runtime python cache
ENV PYTHONDONTWRITEBYTECODE 1
WORKDIR /code

COPY requirements.txt constraints.txt ./
RUN pip install -r requirements.txt -c constraints.txt
COPY src src

RUN gunicorn src.core.wsgi -b 0.0.0.0:8000 --workers 2

  1. create account at docker hub for example https://hub.docker.com/
    build image with corret name like account/repo_name, or use docker tag to retag already existing one to the correct repo address
    use docker push to push to it (after authorized)

  2. create AWS server with docker already preinstalled https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html (simple choice)
    or install manually to some server https://docs.docker.com/engine/install/

  3. just run docker run -it -p 8000:8000 -d docker_hub_account/repo_name:tag_name at machine, to run your web app in background
    (optionally run it from dev machine by prefixing command with DOCKER_HOST=ssh://root@darklab:22 docker run -it -p 8000:8000 -d docker_hub_account/repo_name:tag_name like this. it will run at target machine, if darklab is your host configuration in ~/.ssh/config for auth with rsa and etc and root username of machine
    (Optionally simplify step 2 and 5, by using docker-compose. it will make the commands shortened)

#

to just docker compose build and docker compose up (optionally with flag -d for sending running in background)

heavy knot
#

also my dev machine is AWS which is linux? or is it my local computer which is windows?

rapid sparrow
rapid sparrow
#

install WSL2, then u will have linux sub system

#

allowing you to use in a sane way docker locally

#

and having development pretty much with mostly...full linux compatibility locally

#

there are alternative choices for windows users, but i don't like them to recommend

heavy knot
#

unpacking files is taking forever

#

you have done so much Darkwind The Dark Duck, if you need to bid farewell do so. But your potent friendship and education shall not be forgotten

rapid sparrow
#

wsl2 is close enough to be almost fully fledged linux.

#

checking stuff working locally is always more simple than doing it at remote hosts after all

#

default wsl2 is ubuntu though, but default AWS Linux i think Rhel stuff

#

you would have easier time if your distros match at local and remote target

#

if using ubuntu locally, then makes sense using ubuntu at remote servers. to simplify repeatability

heavy knot
#

I know about half of this! I am not a novice lets go

#

thank frick it done

#

wait the docker will not open!

#

how do I screen shot task manager

#

I am trying to figure out why docker is not running properly, should I redownload?

#

we here

#

can i continue without signing in?

image.png
gentle solstice
#

Yes. Click "Continue without signing in"

rapid sparrow
#

Don't use docker desktop

#

It is cursed and extra problems. Instead of working natively at Linux by grabbing kernel, and working at same host...

Desktop creates another VM and does god awful things

#

Desktop app was made for dummies without Linux

#

If u have wsl2 at least, u can avoid its usage

heavy knot
#

what am I missing, why does docker engine have the downloads for docker desktop? is there a separate docker engine download menu?

image.png
indigo zenith
heavy knot
#

houstin we have a problem

gentle solstice
#

both should work in chrome

#

Did you create it just now?

heavy knot
#

yes

gentle solstice
heavy knot
#

oh ok

#

frick, is there a way to remove the :8000/ at the end?

gentle solstice
#

You didn't change the default port

#

change the bind port in the django launch command

#

Are you using gunicorn or uwsgi?

#

apache/mod_wsgi?

heavy knot
#

how do i check?

gentle solstice
#

If you need to check, you're probably using the builtin server

heavy knot
#

does any of this tell us the answer?

gentle solstice
#

no.

heavy knot
#

heck

gentle solstice
#

How did you launch django?

heavy knot
#

AWS EC2?

#

Django REST api

gentle solstice
#

no, what command did you run?

heavy knot
#

oh wait

#

pip install django

#

pip?

gentle solstice
#

no, that installed django

#

what did you use to launch it?

#

run python manage.py check --deploy

heavy knot
#

oh

gentle solstice
heavy knot
#

nani!?!

image.png
#

plot twist what if i did not launch django and i did not know it?

gentle solstice
#

That means your app is not ready to be deployed.

heavy knot
#

but it seems to work when i run the server no?

image.png
gentle solstice
#

Do you understand why you're getting those warnings?

rapid sparrow
rapid sparrow
#

Docker Engine is completely free on another hand ;]

gentle solstice
#

Try podman desktop instead.

#

(requires podman engine)

#

and looks like vscode (that makes it cursed)

heavy knot
# gentle solstice Do you understand why you're getting those warnings?

idk, the article has many reasons and idk which one, here are the list of possible problems:
`

  1. security.W004: SECURE_HSTS_SECONDS
    Set a value for the SECURE_HSTS_SECONDS setting.
    HTTP Strict Transport Security (HSTS) is a security feature that instructs browsers to only connect to your site over HTTPS. Setting SECURE_HSTS_SECONDS ensures that this feature is enabled and specifies the duration (in seconds) for which browsers should enforce HTTPS after visiting your site once over HTTPS.

  2. security.W008: SECURE_SSL_REDIRECT
    SECURE_SSL_REDIRECT setting is not set to True.

  3. security.W009: SECRET_KEY
    SECRET_KEY has less than 50 characters or is prefixed with 'django-insecure-'.
    Generate a strong, random SECRET_KEY and replace the default one in Django settings:

SECRET_KEY = 'your_generated_secret_key'

Ensure key is kept secret and not exposed in version control.

  1. security.W012: SESSION_COOKIE_SECURE
    SESSION_COOKIE_SECURE is not set to True.
    This setting ensures that session cookies are only sent over HTTPS, protecting them from being intercepted by attackers.

  2. security.W016: CSRF_COOKIE_SECURE
    django.middleware.csrf.CsrfViewMiddleware in MIDDLEWARE, but CSRF_COOKIE_SECURE is not set to True.
    CSRF (Cross-Site Request Forgery) attacks can be mitigated by ensuring that CSRF cookies are only sent over HTTPS.

  3. security.W018: DEBUG
    DEBUG is set to True in deployment.
    DEBUG mode in Django should not be enabled in production as it can leak sensitive information and expose vulnerabilities.
    `

gentle solstice
#

yup

#

all of them

heavy knot
#

you read that fast?

#

idk how to solve all of them?

gentle solstice
heavy knot
#

I know where that is

#

the heck are we changing in settings gamers?

#

so DEBUG should = false?

#

among other things?

gentle solstice
#

yes

#

ideally you replace this file entirely in production.

#

don't commit your security key or database url

heavy knot
#

did I change the settings correctly

#

less warnings but still warnings

image.png
gentle solstice
#

You can use openssl to generate a new secret key

#

openssl rand -hex 32

heavy knot
#

nothing changed?

image.png
gentle solstice
#

increase the number to 64

heavy knot
#

still nothing

image.png
gentle solstice
#

you need to copy the text it prints and use it as the secret key in settings.py

#

and don't share it.

#

clear the screen after you copy it

stable peak
quick lance
#

i am a qustion

#

i have

indigo zenith
robust nexus
#

When i want to create environment in anaconda this happened . How to solve this

thorny shell
#

dunno. Looks like some web server isn't responding, or isn't responding correctly. There are a lot of moving parts in between your conda create call, and that web server; you need to figure out which one(s) are at fault.

brave wagon
#

hey guys. i recently got a mesh router system right, however on top of that, i got a new pc upgrade, so ever since then, my docker has been crashing due to errors that never arose before these changes, does docker depend on a static local ip?

indigo zenith
astral apex
#

FYI

upbeat relic
#

Hey anyone know how I can use Github Actions as trigger for firebase database. If anyone know please help me.

gentle solstice
#

trigger to do what?

indigo zenith
heavy knot
#

why does it keep giving me a https error when I am using http?

image.png image.png
visual oxide
#

What does curl say?

thorny shell
#

interesting. I'd open the browser's developer tools, in order to see a list of all the requests; I'd expect to see a redirect to https:

#

or curl, ya

visual oxide
# 
$ curl -I http://harmonyapp.org:8000/
HTTP/1.1 301 Moved Permanently
Date: Mon, 22 Jul 2024 17:40:34 GMT
Server: WSGIServer/0.2 CPython/3.9.16
Content-Type: text/html; charset=utf-8
Location: https://harmonyapp.org:8000/
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Vary: origin
#

It's a permanent redirect

#

It looks like you're running runserver in production though - you should use gunicorn or waitress instead

#

And use elastic load balancer for tls

thorny shell
#

perks up

#

I'm always looking for an easier way to do TLS. Is ELB easy? Yeah, I know: "Compared to what?"

indigo zenith
indigo zenith
# thorny shell I'm always looking for an easier way to do TLS. Is ELB easy? Yeah, I know: "Co...

Elsewhere this is what I've been doing mostly: https://paregov.net/nginx-proxy-manager-and-lets-encrypt-certificates/

Svet Paregov
Paregov
Nginx Proxy Manager and Let’s Encrypt certificates - Svet Paregov

In my previous for Nginx and Nginx Proxy Manager (NPM), I wrote on how to install NPM, but didn't configure any certificates. Out of the box Nginx Proxy Manager supports Let's Encrypt SSL auto creation and renewal. There is one limitation - you can create certificates only for specific domains/subdomains directly. If you want to

heavy knot
visual oxide
#

Why did you run pip install app?

heavy knot
#

it gave me this error:
ModuleNotFoundError: No module named 'app'

visual oxide
#

That's no reason to pip install app

heavy knot
#

what would the alternative be if it has all these 'app' problems?

sleek minnow
#

I'm looking for some help with release management/versioning in git/github. I recently came across a discussion of it in a book, but can't find the reading again now that I actually need to put it to use. If anyone can recommend some references I'd much appreciate it.

My specific situation is that I'm working in a student team and there's a lot of variation in experience and conventions. Much of the files have orphaned code, large blocks commented out but saved for someone's reference, comments that don't apply to current functionality, etc.

I'm trying to introduce the idea of official releases so that we can have, among other things, a designated clean-up phase in our development cycle (because we sure don't use our linter.)

indigo zenith
sleek minnow
#

Perhaps that part of the matter would be better suited to the architecture channel?

paper moss
#

Hello, do you know any docker service that can be called through API calls and make openoffice or msoffice tasks (e.g. convert from pdf to html, doc's or docx's to pdfs...)?

Thanks

indigo zenith
paper moss
heavy knot
#

I keep on getting this error but i set up my security groups correctly, what do i try next?
[23/Jul/2024 13:35:09] code 400, message Bad request version ('\x02h2\x08http/1.1Di\x00\x05\x00\x03\x02h2\x00-\x00\x02\x01\x01\x00\x17\x00\x00\x00\x12\x00\x00\x00\x05\x00\x05\x01\x00\x00\x00\x00\x003\x04ï\x04í') [23/Jul/2024 13:35:09] You're accessing the development server over HTTPS, but it only supports HTTP. ^C(thought_sync_django-channels) [ec2-user@ip-172-31-19-47 thought_sync_django-channels]$

image.png
thorny shell
#

has nothing to do with security groups -- your request is getting through to your instance

#

You're accessing the development server over HTTPS, but it only supports HTTP. is the money quote

#

if you're running a reverse proxy, I'd carefully check how it's configured. Perhaps it's somehow proxying the request without decrypting it, although I can't imagine how that could be

heavy knot
#

what are the browser's developers tools? this?

image.png
thorny shell
#

yep

heavy knot
#

wait really, i was half joking

#

ok

thorny shell
#

if you click "network" you can see the requests and resposnes.

heavy knot
#

where am i looking?

thorny shell
#

I suspect they'll be normal -- i.e., not https -- but who knows

heavy knot
#

nani? no http or http?

thorny shell
#

those look fine -- they got 200 responses

#

if you clidk on one of the names, I think it will expand and show you

heavy knot
thorny shell
#

ya

#

ok that's https

urban shoal
#

Hi

thorny shell
#

and it got a 200, so that part is working ...

heavy knot
#

is 204 bad?

thorny shell
#

probably not

heavy knot
#

this is 204

image.png
thorny shell
heavy knot
#

wait so maybe that is what we have to redirect?

thorny shell
#

no idea honestly

#

again, take a look at your reverse proxy config -- and its logs, too

heavy knot
#

where are those? also maybe this is a clue, there is no http?

image.png
thorny shell
#

"reverse proxy" is a web server that sits between the public internet, and your "real" web server (django/gunicorn, flask, whatever). It's there to do the TLS magic, and perhaps protect your real web server from DOS attacks

#

it's often nginx, but I hear some people use apache, and some use traefik

heavy knot
#

do i have to add it or does it already exist somewhere?

thorny shell
#

you already have it, or its equivalent.

#

since clearly you're doing SSL mostly right.

#

maybe you're using some Amazon service like ELB; who knows

heavy knot
#

so if i want to test which system i am using (lets say apache)
what do i add to this to make the command to modify the configurations?
/etc/apache2/apache2.conf

thorny shell
#

sorry, I didn't understand that at all

#

you're saying you don't know how your web site is handling TLS?

#

if that's the case, then poke around very carefully in your AWS console

#

unfortunately AWS' UI is terrible, and their docs are bad too

heavy knot
sturdy seal
#

Hello. We have a set of programs, each one relying on their own virtual environments (when finished, that is). Is there a way to specify that environment each time each program runs, in the program itself? Or does that need to be added to a script to start the program?

rapid sparrow
#

could u explain somehow... on code/script/file examples what is happening for you

sturdy seal
# rapid sparrow could u explain somehow... on code/script/file examples what is happening for yo...

This is basically a continuation of my previous issue you helped me with. We ended up selecting Poetry to create our projects, and each project has it's own virtual environment. The projects will run when a user selects an appropriate item from a menu program (text UI). Currently, programs start by the menu, which is written in COBOL (for now), matches the command with a line in a script to start the python program.

#

What I want to figure out ahead of time, for every use that selects a menu item, do I need to specify a programs's specific virtual environment in the starting script? Or is there a way to specify it in program.

#

I'm not very good at putting things into words, so I apologize ahead of time.

rapid sparrow
# sturdy seal What I want to figure out ahead of time, for every use that selects a menu item,...

python programs can be invoked in several ways

  • python3 path_to_script.py # you invoke with whatever global default python
    • python3.10 path_to_script.py # with specific python version
  • .venv/bin/python3 path_to_script.py # you invoke your program with venv environment located at relative .venv address. you can specify absolute address if u wish
  • source .venv/bin/activate and then python3 path_to_script.py # covinient hack for dev comfort. that makes .venv/bin/python3 environment automatically activated and invoking all scripts with it
  • probably there are poetry specific commands to run script with its current env
sturdy seal
#

I'll look into these, and see if poetry has something like that.

#

Thanks.

#

I didn't know if you could set the environment and start the program in the same line.

heavy knot
heavy knot
#

ss -lt | grep apache

-l: listening
-t: tcp

heavy knot
heavy knot
rapid sparrow
#

other options i mentioned are mostly intended for dev users (easy modifying capabilities are preserved) or for deployment to linux servers too

#

if u aim to deploy as desktop application, u can wrap python app with pyinstaller or its alternatives

heavy knot
#

you can also check access from within i.e wget 127.0.0.1 (or a similar command) if there is a listener

#

new error? I have never seen this before. this is different than can not connect?

heavy knot
#

my app looks different when i run it locally, how do fix this?

image.png
indigo zenith
heavy knot
heavy knot
#

i am trying to fix it with nginx, why is sudo yum install make not working?

image.png
grim osprey
sturdy seal
rapid sparrow
rapid sparrow
#

on another hand..

#

...using venvs i think will have easier for you development environment. Easier to modify different venvs and run stuff again

#

so keep venvs 😉

sturdy seal
#

For dev, yes. The only reason for the virtual environments for production is that we have some very specific dependencies for a few very specific programs. But that limits us to Python 3.9.

#

Otherwise I'd just use a production environment.

rapid sparrow
#

u mentioned before it is all run by COBOL? 🤯 that is shocking tbh.

#

i thought it not run anywhere but at US banking

sturdy seal
#

It's in transition. Currently, the majority of programs, and the main menu system are written in COBOL. But we are slowly transitioning to Python.

#

I think it's because when the programs were first being built, they were being built for the accounting department.

#

We make cardboard boxes.

#

I'm not sure why python, honestly, but job security 😂

rapid sparrow
heavy knot