#tools-and-devops

1 messages · Page 21 of 1

brazen forge
#

containers are the wrong tool for this

novel rover
#

It would appear that its the former, as many people have suggested the same thing

novel rover
# brazen forge containers are the wrong tool for this

I would really appreciate if you could give me some guidance on this topic. I'm a student, not a professional developer, and I rely a lot on the guidance of the pros online (like yourself) to know how to move forward in situations like this

thorny shell
#

🤷

#

just don't use containers

#

why are you using containers in the first place?

rapid sparrow
#

it is easy to forward Docker daemon sock into containerized web service

#

your application could be just doing a thing like => received input => registered a thing for launch => if a thing is not launch, then launch container with wireguard

#

we are basically doing a bit of programmatic container scheduling at some point

#

instead of docker, it will be easy to upgrade solution launching the end result in any other container scheduling system

#

if necessary in a future

steep sleet
#

@indigo bloom

#

Hello

spiral pendant
#

Is there a recommended way in PyCharm to set the interpreter / conda environment by default in a dev container?

Moving our project at work to them and the last remaining pain point is not being able to figure out setting the interpreter. I had hopes that changing it once and reopening would leave it set since settings seem like they get pushed from the host machine, but apparently that is not the case.

mental frost
spiral pendant
# mental frost You can use a virtual environment

We’re using microconda at the moment, but im open to swapping to venv.

The problem I’m running into though is keeping the setting persistent, does a venv fix that?

Every time I re connect to a container I have to re-apply the interpreter for the project which is super annoying

vocal parcel
#

hiii im new to discord whats the main pourpose of this server? 'cause i entered it to finid a little job

rapid sparrow
# vocal parcel hiii im new to discord whats the main pourpose of this server? 'cause i entered ...

The main purpose of this server to share information in solving dev problems. Having partners to verify or brainstorm some problem.
Like you are struggling with some ideas or directions regarding dev smth? people are happy to help here.
#career-advice channel in addition can be asked for advices regarding career, resume reviews and etc stuff.
Asking for job itself, or offering paid job is forbidden (Because this is not a place for it, it requires moderation to remove scams and etc, and current moderators do not have allocated time to deal with such stuff)

whole ermine
#

So what's everyone's favorite pre-commit hooks?

rapid sparrow
#

CI friendly written in golang

#

with autogenerated file in yaml for extra configuration of settings

#

enforces Git Conventional Commits upon you

#

and based on that grants you ability to auto generate changelogs and auto suggest next semantic version

gloomy scaffold
#

it's fairly common that I want to sort only some lines in files, so this has been very useful

whole ermine
#

And here I am using isort, trailing-whitespace, and the like 😁

#

Would you recommend for or against using pylint in a pre-commit hook? Currently I'm doing it and the delay isn't too bad.

vocal pewter
#

Personally, I would lean toward only self-corrective hooks in pre-commit. If you have a hook that requires the dev to take more action before being able to commit, make sure it's something worth acting on there and then. Otherwise put it in the CI and have it a merge requirement.

rapid sparrow
#

we would be not just able to afford doing that 😄

#

but otherwise yeah, CI is better place for smth to be enforced

#

i put into Git Hooks... well, only git conventional commits
And automatically redumping my sqlite3 db into dump at some place is necessary before git committing. prevents git commit accordingly if this was needed

twilit forum
#

Should fix a hook to block large files. It's not fun to git fetch 200 MB in the morning 🙄

#

Ruff is a very fast. I have lint and autofixes as push hook to master

twilit forum
thorny shell
gentle solstice
#

Ruff is faster if you care about speed

#

You can also make a github action to auto fix on push

whole ermine
#

I'm in a large org but codebases are actually kind of small since I'm in a niche team. Already have pylint in the pipeline on push, but I also like catching as many things as early as possible, sooo yeah.

twilit forum
#

Just run ALL LINTERS 😶

brazen forge
rich kindle
#

Can a package be a tool

#

Or is the term "tool" used for other things

astral apex
#

Depends on context

rich kindle
# astral apex Depends on context

For example a package that aims for simplicity. E.g: a package that adds some decorators like @asynchronous and other stuff. A package meant to make it easier

astral apex
twilit forum
#

What about vulture, pycodestyle, wemake?...

brazen forge
#

it's open source, contribute :P

gloomy scaffold
#

ruff lowkey supersedes needing to use another linter

twilit forum
#

Like pycodestyle which should be bare minimum annoyed 😅

brazen forge
twilit forum
brazen forge
fleet jewel
#

most are implemented

#

fwiw pretty much every 3rd party rule set has a similar issue for tracking implemented rules, it's quite handy

brazen forge
#

I see

fleet cloak
#

Does anyone use git commit signing on WSL, and if so have you been able to cache the GPG key?

This is my ~/.gnupg/gpg-agent.conf file contents but I have to enter my key every 10 minutes or so (Yes i've killed the agent gpgconf --kill gpg-agent)

default-cache-ttl 34560000
max-cache-ttl 34560000
pinentry-program /mnt/c/Users/User/AppData/Local/Programs/GnuPG/bin/pinentry-basic.exe
gloomy saddle
brazen forge
#

Jane Street isn't the place I'd have thought to talk about uv lol

eager rose
#

actually quite good

vocal pewter
fleet cloak
#

I didn't even know you could sign with SSH. I fixed the issue and all I had to do was add my agent config to Windows
To make it worse I had installed Gpg4Win which uses %APPDATA%\gnupg\gpg-agent.conf as the config file instead of ~/.gnupg/gpg-agent.conf

inner locust
#

Hey all I’m a sysadmin for a university in Boston and I’m a part time lecturer at a community college in my area.

My students have a github portfolio as part of their grade and required some teaching materials.

So I created this guide along with adding the free copy of pro git to help out.

All completely free ofc

https://radiantheavy.gumroad.com/l/sjamuk?layout=profile

Gumroad
GitHub Starter Pack

Ready to level up your coding journey? The GitHub Starter Kit is your all-in-one resource to dive into the world of version control and showcase your projects.Inside this starter kit, you'll find everything you need to get started with GitHub:📘 Crash Course and Setup Tutorial PDF: A step-by-step guide to help you create your GitHub account, conf...

alpine horizon
#

why is the ruff pre-commit script output so different from the regular ruff?
this is what I have in my pre-commit file:

- repo: https://github.com/astral-sh/ruff-pre-commit
  rev: v0.6.9
  hooks:
    # Sort imports
    - id: ruff
      args: [ --fix, --select, I] # `--select I` sorts the imports
    # Run linter
    - id: ruff
      args: [ --fix]
    # Run the formatter.
    - id: ruff-format
      args: [ --verbose ]
#

its so much more verbose, it lists every file-path (even those that it didnt find anything wrong)
I just want it to print the diff of what it did, or what should be fixed

brazen forge
#

show some sample output
also, how are you running pre-commit?

#

manually or the normal when-you-commit

thorny shell
#

my hunch: it's pre-commit that is emitting that output, not ruff.

gentle solstice
#

Also, you should run the isort rule with the rest. ruff can handle it.

rotund gale
#

Hi ya'll I'm working on an open source django project, where would be the best place to post to get feedback?

thorny shell
#

@rotund gale they have a #show-and-tell channel

rotund gale
shy quiver
#

I'm not sure if this is the right channel for this.

Is it possible to make a custom file extension that acts nothing like any of the existing? Because usually, you just make a ".reg" and specify what already existing filetype to interpret our new filetype as. It's just renaming it, basically.

My specific application is secure data storing. You encrypt data with some key(which is the previous yo your current one), then add that key unencrypted, then encrypt all that with a key that is currently stored in RAM. When you exit the program, the key stored in ram is written into that file as that initial previous key

#

So you basically chain encryption keys

rapid sparrow
#

it is a matter only of supported software to run them

frank fractal
lost coral
#

gotcha

fair mantle
#

I have a django+docker question,

How do you guys handle migrations? Say you make changes to existing models or/add models, how do you propagate this changes in prod. My idea was, to have an entrypoint.sh that will run makemigrations and migrate then runserver. Is this good practice?

thorny shell
#

I run makemigrations only on my laptop. In prod, I run "migrate"

# 
COPY --from=poetry-install /bridge/ /bridge/
COPY /project /bridge/project/
WORKDIR /bridge/project
ENV PGHOST=postgres
ENV REDIS_HOST=redis

CMD ["bash", "-c", "poetry run python manage.py collectstatic --no-input && poetry run python manage.py migrate && poetry run daphne --verbosity 3 --bind 0.0.0.0 --port 9000 project.asgi:application"]
#

that's the tail end of my dockerfile

rapid sparrow
#

So... devs just press "migrate" then "deploy" and magic happens.
Smth bad happened? pick another version to deploy for rollback, and go to "shell" for reverting migrations

#

====
At my pet project where i of course don't have such stuff...
...i just made migration as invokable script in container manually.
Just running automigrate on script start...
but if it was serious app, i would have probably left it as manual action

fair mantle
rapid sparrow
fair mantle
#

I think that's my mistake. My migrations folder is gitignored and docker ignored as well.

hazy fox
#

I’ve a memory leak in a live application, if I ssh into the server and start the python interpreter, run tracemalloc in the intrepreter, would I get the statistics for the server? Or would it just print stats w.r.t the interpreter?

fair mantle
hazy fox
swift vapor
#

Hi y'all! I was thinking about "State of the Art Python in 2024" - these are some defaults I came up with. What do you think (not all of them tooling, sry)

  1. Use uv for dependency management (and everything else)
  2. Use ruff for formatting and linting
  3. Support Python 3.9 (but use 3.13)
  4. Use a pyproject.toml file
  5. Use type hints
  6. Use pytest instead of unittest
  7. Use click instead of argparse
rapid sparrow
# swift vapor Hi y'all! I was thinking about "State of the Art Python in 2024" - these are som...

never saw uv, but looks fun for big projects to adopt. marking to check.
I would commend though that for small stuff pip install -r requirements.txt -c constraints.txt is all that is needed, no extra dependency manager on top is justified for small stuff.

Support Python 3.9 (but use 3.13)
ergh, sure, i would comment though that no point to support less than python 3.11 at this point in general. If it is your private app at least
Because it was already released October 24, 2022, so all the ecosystem already caught up with it.

Also python 3.13 was released too much recently, i would recommend using 3.12 instead as latest for the nearby half of a year at least... or a year.
At least if u wish having no weird problems no one had time to fix in libraries to support python 3.13.
This is valid for regular apps.

if you are library maintainer, u are encouraged to switch sooner to support 3.13 and supporting more min python version

Use type hints
sure.
Also consider having auto generated openapi for web projects.
Enforce with properly configured mypy or pyright

Use pytest instead of unittest
without doubts best default in 99%+ cases. unittest justified is almost never.

Use click instead of argparse
pulling extra dependencies is meh unless justified.
So no, unless really needed
For web django projects common to go for django admin command configuration ^_^

Use a pyproject.toml file
ergh? i don't know why this sugestion, but i can say that for package releasing having setup.py is more convinient than static file of configuration, because easier to automate dynamic README, and version and dependencies setup.
I also prefer to use separate configs per tool than messing everything into pyproject.toml, looks cleaner to me option

ruff
having no opinion about it, but estimating that on big projects there is a point for its adoptation.
We just use flake8 in CI though and it works good enough even on million of code lines, so not really a big reason to switch

swift vapor
# rapid sparrow > never saw uv, but looks fun for big projects to adopt. marking to check. I wou...

Great comments! When it comes to pip and requirements.txt what I really miss is a lock file to keep your direct dependencies away from your indirect (pip freeze). And you sort of have to freeze to get reproducible builds.

And ruff is awesome! Since it's so fast I format and lint (auto fix) as a commit hook and it takes like two sec on a large-ish project. 🤩

Tried to put some more background to the default choices in this blog post:
https://anderssundman.medium.com/state-of-the-art-python-in-2024-041c56dc0cae

Medium
State of the Art Python in 2024

What are the best practices for creating a Python application in 2024? Here’s a look at some modern tooling and library defaults.

rapid sparrow
# swift vapor Great comments! When it comes to pip and requirements.txt what I really miss is...

Great comments! When it comes to pip and requirements.txt what I really miss is a lock file to keep your direct dependencies away from your indirect (pip freeze). And you sort of have to freeze to get reproducible builds.
this is why i suggested pip install -r requirements.txt -c constraints.txt
Keep direct manual requirements in requirements.txt, pip freeze to constraints.txt.
Constraints only constrain, they lock versions only

rapid sparrow
swift vapor
#

I didn't even know about the constraints solution! I have to take a look at that. Do you know how it works with dependabot?

rapid sparrow
#

this approach is nice with lib development also, since you could just directly read requirements.txt from setup.py for project building (to have set requirements for your lib installation)
And at the same time reuse same file for dev setup

#

still never used dependabot

swift vapor
#

Nice-to-have is the split of dev vs prod dependencies in a pyproject file, but it's seconday to the lock-file issue

rapid sparrow
#

Still a single pip native command to do that

swift vapor
#

Hehe ofc

rapid sparrow
#

unittest sometimes does have usability btw

#

...if u developed library that has no dependencies...

#

u could be wishing... screw all dependencies and going with unittest instead of pytest

#

also useful during AWS lambda python development, they have difficulties with installig libs, some convinience to go less libraries is somewhat present. nothing forbids to use dev dependencies like pytest though

swift vapor
#

Yeah, the first is definitely not free when it comes to dependencies

rapid sparrow
#

it really storngly pairs

#

Typing + enforcing with mypy/pyright + using data structs library like pydantic that enforces them at run time(alternative solutions are possible)

#

pydantic has inbuilt serialization to and out and can generate openapi, nice default choice, and runtime validations (other options can be just using default data structs, or typed dict, or marshmallow stuff and etc)

#

pydantic can be added pretty much to any project for this reason that follows typing approach

swift vapor
#

That's a really good suggestion. I should write "more good python defaults in 2024" 😂 or you write it

rapid sparrow
#

without data structs people use untyped dicts full of magical numbers and values which really badly affects project quality
And just carrying values in regular classes has its own drawbacks over having convinient data structs

swift vapor
#

Need to drive home now 🚗afk for a few hours

rapid sparrow
#

Since logging is important aspect to simplify debugging of a project, on some comparable level standing with other aspects of project quality... it is kind of imiportant to do for serious projects

#

Some languages made this stuff as proper std libs easy to use

#

python just has article in official docs about it with example on how to do on std stuff only. Third party solutions exist for more convinient approach

rapid sparrow
#

not really feeling like writing about python default is good article tbh

#

very opinionated stuff

#

although plenty of my other articles were opinionated too

#

but hmm... somehow it feels like topic nice to discover on your own

#

i appreciate though mentioning uv thing which i did not saw before, i should check it

alpine horizon
#

I'm not very good with dockers, but hows it possible that in line 34 it names the database "postgres", and then when it passes the name to the API, it says its name is "langfuse" instead (line 15)?
same thing for username and password

rancid schoonerBOT
#

docker-compose.yml line 34

- POSTGRES_DB=postgres```
rapid sparrow
#

my database related code working with multiple production db and needing to migrate it for monitoring / read only users has a similar feature

#

fallbacking to postgres logical db is common due to this database always existing usually within physical instance

#

Or may be he just has wrongly written docker-compose, that can be too

#

he could have forgotten to align variables properly

#

and his code works on older data from already existing volume where he previously already created logical db langfuse and forgot to correct his docker-compose code

#

that is common mistake too

alpine horizon
#

I'm really confused how the service langfuse is even able to connect to this non-existent DB called langfuse??

rapid sparrow
#

postgres to conect logical instance is very common, may be all postgres physical instances have such db

alpine horizon
rapid sparrow
#

Have u checked it is working at all

alpine horizon
#

btw I think you mentioned Hetzner once

thorny shell
alpine horizon
#

what do you recommend for a cheap RDS (postgres) for a startup?

thorny shell
#

EC2 box

#

or, rather, the postgres docker image running in a container on an EC2 box. Or Azure, Hetzner, Linode, whoever

alpine horizon
#

it seems like they dont offer an RDS system, but there are some 3rd party guys that do it on top of Hetzner I believe, like ubicloud

thorny shell
#

the hosted options, like AWS' postgres-in-da-cloud, are eye-wateringly expensive

alpine horizon
#

or do you?

thorny shell
alpine horizon
#

anyone can docker run the image, but backups, connecting to IP directly, and other features wont work

thorny shell
#

last co I worked for -- big serious outfit that you've heard of -- did this. Granted it wasn't a critical piece of their infrastructure, but still.

#

connecting to IP directly is nothing more than exposing the relevant port. Backups can be as simple as a cron job. Look at "cookiecutter-django" for an example

#

if you're a startup, you should expect have to roll your own stuff sometimes, in order to save money, no?

alpine horizon
#

but what do you think of ubicloud? It seems like theyre more expensive then hosting it yourself ofc, but cheaper than AWS RDS

thorny shell
#

nhoi

alpine horizon
#

so I tried doing sudo -u postgres psql -c 'create database langfuse;' to create the DB myself.
then I commented out this part of the docker-compose:
https://github.com/daveebbelaar/langfuse-self-hosted/blob/main/docker-compose.yml#L23-L42
and passed the URI for the DB on my localhost.

but I get the error

langfuse-server-1  | Error: P1001
langfuse-server-1  | 
langfuse-server-1  | Can't reach database server at `localhost:5432`
langfuse-server-1  | 
langfuse-server-1  | Please make sure your database server is running at `localhost:5432`.
langfuse-server-1  | 
langfuse-server-1  | Prisma schema loaded from packages/shared/prisma/schema.prisma
langfuse-server-1  | Datasource "db": PostgreSQL database "langfuse", schema "public" at "localhost"
langfuse-server-1  | 
langfuse-server-1  | Error: P1001: Can't reach database server at `localhost:5432`
langfuse-server-1  | 
langfuse-server-1  | Please make sure your database server is running at `localhost:5432`.
langfuse-server-1  | Applying database migrations failed. This is mostly caused by the database being unavailable.
langfuse-server-1  | Exiting...
thorny shell
#

can't tell what you're doing since I can't see your terminal

#

if I can clone a repo, and issue just a very few commands after that, and see what you're seeing, I'll try to reproduce the problem. But if it requires proprietary code or something, I probably can't help.

alpine horizon
#

nvm found a workaround

whole ermine
#

When browsing discussions about linters, I often see Ruff being highlighted as a full replacement for flake8 and other linters. However, when I look at what rules Ruff implements as of today, it doesn't implement all of flake8's checks yet (in particular, a lot of pycodestyle rules are in experimental status https://docs.astral.sh/ruff/rules/#pycodestyle-e-w).
So it seems like it's not a 100% replacement yet?

Rules | Ruff

An extremely fast Python linter and code formatter, written in Rust.

visual oxide
thorny shell
#

I regularly uninstall pycodestyle, just in case it somehow snuck onto my machine. I hate it.

whole ermine
thorny shell
#

because it complains that I haven't added docstrings.

#

I want to slap it.

whole ermine
#

Ah, hmm, that must be disabled by default in flake8 I think.

whole ermine
thorny shell
#

ah indeed I was

#

I think 🙂

#

honestly I have so many different tools yelling at me, I don't know which is which

whole ermine
thorny shell
#

especially irksome is someone complaining that I have comments like # type: ignore, and that I should include the code number of the error that I want to ignore; but I have no idea what that number might be. I just know that without that comment, mypy breaks 😢

vocal pewter
#

fwiw, mypy's error tells you what type of error you might want to ignore.

sample.py:5: error: Function is missing a type annotation  [no-untyped-def]
Found 1 errors in 1 file (checked 1 source file)

So # type: ignore [no-untyped-def]

astral apex
#

# type: ignore [no-untyped-def]

#

Does it work without the colon?

vocal pewter
#

Good shout.

gloomy scaffold
astral apex
#

you put that back!!

gloomy scaffold
#

ahhahahhah

thorny shell
visual oxide
thorny shell
#

well I keep everything up to date obsessively so I doubt it's that

#

how might I have disabled the error codes?

#

aaaaah [import-untyped]

visual oxide
#

I assume for people who parse the output of mypy rather than relying on the JSON output mode

fleet turtle
#

Hello, I am trying to use locust for api testing in an virtuel environment (venv) but I always run in the same error when I try to start locust:

ImportError: DLL load failed while importing _psutil_windows: Wrong parameter.

I am out of ideas. reinstalled psutils already.

visual oxide
fleet turtle
# visual oxide Do you have the full traceback? 
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "H:\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Scripts\locust.exe\__main__.py", line 5, in <module>
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\locust\__init__.py", line 25, in <module>
    from .contrib.fasthttp import FastHttpUser
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\locust\contrib\fasthttp.py", line 3, in <module>
    from locust.env import Environment
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\locust\env.py", line 11, in <module>
    from .runners import LocalRunner, MasterRunner, Runner, WorkerRunner
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\locust\runners.py", line 25, in <module>
    import psutil
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\psutil\__init__.py", line 110, in <module>
    from . import _pswindows as _psplatform
  File "\\?\UNC\FileserverHaag\home\Dokumente\DHBW\Studienarbeit\Code\Tools\performance\tests\venv\Lib\site-packages\psutil\_pswindows.py", line 35, in <module>
    from ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASS
ImportError: DLL load failed while importing _psutil_windows: Wrong parameter.

Yeah, here it is.

rapid sparrow
# 
$ python3 -m build --wheel "gill"
* Creating isolated environment: venv+pip...
ERROR Source gill does not appear to be a Python project: no pyproject.toml or setup.py

I get confusing error from the building command... which should not be

Screenshot_20241105_012508.png
#

in two other repos it works

#

in another repo i configured it to work like just 2 hours ago and all was okay

#

very weird for error to appear

#

it appears in CI too :/

#

not catching smth

#

Probably i should go to sleep, at morning it will be obvious why

# 
l$ python3
Python 3.11.6 (main, Oct 23 2023, 22:48:54) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gill

if i in same folder turn on python shell and import package, alli is okay hmm

#

package gill does have __init__.py all is okay

#

Wtf. exactly same setup.py wprks in another repo and doe snot work in this one... hmm

# 
import os
from codecs import open
from pathlib import Path

from setuptools import find_namespace_packages
from setuptools import setup

here = Path(__file__).parent.absolute()

with open(str(here / "README.md"), encoding="utf-8") as f:
    long_description = f.read()

build_version = os.environ.get("PACKAGE_VERSION", "")
if build_version and build_version.startswith("v"):
    version = build_version[1:]
else:
    version = "0.1.0.dev0"


setup(
    name="gill",
    version=version,
    install_requires=[],
    long_description=long_description,
    url="https://github.com/user/repo",
    author="author",
    license="UNLICENSED",
    keywords=["django", "django rest framework", "celery"],
    python_requires=">=3.8",
    include_package_data=True,
    packages=find_namespace_packages(include=[
        "gill*",
        "oscar*",
    ]),
    package_data={
        "gill": ["py.typed"],
        "oscar": ["py.typed"],
    },
)
#

hmm exactly setup setup, nothing weird here, it perfectly works in other repos hmm

# 
$ python3 -m build --wheel "gill"
* Creating isolated environment: venv+pip...
ERROR Source gill does not appear to be a Python project: no pyproject.toml or setup.py

Still same strange stuff 🤔

#

um. i deleted all possibles from root

#

this repo has difficult hirerachy of stuff in its gill folder... perhaps smth weird there may be

#

but what.. python wheel is not telling any useful error

#

Oh

#

Managed to run succesfully

#

executed command as python3 -m build -v

#

Strange

#

🤷‍♂️

#

okay in CI the same command is used, strange for Ci to fail
Lets just retrigger CI and see then

#

Oh.

#

CI succeeded 😏

#

Shrugs. dunno why did not work, dunno why it works now

visual oxide
fleet turtle
visual oxide
fleet turtle
fleet turtle
visual oxide
#

Hmm

#

Try with python 3.12.7 and regular pip instead of uv

#

Also it might be that "\\?\UNC\ that's preventing DLL loading

#

What is that

#

FileserverHaag sounds like it's a remote server, generally virtual environments are not portable between systems. You need to make a virtual environment for each physical environment

fleet turtle
visual oxide
fleet turtle
#

Mocked was maybe the wrong expression. So I added the NAS as a Network, and i am writing directly on it. So my documents are not on C: ..., they are now directly on the NAS.

visual oxide
#

I see

visual oxide
#

Did you try with 3.12.7? Did you try with regular pip instead of uv pip?

fleet turtle
fleet turtle
visual oxide
alpine horizon
#

I want to run a docker image from a GH actions to a digital ocean droplet

#

issue is that the ubuntu droplet dosent have Docker installed by default, so what am I supposed to do? (Its like 5 commands to install it...)

#

or am I supposed to log in the console and set it up manually?

rapid sparrow
#

the dark duck comes to the rescue

rapid sparrow
#

Literally Docker on Ubuntu 22.04 image is available there at Digital Ocean

#
  1. there is available more messy choice a bit, Github Actions comes with preinstalled Ansible. it is perfect to run from CI and to target some host machine and install docker there if it is not installed there. Ansible was made to run idempotent, only if it is needing to run
    This choice is mostly good only for pet projects, for most lazy people basically which for some reason were not satisfied with first using
    https://www.digitalocean.com/community/tutorials/how-to-use-ansible-to-install-and-set-up-docker-on-ubuntu-20-04 code like could be executed from within GH without problems
    Ansible as a tool was made for remote executions of some script on some server to install stuff like Docker, literally what it was made for that, and exactly flares well for executing from CI
How to Use Ansible to Install and Set Up Docker on Ubuntu 20.04 | D...

Docker is an application that simplifies the process of managing application processes in containers . This guide explains how to use Ansible to automate th…

#

just using server with already installed Docker is better though

rapid sparrow
#

Yay. 3.13 also has free threading.

sterile nexus
#

So basically this things like a better interactive interpreter are really helpful

ocean mica
#

anyone uses poetry here?

#

needed some help

thorny shell
#

just ask your question

#

@ocean mica just ask

neat pulsar
#

is Poetry a good tool to get into as a beginner?

short peak
neat pulsar
#

I heard of it, but never used it

#

I also never built a real app, just some tutorial stuff

astral apex
neat pulsar
#

which tools are more beginner friendly?

#

I usually just create my project dir in my vscode console and also create the venv there too

astral apex
#

Take a look at uv

gloomy scaffold
#

@astral apex I love your pfp so much

#

so cute :>

astral apex
#

tyty

astral apex
gloomy scaffold
#

my favorite is the paper towel

mint cipher
#

After a few days of trying out different free services to deploy django + celery, I came to the conclusion that its not possible. If you wanna use Django + Celery, you have to host your own VPS. A longer answer of how I find out about it is in my blogpost if anyone is interested to learn more: https://buymeacoffee.com/qoyyuum/the-struggle-django-deployments

rapid sparrow
#

And even then only temporally

#

From 3 to 12 months

#

And very limited

rapid sparrow
#

Since u can use actually VPS, it would be far better experience
I don't like to recommend AWS, it is not noob friendly at all. Highly overly complex

#

Digital ocean would be the smoothest experience, it is noob friendly
The average prices of DO somewhat suck though

rapid sparrow
mint cipher
mint cipher
rapid sparrow
#

big providers like AWS, GCP, Azure give free credits, but they are too complex for novices

#

and DO... is 2-4 times more expensive than Hetzner. Not really nice in a long run at all

rapid sparrow
#

and super stuff... arm prices, twice more hardpower for same price

Screenshot_20241111_033809.png
#

perfect to run pet projects, or even your own game servers 😋

#

besides cloud servers, u could also get from its auction long term cheaper offer for dedicated servers if necessary

mint cipher
#

You're right, the specs on the lowest range beats DO by at least double

rapid sparrow
#

arm beats 4 times 😄

mint cipher
#

Is Hetzner a provider on Terraform?

#

It is

#

Nice

rapid sparrow
mint cipher
#

Can almost easily swap my terraform script for it

rapid sparrow
#

small Ansible Playbooks invokable by terraform provider Ansible-Playbook help me to install some extra stuff if necessary like microk8s

#

i prefer to run projects by just docker though for which Hetzner has already image

#

microk8s is more for experiments

rapid sparrow
#

using it over SSH connection for the provider

mint cipher
#

I never did understood the Docker provider on Terraform. Does that mean I would have some sort of socket access to it remotely?

rapid sparrow
# mint cipher I never did understood the Docker provider on Terraform. Does that mean I would ...

https://github.com/darklab8/fl-darkbot/blob/master/tf/production/main.tf as example

provider "docker" {
  host     = "ssh://root@${module.server.ipv4_address}:22"
  ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", "-i", "~/.ssh/id_rsa.darklab"]
}

module "darkbot" {
  source              = "../modules/darkbot"
  configurator_dbname = "production"
  consoler_prefix     = "."
  secrets             = local.secrets
  tag_version         = "v1.6.10"
  mode                = "docker"
  environment         = "production"
  debug               = false
}
#

DOCKER_DAEMON=ssh bla bla we use

#

and thus able to use remotely

mint cipher
#

ah ok

rapid sparrow
#

this is all usable locally too remotely over ssh with regular docker cli 🙂 its default feature

#

makes very comfy things

#

all terraform

#

that cleans after itself

#

not having any discomfort to run projects like that in years terms

#

same server utilized for long terms remains very organized/clean

mint cipher
rancid schoonerBOT
#

tf/production/providers.tf lines 30 to 32

provider "cloudflare" {
  api_token = data.external.secrets_cloudflare.result["token"]
}```
rapid sparrow
#

thus i could avoid need to configure letsencrypt 😄

rapid sparrow
#

And it is stored in encrypted records in git repo for me 🙂

#

which i backup for several locations by just running git push

#

i like it, finally having some comfortable sense of the password and otp madness 😄

#

when in invoke terraform commands, i have to insert one time password to access those tokens though, if i keep them in password protected folder

#

Originally used AWS ssm stuff, but migrated for pet projects to pass for more long term convinience and ownership of my secrets

mint cipher
mint cipher
rapid sparrow
#

@mint cipher also on unrelated topic regarding tools. Checked your "blog" web site.
https://darklab8.github.io/blog/#how_this_site_was_made
I chose to build mine as static site generator via templ
Very comfy choice in terms of customizing, as i plug any features I wish (from syntax coloration to support of writing in markdown) and full ownership 😋
Nice for writing blog as a code that is autodeployed by Github Actions on every commit

Andrei Novoselov's Blog

Andrei Novoselov's articles, pet projects and favourite stuff

#

as minus i have to make every feature too though, so smth payment related will be probably pain in the ass to add

mint cipher
# rapid sparrow <@930453529192181770> also on unrelated topic regarding tools. Checked your "blo...

I'm trying out BMAC as a way of getting payment. Coming from Brunei, a 3rd world country that is behind in accepting international payments, I'm forced to use something like Payoneer or if I have the extra extra money, Stripe Atlas. BMAC just happens to be free for me to use. They'll just cut a portion of the transaction for each money earned on the site and I think that's fair.

Plus, they do have that webhook event that I have been eager to try out in an actual project and it was fun doing it. Will definitely be adding more of it to other side projects.

On top of that, I need to get into the habit of documenting my findings/learning as I do these things as my own references. I do keep my own notes (I use https://obsidian.md btw and uses Markdown) but if I find that its worth sharing, I post them.

Obsidian - Sharpen your thinking

Obsidian is the private and flexible note‑taking app that adapts to the way you think.

rapid sparrow
#

i tried Obisidan and found too much complex for my purposes 😄 i probably should try again eventually.
Keeping my markdown to vscode... as well as majority of other languages. Everything from vscode.

mint cipher
#

That said, BMAC does lack the syntax code highlights or the ability to let me write my posts in markdown. 🤞 Here's hoping they'd listen

#

The plugins on Obsidian does a bit more. What extensions do you use on VSCode on top of markdown preview?

mint cipher
#

Spoke too soon

image.png
#

A little old tho. 2017 last released

rapid sparrow
#

there exists simple python lib able to watch filesystem and hot reload any command on file changes

# 
version: "3"
tasks:
  sphinx:live:
    cmds:
      - > 
        watchmedo auto-restart 
        --patterns "*.py;*.rst;*.yml;*.md" 
        -i "./docs_build/*"
        --recursive
        -- sh -c "task sphinx:build; python -m http.server -d docs_build/html"

my taskfile.dev command for sphinx hot reloading for example on those files changes

#

it could be mkdocs, or templ i use and etc

#

watchmedo is python lib watchdog[watchmedo]

#

here how i keep writing blog

  dev:web:
    cmds:
      - templ generate
      - go run . --act=web
  
  dev:watch:
    cmds:
      # pip install watchdog[watchmedo]
      - > 
        ~/venv/bin/watchmedo auto-restart 
        --patterns "*.templ;*.go;*.css;*.js;*.md" 
        -i "*_templ.go;build/*"
        -d blog
        --recursive
        -- sh -c "task dev:web; sleep infinity"

reloading for my templ

# 
package main

import (
    "flag"
    "fmt"
    "net/http"
    "strconv"

    "github.com/darklab8/blog/blog"
    "github.com/darklab8/blog/blog/settings/logus"
)

type Component interface {
    Write()
}
type Action string

const (
    ActionWeb   Action = "web"
    ActionBuild Action = "build"
)

func main() {
    var action string
    flag.StringVar(&action, "act", "undefined", "action to run")
    flag.Parse()

    fmt.Println("act:", action)

    web := func() {
        fs := http.FileServer(http.Dir("./build"))
        http.Handle("/", fs)

        port := 8080
        logus.Log.Info("Listening on " + strconv.Itoa(port))
        err := http.ListenAndServe(":"+strconv.Itoa(port), nil)
        logus.Log.CheckPanic(err, "unable to serve serve")
    }
    switch Action(action) {
    case ActionBuild:
        blog.Builder.BuildAll()
    case ActionWeb:
        blog.Builder.BuildAll()
        web()
    default:
        panic("action is not chosen")
    }

where entrypoint command is this for templ stuff

#

so usually i just keep running console command task dev:watch

#

change markdown via vscode

#

and watch browser for changed rendered pages

mint cipher
#

I see

fierce geyser
#

Hi I am totally new here, am I at the right place to ask about Python + Playwright (test framework) ?

visual oxide
#

You'll need a text editor, vscode is popular

visual oxide
#

What is it?

#

Do you have a URL for where you found this code?

warm blade
#

!rule 5, we don't help with credential stuffing

rancid schoonerBOT
#
Rules

The rules and guidelines that apply to this community can be found on our rules page. We expect all members of the community to have read and understood these.

warm blade
#

i see "combos", we both know what those are.

gloomy scaffold
#

then it's rule 19 — that combo is horrendous

gloomy scaffold
#

who combines rice and fries

warm blade
ruby surge
#

Hey does anyone knows how to configure sqlfluff on neovim?

frank fractal
brave zealot
sharp sandal
#

Hi team

#

I have problem with docker of django project

#

Whenever I deploy my django project on aws with docker compose, it is consuming the disk space.

#

What might be the result?

#

I have added wipting old containers and images in Jenkins file

#

Very weird

#

it's taking 0.8G - 1.1G whenever I deploy my service

#

but I think this is normal

#

because

#

the venv has 500 MB

#

and I am using about 6 containers

#

and each image has 50 MB so containers have 300 MB

#

Finally the disk has 300GB

rapid sparrow
#

each time when u deploy new image

#

you have added new set of layers to your local docker daemon

#

if you would have used Docker layers correctly, with properly caching dependencies, the difference between images would be usually only equal to amount of addded new code

# 
copying deps only
installs them

only then copying the rest of a code

stuff like that

#

it would have still increased taken size but in minimal way (only for amount of added code)

#

anyway...

#

you can clean older orphaned stuff with docker system prune command

sharp sandal
#

yes, I did that command

rapid sparrow
#

optionally if you would have used Terraform Docker for deploying, it would have automatically deleted previous image versions before loading the new one

rapid sparrow
#

may be u abuse volumes or smth

#

or some other unused images are accumulated (checkable by docker image ls -a

sharp sandal
#

ok

#

thanks for your kind explanation

ebon kraken
#

@graceful hinge If you can't find any for less you could spin up zsh on windows and use less there.

graceful hinge
#

I actually used oh-my-posh until I moved to starship

#

And I am looking for any available postgresql auto-complete

ebon kraken
opal verge
#

is here have good tools for software testing work

gloomy scaffold
#

that is one of the broadest questions I've ever heard

civic solar
#

Hey, I've been using ipdb for the last 12 years. I was wondering if there's anything new out there that improves upon ipdb or that's "better"? Any recommendations are welcome. Thank you.

thorny shell
#

heh, I only learned about ipdb in the last month or two 🙂

rapid sparrow
#

ipdb I use as last resort in some situations

harsh hemlock
#

Hi guys, do you know how is the new compiler from py13? is it easy to use and supporting all the packages?

thorny shell
#

?

#

python 3.13 goes through the same testing process as all the previous versions

#

I'm sure it works fine

#

as with other new releases, some 3rd-party projects may not have yet published binary wheels for it, so don't be surprised if your existing project doesn't "build" with 3.13. But they will get around to it eventually.

toxic raptor
#

hi random question , so basiclly if i create a Docker of a flask app which work as media manager , can a docker container assess file outside its docker or its it fully isolated ?

thorny shell
#

depends

#

you can "mount" a "volume" so that the container can see files on e.g. your laptop

#

probably best to ask in a dedicated docker discord

steel quail
#

We just have that issue at work, where a docker image fails to process some steps because of a apt-get update/ugrade command. so I'm actually wondering: what's your default go-to about apt in docker?

rapid sparrow
# steel quail We just have that issue at work, where a docker image fails to process some step...

apt upgrade is pointless in general imo... because it is not really justified increase of docker image bloat and building time? it decreazes its reproducability also (there should be desire to aim for the most reproducability of an image building if possible, having more controlled upgrades.)
Increment base image to later version instead.

apt-get update is good, never really had problem with it inside docker. and at work we have good caching implemented for the building part that includes apt operatations so we don't really notice issues regarding it unless we update our Pipfile.lock (that makes us forced rebuilding docker image building stage). We save/pull build stage of docker image in a docker registry for CI, and it pulls build with hash equvalient to Pipfile.lock in next CI steps in order to avoid apt/pipenv sync --system commands run.
Our docker build stage caching mechanism works great with our stateless github runners running in autoscaling AWS group thing

There is a good practice to clear apt-get cache after its usage inside docker image building, i am usually too far/lazy for that.

steel quail
#

yeah I couldnt say which of the two is yielding the issue (I just removed both tbh). the use case here is that I got a new machine with everything fresh on it, and those images started to just fail to build pandas. turns out it was the apt upgrade/update that was causing it

I guess hiding it in the base image is best, indeed

rapid sparrow
#

exceptions i saw only for things like manualled added ppa smth to install latest python

steel quail
#

here it was based on the same image, but the dockerfile tried to install pandas 1.2.1 ; it just fails if I run RUN apt-get update && apt-get upgrade -y first; no idea why exactly. it works fine if not

rapid sparrow
rapid sparrow
steel quail
#

yeah I feel like updating the packages changed smth at the C level, indeed

#

that's how I thought about apt being the cause 😄

olive beacon
#

would anyone be intrested in a black lister / anti nuker

#

so if one of ur admin or a dumb ass k1d tryed to fack with u they aint getting pass deleting none with out a password 😉

rapid sparrow
rancid schoonerBOT
#
Rules

6. Do not post unapproved advertising.

gentle solstice
valid ingot
#

hi

#

have anyone tried to format files on git hooks without staging the files again?

astral apex
#

Have you looked into pre-commit?

valid ingot
#

yeah

#

doesn't work for me cause it tries to run github actions locally and can't disable that as far as I know

#

and also some formatting tools run on node

#

so I have a package.json that installs prettier and some of its plugins, and also husky because it lets me run bash scripts

#

then ruff for python, that depends on the venv shell environment which isn't sourced by any git tool and etc

#

I try to commit from the vscode UI and it fails because it doesn't souce pip activate script

#

so I have to commit from the terminal where it is always sourced

#

thing is, its a Windows app and workstation, so having unix stuff here and there is really annoying

valid ingot
#

I believe the best is to have it unified between github actions and workstation

#

so I'm writing a powershell 5.1 script

gentle solstice
#

pre-commit doesn't run github actions. They're unrelated.

valid ingot
#

and the github action should run with windows nano image

gentle solstice
#

!pip pre-commit

rancid schoonerBOT
#
pre-commit v4.0.1

A framework for managing and maintaining multi-language pre-commit hooks.

Released on <t:1728403777:D>.

valid ingot
gentle solstice
#

did you want a push hook instead that automatically fixes your code on push?

valid ingot
#

then why it lists the github actions and fails because it can't run them

gentle solstice
#

What error are you getting?

valid ingot
#

also formatting

gentle solstice
#

What's the error, what did you do to get the error?

astral apex
valid ingot
astral apex
#

Really?

#

Where?

valid ingot
#

Windows?

# 
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Try the new cross-platform PowerShell https://aka.ms/pscore6

PS C:\Users\marcos> $PSVersionTABLE

Name                           Value
----                           -----
PSVersion                      5.1.19041.5129
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.5129
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
#

on Windows 10 Pro 22H2

#

Powershell Core 7.5 standard library is much much nicer, but I haven't needed anything from it yet except a command/syntatic sugar to double quote strings which is trivial to implement

gentle solstice
#

the exe is pwsh

valid ingot
#

yes

gentle solstice
#

any reason you can't install it?

#

(it doesn't really matter anyway)

valid ingot
#

minimal dependency overhead > verbosity

#

for this project at least

gentle solstice
#

What was the issue you were having with pre-commit?

valid ingot
#

updating index commited unstaged changes

gentle solstice
#

and your pre-commit config?

valid ingot
#

I solved it

#

I was forcibly updating the index insted of updating the cached file

#

before
git update-index --again
after
git update-index --add --cacheinfo 100644 "$blobhash" path/to/file

valid ingot
sonic star
#

Y'all
I'm building a tool that will make you share sensitive info with others
and when a user received a shared variable from another user, it's encrypted
there's a tool to decrypt it by making a request to the project server.
Now when it decrypts it, I can't seem to find a way to set the variables in a way that the user never gets to see or access it
Anyone knows a way to set your environment variables without being about to see it with the env command on linux?

cosmic bolt
#

Who can teach me how to run onnx model on NVIDIA GeForce MX130?
I will pay $100 instantly.

astral apex
#

!rule paid

rancid schoonerBOT
#
Rules

9. Do not offer or ask for paid work of any kind.

red oxide
#

How to trigger a pipeline on gitlab when MR is "merged" or "closed"?

I tried to look into environment but it just doesn't seem to work for me.

Here's how I am using it and would love some inputs to hopefully get this working.

.gitlab/config/shared-ci.yml

.shared_deploy_sandbox:
  stage: deploy_sandbox
  variables:
    <<: *ansible_variables
  before_script:
    - *shared_setup_ssh_script
    - *shared_setup_ansible_script
  script:
    - ./sandbox.sh origin $CI_COMMIT_REF_NAME destroy
    - ./sandbox.sh origin $CI_COMMIT_REF_NAME deploy 2>&1 | tee deploy.log
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_LABELS =~ /need-product-review/'

.shared_destroy_sandbox:
  stage: destroy_sandbox
  variables:
    <<: *ansible_variables
  before_script:
    - *shared_setup_ssh_script
    - *shared_setup_ansible_script
  script:
    - ./sandbox.sh origin $CI_COMMIT_REF_NAME destroy
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && 
           $CI_MERGE_REQUEST_LABELS =~ /need-product-review/ &&
           ($CI_MERGE_REQUEST_EVENT_TYPE == "merged" || $CI_MERGE_REQUEST_EVENT_TYPE == "closed")'

.gitlab-ci.yml

image: python:3.10

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" || $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
...
run_deploy_sandbox:
  extends: .shared_deploy_sandbox
  artifacts:
    paths:
      - ansible/deploy.log
    expire_in: 1 hour

run_destroy_sandbox:
  extends: .shared_destroy_sandbox
...

run_deploy_sandbox works perfectly fine as I want it to but run_destroy_sandbox is not getting triggered on merge or MR close.
Note that I would need to have access to MR branch name in the CI environment so i can use it to execute some task.

Would really appreciate some help, I have been stuck with this issue for days now.

gray obsidian
#

Hey

lilac delta
#

Hey all! The pyenv install instructions on github mention exporting the path through echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc

For MacOS, the more common way to set paths is through /etc/paths or more specifically etc/paths.d/pyenv -- is this a specific choice not to do that in the setup instructions?

broken sleet
#

Good afternoon everyone, I hope you are having a productive day. I would like to ask for your support as I am working on a project for my degree and need recommendations for a couple of things. The project is about deploying and using Odoo. I was initially planning to present it locally on a virtualized server, but I am now considering whether to deploy it in the cloud instead, as it might look more professional and polished. With this in mind, what cloud service providers would you recommend? It won’t be used for production at the moment, just for the demo during the presentation to the jury to determine if I’ll be awarded the degree.

rapid sparrow
brazen forge
#

adding to the shell's path in the *rc file is a common step because the tool's mostly gonna be used interactively

lilac delta
#

From what I've seen, no paths in *rc is far more common than an empty /etc/paths, which I don't think I've ever seen before. But that's kind of anecdotal.

rapid sparrow
#

Prometheus has ability to output data to remote write target into external monitoring cluster if necessary.

Into another Prometheus, or in my case I accept data into horizontally scalable Mimir

unkempt cradle
#

If it’s a production service then it depends on the number of nines you need to guarantee uptime for, but ideally you have some kind of external monitoring for your services in addition to the in-cluster Prometheus. Catchpoint is a SaaS solution many enterprises use, just as an example.

#

If your uptime guarantee isn’t very high you could get away with something hacky like a bash script on a cron that tries to curl your service endpoints (you have health check endpoints on all your services, right?) and sends you an email if it times out or otherwise errors.

#

Yeaaappppp 😂

plain zodiac
#

Generally what is this?

sharp zephyr
#

I need someone who is skilled in doplymenet with django and nginx and docker I have this error
proxy-1 | 172.18.0.1 - - [29/Nov/2024:11:18:09 +0000] "GET / HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36" "-"

# 
FROM nginxinc/nginx-unprivileged:1-alpine

COPY ./default.conf.tpl /etc/nginx/default.conf.tpl
COPY ./uwsgi_params  /etc/nginx/uwsgi_params
COPY ./run.sh  /run.sh

ENV LISTEN_PORT=8000
ENV APP_HOST=app 
ENV APP_PORT=9000

USER root


RUN mkdir -p /vol/static && \
    chmod 755 /vol/static && \
    touch /etc/nginx/conf.d/default.conf && \
    chown nginx:nginx /etc/nginx/conf.d/default.conf && \
    chmod +x ./run.sh

VOLUME /vol/static

USER nginx 
CMD ["./run.sh"]
# 
server {
    listen ${LISTEN_PORT};

    location /static {
        alias /vol/static;
    } 

    location / {
        uwsgi_pass              ${APP_HOST}:${APP_PORT};
        include                 /etc/nginx/uwsgi_params;
        client_max_body_size    10M;
    }
}
# 
#!/bin/sh

 
set -e 
envsubst '${LISTEN_PORT} ${APP_HOST} ${APP_PORT}' < /etc/nginx/default.conf.tpl > /etc/nginx/conf.d/default.conf

nginx -g "daemon off;"
gaunt mist
#

is docker mandatory pre-requisite for learning kubernetes as a novice in deployement concepts?

rapid sparrow
#

Book Docker deep dive will help to learn docker thoroughly

heavy knot
#

Second ^. You can use docker without k8s, but k8s without docker (or another container spec) is not very common.

rocky pewter
#

Is SQL and Docker just basic skills for devs

#

@rapid sparrow

rapid sparrow
# rocky pewter Is SQL and Docker just basic skills for devs

SQL is basic skill yes. it is usable in not web too, just because we have access to high quality sqlite3 db.

Docker... Is more basic skill only for all web related job roles. Web front, backend, data engineering, DevOps stuff.
Its practicality for not web related roles is dubious.

rocky pewter
rapid sparrow
vocal pewter
#

fwiw, Docker is just good to have as a familiarity today for a wide range of areas. It's far from exclusive to any one field. With the growing interest in containerization as a deployment strategy (and all the benefits) you'll likely come across docker files more frequently than not in company or enterprise situations.

brazen iris
#

Anyone who knows about networking

#

I'll will pay money

ionic wadi
#

so i have a new github account, and i want to set up ssh. I've made a new ssh and added it my new github account.

Problem occurs when pushing to the remote as it shows "Permission to [user/repository] denied to [current ssh agent]" error.

can i set multiple ssh for a single computer? or should i just switch the current active ssh?

thorny shell
#

I suspect you haven't told your laptop's ssh client where the new private key is.

#

on Unix, anyway, you can type ssh-add -l to see if the agent "has" your key

ionic wadi
thorny shell
#

what does ssh-add -l say?

ionic wadi
#

im not sure are these safe to paste for public?

#

it shows agents i have added

thorny shell
#

well, it shows ssh public keys that you've added.

ionic wadi
#

oh ok

thorny shell
#

Are you 1,000% certain that one of them is the same one you pasted into github?

ionic wadi
#

yeah i have them, the new one is listed under ssh-add -l

ionic wadi
#

i found something about ssh config files

#

i'll try this out

#

i think it's what i need

thorny shell
#

only other thing I can think of is that your remote specifies a username that is wrong. Mine look like this ```
$ git remote -v
github git@github.com:offby1/bridge-server.git (fetch)
github git@github.com:offby1/bridge-server.git (push)

ionic wadi
thorny shell
#

looks reasonable

ionic wadi
#

looks the same

#

ye

ionic wadi
#

now i know

#

all i have to do is create a ssh config file with a custom Host, and use git@custom_host_name:OWNER/REPO as remote

#

attach the private key to the Host and boom

rapid sparrow
rapid sparrow
rocky pewter
#

is there any open source place or ebook to learn about docker

#

Online courses aren't much in codecademy or no

rapid sparrow
main sparrow
#

Hello, does anybody have experience using pantsbuild or similar? I need some help on how to properly setup a monorepo with multiple prog languages

vagrant thistle
#

ey, I was trying to setup nextcloud server. I did all the portforwarding created hostname from no-ip and added it to my router
But now when I click submit domain on aio page. I get the error "The domain is not reachable on port 443 from whithin this container". Have you opened port 443/tcp in your router/ firewall. If yes is the problem most likely that the router or firewall forbids local access to your domain. You can work around that setting up a local DNS Server.

I even checked my port if it's open or not of canyouseeme.org and it says open there.
Any fix?

rocky pewter
#

so is grabbing info about CI/CD jenkins better beforte going to docker

rapid sparrow
#

unless u have to because of its usage at your current work

rapid sparrow
#

After docker i could recommend using simple to setup and use CI like Github Actions (or Gitlab)

rocky pewter
#

linux status is fine. since docker runs kernal only

#

its burning my cpu fan

rapid sparrow
# rocky pewter so is grabbing info about CI/CD jenkins better beforte going to docker

https://archive.dsicommunity.org/topic_week/perfect_cicd_pipeline/

Nous 2021-10-02 09:44
Jenkins was a first gen CI system. Everything is configured in the UI, which makes it hard to see/approve changes, things are live straight away so any larger changes can break things while you play around with them to see if they will work (as oppose to testing them in a branch first). They where designed to run agents directly on servers which bloated after a while with every tool ever single pipeline needed. Different versions of packages meant to needed different agents and managing it all is a huge pain. Often java focused and everything else is a second class citizen. This also goes for other older tools like teamcity, stash etc. Jenkins is also plugin hell, everything is a plugin and you need a whole bunch to make it a useable tool these days. Which increases the learning curve and makes the whole thing a mess.
I would not use a first gen CI system any more. They should just be left to die. They have tried to tack on more modern features but have all done it badly to keep support for the older ways of doing things.
Second gen CI systems are all config first, where the config is in the repo next to the code. As well as Docker (or at least ephemeral VMs for each job) so the agents are kept clean and essentially reset after each job and each job can install the tools it needs for that job. This includes things like travis, circle ci, github actions, gitlab ci, drone ci etc... so many choices there is no reason to use the legacy tools any more.
I think there is also a third gen popping up now - more abstract than just CI and starting to go down general workflow pipelines. Often kube focused and while you can build CI/CD systems out of them they are more general and no longer tied to only being triggered by Git event but rather any event. This includes things like Tekton, Argo Wrokflows etc type tools.

heavy knot
rapid sparrow
heavy knot
astral apex
rapid sparrow
unkempt zodiac
#

what's the best tool for development on a VPS? i personally want to avoid vscode because my server is too far away from me and loading everything, from extensions to highlights and stuff takes a lot of time.

heavy knot
crimson spruce
unkempt zodiac
#

I'll consider using Git then, but that's a lot of way around

crimson spruce
unkempt zodiac
#

thanks i'll try

rapid sparrow
heavy knot
thorny shell
#

I'm trying to think of an analogy. "Coding without git is like driving without seatbelts", except it's worse than that 🤔

#

"Running with scissors"

#

"Skating on tennis shoes"

heavy knot
#

Juggling unstable explosives. It’s not if, but when everything will blow up in your face.

thorny shell
#

I think I've heard the phrase "juggling chainsaws" before

unique sage
#

self hosted gits make it a nobrainer.. i say that after only setting up my own this past weekend.. finally..

thorny shell
#

I'm lazy, I use gitlab

heavy knot
sour elk
#

does anyone know how I can stop rye from putting sub depencies in the requirments.lock file becuase it puts platform specific stuff

astral apex
placid nacelle
solar bone
#

does any1 know what is the error, i am trying to open autopy after i have downloaded it and it gives me this error, the second picture is proof that its downloaded ???
plz help ive been stuck for nearly an hour

Screenshot_2024-12-07_202026.png Screenshot_2024-12-07_202100.png
astral apex
twilit forum
placid nacelle
twilit forum
delicate oracle
#

Hi,
could anyone please check my understanding on this:

Athena vs Redshift
Athena can't run SQL queries on the data that is present inside the warehouse(For ex: Redshift).
Athena works only on the data present inside S3 bucket.

Athena vs Glue
Catalog can be created by athena service / glue service.
Recommended way to generate catalog is by glue. The reason is simple. Athena can prepare catalog only based on the data from s3.
How about preparing the catalog from may sources like S3, SQL DB, No-SQL DB etc ? Only glue can do that.

If you want to correct/add new thing, pls.

Note: I am not literally comparing the above services. They are there for different purpose. My intention is to compare certain aspects which are command between services.

Also, if glue can do more than what athena can do, whats the purpose of Athena?

vocal pewter
rancid schoonerBOT
#

:incoming_envelope: :ok_hand: applied timeout to @warped charm until <t:1733816715:f> (10 minutes) (reason: duplicates spam - sent 4 duplicate messages).

The <@&831776746206265384> have been alerted for review.

viral marlin
#

!paste @bright pagoda

rancid schoonerBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

viral marlin
#

Use this ^^

bright pagoda
#

while writing my problem I thought of a solution, and when I had to rite it again (because I lost the long text!) i told myself: u can try again.,.. and it worked XD

wraith dock
#

What linters do y'all use with python, how good is it, and does it help when working with a team

thorny shell
#

ruff, it's excellent, I've not used it with a team

vocal pewter
gentle solstice
#

You should also use pyupgrade and eradicate

#

And bandit

vocal pewter
#

Bandit is a good option if you don't have other scanning tools in the CI such as snyk or sonarqube.

pyupgrade is an absolute beast for time saving. I don't run it 100% of the time but I do use it when needed.

Eradicate looks interesting. It wouldn't fly with me though as I frequently commit half-baked code and would be quite frowny-faced at chunks of my proto-types just being removed.

tiny jungle
#

i think you're the first other person i've encountered that uses sonarlint/qube/etc

vocal pewter
#

Personally, I don't but at work we have snyk and sonarqube. Love the former. Not a huge fan of the latter but it makes the enterprise happy so 🤷🏽‍♀️

tiny jungle
#

in my admittedly-limited experience, most of the things that sonar yells at me about that ruff doesn't have to do with complexity

vocal pewter
#

haha, yeah. I have a few modules that sonar will tell me "this has a code smell" and I'm just; yup! It sure do! yarrStinky

tiny jungle
#

i could also do without sonar's java dependency, which process hacker tells me is responsible for about 2/3 of VSC's current ram usage

vocal pewter
#

oh.. oh no.

#

I don't run that crap locally. That's gets thrown over the fence to the CI and I only care if it blocks my merge.

tiny jungle
#

yeah, the local version is the joy of not willing to shell out $160/yr for what's essentially ruff, but before ruff existed, and with a few differences

#

also AI CodeFix and AI Code Assurance is a very strong turn off, regardless of price

vocal pewter
#

Thankfully I don't have to put up with AI code fix tools putting PRs on my repos at work. Yet. I know it will come. The only silver-lining I have there is that I won't have to guard my words like I do when code reviewing some ai-assisted slop. yarrSip

#

To think, I live in the age that we are attempting to automate code churn.

wraith dock
#

Hearing bandit for the first time, I do keep dependabot enabled on GitHub, is it any different

#

Oh nvm, it's different

wraith dock
#

Would you recommend running bandit on PRs or something that everyone can run locally?

#

Or git pre commit hooks

vocal pewter
#

I would run it on PRs and as an optional linter exposed in a local noxfile, tox config, or otherwise. I wouldn't run it in pre-commit because it cannot fix itself. But that is a personal preference

astral apex
vocal pewter
astral apex
# vocal pewter Fast and easy vulnerability scans against the dependencies of the repo with plen...

The auto “fix” PRs are annoying because they don’t re-resolve the whole file, they just bump the one dependency they’re looking at, which often leaves the file in a non-resolvable state, so I have to go manually bump by myself anyways.

Also it’s been really annoying me lately with — 10 seconds after I merge a PR that bumps dependencies, I get an email saying they’ve “detected vulnerabilities”, and then a PR shows up that lists some dependencies in the description but the commit is completely empty — +0/-0

vocal pewter
#

I don't leverage autofix with dependency versions. As you point out, too many things can, and will, go wrong.

astral apex
vocal pewter
#

A weekly report of the dependency scan for anything that need addressing. Other than that, the CI will block if there are critical issues found during build.

#

Like what just showed up in my morning email.
Incident Management Org:
9 projects
99 dependencies
1 known vulnerabilities - medium criticality

Observability Org:
9 projects
76 dependencies
3 known vulnerabilities - all medium

Nothing that needs my attention this week or time on the board for tech debt stories. yarrSip See ya next week.

astral apex
#

Wait

#

What are those numbers?

#

Oh shit

#

Uh…

vocal pewter
#

I didn't know you developed with node modules. darkoLUL

astral apex
#

unfortunately

astral apex
vocal pewter
#

You would not be moving to production with that in my world.

#

And you'd have two weeks from the report to fix them.

astral apex
#

What if you’re still waiting on the upstream?

vocal pewter
#

Everything has its exceptions. You need some pretty good reasons to allow criticals through the pipeline. And there's paperwork to fill out because it will come up on an audit.

#

disclaiming again: this is just the policy of where I work. Mileage will vary.

astral apex
#

I’m trying to do a better job of having a policy for myself

#

Weekly-ish thing of flipping through the GitHub dashboard and the Snyk dashboard and cleaning up what I can

#

Problem is I have no budget for “unnecessary” stuff
So my Snyk, Sentry, Vercel, etc, is all on the free plans and I blow through my quotas every month.

vocal pewter
#

In honesty, that's a "them" problem and your policy should be the minimum needed to cover your ass.

astral apex
#

My boss hasn’t complained about GitHub though. I added Copilot Enterprise to test out the “knowledge bases” and then I realized how big the bill was getting, but he hasn’t mentioned it at all.
It’s so weird, because our GitHub spend is almost the same as our Azure spend, and he sends me messages every month asking me how to reduce the Azure spend, but never even mentions GitHub.

astral apex
#

I’ll have to check what they are when I get into the office

#

I have a bad habit of not checking Snyk because it just red x’es every single PR with “you have used your limit of private tests for this billing period”

#

It actually rolled over a couple days ago, but they immediately sent me three duplicate emails saying I was over the limit, and I’m still getting the red x’s, so,… shrugch

#

Snyk and I don’t have too great of a relationship

vocal pewter
#

I imagine trying to do anything for a company without the support of the company would be a pain in the ass. But we slide away from the topic slightly.

eager rose
vocal pewter
#

It sure do!

devs... tune your tools to reduce the signals. yarrRabid

astral apex
# vocal pewter You would not be moving to production with that in my world.

I have five Snyk projects in a row with one critical vulnerability. They're all the same CVE that's "introduced by your base image (python:3.13-slim@sha256:[...])"
I'm curious, how much paperwork is the exception process for you?
Also, how often do you bump images? I just have Dependabot bump the hashes monthly.

vocal pewter
astral apex
#

Though now Dependabot is annoying me, because it's trying to bump me from 3.12 to 3.13, but my dependencies don't support that
So now I'm not getting any of the new hashes for 3.12 unless I go look them up myself

vocal pewter
#

I've never enjoyed something that auto-updates versions. Even pre-commit.ci.

astral apex
#

How much time do you spend updating versions?

vocal pewter
#

🤷🏽‍♀️ Two months ago we knocked out 18 tasks for version bumps and a handful of house-keeping items. From start to production it took maybe 9 dev hours?

astral apex
#

Define the terms here

#

18 Jira tickets? GH issues? to "update deps"?
Or like specific callouts for specific dependencies from Snyk?

vocal pewter
#

"tasks" would be akin to a jira ticket. Maybe it was more than two months ago. I can't find them in our board anymore. I'm not a product owner. darkoLUL The tasks included:

  • Update something in requests, I forget what (snyk alert)
  • Ensure all repos were on the latest release of an internal library
  • Remove pinned dependencies from dev requirements
  • Ensure the CI was using the artifactory to pull images
  • Replace any *-node images still in use

I think there were a few other little acceptance criteria. It's been a moment.

astral apex
#

As someone who's been bit by both legacy shit being ignored and new shit burning down the farm, I like to know when things change so I can put them on a list to test at some point, to make sure I keep moving vaguely forward to stay within reasonable expectations of supported versions

#

Like my old Ruby app from 2019

#

Sure, nothing needed to change, so I didn't change anything
But then the server died
And I had to rebuild it on a new server
And it started screaming in pain because like 12 of the dependency versions were yanked
But bumping just one at a time caused invalid resolution
Even after removing version markers all together it still couldn't successfully resolve
It took me an entire day just to get a valid lockfile
Then the next day started with trying to add a container but it failed because the container version of Ruby didn't have the right version of a C library for a Ruby Gem with a "native extension"

#

If something sits still for too long I start getting itchy worrying that it's going to suddenly blow up

#

Of course there's also the other side, where bumping httpx took down my infra at 8am Monday morning, so I understanding not making changes just for changes sake

brazen forge
#

where CI tests

astral apex
#

Or if you mean in general, that's the entire point
Letting Dependabot create bump PRs lets the builds and the tests run in CI

brazen forge
astral apex
#

I don't do those

#

I used to, but it was too much effort, and no one used it

#

Now CI yeets merges straight into prod, and everyone is much happier

#

most of the time

brazen forge
astral apex
#

I got fistbumps for three days in a row for "decreasing the time required to deploy new changes"

thin hollow
#

@stuck crow hey kev i saw you in the android studio discord, where did you learn jetpack compose so proficiently any tips?

stuck crow
thin hollow
stuck crow
# thin hollow really is it easier?

It used to be easier to work with Google, but now it is totally opposite. These idiots at Google really become bureaucrats now.

Even they don't have the major market share yet. I totally understand why DOJ wants to break it up now.

thin hollow
#

what do you think the future for android dev is ?

stuck crow
# thin hollow what do you think the future for android dev is ?

If these idiots keep running it, Google will probably sell it too eventually like all the other projects it had before or gave up.

As long as the hardware and software are from different groups, it will never be as good as iOS where both hardware ane software are made in one place. Google's phones suck vs iPhones.

Sorry, I'm frustrated, and don't want to keep talking about it. Have a good one there. My next app will be iOS only.

rapid sparrow
#

Looks weird to me desiring closed ecosystem phone.

rapid sparrow
# thin hollow what do you think the future for android dev is ?

I would recommend to Google things like: mobile development Android vs iOS Reddit
https://www.reddit.com/r/androiddev/comments/18dlz6b/difference_in_app_quality_between_android_and_ios/
It has more opinions of devs that tried both.
So far looks like all recent comparisons in favor of android

Potentially person above just became infected with super successful Apple marketing cult. So let's take his opinion with some doubt and double check things preferably

Reddit
From the androiddev community on Reddit

Explore this post and more from the androiddev community

rapid sparrow
# thin hollow what do you think the future for android dev is ?

I would say at this point of time it is safe to say that Android and Apple are close competitors.

Android has five times more users
But depending on being in a first world country like France, users could come to you with half having Androids, and half with iOS. So obviously today is need to support both phones. Android has more users, but iphone has more rich probably market share to compensate its lower amount of users

stuck crow
#

and Google's phones are such disappointment so far.

#

these complacent idiots still think they are ruling the world

#

even though they are the second tier in both phone and app market

rapid sparrow
# stuck crow Great to know you are happy with your phone, be happy lol That little machine y...

That little machine you have in your hand will never be good enough and show its full potential, if software is not fully tailored for the hardware, like Apple does.
it sounds to me like exactly Apple cult marketing propaganda to be honest.
Exactly the same words other Apple fans repeated to me.

I will to add it as a web dev, there is huge move towards not having importance on which system your software is running.
Having lockdown to specific hardware sounds to me very weird, as it is exactly anti competitor strategy.
From the point of open standards, it looks obvious as Android is way better in being able to run by design on more devices
As users we should desire having more open standards, otherwise we will be charged 3 times more for weird closed ecosystem (like Apple does)

stuck crow
rapid sparrow
# stuck crow Well, if other Apple fans told you the same exact thing, it might contain some t...

The reality is software could unleash its full potential on specific hardware, that's just physics.
we could add this this ideology to unleash potential of specific hardware Even if truthful is not very justified in a modern world.
I mean.. common. We all have more hardware power than we need to. (at least for mobile devices)

Even if this ideology is true... it is not able to be fulfilled by Apple correctly, because they charge 2-3 times overpriced time for their hardware.
Just pick for same price more powerful hardware from other providers. (legit strategy when picking proper desktop PC)
You will get far more raw output, than whatever fine tuning apple can provide. Since they so much overprice their hardware, it just can't provide equal output as a normal other hardware.

So in both outcomes Apple picking is not justified

#

Well, if other Apple fans told you the same exact thing, it might contain some truth in it.
i think you are just Apple cultist at this point, you are already brainwashed by their succesful marketing. That's where Apple has great success. Super marketing for sure.

#

As user that wishes reasonable prices and accessability to a major ecosystem, instead of being locked to small proprietary ecosystem
I will not pick Apple for sure

stuck crow
rapid sparrow
#

Interesting question can araise though... what is worse/better pick, Windows or Apple 🤔 i guess even Windows better than Apple pick today

#

as you get access to plentiful ecosystem of software (which is still all on amd64 and not on arm) and has wsl2 available in addition

stuck crow
#

I own laptops from both Microsoft and Apple as well

#

from a dev perspective, the hardware and software from Apple

#

's ecosystem is better

#

though it is pricer too

#

but Windows is catching up, especially for daily usage

#

from a user that do daily tasks, yeah, you have more options if you use Windows, such as gaming, software options, etc

#

so I'm not actually influenced by Apple's marketing, I tested both

brazen viper
#

Could someone please help me with django/docker? I'm currently making a web app that integrates chart.js to make dashboards. Currently the only way I can get data is from excel worksheets, which I then prepare with python pandas and export ready files in json format. I want to put it all into Docker, however I'm going to need to manually (for now) prepare the files to update my dashboards. I'm wondering how that interaction would work with docker, when I would need to constantly upload new versions of my static files.

I'm just a bit lost on what I need to do first and in what order

analog kettle
analog kettle
#

@brazen viper I must know

brazen viper
# analog kettle <@309723733696970763> I must know

Hi! I'm not sure, I didn't know that's a significant decision, or that there are different kinds. Yesterday I tried making a dockerfile for my app, nginx.conf file and with gunicorn in my requirements file, but i couldn't manage to make it work

thorny shell
#

might's well use python:3.13-slim-bullseye

#

it's current and generic

brazen viper
#

So is image just basically the python version?

thorny shell
#

I don't understand the question

#

the "image" is basically an entire Linux system, with some stuff preinstalled. In this case, the preinstalled stuff is python 3.13

#

[and the Linux system happens to be Debian]

brazen viper
#

Got it, I'll try it today, thank you

gentle solstice
#

note: the python:3-slim images still have 2 versions of python installed.

#

python:3-alpine does not.

lethal basin
distant raven
#

Is there a way to “inject” a dependency into a build or to override a dependency?

I am trying to use the gymnasium library with the box2d feature as a dependency in a hatch project.
This feature depends on box2d-py via pyproject.toml. This package uses setuptools with setup.py to build, but relies on a tool called swig being installed.
My operating system has a swig package, but it is a newer version that no longer supports a legacy feature required by box2d-py.
Luckily, there is a PyPI binary package called swig that provides the specific version of swig.
How do I force box2d-py to be built using the PyPI swig package?
An obvious solution is to have local patched versions of box2d-py and gymnasium, but this is suboptimal.

#

At some point during this process, pip or hatch builds box2d-py and fails.
I would like for it to use the PyPI swig package at this stage.

rapid sparrow
#

for pretty much same purposes

distant raven
#

How do you do it?

rapid sparrow
#

and other ones

#

um, trying to find some example

rapid sparrow
# distant raven How do you do it?

as a basic approach, you just add whatever necessary extra attributes u need to objects

import collections
import sys

from django.conf import settings


def patch_elasticsearch_python_gte_v10():
    """To bypass exception with elasticsearch-dsl = "<6.0.0,>=5.0.0 (for python version gte v10)"""
    if sys.version_info.minor >= 10:
        collections.Mapping = collections.abc.Mapping
        collections.MutableMapping = collections.abc.MutableMapping
        collections.Sequence = collections.abc.Sequence


patch_elasticsearch_python_gte_v10()
#

u need to ensure that your app initialization leads to this patched code always invoked on app start

distant raven
#

Well but this is at build time

#

And it's about a binary being available

rapid sparrow
distant raven
rapid sparrow
# distant raven Is there a way to “inject” a dependency into a build or to override a dependency...

i hack dependencies as i wish with doing things like

import sys
from unittest.mock import MagicMock

import typelog

logger = typelog.get_logger(__name__)


if "uwsgi" in sys.argv:
    # this library is importable only when uwsgi is run :/
    import uwsgi  # type:ignore[import-not-found]
else:
    uwsgi = MagicMock()

Or i could be replacing specifics sublibs of any lib for another lib by doing things like

from raygen import fake_django_app

sys.modules["corsheaders"] = fake_django_app
sys.modules["django_extensions"] = fake_django_app
rapid sparrow
distant raven
#

I understand this principle. But how do you hijack which binaries are available during the build process?

rapid sparrow
distant raven
rapid sparrow
#

you just need to make your override overshadowing previous dependency

#

imagine your lib is depended on smth. that is installed somewhere there (or not installed)

#

just put into repository_root/smth arbitary, whatever u wish
and it will be overshadowing in terms of import and it will be imported instead

distant raven
distant raven
rapid sparrow
distant raven
#

This is not about Python code
This is about the build process literally running a binary

distant raven
#

The binary is not a C extension

rapid sparrow
#

i think u confused me. I thought we talk about something python 🤔

#

and now we are apperently about not python

distant raven
#

Yes, Python build systems

rapid sparrow
#

provide repository example of whatever you have there with what you do.
I lack understanding then to help more precisely.
or wait another person
or provide better explanations regarding what you are trying to do

rapid sparrow
distant raven
rapid sparrow
#

aha. at the level os.system("smth")/subprocess.run("smth")?

distant raven
# rapid sparrow provide repository example of whatever you have there with what you do. I lack u...

It's very simple.

I have the source code of library X.
I invoke pip or hatch to build it.
This reads my pyproject.toml.
It then tries to build library Y.
Library Y depends on library Z.
So it also builds library Z.
To build library Z, there needs to be a tool called W on the system (a compiler).
I don't have the correct version of W.
But there is a correct version on PyPI.
If Z had written into its pyproject.toml that W from PyPI was a build dependency everything would work.

I want to avoid having to locally patch Z and Y.

rapid sparrow
#

okay, got it

distant raven
#

I would be ok with only patching Z locally. But I don't want to have to rebuild everything locally.

rapid sparrow
#

as a simple idea => if u don't have correct W on the system => usually for web apps not a problem installing whatever necessary W u need in a docker image as one of its stages
docker allows easily having System level dependencies like W as part of its build process

distant raven
rapid sparrow
#

just today fought C buildable applications too 😏

#

Okay... you see i had docker image with python 3.9 and i was installing in it python 3.11, and i wished For sure that all apps in docker image used python 3.11 after its installation

# 
ENV PATH /usr/local/bin:$PATH

RUN ln -s /usr/local/bin/python${PYTHON_VERS} /usr/local/bin/python3 \
    && ln -s /usr/local/bin/python${PYTHON_VERS} /usr/local/bin/python \
    && ln -s /usr/local/bin/pip${PYTHON_VERS} /usr/local/bin/pip3 \
    && ln -s /usr/local/bin/pip${PYTHON_VERS} /usr/local/bin/pip
RUN python --version && python3 --version && pip --version  && pip3 --version
#

apperently you can do hack like this. if you specify in your PATH some FIRST folder that has already app that u wish, it will be used first as system level dependency

#

plus i created symlinks in addition to be used from any other regular ways like python3, python and etc instead as a default mean

#

So... my suggestion, if u have correct version of W installable by pip.

#

install it, and set python_location(or_venv)/bin folder as first folder in PATH, like export PATH=/usr/local/bin:$PATH

#

then it should be caught first by your build system

#

you can ensure that the name will be correctly matching by usage of symlinks in addition

#

the recipe is provided with assumption u have linux environment.

distant raven
#

ok

#

I was hoping there'd be a way to make the build system do it itself

rapid sparrow
# distant raven I was hoping there'd be a way to make the build system do it itself

System level C dependencies problem is resolvable only by

  • Installing them on your own to current system
  • Installing them in CI tool (like Github Actions) that builds for you is option too (but docker is easier because local)
    • basically doing things remotely. as advantage has a level of reproducability close to docker (but not, and a bit bothersome to debug github workflows)
    • as advantage the result will be automatable to run on some pushed git tag or created release (but u can just also input docker build result later into CI too)
  • Installing automatically with tools like Ansible (Ansible can do it only.. ergh... targeting your localhost or some remote host, some system will be polluted with attempt to install system level dependencies)
    • or any other tool for automation of configuration of a current OS
  • wrapping system level dependencies properly into Docker (the most reproducable and usable locally)
    • In the old times people used VM machines also and things like Vagrant, but it is a thing of the past and not practical
#

inputting system level dependency from pip itself installed in the process... into build of another lib? i can't imagine how
Simple process assumes any wrapper shell script at least or whatever u wish, that installs first lib first, configures env, and then installs the rest

distant raven
#

*the

thorny shell
dusky orchid
#

pls help

#

D:\ya tout\bot\DiscordBot\bot.py22 RuntimeWarning: coroutine 'BotBase.load_extension' was never awaited
bot.load_extension('cmd') # Charger le fichier cmd.py comme une extension
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Les commandes ont été chargées.
[2024-12-19 01:47:22] [ERROR ] discord.ext.commands.bot: Ignoring exception in command None
discord.ext.commands.errors.CommandNotFound: Command "ping" is not found

astral apex
#

Don't spam across multiple channels

celest locust
#

error: unable to unlink old 'app/Data/Group/GroupCreationData.php': Permission denied

#

on git pull

#

using ubuntu WSL to run the project

#

what is the problem?

brazen forge
gentle solstice
#

It could also be open in another application (windows)

wanton summit
#

anyone has thoughts on "DevOps" book by Jez Humble?

rapid sparrow
#

Noticing existence of this DevOps book from same author, I shall place to read it in nearby days then

grave jewel
#

Terraform is written in Go, and it is at some level compiled down to individual calls to Go functions in the underlying engine.

Can I just skip the Terraform itself and call these Go functions myself? Go seems like a fun language to learn.

vocal pewter
#

You can do everything terraform does with any language. They are just, in the end, API calls to AWS. The benefit of terraform is the hundreds of thousands of hours worth of trial and error poured into the existing tool.

You can also just learn Go. It is a fun language.

rapid sparrow
# grave jewel Terraform is written in Go, and it is at some level compiled down to individual ...

People tend to have fun in inventing their own Terraform Providers by writing this in go accordingly.
Otherwise terraform provider is just convinience to resolve it in a stateful way with terraform help.
Without it => you just use SDKs/APIs of different stuff.
But you could be using Pulumi or Terraform CDK that allows manipulating infra in Golang too (with already handled convinience for statefulness and simplifications)

Anyway... u could also just learn Golang as it is general purpose language indeed
Which is useful to make any sort of program ^_^

fickle mirage
#

I have a fastapi app that I'm using with a docker container. Is there a way to enable hot-reload inside the container when I make changes in my local dev environment? Or am I just going to have to rebuild the container image every time?

rapid sparrow
#

Question only of

#

does FastAPI has its own hot reloading by default? or we need to add external one

fickle mirage
#

yes. it has it's own hot-reload.

#

I'm just concerned that it's going to require re-building the container deployment each time?

#

which can take 10-15 seconds.

rapid sparrow
#

then u will have real time files of current folder

#

and change CMD(optionaly entrypoint) command to hot reloading one for fastapi

rapid sparrow
fickle mirage
#

so I have to setup a volume first?

rapid sparrow
#

yup

fickle mirage
#

and that would not impact production deployments?

rapid sparrow
rapid sparrow
fickle mirage
#

I've never used docker compose. Only dockerfiles.

rapid sparrow
#

then use solution i provided for docker

#

docker run -v $(pwd):/your_app_folder_in_container the_rest_of_your_stuff

#

docker-compose is convient way to write all settings as yaml and then launching as docker compose up ^_^

fickle mirage
#

it looks like asking chatgpt to augment my solution using a local volume has really confused it.

#

can you clarify what your_app_folder_in_container means?

#

Oh, you mean the working directory.

rapid sparrow
fickle mirage
#

Going to adapt it to use docker-compose.yml and then work on adding a volume

rapid sparrow
fickle mirage
#

Yeah. I think I'm confused as to how this works. I have a fastapi_app directory and the Dockerfile copies all the contents of that directory to an /app directory inside the container.

#

So I'm trying to get this docker volume to mount my fastapi_app directory on my machine?

rapid sparrow
#

It is done through volume mounting binding magic 🪄

fickle mirage
#

Ohhh. I see.

#

Okay. so if I run that volume paramter with docker run I do get a bunch of sqlite files written inside the fastapi_app folder. But running uvicorn with the --reload paramter still doesn't pick up the changes.

heady moat
#

do you use vscode, can give devcontainer a try

fickle mirage
#

I do use vscode. I think I prefer to use the volume approach though. Seems clean and not tied to a specific environment.

rapid sparrow
#

documentation how hot reload works

#

it watches only *.py files changes

#

unless u specify extra params

#

enter container from another terminal as

#

docker exec -it container_id_or_name and debug from there

#

try creating files there, see if they will appear

fickle mirage
#

Yeah. I'm modifying main.py. I'm also adding a --reload-dir to my startup script to see what happens.

#

Yeah. I think something weird is happening. I dropped into the container using docker exec -it container_name bash and the changes from my local environment didn't translate over.

#

Okay. I fixed the issue where they are not changing.

#

But reload still isn't working.

#

Feels like stabbing into the dark a bit 😛

rapid sparrow
# fickle mirage But reload still isn't working. 
# https://taskfile.dev makefile commands. Requires installed taskfile for functionality
# useful commands during development
# run `task --list-all` to list all available commands
version: "3"
  sphinx:live:
    # requires `pip install watchdog[watchmedo]`
    cmds:
      - > 
        watchmedo auto-restart 
        --patterns "*.py;*.rst;*.yml;*.md" 
        -i "./docs_build/*"
        --recursive
        -- sh -c "task sphinx:build; python -m http.server -d docs_build/html"

i tend to use watchmedo as universal hotreloader

#

workable with anything

#

it can accept any arbitary command that should be running 🙂

#

and can accept specifying which files to change watching

fickle mirage
#

Yeah. I think I need to step away and come back to this. I'm probably doing something wrong.

rapid sparrow
#

othewrise, check that files u change in container, reflect their changes in repo folder

fickle mirage
#

Will try watchmedo if I can't figure it out in another house.

rapid sparrow
#

cat/nano are useful programs to navigate inside container terminal
midnight commander is useful too for old school people 😏

fickle mirage
#

Yeah I cat ... | grep ... the main.py file to check if I make changes in my local environment are reflected in the container.

#

They are reflected, just uvicorn's hot reload isn't catching on to the changes.

thorny shell
#

ooh ooh "useless use of cat"

fickle mirage
#

Apparently a few folks had tried setting ENV PYTHONPATH=/app in the Dockerfile. But that doesn't seem to work either.

thorny shell
fickle mirage
#

I think I just need to take a break and come back to this. I feel like I've spent more time figuring out environment stuff than actually writing code at this point.

thorny shell
#

.oO("dive", he said)

fickle mirage
#

Got it working. Just needed to step away and give ChatGPT some additional instructions 😛

#

Funny how powering through a problem doesn't really help.

#

Especially when you're Copilot is hallucinating half the code/config

grave jewel
#

Suppose you have Terraform for DevOps and most of the time it works very well.

But once in a while, you need complex deployment logic. Terraform has if-statements, loops, etc but is not a general purpose programming language. For example, you won't be writing an ant-colony optimization in Terraform.

So if you really need some complex logic to deploy a subset of your cloud resources, how can you integrate Python into an otherwise Terraform pipeline?

rapid sparrow
#

This level if High level enough, abstract to make things easy

#

if it is for some reason not fitting the needs

#

there is always an option just using SDK libraries for python clients of specific providers, but it will incur plenty of problems on the way
Terraform CDK/Pulumi handle all the statefulness for you, making things easy. With using API and their clients directly you are on your own

grave jewel
#

I have heard of this. So it could be a way to move some of the more complex HCL script logic to Python, and the simpler stuff can stay in HCL. I will look into this.

rapid sparrow
# grave jewel I have heard of this. So it could be a way to move some of the more complex HCL ...

personally i never encounted yet a case which i can't solve with terraform.
Because i employ extra tools for its Extra Customization/Flexbility.
Lets imagine we need to execute complex code logic, in python or smth to calculate data for the rest of infra.
I just create AWS Lambda resource invokation, my custom code (with input from terraform) is executed there on apply (optionally on plan build if i wish), and i return result to the rest of my terraform resources

#

if even this is not enough => i can utilize Data External terraform resource for hackery of using any arbitary script at a current machine and returning json data from it for the rest of terraform code

#

i can use one custom script execution to input into another one

#

it is hard to imagine situation where terraform would be not enough

grave jewel
#

"extra tools for its Extra Customization/Flexbility." Do these "extra tools" involve Python or another general purpose lang?

rapid sparrow
grave jewel
rapid sparrow
rapid sparrow
#

breaking from it is very dangerous as u can receive very dubious infra then

#

usually infra devs can't be trusted writing a sane code with dynamic typed languages

#

terraform ensures a good and maintainable code is written

#

as long as they don't read smth awful as terragrunt layers at least and try to break terraform with ridiculous reasoning which they justify as DRY

#

Also terraform is stability, all providers support terraform in the most capacity. atlernative are usually worse, less stable and less documented

grave jewel
# rapid sparrow usually infra devs can't be trusted writing a sane code with dynamic typed langu...

One solution to dealing with Devs who don't really code but think they can? Maybe if we could embed Haskell in Terraform. Haskell is a good filter for "if you can use it, you are probably at least an OK software dev". And it is very strictly typed.

I would only use another language if I needed it. If Terraform's loops and etc became too difficult to easily use. But once in a while, that "escape hatch" into general purpose languages is realy needed.

rapid sparrow
somber marsh
rapid sparrow
#

not a fan of functional highly specialized languages madness
Smth smth not productive and not used in real world

somber marsh
#

Some people are massively masking problems so try and seem smarter than others

#

I use tkinter, customtk and python

#

And gsc, java, lua, bpy

rapid sparrow
# grave jewel One solution to dealing with Devs who don't really code but think they can? Mayb...

Terraform loops and etc are very consistent and easy to use once u got used to them
The trick to good terraform writing, is using smartly Terraform Modules for code reusage. Treating those modules like a regular programming langauge Class entity
that has its own Constructor what input variables it will receive, what it will hide in complexity to create and what u will output exactly for other modules.
Terraform modules are very powerful and static typed friendly tool 😋

#

Tbh the only real problem with terraform i have that at some point it does too many API calls 😅 and we need to split one terraform env to many terraform envs for sanity of faster plan/applies

#

If u have trouble something to do with loops/if stuff, usually u need to use locals construct to simplify transofmration of one data to another one. then it becomes very simple

locals {
  data1 = some loops
  data2 = another extra from data1
  data3 = you received some final transformation of data
}

i use it rarely tbh, not often such complex data is encountered to need it. Only if someone messed up somewhere data, writing it difficult for some reason, only then it is needed

grave jewel
rapid sparrow
#

yeah same best coding practices apply that could be learnt from Code Complete book, generic book about writing good and maintainable code

rapid sparrow
grave jewel
rapid sparrow
#

good, i am aws user as well

rapid sparrow
# grave jewel AWS

So... not having public code i guess for that.. except one incomplete code, i probably should not be showing it then

AWS Specific option friendly for other infra devs:
https://docs.aws.amazon.com/lambda/latest/dg/python-handler.html
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function
Anyway, the logic is next one.
You have Python scripts, you use Terraform archive_file data resource to make zip archive into it
you define iam roles for which resource your lambda can access
And you defined aws_lambda_function itself to create Python script executable in AWS infra

Then you input any data, IDs of some resources, filtering searchign, whatever u need for script into aws_lambda_invocation
The script is getting run on terraform apply and outputs data in its result (whatever json u will output from it)
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_invocation
That will be that basically. The result can be used to build plan/create resources for the rest of terraform code
I used that to implement some tricky stuff regarding creating Database users in the right way with all permissions for Datadog monitoring.
Discovering all logic databases on a plan build

====
There is cloud agnostic option to just run scripts from your local machine in addition
https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external
We don't abuse this thing, because it makes things more problematic for your infra dev colleagues, since they will need installed same dependencies for script as u are.
We usually use Lambda functions, because they can make sure terraform plan can be executed by everyone
dependencies are already hidden inside Lambda Function layers/zip achive (in simple measure), or in docker image if u used docker image runtime
At the same time his option is easier to use for you alone (if u are alone user of this code). Since u can skip all AWS lambda stuff

Define Lambda function handler in Python - AWS Lambda

The Lambda function handler is the method in your Python code that processes events. When your function is invoked, Lambda runs the handler method.

Terraform Registry
Terraform Registry
Terraform Registry
grave jewel
#

Interesting. I think for now Terraform + external may be better than Pulumni, as the engine behind Terraform is more powerful (I think)?

And it would only be used when the limitations of the HCL laguage itself bite down.

rapid sparrow
#

if u will create is as terraform and document well, it could be valid used Terraform module then publicly shared, and versioned controlled even just at the level of Github tags for simplicity (you publish new git tag, people can lock themselves to using exact version of a terrafom module written by you)

steel quail
#

do you know if there is a "sandboxed"/"local easy" way to test Kubernetes ?

we would like to give it a try but it would require some architecture changes, and I'd like to test locally first, might it be with docker containers or something like that

short peak
#

I personally hate how Docker Desktop has evolved, it used to be a pretty, light, and simple app. Now it feels overwhelmed by an enterprise mindset

steel quail
#

okay thank! I'll check how it feels like

rapid sparrow
#

to have working ingress u will need to install metallb though, it is covered in their docs (same problem for any self hosted kuber clusters)

#

it is basiclaly kubernetes cluster (with if ncessary multi nodes imitation) through docker containers

#

very local dev option

#

the simpliest version to install self hosted on servers is microk8s i can mention in addition

#

i recommend doing kubernetes adventures with debugging tools like k9s or its alternative

#

it makes things very simple

steel quail
#

ok ok ! thanks a lot

rapid sparrow
#

😅 i have written terraform code that raises Dev version of my monitoring infra through kind cluster
and then with same terraform installs all the stuff

rich wing
#

hello guys, i am very new to this, can someone explain to me how to download

  • Python 3.6 or higher
    *PyQt5
    *PyCryptodome
    *PyInstaller
gentle solstice
#

Why are you using python 3.6?

visual oxide
digital wyvern
sudden lion
#

Is there any package for generating info for citation from PDFs (eg books and especially for papers gotten outside arxiv)

delicate oracle
#

Hi,
Pls help.

Below is docker-compose.yml file

version: '3'
services:
  spark:
    image: jupyter/pyspark-notebook
    user : root
    ports:
      - "8888:8888"  # Jupyter Notebook
      - "4040:4040"  # Spark UI
    volumes:
      - ./in_side_container:/home/jovyan
    environment:
      - JUPYTER_ENABLE_LAB = yes
      - CHOWN_HOME = yes
      - CHOWN_HOME_OPTS = '-R' 
    command: start.sh jupyter lab --NotebookApp.token=''

  spark-master:
    image: bitnami/spark:latest
    environment:
      - SPARK_MODE=master
    ports:
      - "8080:8080"
      - "7077:7077"

When i run docker-compose up, getting the error as PermissionError: [Errno 13] Permission denied: '/home/jovyan/.local'

upper root
rapid sparrow
upper root
#

but why yml

rapid sparrow
upper root
rapid sparrow
#

Because it is programmatically easy to read method for settings, that has human readability

#

Yaml is language agnostic, usable for people regardless of their primary general purpose languages

upper root
#

with making it

astral apex
rapid sparrow
upper root
#

i'm new to python

rapid sparrow
# upper root yeah actualy making the thing

This server is about helping you to make it.
For other people helping making the tool itself, it should be attractively presented first to intrigue for collaboration, and if it will fit interests of a person, then the person could help. I don't see it here any presentation or reason to help making tool itself.

upper root
rapid sparrow
upper root
upper root
#

i made a example of what the config file is like

[about app]

version = 0.0.1

keywords = true ; false ; on ; off

fork = false

[settings]

fullscreen = false

gui scale = 3

dark mode = true

#

this is just a example

rapid sparrow
upper root
#

rn

rapid sparrow
upper root
#

and it's python

rapid sparrow
upper root
#

i'm making a tool with the laguage that was made for this kind of stuff

#

python

short peak
upper root
short peak
#

🤔 ok

upper root
#

the thing mine is different you go and put the source code inside of your project to implement it instead of installing a library

rapid sparrow
upper root
short peak
#

the link I sent you is a python built-in library, no need to install anything. You might need a very good reason to create an alternative since python one is extremely easy to use

upper root
rapid sparrow
#

I created my own ini library... And my own config reading lib too... But I had good reasons 😅 probably

short peak
upper root
#

here is the code

#

the result will be shown in the console

rapid sparrow
#

In first case I had to read unique ini format of game files and wished doing that in static typed way, and having ability to write back without writing write code

In second case I just wished reading configs in Json with comments and optionally reading from env vars, while being typing friendly again

upper root
#

they are titles and i will add real comments in the future

short peak
upper root
short peak
#

I always try to use context managers when available

upper root
#

i used copilot to make this

#

i don't know anything about python

short peak
upper root
upper root
#

you can change values and stuff

#

and it detects the change

#

i recommend using vs code because that is what i used

#

to load and test it

short peak
#

basically allows you to use the feature with a with statement that creates and "destroys" the resources you will use. Again, check the link I sent you, python tomlib built-in option provides it

#

btw, I'm testing Zed on Debian, so far I've enjoyed it 🙂

upper root
#

but i can run linux apps

short peak
#

they don't have a Windows version now, you need to compile it yourself

upper root
rapid sparrow
upper root
#

so anyone pls test mine

heavy knot
#

guys

#

i need alot of help please

#

@upper root @rapid sparrow @short peak

#

can anyone here help

#

it's a python script

#

my CapSolver isnt working, despite i put my auth key

thorny shell
#

please don't ping people

heavy knot
#

Okay

heavy knot
delicate oracle
placid nacelle
median quail
#

how to host flask app?

short peak
potent cedar
#

not sure if this is quite the right channel, but I have a native python module (in C) on github and I have tests that run via github actions. I'd like to have my tests run across multiple architectures - i386/i686 in particular. does anyone have a simple example of this?

rapid sparrow
#

I suspect it is just manual control to specify exact python arch

#

But it should fail on wrong arch

potent cedar
rapid sparrow