tools-and-devops

[distant epoch]

I remember checking Gtihub issues like a year or two ago and it was quite lacking. It was troublesome to setup simple things like labels and main cause of this was lack of bulk edit option

[thorny shell]

• gitlab isn't github
• I wouldn't be surprised if github was better now than when you last checked ... this stuff seems to be moving fast

[distant epoch]

Who wrote Gitlab is Github?

Well, I don't exactly have time to check all of options myself, this is why I'm asking for suggestions and your experience with those tools

Are they 1:1? If not, what are differences?

[surreal ridge]

They're not 1:1, GitLab has many more enterprise features than github

[surreal ridge]

No, Github is Microsoft now and GitLab is still an independant company

[distant epoch]

You mentioned Jira vs Gitlab, so I'm asking about differences between those two

[surreal ridge]

Oh, my bad misread. Honestly they are equivalent for small teams. Is there any specific project management tool you're looking for?

[distant epoch]

I know, I asked if anyone said they are the same, because I didn't catch that part

[distant epoch]

Is it possible to create sprints in Gitlab ?

[surreal ridge]

Yes, GitLab terminology is a milestone

[distant epoch]

Basic features I'm looking for are: stories, epics, labels, tasks, subtasks, dashboards and sprints

[surreal ridge]

Everything is in gitlab as far as I know

Stories would be epics or issues, labels are there (you can create any), tasks/subtasks are there, they are a level below issues, dashboards as well and sprints too!

It honestly makes everything easier from a team member point of view. You have everything in one tool instead of having jira for tickets, confluence for docs, git, CICD in Jenkins or w/e

@distant epoch https://about.gitlab.com/blog/2018/03/05/gitlab-for-agile-software-development/

[distant epoch]

Thanks, will check it

[distant epoch]

Sounds good, I will try it then

[whole night]

I work as a SysAdmin right now but I enjoy development within infrastructure the most, things like automation, developing internal tooling, scripting, etc. I am looking to get into a SWE or SRE role that's about that, but with my current experience it is being quite tough, even though I have experience as a freelance SWE (mostly Java, though). My questions are:

1. Is it easy to go from SRE to this kind of SWE down the line? Or should I go into whatever junior python position I can find and go from there?
2. Is there any kind of project I could add to my CV that would give me experience in this kind of role and will look attractive to recruiters?

[thorny shell]

this is just a guess, but: I imagine your best bet is to find some development task you can do at your current job

this seems like a discussion for #career-advice

[old adder]

Could someone help me with a program or open source tool for next application? I'm not quite an expert on IT and programming, so any help is usefull.
This is the main idea: 1 central computer must control the hardware of different client computers. These client computers perform actions at the request of the central computer. They collect data and send it to the central computer, which stores and analyzes the data.
It is important that the operating speed of the client computers is adjustable.

[restive jasper]

@whole night it would depend on what you actually was doing at SRE position. Like small helper scripts vs something bigger. Anyway you probably would have to learn and have to make some projects. If not for a portfolio, then for getting experience.

2. At least the portfolio projects. It's not easy to make them attractive to recruiters because usually they are education projects or pet projects. Ofc it's better than nothing, but recruiters want X years of experience in a profession

Also personally me wouldn't switch from SRE to software engineering, but that is personal preferences I guess

[whole night]

so would looking for an SRE job where there's a lot of coding/automation be the best approach for me? Because I am interested in programming/scripting/automating the most, but obviously not a fan of SWE when that's focused on an end product. I prefer smaller projects to create for example command-line tooling or scripts tbh

[whole night]

Why is that? Could you explain this more?

[restive jasper]

That is personal preference. I would rather stick with Linux ops then developing in JS or something

Best approach... Develop some pet projects in technologies you dream of working with. Try some junior level interviews to understand what's your current level and what you would need to improve to switch to SWE

[whole night]

Not a fan of JS overall either, but there are a few Python jobs that do sound interesting to me. Mostly within infrastructure tbh

[iron basalt]

I am looking for ways to keep lint & general tooling configuration files synchronised across multiple repositories. Let's say files like .flake8, .pylintrc, to some extent also pyproject.toml as more and more tools allow configuring via it. If I'm maintaining multiple Python repositories which all follow the same code style, I want these files (or portions thereof) to be in sync.
The content of the files evolves as rules are adjusted, added and deleted. Developers will usually adjust the config locally, in the repo where they are working at the moment, but it seems insane to ask them to also open merge requests across all other repos to make the same adjustments. Ideally there would be a single definition that other projects can inherit (and optionally override!), but it gets complex when files like pyproject.toml also contain application-specific config (such as dependencies, versions, authors, ...). Any ideas?

[thorny shell]

wow, good question

ok here's a primitive idea: maintain a separate repo that is essentially nothing but config files (along with, perhaps, a dirt-simple stub program like "hello_world.py"). Have developers base their new projects off of that, and also have them periodically see what's new in that repo and manually apply those changes to their existing repos

@iron basalt ⏫

[iron basalt]

Thanks for the answer! Yeah, we have templates for new libraries and new apps, which contain pre-defined config for our tools. However, as the inheriting repos develop, their configs diverge. It's already difficult to get developers to go update the template when their repo changes, and so it seems basically impossible that they would go propagate that change manually to other repos as well - especially if it means making changes to code that now doesn't pass lint. I considered using a git submodule for the shared config, then at least devs are forced to commit their changes in one place and the process to propagate the changes is straight-forward and doesnt require copy-pasting (just bump the submodule ver & make any necessary code changes). But it falls apart when you have files like pyproject.toml which contain config for various tools and the app as well.

[jagged kiln]

Something I did at work similar to this (albeit with gitlab CI, I'm not sure how painful it would be with GH actions or other CI)
was basically:

• Centralised source of truth repo. All the OG/base/to-be-inherited configs went here.
• CI job (defined in the central source repo) that pulls all of these changes. It can be a bit awkward but we handled repo-specific changes by essentially just letting each repository set a variable to disable the job - a few things you can do here are either let them list which files are to-be-ignored (or to-be-maintained)
  another option would be to disable automatic file pushing and have a bot notify changes by creating an issue instead.

for things like pyproject.toml, I'm not really sure since the thing we did was for a completely different purpose 😅

[brazen forge]

not suggesting this as a solution, just thinking out loud

1. have a source of truth template repo
2. use something like chezmoi1 to apply the changes to the project repos

it can do partial updates to files, so it can update pyproject.toml, for example

(it's meant to manage dotfiles, so this is abusing it)

[thorny shell]

haven't clicked yet but I love the name "chezmoi"

@iron basalt we're approaching the well-known place called "that's not a technical problem; that's a social problem". What is bad about developers not integrating new config stuff into their existing repos?

[jagged kiln]

in my particular case, we just wanted to have consistency across the board as a general organisational policy. made it generally easier for everyone working on across our codebases

[thorny shell]

well sure

I'd hope the devs all agree that consistency is good, and that it's worth a bit of effort

if not, then .... 🤷

[restive jasper]

@iron basalt I never worked with git submodules but maybe this is the answer. On the other hand maybe it's too complicated, or other solutions exist

...like enforcing it not on SCM level but at the ci pipeline level...

[rapid sparrow]

i often hear recommendation: best not ever to use them ^_^ (hehe, unless u have something like C++ code without package manager and it is your last resort ever)

[thorny shell]

"enforcing" may not be worth the trouble.

[iron basalt]

Thanks for all the answers! Lots of interesting thoughts here. I know that people tend to have bad experience with git submodules, but I personally think they are a good solution to some problems. Not all problems though. 😄
@brazen forge That tool seems interesting, I'll take a look at it. Being able to automatically do a partial update of a file seems like something that could help a lot.
@thorny shell I think it's both a technical and a social problem. We want config to be consistent across repos, but don't maintain the consistency manually because it's obnoxious. If we find a process that works & tooling that automates most of it, it won't be a hassle, and the benefits of consistent config will outweigh the burden of maintenance. Such is my hypothesis, at least. 😄

[thorny shell]

I'm moderately skeptical that the benefits are that large

what's the worst-case scenario?

[It's funny; I myself much prefer consistency; I'm a little suprised to find myself arguing against it here]

[restive jasper]

@iron basalt after reading the question one more time, I think that such things are likely to be project specific, repository specific. I mean, linter settings could be project specific, and pyprojcect.toml is also project specific. Not sure if they could be common among multiple projects. For example, any team would/could eventually want to customize linter rules.

Btw why not just use black linter for Python. It has no settings (almost), so the rules will be the same for every project if you just use black.

Regarding ci enforce example I was writing before, in my company they are using gitlab pipeline includes to enforce things, but those things are security checks (such as absence of secrets inside the repo).

[edgy bridge]

Faced a similar scenario at a previous employer. We managed this from the CI/CD pipeline where the linting was just a stage in the pipeline

[edgy bridge]

We also had an agreed basic coding style/standards where all the developers would have to agree on the rules before they became enforceable.

[proud trout]

Hey guys, quick question, for deploying using docker, should I clone the repo on the server and then run it using docker ? or build docker image locally, push to repository (docker hub) and then pull the image in the server and run it ?

[thorny shell]

I'd build the docker image locally, then run it and do some tests, and only push to docker hub once you're confident it works well

[tropic totem]

Wtf

[brazen forge]

hi, is there any problem?

[brazen forge]

https://orbstack.dev
looks interesting 👀

I've been using colima right now, but might try this out

[brazen forge]

some of these points look… designed to be biased

[small grove]

Hm not sure if this is the right place for this question, but here goes.

Does anyone know of anyway I can change remotely switch from MacOs to Linux on a headless intel Mac?

[brazen forge]

as in, install Linux on a remote Mac?

[small grove]

So installing it on the remote mac isn't an issues, I can connect a screen. Its Intel based so compatibility is fine too. The issue would be booting from one to the other remotely

[narrow sonnet]

Would you use an automatic React component generator from backend APIs?

[gentle solstice]

Any ideas what would cause this sometimes?

unable to recognize "/tmp/fn-scheduler-kubernetes-ensure-namespace.HN8t": no matches for kind "Namespace" in version "v1"

this is dokku via kubernetes

running the build again usually fixes it

[left cape]

Hi. I cant natively run python code of app that I develop (M1 CPU)- instead I'm using docker container. Is there any way to vscode resolve libraries that my application is using? Is there with problem like mine, and would share they approach?

okay- I know the solution- devcontainer 🙂

[rigid nexus]

I have a Django system and I would like to architect something in AWS, the notification part of it.
But I want to do something personalized, divided into notifications by service types: push and email.
I have 8 types of notifications: creating a new post, creating a new comment on the post, etc.
The user can choose to receive or not receive each type of message by type of service. Ex: receives push notification of comment creation, but does not receive it by email. How would this look in an architecture within AWS?
Remembering that all notification control is done within Django, through models.
Sending email messages is done using SES and push by Firebase. I know it's possible to do this via SNS, but I don't know how to do it.

The simulation would be:
User X:
Should receive push and email notifications:
post creation
comment creation (but only comments where he is marked as the post owner)

User Y:
Should receive push notifications:
post creation
Should receive email notifications:
comment creation (only comments where he is marked as post owner)

[median remnant]

Can someone point me to a good resource on handling third party version dependencies? I don't mean how to pin versions etc. - I know how to do that in Poetry. I mean more like keeping track of updates to third party versions, deciding when to update etc.

[thorny shell]

one idea: keep your project on github. I'm uncler of the details, but I think they periodically scan your project and find the vulnerabilities for you, and make a pull request with the update. It's really quite slick.

[vocal pewter]

Those are tools that do the scanning and some even submit PRs for updates. Dependabot is one that GitHub runs.

[thorny shell]

ya that's the name

[median remnant]

Thanks! I do keep it on github and I run some tools in Nox like bugbear, I was thinking more about the practices around upgrading versions in general. Like, just create a new branch, pip upgrade, check that nothing breaks and push to main, or are there some more sensible approaches?

[vocal pewter]

Unlimately when you update is up to you. It's certainly work tracking security updates for libraries.

[thorny shell]

well, sure, create a new branch. Upgrading your dependencies is like anything else: it's a code change that needs to be tested and reviewed

if you have good unit tests, then dependabot can do everything automatically.

unfortunately I don't know what control you have over it; in my case, it just started happening without my asking for it.

[median remnant]

Thanks!

[median remnant]

Yes, that sounds like Microsoft

[south forum]

hey,
How would you go about adding an SSL certificate to my Dockerized FastApi app which is hosted on DigitalOcean.
I have tried it with Certbot, but I ran into some problems.

Thanks for responding in advance

[thorny shell]

#1124391627335680051 message

one problem: certificates from LetsEncrypt expire every (I think) 90 days, which means after you've jumped through the hoops to get the certificate, and bake it into your dockerfile, you'll need to do those steps again every three months

maybe you can do your ssl stuff outside of the dockerfile?

[south forum]

The thing is, I never set up any nginx things.
I just use uvicorn to run my app ?

[median remnant]

@south forum You should probably look into fixing the certbot issues since it is the simplest solution. IIRC it relies on port 80 being open for the challenge-response, so might be a firewall issue. If it's available to you, you could also look at setting up one container each of your app, nginx reverse proxy, and certbot.

[south forum]

what I don't understand is this, my app is running on port 80. So I shutdown docker so port 80 is avaiable. But then certbot says it can't reach anything

this is my dockerfile

FROM python:3.9

ENV MONGO_DETAILS=""
ENV MY_SECRET_PASSWORD=""

WORKDIR /app

COPY ./requirements.txt /app/requirements.txt

RUN pip install --no-cache-dir --upgrade -r requirements.txt

COPY ./ /app

EXPOSE 80
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

[median remnant]

@south forum I don't know how you have it set up on Digital Ocean and don't know anything about FastAPI. Haven't used uvicorn (only gunicorn). Can uvicorn handle SSL? Otherwise you would have to set up something like nginx reverse proxy, have it route port 80 traffic to certbot and https to your app (typically traffic from clients to reverse proxy use https and traffic between proxy and app use http)

Start with setting up nginx and have it route traffic to your app. Then add certbot and ssl

[south forum]

somewhat like this?
https://docker-fastapi-projects.readthedocs.io/en/latest/nginx.html

[median remnant]

https://cloud.google.com/community/tutorials/nginx-reverse-proxy-docker

[south forum]

thank you

[bitter tree]

Long story short, I'm trying to merge one Git branch to another. I am wondering how would I achieve this without losing any of my files? So I am trying to merge the README branch into the Master branch as it states in this prompt right here. But I am unsure how I would do this because the last time I tried it, I somehow lost my files and had to get them back.

[rugged hare]

git checkout master; git merge README should do it, without losing files (assuming you've committed them on the README branch)

[bitter tree]

Alright thanks

[restive jasper]

try this one https://pypi.org/project/check-requirements-txt/

or just snyk

[fierce pagoda]

great idea

tkx

[finite fulcrum]

I'm a bit confused by how docker's creating the directory here, I have a volume in docker to persist configuration data, say ./host_path:/some/remote/path:rw, but docker is creating the host path with some weird perms on windows and the container can't write to its directory because of it. If I delete the host directory and create it myself everything works, I'm also fairly sure it worked before as I was debugging an another thing around the volume couple weeks back.

The only difference I could notice was that when listing perms through wsl on the windows fs the docker created dir is

drwxr-xr-x 1 root root 4096 Jul  4 03:12 h2

and the manually created dir is

drwxrwxrwx 1 numerlor numerlor 4096 Jul  4 03:13 h2

I'd assume docker is somehow created the windows dir as some admin, but I can't see any difference in the windows perms (though admittedly I also generally have no idea what's going on around windows perms).

Can I do something in docker so it creates it with the right user or perms? Or should I just track the directory in git with the right perms instead of letting docker create it

[restive jasper]

@finite fulcrum check umask output

[finite fulcrum]

That shouldn't matter as I'd expect it to be either docker managed (if it creates the directory through its wsl distro where I don't do anything), or just non existent if the windows part of docker creates it

[idle cedar]

Configu - Open-source project that puts an end to your configuration Chaos
Check it out: https://github.com/configu/configu

[vague silo]

welcome to the server. what problem does this solve?

[idle cedar]

Modern application development often involves thousands of configurations, which vary across environments and are stored in multiple systems. Managing this involves error-prone, tedious work for developers and hinders DevOps automation.

Configu is a simple yet powerful tool designed to manage application configurations (env vars, secrets, feature flags, and more) at scale across environments – from code to runtime – by providing configuration orchestration along with a Configuration-as-Code (CaC) approach.

You can easily sync and connect it to any configuration store you use (files in a git repo, secret managers, feature flags, databases, etc.) and Configu Orchestrator provides a unified interface that rules them all and allows developers to define and deploy configurations consistently. This ensures that the desired settings are applied accurately throughout the entire software development lifecycle (SDLC). By automating the configuration orchestration process, it reduces tedious manual work and errors and promotes efficient deployments.

It can be used with any programming language/framework and is a super easy setup.

[restive jasper]

Differences with Ansible, Puppet, SaltStack?

like I never heared about it

[rapid sparrow]

Ansible is procedural and idempotent, works one time when u run your playbook to apply. it is pretty much servrless and technically agentless too (having very high community support. Nice tool to augment CI workflows)

Salt stack i heard is having servers and agents, and gets applied changes every 5 minutes

how works Puppet i don't know

P.S. oh it was not a question. never mind 🙂

[idle cedar]

Configu specializes in application configuration management, providing a flexible and language-agnostic solution to handle dynamic configurations at runtime. Ansible, Puppet, and SaltStack, on the other hand, focus on broader infrastructure provisioning and configuration management tasks. While these tools excel at automating server provisioning, configuration drift remediation, and enforcing desired states across infrastructure, they can be complemented by Configu to efficiently fetch and manage application-specific configurations. By combining these tools, organizations can achieve comprehensive infrastructure provisioning, configuration management, and seamless integration of application configurations. Configu's centralized configuration management and dynamic retrieval make it a powerful addition to the infrastructure and deployment workflow, enhancing flexibility and adaptability in managing application-specific configurations.

[rapid sparrow]

sounds like smth like AWS Secret Manager / Vault with client.

Not sure if i would have wished using tool that can affect my immutable container environment runs though. Sounds kind of slightly dangerous and messy a bit 🤔

it is enough for me to inject environment variables mostly at terraform level (because this stuff is easy to review in terms of changes / nicely immutable)

[idle cedar]

Configu takes a configuration-as-code approach that makes configurations first-class citizens, and works across all environments and systems, such as secret managers, feature flags and config databases.

You can easily sync and connect it to any configuration store you use (files in a git repo, secret managers, feature flags, databases, etc.) and Configu Orchestrator provides a unified interface that rules them all

[rapid sparrow]

!rule 6 , delete your stuff it is not appropriate for here

[rancid schooner]

Rules

6. Do not post unapproved advertising.

[rapid sparrow]

<@&831776746206265384> #tools-and-devops message this is a paid stuff with pretty much no access to free version (no self hosted choice). i suggest it is unapproved advertising.

[idle cedar]

I only presented the open source, we have 2 different products

[rapid sparrow]

is your open source version unlimited in users/usage? page #tools-and-devops message kind of tells otherwise.
fully tied to your infrastructure looking like solution with trial / very restricted version only

[heady ruin]

@rapid sparrow I guess application configurations is a wide definition it can be resource handles, credentials to external services, per-deploy values, feature flags etc. Part of the you want to inject at build or deploy time and it can truly be achieved with terraform and some of the you may need at runtime. Configu supplies all of that as a unified interface that can connect to wherever you store your configuration values, fetch them for you on-demand, apply schema validation on their types or other constraints, and then produce the configuration artifacts that best suite for your application whether it is a values.yaml for a terafform deployment project on the ci with the Configu CLI or a feature flag value at runtime with Configu SDK

[frozen zephyr]

Open source or not, this does seem like advertising.

[idle cedar]

This pricing and the website refer to the enterprise product we have, the open source is only for the benefit of the community

[rapid sparrow]

it is not clear that there are no restrictions at open source version.
As far as i browse your website, i observe only trial version / very restrictied version in your second choice

AWS CDK is kind of open source too. AWS is not free in its nature at all nevertheless

there is differentce between Open source and FOSS essentially

[frozen zephyr]

Again, whether the thing being advertised is FOSS or non-FOSS open source or a paid product isn't relevant. Advertising is against our rules regardless.

[heady ruin]

Configu OSS stands alone and is not related to any commercial product. This is similar to terraform and terraform cloud.

[idle cedar]

the website is for the enterprise product we have, you can find all the info about the OSS on the repo https://github.com/configu/configu

We have only shared the OSS we released for the benefit of the community, sorry if it broke any rule.
We have released a Python SDK and we want to help this amazing community

[heady ruin]

It is a Apache-2 licenced project and it is exists to make a good. You can use the Configu Platform as a Configuration Store for your values but you can also use wherever other stores that we support (and that can be and already been extended by the project's community) in the same manner and acheive the same goals.

[frozen zephyr]

It did break rule 6. Please delete these links.

[rapid sparrow]

well, it is nice if it is indeed free and there are no hidden underwater stones
needs a bit more evaluation and trying to check how much it will be useful though xD

[frozen zephyr]

It does not matter if it's FOSS. We don't allow people to accept their products here. There's a bit of leeway if your project is directly relevant to solving a problem someone is asking for help with, but that doesn't seem to be the case here. It seems like you joined this server just to advertise your project.

[heady ruin]

Looking forward to hear what you think 🙂 we are here to learn from this awesome community and improve the project and the Python SDK we just released, this is our only purpose.

[rapid sparrow]

https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
Gitlab documentation is kind of easy to navigate in terms of what is free or paid feature (Free tier, Premium tier, all tiers markings)

https://configu.com/docs/get-started/
your documentation kind of looks like it gets hooked to your infrastructure too from a start

If there was clear instruction how to use your product in a fully self hosted and free way, it would have been helpful (clear words like Self hosted setup should be present)
Or clear clarification that state is kept locally or smth

otherwise i see your product (as first impression), as an offer to donate all my environment secrets into your databases (for future leaking)
and vendor lock self in for future payment lockdown or your infrastructure downtimes

[idle cedar]

Thanks for the feedback @rapid sparrow, we will fix it and make it more clear!
This getting started guide is for the Configu store product.

Configu Orchestrator is the OSS and it includes all the management interfaces (CLI & SDKs), you can use it to manage any configuration storage you use.

For example, here is a recipe for how to use the OSS and manage HC Vault: https://configu.com/docs/hc-vault-w-configu/

[dark crow]

I'm looking for some information on how to build my own API. The API is pretty simple as adding removing and editing a JSON file.

[thorny shell]

pro tip: don't "edit" JSON files. Instead, load it; modify the resulting data; dump that data back out.

[dark crow]

Is there some 3rd party software that is useful?

[thorny shell]

maybe? Are you asking "how do I load and dump JSON"?

if so: https://docs.python.org/3/library/json.html

built in to python

[dark crow]

No I don't want to reinvent the wheel. Im looking for a solution for having my own API hosted on a VPS not sure if I should build it completely in python or use some 3rd party software

[thorny shell]

ok, "having my own API" and "editing JSON" are almost unrelated

start here https://flask.palletsprojects.com/en/2.3.x/tutorial/

[indigo zenith]

https://realpython.com/fastapi-python-web-apis/

[alpine horizon]

why dosent poetry update actually update the version of the dependencies in pyproject.toml?

[brazen forge]

because it's not designed to do so
it'll update to the latest version that matches the constraint in pyproject.toml
if you want to update to the absolutely latest version, do poetry add package@latest

[alpine horizon]

damn thats a shame, I wanted to update everything so overtime I dont endup using very old versions of all these pacakges...

and is "latest" a valid literal in the poetry add command, or you need to find the latest version manually?

[elder solstice]

if you don't specify an upper bound version specifier in pyproject.toml, then the latest version will be compatible with the requirements (unless other dependencies have specified an upper bound for that package)

e.g. you can have

my_package = ">=1.3"

to specify that my_package can be anything later than version 1.3

so whenever that package is updated, poetry update will be able to upgrade the version

[brazen forge]

latest corresponds to ^{whatever is the current latest version}
https://python-poetry.org/docs/dependency-specification/#using-the--operator

[alpine horizon]

then it is a recognized keyword

thanks

[loud bear]

Is poetry's resolver better than pip

[tawny temple]

Does pip implicitly upgrade installed transitive dependencies? Like if A depends on B, I have B installed, and I do pip install --upgrade A, does B get upgraded, or will the backtracking resolver find the latest version of A compatible with the installed version of B?

[thorny shell]

a) try it and see;
b) my guess is: if the new A doesn't require a newer B, it'll leave B alone

[restive jasper]

@loud bear I think poetry uses pip internally

[tardy maple]

Does anyone use sublime? Im trying to figure out what package to install to work with my python files that gives me this auto sugguestion dropdown like this one found in this video:

[undone moat]

I've had auto-suggest pop up for some stuff, but only for builtins really.
Might try: https://lsp.sublimetext.io/language_servers/#python

[tardy maple]

ty

[thin jasper]

yes.

[thin jasper]

yep, but does its own dependency resolution

[knotty kettle]

Are there any tools that compare numbers from profiling runs? E.g. tottime for a function foo() increased by X seconds from the previous profiling run?

[terse prism]

Hi! I am a Senior DevOps engineer. with python script for automated deployment

Highly skilled Senior DevOps Engineer with 5+ years of experience designing and implementing disaster recovery plans, developing CI/CD pipelines

[old wasp]

!mute 1120926030035697734 2d you've been warned about advertising before, consider this your last warning.

[rancid schooner]

:incoming_envelope: :ok_hand: applied timeout to @terse prism until <t:1688962268:f> (2 days).

[mental epoch]

hey y'all. lifetime linux desktop user here ... I've written a Normal ™️ PEP517 python package that is meant to be both a library and CLI tool. I want to package it and distribute it to windows users, though, so I was looking into how to freeze it with a bundled python interpreter so it can be used by normal desktop users

I was looking into PyInstaller (that seems to be the most talked about one?) but it seems to expect a "script" and doesn't seem to really understand a pyproject.toml sort of thing. It looks like it has a lot of config I could mess with, but I've never used it or distributed to windows before -- am I headed in the right direction?

just having trouble finding some docs or a recipe or a walkthrough that explains how I might bundle a whole lib+tool instead of just a "script"

[willow pagoda]

i think the last time i tried this i had to make a venv with the package installed and some script to invoke the entry point manually

[rapid sparrow]

• python is not cross platform compiled pretty much. As best bet, you could be may be using CI runner with windows env to compile it (may be using Wine related stuff could work too?)
  https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners

  Run-PSScriptAnalyzer-on-Windows:
    name: Run PSScriptAnalyzer on Windows
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install PSScriptAnalyzer module
        shell: pwsh
        run: |
          Set-PSRepository PSGallery -InstallationPolicy Trusted
          Install-Module PSScriptAnalyzer -ErrorAction Stop
      - name: Get list of rules
        shell: pwsh
        run: |
          Get-ScriptAnalyzerRule

• Well, using something that is not OS specific like venv. that person recomended above
• Or use another language that is better OS/cross platform compiled (Golang works very well. i use it at linux too, to develop stuff for windows friends)
• you could still try cross OS compile it.. but as far as i know it is not intended feature in python and may be it will not work very well.
• may be in something like Cython, cross compiling will work better

[mental epoch]

ooh, okay. I had just tried doing like pyinstaller mypkg/cli.py and it choked because it didn't understand the imports were to other modules in the package, but setting up a venv first might could fix that, yar

[willow pagoda]

there are some command line options to help with that:
https://pyinstaller.org/en/stable/usage.html#what-to-bundle-where-to-search
particularly --hidden-import for installed packages iirc

[mental epoch]

yeh, aware that python doesn't have good (or any) cross-compiler support, but that's okay. I got a windows machine I use to play video games that I can use to build distributables by hand on an as-needed basis for now. I don't think I need to build it into the pipeline yet 🙂 just looking for a proof of concept at all right now

[rapid sparrow]

Golang flares super well to cross compile to any OS or CPU architecture 🙂
at my linux amd64, i compile for windows/macos arm64/amd64/plentiful of other archs
just a matter to toggle two env vars

[mental epoch]

this is a pure python package that isn't uploaded to PyPI yet but will be soon. my only goal here is to make sure that the CLI tool that's part of this package is easy for a windows user to run w/o manually installing a separate python interpreter 🙂

[mental epoch]

hmmm, yeah, I will play with this and see if I can get it to build something usable, thanks! maybe this is a good blog post if I get it working 😅

[rapid sparrow]

I don't think I need to build it into the pipeline yet 🙂
developing at linux, i find very myself very flabergasted when i need to switch to windows to develop smth.
it is way easier when smth works to compile in my familiar friendly environment ^_^
good idea to setup everything from the start to work smoothly

[willow pagoda]

ive also heard of nuitka for compiling python scripts, though my last attempt using it never panned out due to technical issues i couldnt figure out https://github.com/Nuitka/Nuitka/
codon's another compiler too https://github.com/exaloop/codon

[mental epoch]

huh. spooky

the way I write python, I would be in awe if anything could compile it 😄

[knotty basin]

i have a very weird problem

i have this in a multi stage build

# Set environment variables.
# 1. Force Python stdout and stderr streams to be unbuffered.
# 2. Set PORT variable that is used by Gunicorn. This should match "EXPOSE"
#    command.
ENV PYTHONUNBUFFERED=1 \
    PORT=8000 \
    PYTHONDONTWRITEBYTECODE=1 \
    USER="wagtail" \
    VIRTUAL_ENV=/app/.venv/ \
    PATH="/app/.venv/bin:${PATH}:/home/wagtail/.local/bin"

COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}

then the compose is like

  bfportal_dev_local:
    build:
      context: .
      dockerfile: Dockerfile
      target: dev
    restart: no
    command: bash -c "ls -la && python -c 'import sys;print(sys.executable)'"
    #  && python manage.py migrate --noinput && python manage.py ensure_superuser --username bfportal --email [e-mail] --password 1234 &&  python manage.py ensure_initialization && python manage.py runserver 0.0.0.0:8000
    container_name: bfportal_dev_local
    profiles:
      - dev_local

now there should be a .venv in the container

but there is not

bfportalgg-db_dev_local-1  | 2023-07-10 15:37:50.083 UTC [1] LOG:  database system is ready to accept connections
bfportal_dev_local         | total 40
bfportal_dev_local         | drwxr-xr-x  8 wagtail wagtail 4096 Jul 10 14:45 .
bfportal_dev_local         | drwxr-xr-x  1 root    root    4096 Jul 10 15:32 ..
bfportal_dev_local         | drwxr-xr-x  2 root    root    4096 Jul  6 15:04 .git
bfportal_dev_local         | drwxr-xr-x  2 wagtail wagtail 4096 Jul  8 15:30 __pycache__
bfportal_dev_local         | drwxr-xr-x  6 wagtail wagtail 4096 Jul  9 10:36 bfportal
bfportal_dev_local         | drwxr-xr-x 10 wagtail wagtail 4096 Jul  9 10:36 core
bfportal_dev_local         | -rw-r--r--  1 wagtail wagtail 1353 Jun 24 19:53 gunicorn.conf.py
bfportal_dev_local         | -rw-r--r--  1 wagtail wagtail  654 Jul 10 14:45 manage.py
bfportal_dev_local         | drwxr-xr-x  4 wagtail wagtail 4096 Apr 10 14:54 media
bfportal_dev_local         | drwxr-xr-x  6 wagtail wagtail 4096 Jul  6 16:00 theme
bfportal_dev_local         | /usr/local/bin/python

but if i do

$ docker run -it  --entrypoint=/bin/bash bfportalgg-bfportal_dev_local

it shows up !!!

root@dc796c876909:/app# ls -la
total 44
drwxr-xr-x  1 root root 4096 Jul 10 14:45 .
drwxr-xr-x  1 root root 4096 Jul 10 15:33 ..
drwxr-xr-x  2 root root 4096 Jul  6 15:04 .git
drwxr-xr-x  5 root root 4096 Jul 10 15:28 .venv
drwxr-xr-x  2 root root 4096 Jul  8 15:30 __pycache__
drwxr-xr-x  6 root root 4096 Jul  9 10:36 bfportal
drwxr-xr-x 10 root root 4096 Jul  9 10:36 core
-rw-r--r--  1 root root 1353 Jun 24 19:53 gunicorn.conf.py
-rw-r--r--  1 root root  654 Jul 10 14:45 manage.py
drwxr-xr-x  4 root root 4096 Apr 10 14:54 media
drwxr-xr-x  6 root root 4096 Jul  6 16:00 theme

ahhh

wait

i think i know

the reason

AHHHH im fucking dumb

i was mouting the volume in the same dir as venv

so it was over written

[rotund wyvern]

is it possible to run black on sphinx code blocks, especially in myst format

[loud bear]

I remember having seen a plugin for this

Dunno about the myst part

[rotund wyvern]

i felt like that too

couldn't find it

[loud bear]

search for "sphinx black" or something like that

[rotund wyvern]

yea, i tried that lol

[knotty basin]

any way i can decrease the size more ?

# Use an official Python runtime based on Debian 10 "buster" as a parent image.
FROM python:3.11-buster as builder
RUN pip install poetry==1.5.1
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_IN_PROJECT=1 \
    POETRY_VIRTUALENVS_CREATE=1

WORKDIR /venv
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends git
RUN touch README.md

COPY ["pyproject.toml", "poetry.lock", "./"]
RUN poetry config installer.max-workers 10
RUN poetry install --without dev --no-root --no-cache

FROM node:latest as node_base
RUN echo "NODE Version:" && node --version
RUN echo "NPM Version:" && npm --version

FROM python:3.11-slim-buster as dev

WORKDIR /app
RUN useradd --create-home wagtail

# Port used by this container to serve HTTP.
EXPOSE 8000
# Install system packages required by Wagtail and Django.
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends \
    build-essential \
    libpq-dev \
    libmariadbclient-dev \
    libjpeg62-turbo-dev \
    zlib1g-dev \
    libwebp-dev \
    curl \
&& rm -rf /var/lib/apt/lists/*
# Set environment variables.
# 1. Force Python stdout and stderr streams to be unbuffered.
# 2. Set PORT variable that is used by Gunicorn. This should match "EXPOSE"
#    command.
ENV PYTHONUNBUFFERED=1 \
    PORT=8000 \
    PYTHONDONTWRITEBYTECODE=1 \
    USER="wagtail" \
    VIRTUAL_ENV=/venv/.venv

ENV PATH="${VIRTUAL_ENV}/bin:${PATH}:/home/wagtail/.local/bin"
COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}


COPY --chown=wagtail:wagtail --from=node_base /usr/local/bin /usr/local/bin
COPY --chown=wagtail:wagtail --from=node_base /usr/local/lib/node_modules/npm /usr/local/lib/node_modules/npm
RUN chown -R wagtail:wagtail /app
COPY --chown=wagtail:wagtail ["package.json", "package-lock.json", "tailwind.config.js", "./"]
RUN npm install


# Copy the source code of the project into the container.
COPY --chown=wagtail:wagtail ./bfportal ./

FROM dev as final
USER wagtail
RUN npx tailwindcss -i ./bfportal/static/src/styles.css  -o ./bfportal/static/css/bfportal.css --minify
RUN python manage.py collectstatic --noinput --clear  -i static/src/*

[rotund wyvern]

blacken-docs found it

[restive jasper]

what's the current size of image (unpacked) and what's the size of node_modules?

[tawdry needle]

has anyone tried using Pylyzer yet? curious about experiences with it https://github.com/mtshiba/pylyzer

[sonic kite]

To maybe decrease the size of your Docker image, you can try the following suggestions:

Use a smaller base image: Instead of using the python:3.11-buster image, consider using a smaller base image like python:3.11-slim-buster or python:3.11-alpine.

Minimize installed dependencies: Review the dependencies in your pyproject.toml file and remove any unnecessary packages. Additionally, consider using slim versions or specific versions of packages instead of the full versions.

Optimize the Dockerfile instructions: Combine multiple RUN instructions into a single instruction to reduce the number of layers created. For example, you can combine the apt-get update and apt-get install commands into one RUN instruction.

Use multi-stage builds: If you only need certain files or dependencies during the build process and not in the final image, you can use multi-stage builds to separate the build environment from the production environment. This can help reduce the final image size.

Clean up unnecessary files: After installing dependencies, remove any temporary or unnecessary files created during the build process. For example, you can use the RUN rm -rf <directory> command to remove directories that are no longer needed.

[tawdry needle]

multi-stage builds are also an option depending on what you're doing

oh i see it there

[thorn portal]

that smells like a chatgpt answer 🤔

[tawdry needle]

it does but it's also not wrong

that or it's copy-pasted from somewhere

the only thing that seems hallucinated is the "slim versions" of packages, which typically isn't a thing in python

[sonic kite]

didn't found answer myself, but came closer

was a nice test to do

[tawdry needle]

i think in general it's not a good idea to post chatgpt or other ai-generated content as if it were your own

it's usually obvious, as above

[sonic kite]

ah ok, will tell it within message next time, thx for informing me

[tawdry needle]

plus if you aren't knowledgeable enough to verify the answer, you might end up spreading misinformation

[thorn portal]

hmm maybe it meant like installing a package without extras
e.g. fastapi v.s. fastapi[all]

[tawdry needle]

that's what i would charitably guess. but i think it's just confused and reusing the "slim" terminology from the docker image itself or from other packages. i've never seen someone refer to a python package as "slim"

[thorn portal]

quite right

my only comment re. the original question is:
i would use https://github.com/wagoodman/dive to inspect that's hogging the space and try to cut it down if possible.

i didn't see anything that screamed "i am wasting space!" in your dockerfile (unless you don't actually use your node stack after the npx call?)

[woven tangle]

Hi everyone I'm looking for some DevOps help

I've built an API using fast API and it works when I use it on my local host. The issue that I'm having is that when I try to deploy it to the GCP Kubernetes engine it does not work the way I intend. If anyone would be able to help me with this I would really appreciate it

The API creates ai generated QR codes that are scannable like this one, If anyone has experience deploying to GCP Kubernetes and would like to contribute this is the repo you can message me on here if you want. https://github.com/0xadamm/qrcode_api

[rapid sparrow]

are you aware of GCP kubernetes costs?

the cheapest pricing starts at 72$ per month
https://cloud.google.com/kubernetes-engine/pricing#cluster_management_fee_and_free_tier
Just for the cluster management fee

• Pricing for computing power of used nodes

if u seek deployment to kubernetes for pet project, better to seek smth like.. CIVO or DigitalOcean kubernetes clusters. they are starting from 5$ and 10$ accordingly

although load balancer will add in addition here.

may be cheaper actually deploying to microk8s or container deployment system without kuber

[rapid sparrow]

it is kind of fine for work, but may be not fine for pet projects usage ^_^

[woven tangle]

the issue i'm having with other options is that the api needs a gpu to run does digital ocean offer a container that runs with a gpu ?

[rapid sparrow]

nah. certainly not

[woven tangle]

btw I was not aware 🤣 I have $400 in credits rn

[rapid sparrow]

only biggest providers like AWS/GCP/Azure give such stuff i think

[woven tangle]

dang

[woven tangle]

ya so I have to use GKE its the easiest one I belive

[rapid sparrow]

technically AWS/GCP/Azure are equal in their complexity i think.

[woven tangle]

but anyway if the MVP works I will be able to get investors so I'm not to worried about costs just trying to get the MVP done

[woven tangle]

I should have said easier for me because I'm familiar with Google Cloud offerings

[rapid sparrow]

once upon a time, many years ago i deployed my application to terraform written GKE cluster 😆
currently heavily invested into AWS though

[woven tangle]

got it would you be interested in helping me out if I consider switching to AWS?

[rapid sparrow]

mm yeah, i think so. that is certainly in my interests to help in such matter.

i would certainly yeah help with that

[rapid sparrow]

is there any specific reason why u wish to deploy into kubernetes cluster though?

there are simpler options for sure

[woven tangle]

only because of experience with google cloud but I figured that would be the easiest for me

[woven tangle]

I def don't care what its hosted on not really department so I'm willing to switch

[woven tangle]

I added you as a friend maybe we can continue this convo in dms

[knotty basin]

is this chat gpt 💀 ,

also getting an alpine image for django and poetry is a pain

i went with slim-bookworm in the end

[restive jasper]

is it ChatGPT generated answer?

[rotund wyvern]

100%

[rotund wyvern]

yea

[rotund wyvern]

is anyone using vercel to deploy sphinx documentation

i'm trying to find a decent platform to host branch and bleeding deployments of my docs

i use github pages for my main version, but it's kinda hard to do a dev one

[flat path]

if you have a public github repo, why not use ReadTheDocs?

you can host multiple versions of your docs on there

[rotund wyvern]

i use a custom wrapper: https://github.com/vberlier/mudkip -- i'm not too keen to switch off of it tbh

[flat path]

fair enough 👍

[rotund wyvern]

i'm not entirely sure i could do what i do on readthedocs but i dunno

i also have custom urls

i wish github pages had branch deployments tbh

[finite fulcrum]

Anyone that used a local pypi mirror for caching? I looked into it a while back but couldn't figure out how to mirror a custom index, and waiting for slow docker installs after changing a local package and having to reinstall everything is getting annoying

[near venture]

I'm following this tutorial (https://www.youtube.com/watch?v=mBoX_JCKZTE) trying to learn web scraping using Scrapy. However, I'm having many problems that I would appreciate help solving. My first problem is with setting up a virtual environment. I've done the command and my window looks like the picture I linked. I think that means that I've installed it, but I don't think it's running. How do I make it run on windows? The tutorial says to type source...bin/activate. However, after research I learned that that's how you do it on apple but not windows. My second and final question is how to make scrapy work. I tried just ignoring the virtual env and just moving on. But then I got stuck on the command scrapy startproject bookscraper. After typing this I get the error: scrapy is not recognized as the name of a cmdlet, function, script file, or operable program. I do have scrapy installed and have the file located. I'm pretty sure that this is because I have it in the incorrect path but I don't know for sure or how I would even fix that. Again, any help would be greatly appreciated
Image

[restive jasper]

@near venture this page describes how to do everything incl. Windows:
https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/

[split pollen]

Is there a way to only copy some files via git submodule instead of the whole repo?

[thorny shell]

don't think so

[cedar nacelle]

Has anyone used dynatrace or datadog? If so opinoin? Was looking at prometues and that looks good but it says that push metric should be the exception. I would like to have the ability to make a decorator that would push a metric when the function is called.

One a performance metric and perhaps to join a call to functions that are distributed across different machines

[rapid sparrow]

prometheus is mostly open source solution that is kind of free (you maintain your own infrastructure) and gives time series metrics only for metrics monitoring
yes it has client libraries to have custom metrics pushing
prometheus is part of PLG stack = Prometheus Loki Grafana AlertManager, it has integration with other open source stuff though.

datadog is all encompassing expensive solution, that has logging, metrics, alerts, tracing, database monitoring, profiler and hundreds of integrations with everything, and with corellation of data between all different types of data. metrics stuff with dogstatd is merely fraction of its functionality. Most minimal maintanance cost, all infra is maintained by datadog at their side

So those solutions can do it both, but very different in price/maintanance cost and provided functionality besides that

[restive jasper]

@cedar nacelle prometheus is mainly about continuous metrics monitoring. The stack trace and pushing events, this is distributed tracing - like Jaeger, and probably Datadog and Dynatrace too

[rapid sparrow]

datadog is everything, including distributed tracing and continuous metrics and other things ^_^ (but expensive)

[cedar nacelle]

Like most of us, looking for a low cost solution because not many users. Might be 5 or 10. But there would potentially be be quite a few push events which could be 100s per sec.

I thought that redis might help so that you could push lots of events to it, and pull metrics from resis from promethues at its leisure.

[thorny shell]

I used datadog at my last job. I don't recall how difficult it was to set up, but it was great to use.

pretty good UI, and it was reasonably easy to find stack traces and log messages and stuff.

[rapid sparrow]

indeed. great to use. everything is corellated 😎 everything. traverse logging records to tracing data, tracing data to SQL queries, and in reverse. from logging records traverse to host info, its metrics of CPU/ram at machine and containers. Everything is traversable in all directions

i currently at my job pretty much integrated everything from zero to hero in regards to datadog

missed only setting up dogstatd stuff and some other small things yet

it has its quirks to figure out. but eventually you get a hang of how to make it working to 100% (and me as single person was enough to integrate it through everything)

to make it integrated though, person better be very comfortable with programming language integration happens (in my case python). + some infrastructure language like terraform. + some lambda written in some language (i did it in Go and it works nicely) is necessary to make it work nicely for database integrations + needing hang of AWS(or whatever your cloud provider) a bit, to integrate some related stuff as well
TLDR: devops engineering person will do this integrating stuff well, as long as his dev side is strong enough

we have some small parts of infra in php and ruby, but since i don't know those languages, datadog functionality is barely alive there. Well, PHP is going to be deprecated at our infra anyway, so should be not a big worry -_-.

[worthy agate]

Hi there!
I am having issues with terraform provisioners.
so I am launching AWS EC2 instance, and have wrote provisioners to copy private-key into EC2 instance.
and when I ssh the EC2 instance from my local machine and swithc to root user by 'sudo su -', it does not contain any files inside. I am new to terraform and might explain some staff not correctly.
Here is my provisioner code:

# Create a Null Resource and Provisioners

resource "null_resource" "copy_ec2_keys" {
  depends_on = [ module.ec2_public ]

    
  connection  {
    type = "ssh"
    host = aws_eip.bastion_eip.public_ip
    user = "ec2-user"
    password = ""
    private_key = file("private-key/eks-terraform-key.pem")
  }

  provisioner "file" {
    source = "private-key/eks-terraform-key.pem"
    destination = "/tmp/eks-terraform-key.pem"
  }
  
  provisioner "remote-exec" {
    inline = [ "sudo chmod 400 /tmp/eks-terraform-key.pem" ]
  }

  provisioner "local-exec" {
    command = "echo VPC created on `data` and VPC ID: ${module.vpc.vpc_id} >> creation-time-vpc-id.txt"
    working_dir = "local-exec-output-files"
  }

} 

and here is logs for terraform apply command

[cedar nacelle]

https://www.datadoghq.com/pricing/list/

Wow. You really like datadog. Just had a look at pricing and 1,500 per month for 100 servers is really up there.

[still charm]

Hi, did anyone of you installed python and electron on windows? Which environment do you suggest? visual studio? Is it true, it’s a bit a nightmare to compile an electron project on win?

[rapid sparrow]

i know right. the most horror price is not for servers though

the true horror price is for logging indexing

it makes half of our bill

i needed to implement a lot of terraform rules with filtering most not relevant traffics to fight issues

looking over diagrams of the most noisy methods in status INFO

that filtered like 75% of logging traffic and lowered pricing of datadog to amount that logging indexing is only half of our datadog bill 😅

• we lowered already logging to cheapest one with retention of only 3 days 😅

[cedar nacelle]

OK well that is not going to fly lol

[thorn portal]

i once had an infinite retry loop that pumped so much log that it basically wiped out my profit for a good week 😦

[rapid sparrow]

Huh. Our logging indexes have very low limits. (And good granularity over different microservices)

• We fought price further with good commitment

It would not have protected us from price for ingested logs and spans though (it has no limits)

[cedar nacelle]

Try the #user-interfaces channel instead

[heavy knot]

so i'm prreeeetty sure Im not supposed to get Aborted (core dumped) after I type help('modules') into python3 🤔
(sorry if this is the wrong channel, please direct me if not)

I have a help thread about it now. Im using Ubuntu 22.04 -- and apparently, as my friends have been calling it, I have been "rawdogging" my python environment. I know i know I need to learn how to environment.... im just getting started

[atomic oasis]

Hi there,
I am converting a simple Tic Tac Toe program that works in the terminal, to a popup GUI window.

However I am struggling with "print" commands.
I want to print the gameboard on a Popup GUI window (NOT the simple debug window that GUI provides)

This is my code.

    for x in range(rows):
        print("\n+---+---+---+")
        print("|", end= "")
        for y in range(collumns):
            print("", gameboard[x][y], end= " |")
    print("\n+---+---+---+")

sg.theme('DarkTeal9')

layout =[
    [sg.Text("Please fill")],
    [sg.Text(printGameBoard(), size=(15,11))],
    [sg.Submit(), sg.Exit()]
    ]

window = sg.Window("Tic Tac Toe", layout)
while True:
    event, values = window.read()
    if event == sg.WIN_CLOSED or event == "Exit":
        break
    if event == "Submit":
        break
window.close()```

The gameboard prints like this..

+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 4 | 5 | 6 |
+---+---+---+
| 7 | 8 | 9 |
+---+---+---+

*btw thats just the important bits of code, there is more that i havnt included lol

[brazen forge]

this question isn't relevant to this channel's topic
try asking in #1035199133436354600 or #user-interfaces

[rough hearth]

How did you guys get good at linux?

[thorn portal]

mostly by using it.
then also being annoyed by it and google better ways of doing things.

[rapid sparrow]

At uni we had initial crash course to get used to Linux console way of things

After graduation I just started to use linux as main OS all the time and eventually it got better

[jagged kiln]

Servers.

[rough hearth]

I just started junior devops role and configuring the ssh and mobaxterm got me questioning everything i knew

[jagged kiln]

I started by getting into homelab and general server stuff and it kinda went from there

[rapid sparrow]

That too. Servers and docker containers 🙂

[jagged kiln]

Then swapped to manjaro as my main os from Windows and distrohopped for like a year

general exposure was the best way for me

[rapid sparrow]

Silly you. I found my love from second try. Kubuntu LTS works

[jagged kiln]

heheh yeah I went through a bunch and settled on kubuntu

[jagged kiln]

it'll feel a bit overbearing at first but once you learn more it gets great

[rapid sparrow]

If u a DevOps though, Linux gives freedom from a start. Your tools just start to work correctly
And u don't need to connect to server to using them any longer

[jagged kiln]

ansible will be your closest friend :P

[rough hearth]

I like linux, but setting that ssh on my xubuntu vm, and the fact that the machines are connected via the company intranet was different that anything i did😅

[rapid sparrow]

Ansible can do some types of work, but far from being ideal for everyday server life

Immutable deployments of docker images into container orchestration systems are better.

Dokku / AWS beanstalk /AWS ECS / kubernetes

[rough hearth]

have you guys used rancher? if so how is it

[jagged kiln]

yeah of course, really depends on what you do. I work in low latency environments where something like kubernetes wouldn't work for us

[rapid sparrow]

I like and use Ansible but I fear each time it leads to settings drifted into not as a code documented area each time

Perhaps to try some other solution like Salt. It auto applies stuff every 5 minutes 🙂

[jagged kiln]

Salt, puppet and chef aren't agentless though :/
They're good for different things yk? I usually use ansible + packer to build images so we have a consistent starting point. We rebuild our images regularly so that we can easily just swap out the machines if necessary without really affecting anything

Ansible is really nice for provisioning prior to a snapshot

Obviously if you have mutable architecture then ensuring a guarantee state is a good thing, but different orgs have different requirements. Tis a matter of choosing the right tool for the job which makes our job so fun, since it changes so much!

I've worked with supercomputers, large scale AI and finance and they all have similar but vastly different requirements so it's always really darn hard to figure out what's best 😭

[restive jasper]

I firstly started to use it on my PC as main OS... When Windows Vista came out.
But then I started to do it on servers

[thorny shell]

I'd already used other flavors of Unix, so Linux wasn't much of a stretch

[sturdy tide]

Hey All! What would you recommend for open source loading testing packages?

[graceful light]

are you asking about load testing? Or testing the action of loading up a package?

[toxic bough]

@spice spireI've removed your post because we do not allow advertisements in our server.

[sturdy tide]

load testing (requests per second, response time, etc.)

[thorny shell]

well there's a simple and ancient thing called ab that was intended for that
https://httpd.apache.org/docs/2.4/programs/ab.html

doubt it's enough, but it might be a good start

[thorn portal]

i hear https://locust.io/ thrown around a lot. i have not used it myself, but my business partner said it's legit

[heavy knot]

pip install yowsup2

[thorn portal]

have you tried following the instruction? and please post text as opposed to screenshot next time.

[heavy knot]

what should i do?

[thorn portal]

do you see the text i highlighted?

[heavy knot]

yes

[thorn portal]

follow the link and follow the instruction there. i don't use windows so i can't assit you further.

[heavy knot]

guys what is the best package for making colored text in console or designed download bars in console

[flat path]

!pypi rich

[rancid schooner]

rich v13.4.2

Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal

[heavy knot]

!pypi colorama

[rancid schooner]

colorama v0.4.6

Cross-platform colored terminal text.

[cyan basin]

Hello, first question on this server and idk if this is the right server but I'm making a "color" calculator and I need help in coding outputs and efficiently naming variables. I am just a novice, and using Python.

[thorny shell]

@cyan basin ask in #1035199133436354600

[cyan basin]

Alright, thank you

[vague junco]

Hello everyone! Does anyone know of documentation/tutorial on how to make a private github repo a pip installable module?

[thorny shell]

I think you just want to build a python package the ordinary way, no?

more than you want to know is at https://packaging.python.org/en/latest/

[vague junco]

Thank you! First time doing this 😅

[thorny shell]

ok, be warned it's pretty tedious and confusing 😕

I personally use a tool called "poetry" to make it easier; ask if you're interested in doing that

[vague junco]

Yes please! All advice and tips are very welcomed

[thorny shell]

https://python-poetry.org/docs/ hee hee

let me see if I have a dirt-simple example

... don't think an example would be useful now (although of course you're free to ask)

ok here's the simplest set of steps to create the simplest imaginable project ```
❯ cd /tmp
❯ mkdir -vp project/project
mkdir: created directory 'project'
mkdir: created directory 'project/project'
❯ cd project/
❯ poetry init -n
❯ vi pyproject.toml # delete the line that refers to README.md
❯ cat > project/hello.py
def say_hi():
print("hello")

❯ poetry build
Building project (0.1.0)

• Building sdist
• Built project-0.1.0.tar.gz
• Building wheel
• Built project-0.1.0-py3-none-any.whl

[willow pagoda]

if you're interested in using setuptools instead of poetry, see their quickstart
https://setuptools.pypa.io/en/latest/userguide/quickstart.html
both approaches are pretty simple, though setuptools doesnt give you a CLI to do things - setuptools, poetry, and any other build backend should be installable with pip install . (pip install -e . if you want an editable install)

[vague junco]

Thank you so much to the both of you @willow pagoda @thorny shell! Will do some tests for both 🙂

[thorny shell]

eww, setuptools 🤢

[willow pagoda]

whats wrong with it?

[thorny shell]

fiddly, hard to understand

I suspect that the setuptools equivalent of my little minimal recipe above (#tools-and-devops message) would be a lot longer

kinda like requests vs urllib -- the later certainly works but is hard to use; the former is a nice higher-level wrapper around it.

I assume poetry wraps setuptools.

[willow pagoda]

mkdir -vp project/project
cd project
echo '[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "project"
version = "0.1.0"' > pyproject.toml
echo 'print("Hello world!")' > project/hello.py
python -m pip install build
python -m build```

[tawdry needle]

setuptools isn't particularly fiddly. people just tend to write janky extensions for it

it's much better than it used to be

[willow pagoda]

the boilerplate for pyproject.toml is the most annoying part of that imo, i end up copying it from another repository as a template

[thorny shell]

I end up editing it, as you can see 🤣

[willow pagoda]

according to this it doesnt
https://python-poetry.org/docs/configuration#virtualenvsoptionsno-setuptools

If set to true the --no-setuptools parameter is passed to virtualenv on creation of the virtual environment. This means when a new virtual environment is created, setuptools will not be installed in the environment. Poetry, for its internal operations, does not require setuptools and this can safely be set to true.

[thorny shell]

huh TIL

[tawdry needle]

poetry is its own build backend i think

that is, it's both a "frontend" (as in a CLI that you can use) and a "backend" (as in build-backend = ...)

as per the doc, including setuptools in the env is purely an affordance for IDEs and other tools that assume it is present

[white vector]

heyy, would you have any good source to learn kubernetes? i mean, deploy my apps, both backend and frontend, create the routing stuffs to be able to access those services from outside the cluster?

[toxic bough]

I have these two links bookmarked but I have not checked them out yet
https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-interactive/

https://www.edx.org/course/introduction-to-kubernetes

if I remember correctly, Joe (head of python discord devooops) recommended it

Ah, the interactive tutorial is not available anymore.

[rapid sparrow]

I personally jump started my k8s experience with this nice book

[white vector]

I tried with the official docs but not reaaaally helpful for me lol

probably i need some guide for dumbs 😂😂😂😂

[brazen forge]

https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/

TIL this exists

[slim maple]

Seems pretty good, but obviously you'd need to change a lot of stuff it generates 👀

[sturdy tundra]

Hello i have shp(Esri) file and i have raster file tif georeferenced.my shp file Has a polygons.how to clip tif file by polygons and save every clip polygons from tif to file?

[alpine horizon]

do the workflows in github actions get executed locally on the user machine when they push to the repo, or on the cloud?

[rapid sparrow]

They get executed in machines with GitHub runners

By default public GH runners are used

U can raise your self hosted runners anywhere, in any your infra

[alpine horizon]

and by defualut the GH runners are on the cloud, correct?

[rapid sparrow]

Cloud is just other people machines.
Microsoft could be having their own data centers used to run public runners

[alpine horizon]

true, the point is that its not executed locally

but how can I make the workflow use enviroment variables that I dont want to upload to the repo?

[rapid sparrow]

Unless u run self hosted runner at your own machine, it will not be executed at your own machine
(And not using solution named nectos/act)

[rapid sparrow]

U use GitHub secrets

I usually use more intricate workflow of keeping secrets in programmatic accessable secret storage
(AWS ssm parameters / AWS secret manager / Vault?)

Terraform code extracts them from them and applies to GitHub repo to become GitHub secrets/or env vars available for workflows

[alpine horizon]

you use them at work, or for personal projects?

[alpine horizon]

I'll check it out thanks

[rapid sparrow]

Both. Way more comfortable for long term maintainance

GitHub secrets aren't extractable once defined (unless doing dirty hacks with base64 encoding and running workflows)
I can pick my secrets in AWS secret manager or ssm param if I do / centralized accessability for apps (for deployment purposes)

[brazen forge]

AWS has some GitHub Actions to do that easily btw
https://docs.aws.amazon.com/secretsmanager/latest/userguide/retrieving-secrets_github.html

[agile stag]

Hey guys I was talking about developing a data integration automation program with a buddy of mine where it grabs json data from one website, store into a database and then another website retrieves some data from that database.

He said that we would use Docker for what I was doing and I don’t understand the operation of Docker when it comes to the program. All I know is when a program get containerized by Docker, all I have to do is type in a Docker command on my command prompt to get the program running instead of opening up the project for it to get it running. How else is Docker suppose to help?

[thorny shell]

docker lets you write for your favorite platform -- such as Ubuntu -- even though you might be running on some other platform.

It also lets you freely install dependencies without worrying about what else might be installed, or not, on the host.

It's a bit like having your own VM -- or as many as you want -- on a single computer.

These are nice advantages, but it is, of course, pretty complex to get going, so it's not always worth using.

[rapid sparrow]

Imagine docker image as compiling to binary file in a web development
(And it does not matter if your language is compilable or not)

Docker image contains inside imitated filesystem of a micro OS, and it is launchable at any machine with Linux core.

So, when u build python web dev project to it, it captures all your python libs to inside, all configurational files, all os level dependencies (ideally should be removed at later building step) to compile necessary python libs. It captures all specifics to run web project.

As end result, u get immutable image that froze all your dependencies like a binary file. It does not matter if pip lib is no longer available, your dependency is already gathered into image

As end result, docker image is launchable application at any Linux distribution (Ubuntu, debian, and etc)

Also, u can deploy it onto container scheduling cloud systems directly, without needing to deal with Linux machines
(AWS lightsail container deployment, AWS elastic beanstalk, AWS ECS, AWS EKS and any kubernetes or dokku and etc)

Deployment becomes a matter of changing deployed image version, and u have ability to return back to previous image for rollback in case of trouble
(Easier to revert deployment to previous version, than fixing current one)

As side result, u also have documented as a code truly all necessary dependencies to run your app, which heavily improves maintainability of your project in a long run

---

In comparison to VMs, docker is ultra lightweight and can be launched in hundreds at single machine (depends on how resources hungry application inside it though)

It is achieved via docker reusing current OS kernel to run itself. Docker is having filesystem inside and instructions that makes an illusion for app inside image that it runs on a separate Linux machines (but it does not, it reuses same machine)

As a consequence thought docker is not OS cross platform and not CPU architecture cross platform.
If u built it for Linux amd64, it will run only for Linux amd64 (which satisfies minimum necessary os kernel version)

This should change once WASM platform built docker images become more popular though

Tldr: for interpreted and JVM languages, dockerizarion is very essential thing to have magnitudes easier life.

It helps to compilable languages too, but in less capacity (depends on how os independent compiling result though), and even in this case it still helps by capturing configs inside too anyway

[agile stag]

There’s a lot to unpack here. You said a lot but thank you for the thorough explanation.

So basically, if a program works on my OS, it will be guaranteed to work on other OS’ and that’s because of the Docker image file that Docker creates.

[rapid sparrow]

Correction. If program works on Linux with for example amd64 CPU architecture, then it will work on any Linux OS with amd64 CPU architecture (that can install at least docker or other container launcher)

There are currently in cloud world only two CPU architectures pretty much.
Amd64 still dominates for like 90%, but arm64 started rapidly to grow in popularity for being cheaper hardware power

Buildx tool can help building image for several architectures at the same time / other options exist. But pretty much for single CPU architecture it is enough

[rapid sparrow]

See that I mention Linux explicitly. Windows and MacOS aren't supported, their community of solutions is very crippled and nearly zero.
Person at Windows /macOS as best hope can only utilize full Linux virtualization

So, image built for Linux amd64 and arm64 can pretty much run on 99.9% of modern cloud solutions (if we exclude existence of windows servers as relic of the past)

[agile stag]

Yeah so I’m trying to understand how a program will work on a Linux machine if I containerize a program on a Windows machine (It probably won’t containerize anyways because I had problems doing that with a previous project. I’m sure you remember that day lol.)

For context, I use windows for work and my company wants to host a database on a virtual Linux machine.

[rapid sparrow]

I heard rumors about wsl2 may be able providing full Linux kernel for docker (not guaranteing if it can)
(Depends on what docker will write as OS in docker version command. If Linux, then good, if windows then not)

But if it will not work, then using Docker Desktop (it powers full virt under hood) or full virt machine can work too

Better be not crazy and just developing from Linux though (dual boot for the win)

Makes life simpler in web world

[rapid sparrow]

U could also rent Linux server all the time for work as alternative.
Visual code gives functionality to develop directly over ssh from another machine

Also I saw stuff that provides it as a service with GUI and other stuff. Not a hard stuff to setup all of it on your own though

[restive jasper]

read something about Docker dude

[thorn portal]

what should they read about Docker?
I am sure they will appreciate any link you can send their way instead of just a vague statement like that!

[restive jasper]

like... anything about Docker?

"TechWorld with Nana" is good enough imo

any introduction to Docker will answer this question: "How else is Docker suppose to help?"

[thorn portal]

well i am just trying to get you to help them in a more constructive manner 🙂

[rapid sparrow]

@restive jasper @agile stag
Everyone should read this book. This author is captain of container world and gives material in a very understandable way

(I recommended his book about kubernetes just recently)

[worthy agate]

Hey man!
do you think you could help me out with k8s and aws issue?

[rapid sparrow]

definitely no. because you did not ask your question
https://www.pythondiscord.com/pages/guides/pydis-guides/asking-good-questions/
https://slash7.com/2006/12/22/vampires/
read some guide how to ask good questions and ask them instead of asking to ask.
Also don't ping me. ask everyone. (but don't ping everyone, hehe)

[worthy agate]

I am having issue with my kubectl node.
I have launched all resources using terraform. AWS EC2 Instance with security group, AWS VPC, and EKS cluster also on AWS.
And from my local terminal I did get worker node by running kubectl get nodes, it outputs node that is running, also comand kubectl get namespaces outputs defualt, kube-node-lease etc. And after I created pod by running kubectl run my-first-pod --image stacksimplify/kubenginx:1.0.0 the image is from docker.hub, output showed that pod was created succefully. Exposed the pod by running kubectl expose pod my-first-pod --type=NodePort --port=80 --name=my-first-service output showed that pod was exposed. and than by running kubectl get nodes -o wide I will be take the public ip of node, and by running the kubectl get svc I get port. and if I paste public ip of node and service port example: 35.181.62.190:30023 in the new tab of the browser, it does not open the web page, it just keeps loading.

I am working with kubectl and eks cluster for the first time, so my explanation might be unclear, let if its.

can someone help me out with the issue?

[worthy agate]

I have tried to re-launche all resources from the 0, but still getting same issue. I have tried to access the route 35.181.62.190:30023 from other browsers, and it did not help. I have checked eks security group from aws console for allowed port, and it shows that all ports are able to access it, so its 0.0.0.0/0

did not had other ideas to troubleshoot.

[rapid sparrow]

1. there can be issues with network security group not allowing external access at the chosen port (your can select at least via EC2 instance, assigned security group and to check existing rules for inbound traffic)
2. your EC2 instances can be not having assigned public IP addresses (you can check their network tab regarding that)

3. if you kubectl get svc, shows assigned IP and port, that is already good for last point

so, just 1th and 2th point are potential trouble

[rapid sparrow]

3. An alternative to creating nodeport, is creating LoadBalancer btw. It should be more public by default may be, not guaranteed though

[worthy agate]

so inbound rule is open for all ports, right?

and for 2 case, I also have the launched ec2 instance with public IP

btw, when I do launch all resources, and ready to run kubeclt commands, it always gives me error message Unable to connect to the server: dial tcp: lookup 936220BBA09C04113BB1A72C73AD9E80.yl4.eu-west-3.eks.amazonaws.com on 127.0.0.53:53: no such host unless I update context by running this command aws eks --region eu-west-3 update-kubeconfig --name hr-stag-eksdemo1.

can it be the cause for the issue?

this are two security groups that I have created, but as I can see, both are open for all networks, isn't it?

[worthy agate]

and I am still facing the issue 😓

[burnt gust]

VPNs, which ones are worth using? (is this where this question go?)

[rapid sparrow]

self hosted wireguard is art of beauty

[thorny shell]

https://www.nytimes.com/wirecutter/reviews/best-vpn-service/

[short peak]

I have used ProtonVPN for few years so far and it's good. Sometimes I need to turn it off because for instances, I'm not able to download npm packages, perhaps is the country I chose, I don´t know... ChatGPT doesn't like it when I through VPN either 🤷‍♂️

[normal scaffold]

Yes there is Spiffworkflow Python library that does it: https://www.spiffworkflow.org/

[alpine horizon]

Does anybody here have an example of running all the tests on a OS matrix, using Poetry for installing the dependencies?

(for github workflows, to run on each commit)

[astral apex]

What are "all" tests?
Is your testing framework in the dev dependencies group, or somewhere else?

[astral apex]

Every commit?
To every branch?

[alpine horizon]

All the modules in tests/.
I'm using pytest which is lisetd as a dev dependency

[alpine horizon]

Yes and yes

[astral apex]

Actually
Take a look at https://github.com/ionite34/einspect/blob/main/.github/workflows/tests.yml
I went to double check an example that I knew used Poetry, and this is actually super minimalistic. It's really clean, I like it.

The pip install pytest . is the magic here.
The . will cause CI to see your package, and install it, and pytest grabs, well, pytest.
Then he just runs pytest.

If you want to run on all commits, you can just change

- on: [workflow_call]
+ on: [push]

I've seen lots and lots of people come up with some crazy solutions to install and manage Poetry in CI.
Even here we have our own entire action -- https://github.com/python-discord/bot/blob/main/.github/workflows/lint-test.yml#L34-L37
But you don't need any of that if you just pip install .

It's a little limiting, because it only installs the "core" package, you only get your "main" dependencies, you can't get your dev deps or any other groups -- that's why Ionite has to install pytest manually here
But if you don't need any of that, this is really cool and super simple.

[rancid schooner]

.github/workflows/lint-test.yml lines 34 to 37

- name: Install Python Dependencies
  uses: HassanAbouelela/actions/setup-python@setup-python_v1.4.1
  with:
    python_version: '3.11'```

[rare hamlet]

Is there a lint rule for whichever linter that yells at people who use the "in" operator on a list or tuple instead of a Dict/Set/OrderedList?

[alpine horizon]

wow, how did you do the color change in the markdown??

[alpine horizon]

that workflow seems like what I'm looking for. I've seen some other interesting stuff I might copy as well for the docs, is that from the same Ionite in this server?

[rapid sparrow]

it is git diff syntax highlighting
```diff
- removed code
- removed code2
+ added code
```

It is similar to having syntax highlighing for python code
```py
print(123)
```

[alpine horizon]

oh damn

I didnt know there was diff syntax on markdown

[rapid sparrow]

it is not technically markdown, discord is not pure markdown
They use something similar to https://highlightjs.org/static/demo/ with their own slight modification

some javascript library for highlighting to their electron interface

this documentation pretty much for very good % matches available choices btw

[astral apex]

It is

[alpine horizon]

why people do people say that when testing a package you should install it locally and test that instead of the local files? isnt it just the same thing?

I feel like the only thing that can go wrong is with the imports, that they work locally, but not trough the package, but thats very uncommon.

[astral apex]

Imports is one of the big ones.
File paths is another big one, devs load a config file or a .env from the CWD, then they deploy their package and can't access that file anymore.
How are you seeing that it's "uncommon"? I'm not really sure how often these things happen, but just by testing against an installed copy of your package, you can make sure that doesn't happen at all, because you're going to be forced to do it correctly from the beginning.

[alpine horizon]

I did not think about config files, thats a good point.

[astral apex]

There's many different little things
You avoid almost all of them outright by just doing this one little thing
It's highly recommended

[alpine horizon]

the imports issue, only happend once to me a while back... I dont think theyre that big of an issue once you know how they work

thats fair

but what if you have test/ and package/ in the same directory and you do pip install . -e, how do you know that the tests are importing the pip-package vs. the local files?

[astral apex]

That's why you wrap your package/ in a src/

[alpine horizon]

oh idk about that

in some pacakges thats reserved for the C code, and package/ will contain the Python bindings

idk if thats a standard or what

[brazen hemlock]

guys, this is my docker compose:

version: '3'
services:
  web:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - ./static:/usr/src/app/static
      - ./media:/usr/src/app/media
  
  nginx:
    build:
      context: .
      dockerfile: nginx/Dockerfile
    volumes:
      - ./static:/usr/src/app/static
      - ./media:/usr/src/app/media
      - ./nginx/nginx.conf:/etc/nginx/conf.d/nginx.conf
      - ./logs/nginx:/var/log/nginx # new volume mount

    ports:
      - "80:80"
    depends_on:
      - web
    command: /app/wait-for-it.sh web:8000 --timeout=60 -- nginx -g 'daemon off;'
  
volumes:
  static:
  media:
  

the changes on static and media folder aren't bi-directional, why? (its a dango project, when i add an archive with a view function, it doesn't go to my machine folder too, only the docker container one), can somebody help me?

[indigo zenith]

I would guess you have a permissions problem preventing your container from writing to the folder

[willow pagoda]

not sure about it either, but if i had a library and a python wrapper in the same project, i would put the wrapper in a subdirectory to avoid conflicting layouts: src/ lib.c python_wrapper/ src/ my_package/ ... pyproject.toml ... as for testing, it starts to become pretty important as your build configuration increases in complexity, for example having to include multiple packages, data files, dependencies, plugin entry points, etc.
(additionally if its an extension module that needs to be compiled, you of course can't ever test it without running your build)

[astral apex]

Yeah, and going back to Ionite's work again, he made a Rust extension, https://github.com/ionite34/base2048/, and Rust requires a src/ folder, so he made a src-py folder for the Python typings

You can use whatever you want in Python

[alpine horizon]

Right, you just need to modify 'from' in the pyproject.toml

Its interesting how pandas stores the 'tests/' directory inside the package

https://github.com/pandas-dev/pandas/tree/main/pandas

[topaz gate]

is anyone aware of a tool or program that finds words on a webpage and checks the link and pages it leads to, for that word?

[rapid sparrow]

😬 it is called, web scrapping. A very unethical thing to do

This book has all the stuff to learn it / recounts all different tools to approach it

[topaz gate]

just using it to find a story on a forum

[rapid sparrow]

very shortly telling though, there are pretty much only two approaches

• using requests library for raw pages check (does not render javascript), using beautiful soup to get content out of them
• using selenium to process javascript rendering web sites
  of course those tools have alternatives, but those tools are most stable in their functionality and easy to use

[thorny shell]

it's called "web scraping" (one p), and is not always unethical

it often, but not always, violates the web site's "terms of service"

[topaz gate]

so ctrl f is web scrapping i guess.

[thorny shell]

no, but I think I get your point.

[rapid sparrow]

you mis understand the harm.
of course ctrl +F is not web scrapping
and single downloaded and parsed page.. is not that bad too.

problem araises

• that web scraping bot speed does scale, and they repeatedly do and add inhuman amount of workload onto websites and its databases (if it is forum, it does use database)
  • which in its turn bad in terms of additional money payments for server resources / or less availbility of resources to other people
• also used for unethical purposes like to collect user information for spamming and scams from social networks, or to buy with inhuman speed some rare goods for reselling at 200-300% prices.

it is all becoming okay, if u have web site's owner permission ^_^

[topaz gate]

fair

[alpine horizon]

guys please do not worry about ethicality, most websites collect data on you, so it only seems right to steal some of it back 🙂

[rapid sparrow]

i am kind of afraid of what kind of career u will have (what else next, robbing banks? 😅 )

[thorny shell]

this isn't the forum to debate ethics

[alpine horizon]

@topaz gate the way I look at it is as long as you behave like a regular user, i.e. dont make too many request and dont overload the server your good.

[alpine horizon]

this. this server is for Python help, and unless its something that its actually causing harm to someone, its not your role to tell ppl what they or shouldnt do.

[rapid sparrow]

!rule 5

[rancid schooner]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[topaz gate]

I'm not using for any of that just load the page text finding the word then that maybe ad it the link to text doc it can take all day for all i care

it is aloud on the terms of service just can't steal the data on the site

[alpine horizon]

all good bro, you dont need to explain your reasons to us

just use parsel instead of BeautifulSoup4, thats my advice

[rapid sparrow]

🙈

[solar zealot]

@alpine horizon What's the advantage of parsel? Edit: Ah, okay. Support for xpath selectors is a pretty big win.

[alpine horizon]

exactly, aside from the fact that bs4 has a very shitty and unintuitive API, you are very limited in what you can do, because you can only search for simple patterns, if that.
but parsel has a very simple API, and supports Xpaths and CSS selectors out of the box, so you can do pretty much everything

and it has a fairly big community as well, its a component of scrapy

[misty drift]

sup everyone, i need to send automatic emails with embedded images with python. does anyone have a link where i can learn to do this?

[restive jasper]

@misty drift embedded image means html emails?

[misty drift]

yeah, i've seen some videos where they use html but haven't seen one where they use images in the html's body

[restive jasper]

this actually was a leading question

[warped latch]

I feel like I've never been in a situation where I couldn't select what I wanted with bs4...

anyway, I had a question. I'm thinking about implementing celery to handle sending emails in the background. I was reading about redis and how its an in memory store, and I was thinking about how much RAM is redis going to use out of the box? Is it going to rapidly scale to where it's using a lot more RAM than I want it to?

    An empty instance uses ~ 3MB of memory.
    1 Million small Keys -> String Value pairs use ~ 85MB of memory.
    1 Million Keys -> Hash value, representing an object with 5 fields, use ~ 160 MB of memory.

It seems like that's not bad....

what makes people decide between redis and rabbitmq for this?

[slim maple]

Redis should be ok for celery, I don't think celery would need to use a lot of keys

There are also alternatives to redis like keydb and dragonflydb

[warped latch]

hm, the usecase is just to be a message broker for a task queue. There are so many options and use cases for these tools idk which is best.

the tasks are sending emails mostly

[slim maple]

I think redis works fine with celery, you can always swap it out later

[warped latch]

Im leaning towards rabbitmq

[slim maple]

Why?

[warped latch]

free SSL mainly.

[slim maple]

You don't really need ssl in internal network, but I may be wrong here

[warped latch]

It depends on if your network is compromised or not I believe.

and since the emails are sending very sensitive tokenized links, I dont see why I wouldnt just use it.

besides rabbitmq is a dedicated message broker.. i could be wrong but it seems like it might be more suited for what i want it to do because ofr that.

[slim maple]

I mostly use redis because it's easier to deploy, but there's no wrong choice here

[warped latch]

yeah...

I also may be confusing something with what i said above.

I dont know if the message contains the email or... does the message only contain an instruction to a program that writes the email. I will have to determine this.

[restive jasper]

angular / react / vuejs sites, i.e. client-side frameworks without server-side rendering enabled

[warped latch]

well, that isn't an issue with selecting, that's an issue with the elements not being loaded yet.

[restive jasper]

no, it's because they are loaded by client-side javascript, and bs4 doesn't handle javascript at all

[slim maple]

You could use api in this case

[restive jasper]

such API is usually undocumented, so if you want to use it you have to reverse engineer it

[slim maple]

It's not hard to check how frontend app uses that API and do the same

I'd rather deal with that than parsing html

[gray wing]

idk, if i asked in wrong channel im sorry, but this channel is good for this question for me
https://www.npmjs.com/package/ms
for python is any pip what do the same what this npm?

[ripe tree]

hiya guys! who can rate my translator's code?
i`mport googletrans
from sys import argv
from pyperclip import copy

def words_translate(words, lang):
"""text_translator

Args:
    text (str): enter sentence or word that you want to translate
    lang (str): enter language
"""

if lang not in googletrans.LANGUAGES:
    print('maybe there is no language you wrote')

translator = googletrans.Translator()
return translator.translate(words, dest=lang)

def detect_lang(words):
"""detect_lang

Args:
    words (str or list): enter word/s whose language/s you want to detect
"""

translator = googletrans.Translator()
if words == str:
    translator.detect(words)
    return translator.detect(words)

def all_langs():
return print(googletrans.LANGUAGES)

if len(argv) == 1:
print(
'Welcome to TermTranslate! \nHere, you can freely and easilly translate text.\n')
print('HELP_\n'
"python (or python3) [your path to main.py] [functions] [options]\n"
"functions:\n"
" --wordtran: translate words and texts:\n"
" options:\n"
" 'your word or text' [language that you want]\n"
" --dctlang: detect language/s of words\n"
" options:\n"
" [your word]\n"
" --langs: all available langs\n"
"CAUTION: all texts or few word would be in ''\n")

if argv[1] == '--wordtran':
translation = words_translate(words=argv[2], lang=argv[3])
print(translation.text)
if len(translation.text) > 10:
copy_choice = input('Copy this text? y/n: ')
if copy_choice == 'y':
copy(translation.text)

if argv[1] == '--dctlang':
detector = detect_lang(words=argv[2])
print(f'Language: {detector.lang}')

if argv[1] == '--langs':
all_langs()
`

[tawdry needle]

1. this isn't really the right channel, maybe a help channel would be better

2. you can use a code block for longer sections of code. see the explanation for "multi-line code block" here: https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline-#h_01GY0DAKGXDEHE263BCAYEGFJA

[warped latch]

yeah but you're talking about different issue. I'm talking about the selection of elements using selectors. If elements are loaded, bs4 has the power to select them under any context.

The reason you cannot select elements loaded by js is because bs4 cannot load them, and yes, at the point, you need a different tool, but it has nothing to dowith its ability to select

[tawdry needle]

it's not that bs4 cannot load them, it's that they literally don't exist in the raw html before executing the javascript. so bs4 cannot possibly find them.

[slim maple]

You'd need something that can execute JS (like a browser + selenium) in order to parse them in any way

[warped latch]

yeah, that is better way of saying it.

the elements aren't there though, needless to say. I was just commenting that... another tool that has different selection features isn't necessarily better at selecting elements than bs4, which can select just about anything if it has the html to select.\

you could give me infinite divs with no class or ids at all, and I could select them. It would suck haha

but you could do it

you can even select by other attributes if you need to.

[astral apex]

Isn’t there a bs4 flag to load JS?

[slim maple]

I don't think so

It's ultimately out of the scope of bs4 imo

Also i'd say there are ultimately better alternatives to bs4 like selectolax

[warped latch]

I think I have decided to start with redis, since it does seem easier to set up, and if I wanted to change message broker, I believe it wouldn't be too difficult.

[alpine horizon]

Does JIRA have something like GH Actions for automating the tests?

(Specifically BitBucket)

[alpine horizon]

I doubt that. Can you find an element thats its grand parent has the text which partially matches to "abc" (contains), and one of its ancestors is of class 'div', snd one of its siblings has the class 'x' and the attribute 'type'?

And the node itself has to contain the attribute 'abc'

[warped latch]

generally what you do in situations like that is you select a parent element first, so that you isolate everything within it to select further. if you have to you can literally do div > div > div, it sucks and its prone to breaking on a very slight change of the site. but you can do it.

[alpine horizon]

Yeah it really does suck.

You can write all that in a single XPath query.

[pliant flume]

basically this workflow should run when i create a new release on github

[terse vigil]

i have this problem in my docker volume. my app is made with Streamlit. after modifying code in files in pages folder, localhost successfully updates the changes immediately. however, any changes i made in utils file (which is being imported by files in pages folder), no changes were seen in the localhost unless i restart my container. Changes in the code is seen when i checked the container log in terminal which means the volume is correcly mounted and Docker really picked up the changes, but not the localhost :/.

Here is my Dockerfile and docker-compose.yml with my project structure

Dockerfile:

# base image FROM python:3.9-slim WORKDIR /app RUN apt-get update && apt-get install -y \ build-essential \ curl \ software-properties-common \ git \ && rm -rf /var/lib/apt/lists/* COPY requirements.txt . RUN pip3 install -r requirements.txt COPY . . EXPOSE 8501

docker-compose.yml

version: '3' services: app: build: context: . dockerfile: Dockerfile command: streamlit run Home.py --server.port 8501 --server.address=0.0.0.0 volumes: - ../abtest_frontend:/app ports: - 8501:8501 image: streamlit container_name: abtest

folder structure:

ab_testing_framework |-- ab_test_platform |-- docker |-- Dockerfile |-- docker-compose.yml |-- requirements.txt |-- abtest_frontend |-- pages |-- page1.py |-- page2.py |-- page3.py |-- page4.py |-- utils |-- metrics.py |-- Home.py

[tawdry needle]

can you enter the container in a shell and just do a simple test here? change the file outside the container, then open it with less or whatever inside the container and see if it changed there

unrelated: 1) you probably don't need software-properties-common, 2) ideally you should specify the exact debian version in the image tag, eg bookworm

[night pollen]

Hey everyone👋 I'm not sure if this is the correct channel to ask in so if not feel free to tell me where to go! I've currently been tasked with building a "Job Framework" which will be responsible for scheduling/running many periodic background jobs/tasks. The requirements are essentially:

• Handle running many tasks in parallel.
• Handle dependencies between tasks i.e. only run task B if A completes.
• Get status of a task.
• Store tasks results.
• Be able to monitor task queues, overall health of the system.
• Set up alerts for certain tasks failing, if overall health/certain metrics dips below a set standard.
• Be able to scale to handle a growing workload.

What do people normally use in a production level system for something like this? I know Celery is used quite commonly, but I've also come across tools such as Temporal that provide a nice UI (I know Flower exists for Celery also). If it makes a different we primarily use AWS + Datadog. I'm just trying to gauge really what people mostly use in Production to accomplish this. Thanks in advance for any help.

[thorny shell]

I've not used it personally, but I'd look into celery with the SQS backend

your bullet points sound very straightforward; I'd be surprised if you had to write a lot of code for this

[wispy fossil]

Where can I deploy python scripts that act as listeners for email clients?

[thorny shell]

on your laptop, although that's unlikely to be very useful 🙂

more realistically: some cloud provider like EC2, Azure, whatever.

I think what you want is called an "SMTP server"

[astral apex]

https://blog.cloudflare.com/announcing-route-to-workers/

[wispy fossil]

Thanks!

[mighty anvil]

Hello. Any help appreciated. Bashing my head against a wall here... Trying to deploy Django project to Heroku I get the following error, I’ve tried multiple versions of smart-open and tried removing it from requirements.txt, the invalid specifier is NOT in my requirements.txt (this is: smart-open==4.1.0) or I have tried multiple other versions, all break the build. I’ve tried changing setup tools on Heroku to eg setuptools==40.3.0, no dice (same errors).
python setup.py egg_info did not run successfully. exit code: 1
error in smart_open setup command: 'python_requires' must be a string containing valid version specifiers; Invalid specifier: '>=3.6.*'
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

[thorny shell]

is your project on github where I can see it?

[mighty anvil]

it's a private repo @offby1

[brazen forge]

looks like you need a version of smart_open >= 5.2.0

their setup.py was fixed in this commit

[rapid sparrow]

u can set more fuzzy requirements in requirements.txt like
requirements.txt

types-redis==4.6.0.3
smart-open>=5.2.0
vine

😉

[mighty anvil]

trying that but running into a whole bunch of dependency errors so can't confirm yet. Thanks.

[mighty anvil]

(Django/Heroku/VScode)

[tawdry needle]

honestly, apache airflow

the only downside with airflow is that it's very big and complicated. but it does basically exactly what you want. the only thing it doesn't really do well is queuing up a very large number of homogeneous tasks. it's possible but i think it expects you to have a standalone "task queue" for such a purpose (like AWS Batch)

airflow uses celery internally. you could build your own thing with celery of course, but airflow includes a nice web ui, a declarative-ish interface for defining jobs, a huge developer community, etc

not sure about setting alerts for task failures though. i know we have something like that in our deployed airflow but i didn't set it up and i don't know how much hacking they had to do for it

[spiral oak]

Hi folks, not sure which channel to ask this question, but here goes.
I have a python script which contains some curl commands, that asks the user to enter their username and password as command line arguments.
Now the problem I face is, if the user's password has special characters like $, # etc, these get converted into their ASCII and then the curl command fails because of Authentication error, as the password has different value now.

ex:

python test_script.py admin Pa$$word

The value of password gets converted to Pa130481word

Can someone please help how to fix this. Tried Googling and no good help.

[tawdry needle]

bash and other shells expand $$ to the PID of the current process

you need to put pa$$word in quotes

python test_script.py admin 'Pa$$word'

[spiral oak]

Oh okay, cool. Let me test this out. Thanks @tawdry needle

[warped latch]

Is there some arcane knowledge that would make an nginx docker container use my allow list?

172.20.0.1 - - - - [03/Aug/2023:17:07:35 +0000] "GET /admin/ HTTP/1.1" 403 125 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0"

This is the access log.

    location /admin {
        include /etc/nginx/allowme.conf;
        deny all;
        limit_req zone=limitadmin burst=5 delay=10;
        try_files $uri @proxy;
    }

location block

The allowme.conf file has my public and local ip as well as this in this format

allow 172.20.0.1;

I also could not get it to recognize the official IP list provided by stripe, which I assumed was them not maintaining their IP list, but it seems I just can't get ngninx to respect any allow condition even to myself.

[thorny shell]

are you sure that /admin/ matches /admin? That trailing slash might be relevant; I don't know

[warped latch]

it's not that. In this case, it looks like the reason it didn't work is because I needed to rebuild the container after adding the docker ip.

That being said, I'm still not sure why it was not respecting the stripe ips, but now that I have access logs set up I can re-implement it and see if there is a more obvious answer why after that.

As for the trailing slash, I believe it's the other way around. If you include the trailing slash, then not having it will not register as that location.

[thorny shell]

that makes sense, since I assume it's just doing a simple prefix match

[warped latch]

the thing that confuses me, is that that ip is not going to be relevant in a production environment. I'm not going to be intersted in what the local docker ip is. I'm going to want to know what the actual ip is that is trying to connect.

i guess i will figure it out when i get there

okay, I know why. Its because every IP no matter what is logging as 172.20.0.1

172.20.0.1 - - - - [03/Aug/2023:17:24:00 +0000] "GET /admin/ HTTP/1.1" 200 2486 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0"
172.20.0.1 - - - - [03/Aug/2023:17:34:05 +0000] "POST /stripe/webhook HTTP/1.1" 200 20 "-" "Stripe/1.0 (+https://stripe.com/docs/webhooks)"

here is log format

log_format logformat '$remote_addr - $http_x_forwarded_for - $remote_user [$time_local] '
                     '"$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"';

This might be relevant

    location @proxy {
        proxy_pass http://django;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
    }

So, how do I access the "real ip"

okay i was also wrong about that. I tried accessing from another device on local network, and it correctly blocked me and logged the ip

What I suspect is the issue, is that the simulation of webhook events is coming from my machine, but in production, they would be coming from a real stripe ip.

[indigo zenith]

Is there a reason you're calling curl instead of using Python? Doing it with requests or something would probably solve the problem and just be a lot cleaner in general.

Also not sure if it will help solve the problem but this is nice: https://docs.python.org/3/library/getpass.html

[tawdry needle]

i've seen worse things than subprocess.run(['curl', '-u', f'{username}:{password}', url])

requests is probably better

[heavy knot]

someone checkout #1136762867652239580

[gentle solstice]

subprocess.run(f"ls {file}")

That's bad even in bash

[rotund wyvern]

getpass is a module?!

so random

[thorny shell]

random is also a module

now, if module were a module, that'd be ...

!e

import this

[rancid schooner]

@thorny shell :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | The Zen of Python, by Tim Peters
002 | 
003 | Beautiful is better than ugly.
004 | Explicit is better than implicit.
005 | Simple is better than complex.
006 | Complex is better than complicated.
007 | Flat is better than nested.
008 | Sparse is better than dense.
009 | Readability counts.
010 | Special cases aren't special enough to break the rules.
011 | Although practicality beats purity.
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/SODQP7BFZPFDXPJ54S2WFB6RCM

[gentle solstice]

Module is a type

!d types.ModuleType

[rancid schooner]

types.ModuleType

class types.ModuleType(name, doc=None)```
The type of [modules](https://docs.python.org/3/glossary.html#term-module). The constructor takes the name of the module to be created and optionally its [docstring](https://docs.python.org/3/glossary.html#term-docstring).

Note

Use [`importlib.util.module_from_spec()`](https://docs.python.org/3/library/importlib.html#importlib.util.module_from_spec "importlib.util.module_from_spec") to create a new module if you wish to set the various import-controlled attributes.

[sweet zephyr]

hello

[fierce wyvern]

Hi all, I've just built a tool that can be used to automatically produce documentation for code and scripts. Any feedback is much appreciated 🙏

https://github.com/bitgangstudio/bimbus-simple

[alpine horizon]

how can I create a poetry virtual env using pyenv for the python version?

[restive jasper]

poetry & pyenv are two separate tools. One manages python binaries (different versions of python). Another manages python modules (like pip). You just use both:
https://www.adaltas.com/en/2021/06/09/pyrepo-project-initialization/

[alpine horizon]

Thanks @restive jasper , I tried this tutorial and it kind of worked until Pycharm decided to delete my virtual env for some reason

I fu**ing hate pycharm:
https://youtrack.jetbrains.com/issue/PY-54833/Pycharm-deletes-existing-poetry-environment

[lofty marsh]

Hi can anyone help out pls? I have been struggling for a couple of days with a slight detail on a python script that emulates netcat. The thing is when i get shell and type ls the first time i don't get anything i retype and get the right output but then i type pwd and i get the output of ls and need to retype pwd.... I know it has to do with the buffer but I can't find the way out.... Anyone pls?

[thorny shell]

ask in #1035199133436354600

[lofty marsh]

ok thx

[tawdry needle]

yeah but subprocess.run(['/bin/ls', file]) is a lot better, which is what i was suggestion

[tawdry needle]

hatch for example will look for pyenv on your system and use its installed versions when available. it's reasonable to ask about integrating them

[gentle solstice]

It's bad in bash because usually it's doing via ```bash
for file in $(ls foo); do
:
done

When it could be done via ```bash
for file in foo/*; do
  :
done

[tawdry needle]

well that's another story

[umbral mist]

^^^

these are my dns records

[pine ermine]

Hey, quick question, suppose a friend of mine deleted files from his branch and pushes it to the remote branch, in git

How can I reflect those changes in my local repo?

Like, do fetch or pull also delete files according to the remote branch? (If a file no longer exists, when doing fetch or pull, does it also delete on my local branch?)

[thorny shell]

yes, "fetch" and 'pull' will indeed delete the files

try it and see

if you're scared, make another clone and try it there.

[brazen forge]

git fetch won't delete files. it only updates the pointers to remote refs

git pull also won't delete files, as it doesn't touch untracked files

oh wait, according to the remote branch, yes, git pull will delete files, because that's the change being applied

but you can always get the file back by checking out the older commit that had it

[knotty basin]

In a project of mine, i want to make contributing easy by using a docker env based approach,
ie: python + database + cache are setup by docker compose and the project is exposed using ports
all this i have done, but the last thing which im stuck on is pre-commit,

in the project i have various hooks at various stages (pre-commit and commit-msg),
how can i install and use the said commit without making a seprate env on host (ie : poetry install --group=dev)

pls ping me if you need extra details

[knotty basin]

[brazen forge]

you can use repo-local hooks

[warped latch]

I need some kind of solution for application monitoring, web app firewall, perhaps logging analysis. I need solutions that are ideally free. And they can't be exclusive to some cloud provider, because I am going to be on my own deploying via docker to vps. I already have Sentry set up, but that is for reporting errors.

I'm considering this https://github.com/coreruleset/modsecurity-crs-docker anyone have experience with this?

[bitter ginkgo]

Hello I am trying to host my application on aws ec2 but for some reason it is not sending me to my page

I have changed security group,

some info about my setup
server is running os 3002 port (i have opened it to internet)
I am using ubuntu and tried amazon linux

[rapid sparrow]

1. have u ensured ec2 server has assigned public IP?
2. have u ensured to change security group rule for inbound traffic?
3. have u ensured for ubuntu server to make a rule in ufw firewall or to disable it?
4. have u ensured you bind your application to 0.0.0.0 ip address inside server?

[bitter ginkgo]

have u ensured ec2 server has assigned public IP
ave u ensured to change security group rule for inbound traffic

dont know about last once can you refer me articles that you think could help me

[rapid sparrow]

3. https://linuxconfig.org/how-to-enable-disable-firewall-on-ubuntu-20-04-lts-focal-fossa-linux
4. depends on what u use for web server

[bitter ginkgo]

depends on what u use for web server
if you are talking about technology like flask , I am using expressJs or you are talking about server like apachi then I am not using anything

[rapid sparrow]

what command do u use to run your server?

[bitter ginkgo]

sudo pnpm start

[rapid sparrow]

does it have content of some config with script that is used to run your application?

to what kind of ip it tells it binds on start in logs?

[bitter ginkgo]

yeah it does let me copy paste

http://localhost:3002/

http://172.31.46.0:3002/

NX Web Development Server is listening at http://0.0.0.0:3002/

[rapid sparrow]

great. then problem 4 is not a problem

ensure uwf rule to setup or disable

[bitter ginkgo]

http://172.31.46.0:3002/

[bitter ginkgo]

okay let me read the article

[bitter ginkgo]

Status: inactive```

is that good ?

article does not say it should be active or not

[knotty basin]

that means the firewall is inactive

[knotty basin]

how will the repo-local hooks work when the env itself is in docker ?

also it needs to work with commit-msg type hook

[bitter ginkgo]

What should I do then ?

Why it is not a accessable able still

[bitter ginkgo]

@rapid sparrow thanks for support the issue : ec2 instance was low on spec had to swtich from small to medium

[gentle solstice]

Anyone know how to disable this popup when I try to connect to a host with ssh?

I prefer my terminal prompts to be in the terminal.

Also the hostname seems to be too long, so it cut off the actual question the window was asking.

[thorny shell]

you could try adding that "fingerprint" to your "known hosts" file, but

• I don't know where to find that file on Windows;
• I have a vague sense that this fix no longer works

the typical Unix command-line ssh client offers to do that for you, so that you only ever see this question once per host.

[willow pagoda]

i never knew of that popup's existence, how did you get it?

[gentle solstice]

I found it in mingw /etc/profile.d/env.sh

export SSH_ASKPASS=/mingw64/bin/git-askpass.exe

git-bash won't let me edit it

[thorny shell]

oooooooh it's asking for your password; I thought it was asking you to confirm that you're connecting to the right host

you need to do something to make your private key accessible without a password: either remove the password from it (which is obviously less secure), or see if you can run the ssh-agent

[gentle solstice]

No, the hostname was long, which was cutting off the text limit.

The prompt is fine. The problem is it's not the terminal.

[thorny shell]

well maybe try getting rid of that SSH_ASKPASS variable, then 🤷

[gentle solstice]

When I run ssh via env -u SSH_ASKPASS, it prompts in the terminal like normal.

[thorny shell]

what more do you need to know, then?

that seems conclusive

[gentle solstice]

mingw64 is weird

[thorny shell]

sure it is

Windows is weird.

[gentle solstice]

I'm not blaming this one on windows. I'm blaming it on git

[thorny shell]

And it's weirder when you put makeup and a costume on it and send it out trick-or-treating as Unix.

[gentle solstice]

The only reason I'm using git bash is because my host can't run wsl (no virtualization)

and no internet access

Maybe it was windows' fault.

If ssh does not have a terminal associated with it but DISPLAY and SSH_ASKPASS are set, it will execute the program specified by SSH_ASKPASS.

or maybe it was my terminal app

something was causing ssh to not think it was running in a terminal

[willow pagoda]

where does your ssh client come from? i just tested it in git bash and didnt get the GUI prompt, maybe because my ssh client comes from the optional features in windows's settings

[gentle solstice]

is SSH_ASKPASS set?

it was probably my terminal program.

[willow pagoda]

yup, echo $SSH_ASKPASS prints the same path that you sent earlier

[gentle solstice]

there must be a bug in my version of tabby then

[gritty aurora]

Okay, so, I have a really dumb question

How to activate a virtual environment on linux?

Internet says to execute venv/bin/activate

But there's no bin directory

Only this

A bunch of activate files can be found in the scripts folder though

Also is it possible to launch different programs in different environments simultaneously? If so, then how do I make the program to launch in a specific environment in the systemd file

[rapid sparrow]

Just specify full path to correct venv/bin/python as a launching executor

[gritty aurora]

But there's no bin directory...

[rapid sparrow]

Venv does not need to be activated in order to work

[gritty aurora]

This part is the executor?

[rapid sparrow]

Ergh. What is path of your venv folder?

[rapid sparrow]

Looks like system python, I would not dare to touch it in your shoes

[gritty aurora]

Well, that is something I found in the internet

What I want to know if this is the place to insert a path to the executor

[rapid sparrow]

Impossible. Linux python venvs all have bin folder inside

May be u don't have virtual environments then

[gritty aurora]

I see

[rapid sparrow]

So... U have smth wrong like not created venv

[gritty aurora]

This folder wasn't created on linux

[rapid sparrow]

Start with creating venv

[gritty aurora]

I just hoped I could use it instead of reinstalling stuff into a new one

[rapid sparrow]

Sounds very wrong. Venv is not allowed to be reused in majority of cases even if it was created at another Linux

It breaks when absolute paths get changed to it

[gritty aurora]

Oh, I see

No shortcuts then

Thanks

[rapid sparrow]

There is shortcut

[rapid sparrow]

If u want your application being built ready for usage without venv headache, u need to use Docker for applications
If u have web application at least in its nature

Then u just save it to docker registry and it is ready for launch everywhere at any another Linux machines with one command.

It will auto download image from registry and will get launched

Book Docker Deep Dive teaches how to use it

[gritty aurora]

Okay, I'll check it out

This time I'm just trying to set a simple bot, so I'll just install stuff into a venv manually

[rapid sparrow]

it is really smart to use Docker for even simple bots, because it ensures u have Frozen dependencies, application ready for usage
So when u make new changes, u can easy rollback to previous version, because all deps are frozen, it has everything to run even if dependencies are no longer available for installing at this point of time (of your changes were clumsy and rebuild changes dependencies to break application working when u try to return to older version)

Ability to rollback to previous version matters a lot 😅 for long term maintance of a project

[gritty aurora]

Yeah, you're correct. But I'm lazy

[rapid sparrow]

I know right. I am extremely lazy as well 😞

I need to do better. Much better if I want to reach next rank of quality in a solid way

[gritty aurora]

Indeed

Being lazy is bad, but I can't help it

[rapid sparrow]

music helps to fight laziness 😉

also good amount of food and sleeping in time

i have trouble with later one

i need to sleep in time, then i will have more pool of energy available during a day (with faster energy regenerating speed)

damn. i am a mage which is low on mana.

[gritty aurora]

Sleep...

[gritty aurora]

That message is written at 4am

[prisma yarrow]

Are there good sphinx conventions for documenting python's special methods, given that users don't actually write any dunders when using them?
For example, if a class Zoo implements __len__ this means users will want to len(zoo), not Zoo.__len__()
Yet sphinx will, by default, show Zoo.__len__() in the docs

[wild kettle]

If your CI server originally had a template to build packages with python setup.py sdist bdist_wheel or similar, have you migrated it yet to build with pyproject-build? Were there any rough spots when you did that?

[undone mural]

can somebody help me

out

in Django

[outer ore]

hi, can somebody help me?

in transform .hdr image to vector .svg image. without potrace.

[languid helm]

This channel is for devops questions.

[rapid sparrow]

🤔 i think best we can do to pray for people following the leap of logic from __len__ to len(obj)

i see no other option besides going for custom documentation without autodoc.

which is not really favourable path

always an option to mention len(obj) in docstrings of __len__ if u wish

but i think we can safely assume that people know what for dunder methods

[swift sable]

So riddle me this riddle me that. I'm building a docker container that runs a flask app with a gunicorn webserver, 4 workers. I upgraded the python version of the docker image from 3.9.16 to 3.10.12, and upgraded the gunicorn from 20 to 21. The new image ran fine on my computer but on the server it crashed with RuntimeError: can't start new thread. The old version runs fine with four workers if re deploy it. I can't see any reason in the changelogs why this would happen.

So what on earth could make my new container fail only on the server but not locally and only for the new container but not for the old? The flask app is the same, no code has been changed.

[gentle solstice]

Sadly __contains__ doesn't follow that same logic.

He also asked it in another channel and got an answer.
#type-hinting message

[whole ridge]

Question about best practices in Dockerfiles.
I'm working with someone who has a line in their Dockerfile EXPOSE ${PORT}. I know it's not gonna do anything, but since EXPOSE is for documentation purposes anyway, I kinda think it's a neat way of telling the user how to specify the port to expose. I've never seen it done that way before, I've only seen it used in Dockerfiles with a hardcoded port that's difficult to change at the environment level.
Anyway, I'm supposed to be "polishing up" their Dockerfile. Would y'all discourage that line or let it be?

[restive jasper]

EXPOSE is actually used when you use --publish-all / -P option.

Regarding deciding should EXPOSE be there or not, always remember that this option is stored inside image, and that all images inherited FROM that image will inherit EXPOSE option as well.

[whole ridge]

Oh cool! TIL

[cedar nacelle]

I want to setup a local git repo so that devs can create their own on site projects and manage it. I'm thinking of setting up redmine to do that. Any recommendations to try other than this?

https://github.com/redmine/redmine

looks like redmine.org is down 🤔

[brazen forge]

do you want to host Git repos?
hosting something like Gitea or GitLab might be what you need then

[gentle solstice]

both have docker images

[tawdry needle]

sourcehut is also self-hostable https://man.sr.ht/installation.md https://sourcehut.org/

[vapid bobcat]

Anyone here use vim? 😎

[brazen forge]

plenty people use vim
see #editors-ides

[restive jasper]

Don't ask to ask. Ask your actual question

[dusty turtle]

People familiar with PyInstaller/PyInstxtractor/Decompyle++ I have the binary of a project that was recently updated for compatability and I need some of its source code so I can merge it into a different branch of the same project that I downloaded(Not anything important or work related btw!) Essentially I'm looking for a data.py file, but it is not being spit out directly by pyinstxtractor. Advice of any sorts appreciated.

[undone moat]

It could be in the PYZ folder.

[dusty turtle]

There it is! I have no idea how I missed it.. Thanks a lot!

[heavy knot]

Hey!

So we’ll, I am making a multi tool, but idrk what I should add to it. So I need some suggestions to make!

Dm me if you have a good suggestion

Idrc if it’s hard or easy or medium to make

ill find a way to make it

:)

Or ping me here

[rapid sparrow]

U have to be at least tiny more specific what kind of tool 😉

[heavy knot]

Like, idk actually

Just anything

I now have a calculator built it

[swift sable]

I'll try this question once more before giving up since nobody answered last time.

I'm building a docker container that runs a flask app with a gunicorn webserver, 4 workers. I upgraded the python version of the docker image from 3.9.16 to 3.10.12, and upgraded gunicorn from 20 to 21. The new image ran fine on my computer but on the server it crashed with RuntimeError: can't start new thread. The old version runs fine with four workers if I redeploy it. I can't see any reason in the changelogs of python or gunicorn why this would happen.

So what on earth could make my new container fail only on the server but not locally and only for the new container but not for the old? The flask app is the same, no code has been changed.

[rapid sparrow]

😅 my often problem at work that i explored throughly

[rapid sparrow]

we deploy to AWS ECS servers with, that have c5.9xlarge instance type
Which https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/cpu-options-supported-instances-values.html according to table has
36 vCPU or 18 default CPU cores with 2 threads per core

if we have amount of threads below 900 count for this c5.9xlarge instance type per host (according to datadog metric docker.thread.count), then everything is fine and no "can't start new thread"
if we have more, we start having this problem running out of threads

[rapid sparrow]

TLDR: you are out of hardware resources. Your applications in total consume to much threads. monitoring system is required to discover precise count

to fight this problem further i made next actions

1. created Datadog alert monitor, that notifies us into Slack when some host is running out of threads

2. made sure to have at AWS proper AWS CPU reservation for a running task. Calculated that is to reserve 1 CPU per 20 threads (of metric docker.thread.count), then applications are distributed okay (they crash somewhere at 25-30+ threads)
   (via Dataodog metric explorer built quickly dashboards that allowed me quickly to find out necessary CPU reservations i should set for tasks)

P.S. as open source solution this data can be scraped by Prometheus 😉 if having docker exporter at least connected

[slim maple]

Maybe try using an async worker that won't use actual threads? https://docs.gunicorn.org/en/latest/design.html#async-workers

[rapid sparrow]

sounds like a good solution too to optimize stuff. but monitoring system to investigate a problem should be first priority
because threads could be completely exhausted due to some completely other containers ❗

when i first started to optimize it, we had normal containers that consume 10-20 threads (celery with 4 processes) or uwsgi with same amount of processes
and we had some horrible php applications where people put monitoring agents of NewRelic inside container (instead of having single agent per host). as result they consumed 120 threads per container

fixing this problem alone freed a lot of resources from hosts

[restive jasper]

Probably some quota or some other induced restrictions. Might be ulimit

[rapid sparrow]

in the end we can guess infinity times, but better be having monitoring system of docker thread counts in order to see problem exactly where it happens 😅
no monitoring = no sight.

[restive jasper]

yes, this needs troubleshooter

[swift sable]

Yeah, I guess so. I was just so mystified since the code is the same and docker version used for building the image is the same. Then why would an upgrade of the python base image suddenly use more threads? Doesns't make any sense to me.

[restive jasper]

Try to set ulimits during docker run

[swift sable]

I could but I don't think the ORM handles async stuff in it's current configuration

[slim maple]

By async you mean asyncio?

[swift sable]

Yes, or any other async framework for that matter.

[slim maple]

It's not related to asyncio though

Also depends on the orm

[rapid sparrow]

logical sense is, threads error appear potentially from just having more containers running at the same time 😉
if u do deployment via surge strategy, where more applications run at the same time during deployment

[uncut quartz]

Hi, i'm trying to do a web-scraping task in which i need to extract only the text content of any website's body, so for that i'm using scrapy and scrapy-shell, let's say for example i want to scrape the whole text content of docs.expo.dev - the home page for now, so for this to achieve, will have to target the body tag right? I'm doing the following but i'm getting all the css properties along the text, I tried to use BeautfulSoup to parse html text but didn't work:

in scrapy-shell

fetch("https://docs.expo.dev/")
body_text = response.xpath('//body//text()').getall()
full_text = ' '.join(body_text)
soup = BeautifulSoup(full_text, 'html.parser')
cleaned_text = soup.get_text()
print(cleaned_text)

Can't even see the actual text content of the web page as it is filled with css properties

Can someone help?

[lethal marsh]

Would anyone here be able to help me with deploying a dockerized LLM to AWS? Im having a networking issue and would be really greatful for the help.

[fallow smelt]

Anyone here ever used Lark as part of a vscode language server extension?

[gentle solstice]

Consider creating an issue for a feature to change the output format.

The dev will probably add it.

[south forum]

Hey anyone here?

[thorny shell]

nobody here but us zombie processes

[short peak]

memory... grrr.... memory...

[thorny shell]

bitttzzzzzz

https://tenor.com/biPQN.gif

[restive jasper]

Winnie-the-Pooh took his head out of the hole, and thought for a little, and he thought to himself, “There must be somebody there, because somebody must have said 'Nobody.

[scenic cove]

Hi!
Which LSP if more recommended? pylsp or pyright?
Thanks

[loud bear]

I use pyright and its good

Haven't used pylsp

[astral vector]

anyone know vscode extension to pop that thing when i put caret on some indent line? like the one in pycharm

[golden crag]

Please Fix To This Error

[golden crag]

@next scroll

[golden crag]

@young summit

[vague silo]

Please don't ping random people asking them for help. And please show some effort that you've tried fixing what you're asking first.

[vague silo]

pop the "for _ in range", or do you mean the lightbulb on the left?

[silent sedge]

What's the easiest way to set up a reverse TCP tunnel between two machines? Something just like ssh -R, but I'm thinking maybe there's something more reliable/lightweight than SSH?

I don't want to run a full VPN solution that would mess with system networking stuff (tun/tap etc)

[vague silo]

the most lightweight is probably socat or netcat

but then you're passing data over in plaintext

[silent sedge]

those don't tunnel

[indigo zenith]

#❓｜how-to-get-help (please don't ping me, just do what it says)

[kindred robin]

please explain github to me like baby

[indigo zenith]

First, understand git. Git is for version control. It keeps track of who made what changes to what piece of code so everything can be synced, merged, reversed if needed.

GitHub is a website that syncs with git. It helps teams share code with each other and with the wider public. It also helps you track issues, builds and much more.

The book "Beginning Git and GitHub" is pretty helpful and you can find it free on uh certain websites I won't name here

[brazen forge]

git bisect is what you want

you can git checkout a commit and run whatever code you need

git bisect is handy for finding the bad commit automatically

(automatically: by giving it a command to do the test run)

checkout is for refs

refs can be branches, tags, or plain ol' commits

[dusky olive]

Hello

[silent sedge]

I found frp and rathole: https://github.com/fatedier/frp & https://github.com/rapiz1/rathole

[tawny vortex]

true at least google it one or 2 times

[lofty laurel]

I am currently learning git flow and managing my repo and braches on github.
Since i am learning git I dont know that what sequence of git commands required to do a task like keeping branches in sync with remote repo, making commit to be updated on more than one branch etc. I was doing this by getting help from ChatGPT but sometimes it also misses things which sometime gets me into trouble. So i was looking for some resource like cheatsheet for doing some commonly done thing during git flow

[indigo zenith]

Yeah, that's what happens when you ask ChatGPT instead of actually trying to learn something 😉

Try web searches like "learn git" or "git cheatsheet" and you'll find endless resources.

To learn git to the point that you could work with a team on a complex project, you should plan to read a couple of books (also easy to find and abundant)

[rapid sparrow]

@lofty laurel

agree with dowcet

there are several excellent choices i can recommend

[rapid sparrow]

https://learngitbranching.js.org/ i recommend to play this interactive visual tutorial, it pretty much will solve your problems arleady

Consider to add into the mix Head First Git (https://www.amazon.com/Head-First-Git-Learners-Understanding/dp/1492092517 ) and/or Git Pro ( https://git-scm.com/book/en/v2 )

the first book is more easier/more friendly to learn one, the second one should be more complete to cover up to all advanced topics

[lofty laurel]

Thanks for the help and resources 🙂

[tawdry needle]

never used Head First Git but Pro Git was the primary resource that allowed me to actually get good at git, as opposed to just fumbling around trying to remember what each command did

understanding how git actually worked was incredibly enlightening

[rapid sparrow]

Head First Git appeared just last year. Looks like useful thing to recommend to novices (or less techy people) struggling with git

it is head First after all

[tawdry needle]

i'm not familiar with the head first series but honestly i wish i looked as cool as that guy

[hazy current]

why can't i am able to get develop branch into my local

[hazy current]

i am not sure whether the thing should go like that or not

[brazen forge]

git pull origin develop merges the remote develop branch into you current local branch (which is main)

git switch develop is what you're probably looking for

@hazy current ^

[hazy current]

i want to push my local branch feature/user-admin with remote/develop branch using git push so that at github side it will show me compare and pull request option.

currently it will merge with main on creating pull request

[brazen forge]

I'm not sure I understand what you want the remote-tracking branch of your local branch to be.

[hazy current]

I'm on branch feature/user-admin made some commits and want to push the changes to origin. now the origin/HEAD is on origin/main so its obvious that the pull rquest is gonna merge my local branch to main branch.
this is what my git branch -a shows:

$ git branch -a
  develop
  feature/homepage
* feature/user-admin
  main
  remotes/origin/HEAD -> origin/main
  remotes/origin/develop
  remotes/origin/main

@brazen forge

[brazen forge]

the HEAD pointer just shows the default branch at the time of cloning the repo

you can select the target branch when raising your PR

[olive thunder]

hi, could someone help me. I'm trying to make an overlay, that work even over fullscreen app such as league of legend. All of the solution on stack overflow are basically worthless and i have some kind of pist but i dont know how to make it work. The pist i have is hooking the process and drawing directly over the render of that process, is that something possible to do in python?

[alpine horizon]

not sure if the right place to ask, but which should I install for Ubuntu + AMD?

[brazen forge]

the third one

[alpine horizon]

oh is Darwin for Macos then?

[brazen forge]

yep

[gentle apex]

xd

[rapid sparrow]

linux(amd/x86-64) for regular laptops in 98%+ cases
linux(arm64) for linuxes installed at modern macbooks with M2 and etc processes

[alpine horizon]

Will save this, thanks

[remote furnace]

Hello

Idk if i can ask this here

but anyone can develop a thing for me? i pay if wan't

[rapid sparrow]

u should probably just tell what u need, and then perhaps u can be given advice or even solution 😉

[remote furnace]

I need auto create ubisoftconnect acc with auto 2fa

[rapid sparrow]

!rule 5 9 , u manage to break several rules of this server at the same time 🙂

[rancid schooner]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

9. Do not offer or ask for paid work of any kind.

[uncut latch]

Hello, I was wondering if anyone knew of a terminal based debugger better than pdb? Thanks!

[rapid sparrow]

i use vscode visual debugger that also comes with terminal console inclusion. it is very great and powerful

it can be enabled for application launch, for file launch and for each unittest/pytest launch at least

by default i utilize it from pytest usually. Sometimes using to debug scripts/files too though

[tawdry needle]

pdb++ is a drop-in extension to pdb that has some nicer features

https://pypi.org/project/pdbpp/

[uncut latch]

Hi, thanks for answering! I do most of my coding on headless servers on the cloud and can't use any fancy IDE unfortunately. Do you happen to know of a debugger particularly suited for remote terminal usage?

[tawdry needle]

there's also pudb which is a full-screen TUI type of thing https://pypi.org/project/pudb/ but i don't personally use it

[tawdry needle]

if your code editor supports the "debug adapter protocol" maybe https://github.com/microsoft/debugpy is what you need

[uncut latch]

@tawdry needle Thanks 🙏🙏

[tawdry needle]

i don't know of a general cli DAP client although it would be very useful

[uncut latch]

You're plenty useful yourself!

[tawdry needle]

hah, happy to help

[rapid sparrow]

vscode has plugins like Remote SSH to access full IDE functionality in such drastic situations

then u can use once again full visual debugger (as well as IDE in general 😅 syntax highlighting, intellisense and etc)

[uncut latch]

Thanks I will check it out

[rapid sparrow]

real question though should be... why the hell u need to code from servers in the first place

u have some very wrong infra/workflow if u need this

developing in staging/production is a sign of a really bad processes

[uncut latch]

Lol that is not the case!

[rapid sparrow]

in 99%+ cases it is 😅

[civic mural]

Hey, not sure where to put this question but does anyone know of a good alternative to OneNote that has an API?
Im trying to scrape data from an API with python then using it to keep values in a note up-to-date. I would rather not have to sign up for a sharepoint sub just to access the Graph API

[indigo zenith]

All the obvious alternatives like Google Keep, Notion etc. have APIs.

The only one I've personally worked with is Confluence. That was pretty easy to do what I needed with, but you have to pay for it.

[cerulean hollow]

Why would that be?
I'm not disagreeing, rather, I'm genuinely curious about best practices.

[uncut latch]

Hey , I have been using pudb, and wow, that software is amazing! Thanks for the recommendation

[rapid sparrow]

lets see.
A good modern setup will be having staging and production environment in addition to development environment
Deployment to staging and production will be fully infrastructure as a code and automated.
Before that happens, Pull Requests are usually code reviewed.

That makes next differences:

• Staging/Production are easily reproducable as a code. Easily destroyed and reraised from zero. All changes to infrastructure are changes to git code. They are transparent, code reviewable and git history searchable / documented. That jumps quality by magnitude
• Changes deployed are automatically going through some pipeline of workflow, where you are unit tested, integration tested, linted, code reviewed. That alone jumps quality by magnitude.

what we have without it? dirty polluted state of a production that goes into unknown direction and u have no hope to see who changes what and why (even if u are single person that does it, u will forget all the changes u made in about few months)
You becom essentially hostage of your current infrastructure. One little mess up, and you have hard to repair downtime. While in proper infrastructure u just rollback to previous state within a minute and enjoying not spending many hours to realize what went wrong under pressure of user/sales team complains

[cerulean hollow]

@rapid sparrow thanks for the insight!

[rough marlin]

Is there a decent way to require a higher Python version for development installs than for end-user installs of a package?

To my understanding, pyproject.toml has no way to do this

The context is:

1. A project I contribute to uses 3.8 as the minimum version for end user installs

2. The most recent sphinx version doesn't support 3.8, which breaks readthedocs build

3. The choice seems to be to either make a "wrong" pyproject.toml which states 3.8 despite the dev setup requiring 3.9+, or downgrade sphinx

I'm in favor of downgrading sphinx, but at least one other contributor seems to be in favor of requiring 3.9 for build despite that requiring an incorrect pyproject.toml

[tawdry needle]

you don't need sphinx for the end user env right? you can create a separate env for building the docs

hatch can do this pretty easily

likewise with tox

[rough marlin]

Can you elaborate on what you mean?

At the moment, we just have pyproject.toml.

[tawdry needle]

pyproject.toml doesn't do anything

the [project] section lets you specify a minimum python version for your project, but that's it

[rough marlin]

Yes, as I've noticed much to my annoyance.

[tawdry needle]

if you don't want to downgrade sphinx, i'm saying that you have one python env using 3.8 that's for general development, and another env that's 3.9 for running sphinx

as long as your code runs on both 3.8 and 3.9 it should be fine

[rough marlin]

I think the project maintainer wants to resolve this by requiring contributors to use 3.9+ instead of 3.8, which I disagree with since it complicates "Requires 3.8"

[tawdry needle]

unfortunately that's just how dependencies go

[rough marlin]

Yeah, I'm noticing.

[tawdry needle]

using tools like pyenv it's not that hard to have multiple versions of python in your project

[rough marlin]

I don't think we strictly need anything past 7.1.2 and sticking to it would simplify onboarding for contributors, but it's annoying to use it.

[tawdry needle]

have you considered using tox? it automates setting up environments for tasks like this

there's also a similar tool nox

yes the user does need 3.8 and 3.9 installed, but again it's nearly trivial if they have pyenv

[rough marlin]

I've seen both and I liked nox. The issue for this project is that it's supposed to be a beginner-accessible game framework

[tawdry needle]

ah, i see

[rough marlin]

More tooling means more complexity.

[tawdry needle]

then yes it might be easier to accept that on 3.8 you'll be using an older sphinx version

you don't need to hard-cap it, just let pip do its thing. it won't install a version that's not compatible

[rough marlin]

That's why I was advocating for 3.8-only requirements, but it seems others don't think it's worth it

this is a subjective issue and we don't have good information on the install base.

Imo, 3.8 is worth it since beginners will be on hand me down systems, which might run Win 7

[tawdry needle]

i agree, you want to be flexible with that

but you can roll over to 3.9 when 3.8 goes EoL https://devguide.python.org/versions/

[rough marlin]

I think that's what they have in mind.

[tawdry needle]

i'm surprised that sphinx stopped supporting 3.8

[rough marlin]

https://www.sphinx-doc.org/en/master/usage/installation.html

[tawdry needle]

yeah i see

7.2.0

[rough marlin]

That's a year out

[tawdry needle]

yep

looks like 7.2.0 is not a big release, obviously bugfixes are nice but it's going to be fine for anyone just getting started

[rough marlin]

do you have recommendations on the best way to manage multiple parallel python versions btw?

[tawdry needle]

yes, pyenv or asdf-vm

[rough marlin]

right now Im building from source since it seems easiest

i tried something called act to imitate github actions, but's slow and brittle