#tools-and-devops

1 messages · Page 3 of 1

rapid sparrow
#

firstly, you should be having CI pipeline that installs whatever you will make system in matrix and automatically checking that it is working correctly.
tools like tox/nox are meant to automate testing this stuff locally btw

#

and secondly... i think there is not really answer to simplify it. except, for whatever system you are going to use, try to make it using numbers like requirements.3.9.txt or something like that pithink

#

i mean, each python version has different requirements, sometimes even libraries are needed moved out of deps into project folder of colde, so i think it is the only choice to account everything

elder solstice
rapid sparrow
#

feel free to make some of them shared 🙂

rapid sparrow
#

may be there are already tools like that. dunno. i think it would be easier to make one xD

#

one file with requirements.ini would be having it grouped under different inis
then template rendering to requirements for different versions
and same tool would be installing it if necessary

elder solstice
#

The code is a bit messy but I managed to write a CLI script that checks the versions for an input package

unknown.png
frosty dawn
slim star
#

Hello every body. I'm not sure I'm on the right channel, if I'm not feel free to say it to me (and also that would be great if one could indicates me where to go...).
Here is the situation : I'm not a developper (I did some development some years ago, but just kind of scripting). I'm a Ubuntu user since a while and up to now I managed to compile about everything I wanted to compile on Linux). But... I'm now trying to build a software from source (Ardour) and the waf command is use for that (./waf configure and ./waf). It worked fine on linux, I managed to build and run this software but I can't manage to buid it on mac. I make the assumption that waf is a part of python and would like some assistance to build this Ardour software if possible. Up to now I (think) I passed the ./waf configure step, but the ./waf step exits with errors...

Any help would be great !!!

Thanks

elder solstice
# elder solstice The code is a bit messy but I managed to write a CLI script that checks the vers...

Made it a bit more robust, but I could still be missing something

from itertools import groupby
import json
import re
from typing import Any
from urllib.request import urlopen

from pkg_resources import Requirement, parse_version

import pandas as pd

# Dummy requirement that cannot be satisfied
BAD_REQUIREMENT = Requirement.parse('python<0,>0')

def _get_python_requirement(dist_data: dict[str, Any]) -> Requirement:
    python_version = dist_data.get('python_version')
    if python_version is not None:
        if python_version == 'source':
            requires_python = dist_data.get('requires_python')
            if requires_python is not None:
                return Requirement.parse(f'python{requires_python}')

            return BAD_REQUIREMENT
        else:
            m = re.match(r'[a-z]{2}([0-9])([0-9]*)', python_version)
            if m is None:
                return BAD_REQUIREMENT

            major, minor = m.groups()
            if minor:
                return Requirement.parse(f'python=={major}.{minor}.*')
            else:
                return Requirement.parse(f'python=={major}.*')

    return BAD_REQUIREMENT

def ls_supported_versions(package_name: str, python_versions: list[str]) -> None:
    output = pd.DataFrame(index=python_versions)

    data = json.load(urlopen(f'https://pypi.org/pypi/{package_name}/json'))
    for package_version, release_data in data['releases'].items():
        for dist_data in release_data:
            python_requirement = _get_python_requirement(dist_data)

            for python_version in python_versions:
                if python_version in python_requirement:
                    output.loc[python_version, package_version] = '✓'

    output.fillna('', inplace=True)

    # We only care about the latest patch version for each minor version
    latest_patches = [
        max(columns, key=lambda s: parse_version(s))
        for _, columns in groupby(output.columns, key=lambda s: parse_version(s).release[:2])
    ]

    formatted_output = output[sorted(latest_patches, key=lambda s: parse_version(s))] \
        .rename_axis(index='Python (↓)', columns=f'{package_name} (→)')

    print('[Supported versions]')
    print(formatted_output)
stable coral
#

Hey folks, I've found this setup in an old repo of mine:

python -m venv venv; source venv/bin/activate
pip install -r requirements.txt
python ./setup.py develop

As of 2022, do you think it is still relevant? I've also used virtualenv though venv seems specific to Python 3 (I don't need to support older Python apps). I guess I should also put requirements within setup.py.
Are there more modern/standard/efficient tooling around? I've heard about "tox", "poetry" for instance

rapid sparrow
rapid sparrow
stable coral
#

btw I hit "setup.py install is deprecated. Use build and pip and other standards-based tools" when installing so it's definitely outdated

#

with the vaguest message ^^

stable coral
rapid sparrow
#

guthub actions does exactly same things as tox, but runs at cloud servers remotely it for every commit

stable coral
#

ok I see, I don't really need that currently, more a way to manage packages

#

venv does the trick but requirements.txt is too simplistic

rapid sparrow
#

at least it works super duper fast

rapid sparrow
stable coral
rapid sparrow
#

requirements.txt

flask >= 2.0

constraints.txt

flask == 2.1
click == 2.2
wsgi === 2.2
and etc
#

you mention only root/first level deps in requirements even without versions

flask
#

constraints.txt is not installed, it is just locking which versions are needed used for root and secondary deps

rapid sparrow
#

requirements.dev.txt for example has only first level deps for your dev env

pytest
rapid sparrow
#

it sort of works like augmented venv

#

but locks not just python version and its packages

#

docker locks all necesary binary applciations neede to be installed to linux OS in its dockerfile

stable coral
#

yeah this pattern is great but just time consuming to create the Docker image, relative to a few pip commands

#

that's cool in a team context though

#

ok so I think I'll be ok with venv and well defined requirements + constraints, regarding to package installs

#

and about "setup.py install", the deprecation message doesn't help much

#

how can I bundle a binary based on an entrypoint?

#

on Python 3.10 installed with asdf it doesn't work anymore it seems

#

or maybe my binary ends up in a path that is not in PATH yet

rapid sparrow
# stable coral yeah this pattern is great but just time consuming to create the Docker image, r...

long?! probably u just not having sufficient experience in it. it is brain dead easy with python docker hub images

FROM python:3.10.5-slim-buster as base

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

WORKDIR /install

RUN apt update && apt install -y \ 
    git \
    bash-completion \
    && rm -rf /var/lib/apt/lists/*

COPY ./requirements.txt ./constraints.txt ./
RUN pip install -r requirements.txt -c constraints.txt

WORKDIR /code
COPY . /code
#

any time used on docker is compensated with brain dead easy setups later, and having all installations as a code documented

#

i have commands further simplified with docker-compose and makefile made in python though

stable coral
#

" && rm -rf /var/lib/apt/lists/*" this lines is the kind that bothers me lol

#

but thanks for sharing, the great thing is that once done it will always work indeed

rapid sparrow
rapid sparrow
wooden ibex
rapid sparrow
#

i would mention i don't use already COPY . . though. since of not reliability to use .dockerignore as blacklist to ensure nothing leaks into image
I use of course stuff like COPY src /code, to ensure whitelist copying

wooden ibex
#

ENV format is wrong

#

you are pinning to exact version so bad testing

#

your install git and bash-completion for some reason

#

let it write bytecode, who cares

stable coral
#

it seems that I should replace setup.py by "python -m build" + a pyproject.toml

#

anyway build doesn't work with my old setup.py, and setup.py doesn't work either...

#

forced upgrade

#

but initalizing the pyproject.toml leads me back to poetry

rapid sparrow
#

they are installed to augment my dev environemt

#

visual studio connected to container with installed git

#

has working correctly git modules

#

i develop from within containers with cli or vscode connected to them

tawdry needle
tawdry needle
# stable coral I've found that also: https://stackoverflow.com/questions/62983756/what-is-pypro...

the community is attempting to finally converge on some new standards. this isn't a matter of chasing the shiny and new; the status quo was horrible for the better part of a decade, and despite being a major upgrade over what came before, was still quite nasty and subpar compared to other language tooling. the current churn should be short lived, as the standards are stabilizing and tooling catches up with the standards.

pyproject.toml aims to be a standard place to specify your project/library metadata, like name, dependencies, version, etc. it also aims to be a central location for dev tools (e.g. flake8, mypy) to look for static config information. the intention is that you should be able to replace setup.cfg, mypy.ini, and tox.ini with a single pyproject.toml file.

however, the most important part of the pyproject.toml file is the part where you specify the "build backend" and any dependencies that are needed to build the package. examples of build backends are setuptools, poetry, flit, and hatch.

the idea is that when you run pip install on a package with a pyproject.toml file, pip will read the file to determine which build backend to use, and then hand off the build process to that backend. the backend is then free to do whatever it needs to do in order to build your stuff as a binary archive. setuptools for example will still look in setup.py and run its contents for doing things like building C extensions, but that's only after the build process has been handed off.

#

i should write this on stackoverflow...

#

setuptools specifically intends to deprecate their python ./setup.py interface, so yes, you should eventually plan to change how you build setuptools projects, and the build frontend is probably the best choice for doing so. however this does not mean that you have to throw out your setup.py file, it only means that you need to add a pyproject.toml which declares that your project uses setuptools as the build backend. you can leave everything in your old setup.py if you want.

#

as far as i know, build should work with any pep 517 build backend, so you should be able to use build for any project: poetry, setuptools, etc.

stable coral
#

Much thanks for the details answer, that's indeed worth a SO answer!

#

ok I get the point

#

"python -m build" doesn't run in the same context/folder/whatever so it cannot find "requirements.txt"

#

I don't know yet if its due to Python 3.10, how build work, how python -m work, or my install of Python with asdf

#

but after a bit of struggling I could at least build my CLI tool again with "build" \o/

tawdry needle
#

this doesn't answer your question, but consider heeding the advice in that SO answer f not using requirements.txt as the source of data for install_requires=

stable coral
#

"the community is attempting to finally converge on some new standards. this isn't a matter of chasing the shiny and new; the status quo was horrible for the better part of a decade" haha hello from the JS ecosystem

stable coral
#

so I am not sure how I can imposes versions to the "build" venv without that

#

somehow it kept loading an older version that I probably accidentally installed outside of venv

#

I thought it would have loaded the requirements.txt automatically but no

tawdry needle
#

show your setup.py and pyproject.toml as well as any commands you ran + errors you got

#

!paste

rancid schoonerBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

stable coral
#

indeed so I didn't get how to pick the right packages, if I cannot pass it requirements.txt, that means a lot of duplication

tawdry needle
#

(also your requirements.txt for good measure)

stable coral
#

you'll notice some struggling with packages

tawdry needle
#

much like package.json vs. package.lock

stable coral
#

ok so using the setup.py, pyproject.toml as the source of truth

tawdry needle
#

right. but here's a question: is this an "app" or a "library"? and how do you intend to distribute it?

stable coral
#

it's a binary

#

a CLI tool, it takes my Google Agenda export and generates a CSV with my work per day per client

#

(so sorry it's not clean enough to share, it has my client names everywhere, but you could write that in 2 hours if you are a skilled Python dev)

tawdry needle
#

okay. consider that pinning exact dependency versions means that it can never really be installed outside of a venv, or outside of some other encapsulation/dep-bundling tool like pex

stable coral
#

I have a second app that is more interesting, it's an Electron soft with a Python backend, I am mainly preparing to improve this one

#

hence the need to upgrade my knowledge

tawdry needle
#

well it's probably for the best to install an app into its own env anyway

#

brb a minute

stable coral
#

thank for your help

tawdry needle
#

my guess as to why it wasn't finding requirements.txt: the requirements file wasn't included in the package data files, so it wasn't being added to the ephemeral build environment

#

but regarding general recommendations. if you assume that it's installed into some isolated environment (like a venv) that won't conflict with other installed packages, then go ahead and pin exact package versions in pyproject.toml or setup.py

#

and then if you want to be extra-safe/pedantic about transitive dependencies, you can use pip-compile from the pip-tools project to walk through all of your deps and generate a requirements.txt that acts as a lockfile

#

!pypi pip-tools

rancid schoonerBOT
tawdry needle
#

you can also generate a "constraints" file which is very useful if you have different sets of deps (e.g. one for testing, one for runtime) but that's a separate thing kind of

stable coral
#

package_data={"": ["requirements.txt"]},
this doesn't work, because it it's in the setup config, but you load the file in setup.p

stable coral
tawdry needle
stable coral
tawdry needle
#

i have something like in the makefile my current project at work:

requirements-run.txt: pyproject.toml
    venv/bin/pip-compile --resolver=backtracking --generate-hashes --reuse-hashes --extra=run --output-file=$@ $<
    @printf '$@ regenerated. Make sure to add it in Git.\n'

requirements-test.txt: pyproject.toml
    venv/bin/pip-compile --resolver=backtracking --generate-hashes --reuse-hashes --extra=test --output-file=$@ $<
    @printf '$@ regenerated. Make sure to add it in Git.\n'

# See https://pip.pypa.io/en/stable/user_guide/#constraints-files
constraints.txt: pyproject.toml requirements-dev.txt
    venv/bin/pip-compile --resolver=backtracking --strip-extras --extra=run --extra=tests --output-file=$@ $^
    @printf '$@ regenerated. Make sure to add it in Git.\n'
#

i was calling them .lock instead of .txt but i changed it back to .txt to avoid freaking people out too much 😆

#

i think .lock makes a lot more sense, but inertia is strong among programmers (understandably)

stable coral
#

sounds perfect I am going to try that asap!

#

I am probably going to look for cross-platform patterns as well

tawdry needle
#

oh and requirements-dev.txt is where i list pip-tools, mypy, and a handful of other dev tools

stable coral
#

I recall having issues with windows only packages used for debug that don't install at all on linux

tawdry needle
#

you could use snakemake and/or python scripts (with subprocess), i just happen to be comfortable with make (and a lot of people have access to it through wsl or cygwin or msys anyway)

#

oh, when it comes to windows-specific deps i know nothing

stable coral
#

I've been testing "just" lately, basically make but just to run commands

#

I've seen that in some Rust projects

#

I am tired of bloated package.json scripts so now I use that in JS projects

tawdry needle
#

i saw just too but it didn't seem like enough of a make upgrade, nor did it appear to support actual files as targets

stable coral
#

yeah it's more suited to accomodate with JS dev, which will be scared by make... and just need a way to put scripts outside of messy package.json

tawdry needle
#

i have a mix of "tasks" that i want to run and "targets" that i want to create. make is better at the latter but support the former, whereas afaict just only does the former.

#

yeah if you don't need to create any "targets" then go for it w/ just

stable coral
#

exactly, in JS we don't use the npm scripts for files anyway hence the good fit

#

I hope this will become the standard

#

yarn/pnpm are helping because they don't support the same set of features so package.json are becoming huge messes with perfectly incompatible scripts

stable coral
tawdry needle
#

why don't people write scripts/foo.js and just invoke them from package.json? i've seen this before, people writing big commands right there in a json string. and i don't understand it

tawdry needle
stable coral
#

writing scripts in JS is a mess, google/zx improved it but not enough

#

so people mix bash one-liners, JS scripts, binaries...

tawdry needle
#

i saw zx, looked pretty cool!

stable coral
#

!paste

rancid schoonerBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

tawdry needle
stable coral
#

this is my package.json scripts lol

tawdry needle
#

holy shit

stable coral
#

also you had a "pre"/"post" convention in NPM that is not valid anymore in Yarn 2+

#

this doesn't happen with Just

#

same applies for global config variables, doesn't work

#

many scripts are still one-liners so you can hardly justify creating a whole file for them

tawdry needle
# 
    "prebuild": "(.vn/scripts/is-monorepo.js && rm -Rf ./node_modules/@vulcanjs && mkdir -p ./node_modules/@vulcanjs && cp -R ../../packages/* ./node_modules/@vulcanjs) || echo 'Not in the context of a monorepo, ok'",

i would create a separate script for this 100% every time!

stable coral
#

so yeah Make and Just are probably going to be more prevalent in JS as projects go more mature

#

ok true for this one lol

#

and a few others

rapid sparrow
tawdry needle
stable coral
tawdry needle
rapid sparrow
#

Discovered today I can link multiple argparse interfaces without any imports and without breaking --help command. I can link with subprocess.run sub commands, not just with subparsers originally available there.
It allows me to have dynamically installed plugins to my makefile
And using commands without having all imports imported which would break stuff in environment where deps aren't installed yet

stable coral
#

have to go, much thanks for helping me update my Python-foo (among all, Discord communities are imo the best thing that happened to many languages !)

rapid sparrow
rapid sparrow
autumn hull
#

I've been a Python programmer for several years now. But I've barely used many of the tools available (e.g. setuptools, pytest, flake8, mkdocs, pre-commit, CI/CD platforms) . I want to take my projects to this next level but find it difficult to learn all these different technologies and how I can use them all together easily. Does anyone have a recommendation? Should I just learn one at a time? Or are there good tutorials you know of to learn all these kinds of tools? Thanks in advance!

tawdry needle
full bronze
#

I'd agree with salt. I learnt a lot of these individually and then adding to different projects gives you a good understanding

autumn hull
#

Thanks!

rapid sparrow
mystic void
#

That's my whole point: the article is wrong. And that makes their arguments and reasoning invalid as they rely on false assumptions

alpine horizon
#

how do I replace the master branch with the second one on Git

#

I want to delete Master, and rename Test-iloc to: Master

alpine horizon
#

Hello

#

anybody knows how to solve this

thorny shell
#

well, deleting a branch is git branch -d Master

#

obviously, make sure your commits are reachable before you do that

#

if you don't know what I mean by "reachable", then I suggest you back the whole repo up someplace safe first

rancid schoonerBOT
#

Hey @thorny shell!

It looks like you tried to attach file type(s) that we do not allow (.sh). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a, .csv, .json.

Feel free to ask in #community-meta if you think this is a mistake.

thorny shell
#

This demonstrates renaming

#!/bin/sh

set -x

repos=$(mktemp -d -t git-repoXXXXXX)

trap "rm -rf $repos" EXIT

# Master branch -- create it

cd $repos
git init
git branch -m master Master
cat<<EOF  > TheFile
I have some lines.
Two, in fact.
EOF

git add TheFile
git commit -m "Create a file"

# Alternate branch -- edit it

git checkout -b Test-iloc

cat<<EOF  >> TheFile
Well, OK; maybe three.
EOF

git commit -a -m "modify it"

# Master branch -- delete it

git branch -d Master
git branch -m Test-iloc Master

ls -l
git log -a
git branch -av
sleep 20
long surge
#

How do I configure pylint? When I add the --generate-toml-config flag, it just spews out a bunch of text and I'm not sure what to do with it.

thorny shell
#

I assume you redirect that to a file

long surge
brazen forge
thorny shell
#

call it whatever you want. But presumably pylint will look for a particular file name, so it's most useful to figure out what name it looks for, and name it that

#

or what Moyen said 🙂

brazen forge
#

why I'm saying pyproject.toml is because the config has keys like tool.pylint.something

long surge
#

so this file already exists?

brazen forge
#

if it doesn't make one.

thorny shell
#

it might, it might not

#

lots of different tools read that file

long surge
#

and I put this file in my project directory?

thorny shell
#

so you might have already created it for one of those other tools

brazen forge
#

the >> means redirect to whatever file, but append instead of write.

#

unless you're using cmd or powershell, in which case idk 🥴

long surge
#

is pylint really poorly documented

brazen forge
#

why do you think so? pithink

brazen forge
long surge
#

I can't find anything pertaining to configuration and the website looks confusing 😭

thorny shell
long surge
brazen forge
brazen forge
buoyant trail
#

Hi !

#

I'm developing a virtual assistant, and AI shall we say that,, anyone want to give some ideas?

#

or participate in the making?

#

check out

long surge
buoyant trail
#

cant send that link*

#

pm me !

#

Ill be waiting

#

ideas to do

#

got ~550 lines of code already

#

I want more

#

like Garra - Quero mais

brazen forge
echo cypress
#

hello. How can I split log msgs between stdout and stderr with structlog?

smoky compass
#

Is it possible to have a poetry-managed package that has a setup.cfg without needing duplication between the [metadata] section of setup.cfg and the [tool.poetry] section of pyproject.toml?

#

Perhaps scrap the cfg file altogether and use internal referencing within a project section in the toml?

peak iris
#

miniconda: is this something i should use as a linux user? or am i better off using virtualenv or something like that?

thorny shell
#

I wouldn't use anaconda or miniconda unless you know you need to use them. They sound complicated.

peak iris
#

i'm following instructions for setting up someone else's packages (e.g. some pytorch model) and there are recommendations to use anaconda to manage it, i'm wondering if those are really geared towards someone on windows who is missing good tooling support

#

when i last seriously touched python, maybe 5-6 years ago, the recommendation was some long song and dance about running everything in its own virtualenv that i think is now outdated

thorny shell
#

unfortunately I don't really know anything

flat lotus
#

Anaconda simply combines a virtualenv and a few pre-installed packages

rapid sparrow
#

there is some common trend, to install datascience/machine learning stuff from conda

grand ibex
#

Hi
I am developing a simplified regex tool which will be easier to use regex for popular things like email, phone, date, extration, searching...
I am developing a Python package first time any suggestion will be appreciated also if interested in contribution.
Thanks

rapid sparrow
#

the most crazy people abandon docker-compose in favour of running in kubernetes locally (already tools appear for this like dev spaces / tilt, with different easy to run locally clusters)

#

IDEs have ability to develop from within containers with hot realoading 🙂

#

a way cooler than venv, since we can have as a code OS level binary deps, or side car containers with databases.

wooden ibex
rapid sparrow
#

one minute to resolve deps at the start of a project?!? for first dependency?!

#

common xD

graceful light
#

So I have multiple small setup.py files for cythonizing different individual cython files
Normally I used a windows powershell script (or a bash script for linux) to compile each of these individual files
I want to instead make a python script to handle this stuff so it would be cross-platform, instead of maintaining individual compiler scripts

Reason I mention all that is I was wondering if would be possible to do something like this:
Individual setup.py```py
from Cython.Build import cythonize
from setuptools import Extension, setup

extensions = [Extension("Name.Of.Library", ["src/Name/Of/Library.pyx"])]

setup(
ext_modules=cythonize(
extensions,
build_dir="build",
language_level=3,
),
options={"build": {"build_lib": "src/"}},
)
```and I want to be able to essentially define that into a function, import it, and run it from my script instead of calling subprocess.Popen to handle running python /src/Name/Of/File.pyx build_ext
I'm just not sure how to basically run the function with build_ext

regal creek
#

My docker container is reporting 16GB of memory usage, but the memory usage on the host is only 12GB how can this be?

gentle solstice
#

dedupe

regal creek
#

hm?

gentle solstice
#

memory deduplication.

#

basically shared memory

#

things like the kernel are shared across all containers

wooden ibex
#

Also, that’s a ton of memory

regal creek
#

ok, but how does my container have more ram in use than the host has in use?

gentle solstice
#

the container doesn't see the deduped memory. It just sees it as memory. The host sees all.

#

It sees the memory for what it truly is(n't)

#

memory compression could also be in effect

wheat crow
#

does anyone know a programming software that allows you to highlight a bit of code and collapse it into a small title which you can click on to expand?

short peak
wheat crow
#

visual studio 😅

gentle solstice
#

visual studio code

#

pycharm also supports this

wheat crow
#

im using atom because i couldnt get pycharm to work for some reason

#

i'll just use visual studio ig

short peak
#

I guess even vim supports it with some magic plugin or something

gentle solstice
short peak
#

I think atom got abandoned

wheat crow
#

perhaps

#

i just like its simplicity

#

but ig it's too simple

short peak
gentle solstice
#

You've got 4 months left

graceful light
#

VS Code does everything Atom does and more
I also used to use Atom but just accepted that once MS bought GitHib that Atom would be deprecated some day soon

gentle solstice
#

at least it's not sublime

umbral jay
#

Anyone having problems with conda? it seems to be taking a long time to install.

heavy knot
#

Does any1 here know how i can add a cc gen to my python script and a validator?

uncut crater
#

hi guys, anyone experience with monitoring facebook and instagram? I am searching for partners and would love to get some input on how to achieve this.

lusty sky
#

assert cipher == [1, 1], f"cipher={cipher} (should be [1,1])" :I get "SyntaxError: invalid syntax" But only on macos not on windows or linux. How to fix it on macos?

#

ah I have set up python 2 instead of python3 in the github pipeline

#

I think it is just that 🙂

random lagoon
#

is there a way I can check if someone downloaded a file in my system using python?

gentle solstice
#

What do you mean by downloaded?

frozen anchor
#

Is there any website that gives an overview over commonly used technologies?
I am not talking about the "main" technologies like NodeJS for a MERN-Stack. I am talking about "additional" tech like:
Docker, Kubernetes, Terraform, Jenkins, Sentry; testing libraries; SQL abstractions (ORM, object relational mapping); also stuff like the usage of .env files or "keep a changelog".

Like - I know of the mentioned techs, but have just recently learned about the existance of Jenkins. And I am sure there are many other technologies / tools that are used in production but that I know nothing of.

Would also be great to see an explanation of a real world project that uses some of the mentioned technologies, why they decided to use them and how they are used.

gentle solstice
#

so ci/cd

heavy knot
# 
  -e, --editable <path/url>   Install a project in editable mode (i.e.
                              setuptools "develop mode") from a local project
                              path or a VCS url.

pip install -e . does this mean my pip install updates itself when I change the code?

loud bear
#

can anyone help me setup constraints on package_data based on operating system in a setup.cfg based package?

thorny shell
heavy knot
#

im going in

dawn hill
#

Can you add a single directory (which is a package) to the PYTHONPATH without added all other directories in the same parent directory?

tawdry needle
narrow slate
#

I am currently looking to implement terraform ci/CD and can't decide whether to use Atlantis or builtin gitlab-ci. Does anyone have feedbacks between these 2 solutions?

rapid sparrow
#

you should try to choose GitOps first

#

you would like to see for each commit verification, if deployment became succesful or not and all tests ran succesful, you would like to see ability to rollback yourself to previous succesful deployment (and to know which was succesful), you would like to compare git history between succesful to newely broken states, what was changed

#

Atlantis is convinient of course... but it can be auxilary system only in my opinion. Just a tool to provide additional access to developers, so that they would own infrastructure more

lament ledge
#

Vscode auto complete for .so modules?
Ok maybe I just don't get it how to ever ask this.

So I've been working on creating a python module with boost python. It works have a makefile that makes me a hello.so that I'm able to call functions and make classes from, yata yata. But in vscode there is no auto complete for my .so module.

The thing is I know the doc string for my .so exist because in python interactive shell I can view it.

$python3

import hello

help(hello)

// this will show me the doc string for the module...

But vscode can not give me a auto complete. I've tried pylance, pylint. But nothing I do in my settings.json will give me auto complete for the module.

I've read something about a python stubs or .pyi files and they are supposed to give auto complete for .so modules.

Has anyone been able to get autocomplete in vscode to work for .so modules? How did you do it?

Any advice would be greatly appreciated.

And if you are lost about whatever it is I'm talking about just ask for clarification. I may have to go and ask stack overflow but sometimes they are rude so I'm really hoping someone will understand what I'm asking.

narrow slate
# rapid sparrow Atlantis is ChatOps, Gitlab is GitOps

Well I think that these chatops things is not really a thing. It's just that you type a comment instead of hitting a merge button to apply. I don't really care about that difference. In any way the state would never be stored in the git repo so I don't really get your point

brazen forge
rapid sparrow
narrow slate
#

not really sure if I agree with this pov but I can understand it

rapid sparrow
#

in ChatOps you don't have a history of verified commits that passed tests/deployment, you don't have advantage of seeing git difference between succesful and now broken state

#

you can try implement it being present though

narrow slate
#

well only the plan / apply is handled through comments. everything else is in the repo and the said plan in CI logs

rapid sparrow
#

only then you would see when your infra code is succesful and when not

#

it gives advantages to rollback to previous succesful state, to previous succesful commit

#

seeing difference what changed for infra code as well

#

instead of trying to provide 99.999999% quality, invest little effort into giving yourself easy ability to rollback back 🙂 and fixing in whatever time is necessary.

rugged hare
gentle solstice
#

I think you just imported a repository

#

github no longer allows creation of mirrors

#

It used to be a thing you could ask staff to do and convert an existing repo into a mirror.

#

The new solution (if you have control over the remote) is to configure a post receive hook to push to github

rugged hare
loud bear
#

It makes perfect sense to have a repo on some source control other than github and then have mirrors for it.

rapid sparrow
loud bear
#

sphinx.autodoc is quite vast

#

I can't imagine how much functionality I missed using mkdocstrings

#

I want to embed images in my docstrings. The problem is that the images are stored in docs/img, is it possible to cross reference the image URL relatively in the docstring?

#

Actually I want VSCode to show them. But using relative paths doesn't work, it needs to be a web URL. Should I instead put the link of the image in my repo inside the docstring?

heavy knot
#

Do anyone know how to delete a virtual environment?

thorny shell
#

just delete the folder it lives in

#

nothing special needed

heavy knot
#

Do u know the command in commandprompth?

loud bear
viscid silo
#

Hey guys I'm using the Notification capability in the Plyer module in Python, is there a way to run a certain piece of code when you click or interact with the Notification.

sturdy tide
#

Anyway to determine payload size in postman?

tawdry needle
sturdy tide
tawdry needle
sturdy tide
tawdry needle
sturdy tide
true vapor
#

Is there any python package manager that can work nicely without it being installed at the system or user level?

I want to be able to install the package manager in a venv, then just run pkgmanager install? Reason being that I want lockfiles, or at least something better than pip freeze > requirements.txt

waxen gull
#

!pypi pipenv is my go to

rancid schoonerBOT
waxen gull
#

Can't live without it

true vapor
#

All installation instructions require pip install --user. I want something that can work inside a venv that I make

waxen gull
#

Yeah after you do pipenv shell, you can then do pip install

#

Don't need the --user as its already in a venv

true vapor
#

Interesting. These package managers work better inside virtual envs than I expected

tawdry needle
tawdry needle
#

and actually pdm might use pip internally too.. let me check

#

yeah nope it still uses pip internally

true vapor
tawdry needle
#

i like that it doesn't "take over" your project like poetry

#

it just manages packages

#

also pep 582 is cool

#

!pep 582

rancid schoonerBOT
#
**PEP 582 - Python local packages directory**

Link

Status

Draft

Python-Version

3.12

Created

16-May-2018

Type

Standards Track

true vapor
#

Agree entirely about both things. Never liked poetry, but pdm seems great so far

polar notch
#

anyone have some good suggestions on a free, sqllite modelling tool, I kinda liked Moon modeller, but it doesnt support sqllite in the free version

primal swallow
#

Hello! How i can stop someone to text here?

unknown.png
tawdry needle
#

unfortunately it doesn't support async natively yet, but with anyio it's not that hard to run queries in background threads

#

(the built-in sqlite3 module doesn't support async anyway)

gentle solstice
heavy knot
#

Hi, suppose I have a feature branch already pushed to remote repo. I need to rebase it to master remote branch. What is the cleanest way to do this?

When I try to rebase, git will always suggest git pull after and the commits look duplicated…

inner pollen
#

Checkout to feature and do git rebase master

heavy knot
gentle solstice
# 
git fetch -A
git rebase -i origin/master
#

that will open your editor to let you pick, reorder, drop, squash, edit, or fixup commits.

#

the more you reorder or fixup, the more conflicts you'll have to fix later.

chilly swallow
#

Please I need help with django, I’m try to send an email with mailjet with django but it returning error “connection unexpectedly closed “

heavy knot
#

@gentle solstice thank you!

marsh folio
#

is sphinx still the go-to documentation generator? looking for something to standardize on at work. low friction is a must, but i also want to avoid wasting time on some new fad that won't stick around

thorny shell
#

I'd say "plain text" is the go-to documentation generator, but that's just me

#

I haven't played much with sphinx, but I doubt anyone would call it "low friction"

#

it seems complex

marsh folio
#

nah i need at least auto-generation from docstrings

thorny shell
#

I'd use doctest for that tbh

#

then at least you'll know your documentation is correct.

marsh folio
#

and markdown output so the repo's README.md is the entry point

thorny shell
#

in my experience, most documentation is wrong.

#

yeah markdown is fine

marsh folio
#

doctest is kind of orthogonal to this, but a very good suggestion

thorny shell
#

I guarantee you, if you spend a bunch of time on fancy doc-generation stuff, your docs will suck anyway.

#

they will not be maintained, and may wind up worse than useless.

marsh folio
#

yeah, i know. i'm trying not to spend a bunch of time. set up the pre-commit hook once and have it generated from source

thorny shell
#

doctest, man

marsh folio
#

and it's only for library code, not our whole codebase.

thorny shell
#

nothing else (that I know of) has a chance of being maintained

#

of course I'm assuming that you'll actually run the doctests on every build 🙂

marsh folio
#

pre-commit, yeah

#

i'll take a solid look at whether doctest can be the whole solution, thanks

#

the strike against it (for our use, not a flaw) is that we have typescript repos, too, and if one tool can handle both, it's a huge plus

#

having a consistent set of standards, output format, build process, etc. across languages is a huge advantage if we can get it

rapid sparrow
stable pewter
#

Hello, does anyone know what python module to use for physics units?

I want to work with a module that:
-has a large number of pre-existing units
-has an easy way to create a new unit
-has an easy way to convert a unit to another

I tried scimath (don't know how to create a new unit) and unum (not that big database so a lot of time to create new units) does anyone know of any other modules that work for them ? (or help me with creating new units in scimath if you know how)

thorny shell
marsh folio
#

a separate but related question: is there an automated way to do a "regression test" on the public interface of a package? i don't want someone (coughcough me) to innocently improve the name of a function parameter and break a caller in a different repo who passes it by keyword

thorny shell
#

wouldn't boring old unit tests catch that?

stable pewter
marsh folio
#

sure, if i know how to write a unit test that can catch any change to the interface and remember to write one for every method/function

#

and if i use the IDE's "refactor" function to do the ill-advised name change, the unit test will get helpfully refactored, too 🙃

thorny shell
thorny shell
marsh folio
#

alright

#

"don't do that" is, historically, not so reliable

thorny shell
#

seriously?

#

if that's a real risk, maybe move the tests to a separate project

#

I assume most IDEs will confine their refactoring to a single project

marsh folio
#

move the tests to a separate project? oof

thorny shell
#

yep

marsh folio
#

i want my coworker who handles devops to still be my friend

thorny shell
#

it's an unusual step, but you seem to have an unusual problem

marsh folio
#

thanks for your help @thorny shell, i'll consider the advice

tawdry needle
subtle swift
#

If anyone who can code really well in Python please dm me to help me create a system for my school’s Canteen!

loud bear
wary oasis
#

Trying to run a simple fastapi app using uvicorn and docker. When I use txt uvicorn app.main:app --host 0.0.0.0 --port 8000 The server runs and I can get a response at the address. Doing the following in docker seems to not be working as the address is showing txt Firefox can’t establish a connection to the server at 0.0.0.0:8000. bash (env-jarrodh-dev) jarrod@jubuntu:/media/jarrod/D/Projects/jarrodhdev$ docker build -t myimage . Sending build context to Docker daemon 9.216kB Step 1/6 : FROM python:3.10.7 ---> e285995a3494 Step 2/6 : WORKDIR /code ---> Running in a4b4c729a524 Removing intermediate container a4b4c729a524 ---> d951c1ee0196 Step 3/6 : COPY ./requirements.txt /code/requirements.txt ---> 3ea538106cca Step 4/6 : RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt ---> Running in 3f06215d9f68 Collecting anyio==3.6.1 --SNIP PIP INSTALLS-- --END SNIP-- Removing intermediate container 3f06215d9f68 ---> 17ea74b9f562 Step 5/6 : COPY ./app /code/app ---> 1b67559a1a45 Step 6/6 : CMD ["uvicorn", "app.main:app", , "--proxy-headers", "--host", "0.0.0.0", "--port", "8000"] ---> Running in 98ee463d8ab4 Removing intermediate container 98ee463d8ab4 ---> 76c30d9a2453 Successfully built 76c30d9a2453 Successfully tagged myimage:latest (env-jarrodh-dev) jarrod@jubuntu:/media/jarrod/D/Projects/jarrodhdev$ docker run -d --name mycontainer -p 8000:8000 myimage f17c18e2f0485dbad623ae93a93a29eac5103a80495d6d8aa80dbfb175312c64 I can build the image and run the container, but it just doesnt seem like the server is running... Here is my Dockerfile ```docker
FROM python:3.10.7

WORKDIR /code

COPY ./requirements.txt /code/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

COPY ./app /code/app

CMD ["uvicorn", "app.main:app", , "--proxy-headers", "--host", "0.0.0.0", "--port", "8000"]

unknown.png
tawdry needle
# wary oasis Trying to run a simple fastapi app using uvicorn and docker. When I use ```txt u...

are you typing http://0.0.0.0:8000 into your browser? try http://localhost:8000 instead, or http://127.0.0.1:8000 which is what localhost usually resolves to. 0.0.0.0 is not a real ip address that you can access. see https://en.wikipedia.org/wiki/0.0.0.0

0.0.0.0

In the Internet Protocol Version 4, the address 0.0.0.0 is a non-routable meta-address used to designate an invalid, unknown or non-applicable target. This address is assigned specific meanings in a number of contexts, such as on clients or on servers.

wary oasis
#

Still cannot connect. When I use 0.0.0.0 without Docker, it does work. But I tried localhost:8000 and got nothing

tawdry needle
wary oasis
#

still "Unable to connect"

tawdry needle
#

you can also use curl to make sure it works from the command line. curl 'http://localhost:8000'

tawdry needle
#

try also adding EXPOSE 8000 in the dockerfile, and run with docker run -P

wary oasis
#

I'm not getting anything from uvicorn in the console maybe because I ran the container headless?

tawdry needle
#

is uvicorn actually in your PATH in the container?

wary oasis
# 
FROM python:3.10.7

WORKDIR /code

COPY ./requirements.txt /code/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

COPY ./app /code/app

CMD ["uvicorn", "app.main:app", , "--proxy-headers", "--host", "0.0.0.0", "--port", "8000"]
tawdry needle
#

oh that's it

wary oasis
#

How would I add that

#

I'm followign the tutorial on the FastAPI docs, but it didnt mention that

tawdry needle
#

idk, i never had this issue myself and your dockerfile looks fine

#

just running through ideas

#

when you run uvicorn on the console, you should see nice logging output, at least indicating app startup

short peak
#

try inspecting container IP and use that instead of localhost

tawdry needle
#

do you see none of that when you run the container? you should

wary oasis
#

Ok adding EXPOSE 8000/tcp and trying again

short peak
#

also, ensure port is exposed when you do docker ps

tawdry needle
short peak
#

I've never used EXPOSE statement before

wary oasis
#

Ok I ran the container without headless mode and got this output, probably means I need to edit PATH in dockerfile? ```bash
(env-jarrodh-dev) jarrod@jubuntu:/media/jarrod/D/Projects/jarrodhdev$ docker run --name mycontainer -p 8000:8000 myimage
/bin/sh: 1: [uvicorn,: not found

tawdry needle
short peak
tawdry needle
#

it's looking for the program [uvicorn,, as if that CMD command was a single string being passed to /bin/sh

#

...what the heck

#

so to answer your question: it doesn't work because uvicorn never even starts

#

i also didn't realize what you meant by "headless" mode, you are using -d which yes will prevent logs from being shown on the console

wary oasis
#

oh wait

tawdry needle
#

you need to look in docker logs for that

wary oasis
#

that'll do it

tawdry needle
#

if that's the problem, this is a bug in docker's input validation and error handling

#

but good catch

wary oasis
#

Ok building and running again to check

#

Thank you all 🙂

unknown.png
#

silly mistake

tawdry needle
#

...that's a docker bug imo

#

the empty argument caused it to parse the whole input incorrectly as a string. wat??

short peak
#

yeah, it should complain on building

#

though CMD statement is used on container creation, no?

wary oasis
tawdry needle
#

man, docker doesn't even have a public bug tracker from what i can tell

#

i am definitely going to switch to podman as soon as they have good buildkit support

wary oasis
#

Does it have a github for creating issues?

#

docker

#

Oh yeah theyre on github.

#

Does seem odd that such a big entity doesnt have a simple bug rreport feature

tawdry needle
#

not surprising imo, it's a for-profit company that happens to run several related open-source projects

#

the problem is that it's hard to know which component of docker to actually file the bug report with

wary oasis
gentle solstice
#

Add a space after [

wary oasis
# gentle solstice Add a space after [ 
docker run --name mycontainer myimage
/bin/sh: 1: [: missing ]
``` 🙃  ```docker
CMD [ "uvicorn", "app.main:app", ,"--proxy-headers", "--host", "0.0.0.0", "--port", "8000"]
gentle solstice
#

Space before ] too

wary oasis
#

different error intersetingly

gentle solstice
#

You should consider using gunicorn and configuring most of your cli options with env vars

wary oasis
#

with space at beginning and end

#

docker just doesnt know what to make of it

gentle solstice
#

You've got 2 commas next to each other

wary oasis
gentle solstice
#

You could just make it a single string.

wary oasis
#

Really? I did think it was weird to have to do it all in parts

#

I am brand new to docker, just following the small tutorial in the FastAPI deployment docs

gentle solstice
#

The container basically runs $ENTRYPOINT $CMD

wary oasis
#

I am reading the official docker getting started tut now

gentle solstice
#

By default, the entry point is sh -c, and the command is python

#

A lot of containers make their cmd /.docker-entrypoint.sh which runs or sources /.docker-entrypoint.d/*.sh

weak shell
#

does anyone perhaps know how to join a roblox game using the ro_py api? ( i don't know which channel to say this in )

tawdry needle
#

also i asked them about buildkit because this is clearly a docker bug, and i was curious about where in the docker "stack" the bug occurred

#

clearly it's somewhere in the dockerfile parser

#

show your code and explain what exactly it is that you are doing

#

!paste

rancid schoonerBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

native scroll
#

How can I edit a specific commit

thorny shell
#

define "edit"

#

usually you undo it, make some changes, then redo it

native scroll
thorny shell
#

yeah, undo the commit, make a new one that's just like it

native scroll
#

by git checkout <commit>

#

?

thorny shell
#

hm maybe not 😐

#

if the commit you want to edit is the most recent one, it's pretty easy -- just make your changes, stage them, then do "git commit --amend".

#

But if it's older, you'll have to do an interactive rebase, which I can't walk you through 😐

#

[and of course, if you've already published the stuff you want to change, you probably shouldn't edit at all, and instead just publish a new commit that fixes the problems]

native scroll
#

well yes its an old commit

#

but I cant publish a new commit

#

because I added lots of lines that I gotta test before production

#

so i just wanna adjust an old commit

thorny shell
#

I'm annoyed that I can't find straightforward instructions for this in that git-scm book; I thought it was pretty slick

#

if you choose to do that, I strongly suggest you practice on a throwaway repository first

#

or even use a decent GUI that makes this easy (I only know of one, but trust me, you don't want to use it)

native scroll
#

is there a safer way

#

like

#

get this commit to another branch and work from there

thorny shell
#

oh sure

#

the safest way is to just clone your existing repo elsewhere, and go crazy on that clone

#

if you hose it, so what?

#

if you finally get it all working nicely, then you can publish from it

native scroll
#

k

#

gonna try

#

just cloned the repo

#

gonna rebase now

#

ok so I've just did a rebase but I got this

#

which one to modify lines

thorny shell
#

you probably want "e".

#

Like I said, I can't walk you through this.

#

That's why I recommend that you practice on a throwaway repo.

native scroll
#

I can't even click on it lol , I'm using vscode

tawdry needle
#

then the git command line will proceed

#

it's just like editing a commit message. git waits for the file to be saved and closed before proceeding.

#

git will then pause the rebase process at the commit you marked with edit, then you can do whatever you want, e.g. git commit --amend

#

then do git rebase --continue to proceed

lyric tulip
#

Hi all,

#

Trying to think of the best way to do this. Want to make a discord bot that uses Docker, but uses a package that I'm actively developing locally. How would I achieve this? Mount as a volume? I understand volumes aren't recognized in the DockerFile, so would I just pip install in some entry script? Or should I use a separate image for just a python library?

rapid sparrow
#

what are you trying to achieve?

lyric tulip
#

I want to use docker, but have an external package that I install as an editable dependency that is not in my working directory (pip install -e)

rapid sparrow
#

pithink
editable dependency, you mean you wish to run your docker container with hot reloaded dependcy from elsewhere in your filesystem?

#

if yes, then it is just a volume

lyric tulip
#

Yes that is what I want

#

so to install it though where would this happen? In an entry script? I understand you can't reference the volume paths in the Dockerfile itself

rapid sparrow
# 
version: '3.8'
services:
  app:
    build: .
    logging:
      driver: "json-file"
      options:
        max-buffer-size: "4m"
        mode: "non-blocking"
    command: sh
    environment:
      DOCKER_HOST: tcp://dind:2375
    volumes:
      - ".:/code"
  dind:
    image: docker:20.10.12-dind
    expose:
      - 2375
    environment:
      DOCKER_TLS_CERTDIR: ""
#

it boilerplates all your docker settings into yaml

#

for example also pointing volumes to run it

#

then raising your docker container would be just docker-compose up command 🙂

wary oasis
#

Just learning about docker bridge netowrks, is that what this is? ```bash
jarrod@jubuntu:~$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:c1:c3:a3:66 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

rapid sparrow
#

three networks are defaults

#

and two networks were raised as default bridge networks to my docker-compose alignment of containers

#

tbh, no idea. but logically it should be one of those networks in theory

wary oasis
rapid sparrow
rapid sparrow
# wary oasis Ahh new command unlocked! `docker network inspect bridge` confirms ```txt "Confi...

i am bad at networks, all i know that host network makes container running in same network space as of a host
and bridge network makes isolated sub network to me to run some group of containers (default option), which exposes only through ports to host and other external networks
with expose in bridge network i expose one container to another one without revealing it to external network

wary oasis
#

That sound familiar to what I'm reading, and I think the default is "bridge" meaning normally containers cannot communicate unless they are on the same bridge netowrk

rapid sparrow
#

that's why we can point in docker-compose network settings same network name 😉

#

then it will create bridge network

#

to which we can connect multiple containers

wary oasis
#

Yeah I am going to run nginx, ddclient, and my website in a docker-compose.yml and I need to network nginx with the app

rapid sparrow
#

technically same can be done with regular docker, but why to bother when we can boilerplate it into yaml

wary oasis
#

Yeah docker-compose seems very useful

rapid sparrow
#

it creates by default same shared bridge for them

wary oasis
#

Oh I like that!

rapid sparrow
#

well, except probably exposing their ports to each other, with expose: port for internal communications 🙂

wary oasis
#

That makes sense

rapid sparrow
#

i just finished writing docker-compose for self hosted github runner. Feeling yay.

#

finally raising this self hosted runner without interactive actions

#

pexpect library for the win, to turn crappy software into semi-usable software

dusky cloak
#

Hi there,

I'm not sure if this is the right channel, but I would like to ask if someone has any experience regarding integration of SAP SuccessFactors?

tacit trellis
#

How do people normally handle GitHub projects? If I posted my database seeder, would people leave comments or make minor improvements? I don't want overhauls of the code, but I'm striving to get more of an authentic GitHub experience and don't know where to turn.

rapid sparrow
#

if alone dev = centralized workflow

#

if organization = usually feature branch workflow

#

if open source = usually forking workflow for external devs, and feature branch workflow for really close inner cycle.

#

if many devs, big project, kinky git users = then gitflow

#

if super kinky git users and just experienced devs = they can go for trunk workflow

tacit trellis
rapid sparrow
# tacit trellis I meant more in regards to here. I am a solo coder doing it as a data science pr...

pithink miraculously writing content interesting to other people
with experience it should come eventually. When you became professional, your material at some point should become interesting to other people if you will form it correctly for colloboration/document it understandably to others.
(or just being TikTok/Youtube person with a hunch to media stuff, then you could make as interesting material of less professional quality)
and well... making somewhere article about it to boost at least initial kick of users, just mentioning in code communities.

tacit trellis
rapid sparrow
#

it depends on what you project does

tacit trellis
#

I see #databases. I guess that would be a good place to start because it seeds random data into databases.

#

And then allows the user to query it and whatnot. It's designed to allow a data scientist to search for trends in a completely randomized environment where one could not simply predict outcomes (in theory).

rapid sparrow
tacit trellis
#

I don't write it to do what has never been done. I do it to provide work upon which can be improved.

tawdry needle
#

(you can post it here and someone might be happy to review it if they're not busy)

#

we used to have a separate channel for posting projects but it became spammy

tacit trellis
#

I've never been forked, so I'm a virgin.

rapid sparrow
#

so u would have to convince people, why your solution is trully better in some aspect
(in your article for example 🙂 )

tacit trellis
#

You've got me all wrong. I'm not trying to corner the market. I'm using this concept as a springboard.

tawdry needle
#

if people start using your software, they will eventually start filing bug reports. triaging bug reports isn't really something to aspire to imo.

tacit trellis
rapid sparrow
tacit trellis
#

Maybe I'll just stick to talking to people about like, "What should a database program that deals in widgets provide for the end user?" and just go from there.

tawdry needle
#

i'm not sure why darkwind is discouraging you so much here

#

people write their own versions of things that already exist all the time

#

it's a great way to learn, if nothing else

rapid sparrow
#

well, point taken.

tawdry needle
#

and sometimes you have a new or different idea about how to do something, and other people actually start using it

tacit trellis
#

@rapid sparrow That is the evolution too. It will have ways to interact with users that are created, and items.

#

Basically, I want to build it into a seeder AND database application where you deal with a fictional company and how to make telephone lists for cold calls, check product inventory, etc.

#

Very early in the works.

tawdry needle
#

publishing software as open source is good in general. you're just not likely to end up engaged in some highly intellectual social communal programming experience. open source dev and maintenance can be a challenge. but it's also tremendously rewarding to finish something, and publish it, and see other people enjoying what you've published.

tawdry needle
#

so i'm with darkwind on maybe tempering your aspirations a bit

rapid sparrow
tawdry needle
#

indeed

#

there's nothing wrong with programming for fun, but it's important to keep in mind that, if you are writing code for other people to use, you should keep in mind what those other people might want. keeping your goals in mind is very very important when working on any long-term project, software especially because it's so easy to get distracted and just mess around with 100 things and never get anywhere.

#

(messing around with 100 things is also a great way to learn, but maybe not the most efficient way)

tacit trellis
#

I probably haven't sold it very confidently, I'll admit.

tawdry needle
#

is this like an extended programming exercise? as if you were contracted by some company to build their call center system, but it's a made-up company?

tacit trellis
#

Yeah! That's a good way to put it.

#

With a built-in database seeder because I didn't want to use an alternative. Lol

#

At most, it's probably going to be a resume proof of concept.

tawdry needle
#

i admire your creativity. this could be a good exercise in software architecture.

however, keep in mind targeting a made-up task makes it easy to cheat and adjust the task if and when things get difficult. you also aren't forced to contend with "unknown unknowns". it's also literally not useful to anyone unless you wanted to try to build a game of some kind around it. so i would argue that this is not a good resume project, and if you are looking for a resume project i suggest doing something more practical.

#

there might however be very practical individual components in here

tacit trellis
#

Okay, some gems in the dirt, so to speak.

tawdry needle
#

indeed. also it's not dirt in and of itself, but it might not be the best use of your time and energy

#

e.g. the test-database seeder is an interesting project, although maybe a bit boring and like darkwind said, good ones already exist. database seeding isn't really a challenging or interesting task from a design perspective

#

however, making a little game where you have to work in a call center without getting fired... that might be fun 😛 (or just stressful to play, lol)

tacit trellis
#

I think my task now is interacting with the seeded database.

#

Like, pulling random customers and putting them on a list, for example. Maybe generating a webpage to view them on.

tawdry needle
#

sure, if you want to just learn basic things like "how do i read from a database and move data around" then sure, i think this is a good project

#

right. you can practice things like making web dashboards and designing apis

tacit trellis
#

The seeding is essentially done for now. Though, probably not done cleanly. Lol

tawdry needle
#

the fact that it's a fake call center becomes less relevant if you start to build nontrivial real software around it

tacit trellis
#

So, I should spend less time on installation and seeding and instead focus on the actual components of database management.

tawdry needle
#

i would say so. get it to a point where it works well enough for what you need, then move on to other tasks

#

this will be time very well spent if you learn how to use and administer databases and how to use popular packages like sqlalchemy

#

maybe fake data isn't ideal, but you already did that part. so it's not worth undoing work you've already done

tacit trellis
#

Is SQLAlchemy one that seeds as well? I'm looking through the website now.

#

Seems like it is. I'm looking at a script that has a Customer class that appears to be creating data.

#

If nothing else, I learned about a new tool for seeding a database and thus less to have to focus on when dealing with the "game" of how to not get fired at a call center. 😄

#

Though, probably working on a script instead of being on calls would be the antithesis of good call center work if we're getting really into it.

tawdry needle
#

oh

#

sqlalchemy does not generate fake data for you

#

people use libraries like factory boy in conjunction with it, which is what darkwind was saying

tawdry needle
#

your score is the % of time at work spent not working while still meeting performance targets

#

like one of those "program an ai and fight other players" games combined with one of those old flash games where you run a restaurant

tacit trellis
rapid sparrow
#

Django is needed only if u need server side to it
Making board of scores, profile, making it multiplayer game

tacit trellis
#

Guess I will have to for that, huh?

#

Thanks for the pointers, @rapid sparrow and @tawdry needle! I appreciate the honing of my focus.

rapid sparrow
# tacit trellis Dang it. I had a place asking if I code in React.

U a welcome. Also saying that learning curve is smaller and easier for Vue.js, but React is thrice more popular in job market. As backend Dev it was enough for me to try Vue. Js, but if I had aimed for serious Frontend career, I would have probably went for React

rapid sparrow
heavy knot
#

Does anyone know why this doesnt work in Colab, when I run it on VSCode it works! I need to use colab in order to train my CNN faster

tough heart
#

Hiya, I'm looking to set up github actions to be able to release my package on PyPi automatically. I am following this page from the python packaging user guide: https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
It tells you to put some secrets in github and to put a yml file in .github/workflows. How does pypi know what version to release? Now it just tries to re-release an existing version, and that, of course, fails:

ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/          
         This filename has already been used, use a different version. See      
         https://test.pypi.org/help/#file-name-reuse for more information.
#

All I did now was adding some secrets from (test)pypi to github and adding this to a file in .github/workflows:

    - name: Install pypa/build
      run: >-
        python -m
        pip install
        build
        --user
    - name: Build a binary wheel and a source tarball
      run: >-
        python -m
        build
        --sdist
        --wheel
        --outdir dist/
    - name: Publish distribution to Test PyPI
      if: startsWith(github.ref, 'refs/tags')
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        password: ${{ secrets.TEST_PYPI_API_TOKEN }}
        repository_url: https://test.pypi.org/legacy/
rapid sparrow
tough heart
#

Yeah that would make sense

#

This yaml file is so disgusting to work with, you basically copy stuff and hopes it works

rapid sparrow
# tough heart This yaml file is so disgusting to work with, you basically copy stuff and hopes...

Yaml is not at guilt. At guilt ability to execute it only when publishing new version to the cloud.
there are two choices to resovle it

  1. Writing your own scripts, you can fully test locally, and just inputing interface to their invokation into pipeline. (Through argparse changed arguments if necessary). Comfortable since you can check and test each job individually.
  2. I think i saw somewhere a way to execute GA pipeline locally, perhaps this can be explored too
tough heart
#

I'm not saying yaml is at guilt, at guilt is my puny human brain. I'm just saying I find it disgusting to work with.

rapid sparrow
#

Because you don't own the process enough. Test it fully locally

#

just insert result into pipeline code

tough heart
#

That's part of the problem, the thing to test is "does it do X when I push a commit". That's not possible to test locally, because by definition pushing to remote is not local

#

I think github maintains its own version number and uses that maybe?

rapid sparrow
#

just echo all the predefined CI variables and see for itself

tough heart
#

What are all the predefined CI variables?

#

Oh, you mean I should just run the commands in the yaml file locally?

#

I'll guess I will create a test package/repo to test setting this up

#

What a clusterfuck

rapid sparrow
#

i mean, that i think it is great, when your CI just executes your commands from makefile

#

which you can execute locally fine too

#

it makes certain predictability?

#

and easy to test situation 🙂

rapid sparrow
#

gathering all those commands as python package already pithink

#

if necessary, CI only overrides some parameters like --image={{github.blabla}} for my argparse CLI

rapid sparrow
tough heart
#

I see, you actually seem to have some grasp on what happens behind the screens

#

Also I only just find out that what you set as uses is actually just a reference to a github repo

#

So uses: checkout@v1 uses tag v1 of the actions/checkout repo

#

Ah, OK, I am slowly understanding more. I was confused by this YAML syntax:

    - name: Build a binary wheel and a source tarball
      run: >-
        python -m
        build
        --sdist
        --wheel
        --outdir dist/

This of course is just python -m build --sdist --wheel --outdir dist/ but I was confushed by everything being on its own line 🤦‍♂️

#

OMG I am an idiot

rapid sparrow
tough heart
#

I have specified the version right there in the setup.py of course

#

Of course this gets used when I run python -m build

#

There are 63 different ways to write multi-line strings in YAML.
LOL did I say anything about YAML being disgusting?

tawdry needle
wise stag
#

I'm looking for a compact device to run a .py and output a basic gui to a display. Basically need something to replicate the functionality of an Arduino except I don't want to learn another language.

I'd go with raspberry pi but they're totally sold out in my country. Any advice for alternatives?

tawny temple
wise stag
tawny temple
#

It's not something I can speak to unfortunately. I've never used micropython, only C++ with microcontrollers. I have heard of it though, so figured I'd throw it out there.

#

Again #microcontrollers may have people that can answer that. You could also ask in the Adafruit Discord server

wise stag
tough heart
#

Still struggling a bit with github actions. I'm following https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ and based on that I've added the following job to my workflows file:

  build-n-publish:
    name: Build and publish to Test PyPI
    if: startsWith(github.ref, 'refs/tags/v')
    needs: build
    runs-on: ubuntu-latest
    steps:
    - name: Install pypa/build
      run: >-
        python -m
        --wheel
        --outdir dist/
    - name: Publish distribution to Test PyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        password: ${{ secrets.TEST_PYPI_API_TOKEN }}

Triggering works like expected, however, the "Install pypa/build" step fails:

Run python -m build --sdist --wheel --outdir dist/
ERROR Source /home/runner/work/mypackage/mypackage does not appear to be a Python project: no pyproject.toml or setup.py
Error: Process completed with exit code 1.

This does make sense, since there is no setup.py in mypackage/mypackage; it's in mypackage... It's not clear to me why it uses mypackage/mypackage as the working directory, or how to fix this. Any suggestions?

tough heart
#

Ah, I think I know what's going on, the mypackage/mypackage path is not the subfolder in my dir, it's the default working directory. However, the repo is not checked out because that was done in a different job and apparently jobs have independent state.

south forum
#

Hey, im trying to deploy my flask app to DigitalOcean but im lost.

I created a droplet to run a py script with cronjob every 24h.
I created a webapp with my flask webapp
How do I now connect these so that the py script returns get updated into the db from my web app?
Thanks for any help or resources / documentation

rapid sparrow
south forum
rapid sparrow
#

i did not attach selenium module yet though

#

scraps with only other means fo rnow only

#

i had code for selenium scrapping too, but did not refactor into my 2.0 version yet

rapid sparrow
rapid sparrow
wooden ibex
#

Selenium is browser driver

rapid sparrow
# wooden ibex What?

there was a hidden dm msg from him, my answer was to it:

[8:08 PM]
hey man
[8:08 PM]
do you have some resourcess regarding deployment ?
[8:09 PM]
I can't get it to work with DigitalOcean :/
rapid sparrow
south forum
wooden ibex
rapid sparrow
#

Docker-compose and Ansible are both fine starting options to deploy it into DigitalOcean

wooden ibex
#

Just docker compose and some scripts. Ansible is for when you get a fleet of stuff and even then, Kubernetes might be better option vs Ansible

rapid sparrow
#

can't really argue with that, but Ansible is still kind of experience that should be tried before Kubernetes.

wooden ibex
#

Why?

#

Because everyone loves dealing with OSes except they don't

rapid sparrow
#

People should understand Linux to full capacity before jumping to layer on top of it

#

for deeper understanding of processes

rapid sparrow
#

Linux is raw SQL, Kubernetes is ORM.

wooden ibex
#

Ansible isn't required to learn Linux

rapid sparrow
#

some configuration management tool is required in order to learn how to automate Linux.

wooden ibex
#

You are like saying, learn RAW SQL so you can do Graph Database

wooden ibex
#

Since most configuration in Linux is just installing a few packages

rapid sparrow
# wooden ibex No, you write scripts. Watch ```bash #!/bin/sh echo "Hello"```

bash scripts are reaching quickly maximum to their scalability in terms of code growth. I would say they reach it when code is more than oneliner.
even scripts in python reach it quickly too for installation purposes. as makefile they work nicely though.
Better to use language/ecosystems meant for configuration management. Because they scale for stuff bigger than 5-10 commands. Plus soo much of reusable stuff in this ecosystem 🙂

wooden ibex
#

For docker, most of time you just your configuration management is just "install docker"

#

Ansible is fine if you are old school and running large fleet of Linux machines but it's not required for most modern setups esp on Public Clouds

rapid sparrow
#

It is not required, but it is important learning experience. ¯_(ツ)_/¯

wooden ibex
#

and thus learning Ansible isn't required. And if they are running Kubernetes, they have management layer that's not Ansible

south forum
#

if I host a flask app, the app "runs" all the time right?
so If I would put a "sleep(24h)" in a while loop it would run once through the loop each 24h right? 😄

rapid sparrow
wooden ibex
#

so learn Linux, Learn Docker, then decide if you want to learn more Linux management using Chef/Puppet/Ansible/SaltStalk or learn Kubernetes or learn a public cloud

wooden ibex
rapid sparrow
wooden ibex
#

I know plenty of groups running Linux VMs without Ansible at all. They use Azure/AWS script injection to set config script at start up time

wooden ibex
rapid sparrow
#

you can't see its advantage to have more scalable and more maintanable code with config management. Scripts have limit they can cut.

wooden ibex
#

I have busted Powershell script to run/install microk8s on some Ubuntu boxes

wooden ibex
rapid sparrow
#

and as i mentioned, i said already that Ansible(and other config management tools) is not really required after jumping to kubernetes. But I will stand on it being important learning experience before jumping to layer of abstraction higher. Because developer needs to be aware of all posibilities, tools to have at his disposal. Understanding better processes in between.

wooden ibex
#

Will it die off? of course not, old stuff never dies. Is it modern future people need to know? Not unless Linux Sysadmin is in your future.

#

Learning Linux -> Learning Docker -> Learning Kubernetes 🤮 is better future

rapid sparrow
wooden ibex
#

Linux is useful because most people will need to manipulate Linux in docker. Just moving stuff around like moving node dist output into root and such

rapid sparrow
# wooden ibex Learning Linux -> Learning Docker -> Learning Kubernetes 🤮 is better future

The problem with all of it is kind of similar to how stuff in python works and Data Structure and Algorithm displine (if to give alternative analogy). Do you need to know how under hood array, dictionaries work at memory level to write some stuff? or even how it is turned into assembler commands and scheduled into chain of commands to be executed at CPU processor?
Surely you don't need. But each additional piece of knowledge for underlaying technology underneath it, makes your understanding deeper, allowing more performance optimization at conscious level performed ¯_(ツ)_/¯

wooden ibex
rapid sparrow
# wooden ibex If you want but time is finite so at some point you leave the details to someone...

can't disagree with that. But quite often this approach yields good results to me.
At least before jumping to Frontend framework technologies, i made sure to learn as much as possible of old school HTML, CSS, vanilla JS, browser API posibilities.
It served me well, seeing through frontend framework syntax sugar and seeking features i knew there should be reimplementing some technology, or seeing that this technology was not implemented in frontend framework at all.
Learning basics/fundamentals leads to better awareness of what is going on.

indigo zenith
#

If you don't need docker then just set up a cron job to run the scraper... That's about it unless there is more to your question?

south forum
indigo zenith
south forum
#
  • I deployed it without that variable and got an "install package error: "greenlet".. any idea 😄
indigo zenith
rapid sparrow
#

Easily replicable deployment to Dev and Prod

wispy orbit
#

this is quite urgent, so please have a look 🙂

heavy knot
#

is it possible/how do you use pip to install a package as a command, as in, make a folder in /usr/local chmod +x a file and add the folder to the PATH

brazen forge
heavy knot
heavy knot
#

Should my pre-commit with isort, black, etc. only check for errors or also fix them?

#

As in, should I run these tools with —check in pre-commit or not.

lyric tulip
#

Hi all, following some articles in CircleCI to integrate into my project, but it seems that they always use another docker image from circleci to build and run tests. Why wouldn't they build from your own DockerFile? Does your DockerFile only matter for deployment to the VPS?

tawdry needle
heavy knot
#

Question: Is it possible to iterate through a directory of files, consider them python files and attempt to call a static function within them(assuming they have that function)

rapid sparrow
#

Practically it is sounding as potential bad decisions

gentle solstice
#

You can use glob.glob("*.py"), then importlib.import_module(file[:-3])

#

assuming it's in the path.

frozen oxide
#

Does PDM allow you to install different python versions? I've got package installation down but I'm struggling with sorting out how to use it to install versions of python.

tawdry needle
#

so you would first install your desired version of python using whatever tool makes sense. then you would use that version of python to install pdm.

frozen oxide
heavy knot
#

Basicially, I am generating python scripts to import materials and generate the node graph

#

And I have one master script that needs call them

narrow slate
#

Does anyobody know how to generate files in a repo from CI ? I am basically looking to integrate terraform-docs within my CI for it to generate the readme file.
What I am thinking is to .gitignore the README.md and generate it within the pull request but I am not sure if this it the right approach 🤔

south forum
#

If created a docker container with my flask app now.
How do I deploy this now to digitalOcean? 😄
( and I bought a domain :P)

#

Really new to the entire deploy / server stuff, but got told using docker is best

south forum
#

So I followed a tutorial, and the docker runs fine in the console on the droplet, but wen I open the ipv4 + port address I get " site took to long to respond" ..?? help pls

indigo zenith
rapid sparrow
rapid sparrow
#

curl http://localhost:port/

south forum
#

Hey yeah, I actually got it to work now. Still not sure whats going on tbh

loud bear
#

why tf does codecov/patch action keep on failing?

timber umbra
#

I accidentally committed (only locally) a file I shouldn't have (a database). What's the easiest way to fix that? My desired state is the same commit except this file should be in .gitignore.

brazen forge
timber umbra
tawdry needle
#

let's say you have a commit history starting with A, and you make a bad commit X:

A ← B ← C ← D ← X (branch:main)

when you run git commit --amend, you make a new commit Y

A ← B ← C ← D ← Z
              ︎︎↖ Y  (branch:main)

and basically leave the old commit X without any branch attached to it

#

so this does roll back in the sense that you are making a new commit with the changes rolled back, and then rewriting history

rancid schoonerBOT
south forum
#

My flask webapp project :

~/ProjectFolder
    | --Dockerfile
    | --app.py
    | --scrape_data.py
    | --chromedriver.exe
    | __/.venv
    | __/my_project
        | -- __init__.py
        | -- models.py
        | -- forms.py
        | -- db_connector.py
        | -- database.db
        | __/routes
            | -- allmyroutes
        | __/static
            | -- allmycss/stuff
        | __/templates
            | -- allmyrtemplates

The "scrape_data.py" scrapes a site using selenium and calls a "write_to_db" class in "db_connector" => send data to db

I now want to add a cronjob which runs "scrape_data.py"every 24h.
But I don't know how to get this done.
For deployment I am using DigitalOcean 1-click Docker App

Thanks in advance

Dockerfile:
FROM python:3.8-slim-buster

COPY . /app

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt

ENV FLASK_APP=app.py
ENV FLASK_ENV=development

EXPOSE 5000

COPY . .

CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]
mental steppe
#

Un, not that familiar with that. It sort of seems to me like docker containers are intended to be more ephemeral

indigo zenith
# south forum My flask webapp project : ```py ~/ProjectFolder | --Dockerfile | --app.p...

A good survey of the different approaches to cron + docker https://www.howtogeek.com/devops/how-to-use-cron-with-your-docker-containers/

How-To Geek

Running background tasks on a schedule is a standard requirement of backend services. Getting setup used to be simple – you’d define your tasks in your server’s crontab and call it a day. Let’s look at how you can utilize cron while using Docker for deployment.

gentle solstice
#

K8s has better cron jobs

rapid sparrow
gentle solstice
#

And apscheduler

heavy knot
#

Anyone here with AWS EC2 experience?

rapid sparrow
hallow cypress
#

Hey guys I want to learn GCP is there any course you guys can recommend for someone who doesn't have any prior experience on neither cloud nor GCP..

brazen forge
brazen forge
wooden ibex
heavy knot
obsidian marsh
#

im using cibw to build a package and everything builds properly, but the job fails because a single linux_x86_64 wheel gets built as well

#

any ideas?

heavy knot
wooden ibex
#

It’s pretty advanced, documentation is best source

heavy knot
#

Recently I engaged in a project to make a bot for a game, but I got in a problem that I can't solve: every time I read the game screen, the game closes (possibly the anti-cheat crashing the game), do you guys recommend anything that I can do to read the game screen without being detected?

brazen forge
alpine crane
#

is packaging on topic in here? like setuptools distutils etc?

strange crown
#

Hey guys, I need a help ! How to switch to master branch in Git Bash ?

brazen forge
brazen forge
obsidian marsh
loud bear
#

Thank god codecov does merge my coverage reports automatically

wary oasis
#

So I have this docker-compose file that runs a reverse proxy (nginx proxy manager). I'm just wondering how I should handle adding a new app to my website. E.g: I'm making a clock app that tells the current time that will be hosted at clock.mydomain.com. So I will config the reverse proxy to route requests to that domain to point to my clock app running in another docker container. Can / should I add the clock as a service to the same docker-compose? If I have other services running as part of the compose, can I make changes to one service without interrupting the others? Or Would I need separate docker-compose files to to that?

unknown.png
#

Example: I have the proxy manager and clock app running from one docker-compose, but then I want to add a third app / service. Will adding a new service disrupt the other 2 if they are in the same docker-compose file?

rapid sparrow
wary oasis
rapid sparrow
rapid sparrow
wary oasis
#

Each service will depends on the proxy manager being up. But besides that, there will be no reliance on other apps, so optimally I could stop or delete an app without effecting other things

wary oasis
rapid sparrow
#

I joined composes into one network nevertheless

#

For easier communications between services in different composes. Wishing to keep DNS name resolving, instead of using IP addresses

wary oasis
#

Yeah I went with this approach too yml networks: default: name: scrappy external: true Because I'm using the proxy manager, I'm putting everything on the same network as that so that I can simply route things to http://clock:8080 since they are all on the same bridge network

#

Hmm Can someone spot the problem here?

unknown.png
#

Oh I think I need to use the dockerfile directive

#

got it ```yml
services:
app:
build:
context: .
dockerfile: Dockerfile.dev

networks:
default:
name: bridge

icy sonnet
#

What is best practice directory layout for module development? I currently have something like ```project/
├─ dev_script.py
├─ package/
│ ├─ module1/
│ │ ├─ init.py
│ │ ├─ file1.py
│ │ ├─ file2.py
│ ├─ module2/
│ │ ├─ init.py
│ │ ├─ file3.py
│ ├─ init.py

#

I work out of dev_script.py and as things progress move functions/code to relevant files

#

But I always find I end up jumping through a bunch of hoops to import things correctly

#

especially if package/ isn't a top level folder in the project, but is a subfolder somewhere. then I have a bunch of things like import py.src.package.module

#

Maybe directory layout is not the problem and I should be using some other pattern to do active development

iron basalt
#

sometimes I follow a similar pattern, calling individual bits of code from a git-ignored dev.py module

#

in most cases, I just place the code where I think it should be, and let pycharm import it for me

loud bear
#

Hell I put my outermost public API (2 functions) in __init__.py to avoid creating another module and init.py remains almost empty anyways

gentle solstice
#

as long as you import it into __init__ to make it available

south forum
#

has anyone experiences with running selenium webdriver on an ubuntu server? (flask app)
Locally it works fine but on server I get "No chrome executable found on path" error.

indigo zenith
#

If you're on a newer version of Ubuntu look for newer instructions

rapid sparrow
#

which is possible at least like this

apt update
apt install -y wget
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg -i google-chrome-stable_current_amd64.deb; exit 0
apt-get -y -f install
#

may be in other commands too

karmic jay
#

Hi guys, I want to create a dev environment using proxmox. What I mean is I want to use visual studio code -> connect to a LXC container and create dev environment where I install the python3.7 or 3.10. I hate fighting with dependencies on my windows machine. Install npm, and many more stuff.. sorry i'm all over the place.

indigo zenith
#

WSL2 is probably where I'd begin though, it's pretty quick and simple to set up

rapid sparrow
south forum
rapid sparrow
south forum
# rapid sparrow What? U aren't root of your own Dev machine? <:pithink:652247559909277706>

digital Ocean forum:


App platform is a managed way to host applications – you give the location of your git repository or container image, and it deploys the app for you. You can configure features like custom domain names through the console after deploying your app. It’s unlike a droplet (https://cloud.digitalocean.com/droplets) which give you your own virtual machine. There isn’t a way to log into app platform, since it’s not a VM ```
rapid sparrow
#

just put chrome into your docker container

south forum
#

is chrome = chromedriver.exe ?

rapid sparrow
loud bear
long surge
#

I'm learning about pyproject.toml and on PEP 518 https://peps.python.org/pep-0518/ it is said that pyproject.toml specifies "build system requirements" to "build" the project. What does it mean by "build"? I thought python was interpreted, not compiled.

brazen forge
#

build as in building a package

#

to be installed using pip and what not

long surge
#

what does that mean

brazen forge
#

have you ever installed a package using pip install?

karmic jay
south forum
#

Do I need to put the google-analytic tag in every page, or is it fine if I put it into my base template ?

rapid sparrow
#

it is same code for all pages anyway

#

u would be breaking DRY principles and going WET otherwise

south forum
woeful wagon
#

I am trying to run a github actions manually and for whatever reason when I try to run it seems like its not running at all and does nothing. I dont see a failure a started or anything.

this is my first time writing a github actions script

here is the github actions that I am using

name: Compile GUI for Ubuntu
# run this job manually
on: workflow_dispatch
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      # used to checkout different branches
      - uses: actions/checkout@v3
        with:
          # checkout my specific branch
          ref: main
      - uses: actions/setup-python@v4
        with:
          python-version: "3.10"
      - name: pip install dependencies
        run: pip install -r requirements.txt
      - name: compile with pyinstaller
          cd ./src/python/
          python -m eel gui_main.py ../web --onefile --noconsole --clean```

I basically want to a github action that I can manually trigger that uses the code that I have in the main branch to compile the python code and then push the compiled executable to a new branch called executables

feel free to @ me
heavy knot
#

whats the change?

unknown.png
#

i tried both of these but no word change highlight

unknown.png
tawny temple
#

Maybe the line endings (lf vs crlf)

heavy knot
tawny temple
#

I think that is indicating a trailing newline was added. Not the same thing as the line endings I mentioned earlier.

woeful wagon
#

whats the cleanest way to push from a github actions bot to another branch?

#

I have been seeing people say do things such as

git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config user.name "github-actions[bot]"```

I am wondering if there is a clean way to do it like

git config user.email "${ github_actions_bot_email }"
git config user.name "github-actions[bot]"```

#

found it!

#

it was git config user.name github-actions git config user.email github-actions@github.com

rapid sparrow
#

i am having an issue with pipenv. I need to update one library

#

if i update with pipenv update lib it updates so many libs, that it breaks tests

#

if i update with pipenv update lib --keep-outdated it is not updating the necessary lib, it only updates some libs around (much fewer)

#

how to update my lib only more reliably with pipenv?

#

nvm, found very useful thread

subtle sundial
#

Hi,

RE: Python Sphinx Documentation

Is there a way to have the github external source code embeded in a sphinx page?

# This works but needs the file locally or relative to the source. I don't want that
.. literalinclude:: example.py

# I want something like this (i.e. external link but this does not work despite it being just a .py file)
.. literalinclude:: https://raw.githubusercontent.com/bentraje/blender-python/master/animation-rigging/constraint_param_from_armature_export.py

I already tried adding 'sphinx.ext.linkcode', 'sphinx.ext.viewcode' in the config file but it still does not display the code.

P.S. posted this on the available help channel but unfortunately didn't receive anyreply 😦

brazen forge
# rapid sparrow https://github.com/pypa/pipenv/issues/2665

I don't expect this to ever work as expected, given how tangled up this code is. I moved on to poetry ages ago, primarily because of this.
https://github.com/pypa/pipenv/issues/2412#issuecomment-988588499 🗿

GitHub
Upgrading a dependency with --selective-upgrade doesn't seem to wor...

Running using the latest release (2018.6.25), selective upgrades don't appear to work as expected. (If I'm doing something wrong, please let me know :-) ) This was originally filed ...

rapid sparrow
#

Already even tried deleting

#

It keeps records in changed state lock

brazen forge
#

have you tried... not using pipenv lemon_smug

rapid sparrow
#

Legacy. Not my choice

brazen forge
#

couldn't you update the version spec in the Pipfile and then try pipfile lock or something?

#

or does that update the versions for the others as well?

#

hmm. it will I guess pithink

rapid sparrow
brazen forge
#

I have used pipenv only once for cdktf and it was not pleasant

rapid sparrow
#

Lock is good idea though, I remember there is mentioned in one of projects docks regenerative command

#

It should be probably this to bypass this

tawdry needle
#

it's not (yet) possible to incorporate a constraints file when generating a requirements file, unless you literally insert -c in the requirements.in file, which isn't possible when using pyproject.toml. so it's entirely possible to generate a requirements file that does not meet the constraints!

rapid sparrow
rapid sparrow
tawdry needle
rapid sparrow
#

my projects are docker web projects, not an issue

tawdry needle
#

so you need to first extract the pyproject deps and all the extras to a temporary requirements.in file, insert -c at the top, and run pip-compile on that. messy

#

would be great if pip-compile had a -c option itself

rapid sparrow
#

with making rule for deps version only of one blocking to upgrade library

#

and upgraded everything forward according to available rules

#

Kind of good in some sense tool, because it forces to move u forward xD

graceful light
#

I was looking around to find tools that would scan and analyze your Python code to possibly find bugs or security issues, right now I've come across Amazon CodeGuru and SonarQube, but these are some bigger enterprise tools whereas I was looking for something preferably open-source. Anyone know of any?

#

I'm already using black, isort, and flake8 for formatting, import sorting, and general linting, but figured finding bugs and/or security issues would be helpful on top of that

shadow crow
brazen forge
#

!pypi bandit

rancid schoonerBOT
brazen forge
#

!pypi safety

rancid schoonerBOT
#
safety v2.2.0

Checks installed dependencies for known vulnerabilities and licenses.

graceful light
#

I did just come across safety, going to have to add all of these to my pre-commit and GH Actions workflows
Thanks, guys!

mystic void
loud bear
loud bear
#

Afaik most services like codacy codefactor just run pylint and a bunch of tools you already know to gather "metrics" and "insights"

brazen forge
loud bear
#

Many people don't use GH?

#

I haven't seriously found a project from pypi taking me to gitlab or something since like eternity

brazen forge
#

think private projects that are hosted in on-prem/private GitLab

loud bear
#

So why does sphinx autosummary not add module level docstrings in the table it generates?

calm crescent
#

where can i ask about making bots

tawdry needle
#

i was just wrestling with autosummary last night, :recursive: didn't appear to have any effect and i couldn't figure out why

#

apparently there's a sphinx extension to generate epydoc-like api reference pages, i would love that

loud bear
#

it can be achieve with a template

#

although i believe autogenerated is too much control lost for smaller apis

thorny shell
tawdry needle
#

yeah i actually kind of love/hate sphinx and restructuredtext.

thorny shell
#

I've never used 'em in anger

tawdry needle
#

love because of what it can do, but hate because of how clunky and awkward it sometimes can be

#

i think the sphinx user experience has improved a lot recently

#

it's still a little awkward to use a python script to generate a makefile, but the file itself is small and readable and easy to customize

#

plus sphinx-build i think now does its own "make-like" thing where it doesn't rebuild unchanged pages

#

and some of the HTML themes are pretty good

#

in hindsight, we probably should have been writing docstrings in sgml. but failing that, the real killer feature of rst and docutils is the ability to define your own semantic markup directives, and sphinx includes a huge amount of useful directives built right in

#

the only reason that sgml/xml would be preferable to rst is that you can arbitrarily nest tags in sgml/xml, but rst syntax is fairly "non-composable" (e.g. you can't easily apply bold or code formatting to the text of a hyperlink). whereas most markdown implementations do a pretty good job of this

thorny shell
#

I don't think I've ever seen SGML used anywhere in the last 20 years

#

if I had to write SGML I'd complain

tawdry needle
#

there are some documentation tools based on it, although idk if they're in wide use

#

it's literally markup language, i hate that it and its descendents were co-opted for serializing structured data

thorny shell
#

that too

#

in his defense Tim Bray is a wonderful writer and blogger 🤣 (he invented XML)

tawdry needle
#

cool, i haven't read anything of his

#

i mean okay... the closing </ thing is kind of annoying

def add(x: float, y: float) -> float:
    """Add two numbers.

    <param name='x'>First number.</param>
    <param name='y'>Second number.</param>
    <return>A number.</return>

    <example>
      add(1, 2)
    <example>
    """
#

aren't they technically optional in sgml?

# 
def add(x: float, y: float) -> float:
    """Add two numbers.

    :param x: First number.
    :param y: Second number.
    :return: A number.

    .. code-block::

       add(1, 2)
    """
#

the rst is definitely cleaner-looking

#

but you basically can't do that with markdown at all, there's no option for adding your own semantic directives

thorny shell
#

honestly I wouldn't write that sort of documentation unless there were a gun to my head

#

afaict, there's never any mechanism to ensure it stays accurate, hence nobody maintains it, hence it's wrong, hence I never even bother reading it

#

doctest is the only per-function doc I'll write, for exactly that reason

true vapor
thorny shell
#

good docs are indeed useful

#

IME, though, most docs are worse than useless

loud bear
#

Docs written by a non-dev of the project are generally better imo

thorny shell
#

wouldn't surprise me

#

they'll make fewer assumptions

#

and will ask obvious but important questions like "yeah, OK, but that problem does this function solve?"

true vapor
tawdry needle
#

bad docs are entirely a cultural problem

#

imagine trying to use pandas or numpy without per-function reference docs

thorny shell
#

indeed

#

my gripes are around internal tools, not third-party libraries

#

the latter have good docs almost by defintion: if they didn't, I wouldn't use them 🙂

#

somehow I feel like drawing @dry hatch 's attention to this discussion

south raft
#

hi, do you guys know free cloud servers?

heavy knot
#

How do I fix "Something went wrong. Please restart WSL with the following command and try again..." for Ubuntu Terminal 20.04 LTS