#tools-and-devops

1 messages ยท Page 69 of 1

tawdry needle
#

fastmail is super cheap too for small businesses

#

and then you actually get a real inbox, tech support, etc

#

depends on your needs i guess

#

for just sending alerts to yourself i do think it'd be nice to be able to run an smtp server in a container

wooden ibex
tawdry needle
#

because not-ms

#

idk

wooden ibex
tawdry needle
#

yeah if i was running a business w/ employees i'd probably want something "industrial" like MS

#

SSO and stuff

wooden ibex
#

and fastmail isn't a mail sending service, it's email server

#

I'm talking about Applications sending emails like reminders, password resets and like

#

for that, use Sendgrid OR Amazon SES OR whatever

tawdry needle
#

oh yeah for sure

rapid sparrow
#

Mail protocols are so friendly, that they are not really different from rest apis

wooden ibex
#

SMTP is PITA to deal with

rapid sparrow
#

The only real requirement, having turned off protection from bots

wooden ibex
#

but Sendgrid and SES have RestAPI for email

rapid sparrow
#

Nice I guess. Rest makes life easier

#

I already was told by other devops that

#

Alertmanager integrates with sendgrid easily

#

So, that is my choice then

pastel wing
#

if I use docker-compose with --scale flag, does docker load balance it automatically?

rapid sparrow
heavy knot
rapid sparrow
#

hmm, why do I have a feeling, that it is another web scrapping steam bot? ๐Ÿค”

rapid sparrow
rancid schoonerBOT
#
Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

heavy knot
#

it's simply for a usernames list of names for my program

#

no cracking or any sort involved

rapid sparrow
#

do you parse the steam web site for that?

heavy knot
#

I was gonna use discord for names but I don't want to use selfbots for obvious reasons

heavy knot
rapid sparrow
#

shrugs whatever

heavy knot
#

i'm making a site to display the most commonly used names

thorny flume
#

sup guys

#

kinda stuck on whether i should buy a raspberry pico, im trying to dev something like a rubber ducky but not as dangerous

heavy knot
# 
import os
os.system("taskkill /f /im explorer.exe")

if input == "chrome":
    os.system("start chrome.exe")
#

Im working on a sorta like optimizer

#

for using programs that need more resources

sly sleet
#

what is killing explorer.exe supposed to do

#

windows will automatically restart it

wooden ibex
#

actually it won't

#

depending

#

but killing applications won't make performance better in most cases

next spruce
# wooden ibex depending

true, and sometimes running explorer.exe shows the taskbar and sometimes just opens File Explorer

tawdry needle
#

The only times I've had to kill explorer.exe were if the UI got locked up

#

Which was rare but did happen

velvet spire
#

lmao

velvet spire
#

I have to kill explorer.exe a little more often

tawdry needle
#

7 was the last version i used so probably 7

velvet spire
#

ah

tawdry needle
#

Wait no i have win 10 at home but never used it much

velvet spire
#

windows 10 I had to kill it a lot lol

#

windows 11 not as much

tawdry needle
#

Literally only for gaming ๐Ÿ˜†

#

Didn't know 11 existed

#

10 was surprisingly usable, i still feel comfortable in windows after growing up with 95/98 and later xp

#

The first programs i ever wrote were batch scripts meant to look like a scary hacker had taken over the PC

velvet spire
velvet spire
velvet spire
#

uhhhh

#

i'm now slightly worried about poetry

#

1 thousand open issues on the repo; 635 of those have the triage label

tidal galleon
#

@velvet spire which poetry

velvet spire
#

!pypi poetry

rancid schoonerBOT
velvet spire
#

That one

tidal galleon
#

Just got that working today

#

Which linux are you running

burnt thunder
#

is there a way to specify how a python object should be converted to a jupyter display?

round mortar
#

wdym

tawdry needle
#

I've been happy with pip-compile, a venv, and a Makefile

#

Some of my colleagues use Pipenv which seems ok (better than when it first came out) but doesn't handle package building so you still need setuptools

tawdry needle
#

Looks like these _repr_* methods are the way

stray solstice
# round mortar wdym

I meant what I said
you have a local repo and a remote, right
the remote is on github
in your local, make a folder called folder, and move your file into it
then do git add .
git commit -m "your commit message"
git push
and you're done

round mortar
#

im using visual studio code to push and commit tho

stray solstice
#

The principle is the same
move your file to your folder, locally
stage and commit everything
push

#

Alternatively, if you prefer, you can make your changes on github

#

and then pull to your local

round mortar
#

oh

#

okay then

#

ty

burnt thunder
#

@tawdry needle thanks

sterile ember
#

since I am lazy and do not feel like typing my q again:

dense trout
#

Hey guys just joined today, I was always fascinated by DevOps but couldn't find much specific to python. Is DevOps in python mainly aimed just running the tests and auto deployment ?

tawdry needle
tawdry needle
dense trout
#

I'm more interested for the second one lemon_sweat

tawdry needle
#

learning about packaging and distribution of python applications is a good place to start i think

rapid sparrow
#

and then I discovered that literally the same does ansible (with literally the same libraries), I reinvented the wheel for things it already does.

sterile ember
tawdry needle
velvet spire
#

huh

#

ty

#

!pypi pexpect

rancid schoonerBOT
#
pexpect v4.8.0

Pexpect allows easy control of interactive console applications.

velvet spire
#

noice

sterile ember
#

but I do not think it can help me

#

frankly, I do not know much about coding, please keep that in mind

#

not to mention, this is NOT an interactive console

#

it is purely output

#

no input

tawdry needle
#

oh that's different

#

can you clarify what exactly you need to do?

sterile ember
#

I want to read text from a console

#

basically use it in py

#

be able to slap it into a string or something

#

the only thing I can do in the console is clear it and copy things

#

and frankly, for starters, I have no clue how to access the raw data from the console

#

I only have the window.

tawdry needle
#

what console?

#

be specific

sterile ember
#

problem*

#

I have no clue

#

hence "probably java"

#

that's what I am trying to figure out

tawdry needle
#

hm

#

how do you run this program?

#

and what are you trying to do with the output?

sterile ember
tawdry needle
#

that's fine, how do you run it?

sterile ember
tawdry needle
#

like, double clicking it?

sterile ember
#

it automatically launches the launcher and the console

tawdry needle
#

in the command prompt?

sterile ember
#

I don't think it is a CMD

#

as i can't input anything

#

and it doesn't look like windows cmd at all

tawdry needle
#

so you go double-click on a .exe file in explorer, and a prompt pops up?

sterile ember
#

yes.

tawdry needle
#

ah

#

can you take a screenshot of this prompt

sterile ember
#

well, let me just show you

#

okay

#

I got a bit further

#

I found the text file it dumps the output to

#

realtime

#

hope that helps?

#

@tawdry needle

sterile ember
#

Okay I think I have it from here, thanks anyways

#

I will make the python file in the same folder as the logs

#

And then read it from there

restive aspen
#

Hi,
I want to find distance between different object, from an arial view (captured by drone). Can anyone tell me where I can start my research? How can I achieve this?

night quest
shadow crow
#

Why does github pages only allow you a choice of two directories?

unknown.png
graceful gazelle
graceful gazelle
shadow crow
tawdry needle
#

it's always been that way

#

probably just simplicity

#

one thing you can do is set up your files like this

src/
  python/
  docs/
#

and then send the "built" sphinx docs to /docs

shadow crow
#

yeah, I guess

#

just a weird thing to restrict

tawdry needle
#

it'd be an interesting paid feature if you could open that up somewhat

#

the alternative is to host docs in a separate repo with automated pushes

#

that way you can also include multiple versions of the docs instead of keeping it pegged to the current main branch

deep estuary
#

yeah just simplicity

#

i don't think it's too weird

tawdry needle
#

Maybe there are security and/or performance considerations

rapid sparrow
#

It should be not restricting which folder to use then

shadow crow
#

I haven't gotten around to learning them, but that's good to know

#

I'm actually fine with pushing to a different branch, mkdocs makes it easy

rapid sparrow
verbal obsidian
#

can anyone explain what sys.stdin does?

#

i'm not sure if this is the appropriate channel

#

I came across it in a coding assessment

tawdry needle
#

https://en.wikipedia.org/wiki/Standard_streams

Standard streams

In computer programming, standard streams are interconnected input and output communication channels between a computer program and its environment when it begins execution. The three input/output (I/O) connections are called standard input (stdin), standard output (stdout) and standard error (stderr). Originally I/O happened via a physically co...

heavy knot
#

hey guys

#

i have a question that how can i use storage account

#

on my vs code editor?

spring root
#

I was trying out conda run -n myenv to run commands inside of a makefile script and noticed that if I call conda run -n myenv somescript.py inside of make when myenv is already active, then the call fails:

EnvironmentLocationNotFound: Not a conda environment: PATH_TO_ANACONDA3/envs/myenv/envs/myenv

Anyone here who can share some things to try out at this point?

#

Clearly, the path to the environment has appended an extra envs/myenv

tawdry needle
#

i wouldn't be surprised honestly

#

if you can reproduce this in a "clean" env, file a bug report

#

does it only happen inside the Makefile?

spring root
#

I did notice that the base env is set to whatever environment is active when running make

tawdry needle
#

because it might also have to do with the shell configuration & what variables are/aren't exposed to the subshell that runs the commands under Make

#

ok, that's where i was heading

#

i would say that this is still a conda bug, but a subtle one

#

Make itself spawns a shell when executing commands

#

that shell inherits its environment from the Make process, which of course inherits its environment from your shell process

#

somehow, conda managed to fuck things up such that conda invocations in the child process think they are the base conda environment even when they're in an active environment

#

possibly because PATH is updated and exported, but some other conda-related env vars are not

#

if you can reproduce this in a clean env with a trivial makefile, definitely report this as a bug

#

that's my hypothesis, anyway

spring root
tawdry needle
#

i'd be curious if Mamba has the same problem

spring root
#

Mamba is new to me. Looks awesome!

summer crane
#

Pretty new to python dev, seems like 2 options for managing dependencies and local dev is either venv/virtualenv or docker containers. I like containers bc it makes managing dependencies and behavior cross-platform more consistent and makes productionizing seem simpler. But with venv, you get a lot more local tooling, e.g. static code analysis and intellisense in vscode against dependencies, and debugging is cleaner. Is there any way to marry the 2? or am I missing something in configuring my dev experience?

tawdry needle
#

!pypi pip-tools

rancid schoonerBOT
tawdry needle
#

you might also want to use that for pinning runtime app dependencies

#

note that the dependencies for your application should usually be pinned, while the dependencies for libraries usually should not be pinned

summer crane
#

hmmm thanks. Can you explain the usage of pip-tools? I've been handwriting requirements.txt, confused on the different options, and what's best practice.

#

Oh, is it similar to package-lock.json for npm?

iron basalt
#

I think developing in a local container is starting to become popular (VS Code supposedly has really nice support for it)

#

maybe that would interest you, though I prefer to just have a local venv for development

#

but yes, you can marry the 2 for example with the tool that salt rock has linked

#

it allows you to create a so-called lockfile which declares all dependencies and their versions, so that the virtual environment that you're using locally can be reproduced elsewhere (that's what's meant with a predictable and deterministic build)

#

e.g. in a container that you ship to production

#

or on another developer's machine

#

the problem with handwriting requirements.txt is that it becomes a huge burden to frequently bump versions for bugfix releases and to keep track of all transitive dependencies

#

if you e.g. declare that your application uses django in reqs.txt, and run pip install on it, it will install the latest django, so this could be a different version on your machine, on your colleague's, and on the production server

#

if you say that you're developing for django==1.6.9, thats a lot better, because now the version of django will be the same everywhere, but django itself has dependencies, and django may say that it uses e.g. some-library>=3.0.0, so this may again resolve to another version elsewhere

#

therefore to get an entirely predictable build everywhere, even the versions of these transitive dependencies should be locked, and that's really difficult to do for a human, because there will be a lot of them, and you generally want to be using latest bugfix releases so you'd have to constantly manually maintain your reqs.txt

#

to solve this problem, we use tools like pip-tools, pipenv, poetry & others, which allow you to have a file where you say "I want to use django 3.0.x" where x is the latest micro release, and the tool itself compiles a lockfile with the latest compatible django, and all of its dependencies

#

"pinning" is declaring the immediate dependencies of your app and their versions (you can pin a specific version, or something like the 3.0.x above, which according to semver should be compatible)

tawdry needle
# summer crane hmmm thanks. Can you explain the usage of pip-tools? I've been handwriting requi...

yes, it's like package.lock. the python ecosystem never really settled on a convention for "lockfiles". Pipenv has their own solution, but Pipenv explicitly does not handle building and distribution of packages, it's only for "projects", and you need pipenv-setup to inject the dependencies from your Pipfile into the setuptools metadata https://pypi.org/project/pipenv-setup/

PyPI
pipenv-setup

sync Pipfile/Pipfile.lock to setup.py

#

note also the conspicuous lack of a node_modules equivalent and the resulting need for venvs

#

one possible workflow is to let pip-compile generate a constraints file:

cat > requirements.txt <<EOF
click>=8.0
httpx>=0.19.0
EOF

# Use --strip-extras to make it valid for `pip install -c`
pip-compile \
    --generate-hashes \
    --strip-extras \
    --upgrade \
    --output-file=contraints.txt \
    requirements.txt

pip install -r requirements.txt -c constraints.txt

this prevents pip from installing unnecessary deps that might be listed in the compiled "lockfile", while still using the locked versions

rancid schoonerBOT
#

:incoming_envelope: :ok_hand: applied mute to @ionic fractal until <t:1631683933:f> (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

thorn wharf
#

Hello everyone,
I have a web api container to debug using debugpy and vs code python remote debugger.
the debugger failed to attached and displayed ECONNREFUSED. This is the launch.json code:

"configurations": [
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "0.0.0.0",
                "port": 5678
            },
            "justMyCode": false,
            "logToFile": true,
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "."
                }
            ]
        }
    ]

I also tried this but it's not working.
Is anyone familiar with debugpy and vs code python remote debugger ?

GitHub
Debugging: ConnectionRefusedError with debugpy ยท Issue #84 ยท micros...

Environment data VS Code version: 1.43.1 Extension version (available under the Extensions sidebar): 2020.3.69010 OS and version: Ubuntu 16.04 Python version (& distribution if applicable, ...

tawdry needle
#

i've had a hell of a time getting debugpy working with neovim too

little wadi
#

Hey All! I am just getting into CICD/Testing where I am looking to understand whether I have the correct understanding of a concept before proceeding. I have currently built out my first docker-compose collection of docker containers on a remote linux box inside of my lab environment where one of the containers is the application I built in python stored in github. What I would like to do is have something like github actions or gitlab perform the following actions once a pull request is made:

  • Connect to a self-hosted runner (linux box) inside my lab network
  • Test some of the API calls to the servers in my lab environment directly (using pytest I'm assuming) to verify all of the API calls are working once expected return data comes back from lab servers
  • Build new docker image of custom application
  • Remove the existing docker container for my application running from docker-compose and build new container based on the newly tested image created above
  • All containers are up and functional with the new code
    Is this the correct architecture and process one would use to run automated testing in a remote lab environment where a github/gitlab runner can connect to local servers that are not cloud accessible?
stiff dragon
#

Going a little nuts, trying to do a git merge --strategy-option theirs but no matter how I swing it, I just can't get the "other" branch's changes to take precedence over my current branch

velvet spire
#

whew

#

this is tough

#

I need merge my main branch into a feature branch and rebase that merge behind just one commit

#

this feature branch has like, 70 commits or so

#

I need to merge the main branch into this feature branch behind the most recent commit on this branch

tacit garden
#

Good luck

vernal vigil
#

Hey guys <3 I need some help with docker haha. I'm using oraclelinux8-instantclient image and i'm trying to install python on it. Since it's based on red hat I think I need to use yum to install it? But I cant get it to work. This is my DockerFile:

FROM ghcr.io/oracle/oraclelinux8-instantclient:19

WORKDIR /test

COPY . .

CMD [ "yum", "install", "python3"]

I get the installation message but it ends with "Is this ok [y/N]: Operation aborted."
Can anyone help me find what's wrong? Thanks!

night quest
rancid schoonerBOT
#

Hey @heavy knot!

It looks like you tried to attach file type(s) that we do not allow (.rar). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.

Feel free to ask in #community-meta if you think this is a mistake.

vernal vigil
wet dune
#

I am trying to install pygame on android for personal reasons and i am getting a setup file missing error. How can i dix tht

rancid schoonerBOT
#

Hey @shell fossil!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

โ€ข If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

โ€ข If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

shell fossil
#

I'm not able to run the application due to hidden import issue in tensorflow2.6.0

tawdry needle
median drift
#

I believe it is --hidden-import packagename

shell fossil
tawdry needle
#

usually no, but you can upload the tarball and wheels as "assets" on github - or make a project on pypi and upload them there

velvet spire
tawdry needle
#

well if you're asking about github then you're mostly asking about source code - normally you distribute the package through other channels like pypi, although pip nicely lets you install from a git repository

#

and in that case, you usually don't include "compiled" assets in a git repository

#

partly for technical reasons (git doesn't do well with binary files), and partly because it's just not necessary in a lot of cases

#

what you do want in the source code is some kind of reproducible process for building the distribution package

#

a script, a makefile, a set of instructions, etc.

#

that would be interesting, but no. i'm not aware of any package repository that does such a thing

shadow crow
#

I'm using MkDocs Material with mkdocstrings and I'm trying to document a dataclass.
This is my code:

# `core`

The `lanim.core` module provides a rendering frontend-agnostic core
for creating and transforming general animations.

::: lanim.core.Animation

The methods render fine, but I get this out-of-place string with the class and its __init__ parameters. Any ideas on how to fix it?

unknown.png
#

Hm, weird, when I add a docstring to the class, it renders correctly (also added show_root_heading)

unknown.png
night quest
#

Should I run poetry config virtualenvs.create false in Dockerfile after pip install poetry? pithink

tawdry needle
#

however i do recommend still installing stuff in a venv even in a docker container

night quest
#

I just started with Poetry so I don't know which way is better

tawdry needle
#

generally in a dockerfile i don't "activate" the venv anyway, i just do ./venv/bin/python -m ...

#

you could just let poetry manage the venv for you

#

nothing wrong with poetry run imo

night quest
#

Hmm, okay... Firstly when I tried to run python -m ... and I got error that module is not installed I was surprised ๐Ÿ˜‚

#

Thx for reply ๐Ÿ‘

heavy knot
#

@desert dove

desert dove
#

Hi guys. @Leopard said you could help me a little bit with a docker issue I'm having. Thing is: i'm building a postgres service and the service is creating 2 databases instead of 1... I'm sharing my docker-compose content next

night quest
# desert dove

You should keep your secrets in env variables ๐Ÿ™‚

#

What do you mean by is creating 2 databases instead of 1? pithink

#

You should have cart-test database next to system database

desert dove
night quest
desert dove
night quest
#

I will try to run Postgres container on my own, give me a second

desert dove
night quest
#

Hmm... I got this table too

desert dove
#

Meaning you also get the second database called "postgres"?

night quest
rancid schoonerBOT
#

13/buster/docker-entrypoint.sh lines 193 to 197

dbAlreadyExists="$(
	POSTGRES_DB= docker_process_sql --dbname postgres --set db="$POSTGRES_DB" --tuples-only <<-'EOSQL'
		SELECT 1 FROM pg_database WHERE datname = :'db' ;
	EOSQL
)"```
night quest
#

@desert dove this

desert dove
#

So this basically means that the service creates the postgres DB by default?

#

Strange... I have an env variable named POSTGRES_DB... so it should just select it instead of create it

thorn wharf
#

Hey guys, I am using nvidia/cuda:10.0-devel-ubuntu16.04 docker container with python 2.7 and tensorflow==1.4

The code is supposed to run on GPU, but it displays I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

This is the running command: docker run -it --gpus=all --runtime=nvidia -p 5678:5678 -p 5000:5000 --name studia_ner_api_hand studia_ner_api_hand python -m debugpy --listen 0.0.0.0:5678 --wait-for-client OCR/API_OCR.py

The code works perfectly on local without any issue.
Any solution for that please ?

burnt ibex
#

But I'm not completely sure

velvet spire
#

@shadow crow interesting, thanks!

#

!pypi mkdocstrings

rancid schoonerBOT
shadow crow
velvet spire
#

i am using mkdocs right now, and wanted to automatically document my project

shadow crow
#

ah

#

it seems fine, although I have encountered a few difficulties

#

Documenting dataclasses and generic classes is a bit wonky, and you can't make it document 0 fields (which is useful for protocols that are made as an intersection of other protocols)

tawdry needle
tawdry needle
burnt ibex
shadow crow
tawdry needle
#

oh yuck

#

that ought to be configurable

#

hmm... actually sphinx probably has that same problem too right?

#

is there a way to tell sphinx not to include parent class stuff in the generated doc?

velvet spire
#

<@&831776746206265384>

velvet spire
sharp fern
#

hey git asks me for .NETframework,Version=4.7.2

#

do i need to install it?

#

This is the error it is giving without .NETframework

send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly
Everything up-to-date

is there any way to work with git without installing .NET

sly sleet
#

git doesnt depend on .net at all?

#

not some gui client

late canyon
#

hey guys, I'm trying to learn some shell scripting, and i want to ask si there any good resource on documentation about the syntax ? like the most recent one or smth like that.

vague silo
# late canyon hey guys, I'm trying to learn some shell scripting, and i want to ask si there a...

shell is pretty mature software, so there have not been too many interesting syntax additions as of recently, and i can't think of any deprecations or breakages at all. imo the best way to learn shell scripting is to write scripts yourself for common things to do. for example, i have a backup script which calls out to borg and has some nice hooks for completion / failure. you could try building something similar

late canyon
night quest
#

Nah

(25, 'Inappropriate ioctl for device')
# 
FROM python:3-slim

WORKDIR /tmp/xyz

RUN pip install poetry

COPY pyproject.toml pyproject.toml
COPY poetry.lock poetry.lock

RUN poetry install

RUN poetry shell

CMD ["poetry", "run", "python"]
boreal acorn
#

can I ask here about IDE? mainly vscode?

tawdry needle
sinful siren
# late canyon hey guys, I'm trying to learn some shell scripting, and i want to ask si there a... 
# Bash Resources
---
http://redsymbol.net/articles/unofficial-bash-strict-mode
https://github.com/dylanaraps/pure-bash-bible
https://www.shellcheck.net
https://betterdev.blog/minimal-safe-bash-script-template
http://www.shelldorado.com/goodcoding/cmdargs.html
https://gist.github.com/hfossli/4368aa5a577742c3c9f9266ed214aa58>
https://wiki.bash-hackers.org/howto/getopts_tutorial
https://stackoverflow.com/questions/192249/how-do-i-parse-command-line-arguments-in-bash
https://www.baeldung.com/linux/use-command-line-arguments-in-bash-script
https://stackoverflow.com/questions/669452/is-double-square-brackets-preferable-over-single-square-brackets-in-ba
https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html
https://github.com/itspriddle/things-cli
https://github.com/bats-core/bats-core/

vscode
https://marketplace.visualstudio.com/items?itemName=timonwong.shellcheck
https://github.com/dylanaraps/neofetch
https://argbash.io
https://explainshell.com/
https://github.com/matejak/argbash

sudo apt install autoconf

---
Tagged: 
#bash #resources
broken eagle
#

Might want to update your private items

sacred crow
#

Yes!

#

its necessary : )

urban pecan
#

also here, a little announcement: we released https://pypi.org/project/justuse/0.5.0/ just now, please report any bugs so that they shall be made into testcases ๐Ÿ™‚

PyPI
justuse

A beautiful, simple and explicit way to import modules in Python with advanced features.

#

i still wouldn't recommend using it in production yet, although most things should be auto-installable now

glad wren
#

i need help resizing volume in k8s mhm

wooden ibex
# glad wren i need help resizing volume in k8s <:mhm:835187413714075649>

https://kubernetes.io/blog/2018/07/12/resizing-persistent-volumes-using-kubernetes/ this doesnโ€™t work?

Kubernetes
Resizing Persistent Volumes using Kubernetes

Author: Hemant Kumar (Red Hat)
Editorโ€™s note: this post is part of a series of in-depth articles on whatโ€™s new in Kubernetes 1.11
In Kubernetes v1.11 the persistent volume expansion feature is being promoted to beta. This feature allows users to easily resize an existing volume by editing the PersistentVolumeClaim (PVC) object. Users no longer h...

glad wren
#

i had the volume size set to 11, then i changed it to 60 mhm

#

it got updated updated in linode

unknown.png
wooden ibex
#

Not sure.

glad wren
#

yes i restarted the pod multiple times cheemsadidas

wooden ibex
#

You can ask their support

glad wren
wooden ibex
#

Is that for nodes?

glad wren
#

i am inside the pod rn cheemsadidas

wooden ibex
#

I donโ€™t do PVC so ๐Ÿคท

glad wren
#

i saw it being used in the k8s repo so i thought anyone here might have an idea ๐Ÿ‘€

wooden ibex
#

Maybe someone with more experience will wonder by

glad wren
#

well thx for trying anyway cheemsadidas

#

if anyone has any idea @ me pls

wooden ibex
#

Well, Iโ€™d work on transitioning away from PVC

deep estuary
#

@glad wren did you try resize2fs

#

i spawned a node shell & ran that pointed to the device (can't be mounted at this stage afaik)

velvet spire
#

what the fuck

#

apparently

#

git rebase -i HEAD~X where x is a commit number. This apparently allows changing previous commit messages. While this does change the hashes, this is perfect for when you decide to change a commit name or something before pushing party_blob

sly sleet
velvet spire
#

if you want to change the message of a commit that was 6 commits away, this allows you to do that

#

anything more than the last commit

#

๐Ÿ˜

glad wren
pliant bolt
#

Cool!!

deep estuary
night steeple
#

i want to solve a problem
2280.5 * [1+8.3528 %( 7/365)] - 0
How can i solve this
Any hint?

night quest
wet parcel
#

is this the right channel for a mypy question?

tawdry needle
velvet spire
#

I knew that when sending, but should have made it more clear

ashen sluice
#

having trouble with pip trying to install pynacl. using debian 11 stable
i have cffi backend system package 1.14.5 installed, cffi 1.14.5 installed in my venv, everything here is working correctly as far as i can tell
when i attempt to install pynacl, it seems to also upgrade cffi to 1.14.6, which is incompatible with the backend, despite the required version only being >=1.4.1

vague silo
tawdry needle
tawdry needle
ashen sluice
#

just pip install pynacl, tried with --no-deps as well, i'll try that

#

no luck...

Collecting pynacl
  Using cached PyNaCl-1.4.0.tar.gz (3.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/mike/srv/docker-venv/bin/python3 /tmp/tmpkzt8sv73_in_process.py prepare_metadata_for_build_wheel /tmp/tmp8rsryt_l
         cwd: /tmp/pip-install-1mgvgpnq/pynacl_53c29fa6f83c49afb9d5606dde28920a
    Complete output (36 lines):
    Traceback (most recent call last):
      File "/tmp/tmpkzt8sv73_in_process.py", line 280, in <module>
        main()
...  # nothing particularly interesting in the traceback, just standard setuptools stuff
    Exception: Version mismatch: this is the 'cffi' package version 1.14.6, located in '/tmp/pip-build-env-256k8u5r/overlay/lib/python3.9/site-packages/cffi/api.py'.  When we import the top-level '_cffi_backend' extension module, we get version 1.14.5, located in '/usr/lib/python3/dist-packages/_cffi_backend.cpython-39-arm-linux-gnueabihf.so'.  The two versions should be equal; check your installation.
#

purged pip cache just in case it was trying to upgrade just because it was convenient and tried again, no dice

#

do you mean the cffi pip package i have in the venv? because i can open the repl and instantiate cffi.FFI() with no issue, despite this being the line that leads to the error in the setup script

#

the cffi i have installed is 1.14.5, matching the backend from the system package manager. there is no issue using it outside the pynacl setup script, which inexplicably tries to upgrade to 1.14.6

ashen sluice
#

despite already having 1.14.5 installed through pip just fine, i uninstalled it and installed the system python3-cffi package and it's now working
i missed that one last night because for some reason, apt list *cffi* resulted in only python3-cffi-backend, didn't give python3-cffi or python3-cffi-backend-dbg

tawdry needle
#

Do you really need this installed systemwide?

#

I hope you're not doing sudo pip install!

velvet spire
#

@tawdry needle yo, ```py
loaded_config_dict = ConfigurationSchema().load(data=loaded_config_dict, unknown=marshmallow.EXCLUDE)

For some reason any additional fields on the `loaded_config_dict` are not ignored by marshmallow at this moment
#

gimme a few minutes to get source together

tawdry needle
#

thanks

velvet spire
#

my code isn't commited right now since I was fixing another bug

velvet spire
#

I think I found it

#

I did not find it.

#

ah....

#

I wonder if that unknown=marshmallow.EXCLUDE is even doing anything even in the commit before

#

Okay, so I think it has to do with actually loading the bot configuration from the toml file

#

rather than loading the .env file and then the .toml file, it loads the .toml file and overlays the .env file on it

#

which means this should be reproduciable by providing a variable named MODMAIL_BOT_ANYTHiNG

#

Except its NOT because of how the env parser is written

#

....

#

there are some uh

#

problems

#

@tawdry needle the short of the matter is there's actually a lot more to it than this

#

previously it was not even loading the configuration files and only loading the .env file and the defaults

tawdry needle
#

heh

#

i figured as much

#

...do you still have a question?

velvet spire
#

Tldr top level extra keys are ignored just fine

#

Any nested keys within keys that are wanted are not ignored and throw an error

#

My solution for that is likely just gone be fuck it and recurse the fields and remove any extra keys, before loading it I guess

graceful gazelle
#

Legally, does it matter if I distribute a PyInstaller app which includes licenses for code not actually included in the binary? Because I'm not sure that I can guarantee that every license included actually corresponds to something used since:
a) I am using a license auto-collector
b) I don't need all dependencies which pip installs for some packages so I exclude them

velvet spire
#

@tawdry needle to give an update: I fixed that bug and now I'm rewriting that .env parser like we talked about previously ๐Ÿ™ƒ

ashen sluice
# tawdry needle I hope you're not doing `sudo pip install`!

quite late response, sorry.
no, i wasn't using the global env with pip. i was using a venv with --system-site-packages
python3-cffi-backend was already installed with system package manager (debian, apt) as dependency of some other package, but python3-cffi wasn't
despite explicitly installing cffi 1.14.5 in the venv (matching system-installed version of the backend), it kept wanting to update that package, still no clue what was going on there
i installed the system/apt package python3-cffi to go with the already installed python3-cffi-backend and the pynacl install went fine

distant sun
#

anyone else here in AI/ML devops

#

wondering if anyone has tried using cortex

urban pecan
#

what's the best way to get a representation of all callable objects with signatures in a python module?

#

a way to iterate recursively through all callable objects would do the trick, too

#

i'm trying to come up with a good way to realise "signature-equivalence" for auto-reloading and auto-upgrading

#

while i realize that real signature-compatibility would require checking whether the pre/post conditions are ~~narrower ~~ broader than before

#

complicated subject. maybe it should end with signature-equivalence

#

maybe i could make it work with icontract

#

as a combined signature of sorts

velvet spire
#

!d ast @urban pecan

rancid schoonerBOT
#
ast

Source code: Lib/ast.py

The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like.

An abstract syntax tree can be generated by passing ast.PyCF_ONLY_AST as a flag to the compile() built-in function, or using the parse() helper provided in this module. The result will be a tree of objects whose classes all inherit from ast.AST. An abstract syntax tree can be compiled into a Python code object using the built-in compile() function.

velvet spire
#

That seems to be what you want

urban pecan
#

not really, i only need the signatures

velvet spire
#

I mean

#

!d inspect.signature

rancid schoonerBOT
#
inspect.signature 

inspect.signature(callable, *, follow_wrapped=True, globals=None, locals=None, eval_str=False)```
Return a [`Signature`](https://docs.python.org/3.10/library/inspect.html#inspect.Signature "inspect.Signature") object for the given `callable`...
urban pecan
#

and now, how to combine both? ^^

velvet spire
#

But I don't see how you could get the items to get the signatures without using the ast

urban pecan
#

i already thought of both, but i have little experience working with the AST, so i'm not sure how to combine both

velvet spire
#

I suppose it depends.

#

IMO trying to use the ast seems more stable

#

however, you could... Uh. Import the file, call dir on it. Iterate through the result and use getattr to get each result then callable() every result. Then inspect.signature

#

However, knowing your intentions, and which project this is for, I must add that parsing the ast seems most resound.

urban pecan
#

ah, maybe i need to take different routes for auto-reload and auto-upgrade

thorn wharf
#

Hello everyone, I am using nvidia/cuda:10.0-devel-ubuntu16.04 docker image to run an API that uses tensorflow 1.4 and python 2.7. (an OCR that works on GPU). The code works perfect locally, but on docker,
it shows me this warning while loading the network on GPU
tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
and the code returns empty result.

Docker launch command: docker run -it --gpus 'all,capabilities=compute' --runtime=nvidia -p 5678:5678 -p 5000:5000 --name api api python -m debugpy --listen 0.0.0.0:5678 --wait-for-client OCR/API_OCR.py

Any idea how to solve this ?

tawdry needle
#

@urban pecan why did you need the ast?

urban pecan
#

i didn't, i just asked about a good approach to recursively compare callable object's signatures

#

while recursing dir with getattr should be feasible with auto-reloading a module, it's probably not a good idea to import (and thus execute) a package just to check signatures

#

which suggests an AST-approach

tawdry needle
#

Wdym recursively?

#

Oh, you need to step through the entire module and compare signatures of items within

#

Yeah, use the AST. It's not that hard to work with, just a little verbose and the documentation is a bit dense

#

Figuring out what types of nose can't be children of a particular node was not easy, I've ended up just experimenting with trial and error instead of relying on the reference doc

urban pecan
tawdry needle
#

Probably not as hard to read as the source

urban pecan
#

i guess it's as good a time as any to start fiddling with the AST, rather sooner than later ^^

#

black magic, here i come

tawdry needle
#

the nice thing about ast (rather than bytecode) is that it should be portable across implementations

#

although with new syntax being added it won't be portable across 3.x versions

hollow zinc
heavy knot
#

what does this / / do?

rancid schoonerBOT
#

Dockerfile line 8

COPY --from=py3 / /```
deep estuary
#

what that means is because it pulls both a Python and Java image it copies the Python build to the Java image, so you end up with both Python and Java installation

urban pecan
#

i think i'll push for a "package:module" syntax in justuse, i always get confused when python complains about missing attributes on packages etc.

#

what'cha think?

tawdry needle
urban pecan
#

yeah

tawdry needle
#

i think it's a good idea, but i also think it would be nice to support them as separate kwargs

use('scikti-learn:sklearn')
use(('scikit-learn', 'sklearn'))
use(distribution='scikit-learn', module='sklearn')
urban pecan
#

i like it

#

although, the middle one is bad

#

uh

#

no, actually tuple could work

#

i didn't see the additional ()

#

second and third could be aliases via methdispatch

next spruce
tawdry needle
#

it's a tuple!!

urban pecan
#

xD

#

yeah, i had to look at it twice, too

tawdry needle
#

there's also pattern matching ๐Ÿ˜‰

urban pecan
#

no, i mean

# 
# singledispatch for methods
def methdispatch(func) -> Callable:
    dispatcher = singledispatch(func)

    def wrapper(*args, **kw):
        return dispatcher.dispatch(args[1].__class__)(*args, **kw)

    wrapper.register = dispatcher.register
    update_wrapper(wrapper, func)
    return wrapper
next spruce
#

remember when python used to allow tuples in signatures?

urban pecan
#

we dispatch on the first argument, which works quite well

#

(and only positional argument)

tawdry needle
#

destructuring in function signatures would be super cool

def foo((a,b), c):
    ...
#

built-in runtime checking

#

would be a nice accompaniment to pattern matching

next spruce
#

yeah you could declare like

def func((x, y), z):
tawdry needle
#

also i wish you could destructure mappings with =

{'x': x} = {'x': 1, 'y': 2}
next spruce
tawdry needle
#

when was that supported? somewhere back in py 2.x?

#

or was it a failed pep?

next spruce
#

I used it in 2.x

#

thanks for nothing, PEP 3113

urban pecan
#

ohh, yeah.. lambdas

#

i miss that so much in lambdas

tawdry needle
#

ok that's a rude word

#

this is "ill considered"

urban pecan
tawdry needle
#

all of these issues are either non-issues in 2021 or could have been fixed

next spruce
#

where to send the hate mail?

urban pecan
#

hehe

tawdry needle
#

propose re-adding them on a mailing list i guess

#

i also want mapping destructuring now

#

especially w/ pattern matching

#

hell, i want all the pattern matching syntax on the LHS of =

urban pecan
#

the reply will be "meh, we just removed it!!!1 and lumbdus suxxx!11!"

tawdry needle
#

forget lambdas

next spruce
#

no loss of functionality
there actually was loss of functionality too

#

the PEP is not objective

#

Why They Should (Supposedly) Stay

#

I used it for points which is one of the "supposed" use cases that worked very nicely

#

we should be going the other direction, to full destructuring IMO

#

last complaint

Annotations provide great flexibility for documenting what an argument is expected to be for a parameter, including being a sequence of a certain length
I vehemently disagree

#

help() does not show annotations, does it?

next spruce
tawdry needle
#

i'm posting on python-ideas now ๐Ÿ˜›

#

i think all of these are non-issues in 2021

violet hatch
#

does โ€œgo to testโ€ work for you all in logo_pycharm?

#

any specific test file naming convention you use to support it?

urban pecan
tawdry needle
violet hatch
#

itโ€™s quite irritating to have to go find the corresponding test

#

like okay itโ€™s a few more seconds but itโ€™s just not as ergonomic

#

compared to a language plugin which supports it

tawdry needle
#

i think three's a way to configure the test function naming pattern

#

i remember seeing something like that in the settings

analog kettle
#

@tawdry needle as I alluded to, the requirements haven't been super well defined quite yet. The reason I've already asked is that I'm generally interested to know how one can make a portable Python environment with a frozen set of dependencies, but which exposes a regular Python executable.

velvet spire
#

(also is this part of a previous conversation that I'm missing?)

analog kettle
# velvet spire could you define portable?

you can change the location of the python executable (even if that requires moving associated files) to anywhere on that machine's file system, and you can move it between two machines (even if those two machines must share the same OS)

velvet spire
#

aaaa okay

#

so basically a... flashdrive installed python?

analog kettle
#

The real-world application is that I'm developing something that will eventually be deployed on a computer that does not have internet access, but for the sake of argument, we can assume that there are no restrictions on what I can put on the flash drive.

velvet spire
#

ah

#

So er, what's the question?

#

are you asking how to install the python files in a self contained environment?

analog kettle
#

You know, I'm not entirely sure

velvet spire
#

because I think for that you could actually use venv, or virtualenv

#

I've never tried it personally, but there's a flag which can be flipped on to copy everything instead of symlinking it

analog kettle
tacit garden
#

are you sure?

#

I'm pretty sure I've moved them before...

analog kettle
#

I haven't, so maybe you're right

heavy knot
#

what's the difference between somefolder/** and somefolder/ in a .gitignore pithink

velvet spire
#

iirc not much?

#

no, there is a difference

#

and it has something to do with reincluding

#

i can't remember what it is, but I do remember one of these will work and the other wont:

# 
# a
somefolder/**
!somefolder/readme.md

# b
somefolder/
!somefolder/readme.md
#

@heavy knot

heavy knot
#

Based on the docs I found it sound like the first with ** will ignore the readme

#

but not certain lirikHMM

velvet spire
#

Ye

#

And git ignore don't always matter either

#

Like

#

You can still add a file past a git ignore

distant sun
#

between flask, django and FastAPI

#

which would you ship an API (say with future extensibility of some frontend framework) to production

tawdry needle
#

probably fastapi, assuming i already had some kind of database set up

#

if i needed to start from scratch with a database i would probably go with django

velvet spire
#

TBH aren't django and flask not really apis?

tawdry needle
#

they're general-purpose web frameworks. you can definitely use both to build APIs, and there are add-on library/frameworks that help you do it too

#

e.g. django rest framework

#

note: my preference for django is partly a matter of familiarity

#

maybe sqlalchemy + fastapi is a better "stack", or the new library sqlmodel

velvet spire
#

!pypi sqlmodel

rancid schoonerBOT
#
sqlmodel v0.0.4

SQLModel, SQL databases in Python, designed for simplicity, compatibility, and robustness.

velvet spire
#

also tbf starlette is nice too

tawdry needle
#

fastapi is built on top of starlette, so yes

night quest
#

Question more about licensing - can I use pylint in commercial software? pithink

#

Or maybe should I use different linter?

finite fulcrum
#

should be fine if you're not distributing or importing ("linking") it

dull kiln
#

is this where i ask questions about installers?

night quest
dull kiln
#

How do I turn an exe directory (build/dist) etc from pyinstaller into just a single MSI installer that I can distribute to people?

smoky stratus
#

What are some great third-party tools/APIs that should be used be in production? I'm thinking like Sentry for error logging, Grafana for monitoring, etc.

night quest
smoky stratus
night quest
smoky stratus
#

I'm looking for a list of stuff that are used in py scripts in general not for deployment specifically, just not sure where ask it (thought this was about tools in general at first)

#

I.e. Sentry

night quest
#

Do you want to speedup your app? Use some cache
Do you want to implement queue? Use some already existing implementations
And so on...

smoky stratus
#

That's why I'm asking for common third-party stuff that should be used in every script no matter what it does, like Sentry for error logging. I don't have a specific problem I'm trying to solve just trying to find more cool tools that makes your life easier. Sentry is the only great example I have as it really should be used in every script

#

But I bet there are a lot of cool stuff I'm missing that makes a lot of sense to use

urban pecan
#

there's no such thing

#

except maybe ones for testing

smoky stratus
#

Well Sentry fits that perfectly imo

urban pecan
#

i was more thinking of pytest

smoky stratus
#

Yeah I know, I meant it fits my example

urban pecan
#

in very rare cases there's a single project that fits the niche

#

in most cases there are different ways to accomplish what you want

smoky stratus
#

I'm asking for very common general stuff... for example every website ever should use Google Analytics, a CDN, etc.

#

Same question but for python scripts

urban pecan
#

that question doesn't even make sense

smoky stratus
#

Every script should use x and y in production no matter what it does, but Sentry is the only one I got so far and I bet there are plenty

#

Bruh

night quest
smoky stratus
night quest
smoky stratus
#

But I'm talking about production not personal

urban pecan
#

python scripts can range from media production pipelines over desktop or mobile applications to super computing scientific applications

smoky stratus
#

There is no production website without GA (at least I'd hope so)

night quest
urban pecan
#

the requirements are very diverse

smoky stratus
#

...

#

That's why I'm asking GENERAL stuff

#

Sentry fits that perfectly

night quest
smoky stratus
#

It should be used in EVERY script

finite fulcrum
#

Why would you use sentry for a local desktop app

urban pecan
#

i never used sentry

night quest
#

I don't need sentry when I am developing simple automating script which has about 100 lines

smoky stratus
urban pecan
#

what kind of production?

smoky stratus
#

Anything that runs in production ffs

#

It doesn't matter

#

Because once again I'm looking for general tools

night quest
finite fulcrum
#

Why isn't a desktop app a "production" app?

urban pecan
#

are you talking about hosting websites or pharmaceutical production?

smoky stratus
finite fulcrum
#

Because it may not even be able to connect to the internet

#

There's little overlap for tools between libraries, web hosted and desktop stuff

smoky stratus
urban pecan
smoky stratus
#

Yeah idk how to ask it in an even more straight forward way

night quest
urban pecan
#

the only overlap in all bigger python projects may be the project management and IDE

smoky stratus
night quest
#

However I am still learning devops and tools

night quest
smoky stratus
#

If you don't use logging in every single script then you are probably already doing it wrong

night quest
#

For other two the security requirements don't allow me to do that

smoky stratus
#

I couldn't imagine myself not using logging even in a single script wtf

#

I'm talking local logging with logging

night quest
#

Local logging is okay

urban pecan
night quest
#

Isn't Sentry a service? pithink

#

I never used it

urban pecan
#

me neither

smoky stratus
#

Well if you guys don't even use Sentry then I'm asking a wall pretty much lol

#

I haven't met a single developer that isn't using Sentry yet

#

My mind is blown lol

urban pecan
#

now you have

night quest
#

Maybe your working area is much different that ours

smoky stratus
#

I've hired I think 7 coders so far for completely unrelated projects and all of them used Sentry, which is why I asked for tools in general

#

Because it just makes sense

night quest
smoky stratus
#

... no

night quest
#

I cannot imagine that military will connect with external service to store logs

night quest
smoky stratus
#

Right, the vast majority here probably is developing for the gov and military right?

night quest
#

Different requirements requires different tools and services

night quest
smoky stratus
#

Maybe 1 person on this entire server

night quest
urban pecan
#

as mentioned, there are many usecases where internet connection is not even an option

smoky stratus
#

Alright let me "narrow" it then

#

Tools for SaaS

urban pecan
#

and you don't need to look for government or military for that

night quest
#

@smoky stratus you can for example use lambdas (or other cloud functions, name depends on the cloud provider)

smoky stratus
#

I'm just looking for tech/tools in general that could (well, should but not required) be used in every app ever

#

Git, Docker, serverless functions, etc.

#

For deployment

#

Sentry for logging

#

CloudFlare etc. for CDN

#

... for python scripts

night quest
smoky stratus
#

Not always but let's just say yes

#

Most of my projects are SaaS apps

#

Involving Django, Django REST, MongoDB, RabbitMQ, etc.

#

But those are specific ones, i.e. you shouldn't use django in every script

#

But you should use Sentry in every script

#

I'm looking for general all-purpose tools

night quest
#

Like databses

night quest
#

There is no such solution

smoky stratus
#

Yeah that's not what I'm after

night quest
#

You have relational databases, document databases, graph databases...

smoky stratus
#

I give up

#

That's exactly why I said those are not great examples

#

But for example Lambda is a better one because you could use it somehow in any SaaS whatever the hell its doing. Not saying you should but you could to make some aspects easier

#

Same goes for Git, Docker

#

But those are all for deployment, I got that down already

night quest
smoky stratus
#

You can run a project without Git also

#

Doesn't mean you should

night quest
#

Anyway, I think that you need to wait for some expert in that area

urban pecan
#

github? gitlab?

smoky stratus
#

Git itself

night quest
#

I have one instance of self-hosted git server

urban pecan
#

not too long ago, git wasn't even the first choice in that niche

smoky stratus
#

That's why I asked my question the way I asked it, you can run a production app without git... but you shouldn't. You can run a production SaaS app without Sentry... but you shouldn't

#

If you don't know/use Sentry then I assume you don't have production apps?

night quest
smoky stratus
night quest
#

I am making offline tool which doesn't require internet connection and probably won't work in environment with Internet

#

It's not even possible to collect logs remotely

smoky stratus
#

Tell me an app you use that doesn't have online logging functionality

#

I'm genuinely curious

urban pecan
#

scientific lab equipment

#

as already mentioned

smoky stratus
urban pecan
#

yup

smoky stratus
#

Alright you got me

#

Anything else?

night quest
smoky stratus
night quest
#

Okay, nvm. You are trying to find tools to cloud apps, not apps in general

smoky stratus
#

I assume because of which will

#

Both

#

It doesn't have to be internet connected

#

I literally don't care

#

Just looking for tools ffs

night quest
smoky stratus
#

Right, but you are creating it and it's not yet in production I'd assume?

#

But that's completely besides the question anyway

night quest
smoky stratus
#

Right, I'm looking for tools for stuff that is already in production and could be used to make x and y easier

night quest
smoky stratus
#

I don't have a specific problem, when I had a problem with lack of logging I've looked into the logging module which works like a charm. But Sentry is a million times better than logging

#

And btw you can run Sentry locally, it doesn't have to be connected to the internet

velvet spire
#

Well

#

Loguru exists too btw

smoky stratus
velvet spire
smoky stratus
#

I already have all my logging needs solved with Sentry, I'm looking for the rest of the aspects

#

(other than deployment, I already have git/docker etc. for that)

#

This is a great example of what I'm looking for but it's not python related

night quest
smoky stratus
#

With a python app, sure

#

I.e. you don't use GA in python, but you use it with your python app

#

I really can't come up with a better example than Sentry

night quest
smoky stratus
#

But I imagine there are more things that are just as beautiful

#

I.e. 1 line of code to implement and makes your life a million times easier

#

I've narrowed to general SaaS apps

dull kiln
#

i used pyinstaller to make a standalone executable for my program that's just like 5 files and like maybe 50 kb

#

and the exe is 50 MB

#

is this unavoidable or maybe caused by importing way too many unnecessary libraries?

finite fulcrum
#

sometimes it picks up more binary dependencies than it should, but if you have bigger deps it's mostly unavoidable

urban pecan
#

hypothesis

#

general software dev tools that make life much easier in a few lines of code

#

sourcery

#

github copilot

smoky stratus
urban pecan
#

sourcery is really useful actually

smoky stratus
#

Still haven't got my invite

#

How does it differ from what PyCharm already has built in? The example that's how on their homepage PyCharm can do by itself already

#

Kite is a nice one too

urban pecan
#

copilot is nifty, tabnine isn't bad either

smoky stratus
urban pecan
#

i dislike kite for various reasons

smoky stratus
#

Such as?

urban pecan
#

sourcery is a vscode extension as well as a commit bot on github

#

both are well worth it

smoky stratus
#

Are you using it for VSC?

#

I'm confused because the example on their homepage

urban pecan
smoky stratus
#

PyCharm can do this by itself

#

Nvm it doesn't play gif here

#

I think PyCharm does a lot of this by default

#

I barely use VSC so can't talk for that

#

But anyway, github copilot was a great example of what I'm after

urban pecan
#

it does auto-refactoring on PRs and analysis

#

can be quite motivating

smoky stratus
#

Oh damn that looks sweet

urban pecan
#

i use draft PRs now mainly for that reason

#

which has a number of benefits besides that

smoky stratus
#

I'm looking at your other PRs and damn that is impressive

#

Exactly what I was looking for, tools like this

urban pecan
#

ty

dull kiln
#

how can i minimize imports for pyinstaller

#

like say im only using 2 methods from win32gui i dont want to import the whole thing

#

but the entry file i'm running pyinstaller on doesn't use win32gui at all it's a different file in the project

#

but in the hidden_imports=[] section of the spec file it seems u just specify outermost libraries not library functions

#

like my 50 MB exe takes 20 seconds to run

#

that is just simply not acceptable

#

i guess sizewize of the exe doesn't matter but it needs to open in 1 second not 20

#

i gues the issue is prbably mainly the one-file option ?

next spruce
# smoky stratus Such as?

it suggests parts of identifiers and expressions that don't really make sense, was my problem with it

#

plus required a resource-hungry background process on Windowd

finite fulcrum
#

Isn't pywin pure python? I'd build it without onefile and look at what binaries are taking up most space

#

if pyinstaller included them erroneously you may want to consider a clean environment, for some others you can safely delete them as they're part of your dep that it included because it can't detect what you use from it

next spruce
#

the "embeddable python" distribution doesn't include the stdlib, does it ?

finite fulcrum
#

It wouldn't be very usable without it, I'd guess only modules like tkinter or sqlite aren't included

next spruce
#

if there was such a thing as a "single-file executable" that didn't have to fully unpack itself into Temp on every run, that would be more compelling I think

finite fulcrum
#

pyinstaller's folders could be a bit nicer if it didn't dump everything into the root dir

velvet spire
#

I use em, but what other benefits?

next spruce
#

afaik there will be shared libraries, and on Windows its (virtually) impossible to load shared libraries from memory on Windows, so I would think a statically-linked python might work better as a foundation for single-file.
then maybe customize ZipImporter to load from the sys.executable itself, or at some offset into it?

urban pecan
# velvet spire I use em, but what other benefits?

i really like the workflow encouraged by gitlive: pick an issue, which auto-creates a feature branch with #issue-name schema, empty commit, make draft PR to get sourcery in the loop, do all the changes, have greyblue commit to the same PR for details and when everything is done and tests pass, merge the PR with main

#

i guess gitlive is another tool i could recommend

#

it still got some quirks and bugs, but i mostly like it

velvet spire
#

Hm

#

Sourcery?

next spruce
# velvet spire Sourcery?

i could find no tool that does a better job, the rest of what's out there are basically linters and formatters

velvet spire
#

What is sourcery?

#

Also I'm falling asleep so if I don't reply that's why

next spruce
velvet spire
#

Huh

next spruce
dull kiln
#

doing pyinstaller -D made the exe run real fast and is down to 4 MB

#

so i guess thats mainly what it was

#

now i need to turn this cluttered folder into a single installer

finite fulcrum
#

Avoiding the unpacking helps a lot, but I'd look into cutting down the size for distribution if you don't have deps that are that large

tawdry needle
dull kiln
#

deps what do u mean by this

finite fulcrum
#

Your code's dependencies

dull kiln
#

oh right

agile lark
#

Is it possible to set up GitLab running without a domain and using self-signed cert? It's for local dev work only (no intention to open it up to the public). I'm not on any VPS services too, just me and an old laptop-as-a-server that runs Ubuntu

urban pecan
next spruce
lavish charm
#

Hey I am trying to chase down a memory leak using PyCharm involving tkinter/pil is there a better application to pick through memory snapshots?

wet parcel
#

anyone familar with pipenv and setup.py? trying to do a monorepoish type setup.

brazen roost
#

guys , can anyone help me about how to install pyspark docker image from here
https://hub.docker.com/r/jupyter/pyspark-notebook

i need specifically to setup spark version 3.0.1 , i'm new to docker and installed image which apeared to be a newer version , how can i change it

dull kiln
#

anyone know a good guide for innosetup use

heavy saddle
#

I have long question in #๐Ÿคกhelp-banana message, gist is:

Does it make sense to harvest .so libraries from the manylinux docker image to put in a wheel?

indigo zenith
#

Hopefully basic git question. I'm not using branching or anything currently, just tracking changes in a simple repo where I'm the only developer. I have an unused function that I want to remove for now but may want to come back to later. What's the easiest way to track something like this?

night quest
indigo zenith
silver gull
#

how should i clone a repo and then make a new repo with that other one?

#

like how do i clone repo A into a new repo B that im making?

rich nymph
#

Umm, hello
I remember installing git, but when I try to use this I get this error. Could someone help me sort this out, please ping me if so.

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

this happens when I run the command git

night quest
night quest
heavy knot
#

Hello, I have a tiring problem with git

#

(I use pycharm)

#

I have a file with .db format

#

I commited and pull my folder

#

all files changed with the thing where you can choose for each change in the code which version you keep

#

but the .db file also changed

#

I choose for each files but not in the db because it's unreadable

#

hii

#

so to fix it, how can I remove the files from git, I don't need to pull it

#

or how I can get back to the last commit, now it write this when I try

unknown.png
heavy knot
#

ok i found the problem, now it's fixed

#

thanks for your help xD

obsidian sigil
#

Has anyone used GitHub Actions to build and deploy to IIS?

#

I used to use Team City and Octopus Deploy to build and deploy. I would love some advice on best practices.

heavy knot
#

Is there any significance of the | I keep seeing after run in github actions other than to tell it to read the next lines?

#

Like ... run: | somecommand1 somecommand2

burnt ibex
#

That just means the input spans multiple lines rather than one. It's part of the yaml standard iirc

heavy knot
#

I see, so run: somecommand1 somecommand2 would fail?

burnt ibex
#

Umm, I'm not sure. Here's what I see in the yaml spec though (first code block)

unknown.png
#

Not sure if that's related though

heavy knot
#

I have a github action that runs a python script to generate a file then pushes it back to the repo ... - name: Run syntax maker run: | python src/syntax_maker.py git config --global user.name 'GitHub Publish Action' git add syntaxes/many.tmLanguage.json git commit -m "Automatically made many.tmLanguage.json syntax." git push

#

Problem is sometimes the file is already there unchanged so the commit complains nothing to commit, working tree clean and the action fails

#

Is there a way I can have the action still succeed if no changes are needed pithink

#

like if there is new stuff, commit and push it, else, do nothing

heavy knot
#

this seemed to do the trick discre3Yay if [[ `git status --porcelain` ]]; then git commit -m "Automatically made many.tmLanguage.json syntax." git push fi

velvet spire
#

For a dotenv file holding defaults or required variables, what would you name it? Does not have to be on this list.

.env.example
.env.sample
.env.template
valid socket
#

Would anyone mind helping me setup a DigitalOcean VPS for a discord bot? I'm super confused as this is my first time working with VPS' thank you! :)

raw gate
#

I've got a python codebase that runs on production and development (local). I have some scripts that people on my team (which are technical but not developers) are sometimes running and they need to be with production env.
I have python-dotenv and a prod.env file with the redis connection details for example.
The thing is, when running from your own dev machine, you need to tunnel the prod redis connection to localhost in order to connect to it.
How do you usually handle these scenarios?

rapid sparrow
#

I used digital ocean recently to get VPS for myself too

rapid sparrow
#

Do u wish using prod redis for dev?

raw gate
#

@rapid sparrow I have some scripts that should run with production creds (for example, connecting to the redis production server), but they're running from a developer's laptop.
To get network access to the production instance, SSH tunnelling is used. Hence, the redis hostname (and maybe port) is now localhost and no the actual internal production hostname

#

so I can have another env file like production-local.env that works on a dev machine (after setting up the SSH tunnels), but I just wanted to know what others are doing/thinking regarding this

rapid sparrow
#

Ah, so u wish to know alternative to ssh tunneling to prod redis, or alternative to .env files, right?

#

I use linux for dev. I just raise docker container with redis in my dev PC

#

I heard some Windows 10/macOSes versions can do it too now

raw gate
#

yeah that's not exactly what I'm asking

lucid umbra
#

hello guys, anyone knows if there is a fullnode implementation in python? like a library that I can install with pip

velvet spire
#

how can i push a branch with git without checking it out?

finite fulcrum
waxen gull
#

I don't know if this part of devops or software-design. Forgive me if I'm in the wrong channel for this. I want to run a python script on a cron job but I want all of this handled in docker containers. A bit of research leads me to this approach:

1 docker container for the python app script with the python slim image
1 docker container specifically to run docker run --rm app script installed with ubuntu image

Then put them together with a docker-compose.yml.

Is this the right way? Or should I merge the 2 docker images into 1?

polar gazelle
#

hello

#

I have main.py that want to run with docker-compose run, main.py take two argument like this main.py --data_dir /path/to/folder

how should I define command in docker-compose.yml for this
and how should i specify docker-compose run command

smoky compass
#

Is anyone else using Sublime Text as their editor and have a fairly comprehensive Python setup (e.g. linters/mypy/lsp)?

rapid sparrow
smoky compass
#

Haha. I have never gelled with anything heavier and I like my setup, I was just wondering if anyone else used it so I could see if I was missing anything cool

stable kite
#

Hi guys Ive encountered a weird problem using gitlab: I have a group with projects but the problem is that my group ci variables are only inherited by projects present before I added them but news project dont inheritate them I dont understand why

wet parcel
#

anyone familiar with pipenv and setuptools?

waxen gull
velvet spire
polar gazelle
polar gazelle
gritty lion
tawdry needle
#

@wet parcel following up:

โ”œโ”€โ”€ app
โ”‚   โ”œโ”€โ”€ main.py
โ”‚   โ”œโ”€โ”€ Pipfile
โ””โ”€โ”€ somelib
    โ”œโ”€โ”€ __init__.py
    โ”œโ”€โ”€ lib.py
    โ””โ”€โ”€ setup.py

can you post the contents of app/Pipfile? note that pipenv has weak support for "local" packages and editable installs, see https://github.com/pypa/pipfile/issues/90

#

also, @wet parcel i just noticed that somelib is structured incorrectly. it should be like this:

.
โ”œโ”€โ”€ app/
โ”‚   โ”œโ”€โ”€ main.py
โ”‚   โ”œโ”€โ”€ Pipfile
โ””โ”€โ”€ somelib/
    โ”œโ”€โ”€ somelib/
    โ”‚   โ”œโ”€โ”€ __init__.py
    โ”‚   โ””โ”€โ”€ lib.py
    โ””โ”€โ”€ setup.py

and setup.py should be like this:

from setuptools import find_packages, setup
setup(
    ...,
    packages=find_packages(),
    ...,
)
#

there is a difference between an "import package" (a module that can contain other modules) and a "distribution package" (something you distribute e.g. via pypi). the naming choices are unfortunate here, so it's sometimes easier to refer to a "distribution package" as a "distribution"

#

setup.py defines a distribution, which can contain one or more modules/packages

#

with the setup above, you should be able to write from somelib import lib in app/main.py

wet parcel
#

@tawdry needle thanks so much! I was structuring it incorrectly. clearing up the meaning distribution from other packages helped greatly.

tawdry needle
wet parcel
#

@tawdry needle do you if I can change that second dir to somethign else?

โ””โ”€โ”€ somelib/
    โ”œโ”€โ”€ somelib/ <---- this one?
tawdry needle
velvet spire
#

holy

#

this is excellent, i don't have to do the work for it now

#

nearly half of the commits to the linux kernel have been since 2015.

normal bone
#

Can I ask you a question about ffmpeg?

thorn wharf
#

hello guys,
hope you are all well.
I faced output clipped, log limit 1MiB reached error while building a container.
I added to /etc/docker/daemon.json these 2 lines:

  "log-driver": "json-file",
  "log-opts": {"max-size": "100m", "max-file": "2"}

I ran systemctl reload docker. The error persists when rebuilding.
any advice about how to handle properly this issue ?
PS: I don't want to use buildx since I am in production.

velvet spire
#

of the 33 respositories of this guy and it seems like all of them depend on a different project of theirs

heavy knot
leaden tartan
#

I think instead of pushing everything directly to app.domain.io we push it to something like dev.domain.io for testing and if everything seems fine we push it to app

Can someone tell me if this is even possible? It's a nestjs app. Are there any courses available to help me with this? Any help will be appreciated.

We are using AWS EC2 for the server

velvet spire
#

how can i make a github action step of a larger workflow work only if in the event a specific path was edited?

heavy knot
#

echo ::set-output name=was_edited::$value

#

then get it in another

#

${{ steps.previous_step_id.outputs.was_edited }}

velvet spire
#

but like, how do I know if it was edited in the pushed commits?

heavy knot
#

setting value...
run: |
value=???
not sure. some fancy bash 1 liner... that git diffs, pipes that to grep etc or use find. or checks last modified date of the file?

robust sparrow
#

What is the easiest way to debug memory consumption? I have numpy arrays as attributes to classes

#

ฤฐt goes upwards of 20gbs for views of 40 2500x1500 images :(

wooden ibex
robust sparrow
#

I guess that's whay you mean

tawdry needle
#

i think numpy arrays have their own "sizeof" method

#

note that an 'object' dtype array probably will be considered an array of references and will therefore underestimate the true size. but for things like numbers it should work well

finite fulcrum
#

A memory profiler ought to help

obsidian sigil
#

Does anyone know where to fix this issue? I am running GitHub actions, and I'm receiving "Error: Process completed with exit code 1."

SNAGHTML464438.png
night quest
#

Anyone used buildbot? Is it worth to learn?

wooden ibex
night quest
wooden ibex
astral olive
#

Hello, I have a question.
I sometimes work on a project at my school from a school computer.
I would like to use github pull and push, possibly without login.
I don't want to enter ssh keys on the school computer, because i don't always get the same computer, and typing in the ssh key every time would be a hassle
can i pull with https clone, but then i can't push.
can i push without login, or with some kind of login that doesn't save my password?

night quest
#

Make sure to remove repo (or change remote) when you leave this computer

astral olive
#

thx

astral olive
night quest
rancid schoonerBOT
#

Hey @civic grove!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

โ€ข If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

โ€ข If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

civic grove
# 
# syntax=docker/dockerfile:1
FROM python:3.8-alpine
WORKDIR /code
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
RUN apk add --no-cache gcc musl-dev linux-headers libressl-dev libffi-dev make
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
EXPOSE 5000
COPY . .
CMD ["flask", "run"]

this is my entire dockerfile

#

!pypi cryptography

rancid schoonerBOT
#
cryptography v3.4.8

cryptography is a package which provides cryptographic recipes and primitives to Python developers.

civic grove
#

^ this i believe uses greenlet

#

i think i have to install a dependency to the docker instance(?alpine) directly but i can't figurefind out which one

#

i added build-base to RUN and now it's working

graceful gazelle
#

How would I go about getting GitHub Actions to log the output of an ongoing step in real time instead of having to wait for the step to finish?

#

Also, if I have a Google Cloud VM and I initialise it as a self-hosted GitHub Actions runner using the browser console, then will jobs still run if I close that browser console window/computer dies?

south swift
#

someone knows shell command ?

short peak
#

just ask the question ๐Ÿ™‚

rapid sparrow
south swift
#

I just need to know is shell is the english name ๐Ÿ˜… ?

#

I need to translate my courses in english

rapid sparrow
south swift
#

Thanks, have a nice day y'all

tiny dove
#

Hello everyone

#

I'm trying to get performance data into a python script and then process line by line and output elsewhere

#

It's a typeperf command for windows I'm currently using to get performance stats

#

Are there any pointers to how I can extract said stats using python?

#

Thanks

tawdry needle
# tiny dove Are there any pointers to how I can extract said stats using python?

it depends on the format of the data, but usually you can process data line-by-line by looping over a "stream" such as stdin or a file object. for example:

import sys
from collections.abc import Iterable
from typing import TextIO

def process_line(line: str) -> str:
    """Perform some meaningless data processing for demonstration."""
    return f'<< {line} >>'

def process_stream(stream: TextIO) -> Iterable[str]:
    """Process an entire stream, line-by-line."""
    for line in stream:
        yield process_line(line.strip())

def sink_to_stdout(lines: Iterable[str]) -> None:
    """Do something with the processed data."""
    for line in lines: 
        print(line)

def main():
    if len(sys.argv) == 1:
        # If no args, read from stdin
        sink_to_stdout(process_stream(sys.stdin))
    else:
        # Process each input file one at a time
        for input_path in sys.argv[1:]:
            print(f'=== Processing: {input_path} ===')
            with open(input_path) as input_file:
                sink_to_stdout(process_stream(input_file))

if __name__ == '__main__':
    main()
tiny dove
#

@tawdry needle Thanks. I'll like to ask. I am currently working with a typeperf command in windows. I would first execute the command in main and then process each line right?

tawdry needle
#

i'm not sure what you mean by that, nor do i know what typeperf is

#

if you run that script as python3 <scriptname>.py, the stuff inside if __name__ == '__main__': will be executed. so in this case, that would execute the main() function, which checks cli args, reads from stdin, etc.

tiny dove
#

Typeperf TL/DR collects windows performance stats

#

Hmm, okay

#

Okay, thanks

tawdry needle
#

yes, you could also import <scriptname> (without .py) in a python console, but main() relies on checking the original cli args so that probably wouldn't work. you'd have to write a different function that takes a filename as input, for example

#

this is just a demonstration of how you might structure a "stream processing" application in python

#

more generally you can just do

with open(filename) as filehandle:
    for line in filehandle:
        ...
tiny dove
#

Thanks. I appreciate it

tawdry needle
#

noting that sys.stdin does not need to be "opened"

graceful light
#

So for my GitHub Workflows I run black, isort, flake8, and pyright for formatting / error checking the code, but what do you guys use aside from that? Do you have any spell checkers for things including comments?

hallow basin
#

Does anyone know of a good programming software?

graceful light
flat path
thorn wharf
#

hello guys, I have a container that must have a program called kaldi (https://github.com/kaldi-asr/kaldi).
I have to run this command for configuration purposes: cd /opt/kaldi/tools && extras/install_srilm.sh.
When I put it in RUN, it always raises an error, but if I build the image without it and run it inside the container, it works perfectly.
I thought maybe the displayed message after installation is seen as an "error". I tried to redirect it to /dev/null in vain.
any Idea about how to solve this ?

tawdry needle
wicked rivet
#

Is using pyenv on a server that doesn't have a modern version of python 3 best practice or is there a better alternative these days?

tawdry needle
#

docker perhaps

wicked rivet
#

Not a good option for this client ๐Ÿ˜ฆ

tawdry needle
#

i don't see anything wrong with pyenv then

#

i'd use pyenv-virtualenv as well

#

better than building it from source, although i guess there's always the risk that the pyenv maintainer is compromised and slips in a crypto miner or secrets logger

#

building python from source isn't that bad, but you do need some deps and like all things it might take some fiddling to get it working on the client's specific environment

thorn wharf
# tawdry needle no, the "error" has to do with the exit status of the script. if it's non-0, it'...

thanks for you helpful answer โค๏ธ this is the output:
make[2]: Nothing to be done for 'install-exec-am'. test -z "/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs" || mkdir -p -- "/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs" /usr/bin/install -c -m 644 'README' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/README' /usr/bin/install -c -m 644 'INSTALL' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/INSTALL' /usr/bin/install -c -m 644 'COPYING' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/COPYING' /usr/bin/install -c -m 644 'AUTHORS' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/AUTHORS' /usr/bin/install -c -m 644 'ChangeLog' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/ChangeLog' /usr/bin/install -c -m 644 'NEWS' '/opt/kaldi/tools/liblbfgs-1.10/share/doc/liblbfgs/NEWS' make[2]: Leaving directory '/opt/kaldi/tools/liblbfgs-1.10' make[1]: Leaving directory '/opt/kaldi/tools/liblbfgs-1.10' This script cannot install SRILM in a completely automatic way because you need to put your address in a download form. Please download SRILM from http://www.speech.sri.com/projects/srilm/download.html put it in ./srilm.tar.gz , then run this script. Note: You may have to rename the downloaded file to remove version name from filename eg: mv srilm-1.7.3.tar.gz srilm.tar.gz

tawdry needle
#

it sounds like you are expected to download the script from a specific location that requires you to put in your address, presumably so they can add you to a mailing list

#

so you need to obtain that tarball file first as per the instructions, and COPY it into the container before that RUN command

thorn wharf
#

it seems that they updated the script to make it auto download.
Thank you

rancid schoonerBOT