#tools-and-devops

1 messages Β· Page 67 of 1

supple venture
#

say you want to remove last 5 commits, then you could do HEAD~5

#

you need to git push force

velvet spire
#

You do have to git push -f

#

But

supple venture
#

it works for me

velvet spire
#

Github still saves that commit

#

So ur both right!!

supple venture
#

yes but it is not history

night quest
#

I mean not in branch history but commits were available via links

supple venture
#

yes

velvet spire
#

Yeah

supple venture
#

right

night quest
#

πŸ‘

#

Ninja commits ducky_ninja

rapid sparrow
#

it can be done, but it is magic beyond my level

#

as easy way... just git rebase -i I guess

night quest
unique magnet
#

yeah, I will rebase, but I'll also tell them to just get a new token, I'll do rebase just to keep the history cleaner

rapid sparrow
#

I know that there is some smoother way to rewrite history perhaps

supple venture
rapid sparrow
#

I saw mention about it in some article
how to handle transition from no formatting / formatting by another tool to black with keeping your git history clean
not remembering already what was recommended about it

supple venture
#

and merging branches

rapid sparrow
#

it offers like 10 different options to apply for each separated commit

supple venture
#

yeah, i was saying the main thing i use that for

rapid sparrow
#

you can select one commits to drop, others to squash, others to reword

#

and e.t.c.

supple venture
#

git rebase -i HEAD~5 D:

#

my most used command

#

lol

#

and git push -f

rapid sparrow
#

to use git revert
instead of git reset --soft HEAD~1

#

but habits die hard.

supple venture
#

^^^ so true, whats the difference in the two?

rapid sparrow
#

git reset --soft HEAD~1 cancels the last commit, while keeping the code from it.

#

but if you do it in public branch...

rich remnant
#

you can't, it's pointless to try and remove it, the key has already been crawled and published by automated tools - you must simply admit the fault, revoke and replace the API key, and move on .. it's 100% waste of time to try and remove it

rapid sparrow
#

you break their branches, since your branch would have diverted git history from theirs, and they will not be able to pull commits

#

git revert cancels the commit by making another reverting commit

#

it is safer to use for public branches

#

it can be pulled by others without breaking their stuff

#

(no need for git push with force also)

rich remnant
supple venture
#

yea, but i never push to main branch :D, i wouldn't like to have two commits for an unnecesarry thing

rich remnant
#

these guys had a public service for viewing such pushed secrets basically in real time

rapid sparrow
rich remnant
#

https://www.shhgit.com there seems to be a bit filtered version on their website right now

Keep your secrets, secret

We help secure forward-thinking development, operations, and security teams by finding secrets across their code before it leads to a security breach.

supple venture
#

hmmm, well personal opinions, just doing a soft reset is better than reverting and then squashing

burnt thunder
#

github also has their own secret checking thing, but I am not sure how to opt into that

supple venture
#

gitguardian is also there

rich remnant
#

anyway after publishing the secret there is no point in trying to "undo the damage", it's done, you can't undo it, you must invalidate the credential ASAP and then you can just do a normal commit to delete the secret

rapid sparrow
#

indeed. the damage is irreversible anyway

wooden ibex
#

Piracy would be 99% of the streams

#

Trust and Safety is massive cost of Discord, one reason Microsoft was hesitant, they have experience with XBox Live

urban ore
# 
on:
  workflow_run:
    workflows: ["Lint & Test"]
    branches:
      - main
    paths:
      - bots/**
      - pyproject.toml
      - poetry.lock
    types:
      - completed
#

I'm so confused, why does this workflow run even if the file changed is not in any of the paths

#

I changed README.md and it still runs

rapid sparrow
urban ore
#

github

rapid sparrow
#

this example offers on push too

#

official documentation

# 
on:
  # Trigger the workflow on push or pull request,
  # but only for the main branch
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  # Also trigger on page_build, as well as release created events
  page_build:
  release:
    types: # This configuration does not affect the page_build event above
      - created
#

hmm, path thing is offered too

on:
  push:
    paths:
      - 'sub-project/**'
      - '!sub-project/docs/**'
urban ore
#

Ye but I want the workflow to trigger only if its in the main branch, if the directory bots is changed and if the workflow Lint & Test is completed

rapid sparrow
#

perhaps you need having paths in ' '

urban ore
rapid sparrow
#

they offer how to ignore file changs too

on:
  push:
    paths-ignore:
      - 'docs/**'
supple venture
#

Or you can add a ! Exclamation mark to do opposite behaviour

rapid sparrow
#

yeah, that would be more convinient

supple venture
#

Like !docs/**

urban ore
urban ore
urban ore
finite fulcrum
#

Is it possible for pycharm to display the .git folder in the project view, while also not indexing it for things like searches?

tawdry needle
#

you can set folders to be "excluded" from indexing

#

you can probably also enable showing hidden folders but idk how

finite fulcrum
#

The only thing I've found are "Ignored Files and Folders" under Editor, which makes it show up in the tree but also makes project wide searches look into it

spark steeple
#

Hi,

My python isn't that great yet and I'm still trying to learn many of the basics.

I have a lot of code in JavaScript and I need to convert it to python. Are there any reliable programs that do a decent job with simple code?

Please tag me. Thank you in advance.

tawdry needle
tawdry needle
spark steeple
finite fulcrum
tawdry needle
#

How often do you need to access the .git directory?

#

I guess it's useful for config, excludes, description, etc

finite fulcrum
#

not too often, but it's annoying to step out of the ide just to grab the file and open it in the same ide

rich remnant
terse shadow
#

Does anyone here have any experience in screen-scraping tools?

#

specifically tn3270 emulators

sly flare
tiny karma
#

I have the main branch on my local clone, but there are two more branches on origin that I'd like to get into local. How do I do this?

unknown.png
#

Never mind got it

leaden tartan
# tiny karma

A shorter command is git checkout -t origin/branch

shadow crow
rich remnant
#

they used to have a real time unfiltered unobfuscated stream of those, that was sure fun times .. now they obfuscate usernames and reponames a bit, and claim to notify owners first, so dunno

proud python
#

Please any answers if possible inside stack

shadow crow
#

Is there some simple example of a mypy plugin to get how they work?

wooden crescent
#

Granted I’ve only been working with AWS for three weeks now, but dear good why does it get so much hype. Even though azure is IAAS and not PAAS it is much more usable then AWS. I’ve been trying to figure out how to have my jenkins space connect to the code commit repo for an absurd amount of time. Also the horror that is S3 buckets and IAM keys.

tawdry needle
#

doesn't look that bad really

shadow crow
wooden ibex
sterile notch
#

Keep getting this while installing mayavi

proud python
sterile notch
#

Building wheel for mayavi is stuck

finite fulcrum
#

is my pycharm being weird or does anyone else get a completely wrong inferred type on item with

for item in sorted(range(10)):
    ...

?

#

Well I suppose it isn't completely wrong but it definitely could do better, it looks like it's having trouble with the typevars in the stubs

a: dict[int, None] = dict.fromkeys(range(10))
for item in sorted(a.items()):
    print(item)

also gives int for item

tawdry needle
tawdry needle
finite fulcrum
#

for the first snippet it gives SupportsLessThan which is a protocol from the stub file

#

looks like it just gives that because the typevar is bound to it

tawdry needle
#

I actually think that's not wrong

#

What is the return type of range in the stub file?

#

That protocol? Has range worked on strings this entire time??

#

I am on my phone so I can't check right now

finite fulcrum
#

the iteration on range is Iterator[int]

finite fulcrum
tawdry needle
#

I was kidding πŸ™‚

finite fulcrum
#

adding the typehint to b here makes it work properly so I guess it has problems with extracting that from the typehinted __iter__

def f(a: collections.abc.Iterable[T]) -> list[T]: ...

b: collections.abc.Iterator[int] = range(10)

for i in f(b):
    ...
tawdry needle
#

I would usually write Iterable and not Iterator but yeah

#

Does mypy infer correctly if you use reveal_type?

finite fulcrum
proud python
tawny temple
#

@round mortar No, the tool itself doesn't take much space. For example, on Windows, the portable version seems to be about 50 megabytes. I don't have on hand its size on other operating systems.

round mortar
#

oh

#

nice thanks

tawny temple
#

That being said, the repositories you clone (if you're unfamiliar with that term, it basically means to download code onto your computer) can be of an arbitrary size.

round mortar
#

im gonna use it with vs code so yea

round mortar
#

or if i delete it will cause problems with the github repo

tawny temple
#

If you pushed your changes to GitHub, then all your code is saved on their servers. You can then do whatever you want with the repositories on your computer, including deleting the folder, without it affecting what's on GitHub.

#

Well, not "whatever" you want. Of course, if you push more changes to GitHub, then that will affect what's on their servers. But that requires running git commands. Just deleting a folder doesn't tell git to push any data.

round mortar
#

okay thanks

alpine ridge
#

aws free account also needs credit card details, do they charge anything ?

leaden tartan
alpine ridge
#

Oh, okay

tulip forge
#

Hi, is there a good "automating git stuff" library out there? I'm working in a team with 99 git repos (not literally) and would like to fetch them, show some stats about what's up today (many changesets in repo XX), etc

tame lodge
#

how do i connect my postgres db to vultr?

vague silo
vague silo
tame lodge
tulip forge
vague silo
vague silo
tame lodge
vague silo
tame lodge
vague silo
#

Okay. Do you want to do the simple way or the correct way?

tawny temple
#

Why do those two things have to be mutually exclusive πŸ˜”

vague silo
#

I know right πŸ˜”

#

The simple way is to add
host all all 127.0.0.1/32 trust
to /etc/postgresql/VERSION/main/pg_hba.conf, run systemctl reload postgresql@VERSION-main, replacing VERSION with your installed postgresql version, and then connecting with your bot to the database you want to, and you can use the postgres superuser.

The correct way is to use host BOTDB BOTUSER 127.0.0.1/32 md5 to said file, run said command, then creating the bot database user via createuser, and connect as that user. That will also involve storing the credentials somewhere secure

#

Another option is to run your bot account on a locked down linux user (which you should do anyways), then create the same named user in postgresql, and you can connect to the database user withotu password (from the operating system user, via the postgresql unix socket)

#

that's a bit more complicated but also very nice

tame lodge
#

Alright, thanks

finite fulcrum
#

can I open the corresponding local file instead of only displaying the diff from the PR tab in pycharm?

gritty tangle
#

Any mac users here? Especially VS Code users? ❀️

frank ravine
#

Hi I am looking for something that can scan an image for something uniformally circular, then cut that circle from the image and create a new image file with just that circle in it

rapid sparrow
frank ravine
#

oh boy

rapid sparrow
#

Segmentation/classification problem

#

You could make it of course perhaps with more primitive instruments. I think I read somewhere how to solve it with with a bit simpler algorithm

#

But it would be essentially same field

frank ravine
#

I actually just found a github repository of exactly what I'm trying to do so I'm going to check that out and see(probably) if they found an easier way to do what I want

rapid sparrow
#

;) that is great

tawdry needle
frank ravine
#

I'm using openCV but it seems to only check the difference between the exact same image in the same place, IE if i took two pictures of something IRL it seems to show differences all over because everything isn't completely lined up

#

which is why I was looking to find a way to scan for only certain parts and put those onto a fresh image to compare so everything would be lined up

tulip forge
#

Lenz in my experience, scikit-image is easier to use, opencv is faster. Do you have any code, though? I don't get the question (from the description)

#

@frank ravine that was for you. And, if it's really close to circular, you probably don't need a neural net, but maybe a convolution trick of some sort will do (faster, easier)

ruby spoke
#

What library do you reccommend for sending mouse input and keyboard presses in python

tulip forge
#

@ruby spoke do have a look at pyautogui!

pastel wing
#

What does ** do in .gitignore when used like this? **/FolderName ?

thorn wharf
#

hey guys, I am new to docker and I want to create a conda env inside a docker image with libraries installation. This is my code:

FROM anibali/pytorch:1.5.0-cuda10.2-ubuntu18.04
COPY requirements.txt requirements.txt

RUN bash ~/miniconda/etc/profile.d/conda.sh &&\
       conda init bash &&\
       conda create --name layoutlm python=3.7 &&\
       conda init bash &&\
       exec bash &&\
       echo "source activate layoutlm" > ~/.bashrc &&\
       pip install -r requirements.txt

when I run the container and do pip freeze, the requirements are not installed.

How can I solve it ?

rotund widget
#

Anyone available to help for some git issue

pastel wing
#

403 means unauthorized

rotund widget
#

I have ssh key installed on my local

#

but it still pop up this error

pastel wing
#

well you are trying to use a https connection not ssh

rotund widget
#

Sorry, I dont understand the differnce ......

#

how to resolve this

pastel wing
#

use ssh for ssh key or create a pat for https

rotund widget
#

So you mean when I clone repo from gihub, I need to use ssh address?

#

I dont know how to switch to SSH now

#

I will try google it. At least now I know what the issue is

#

Thank you btw

pastel wing
rotund widget
#

Thanks bro you save me. I have been trying to figure this error out for like 2 hours....

pastel wing
#

no problem πŸ™‚

gray juniper
#

Hi, I am trying to setup codespeed python package, but the documentation is not clear how to setup it up for a new project, can someone help? Is it available only for Django applications?

rapid sparrow
#

In any path

pastel wing
rapid sparrow
tulip hull
#

Hey guys, I was wondering how I could get my exe to work on another person's PC, I have the exe working on mine but when I gave it to someone else, it just crashed midway - maybe because of the packages etc

pastel wing
tulip hull
#

python

heavy knot
#

Hello, i have a question, can use github for permanantly runed programs

#

like a discord bot per exemple

burnt ibex
#

Like projects that are always running?

heavy knot
#

sorry for bad english i'm frensh

burnt ibex
# heavy knot yes

Yeah, sure! You can put any type of code on GitHub, as long as it doesn't violated their ToS.

night quest
burnt ibex
#

GitHub won't run the program though, only store the code.

night quest
#

You can run short actions (GitHub actions)

#

Like building binaries for new release

heavy knot
#

ok thanks, i'll try tomorow

dawn socket
#

has anyone used the pywin32 software?

#

library

wooden ibex
dawn socket
#

@wooden ibex what other libraries do you recommend in place of it?

wooden ibex
#

Depends on what you want to do

dawn socket
#

I heard of pyautowin but I haven't seen it be able to send keys to inactive windows

#

or unselected windows

wooden ibex
#

oh, afk bot

dawn socket
wooden ibex
#

apps need admin privileges to interact with other apps

velvet spire
#

hey I wouldn't say it was an afk bot instantly lol

#

oh nvm

wooden ibex
#

but probably pyautowin is better for what you need

dawn socket
#

sounds good, yeah the idea I had was the chat has control of certain keys in the game while I get to still interact with the game. But in case I go onto other windows to check things I didn't want the server to continue sending keys to my pc.

#

i'll take a look

scenic wren
#

you try autoit / ahk module of python . you can send keys as well as taps to inactive windows. i was able to do it.

small echo
#

how do pypi mirrors work.... do they only get packages on demand or do they clone the entire metadata like apt and yum will so you can seach locally? if so, where is the metadata for pypi?

rancid raven
#

ghosting problem with command prompt in #help-popcorn if anyone could take a look that'd be great

tawdry needle
# small echo how do pypi mirrors work.... do they only get packages on demand or do they clon...

It depends on how you set things up.

You can set up your own "package index" and use --extra-index-url in Pip, as in https://packaging.python.org/guides/hosting-your-own-index/ and https://mothergeo-py.readthedocs.io/en/latest/development/how-to/private-pypi.html. This way you aren't mirroring PyPI at all, but you're adding an extra location for Pip to search before searching PyPI itself.

If you really do want to host a mirror, you can use Devpi (https://pypi.org/project/devpi , https://devpi.net/docs/devpi/devpi/stable/%2Bd/index.html) to actively mirror PyPI by fetching files once from PyPI and locally caching them thereafter. Or use Bandersnatch (https://pypi.org/project/bandersnatch/) to "statically" dump the state of all packages on PyPI, which will need to be refreshed periodically. In both cases, you can use the --index-url instead of --extra-index-url so your client never has to touch PyPI directly.

PyPI
devpi

(deprecated, install devpi-server, devpi-client, devpi-web instead)

PyPI
bandersnatch

Mirroring tool that implements the client (mirror) side of PEP 381

small echo
#

I'd like a local cache so if I have a few projects using foo at v3.14.159 then I don't have to hit the web. And if I'm messing with docker files it also shouldn't need to hit the web. For working on trains. On planes. Or Jenkins locked in chains.

#

A bit like nexus or artefactory tbh

native hinge
#

Issue regarding git add, can you help me at #help-pear ?

tawdry needle
#

i agree i like the idea of not having docker spamming pypi every time i need a clean rebuild

small echo
#

Amazing. Will check it out.

heavy knot
#

How to run codes in github || sry it is not releated to python but sadly there is no github discord server || :)

velvet spire
#

wdym run codes in github || ||?

heavy knot
#

I dont find a button to run code

velvet spire
#

there isn't one

#

you can't run code on github

heavy knot
#

:(

#

I am stuck w/ intel atom and 2gb ram windows 7 32but

#

*bit

velvet spire
#

what are you trying to do?

heavy knot
#

I thought i would code on github

heavy knot
velvet spire
#

ask in an offtopic channel, someone will have some ideas

#

!ot

rancid schoonerBOT
leaden tartan
#

Just go to a repository and click . on your keyboard

velvet spire
leaden tartan
#

But this service will be paid September onwards

#

You can always use gitpod

velvet spire
leaden tartan
#

I had a fairly good experience with it

leaden tartan
velvet spire
#

unless you can tell me otherwise, I thiiink that . will remain free?

leaden tartan
#

Yeah that's just the viewer i think

velvet spire
#

because you aren't actually running in their codespaces

leaden tartan
#

You can't really run the code

velvet spire
#

yeah exactly

leaden tartan
#

Use gitpod its free for 50.hrs a month

velvet spire
#

not the online editor

leaden tartan
#

Nope, couldn't get into the beta

velvet spire
#

(which is literally just vscode in browser)

leaden tartan
#

Thats what gitpod is too

#

Their customer support is absolutely amazing too

velvet spire
#

wait

#

gitpod is literally vscode in browser huh

leaden tartan
#

Yes

#

And something called theia too

#

Can't say i liked it

#

We used gitpod for a college project

velvet spire
#

nice

leaden tartan
#

Since i couldn't install docker on all my teammates computers

#

Some of them didn't support it at all

velvet spire
leaden tartan
#

Yep it was in the works a few months ago when i last used it

velvet spire
#

what the f

leaden tartan
#

Good riddance lmao

velvet spire
#

is this a free test

#

or what

leaden tartan
#

?

velvet spire
leaden tartan
velvet spire
#

why is double hours free

leaden tartan
#

Oh wow

velvet spire
#

github student pack

leaden tartan
#

The ones who used gitpod didn't come close to 50 hrs tho

velvet spire
#

that's why

leaden tartan
#

Ahh i have that too

#

Didnt see it then

#

Maybe its something new

velvet spire
#

also

leaden tartan
#

Student unleashed lmao

rapid sparrow
heavy knot
velvet spire
heavy knot
small echo
#

@tawdry needle I took a look at bandersnatch. Thanks! It's a cool project and will be a good start for my work.

frail yarrow
#

any tools to turn this into exe

tawdry vale
quiet mason
# 
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY *.sln .
COPY aspnetapp/*.csproj ./aspnetapp/
RUN dotnet restore

# copy everything else and build app
COPY aspnetapp/. ./aspnetapp/
WORKDIR /source/aspnetapp
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
====>   FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=build /app ./
ENTRYPOINT ["dotnet", "aspnetapp.dll"]
```I get to "FROM mcr.microsoft.com/dotnet/aspnet:5.0" and get the error `Get https://mcr.microsoft.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)` Is this a network provider error?
small echo
#

is the error coming from your laptop or a CI instance?

#

so i think it's a bug on their end.

quiet mason
#

Does it work on your device?

small echo
#

no it doesnt work on mine

quiet mason
#

Imagine that smhlemon_angrysad and I need this to work tomorrow

small echo
#

i think that's why its broken. the universe determined that you had a deadline so it decided to break everythin

quiet mason
#

πŸ˜†

rapid sparrow
tawdry needle
small echo
#

bandersnatch? no i wont use it. it makes a full mirror with the wheels. this could be an issue with e.g. tensorflow which is stonking.

flat path
#

Bandersnatch maintainer here, you can filter what packages it mirrors fyi

#

There's quite a few filtering options but I don't remember πŸ˜…

small echo
#

oh hi @flat path ! Thanks for the cool project!
FWIW the docs and the code are out of step wrt the configuration and defaults. e.g. you need to set various values even if the docs say they have a default value. This is because fallback= is not used everywhere it should be in mirror.py

#

it's often useful to have a mirror.conf.example which can be chopped up by users

sly flare
flat path
abstract lily
#

hi, does anyone know any extension for vscode to parse the structure of a python script?

small echo
#

whats the structure of a python script? You mean to summarise the functions and types like you might have in pycharm?

small echo
#

The normal python one does. You need to go to View -> Open View ... "Outline"

abstract lily
#

ΒΏmaybe exist a tool more visual? like visustin

small echo
#

Not as far as I'm aware. If I can be frank these tools don't work well as they scale to larger code bases.

abstract lily
#

yeah, I have tried 2 and they do not understand part of the code

#

thanks for your help

heavy knot
#

Hello, I need help to use heroku with python

velvet spire
#

whatcha making?

heavy knot
#

I would like to create a discord bot

velvet spire
#

just added heroku support to a project of mine so

#

ah.

heavy knot
#

so i don't need django

velvet spire
heavy knot
#

OK, thanks 🀣

#

so there is no way to keep running a bot without paying ?

small echo
#

should we move the chat to #discord-bots ? Otherwise, you may want to consider free tiers of google cloud or AWS in conjunction with e.g. free tier of mongo atlas

sharp patrol
#

docker run --gpus 1 -t rt:2.0 -c "clear && conda activate env && cd example && python main.py" does not work but docker run --gpus 1 -ti rt:2.0 and doing those commands manually does. Is there a way to make it work without doing it manually? I tried a bash script but that didn't work either

#

In the first example it just hangs

small echo
#

docker ru n--gpus 1 -t rt:2.0 docker-entrypoint.sh hangs? is docker-entrypoint.sh copied into the image?

#

is bash installed or do you need /bin/sh?

sharp patrol
sharp patrol
#

so in the entry point ill add /bin/sh?

small echo
# 
$ cat docker-entrypoint.sh
#!/bin/bash
clear && conda activate env && cd example && python main.py
#

clear shouldn't be needed πŸ€·β€β™‚οΈ

sharp patrol
#

ah yea i just had it there haha sry

sharp patrol
#

ty for your help btw

small echo
#

it should be in your directory with the dockerfile and then in the dockerfile you have a line with COPY docker-entrypoint.sh .

sharp patrol
#

ok sounds good ill try rn and rebuild

#

can i ping u after ?

small echo
#

it's after midnight in belgium so ... probably not

sharp patrol
#

ah ok sounds good

#

hope you sleep well :D

#

thanks for your time

small echo
#

thanks

sharp patrol
#

hmm no luck πŸ₯²

#

i have no idea whats up w this

cobalt matrix
#

hi idk if this is the right chat but: is there someone known with the bert-base-uncased extension form huggingface

neat grove
#

Can I use this channel for Telegram bots?

#

Or is there a better channel for things like that.

velvet spire
#

@warm pollen heyyyy, I heard you're pretty good at git...

warm pollen
#

lol, sorry but I'm currently at work

velvet spire
#

Aight, lol. I'll still send it anyhow since it's a public channel πŸ˜›

#

Hypothetically, if the branch that a feature is being merged into was force pushed, how would I rebase my changes onto that new branch, without rebasing the entire branch. Tldr I need to get a few commits off of that branch without getting all of them.

tawdry needle
#

can't you do a normal recursive merge?

#

actually i don't see why rebasing would be an issue either

#

unless they force pushed changes that are incompatible and then you'd get rebase conflicts

velvet spire
#

we ended up deciding not to do it

tawdry needle
#

better to resolve 1 merge conflict than to resolve a chain of rebase conflicts imo

lost mantle
#

Hi, i need some help in creating a CLI package which is to be installed via pip install git+<url> During testing when i install it via pip install --editable . it works fine but when i push to git and then do it, it throws an error saying there is no module named edge

#

this is my directory structure

unknown.png
#

my setup.py file

from setuptools import setup

setup(
    name="Edge",
    version="0.1",
    py_modules=['edge'],
    install_requires=[
        'Click',
        'boto3',
    ],
    entry_points='''
        [console_scripts]
        edge=edge.main:main
    '''
)
tawdry needle
#

@lost mantle this only works because you're running it in the current directory. py_modules=['edge'] isn't sufficient for all modules. you need to list packages in the packages= parameter, usually you use find_packages to assist:

from setuptools import find_packages, setup

setup(
    name="Edge",
    version="0.1",
    packages=find_packages(),
    install_requires=[
        'Click',
        'boto3',
    ],
    entry_points='''
        [console_scripts]
        edge=edge.main:main
    '''
)
#

also the user should write pip install 'git+<url>#egg=edge'

lost mantle
#

@tawdry needle thanks a lot, it worked!

#

what does #egg=edge do?

tawdry needle
#

you can test this without pushing to github by creating a 2nd git repo on your filesystem and installing with pip install git+file:///path/to/test/repo#egg=edge

tawdry needle
# lost mantle what does `#egg=edge` do?

in this case, it tells pip what the name of the "project" is, which it tries to infer from the url. if you omit it, pip has to re-clone the repo every time the user installs or updates. the egg= parameter should be whatever is in edge.egg/top_level.txt

velvet spire
#

use poetry

lost mantle
#

one other thing, i am doing these imports in main.py

from .commands.init import Initialize
from .commands.build import Build
from .commands.upload import Upload
from .commands.deploy import Deploy

if there a way to make this short by doing something like from commands import Initialize, Build, Upload, Deploy

velvet spire
#

yes

lost mantle
#

i tried putting this in commands/__init__.py

from .init import Initialize
from .build import Build
from .upload import Upload
from .deploy import Deploy
#

but it didnt work

velvet spire
#

wdym

#

from that you can just do import commands and they will all be part of that

velvet spire
lost mantle
#

so you are saying without mentioning file name i can directly import them?

velvet spire
lost mantle
#

i had tried that previously but it didnt work, let me try again

velvet spire
#

make sure you've saved your files lol

lost mantle
#

oh so it doesnt work when i test it with pip install --editable . but works with pip install git+

#

why is that?

velvet spire
#

i have no idea

#

I use poetry for package management so

lost mantle
#

oh anyway the primary method using git is working so it should be fine i guess

velvet spire
#

lol

#

absolute imports > relative IMO

tawdry needle
#

it will be from . import commands if you're using relative imports

#

otherwise if you're using absolute imports it will be from edge import commands

#

or just import edge.commands then you can write edge.commands.Initialize

lost mantle
tawdry needle
#

yeah

lost mantle
#

it worked thanks!

tawdry needle
lost mantle
#

it worked now

velvet spire
#

side note: i have code and cannot find the logic where something i just wrote happens

#

don't be like me

#

oh just found it whew

lost mantle
#

thanks a lot @tawdry needle and @velvet spire πŸ™‚

velvet spire
#

salt rock lamp did it all don't thank me lol

lost mantle
topaz aspen
#

I have a docker image that I've been using for local development now and then, and need to be able to use gsutils within it - things like gsutil ls, or making calls to bigquery etc from notebooks.

I'm not sure what the typical workflow is for doing this though, and as I'm inexperienced with docker i'm wondering whether I just don't know what to google here.

I don't want to use a different base image - I'm using ubuntu and want to continue doing so, and I'm (for now) only trying to do this on a local machine, not a VM on gcp.

Any pointers on going about setting this up would be appreciated.

rapid sparrow
#

It will make you access inside the container

#

Find the steps to install things for gsutils

#

Add the necessary steps to dockerfile

velvet spire
#

<@&831776746206265384> scam filter fail

#

ty

#

(user id 755032484026122250 if they deleted it themselves)

toxic bough
#

nope, we gottem

heavy knot
#

some who know well heroku here ?

velvet spire
#

...why would you do that?

#

this

#

πŸ€¦β€β™€οΈ

shadow crow
#

@heavy knot Please don't ping anyone for help.

#

Only ping moderators if you need something moderated.

merry patio
#

How do you handle logs? I am looking for tools to better format my logs and maybe process and filter them

leaden tartan
merry patio
#

Is colorizing your logs useful? what do you think?

rotund furnace
#

I just took a look at the loguru suggestion above which looks really nice - notably the formatting of the stack trace.
I'll be adding that to a tech improvement board at work to see if there's value to be had.
Thanks for rec!

deep estuary
#

loguru is neat

latent robin
#

Traceback (most recent call last):
File "c:\Users\isaYILDIZ\Desktop\Decoy\app\worker.py", line 7, in <module>
from database import *
File "c:\Users\isaYILDIZ\Desktop\Decoy\app\database.py", line 274, in <module>
class Lobby(BaseModel):
File "c:\Users\isaYILDIZ\Desktop\Decoy\app\database.py", line 280, in Lobby
members = ArrayField(IntegerField, 5, default=[])
File "C:\Users\isaYILDIZ\AppData\Local\Programs\Python\Python39\lib\site-packages\playhouse\postgres_ext.py", line 171, in init
self.__field = field_class(**(field_kwargs or {}))
TypeError: peewee.IntegerField() argument after ** must be a mapping, not int

plush peak
#

I want to make code that presses ctrl + v + enter and waits the time you input into a window that pops up when you open the program

velvet spire
#

ah yes, when you accidently make a commit instead of amending it πŸ˜›
git reset HEAD^ --soft && git commit --amend

tawdry needle
velvet spire
#

hm, I should change that tbh

#

rather than modify the shit outta root, should modify it outta my own module

tawdry needle
#

all of these files would be under myapp/, so logging.py is the myapp.logging module

#

usually i also have a custom formatter in setup_toplevel_logging

#

as well as other stuff like filters for doing data type conversions

velvet spire
#

i.

#

(you didn't get ghosted, just lance deleted my bm invoke)

tawdry needle
#

πŸ™‚

velvet spire
#

πŸ₯Ί

tawdry needle
#

not offhand. but i did once set up a logging filter that pulled the current request ID off flask.g when it was present

#

that was probably one of the most useful things i ever did in python

velvet spire
#

well dang

#

might restructure my current logging into __main__.py

tawdry needle
velvet spire
#

rn its in __init__.py because I mean like, you shouldn't be importing this specific project, and if you are, you're doing it wrong

tawdry needle
#

that's fair enough. i still like to make my own applications as "library"-like as possible

#

for my own sake if nobody else's

velvet spire
#

its a discord bot lel

pastel wing
velvet spire
#

might move logging to __main__.py regardless tho

tawdry needle
pastel wing
#

Yeah it's creator made Fossil

#

Also you can import/export from/to git

velvet spire
#

speaking of git.. have you ever used a git client in python?

#

I'm thinking of using dulwich

#

since it doesn't require git to be installed

pastel wing
#

What is the use case here?

surreal apex
#

hi, i want to do kind of game bot it will see my screen and move the character with mouse and keyboard.which library should i use

tawdry needle
#

never heard of such a thing. i tried using libgit2 bindings once but i think i ended up just using subprocess for whatever i was trying to do

velvet spire
tawdry needle
#

"not use the shitty git CLI"

pastel wing
#

Just use shell scripts/commands ?

velvet spire
#

I'd perfer to use python, since git might not even be installed

#

funny thing is

#

!pypi dulwich

rancid schoonerBOT
velvet spire
#

this apparently doesn't require git to be installed

tawdry needle
#

i'm reading this now... must have been a huge amount of work

#

i assume the C backend is libgit2

velvet spire
#

pure python

tawdry needle
#

it says there's an optional C backend

velvet spire
#

right

#

oh

pastel wing
#

Python snail sadly

velvet spire
#

lmao one reference to libgit2

rancid schoonerBOT
#

dulwich/tests/test_objects.py lines 305 to 312

class ShaFileTests(TestCase):
    def test_deflated_smaller_window_buffer(self):
        # zlib on some systems uses smaller buffers,
        # resulting in a different header.
        # See https://github.com/libgit2/libgit2/pull/464
        sf = ShaFile.from_file(BytesIO(small_buffer_zlib_object))
        self.assertEqual(sf.type_name, b"tag")
        self.assertEqual(sf.tagger, b" <@localhost>")```
tawdry needle
#

wow someone reimplemented all of git

#

this is really impressive

velvet spire
#

The reason to use dulwich over anything else is to ensure minimal dependencies, if I want to install something from gitI'd rather use dulwich over subprocess and git πŸ™ƒ

tawdry needle
#

dependencies or not, this might just be a lot nicer when scripting git things in general

velvet spire
#

ironically one of my dependencies is currently a git repo so

#

lol

#

there's also GitPython but it does require git to be installed Β―_(ツ)_/Β―

tawdry needle
#

heh, that is probably a wrapper around subprocess

velvet spire
#

and is in maintainence mode

#

i've used pygit2 as well

#

There is one thing I could do which I haven't tried yet

#

but github allows a zip download of every single commit

#

so I may just be able to do that instead

#

the question is does gitlab support that too

#

yes

#

yes it does

#

which means I probably don't even need dulwich πŸ‘€

#

I think I'm branching out (😏) of the topic of this channel, could we move to a help channel or something else?

tawdry needle
#

this seems on topic to me

velvet spire
#

well, the entire feature is actually downloading and importing python modules from github or gitlab repos

#

not sure if downloading it as a full repo (shallow ofc) would make more sense, or as a single zip

#

there is...

#

!d zipimport

rancid schoonerBOT
#
zipimport

Source code: Lib/zipimport.py

This module adds the ability to import Python modules (*.py, *.pyc) and packages from ZIP-format archives. It is usually not needed to use the zipimport module explicitly; it is automatically used by the built-in import mechanism for sys.path items that are paths to ZIP archives.

Typically, sys.path is a list of directory names as strings. This module also allows an item of sys.path to be a string naming a ZIP file archive. The ZIP archive can contain a subdirectory structure to support package imports, and a path within the archive can be specified to only import from a subdirectory. For example, the path example.zip/lib/ would only import from the lib/ subdirectory within the archive.

velvet spire
tawdry needle
#

doesn't pip already do that

velvet spire
#

they may not have installation files

#

they're extensions of this project, to be put in its own source

tawdry needle
#

i see

velvet spire
#

its an addon system for a bot I'm writing

tawdry needle
#

i would use the zip archive if you know the host supports it, and fall back to git shallow clone otherwise

velvet spire
#

yeah

tawdry needle
# 
if is_github(host):
    path = fetch_ziparchive_github(repo_url)
elif is_gitlab(host):
    path = fetch_ziparchive_gitlab(repo_url)
else:
    path = clone_shallow(repo_url)
velvet spire
#

and then how would I load them, would it make sense to extract the zip or just read everything in it without extracting using zipimport?

tawdry needle
#

if you know you only need 1 file then just extract that file

velvet spire
#

wait you can extract single files

#

i should have remembered that

#

then in that case, makes more sense to extract all of the files we need

#

especially given this:

Note that if an archive only contains .py files, Python will not attempt to modify the archive by adding the corresponding .pyc file, meaning that if a ZIP archive doesn’t contain .pyc files, importing may be rather slow.

#

thanks!

small echo
#

you can extract single files from zip, yes. and sometimes from tarballs (tgz, tar.gz if your file is in the beginning of the tar file you only need to decompress the beginning)

heavy knot
#

How can i open an website with an proxy in pycharm, and let it close the website after like 30 seconds or something
and i dont understand how i can let it do an automated command only how to let it open the site

small echo
neon jungle
# 
PS C:\Users\space> conda install tensorflow keras
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
pkgs/main|pkgs/main/win-64::python==3.8.11=h6244533_1
pkgs/main/win-64::python==3.8.11=h6244533_1

what does this mean? From what I see it's saying that the python install isnt compatible with itself, wtf?

tawdry needle
#

i'm not sure what pkgs/main| means in this case. but that appears to be the difference between the two

neon jungle
#

I figured it out, i ran it again with just installing tensorflow and apparently tensorflow is incompatible with 3.8

tawdry needle
#

huh

neon jungle
#

which doesnt make sense since it should be

tawdry needle
#

what channels do you have selected?

neon jungle
#

i have conda forge in my defaults but thats it

# 
conda install tensorflow
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
                                                                                                                       |failed

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - tensorflow -> python[version='3.5.*|3.6.*|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

Your python: pkgs/main/win-64::python==3.8.11=h6244533_1

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
#

oh and pytorch is also in the defaults

tawdry needle
#

huh, looks like it just isn't packaged for 3.8+

#

try using -c conda-forge or set conda-forge above defaults in your condarc

#

the anaconda defaults generally seem to be behind

neon jungle
#

installing it works in a test env with 3.9.5

#

but this env has 3.8.11

tawdry needle
#

huh

neon jungle
# 
auto_activate_base: false

always_yes: True

default_channels:
  - conda-forge
  - pytorch

create_default_packages:
  - pip
  - ipython
  - pylint
``` That's my full .condarc file
tawdry needle
#

ah

neon jungle
#

i'm gonna try making another blank test env with 3.8.11 and see if it works there

#

yeah doesnt work in a blank test env either

#

i figured it out, it was a stupid mistake

neon jungle
#

therefore it wasnt checking the default anaconda repo

neon jungle
#

Changed my .condarc to

auto_activate_base: false

always_yes: True

channels:
  - conda-forge
  - pytorch
  - defaults

create_default_packages:
  - pip
  - ipython
  - pylint

and it works now, thanks a lot for the help!

latent canyon
#

Error while installing rabbitmq in m1 mac : /usr/local/sbin/rabbitmq-server: line 154: 5407 Segmentation fault: 11 start_rabbitmq_server "$@"

wooden ibex
primal cloak
#

i want to install git. and it asks me what editor should i choose? what do you guys think is the best editor?

pastel shuttle
#

atom

night quest
#

I don't like to leave from terminal and wait for VS Code/Atom/whatever

primal cloak
#

okay, but i can change it in the future right? or do i need to reinstall to change the editor?

night quest
primal cloak
#

okay thanks

night quest
rapid sparrow
velvet spire
#

or your IDE ^

heavy knot
#

Help me in this

#

!paste

rancid schoonerBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

heavy knot
brazen dragon
#

Hi, everyone

#

How to crack pycharm 2021.2?

tawdry needle
rancid schoonerBOT
#
Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

toxic bough
rancid schoonerBOT
#

:incoming_envelope: :ok_hand: applied warning to @brazen dragon.

night quest
#

Yes

#

Or git clone ...

night quest
#

πŸ‘

velvet spire
#

Thought this channel might appreciate it, lol

#

(no that is not my reddit)

chilly jacinth
velvet spire
#

You work for deezer?

chilly jacinth
#

No i do not

#

The project is licensed under the MIT license, and is freely available to the public.

velvet spire
#

So then, how about linking your source code? πŸ˜„

chilly jacinth
#

I do not have to as it is not a requirement of the license.

velvet spire
chilly jacinth
#

Ah sorry, I did not realize that tools was referring specifically to dev tools

velvet spire
#

& if youre sharing a closed source project, I would use! offtopic

rancid schoonerBOT
chilly jacinth
#

awesome thank you!

austere verge
#

I am using ubuntu 20.04 and I want to write a bash script that checks if specified scripts crashed in the background. Can anyone provide a bash script code for that?

rugged tiger
#

Hi all, how much django do you need to get started and be dangerous?

#

It's a big framework and I don't know if I know enough

rapid sparrow
rapid sparrow
#

Additional map a bit more python oriented

plan_backend.png
small echo
# 
pyenv install --list | grep '^  3\.10'
  3.10.0b4
  3.10-dev

Where's rc1? 🀌

gentle solstice
#

I am getting dns resolution errors when I attach my docker container to a non-default bridge network (created by compose). i.e. connecting to login.microsoftonline.com results in a "Name or service not found" error in httpx

tawdry needle
gentle solstice
valid reef
#

I see some people using a editor that has a terminal "built-in". Is that a specific editor or can multiple of them do that?

#

I'm now just constantly switching between powershell and sublime

sly sleet
#

there should be sublime plugins for that

#

@valid reef

valid reef
wooden flower
#

Perhaps can I automatically make an SSL certificate by using docker ?

gentle solstice
#

I know kuberneties can do it

wooden flower
#

Okie

deep estuary
#

for docker specific

rapid sparrow
rapid sparrow
#

nobody tried to build nginx from binaries to remove header "server: nginx"?

#

I tried but for some reason it did not work as I wished

# 
RUN sed -i 's/Server: nginx/Server: Apache/g' /nginx-1.20.0/src/http/ngx_http_header_filter_module.c
RUN sed -i 's/nginx\//Apache\//g' /nginx-1.20.0/src/core/nginx.h

not sure where else I am supposed to hide

fervent osprey
#

Hey guys, what are some good ways to deploy your own python scripts to a small (~10), medium (~50-100) or large ( 100+) amount of people? Are there any industry standards or clearly better solutions?

rapid sparrow
#

that would be the minimum I would do

fervent osprey
#

fair, but how do you deliver the tool to the people? I mean, an example solution I can think of is to put up a Flask app and give them the public URL, this works well for 10 people, but for 50 it's already prohibitive.

rapid sparrow
#

then you launch it with gunicorn (instead of python)

#

with multiple workers

#

they will be able to process a lot of people in parallel (sort of inbuilt multithreading)

#

(it handles auto restart of died workers too)

#

it will be able to process thousands of people ;b

fervent osprey
rapid sparrow
#

if you have 4 core PC

#

it will be 8 workers

#

they will be able to process a lot of people

fervent osprey
#

that's cool, i'll definitely check the tool out!

rapid sparrow
#

gunicorn is not capable to serve static files (html/css/js)

fervent osprey
#

lol I'll write down all of this and have an extensive google search. I have a vague knowledge of all these terms and I guess i'll have to know them inside out pretty soon

rapid sparrow
#

this rabbit hole is a deep one πŸ˜‰

fervent osprey
#

so let's say this takes care of the small and medium problem, what if you want to deploy an app to, i dunno, 1000 people?

rapid sparrow
#

I could say you more tools to simplify deployments
but it will increase the amount of tools to learn%

rapid sparrow
#

u could launch redis server, connect to your application
and to use it to cache stuff, so the repeated things would not request sql database too many times

#

plus you can set caching at level of nginx too I think

#

then some of your urls would be served super duper fast before it touches the gunicorn

#

mm...

#

besides caching (btw varnish is popular as web accelerator too)

#

the often used technique named load balancing

#

we distribute the load between multiple equal servers with the same application

#

and using haproxy to send user randomly to one of them

#

of course it all... adds complication to setup things up....

#

...that's why we use docker, docker-compose tools to wrap it into containers
and deploy to servers with remote tools like ansible for quick setup from the code of your project to multiple servers

#

anyway, it is a deep rabbit hole

#

I still wish to learn terraform and k8s one day, which would simplify the process further %

fervent osprey
#

lol i bet. So if I want the weight of the calculation on the user instead, I'd go docker? Like If i want to pack up my scripts into something that resembles a .exe

rapid sparrow
#

docker is a way to pack all your dependencies into... analog of exe for linux servers.

fervent osprey
#

huh that's cool, definitely going to check pyinstaller

velvet spire
#

!pypi ansible

rancid schoonerBOT
ancient umbra
#

Packages discussed here?

tulip forge
#

@fervent osprey there are platforms like heroku, aws, azure, ... that let you pay as you go for hosting, some are even free up to some point

#

kubernetes: Can I ask here? Is there a dedicated server/slack/... somewhere? And how does livenessProbe work - if one container fails, what happens to the others in the same pod?

tulip forge
#

@fervent osprey did you check out anvil? Never tried it but maybe it is relevant for you

shadow crow
#

This is my Dockerfile: ```dockerfile
FROM python:3.9-slim
STOPSIGNAL SIGQUIT
ENV POETRY_VIRTUALENVS_CREATE=false
RUN pip install -U poetry
WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN poetry install --no-dev
COPY . .
RUN python -m migrations
CMD [ "/bin/sh", "debug.sh" ]

At `RUN poetry install --no-dev` I get this:

OSError
Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:227 in cert_verify

OSError
Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:227 in cert_verify

What's wrong, and how can I fix it? Google didn't show anything useful
#

These are my dependencies:

python = "^3.9"
starlette = "^0.16.0"
asyncpg = "^0.23.0"
uvicorn = {version = "^0.14", extras = ["standard"]}
aiohttp = "^3.7.4"
python-dotenv = "^0.19.0"
yoyo-migrations = "^7.3.2"
psycopg2 = "^2.9.1"
reef jay
#

Looking for suggestions. I have a microservice style architecture on small python scripts hung of a message bus. Currently each script launches in a docker of it's own coordinated by docker-compose. Presently I have a dozen of these scripts, but it looks like that will double in short time. As all of these run on the same physical host it is starting to feel a bit in efficient running so many dockers for such small scripts.

#

I am thinking of a way to manage multiple scripts per host. Currently everything binds/bridges to the same local IP anyway.

quiet mason
#

I run this command =>docker run -d --rm --name mongo -p 27017:27017 -v mongodbdata:/data/db mongo and get the error Error response from daemon: invalid volume specification: 'mongodbdata:/data/db'. I am not like into devops per say just a humble web dev but this was done in a tutorial and worked flawlessly, I need this container so that I can host on azure

strange wing
#

hi so i converted an py file to exe and i want to to auto updates for the file how can i do tthat?

gentle solstice
shadow crow
# gentle solstice add `ENV SSL_CERT_DIR=/etc/ssl/certs`

That didn't help.
I think I figured out the issue -- I had requests in dev-dependencies. Because I was running it in --no-dev mode, poetry planned the removing of requests and removed it, which made poetry really sad. So I fixed it by enabling creating a venv.

#

Not sure if it's an issue with poetry, or if I'm really supposed to make a venv.

gentle solstice
#

Using a venv allows you to easily copy it to another stage

shadow crow
#

I'm just starting out with docker, so I looked at how it's done in python-discord's projects on GitHub

#

is that generally a bad idea?

rapid sparrow
shadow crow
#

πŸ€” not sure how that helps

rapid sparrow
#

just a bit more comfortable debug

#

btw, is poetry your preferable env?

#

I am usually used to venv

shadow crow
#

poetry doesn't mean 'not venv'

#

by default, poetry does create a venv

rapid sparrow
# 
 OSError
  Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
  at /usr/local/lib/python3.9/site-packages/requests/adapters.py:227 in cert_verify

  OSError
  Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
  at /usr/local/lib/python3.9/site-packages/requests/adapters.py:227 in cert_verify
#

anyway

#

you can access the inside of the container

#

and see, if it is missing the certificates

#

if it is missing.. you know, to try finding a way to copy them, where they are expected

shadow crow
rapid sparrow
#

all right

shadow crow
#

we discussed in #dev-contrib and decided that creating a requirements.txt from poetry and then installing from it was the best solution

rapid sparrow
#

xD

#

so the best solution is to not use poetry then

shadow crow
lost dust
#

poetry or pipenv? ThinkO_O

#

honestly the only downsight of poetry vs pipenv that I see is that poetry doesn't have as straightforward install procedure, so it might be a turn back for any poor souls that decide to contribute to my projects, or just setting them locally

velvet spire
#

Wdym?

#

Poetry is pretty easy to install and use

#

Pipenv is slow and harder to use IMO

lost dust
#

is it? From what I see in the docs (first time using poetry) I have to run some scary script

velvet spire
#

You mean to install poetry?

lost dust
#

ye

velvet spire
#

You can also install poetry via pipx or pip

#

Tldr can install poetry same way as pipenv

lost dust
#

pip install pipenv
vs
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -
I can use pipx but then that's additional thing to install ChirunoSlain

velvet spire
#

Or just use pip install poetry then

finite fulcrum
#

going through pip should work fine in most cases

velvet spire
#

I use the script first most, then pipx, then pip if necessary

lost dust
#

not sure what dependencies it has? Wihat are the chances it will conflict if I install with pip

finite fulcrum
#

and well, you only install it once

velvet spire
#

It has 25 dependencies iirc

#

That's why I use pipx lol

lost dust
#

ok time to look up pipx KannaSus

velvet spire
#

Therefore it's just pip install pipx && pipx install poetry

lost dust
#

that looks nice

velvet spire
#

Yeah I use pipx for some of my dev packages I use on all projects

lost dust
#

just pipx install poetry will work properly?

velvet spire
#

Mostly black and pre-commit are on pipx

#

Yeah

lost dust
#

looks good, I'll probably switch to poetry then

#

I've also heard about PDM, but it looks really new

velvet spire
#

My dev tools are always pyenv for python version, then use 3.9.6 to install pipx, then install black, autopep8, flake8, and pre-commit with pipx
Install poetry via the script piped to 3.9.6, and then poetry env use 3.9.6, and wait for poetry install to install my 100+ dev dependencies

velvet spire
#

@lost dustany other questions rn?

lost dust
velvet spire
#

cool

pearl glade
#

@olive path wonderful play

olive path
#

ty ty

heavy knot
#

i am trying to deploy my django and react project on elastic bean stack. but i am getting the following error

File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/gunicorn/util.py", line 350, in import_app
Aug 28 05:36:35 ip web: __import__(module)
Aug 28 05:36:35 ip web: ModuleNotFoundError: No module named 'home/wsgi/prod'
Aug 28 05:36:35 ip web: During handling of the above exception, another exception occurred:
Aug 28 05:36:35 ip web: Traceback (most recent call last):
Aug 28 05:36:35 ip web: File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Aug 28 05:36:35 ip web: worker.init_process()
Aug 28 05:36:35 ip web: File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 104, in init_proces

here is my project django.config file in .ebextensions

option_settings:
  "aws:elasticbeanstalk:application:environment":
    DJANGO_SETTINGS_MODULE: "home.settings.prod.py"
    "PYTHONPATH": "/opt/python/current/app/home:$PYTHONPATH"
  "aws:elasticbeanstalk:container:python":
    WSGIPath: home/wsgi/prod.py
    NumProcesses: 3
    NumThreads: 20

my root tree and home tree

//root
.
β”œβ”€β”€ README.md
β”œβ”€β”€ build
β”œβ”€β”€ buildspec.yml
β”œβ”€β”€ db.sqlite3
β”œβ”€β”€ env
β”œβ”€β”€ home
β”œβ”€β”€ manage.py
β”œβ”€β”€ node_modules
β”œβ”€β”€ package-lock.json
β”œβ”€β”€ package.json
β”œβ”€β”€ public
β”œβ”€β”€ requirements.txt
└── src

//home
home
β”œβ”€β”€ __init__.py
β”œβ”€β”€ __pycache__
β”‚   β”œβ”€β”€ __init__.cpython-38.pyc
β”‚   └── urls.cpython-38.pyc
β”œβ”€β”€ settings
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ __pycache__
β”‚   β”œβ”€β”€ base.py
β”‚   β”œβ”€β”€ dev.py
β”‚   └── prod.py
β”œβ”€β”€ urls.py
└── wsgi
    β”œβ”€β”€ __pycache__
    β”œβ”€β”€ dev.py
    └── prod.py
#

any kind of help will help me a lot. thanks

#

also please ping me.

strange wing
#

how can i add an auto updater into my exe file and i convertet the file from py to exe

rapid sparrow
strange wing
rapid sparrow
#

hosting server with updates, and attaching those self updates being downloaded from it

strange wing
#

how

rapid sparrow
#

simple scheme...

#

you split your application into two parts

#
  1. updater/downloader
#
  1. your main application
strange wing
#

yes

rapid sparrow
#

updater just checks and downloads if necessary, the newest version from the server with updates

#

and launches your main application, whatever it is current or downloaded new one

civic grove
#

uhm i was trying to use github actions and this happened for python3.6, can anyone decipher this?

unknown.png
civic grove
#

nevermind i think gh was acting weird, i removed 3.6 anyway since it didnt have dataclasses yet

buoyant atlas
#

Do anyone have experience with Google cloud platform and Kubernetes?

#

I have few questions

civic grove
#

i am back with a question

# 
name: Python package

on: [push]

jobs:
  build:

    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: [3.7, 3.8, 3.9, 3.10.0-rc.1]

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install flake8 pytest

          # the line below fails for windows and possibly macos
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

          # ideally i want 
          # if windows -> do appropriate command to find requirements.txt
          # if macos -> ...
          # if ubuntu -> ...

      - name: Lint with flake8
        run: |
          # stop the build if there are Python syntax errors or undefined names
          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
      - name: Test with pytest
        run: |
          pytest
#

i need help setting the line that pip installs requirements.txt. basically windows doesnt support bash and i need it so that it can detect which os it currently is running on and then execute the correct statement (this is for github actions)

tawny temple
#

Have you tried specifying shell: bash?

civic grove
#

i actually have not, i will try that

tulip forge
#

@civic grove can't you simply skip the whole if? Don't you know that there is a requirements.txt anyway?

civic grove
#

good point

#

i did remove it for now

civic grove
#

i'm a bit surprised though, how does windows support bash?

tawny temple
#

The runners on actions have additional software installed

wooden ibex
#

But I would switch to Powershell for Windows to make your life easier

civic grove
civic grove
wooden ibex
tulip forge
#

@wooden ibex many python programmers use windows. I do. (Not by choice, not at all. )

wooden ibex
rapid sparrow
deft plover
#

Does linix mint have a disk portioning tool

echo dirge
velvet spire
#

Pretty much, yeah

echo dirge
#

poetry install requests... then it will create a virtual environment like pipenv?

velvet spire
#

Er, no

#

What are you trying to do?

velvet granite
#

Hey anyone know an aarch64 headless chromium driver for Selenium? It auto downloads x86/x64 ones which isnt supported(obviously)

echo dirge
#

poetry is alternative to what? pipenv or pip or pipx?

odd geyser
#

I cant install Ta-lib on windows. What should i do?

wooden ibex
tawny temple
#

But it is a common language for automation tools which can conceivably be used on any OS

vague silo
#

has anybody read "The Phoenix Project"? would you recommend it?

wooden ibex
vague silo
#

thank you!

velvet spire
# echo dirge poetry is alternative to what? pipenv or pip or pipx?

poetry somewhat of an alternative to pipenv. Pip and pipx are different things.

Pip is the Python package manager, I think that poetry may even use it internally.

Pipx is a way to install python packages into their own lil virtual environments. You can use pipx to install pipenv and poetry.

Pipenv manages a venv for a project, but only one venv, and imo pipenv is incredibly slow.

Poetry can manage multiple venvs for one project, and is much faster, and has dependency manangement, so as long as your requirements have at least one combination of supported versions, it will find that combination.

tawdry needle
#

There is also Flit, which does the same things as Pipenv and Poetry. It somehow received less attention than the other two, not sure why.

tulip forge
#

pipenv used to be unmaintained for a long time. We have bad experience with it in our team. Broken stuff; not just slow

#

There's also conda which is same but different - better for data analysis, simulations, numpy etc with C or Fortradependencies

#

Fortran

potent peak
#

Hey
Where to Host python Webapplicatoins normally? (im from the php universe). Are there Manged Python hosters or, do i need to setup a root server by myself?

rapid sparrow
potent peak
#

And how to manage to start the application after a reboot?

#

@daring forum cronjob?

rapid sparrow
#

supervisorctl as installed solution

#

or pack to docker, and use docker-compose restarting functions

potent peak
#

Okay, but then you need to manage the deamon by your self?

rapid sparrow
#

it will make it auto start on restart

potent peak
#

And for SSL Setup an apache Reverse proxy? or what is the way to go there?

rapid sparrow
potent peak
#

does certbot work with nginx (im a nginx noob) πŸ˜„

rapid sparrow
potent peak
#

Nice thanks! How do you deploy your apps? (For My Job i use GitLab ci push deployments)

rapid sparrow
potent peak
#

i just used ansible to "sync" my development computers so far

rapid sparrow
#

one day I would wish to learn k8s

potent peak
#

me to xD

#

i started to watch some tutorials.... but is confusing stuff πŸ˜–

rapid sparrow
potent peak
#

How do you manage environments variables ? or "what files to gitignore" normally in pyhton projects

rapid sparrow
#

nginx offers many other functionality.. but lets have it not mentioned for now

rapid sparrow
potent peak
potent peak
rapid sparrow
#

I need log aggreagtor more desperately than k8s at the moment

#

and then I would need learning kafka first for event based architecture

potent peak
#

log aggregator?

rapid sparrow
#

elastic or loki

potent peak
#

ah, we use logstash > es > grafana

rapid sparrow
#

yeah

#

anyway, k8s has awesome interactive official docs
they enable you temporal linux OS in your browser just for training

potent peak
#

but i do not have root access on this environments, so its a bit limited what we can do

#

yeah i should give k8s a try again... but im to involved in many other projects and there is soooo much to learn

rapid sparrow
rapid sparrow
potent peak
#

do you know scrapy?

rapid sparrow
#

I am at the moment BackendArchitectDevSecOps, fulfilling three-four roles at the same time

rapid sparrow
#

not needed yet

potent peak
#

haha yeah me to BackendArchitect/DevOp/EndToEndTestSetup/TeamLeader

#

but in PHP

#

mostly

rapid sparrow
#

xD

#

all right, add QualityAssurance to my titles too then

potent peak
#

yeah i like "Clean Code"

rapid sparrow
#

me three

#

going to learn DDD as next step, not tried yet

#

a book of Eric Evans awaits me about it

potent peak
#

We more like FDD

#

Fear Driven Design xD

rapid sparrow
#

I am more of a Learning Driven Design

#

I can admit, that I often choose tools just for the sake of learning them ;b

potent peak
#

We had a Big Monolith application and now we go to a More Service Oriented Architecture

rapid sparrow
#

oh, I jumped straight to Microservices

#

I've read recently two good books from O'Reilly about it, to make sure everything goes right

potent peak
#

I find it hard to define what is a microservice and what not

rapid sparrow
#

*From monolith to microservices" and "Building microservces"

potent peak
#

We got some tiny applicatons in our ecosystem e.G an Onboarding form another Identiy Provider

#

but IDK if i should call this a microservice or just a Service

rapid sparrow
# potent peak I find it hard to define what is a microservice and what not

I just have certain goals I follow in microservices
keeping as much of my applications in stateless state for easier horizontal scaling and for easier fault tollerance. if it died, all right, I can replace with clone.
and at the same time to have as less rippling effect as possible, when I change smth

rapid sparrow
#

so many similarities

potent peak
#

Web Devlopers live

rapid sparrow
#

if I enable firewall first

    - name: Allow connections on multiple ports
      ansible.builtin.iptables:
        chain: INPUT
        protocol: tcp
        destination_ports:
          - "5432"
          - "22"
        jump: ACCEPT
        rule_num: 1

    - name: conntrack allow RELATED and ESTABLISHED
      ansible.builtin.iptables:
        chain: INPUT
        ctstate: ESTABLISHED,RELATED
        jump: ACCEPT
        rule_num: 2

    - name: Block all the rest
      ansible.builtin.iptables:
         chain: INPUT
         policy: DROP
         rule_num: 5

docker is not wishing to connect %
ERROR: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io: Temporary failure in name resolution

rapid sparrow
#

probably all I need

    - name: Allow connections for docker
      ansible.builtin.iptables:
        chain: INPUT
        protocol: tcp
        source: registry-1.docker.io
        jump: ACCEPT
        rule_num: 3
urban pecan
#

it sounds like an issue for a help channel, but according to my google search that's slightly more complex than anticipated: how to get a list of all dependencies for a specific package (using pip for example) without installing them?

rapid sparrow
urban pecan
#

check.

#

but still - no installation?

velvet spire
#

@deep estuary ..somehow the github filter worker just filtered a pull request creation by my own account

deep estuary
#

lmfao

#

link?

#

& what is the response in the GitHub webhook thing?

velvet spire
#

what do you mean link?

deep estuary
#

to the PR

velvet spire
#

ah

deep estuary
#

what do you see in honeycomb & the webhook page

velvet spire
deep estuary
#

well, is it a 205 or 200

#

sorry, 203 or 200

velvet spire
#

203

#

wait....

#

one sec

velvet spire
#

I just disabled an event that was sending that discord was never emitting, but didn't notice previously---that event is workflow jobs and is sent 4+ times for every commit.....
Disabling that will fix the problems, since the actual problem here is it got 429ed

#

I redelivered the pull request creation, and hopefully this will help in the future since its gotten ratelimited before

deep estuary
#

lol

civic grove
#

i'm new to releasing packages and versioning and devops in general, what's a common workflow for keeping PyPI release and Github release in sync? do you just publish(in my case poetry publish) whenever you create a release on github?

night quest
civic grove
#

ohhhh

harsh summit
#

This might be completely off-topic, if so, I apologize. But - does anyone have experience with neovim? I have a super dumb question, but can't seem to get it to work.

jolly falcon
#

I got a really strange annoying issue, wondering if anyone knows a solution. If got tensorflow running on my machine, and via python ide i can do something fine, but as soon as i try attempt that in a program that supports python scripting, i get an error "tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize"

indigo zenith
#

Are you sure that the "program that supports python scripting" is configured to use the same Python installation your IDE is?

jolly falcon
#

that is what ive been checking, seems so

#

all i have is a py.exe in windows, and then python in appdata/programs

#

trying both via cmd prompt works

#

trying inside each interp works fine

#

but then the program(DaVinci Resolve) gives that error in consolle

#

even more bizaare. I tried to get the Resolve python scripting to just run 'py scriptname.py' as a subproccess

#

and it gives the same error lol

#

but its opening up a whole new cmd prompt and new python that is external

#

and doing CPU tensorcore via Resolve works fine

#

print(os.environ['PATH']) is the same on both

#

super strange

#

its happening on loading up a tensorflow model

jolly falcon
#

hmm

#

"With the new DaVinci Neural Engine using NVIDIA CUDA 10 and TensorFlow acceleration"

#

could it be that there is some sort of built in tensorflow conflicting ?

rugged flame
#

Quick question: is it possible to check out a specific commit on Github?

finite fulcrum
#

yes

wary helm
#

hello i froze some python code with cx_freeze. it returned me no errors but all my methods with which i was writing files or creating directory are no being called anymore. Idk if this is the right Chanel to ask ... but ahh pls help | no one on the internet seem to have had that issue strange --> i put my program folder into another folder of the same name now it works - strange

finite fulcrum
#

you can substitute the branch for a rev

rugged flame
finite fulcrum
rugged flame
rugged flame
#

When doing a PR review, is there a way to leave comments on any part of the code in the working tree, rather than just the parts in the diff?

#

On Github, that is.

finite fulcrum
#

Not to my knowledge

#

only a few lines around the diff, but not for the parts you expand

jolly falcon
#

you click the lines dont you, and it lets you add a comment

#

oi any part of the working tree, dont think so. You can prob reference it in an issue post or something

tulip forge
valid ice
#

hello, i am trying to create an exe using pyinstaller, the exe is getting created just fine but the program requires a pickle file which contains credentials and an ico file which has the application icon. could someone please advise how could i achieve this ?

reef jay
#

anyone up for a devops question?
Got a bunch of microservice style python scripts, currently running each in a docker container. I like the architecture but it seems a little overkill. Is there any middle ground python process manager that I can run on docker which could run 2, 3 or more scripts.

rapid sparrow
#

create cfg to run your thing

#

sydo systemctl daemon-reload, load daemon configs

#

sydo systemctl start daemon_name.serivce, to enable it

#

this solution is better suiting when you don't use docker though

#

if you use docker, I think probably better putting everything to separate container

#

because it will allow easily making health checks

#

hmm, u could add your own custom healthcheck to container though, which would check health of multiple apps

#

then it will matter less if more than one process running in container

reef jay
#

Better than my current approach which is basically instantiating multiple service objects (basically mqtt event handlers) sharing a cache in the same process. eg. All my radiator and boiler controllers are in the same process. All the schedulers run in one process. One process per docker. However, this limits flexibility, in the ability to update or restart a single component within. It also provides the ability to circumvent the architecture and sneaky things across in memory, which leads to bugs.

#

I'm still at the stage of wondering if it's worth going to aggregate process dockers and a management API. It might not be worth it. Asides a bit of clean up once in a while, the one docker, one process resulting in about 20 containers running, isn't that bad.

rapid sparrow
reef jay
#

Yea, I have docker-compose. I think one thing would help is using a single parameterized image, rather than building each

#

they are all alpine with a custom pip file

rapid sparrow
#

anyway, the daemon solution above, can be nicely put into ansible

#

that's why I like it

#

so, essentially we could have the same... building instruction from zero ;b

#

without any unnecessary dependencies

neat grove
# 
from typing import Text
from telegram import *
import telegram
from telegram.ext import *

import latino
from latino import lat_ita
from latino import lat_eng

bot = Bot("censored")

updater = Updater("censored", use_context = True)

dispatcher = updater.dispatcher


#if you want to make another command for your telegram bot
#start copying there

def test_function(update:Update, context:CallbackContext):
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = "Hi! I am a Latin dictionary! At the moment I support only two languages, English and Italian.\nThe complete list of my commands is avaible with /commands.\nIf you have some problems or any type of issues you can type /help."
        )

start_value = CommandHandler("start", test_function)

dispatcher.add_handler(start_value)

#and finish here

#start pasting here your new commands


def commands_list_function(update:Update, context:CallbackContext):
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = "Command list:"
        )

start_value1 = CommandHandler("commands", commands_list_function)

dispatcher.add_handler(start_value1)
#end


def word_function(update:Update, context:CallbackContext):
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = latino.lat_eng()
        )

start_value2 = CommandHandler(, word_function)

dispatcher.add_handler(start_value2)
#end

#do not tuch this
updater.start_polling()
#do not touch this

Hi! I'm trying to make a Latin dictionary in a Telegram bot. I did all, but now in the last function I don't know how to implement that if I type every Latin word I want it automatically recognizes it as a word in the library latino. How can I do it?

#

It is very hard, I know, but if you help me I will really appreciate it.

fallow terrace
# 
from  latino import lat_eng, lat_ita
from typing import Text
from telegram import *
import telegram
from telegram.ext import *


bot = Bot("")

updater = Updater("", use_context = True)

dispatcher = updater.dispatcher

def test_function(update:Update, context:CallbackContext):
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = "Hi! I am a Latin dictionary! At the moment I support only two languages, English and Italian.\nThe complete list of my commands is avaible with /commands.\nIf you have some problems or any type of issues you can type /help."
        )

start_value = CommandHandler("start", test_function)

dispatcher.add_handler(start_value)


def commands_list_function(update:Update, context:CallbackContext):
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = "Command list:"
        )

start_value1 = CommandHandler("commands", commands_list_function)

dispatcher.add_handler(start_value1)
#end


def word_function(update:Update, context:CallbackContext):
    word = update.effective_message.text.split()[1]
    bot.send_message(

        chat_id = update.effective_chat.id,
        text = lat_eng(word)[0]["traduzione"]
        )

start_value2 = CommandHandler("eng", word_function)

dispatcher.add_handler(start_value2)
#end

#do not tuch this
updater.start_polling()
#do not touch this

the solution

finite fulcrum
#

I'm remaking an app, where practically the whole source will be rewritten (except for a few files like utils) but the behaviour will mostly be kept the same, would it make sense to make a new orphan branch to then replace main with, or to incrementally update the old tree? (I don't care about forks as there are no active ones)

wooden ibex
#

In case you need to make bug fixes to old app

finite fulcrum
#

That's also not a huge concern as it has been around 2 years since I did any meaningful contribution to it and most things will require the rewrite

#

I'm wondering whether it's worth it to preserve the old history, if it's not going to be valuable in the future considering I'm rewriting it as a whole and will make the new commits a bit uglier unless I make an "initial" commit from master that will just replace it whole anyway

wooden ibex
slow cave
#

Hey, I have couple questions about using tox.

  1. Is it worth moving to tox in a project that is proprietary/corporate and does not need support for different python versions, etc., so it won't be using the "matrix" python/libraries versions feature of tox?
  2. Do you think it is worth having both Makefile and tox in a project (since tox is kinda like virtualenv and Makefile combined)?
rapid sparrow
#

"/" sort of makes folders in my branches

#

From time to time I even open it

tawdry needle
#

i personally like makefiles a lot, i tend to use them to provide a unified set of "entry points" into the dev side of my projects

winter spruce
#

please ping me when help, thanks!

#

also, does unittest support async functions

velvet spire
#

your workflows are taking 1 to 4 minutes...

winter spruce
#

its mainly CodeQL

#

it takes 4 mins

velvet spire
#

yeah

winter spruce
#

can I cut it down?

#

I didnt cache CodeQL alone, because if I do, I might hit the cache limit

velvet spire
#

idk, I don't use codeql

winter spruce
#

wait, do I even need codeQL

velvet spire
#

for starters, I think you would only need to run codeql if a *.py file changed

#

rather than this

winter spruce
#

my tests run in parallel across different python versions and os

winter spruce
velvet spire
#

tldr these

rancid schoonerBOT
#

.github/workflows/codeql-analysis.yml lines 7 to 10

paths-ignore:
  - 'docs/*'
  - '*.rst'
  - '*.md'```
winter spruce
velvet spire
#

trying to find the right docs one sec

winter spruce
#

okay, thank you!

velvet spire