tools-and-devops

[rapid sparrow]

pip3 freeze > requirements.txt, it freezes current used versions of libraries

allowing them being reinstalled later with pip3 install -r requirements.txt

[woven jetty]

pip freeze > requirements.txt after install or instead of install?

[rapid sparrow]

after install

also be sure to work in venv

it will make sure your project has only dependencies used for the current project

[woven jetty]

FROM tiangolo/uwsgi-nginx-flask:python3.8
RUN apk --update add bash nano
ENV STATIC_URL /static
ENV STATIC_PATH /var/www/app/static
COPY ./requirements.txt /var/www/requirements.txt
RUN pip install --upgrade pip
RUN pip3 install notion-client=0.8.0
RUN pip install -r /var/www/requirements.txt
RUN pip freeze > requirements.txt

as such?

[rapid sparrow]

well, yeah. but use also pip3 as i said

[woven jetty]

ah ye sorry

[rapid sparrow]

and I would recommend probably putting notion-client string after requirements.txt line

it feels better

[woven jetty]

Done that now

Ill try do sudo bash start.sh now

This time I got a different error way earlier

https://i.imgur.com/5Yj90NU.png

I managed to get past that one

however

Actually now it seems to have installed notion-client:D

Nice!

Time to try and figure out why I am getting Internal Server Error now :S

[devout temple]

even tho i have liblzo lib installed, i still seem to geting to No module named lzo. what may be the potential cause?

[tawdry needle]

can you be more specific? liblzo sounds like it contains a shared library, not a python package. you might need to install it with pip, or look for something called python-lzo in your package repository

(note: do NOT run pip install as root)

(you will probably break your system if you do that)

[serene seal]

I'm a little confused why when I try to execute the equivalent of a makemigrations command to my docker-compose it isn't seeing any of my env variables but my app itself does

$ docker-compose run fastapi alembic upgrade head
The DB_USER variable is not set. Defaulting to a blank string.
The DB_PASSWORD variable is not set. Defaulting to a blank string.
The DB_NAME variable is not set. Defaulting to a blank string.
The PGADMIN_EMAIL variable is not set. Defaulting to a blank string.
The PGADMIN_PASSWORD variable is not set. Defaulting to a blank string.
The CELERY_BROKER_URL variable is not set. Defaulting to a blank string.
The CELERY_RESULT_BACKEND variable is not set. Defaulting to a blank string.
The barbot_dev_key variable is not set. Defaulting to a blank string.
The sentry_dev_key variable is not set. Defaulting to a blank string.
Creating compose_fastapi_run ...
Creating compose_fastapi_run ... done
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.

as an example

[rapid sparrow]

At least docker-compose.yml

[indigo zenith]

LOL nice... that is the appropriate response to so, so many questions 🤣

[rancid schooner]

:incoming_envelope: :ok_hand: applied mute to @serene seal until <t:1642827517:f> (9 minutes and 59 seconds) (reason: newlines rule: sent 119 newlines in 10s).

[rugged flame]

!unmute 266766446665793545

[rancid schooner]

:incoming_envelope: :ok_hand: pardoned infraction mute for @serene seal.

[serene seal]

❤️ apologies, didn't realize it was over the limit like that

[rugged flame]

Please use the paste service for large amounts of code:

!paste

[rugged flame]

No worries, it's just that long code blocks with many line breaks tend to trip the spam filter.

[sweet token]

How do I find an stl file in the inspect element?

[serene seal]

It's quite a sizeable compose at this point 😛

https://paste.pythondiscord.com/ikaboconul.http

[serene seal]

Pasted up the code 🙂

just to consolidate so the post is easier to read/respond to:

I'm a little confused why when I try to execute the equivalent of a makemigrations command to my docker-compose it isn't seeing any of my env variables but my app itself does

running the following command:

docker-compose run fastapi alembic upgrade head

$ docker-compose run fastapi alembic upgrade head
The DB_USER variable is not set. Defaulting to a blank string.
The DB_PASSWORD variable is not set. Defaulting to a blank string.
The DB_NAME variable is not set. Defaulting to a blank string.
The PGADMIN_EMAIL variable is not set. Defaulting to a blank string.
The PGADMIN_PASSWORD variable is not set. Defaulting to a blank string.
The CELERY_BROKER_URL variable is not set. Defaulting to a blank string.
The CELERY_RESULT_BACKEND variable is not set. Defaulting to a blank string.
The barbot_dev_key variable is not set. Defaulting to a blank string.
The sentry_dev_key variable is not set. Defaulting to a blank string.
Creating compose_fastapi_run ...
Creating compose_fastapi_run ... done
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.

as an example

Docker Compose File:

https://paste.pythondiscord.com/ikaboconul.http

Any ideas what could be causing the above behavior? It works fine and has the env variables when I do Docker-Compose up but when I do the above command it can't find the env variables.

[rapid sparrow]

U used environment before in other container code
environment:
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASSWORD}
- POSTGRES_DB=${DB_NAME}

But u did not use them or env_file syntax to transmit any env values to FastAPI
I don't understand why u expect FastAPI to have set any env vars then

[serene seal]

What is wrong with that?

Oh, I see what you mean so I would have to pass all of the environment variables into the fast API done?

I would have thought it would have access to the environment variables if they are globally available but I guess now that you point that out it makes sense

[rapid sparrow]

Yes

[rapid sparrow]

Use env_file syntax if u wish reusability

U can set same vars in it
And then uploading to containers same file with env vars

[serene seal]

How do I do that? Are there any shortcomings to note about doing it that way?

[rapid sparrow]

web:
  env_file:
    - web-variables.env

web-variables.env

VAR1=123
VAR2="467"

[rapid sparrow]

The only shortcoming is that it is ill advised to store important secrets and submitted to repo.
Same problem as if u had them written in docker-compose env vars directly

And that u can set only string values with it

Not sure if comments are allowed in it

[rapid sparrow]

In theory, setting up env values in docker compose file directly could be better because it could be having syntax to read env values from the operational system it runs in

So docker compose file will just redirect them from current OS to container

[serene seal]

Excellent! Thank you, I will try this in the morning! Appreciate the explanation by the way!

[heavy knot]

I'm so stupid, any help? I just want a little tool/program when you click the Generate button, it types a sentence for you.

[woven jetty]

@rapid sparrow Hey so just wanted to thank you for the help yday, managed to get my flask app up and running

When it came to you saying you would suggest me to use venv, is it this you mean? https://www.arubacloud.com/tutorial/how-to-create-a-python-virtual-environment-on-ubuntu.aspx

or should I change something in my dockerfile?

[rapid sparrow]

Yes. In containers it is useless.

But when u develop outside of containers, it helps to control dependencies for python application

[woven jetty]

Ye so it will look something like

My Ubuntu
My Venv
Container

[rapid sparrow]

Instead of polluting global python in the whole system
U create local python that has dependencies only for current project

[woven jetty]

Thanks ^^ Ill give that a try

[rapid sparrow]

Look it like

Ubuntu
My Venv during development

Ubuntu
Container during staging testing

Sometimes we need venv inside containers

But rarely

[woven jetty]

AH

Think I start to understand it now^^

[rapid sparrow]

Then it would be

Ubuntu
Container
Venv
App

[rapid sparrow]

The point is ability to

pip freeze dependencies for current application easily

And not having conflicts in dependencies with other apps
During development

[woven jetty]

yes

[serene seal]

Upon further consideration of the question last night regarding alembic upgrade not seeing the environment variables from the container, it shouldn't even need most of those environment variables, the only ones I imagine it would actually need are the database access variables. I am going to fiddle with it and see if it actually applies the migrations to the database table despite this issue and if it does I guess it is a non-issue but if it doesn't then I will have to figure out what is going on

[serene seal]

so it made my migrations seemingly with no issues, traceback:

$  docker-compose run fastapi alembic revision --autogenerate -m "New Migration"
Creating compose_fastapi_run ...
Creating compose_fastapi_run ... done
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.autogenerate.compare] Detected added table 'post'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_post_id' on '['id']'
INFO  [alembic.autogenerate.compare] Detected added table 'server_controllers'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_server_controllers_id' on '['id']'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_server_controllers_server_id' on '['server_id']'
INFO  [alembic.autogenerate.compare] Detected added table 'user'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_user_id' on '['id']'
Generating /app/alembic/versions/b9b585a270ec_new_migration.py ...  done

but when I make the migrations it doesn't seem to actually be running the upgrade:

$ docker-compose run fastapi alembic upgrade head
Creating compose_fastapi_run ...
Creating compose_fastapi_run ... done
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.

[woven jetty]

Hey @rapid sparrow So I got another question.
Just had time to experiment a lil with the venvs. So I created a venv, activated it, however all the dirs from my "real" environment were also on the venv. I did a pip install requests on the venv, and then deactivated the venv and saw it hadnt installed requests on the "real" environment so when it came to that all be good. So I guses my question is, what stays in the venv and what doesnt?

Since my folders from my real environment also exists in the venv

I hope my question make sense

is the venv only for pip packages ?

[rapid sparrow]

Venv creates a full basic environment with standard packages plus everything u will install to it

Venv is full python of certain version packed in one place

It has only one weakness

Venv can't be relocated. It depends on absolute paths. It can be relocated in some cases between operational systems, as long as destination path copies origin

[woven jetty]

Okay

So a venv is a python environment

[rapid sparrow]

People usually freeze venv dependencies and reinstall venv fully later

[woven jetty]

A venv isnt a FULL virtual environemnt as in, if I make a folder in my venv, the folder wont be "outside" the venv?

[rapid sparrow]

What does it mean

[woven jetty]

Well I am trying to understand how "far" the venv goes

As in

After I activated my newly made venv

I could see I could ofc do pip install X

and X would only be inside the venv

so if I deactivate the venv X wouldnt be there no more

which is great

but

[rapid sparrow]

Venv is just independent copy of python and pip installation.. That u can create for every project

[woven jetty]

ye that answered my question:P

venv isnt a full virtual "ubuntu" environment persay

its "just" a separate python installation persay

[rapid sparrow]

No. It is not related to virtualizations

It makes isolated only python environments

[woven jetty]

ye okay^^

So it's "just" for development purposes I guess?

You still run your docker outside the venv ?

or

Also how do you freeze venv dependencies ?

[indigo zenith]

Usually pip freeze > requirements.txt

[woven jetty]

Ah thanks

Last question for tonight

On my windows environment I can do auth=os.environ["BLAHBLAH_TOKEN"]

how do I add values to system environment on ubuntu? and how do I access their values as I do in the above example

[finite fulcrum]

given a .cer file, how can I verify that an executable was signed by the cert it was exported from?

[indigo zenith]

I can't think of anything majorly different between Windows and Ubuntu in that regard. Check the documentation for os.environ but the example you gave should work cross platform

[indigo zenith]

What OS? With Python or you don't care how?

[finite fulcrum]

On Windows, through python

[serene seal]

So I'm interested in using Redis to cache API calls in my FastAPI app, I found Redis has some documentation about doing exactly this -

https://developer.redis.com/develop/python/fastapi

However, they want to use aioredis, I am using redis via Docker-Compose and I don't think I'm pulling the aio version - how can I correct this? I'm going to see if I can make sense of what needs to be done and see if I can better understand the changes required. Anyone have any insight?

[woven jetty]

@rapid sparrow By chance you also know a lil about nginx x)?

[rapid sparrow]

my main web server by accidental chance

[woven jetty]

Well I made a self signed certificate

if I go to https://IPHere it works

however

With my flask application I have urls like https://Iphere/XXXXXXX

And that wont work, only works on http

so I cant get any webhooks on https atm

Do you know a solution to get the /XXXXX to work?

[rapid sparrow]

very strange domain, which is not in any domain zone

[woven jetty]

Well

I havnt bought a domain

so I am just connecting with my public IP

Hence why I self signed

[rapid sparrow]

I guess I will be quoting this man very often

[woven jetty]

lmao

Well first of all

http://167.99.229.124/
https://167.99.229.124/

As you can see these work

but

[rapid sparrow]

yes, regular nginx default web page

[woven jetty]

my flask "routes" doesnt work on https:
http://167.99.229.124/zendeskIssue - works
https://167.99.229.124/zendeskIssue - Doesnt work

Ye I dont have any web "pages" persay

Just gonna use this for listening to webhooks

basically

So dont have a fancy webpage xd

What code do you want to see the nginx/sites-available/default?

or

or were you thinking of something else?

[rapid sparrow]

I don't see any Nginx code that is used to direct to flask

[woven jetty]

and that is what I am curios about how to go about

[rapid sparrow]

here is an example of very dumb code that does that

[woven jetty]

I have build my Flask project in /var/www/NotionProd

along with my Dockerfile

https://i.imgur.com/jeEuvak.png

[rapid sparrow]

upstream django {
  server web:8000;
}

server {
   listen 80;
   listen [::]:80;
   return 301 https://$host$request_uri;
}


server {
  listen 443 ssl;
  listen [::]:443 ssl;

  server_tokens off;

  # # location of SSL certificates
  ssl_certificate /app/web/ssl/ssl.crt;
  ssl_certificate_key /app/web/ssl/ssl.key;

  location / {
    proxy_pass http://django;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
    proxy_redirect off;

  }

  location /static/ {
    alias /app/web/build/static/;
  }

}

[rapid sparrow]

the add_header thing not sure if would be useful for you

read it with ignoring it for now, but just keep in mind in case of being needed

actually I will delete it

in order to be not confusing

as possible alternative static files
instead of

location / {
    alias /app/web/build/;
  }

can be served like

root /app/web/build;
location / {
    try_files $uri $uri/ =404;
 }

not really knowing the difference 🤔

[rapid sparrow]

and just to be sure I offered nginx config with http 80 strict redirect to 443 https

i am not allowing http in my config, just redirecting it

[woven jetty]

Ah fair

Someone just called so i need to try this in a bit

Thanks s ton tho

[woven jetty]

Is this the entire file in sites-available?

[rapid sparrow]

yes

replace default.conf and have happiness

[woven jetty]

https://i.imgur.com/BEU36Mu.png like this ye?

also what is the "upstream django" part on port 8000?

When I do sudo nginx -t
I get:
https://i.imgur.com/4tEL23z.png

[rapid sparrow]

it is my framework django instead of flask

i have it available at localhost:8000

in another container 🤔

of docker-compose or kubernetes-pod

so it has domain django to address it, that is resolved to its ip address

[woven jetty]

hm

Well

I post to http://167.99.229.124:56734/zendeskIssue

and want this to be https

[rapid sparrow]

replace with localhost:5000 or whever you host your flask

[woven jetty]

So 56734

web is your server name?

Should I replace server web:8000; with 167.99.229.124:56734

?

It seems to say syntax is successful now

after replace web:8000 with my serverip:flaskport

hm still goes to http and not https

I have opened up in my firewall, I have done a:
sudo systemctl restart nginx

I have even rebooted

but I still cant access https://167.99.229.124:56734/zendeskIssue

only as http

[rapid sparrow]

    log_records = []
    proc = subprocess.Popen(
        f"echo 123; sleep 2; echo 456; sleep 2; echo 789",
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        shell=True,
    )

    for line in proc.stdout:
        line = line.decode('utf-8')
        print(line)
        log_records.append(line)
    proc.wait()

I wonder how the hell it works

WHY IT WORKS
there is no proc.wait() in the loop 🤔

still nice that is works though

[woven jetty]

lol

[rapid sparrow]

it dublicates stdout log in real time

to stdout and to variable 😉

during command run

[woven jetty]

but odd I cant get my shit to work lol

Gonna see if I can find a Linux/Ubuntu discord and pay someone to fix this for me lol, tired of nginx lol

Thanks for trying tho

[heavy knot]

How could I add authentication my application. Like someone needs to have a license and enter it and I don’t wanna handle it in python I just want an api for this that is secure

[rapid sparrow]

Where to set Celery worker settings? I wish to declare worker_prefetch_multiplier = 1

oh yeah, found it is possible to set at least in worker launching CMD options

[heavy knot]

👋 Hey everyone, I'm Alex and am working on https://cogram.com/, a coding assistant for data science. We're currently exploring context-awareness of a team's shared codebase, eg. given a query in natural language or a line of code, Cogram would suggest relevant code from a team's Git repository. To understand if this is actually useful --

**Do you lose time searching for code snippets in a shared codebase, if you work in a team? Is this a pain point?
**
Would love to get your thoughts on this!

[rapid sparrow]

no. I have CTRL+Shift+F to use in VS Code in order to find any code snippet

It will render me comfortably at the left all code occurencies in the project

with easy moving to them by click mouse one time, to show them at the right

I have even flexible settings to search by REGEX record

for when I need more powerful search

[crisp wing]

I wonder what are monorepos used for

I know I don't need it and don't want them yet but just curious

There's currently two tools I know for it, Nx and Turborepo (now a Vercel product)

[empty fable]

How to use version control in vsc ssh fs extenstion?

Said no git installation detected, i think it meant local not ssh

[rapid sparrow]

I just wish to mention that gunicorn has beautiful icon

[crisp wing]

G r e e n

[vernal mirage]

Hi all, I've got a tech test interview on Thursday for a python infra role (CI, testing etc.)
What sort of questions / problems should I be researching??

Could do with some pointers

[heavy knot]

thanks for sharing this!

[gilded lichen]

I need help in figuring out what's wrong with my exe... The exe contains same code as the py file https://paste.pythondiscord.com/aqapuhenob.py
But for some reason when i open a not allowed app, while it used to work in pycharm it doesnt notify nor play audio in the exe edition.

[rancid schooner]

Hey @gilded lichen!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

[gilded lichen]

https://paste.pythondiscord.com/tomokazike.sql

logs

i really need to fix it please help me.... I want the exe to work :(

[fading blaze]

So I installed pygraphviz by first downloading and running the exe for graphviz and then by running the pip command they said to run in the pygraphviz docs (https://pygraphviz.github.io/documentation/stable/install.html#windows-install), in a venv.

Then I made a jupyter kernel for that venv and tried importing pygraphviz in a notebook with that new kernel, which didn't work.

Then I tried installing pygraphviz through jupyter directly using sys.executable as jakevdp suggested in his blog:

In [7]: !{sys.executable} -m pip install --global-option=build_ext --global-option="-IC:\Program Files\Graphviz\include" --global-option="-LC:\Program Files\Graphviz\lib" pygraphviz

then I tried importing pygraphviz, and got this error

     11 # Import the low-level C/C++ module
     12 if __package__ or "." in __name__:
---> 13     from . import _graphviz
     14 else:
     15     import _graphviz

ImportError: DLL load failed while importing _graphviz: The specified module could not be found.```

getting this error trying to import pygraphviz even though both pygraphviz and graphviz appear to be installed

[tawdry needle]

@fading blaze if you look in C:\Program FIles\Graphviz\lib, do you see a _graphviz.dll or something like that?

[fading blaze]

No I don't see a file like that

Also I tried import pygraphviz as pgv in terminal python and it worked fine. The error only happens in Jupyter notebook

[fading blaze]

okay, i got it working in google colab using this: https://stackoverflow.com/questions/58535604/installing-pygrahviz-in-google-colab

I'm still not sure why it didn't work in plain jupyter but colab will work

[tawdry needle]

this suggests that you might have 2 different python installations, and that it is installed cocrrectly in one but not the other

how did you install jupyter? you might need to set up a jupyter "kernel" that points specifically to the python that you used to install

what does py -0p show?

[fading blaze]

Yeah that makes sense

I think it also would explain why I installed pgv in the venv, made a kernel from that venv, but when i made a notebook using that kernel importing pgv didn't work (it said module not found or something).

(That lead me to this blog [https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/] and using !{sys.executable} install pygraphviz or whatever within Jupyter itself. It no longer said module not found when imported but got the new errorDLL load failed while importing _graphviz..)

[fading blaze]

 (venv)         C:\[venv-path]*
 -3.8-64        C:\Python38\python.exe
 -3.7-64        C:\Program Files\Python37\python.exe
 -3.6-64        C:\Users\[user]\AppData\Local\Programs\Python\Python36\python.exe
 -3.6-32        C:\Users\[user]\AppData\Local\Programs\Python\Python36-32\python.exe```

i installed jupyter using pip install notebook iirc

[brazen locust]

you can launch a Jupyter notebook without prior installation, just blogged about it https://upcycled-code.com/blog/executable-documentation/

[small parrot]

How can I zoom out using the geckodriver for selenium

[indigo zenith]

That's a Google question ;) https://stackoverflow.com/questions/32135085/how-to-zoom-out-of-page-using-python-selenium

[small parrot]

browser.execute_script("$('#values').css('zoom', 5);")

[small parrot]

how you zoom out to a 80%??

[indigo zenith]

I would assume its driver.execute_script("document.body.style.zoom='80%'")

[manic temple]

Hi!!

I am trying to test bls signature verification in local GO etherem environment
from web3.providers.eth_tester import EthereumTesterProvider

How can i point it to localhost ??

[rapid sparrow]

I invoke my celery tasks like task = my_task.delay(params)
and I discovered that if all workers are busy... the task is not taken and my_task.delay(params) is awaiting in sync mode, until someone frees the queue

I did not expect such behavior 😦

I expected the task being sent to redis, and being awaiting until some worker is free to process the task

is it possible?

[rapid sparrow]

just to be sure I launch my workers like that 🤔
2022-01-25T08:43:42.531386814Z stdout F venv/bin/celery -A core worker -l INFO --prefetch-multiplier 1 --soft-time-limit 1200

[rapid sparrow]

nvm, SOLVED, wrongly utilized async library to get them started

[visual heron]

Hi

[devout tangle]

Hey everyone, which tools do you use to manage your dependencies? pip-tools? poetry?

wondering what the "go-to" is these days

[rapid sparrow]

I use just venv

and pip freeze from inside the venv

python3 -m venv venv
source venv/bin/activate
pip install pytest 
pip freeze > requirements.txt # saving stuff
pip install -r requirements.txt # reinstalling from zero saved

[devout tangle]

uf hahaha

[rapid sparrow]

I keep requirements.txt for prod ready stuff
and requirements.dev.txt that contains my dev dependencies too

[rapid sparrow]

what's a constraints.txt?

https://pip.pypa.io/en/stable/user_guide/

I don't get it 🤔

what is the use case

https://stackoverflow.com/questions/34645821/pip-constraints-files

Oh I see

Videos are useless

anyway, If I got it right... constraints are for the all secondary-nested dependencies

without forced installation of them

I guess it is cool indeed thing to use
https://luminousmen.medium.com/pip-constraints-files-b684992705b6

increased readability of requirements.txt is surely welcome

the only regret I have

that I already have a mess without it 😉

good oportunity to refactor though

I am passionately hateful of any learning videos 😉

in my opinion they are mistake in 99% of cases, a mega huge waste of time

books, docs and texts are always better in content and faster to read

I don't care how much they are reputable.
I tried to listen even the most expensive high quality videos like MongoDB made to represent their product

it was still useless in comparison to docs

While someone tries to explain smth in video

I can re read something 4 times at least

with navigating quickly where I struggle and finding quickly more detailed explanations and additional info

with reading faster where I get it

Text is easy to navigate and comprehend, it is UX is perfect. Videos are not.

and I had 6 years of university also full of useless lections, where teacher was in uncomprehendable way something bubbling for few hours every time

once I understood this type of format is useless for me, I skipped 95% of them , and prefered reading and practicing on my own ;b

skipping almost all lections did not stop me from succesfully graduating.

I can say there were exceptions. Some teachers could give materials in a good way. But they are exceptions. Very rare exceptions

I needed few months ago to learn all basics about Kubernetes, I made a go through The Kubernetes Book and in fastest possible time learned all the basics

same thing before I did for Ansible with the book Practical Ansible 2

Books are giving me fastest way to learn all the basics and to start doing practice

And I can check their reviews at Amazon, to find the BEST books, which are having highest scores and reviews, from hundreds of people

To get the best quality material

to learn basics and advanced levels

[indigo zenith]

"Reading is better than watching, and doing is better than reading. " I think that works as a general rule for most people most of the time when it comes to mastering new skills

I agree though that the docs are not always the best thing to read first

[rapid sparrow]

well, that's why probably as one of best alternative ways to start doing something are the Interactive Tutorials

they are quite good often too

a healthy mix of reading and practice at the same time 😉

[indigo zenith]

I just find I never have the patience for videos, takes so much longer to get the same information

If I must I skim the transcript

[olive path]

How do I get started running Nginx on my Windows computer (I need it for the TLS support with WebSockets)? I have a nginx.conf file from an open source project I have checked out.

[vernal harbor]

Hey all -- having an issue with AWS / Lambda / Async/SQLAlchemy

Somehow SQLAlchemy session is being called twice

[uneven saddle]

Hi all , AWS cloud engineer anyone here ?

[rapid sparrow]

https://www.pythondiscord.com/pages/guides/pydis-guides/asking-good-questions/

What Not To Ask
Q: Can I ask a question?¶
Yes. Always yes. Just ask.

Q: Is anyone here good at Flask / Pygame / PyCharm?
There are two problems with this question:

This kind of question does not manage to pique anyone's interest, so you're less likely to get an answer overall. On the other hand, a question like "Is it possible to get PyCharm to automatically compile SCSS into CSS files" is much more likely to be interesting to someone. Sometimes, the best answers come from someone who does not already know the answer, but who finds the question interesting enough to go search for the answer on your behalf.
When you qualify your question by first asking if someone is good at something, you are filtering out potential answerers. Not only are people bad at judging their own skill at something, but the truth is that even someone who has zero experience with the framework you're having trouble with might still be of excellent help to you.
So instead of asking if someone is good at something, simply ask your question right away.

[heavy knot]

I don't know if this belong to tools-and-devops, but everytime when I try to run different Python programs, I get same output: "Python nije prona"

[indigo zenith]

I don't know what language that is, what does it mean in English? Do you have Python3 installed?

[heavy knot]

Yeah, I have Python3 installed it. Reinstalled it and anyway same error. Message is in my native language is in my native language and it means "Python isn't found", but it's not fully printed, full message would be "Python nije pronađen." Guess because of letter "đ"

And if I run python3, just that, then Microsoft Store is opened

[heavy knot]

So it seems that for Windows there is no python3 command, but py

[tawdry needle]

you can set it up to use python.exe or python3.exe, but yes, windows comes with a special launcher called py.exe

[harsh nexus]

Does anyone know about a crypto payment gateway that does not require Django?

[dull root]

Oh you wanna learn the fundamentals 5000000 times over again? Have a banger salary of 200k? Well watch this shit.... aNd yOu wiLl bEcOmE a PrO wItH sOmE sImpLe PrOJecTs.

[dull root]

Read the docs. Thats where the key to success is.

[tawdry needle]

"learn the fundamentals" is generous. "have someone blab examples and weak incomplete explanations at you" is more apt for a lot of videos that i've seen.

[tawdry needle]

there is no such thing as an "https key". you just need to change the git remote url, you don't need to change the actual data that you cloned; it's identical.

by default, the remote will be called origin, so you can do this:

git remote set-url origin <new-url>

in this case, the <new-url> should be the ssh url, so for example you might write:

git remote set-url origin '[e-mail]:aurimus/my-excellent-project'

[crisp kettle]

What's the guidance for science-related python packages with a build system in the context of apple silicon / M1? e.g. "how to install numpy / pandas" on m1 is a popular search term and there's a bunch of stuff for one-off installs, but what about in the context of a system like poetry? I'm starting to think that the best way to do that on m1 is to just do "rosetta-only" & skip the arm64 stuff entirely

Especially considering some of our projects are still specified for py3.7 and most of the 'fixes' are probably only landing on 3.8+

[tawny temple]

Hello, what's the advantage of pip-sync a.txt b.txt from pip-tools over pip install -r a.txt -r b.txt? It seems like pip-sync may support some extra arguments, but I am only curious about this very basic use case.

[next scroll]

from what I understand, the advantage is that you can pip-compile sets together.

EG if you had base reqs, dev deps and N other deps, you could compile base+dev together into a single file and then pip-sync just that single text file to install base and dev.

You can have arbitrary breakdowns and sets

[smoky compass]

Does anyone have a preferred tool for virtualenv management compatible with M1 Macs? Something similar to pyenv-virtualenv for example?

[tawdry needle]

is pyenv-virtualenv not compatible with m1? try virtualenvwrapper if you like "centrally"-managed venvs. but it isn't pyenv so it can't install different python versions

[tawdry needle]

i think pip-sync is pip-compile + pip install

[maiden shadow]

How can I set up a remote development environment in AWS?

[sly sleet]

How would I split submodules (that are all under one module) in a pip package into separate installable components?

[tawdry needle]

you have foo.a and foo.b, and you want to make them two separately-installable distributions? you will need a namespace package

@sly sleet
https://packaging.python.org/en/latest/guides/packaging-namespace-packages/
https://www.python.org/dev/peps/pep-0420/

[smoky compass]

Oh maybe. I was just under the impression it was basically unmaintained at this point

[tawdry needle]

i am not sure, it might be

[indigo zenith]

Spin up an ec2 instance and have it... Beyond that depends what you're trying to do exactly

[maiden shadow]

@indigo zenith my primary goal is to allow a team of devs to use remote development

Does that mean each dev has to have their own ec2 instance?

[indigo zenith]

Or one with separate user accounts. It probably makes more sense to have a docker image that everyone runs locally but again it all depends on what you are trying to do

[maiden shadow]

I want to use ec2 so that I can take advantage of powerful ec2 instances

[indigo zenith]

I see. Well whether you can share one or each need a sperate one just depends on how (and when) you need to use them

[maiden shadow]

I was looking into code server

I think I have to use it with EFS. This way we can use different ec2 easily by just attaching the efs to that ec2. Not sure if that makes sense.

[deep crow]

hey i have a generel question, first of all im just a few days old learning python and i dont understand why "return" is usefull if u just can use print if there is an example where "return" is usefull please show me one cause i dont understand where "return" is more efficient than print or for what "return" is even efficient for

[indigo zenith]

Return passes a value from inside a function to the outside. Print displays a value. Totally different things. If you start using functions that do anything other than just print, it will make sense

[analog portal]

i'm using opencv and i need help with 2 things:

1. it takes about 20 seconds for my code to open my webcam when i open it
2. i want to be able to choose which camera on my computer i want to open (i only know how to open the default camera. i have windows 11)

cap = cv2.VideoCapture(0)
cap.set(3, 1366)
cap.set(4, 768)
cap.set(10, 100)
    while True:
        success, img = cap.read()
        cv2.imshow("camera", img)
        if cv2.waitKey(1) & 0xFF ==ord('q'):
            break

[sly sleet]

what happens if you want to do something later with the value returned from a function

can't do that with print

[neon jungle]

I recently switched to flake8 as a linter instead of pylint, but now whenever I get an error it just highlights the first character of the line and not the whole line

its not that bad but it makes it kind of hard to see where errors are, do you guys know of any solutions?

[stuck beacon]

Hello. I'm trying to compile Python3.8.9 from source on Windows 10 but getting compiling errors. I've tried a couple of version of python so far and got the same error below:

fatal error RC1116: RC terminating after preprocessor errors

Can anyone here help?
There is an issue for this https://bugs.python.org/issue45220

[patent edge]

is this vscode?

[neon jungle]

yes

[gloomy quail]

maybe you could try ```py
import tensorflow.keras.backend as K

I don't know if it will work. Just a suggestion

[neon jungle]

I’m not trying to fix the error, the error is just that K is unused, I’m trying to make it so that vscode underlines the whole line and not just the first character

[indigo zenith]

This thread is about PyLint but it may be relevant: https://github.com/PyCQA/pylint/issues/3061

[sand spoke]

Best recommendation for a system cleaner for PC windows 10 ?

[indigo zenith]

I avoid them in general as they seem to often do more harm then good and you need to be really careful. But if I really need one I go for BleachBit https://www.bleachbit.org/download/windows

The most popular seems to be CCleaner but I trust BleachBit more as it's is open source

[heavy knot]

I am having trouble importing a class within a project using VS code. Can someone help me out please

[indigo zenith]

What are the paths of the relevant files, how are you doing the import exactly and what is the error message/issue you are having?

[twin valley]

Anyone having weird issues with Python3.10, Poetry, and PyCharm? The Python3.10 virtualenv managed by poetry “seems to be broken” every time I try to poetry install

I guess PyCharm is irrelevant there

Nvm just needed to sudo apt-get install python3.10-venv

[reef latch]

anybody use kafka stream jobs in a python app setting?

[mortal zodiac]

👋🏾 Hi all, I have a few Git questions if anyone has a minute?

I'm working off a branch from main

But I need to create a new branch now and have changes on my previous branch that I need on this one

How would I go about doing that?

[hot bison]

if you create a new branch, it will include your local changeset

[pliant meadow]

SpiderDevelopers, I desperately need help. So I have spyder 5.1.5 installed, and I'm developing a UI based on PyQt5. According to the requirements for spyder 5.1.5, it can only support PyQt5<5.13. However, I have installed in my environment PyQt5 5.15.6 and spyder still runs, but if I run print(PYQT_VERSION_STR) within some code inside spyder, it spits out 5.12.3. But if I run pip list inside a console within spyder, I still see that my PyQt5 version os 5.15.6. What is happening??

[pliant meadow]

Ok, I found that in my anaconda navigator I have this pyqt package which seems to be the one spyder is using, but I have no idea what package that is

and why it's not using pyqt5 from python

[ionic quiver]

Hey, do you guys add a license for your private projects on github?

[tawdry needle]

it sounds like you have multiple versions of python installed, and/or possibly multiple conda envs

[pliant meadow]

That wasn't it, it was something weirder, but somehow I think it may be working now

That dumb module "pyqt" is installed by anaconda when installing spyder, and I believe it encompasses pyqt5 5.12.3. On the other hand, I had used pip to install the latest version of pyqt5 (5.15.6) on the same venv. So it looks as though I had two versions of pyqt5 within the same env, but one was packed inside "pyqt" and the other in "pyqt5". I then uninstalled the pkg "pyqt5" through pip, and it worked (?)

it was a complete headache. Plus, I was working with Win 7 which made things worse, since I could use the latest anaconda installer, updating conda was taking forever, updating packages was also a nightmare.... I just hope it'll work from now on

[tawdry needle]

good catch. you sometimes have to check on pypi to see what things are, e.g. reading their descriptions

other times packages are aliases for other packages

!pypi pyqt5

[rancid schooner]

PyQt5 v5.15.6

Python bindings for the Qt cross platform application toolkit

[tawdry needle]

it looks like there isn't even a pyqt package on pypi... you might have also been mixing up conda packages and pypi pacakges

[tawdry needle]

i always suggest to never use anaconda and just stick to conda (i.e. the miniconda installer)

the anaconda gui doesn't really add much convenience, and they jam so much garbage into the base environment that it makes a big mess

[indigo zenith]

Private projects that nobody else can see? No.

Personal projects that I make public? Yes.

[tawdry needle]

doesn't github just detect any LICENSE.txt or LICENSE or COPYING etc file in the repo?

[sand thistle]

i am getting a 504 error, but all my infrastructure seems to be up

i have checked my service providers as well and nothing seems to be down

[tawdry needle]

what does your infrastructure consist of?

[sand thistle]

nginx, docker, mysql are all up

[tawdry needle]

504 is gateway timeout: this suggests that maybe your application is taking too long to respond, and something in between the client and your application is giving up

is this all running on a single server? or in some kind of cloud services setup?

i don't know what you mean by that

[sand thistle]

sorry uhh

its one big thing on one server

[tawdry needle]

my first guess is that your application (in docker?) is failing to connect to mysql, and the python application is itself timing out trying to connect to mysql

i have similar errors when i misconfigure my docker containers at work, they time out trying to connect to the database (not mysql in my case)

[sand thistle]

you want me to log in to mysql

[tawdry needle]

well... yes? and no? your application obviously has to provide credentials for the database, or at least connect to the database at the right url

you have it set up like this?

outside world  -->  nginx  -->  docker container  -->  mysql

are docker containers even allowed to access network addresses outside of the container?

[sand thistle]

uhh this configuration has been up for a long time

about a month ago i changed some dns settings

for a sub domain leading to this container

anyway,i havent changed anything

[sand thistle]

alright i mean i was able to log in to mysql

[tawdry needle]

at minimum, start by checking the nginx logs

it should tell you if the 504 is being returned from nginx, or if it's being returned by the python app itself

[sand thistle]

[error] 27648#27648: *2177993 upstream timed out (110: Connection timed out) while reading response header from upstrea

@tawdry needle

[tawdry needle]

ok, and are you sure that nginx is pointing to the right upstream?

show your nginx config

!paste

[rancid schooner]

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[sand thistle]

if it was misconfigured wouldn't it have had an error if i made DNS changes 2-4 weeks ago

i dont feel ok sharing the nginx config

[tawdry needle]

hard to say, need to see the config. at minimum, can you share the part of the config that points to the python application?

are you using proxy_pass? something else?

[sand thistle]

i dont see proxy pass

i changed the A record for the main domain, but its the subdomain thats having connection issues

[sand thistle]

the issue is i dont see it pointed

[tawdry needle]

so how is nginx "connected" to the python application?

[tawdry needle]

maybe you misconfigured the server_name directive in nginx

e.g. it's failing to match the correct server block because you set the server_name directive wrong, and it's falling back to the default_server server

i recommend reading this doc if you don't know what i mean https://nginx.org/en/docs/http/request_processing.html

it's a bit hard to explain

[sand thistle]

i see

[scarlet olive]

Hey Pythonistas. I work with a pretty active small dev team as a front end dev. Dev teams will publish their environment files with updates to a .example file but not mention it or maybe mention it in a group chat somewhere. Well, if I go a few days or longer without upgrading one of the services on my remote dev box then I have to go and find and rectify the environment variables. Sometimes this is a pain so I wrote a little script to check them, compare them and offer to merge them with some user input. Works like a charm.

I am trying to publish it to pypi and since last time I did so I see they recommend now a .cfg file. I want the command for the package to run the script but but console_scripts doesn’t seem to work

(Like user doesn’t have to run python3 -m envcheck, rather just envcheck)

I took a look at how black package does this and the only different I see is that they’re still using setup.py

Has anyone published a package with a setup.cfg containing a console script to invoke commands directly?

[scarlet olive]

For anyone interested…. It does work with setup.py just wondering why not with .cfg 🤷🏻‍♂️

[tawdry needle]

maybe you just had the syntax wrong in the .cfg version?

[sand thistle]

it was my app, not the devops stuff

[gray shell]

I love copilot

[civic grove]

response_model=UserCreate , that will give the password in the response as well lol

[maiden shadow]

I created a pylint plugin. However, I only want the plugin to run on specific modules. How do I specify that?

[grand vector]

is there a good automatic code mapper for python? I need to see a flowchart of my code and i reeeeaaaallllly dont wanna manually craft it

[open robin]

I recently made performance improvements to a library I maintain. Now I'm looking for a tool that can easily generate nice graphs showing the performance improvements. And by easily, I mean all you have to do is specify that you want to compare mylib_v1.foo(input) against mylib_v2.foo(input) (and specify some increasing input sizes), and the tool outputs a nice png or something that you can conveniently link to from your release notes. But I'm having trouble finding something like this, which is surprising since it seems like a pretty common use case. Any suggestions?

[tawdry needle]

you can use matplotlib to generate graphs in general, or seaborn for a higher-level plotting interface

[blissful ice]

Hi! I'm trying to adopt some package-requirements scheme for my projects. So maybe a simple requirements.txt or perhaps poetry? Bue either way, I'm unclear about how to specify "variant" dependencies like OpenCV or CuPy. Like, I don't care if you (as a user) install opencv-python or opencv-python-headless and I don't want to force you into one if you prefer the other. And with CuPy it's crucial not to take the decision away because I (the package) can't decide for you if you should be installing cupy-cuda113 or cupy-cuda114.
So how does one handle this? Only specify packages where a singular decision exists and inform users that the requirements are incomplete and require manual decisions?

[torn bane]

hi, I'm used to using pyenv virtualenv in linux and in WSL, is it usable in Windows? I installed pyenv-win and virtualenv using the direction in this one writeup I found on google (RKadeZone), but it looks like it's still using the local installed python and not the version I installed with pyenv

seeing as how I installed 3.9.6 and python --version says 3.9.5 which is what I had installed on windows

[torn bane]

Probably have to figure out some other tool for making virtual environments in windows

[rapid sparrow]

i can say for sure that virtual venv (also known as python3-venv) is usable in windows, not knowing about pyenv

[full bronze]

What's wrong with using the default venv implementation? Or rather, why would you use another package for a venv?

[sly sleet]

venv, being a stdlib module, is missing features

https://virtualenv.pypa.io/en/latest/

[torn bane]

my main problem with venv originally was that I didn't understand how it works or how to use it, where as learning how to use pyenv virtualenv was quite straightforward

now that I've used python for 4-5 years, I still don't understand how venv works or how to use it

[sly sleet]

it works exactly the same way

just with slightly different syntax

[scarlet olive]

Here’s a package I wrote (was asking about a little above). Working on some a more structured version for a real release but this is working and if anyone has suggestions I’d be happy to hear them https://pypi.org/project/envcheck/

[uneven saddle]

Need help AWS .

Folder of different users in ONE bucket only and each user can view access just there folder .

Need json code policy for this.

[tawdry needle]

You could use a "constraints" file and list all of those variants in the constraints file, at least. I'm not sure about specifying alternatives to satisfy an abstract requirement... another option is to use different requirements files? it kind of depends on why an end-user might choose a different variant

[mild swift]

I'm trying to migrate a dockerfile from 3.9 to 3.10 - but I find that apt-get install python3.10-dev fails in the official 3.10 base images I've tried - is there a way to acheive this?

[smoky stratus]

Enabling ufw makes my GitHub actions runners disconnect/timeout, everything else works as expected since all outbound is allowed. What could be the issue?

[rancid schooner]

:incoming_envelope: :ok_hand: applied mute to @wintry hearth until <t:1644155347:f> (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

[heavy knot]

which one

[burnt ibex]

The current one is fine

[heavy knot]

ok

use bundles openssh or external openssh

?

[heavy knot]

?

[burnt ibex]

bundled is fine

[heavy knot]

?

[burnt ibex]

either is fine

[heavy knot]

@burnt ibex

[burnt ibex]

that is fine

[heavy knot]

use MinTTY or windows cmd?

[burnt ibex]

cmd is fine

[heavy knot]

do i need a credential helper

git credential manager or none

[burnt ibex]

git credential manager

[heavy knot]

whic

both or?

[burnt ibex]

both

[burnt ibex]

just click install

[heavy knot]

done

okay git worked

thanks

@burnt ibex do i need cmake added to the path for dlib to install

[burnt ibex]

I have no idea

[wet dune]

hi there

I am making my first "lib" its a nasa api wrapper

I just dont really know how to upload the package

If someone can help

Thats great :)

[heavy knot]

Hey, does anybody know how to run bash commands in python?

[rapid sparrow]

import os
status_code = os.system("your command")

or you can use subprocess library for a more advanced capabilities

to capture output

or even capturing output to stdout and python code at the same time in real time

[heavy knot]

so it would be like

`import bash
import time

try:

while True:
    bashCommand = "sudo ./anti-ddos.sh"
    print("ddos check..")
    time.sleep(5)
    print("all possible ddos attempts eliminated")
    time.sleep(5)
    print("you can disable ddos check with ctrl+c")
    time.sleep(20)`

would it be like thaT?

[sly sleet]

os.system considered harmful

subprocess is objectively superior in every way

and less chance for vulnerabilities

[tawdry needle]

import bash?

[heavy knot]

yes. you can install it wit pip3 install bash.

[tawdry needle]

https://pypi.org/project/bash/ this? i don't see much benefit of using this

nor do i understand the point of it. i think you should stick with subprocess.run for all basic use cases

!d subprocess.run

[rancid schooner]

subprocess.run

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, ...)```
Run the command described by *args*. Wait for command to complete, then return a [`CompletedProcess`](https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess "subprocess.CompletedProcess") instance.

The arguments shown above are merely the most common ones, described below in [Frequently Used Arguments](https://docs.python.org/3/library/subprocess.html#frequently-used-arguments) (hence the use of keyword-only notation in the abbreviated signature). The full function signature is largely the same as that of the [`Popen`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen "subprocess.Popen") constructor - most of the arguments to this function are passed through to that interface. (*timeout*, *input*, *check*, and *capture\_output* are not.)

[tawdry needle]

i wonder if sudo will even work in a subprocess

that would be the only sticky thing

you might have to put your password in at each loop, which wouldn't be good

[heavy knot]

I am trying to make a python script that runs a bash command as a loop.

[tawdry needle]

well this isn't a bash command

sudo is a standalone program on your computer

[heavy knot]

like once every 30 seconds

[tawdry needle]

which runs another program as a given user

[heavy knot]

ooooh.

[tawdry needle]

literally sudo means switch user do, i.e. "do something as another user"

its name is based on the su command, which just switches the user and starts a new shell process

[ionic quiver]

anyone has experience packaging python app and automatically adding it to releases with github actions?

[indigo zenith]

Don't ask to ask, just ask

[heavy knot]

I thought it "super user do".

[indigo zenith]

Makes sense but su works with any user, not just root

[ionic quiver]

was wondering how would i go about building my app on push and put the built file to releases

[tawdry needle]

you are looking for a "CI/CD" tool

if you use github for your code, you can probably use github actions for this

[ionic quiver]

yeah im using github actions

i can build but how would i publish it in releases afterwards?

[tawdry needle]

that i'm not sure. maybe you have to use the github api

[mystic void]

depends on what you mean by "releases". That could be as simple as a tag in your repo, to multi-stage complex processes

(ex: pushing some generated artefacts such as docker images, tarballs, or to pypi or updating some revision numbers automatically and announcing the release somewhere like slack/discord/email)

[tawdry needle]

i assumed they were talking about "releases" on github specifically

[serene mango]

Hi can I ask for docker help here (because it's not a python issue)

I'm getting a lot of dependency errors while trying to build a docker image of my app.

Stuff like "Failed to build multidict yarl" and a lot such errors (like for example unable to pip install git+ for which i have to include git too)

Is it possible to just use the existing python environment and then not to worry about pip installing at all? If yes, how do i do it

[ionic quiver]

why not install dependencies on your container

@serene mango

[serene mango]

Hold on, i'll send all the errors one by one haha

1. I also need git to install packages directly from github (which i managed to solve so that isnt an issue)
   (i'll get the other errors)

FROM python:3.9
RUN apt-get update && apt-get install -y git

COPY ./requirements.txt .

RUN pip install -r requirements.txt

COPY ./src ./src
COPY .env .env 
CMD ["python", "src/main.py"]

Here's my Dockerfile

I'm experiencing this "glitch" where it installs so many times

it gets ALL releases of discord.py lmfao

[ionic quiver]

what does your requirements.txt look like

you did pip freeze right

[serene mango]

Yes, actually I managed to fix that issue somehow, and here's the latest error:

OSError: [Errno 99] error while attempting to bind on address ('::1', 20000, 0, 0): cannot assign requested address
Exception ignored in: <bound method BaseEventLoop.call_exception_handler of <_UnixSelectorEventLoop running=False closed=False debug=False>>```

docker run -p 20000:20000 container_name

[ionic quiver]

not sure

[serene mango]

docker run --expose 20000 container_name

I've also tried this, same error

[tawdry needle]

this is pip attempting to resolve a version conflict, so it's fetching multiple versions

[serene mango]

oh

How I fixed it was, I install then delete then install again in the same process

i don't know why it worked 😄

[tawdry needle]

at least... i think so? normally it has more output about "finding a compatible version"

[serene mango]

it didnt print that this time

[south dawn]

what do u guys use to code

[edgy lake]

Can someone help me write this python code, Im tryna grab duplicate images from one folder then sort them in next folder. I have no python experience but the code is here $$$

https://github.com/philipbl/duplicate-images#:~:text=This Python script finds duplicate images using a,been rotated%2C have changed metadata%2C and slightly edited.

[flat summit]

https://github.com/Disk-MTH/Tradlib

[indigo zenith]

That's usually a version comparability issue. 1) Make sure the version of pip inside your container is the same as the one you used to make your requirements.txt. 2) Try allowing an older version of the problem package

Looks like you sorted it out, but for future reference

[heavy knot]

Can anyone make good OpenCV python object detection script??

[snow zinc]

!rule 5

[rancid schooner]

Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

[heavy knot]

Thats part of python @snow zinc OpenCv

[snow zinc]

I'm not sure if we log edits, but I'm pretty sure that their original message, before editing it two minutes after I sent my message, was directly asking for help with cheat programs
and OpenCV is not really on topic for this channel either tbh

[barren hamlet]

Hmm

[heavy knot]

Hello just a quick question related to gitflow. Does develop branch off master? Or is it an independent branch?

https://nvie.com/posts/a-successful-git-branching-model/

[heavy knot]

According the diagram, develop branches off master

[solemn marlin]

In the first version of your App the develop would bee a copy of the master branch. In next versions you would implement all the new features to the develop branch and when you are ready to release just merge the dev branch to the master one.

@heavy knot

[heavy knot]

Roger that, thank you so much

[wet dune]

Im not sure if this goes here lmao. Im making a NASA API wrapper, and i am currently working on the output and error handling. Any suggestions i could add on the output to be more like, easy to read or helpful to the user?

[wet dune]

Example of APOD(Astronomy Picture of the day) request

[rancid iron]

Is this the channel to ask questions about python lining tools (flake8, pylint,...)?

[flat path]

Mhmm, you can certainly do that here although type checking ones should go to #type-hinting

[solemn marlin]

It looks nice to me

Maybe insert a break line after the “Response MSG” but idk if that would be really nice

[rancid iron]

I just dialed down the severity of flake8 warnings in vscode settings. All G

[idle depot]

Hi, Im looking for a good tool to monitor all my computer stack (usually use python/nodejs/react) or alternatively monitor docker containers.

[rapid sparrow]

it is common to use ELK or PLG

i am an adept of PLG

PLG = Prometheus + Loki + AlertManager + Grafana

ELK = Elastic Logstash Kibana

[idle depot]

are they all free? (forgot to mentions that i am liooking for something free for personal use)

[rapid sparrow]

yes

at least they are open source and have free versions

Grafana has some sort of paid version, not sure about details

others should be having paid versions too, but it is not necessary to use them I think

I use perfectly for free for now

[idle depot]

Great thanks for the help

[rapid sparrow]

u a welcome
Just to be sure
Prometheus is monitoring tool, tells it is alive, CPU load, memory load and e.t.c. Metrics checking tool basically, to gather from all servers in one database, it has inbuilt visializer
Loki = Log aggregator, collects logs from all your applications, and gives simple query language to search through them
Alert Manager = can raise alarms based on conditions and send you notifications about it to Email/Discord/Slack/whatever
Grafana is just a visualizer for more comfortable usage of all previous things, quite useful for Loki especially, since it is not having inbuilt visualizer

[idle depot]

yes this is exactly what i am looking for. thanks.
I tend to code fast and sometimes overlook process/memory leaks etc. I just think it will help me monitor the stack while working.
Is there any revommended "All in One" docker container installation you know of? Or i should just install all separably

[rapid sparrow]

There is, if you are in Kubernetes/Helm ecosystem for example

https://grafana.com/docs/loki/latest/installation/helm/

it looks like they are offering some parts of the full package in other types of installations

docker compose version has Lok + Promtail + Grafana, not bad for start

[idle depot]

I use only docker locally... so..

[rapid sparrow]

try docker-compose version then

it is without prometheus, but pretty much looking cool

[idle depot]

tnx

[shadow crow]

Do you think it's okay to split a change between several commits, if in some of these intermediate commits the tests get broken?
Or should the system be in a working state on every commit?

[solemn marlin]

O think its fine

Specially because you could make chances related to other features in order to achieve your end goal

[shadow crow]

but then it's harder to revert to a specific commit

and if you're browsing code on a particular commit, it might be really confusing how it works (because it doesn't - it might not even compile)

[solemn marlin]

Pros and cons

Maybe creating a commit name pattern to specify that you are in the middle of an new feature should work

And split your feautures into new branchs

[shadow crow]

I wish there was a way to group commits in a tree somehow. Like, compose a commit out of sub-commits, without creating a new branch

[solemn marlin]

I Mean is creating an new branch then merging that bad of a decision?

[shadow crow]

I guess you can squash-merge

but then it's tricky to view the individual commits, you need to keep that branch alive and switch to it

[wet dune]

Oh yeah good idea! Thx a lot :) I really appreciate that bro. I will fix it right now

[silent sedge]

Are there any static opentelemetry renderer? A collector that will generate static reports?
So for example, I can use it in CI, it will collect spans into static files I can expose as artifacts for the CI job?

[silent sedge]

looks like opentelemetry-collector can output a JSON file. Did anyone try that on CI?

[narrow slate]

In an aws context, where would you put internal tooling like code review, ci, cd to host it in the safest way ?
The best approach I have in mind is a separated account which would make use SSM to reach other accounts / subnets.

[solemn marlin]

Can you elaborate more your question ?

[narrow slate]

I don't think i can make it more elaborate :/

you have to deploy staging and production envs within aws, let's say both env have their own aws accounts, where would you put versioning/ci/cd tooling and how would you achieve the provisioning, let's say on k8s clusters

I have my own answer, i'd just like to be challenged

[silent sedge]

Another VPC at least

if you're using root accounts for anything, another root account

[narrow slate]

if it's in another account, there is of course another vpc yes

the question is what would you use to reach this k8s cluster and provision it ?

would yu just peer vpcs ?

[tawdry needle]

Leave comments about what is broken (including in the commit message), and don't push to a main/trunk branch until it works

[shadow crow]

That's what I usually do. I sometimes add TODOs in the ling description of the commit

[frozen zephyr]

I've got a coworker who insists on having at least two commits per PR, one that introduces a new failing test, and one that makes the test pass

[carmine venture]

Hi everyone. I wrote a package that lets me keep a single-source-of-truth for all of the .env files in my codebase. I'd like to pass it along in case it can help any of you as well. Cheers!
https://github.com/MikeChurvis/compile-env-files

[rapid sparrow]

It sounds as an interesting idea 🤔

TDD at git level

[rapid sparrow]

It does not make sense to have system working at every commit in branches

Because we use CI Cd tools that run at every commit to check integrations with other services
Which are meant for us to save our time from testing

It could be making sense to have every commit working in master branch

U could be squashing (uniting all commits into one) before merging branches into master

[frozen zephyr]

We also don't do fast forward merges when merging a PR that has more than one commit, so the trade-off is that you can no longer revert normal individual commits, only merge commits

That said, we rarely revert anything anyway. It's usually easier to roll forward and fix the fix than to throw it away entirely

[civic grove]

do jobs not retain "side-effects" when using needs in github actions?

name: C/C++ CI

on:
  push:
    branches: [ main ]


jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Build
      run: |
        make all

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run tests
        run: |
          chmod +x ./runtests.sh
          ./runtests.sh

my build job basically just makes a bunch of .o files inside ./bin/test/ and test job needs those files to run tests, but i get this

./runtests.sh: line 14: ./bin/test/test_*: No such file or directory

obvoiusly i can just put them in the same job but i was wondering if there is something i can do so that build and test are different jobs

i'm assuming this is due to the 2 jobs having different runners

[chilly oriole]

I don’t know if this is the proper spot for a question, but I got emails from apple, not sure it’s really apple about developers license and certifications but my laptop puts up massive warnings about downloading anything from the page. And the copyright date at the bottom is still 2018. Real or fake? Please and if this is the wrong spot for this can someone help me find the right one. Please and thank you.

[mystic void]

Your computer warning you is a massive flag.
You can also check the usual spots:

• Look at the email addresses, if they have some typo or weird domain
• If the message has a threatening town
• If they contact you out of the blue without you taking any action
• Paste some of the title/sentences of that email in google to see if it is a known scam

[jade birch]

Hey there all! I'm looking for a Unix-based command line of some sort for Windows systems that can run commands installed on the Windows system itself. I have previously used sub-system Unix, which has its own package manager but am trying to move away from that more to something that is ideally just aliasing or otherwise using the Windows libraries itself rather than running a subverted operating system. Currently I am using Git Bash but it seems to have a number of issues, particularly with Python commands, and I am now looking for an alternative as I'm tired of dealing with random crashes if I do not provide arguments to commands that otherwise would normally function fine. (Example: If I run py it hangs indefinitely rather than dropping into the Python console as normally would occur; I cannot find any online forums addressing this bug, though if you know of it/have seen one please let me know. I may not be searching the right thing.)

Ideally whatever tool would be able to handle:

-javac and java (my university uses these languages for teaching)
-git
-python
-ls, cd, grep, and other basic unix utilities/commands
-ideally 'vim' and 'code' commands```

Obviously, even if you know of a tool that fulfills only some of these conditions I'm trying to gain as much knowledge as possible of the tools available so I can select what's right for me so please do share; thank you!

[civic grove]

wsl2 is your best choice

[tawdry needle]

Cygwin or Msys2

Unlike WSL, those systems actually run as native windows applications

however i am curious what the problems are

git bash i believe is like a stripped down msys environment

[jade birch]

I’m not really sure; didn’t use to be a problem, but just has had issues constantly that a reinstall has not resolved

[jade birch]

Will look into it too

Thank you both

[topaz aspen]

PipEnv - a fresh approach that is going to become the “official” Python way of locking dependencies some day

From : https://pip-compile-multi.readthedocs.io/en/latest/why.html , is this true?

What are people typically using for project management at the moment? I've found poetry problematic at times, pip-tools seems reasonably sound tho missing out on some of the polish that poetry has, and I've not tried pipenv.

[idle ridge]

Hey, I've got a GUI python dekstop app that I'm trying to deploy (with all its dependencies). Pyinstaller takes care of all the regular dependencies that are used in the code, but the app also depends on an external command line python program. The app runs commands for that program. If a user doesn't have this cmd line program installed then obviously the app won't function. What's the best way of packaging this cmd line prog with a version of python so that the user doesn't have to install it? I have a couple ideas on how to do it but I wasn't sure.

[tawdry needle]

If you can explain the problem then maybe someone can help!

[tawdry needle]

pipenv is hopefully not becoming official, although the python packaging tutorial i think uses it

pip-tools and setuptools is still the way for me

[topaz aspen]

ive gone back to this from poetry

well i use pip compile multi with pip-tools

[tawdry needle]

pipenv isn't bad, but it explicitly does not support "packaging" in the sense that it cannot build wheels and maybe even is not a pep 517 build backend

[topaz aspen]

i've never actually build anything without using poetry 🤔

would be interested in seeing a poetry -> pip-tools conversion for that i guess, is it just having a setup.py ?

[wicked edge]

Hey, I've just read through the blog post https://dev.to/frostming/a-review-pipenv-vs-poetry-vs-pdm-39b4, and I'd like to hear the other side of the story, if there is one. Is PDM definitely better than poetry, or does poetry have its advantages also?

[heavy knot]

Hi all, I'm here and i need some head ups.

Really wanna learn python programming to become a pro, but to skip that part first, i need a python programmer that can work on mailgun mailing sdk and a developer to help with developing an SMTP bulk email sending scrip, hit up my i box if you can refer to anyone available I'll be pleased to deal 🤝

[olive falcon]

hi, how can I use the keyboard module to print out the keystroke that the user pressed

[jade birch]

Here's the main one:

I am now hanging indefinitely.

Ctrl + C is not accepted.

I have to restart the terminal.

It does not matter what directory I am in when I run this.

This is the behavior even if I reinstall both the terminal and Python with nothing else involved.

[gentle cave]

Lucknow Super Giants: KL Rahul, Ravi Bishnoi, Marcus Stoinis, Deepak Hooda, Jason Holder, Manish Pandey, Quinton de Kock, Krunal Pandya, Mark Wood, Avesh Khan, Ankit Singh Rajpoot, Krishnappa Gowtham, Dushmanta Chameera, Shahbaz Nadeem, Manan Vohra, Mohsin Khan, Ayush Badoni, Kyle Mayers, Karan Sharma, Evin Lewis, Mayank Yadav

this time we have lucknow too in ipl

yeeaahhhhhhhh!!!!!

[tawdry needle]

wdym by reinstall? linux?

but yes that is not normal

[opaque gorge]

Is there a better way to do this in github workflows? This is just so annoying: ```yaml
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.7", "3.8", "3.9", "3.10"]
include:
# Specify paths here, so we know what to cache
- platform: windows-latest
POETRY_CACHE_DIR_SUF: .cache\py-user-base
PYTHONUSERBASE_SUF: .cache\py-user-base
TOXDIR_SUF: .tox
- platform: ubuntu-latest
POETRY_CACHE_DIR_SUF: /.cache/py-user-base
PYTHONUSERBASE_SUF: /.cache/py-user-base
TOXDIR_SUF: /.tox
- platform: macos-latest
POETRY_CACHE_DIR_SUF: /.cache/py-user-base
PYTHONUSERBASE_SUF: /.cache/py-user-base
TOXDIR_SUF: /.tox
steps:
# I skipped some basic steps here

  - name: Test with tox
    run: python -m tox
    env:
      PIP_USER: 0  # We want tox to use it's environments, not user installs
      PYTHONUSERBASE: ${{ github.workspace }}${{ matrix.PYTHONUSERBASE_SUF }}
      POETRY_CACHE_DIR: ${{ github.workspace }}${{ matrix.POETRY_CACHE_DIR_SUF }}

It's really annoying that windows is using `\` for paths instead of `/`, and it forced me to just set suffix matrix variables which I then append to `github.workspace` to construct an os-dependant environment variable, and this needs to happen in each step in the workflow (maybe not with all 3 variables each time, but still)  (link to the entire workflow: https://github.com/Dinnerbone/mcstatus/blob/f6b6d4050fc0d8441eec810617968a124d366b26/.github/workflows/tox-test.yml)

[finite fulcrum]

doesn't / work? Windows should accept it everywhere for file paths

[opaque gorge]

huh, it does?

let me test that

that's very interesting, it indeed does seem to be fine with /

thanks

[jade birch]

I meant Python. I’m incompetent.

I have to reinstall Linux on a machine later today; loaned someone my laptop and they uninstalled a package they shouldn’t have that ripped like over a gig worth of packages off the OS

And I do not want to spend time rebuilding them

So probably got mixed up while thinking

But basically reinstalling Python and Git Bash didn’t help

[rapid sparrow]

U probably deinstall python? ;)
Python is dependency to a lot of stuff in ubuntus

Anyway, better reinstalling OS in such cases

[jade birch]

No someone else did something stupid

That wasn’t what my question was about though

lol

[rare mirage]

Hello!

Quesiton:

Look at setup.cfg at https://github.com/pre-commit/pre-commit-hooks

Does anybody know where does that file come from?

What tool is it from?

I'd really appreciate you to tell me if you know!

[visual oxide]

setuptools parses that

Pep 621 is the new way of doing that

[rare mirage]

Awesome! Thank you very much!!!

So, an old way of adding plenty of metadata to your package, was to write a setup.cfg? And now, the new way, is to create a pyproject.toml instead?

[rapid sparrow]

Is there python packaging manager that allows easily dumb package publishing to access dependencies.
Something like sharing in online git is enough

https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/
https://gist.github.com/javrasya/e95ade856ff42e4649972f8a54368459

Oh it is possible with pip

Curious

[tight coral]

thanks

so i have a repo initialized on my vs code and in github

and i made changes on another computer and committed them

[shy yoke]

👋

[tight coral]

how do i get those changes from the repo onto my original computer? do i clone the repo again?

[shy yoke]

ahh ok was about to say to commit them first

[civic pilot]

if you already committed the changes, i think yeah just go ahead and clone it again

but

[tight coral]

oh ok

[shy yoke]

^

[civic pilot]

i dont really work with vsc & github usually-

so take everything with a grain of salt lol

[tight coral]

do you use cli or something noramlly?

because im fine doing it with that as well

[shy yoke]

you could prolly just transfer the folder to the new pc or something

copy* it if you want it on both

[civic pilot]

yeah

honestly though a git clone would be the simplest answer lmao

[shy yoke]

you can alos create ssh keys with git

[tight coral]

so ive got my folders with like yesterdays stuff in it and i want to update the files to todays stuff that i edited on my other ccomputer, i should just clone it?

[civic pilot]

yeh i'd suggest move the files on the current pc somewhere else just in case something goes wack with the clone, and then clone the repo from your updated branch or whatever

[flat path]

if you already cloned the remote repository you should be able to do git pull as long there's no unsynced changes locally

not sure if this is less complicated than just moving files with an USB stick, but it is an option

[civic pilot]

ooh

that's cool lol

[flat path]

cloning a remote repository also sets up a remote (you can list it via git remote -v) for git push and pull operations

[tight coral]

ok ty :) @flat path

even if i have files that arent the same as the current online repo, pull says all up to date, why? @flat path

[flat path]

hmm, did you push to the same branch on that other computer?

[tight coral]

yes, there is only 1 branch

[flat path]

Do you mean the files have changes locally that don't exist online or there's changes online that weren't copied down?

[tight coral]

changes online that arent copied down when i do "git pull"

if i look at git log it shows (HEAD -> main, origin/main) on a commit what is this? does it have somethign to do with branches?

[flat path]

so HEAD is the checked out ref in which case is a branch

the origin/main bit indicates the online version of main has the same commit locally

so either the local copy of the remote is outdated (those are those remote-tracking branches are) or something else is going awry here

[tight coral]

git pull https://github.com/REPOURL main

this is a valid command?

to get main branch

[flat path]

I think that should work but I don't think it does anything different unless the remotes are configured weirdly.

[tight coral]

ok thanks for the help :)

[flat path]

another option is do git reset --hard origin/main which forces the local index to be identical to the other index (which happens to be on the origin remote) but that's well, a bit dangerous since it will overwrite non-synced local changes.

[tight coral]

ok cool

one time i did that and i hadn't committed for a couple hours 💀

was not fun

[flat path]

yeah I have done that before, it's why I stash or make temp commits now so even if I do an accidental reset I have the reflog or stash to save me

[tight coral]

oh cool

[flat path]

I should probably make frequent disk backups too but I'm lazy

[tight coral]

ok

lol

yeah thanks for help with that

[flat path]

no worries, it's in times like these I question whether I really know how to use git anywya 😄

[frail lagoon]

Hi everyone, I have a question on how to develop a Package at the same time as a program using it. Should I :
A . develop the package in a repository appart from the main application essentially doing commits and push AND THEN testing it in the main application
B. develop the package in a repository within the main application at the same time. (having to mess with local imports etc ?)

Happy to clarify , sorry english isn't my first langage 😢

[somber lion]

The first is probably much better.

Note that you can pass a directory path to pip install, to get it to install from disk if you want to test with in-progress versions.

[frail lagoon]

Thank you very much, make sense

[tawdry needle]

You can also use pip install --editable to avoid having to re-run pip install after every change

However before releasing a package you should test in a clean env without --editable just to make sure it all works

[tender rose]

does anyone know how to merge the commit of the original repo to the fork?
ie: the main repo can be so many commits ahead, how do i commit those changes into my fork?

[desert blaze]

You can click Merge upstream button on github idk the command for it though

[tawdry needle]

clone your fork, set the original as a remote (e.g. upstream), merge the remote with your local clone of the fork (git pull upstream), push your fork back to github

[frail lagoon]

got it thank you

[mellow mirage]

what should i use to paste my code?

[heavy knot]

hey, im trying to run selenium in docker, but its getting stuck at the creating driver and just stays there without any error code. is there a way how i could debug this to find the issue?

[heavy knot]

made any progress?

[heavy knot]

nope used heroku for it in meanwhile

[tawdry needle]

https://paste.pythondiscord.com @mellow mirage

sorry if you got duplicate notifications

[heavy knot]

Oops, wrong channel. Will delete shortly

[rancid schooner]

:incoming_envelope: :ok_hand: applied mute to @astral field until <t:1645010438:f> (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

[finite fulcrum]

Is there any way to get poetry env use to work on windows without passing the full path to the executable? I tried creating symlinks to the version executables that follow what poetry uses but that opens an another terminal when ran for some reason and messes things up

[gentle hill]

How can I install pip for Python 3.10 on Ubuntu? I already had 3.8 installed, where python3 --version returns 3.8.10, and then I followed this guide to install 3.10 from the deadsnakes repository: https://computingforgeeks.com/how-to-install-python-on-ubuntu-linux-system/.

Now if I run pip --version I get this back: pip 20.2.3 from /home/aleister/.local/lib/python3.8/site-packages/pip (python 3.8). There is no python3.10 under ~/.local/lib, I only see 3.10 under /usr/bin.

How do I get the command pip in my terminal to run the pip for Python 3.10?

[finite fulcrum]

try pip3.10

[gentle hill]

"command not found"

If I try curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 I get a traceback and a "No module named 'distutils.cmd'"

I had to do a sudo apt install python3.10-distutils before the curl worked. Now my pip is a happy 22.0.3 from python3.10 site-packages

[polar forge]

Question:
Is it safe to delete ~/.local/lib/python3.8/site-packages on Ubuntu 20.04? I kind of screwed up, and did not use venv for my project!

[velvet spire]

prooooobaly, since its in your local directory and isn't the global one

but you should uh, wait for someone else to tell you yes or no

[rapid sparrow]

No

It is a way to delete half of a system

Default python is dependency to half of GUI systems at least

[mossy fulcrum]

why did no one ever make a free soundboard? is it so hard?

basically all it has to do is load a file and play it

[rapid sparrow]

surely, easy.
But that sounds like traffic intensive thing to make
high payments for the internet traffic

So... what's the proffit

all optimizations are going to cost you some development time too

and BOOM it becomes not really easy, you will need to know how to handle Data Intensive Applications

So again comes the question, what's the proffit? 🙂

[mossy fulcrum]

why internet traffic?

you mean for the download?

[rapid sparrow]

and consider high amount of competitors in this field. Are you ready to compete for the users from. in Spotify and etc

[mossy fulcrum]

you get what a soundpad is

[rapid sparrow]

Yes, downloads have an internet traffic cost

[mossy fulcrum]

yep for that i'd use github

soundpad = tool to play sounds on your mic

like soundboard on steam or voicemod

just that these cost

[rapid sparrow]

you mean, a tool to make sound effects to your microphone

like changing your voice?

basically working with Virtual Audio Channels

[mossy fulcrum]

yep

i just wondered how hard it is to program sothing like tha

[rapid sparrow]

it does not really sound useful to have as a web app

[mossy fulcrum]

no

[rapid sparrow]

it sounds more like dekstop app

[mossy fulcrum]

yep

[rapid sparrow]

for integrations with discord/skype and etc

[mossy fulcrum]

thats what i wanted it to be

doesnt have to integrate

[rapid sparrow]

All you will need to create Virtual Microphone

and redirecting transformed into it

[mossy fulcrum]

do you know a lib for that?

[rapid sparrow]

No idea, i am not a desktop dev

I am web dev only

[mossy fulcrum]

oh ok

its not that urgent anyways since i have less to no coding experience

had to learn python before i'd start that

[polar forge]

Hey Darkwind, cheers for confirming that. I know that people are always raving about Ubuntu and Linux, but at least on macOS we don't have that particular problem.

[rapid sparrow]

i think it is possible to delete installed stuff safely by...

just run pip freeze in your global pip

and uninstall everything that it has

pip freeze | xargs pip uninstall -y

pip is not used in ubuntu

ubuntu uses just python without the pip

https://stackoverflow.com/questions/11248073/what-is-the-easiest-way-to-remove-all-packages-installed-by-pip

tbh, global pip is not even really needed

common, we all use usually just venv pips

I don't have even installed global pip 🙂

[polar forge]

y

[rapid sparrow]

my pip is coming straight from venv

[polar forge]

yea! but one misstep and you have some weird library in your site-packages!

how can I explain...

This stuff annoys me

It gets unwieldly

If you have ideas/pointers, besides "venv", to keep this under control and clean, I'd be really interested

[rapid sparrow]

requirements.txt + constraints.txt makes a really safe usage of pip

pip install -r requirements.txt -c constraints.txt

we dump pip freeze into constraints.txt

and we copy only the primary dependencies into requirements.txt

requirements.txt are the dependencies which are installed
constraints.txt dependencies aren't installed, they just lock all dependencies including to secondary dependencies

[polar forge]

hmmmm!!!

all this stuff is new to me

sounds great

(it would solve another problem I often run into)

[rapid sparrow]

yeah, my problems too. just recently discovered about it

needing to refactor all my requirements.txt files now

[tawdry needle]

i did this, it was very useful because we have specific packages installed in our org's docker base image

so i can put those package versions into a single shared shared-constraints.txt, without polluting the requirements.txt of single library/app and possibly pulling in extra deps that aren't needed (e.g. for development builds outside of docker)

but of course my changes were rejected 😆

[rapid sparrow]

i am at the moment reading SEO and thinking if Vue.js virtual JS DOM really hurts my SEO or not 🤔

everything says it should not for new google crawlers

but it still seems like a more sure way to make at least prerender

[tawdry needle]

i know basically nothing about that topic

did you mean to reply to a different message?

[rapid sparrow]

nah

I am having no experience too. Just worked a bit with parsing in the past
And reading cool book

It looks like useful book

[rapid sparrow]

Indeed. much clearner the mess than without it

Tbh I wish also...

trying to pip install from github cloud repositories directly

it would be nice for private pip packages

I wonder how well would it work

I looked at the way to setup private pip server, it is possible too, but looks like a bit of troublesome a bit to have personal pip server

not sure which solution is better

[tawdry needle]

i believe it's pretty straightforward with the right tools

[rapid sparrow]

technically yes, private pip server should be not a problem 🤔

[tawdry needle]

i have seen the combo of bandersnatch + devpi recommended before

[rapid sparrow]

if you have CI/CD pipeline, it should be all working stably all the time

[tawdry needle]

what challenges did you have?

[rapid sparrow]

I looked over Golang publishing system, and fell in love with how easy to publish packages there
I wish to reuse my private python code more often too

[tawdry needle]

seems like you don't even need bandersnatch, you can just use devpi and it will cache files from pypi

[rapid sparrow]

I wish to make reusable private python packages

perhaps even just git submodules can be used in my case

[tawdry needle]

git submodules can work too, good solution when you need/want to use "vendored" dependencies

[uneven falcon]

Hey,

I want to fork my github.com project to a github.on_corporate_site.com does anyone know if this is easily possible (via Browser)?

Command line is fine as well. Currently so no way around a local copy

[rapid sparrow]

Git clone original_url
Git remote set-url origin url_of_empty_repo_in_your_org
Git push

Forking in original way should be possible only within same git provider domain
Between different git providers, we clone and push
Or invoke migrating procedures if they exist, if we wish to transfer issues history and etc

[woven saffron]

Im curious, if I use git's hooks like so

repo
|-hooks
  |- post-receive

and run

>git config core.hooksPath hooks

it should run the hook after pushing from my local computer

Is there a way to get the post-receive hook running, even if someone is only editing the file directly in the github website, e.g., editing a readme?

Also, I reckon that every user would have to use >git config core.hooksPath hooks on their local pc, anyways..
Is there no better way to make bamboo run on every change 😦

[tawdry needle]

it should be theoretically possible to set up a git host that "forks" directly from another hosted repo

as far as i know, a hosted repo like on github is just a bare repo

so it should be possible for github et al to have a feature like "clone from external source"

and im actually a bit surprised that they don't, now that i think about it

maybe there's an additional technical obstacle

[indigo zenith]

Old question I know, but I just learned about this: https://docs.aws.amazon.com/workspaces/latest/adminguide/amazon-workspaces.html

[compact folio]

Ugh i done goofed.

Didnt notice i was working while it was rebasing

and after quite a few changes i did git rebase skip

Is there any way to go back?

If i do git reflog

I can see

[tawdry needle]

find HEAD@{1} in the reflog, do git checkout on that commit, and make a new branch there

@compact folio ☝️

that should be the commit where HEAD was pointing before the rebase

the old commits usually won't be "garbage collected" right away, so you just have to put a label back on the chain of the old commits in order to access them again

the entries with HEAD@{0} and rebase: i think are all of the rebase steps

oh cool, i just found out about ORIG_HEAD

so you can also probably do git branch mybranch-saved ORIG_HEAD or whatever

https://stackoverflow.com/a/692763/2954547

http://git-scm.com/docs/gitrevisions

ORIG_HEAD is created by commands that move your HEAD in a drastic way, to record the position of the HEAD before their operation, so that you can easily change the tip of the branch back to the state before you ran them.

[glacial temple]

Pip is not installed in my computer?

How to install pip? I’ve installed Python already

[tawny temple]

Try python -m ensurepip

If that doesn't help, try following https://pip.pypa.io/en/stable/installation/#get-pip-py if it still doesn't help, then reinstall python I suppose

[split cloud]

Try to run sudo get update pip

[sly sleet]

this is windows...

[heavy knot]

Hello I'm pretending to execute mypy by means of a bash script that looks like this:

#!bin/bash

set -x
set -e

mypy --config-file=mypy.ini my_directory

However whe I try to run the script bash mypy.sh, it prompts mypy: command not found. Any idea about how to fix it?

[tawdry needle]

try py -m pip -V

it's possible that pip just isn't on your PATH

[tawdry needle]

what do you mean by "pretending"? when you type mypy on the command line, does it work, or do you get the same error?

btw i also recommend -u and -o pipefail in bash for an "extra safe" script"

also normally a shebang needs to be an absolute path, no? #!bin/bash looks wrong, i'm surprised that even runs

#!/bin/bash

[heavy knot]

Yes when I do it in the command line it works: mypy directory

It propmts type mismatches

[heavy knot]

Ouch when I put python --version within the bash script, it propmts python: command not found as well

I feel like it's not using the conda environment

[tawdry needle]

indeed, it isn't. this was important information

running a script with bash starts a new shell, and only exported environment variables will be inherited by the new shell

you should be running scripts in the current shell session, using source

source runs a script line-by-line

also i would just remove the shebang, or at least fix it to be syntactically correct

another option would be to actually activate the conda environment from inside the shell script, but there could be issues there too

[heavy knot]

Yeah that's the thing, the PATH variable within the shell script is different

[heavy knot]

I'm gonna use that as well

I think that I prefer to not activate the conda env within the script at least for this use case, then for another shell scripts that take care of building process I will.

[heavy knot]

This one would be syntactically correct, I think so
#!/usr/bin/env bash

[finite fulcrum]

I'm looking into doing pre-commit properly instead of running my local tools through it, as it's a bit slower when using poetry as that adds an additional intermediate python process, but using additional-dependencies with strict version requirements seems rather limiting, and unlocking the requirements could produce different environments, is there a better way of doing that?

for example I have

flake8 = "~4.0"
flake8-annotations = "~2.7"
flake8-bandit = "~2.1"
flake8-bugbear = "22.1.11"
flake8-docstrings = "~1.6"
flake8-isort = "~4.1"
flake8-print = "~4.0"

as deps for poetry, but moving the pre-commit requires manual version updating for the plugins

[rapid sparrow]

how about just using CI tool to run everything on commit

[tawdry needle]

that would also work. but it at least needs to be an absolute path, with a leading /

[finite fulcrum]

huh? I'm not going to push every single commit individually just so I have some linting done on it

I'm thinking of leaving flake8 to be managed by poetry but that seems like a weird disconnect between it and other tools

[rapid sparrow]

i mean, test locally as you wish, but CI tool would be making sure if you did not forget to do it

[heavy knot]

Now I'm trying to figure out how to let that new shell know that it has to use my conda environment

[finite fulcrum]

It's not like you can forget to run a pre commit hook, and the same is used in CI to ensure it's consistent, it just seems annoying to manage the versions properly if using pre-commit in the way it's intended to be used

[tawdry needle]

alternatively you could invoke the mypy in the conda env directly

[heavy knot]

Yeah I mean, if I type mypy directory where I want mypy to check type mismatch it works perfectly

But I wanted to add some automation by means of a script

[tawdry needle]

here's a recipe to get the absolute path

#!/bin/bash 
env_name='my_env'
env_dir="$( conda info --envs | grep -Po "${env_name}\K.*" | sed 's: ::g' )"
"${env_dir}/bin/mypy" --config-file=mypy.ini ./my_directory

found it on stackoverflow, might need to adjust the grep and sed stuff

https://stackoverflow.com/a/69153217/2954547\

[heavy knot]

Wow, that's a beauty, let me try it really quick

[heavy knot]

It tells me conda: command not found

[tawdry needle]

ah. i think the default "conda.sh" bash setup creates a shell function instead of setting PATH

that would be the reason

try using source or you'll have to adjust your shell config

[heavy knot]

I'm on Windows, but I feel like i will replicate what I have right know on wsl

[tawdry needle]

are you using the "anaconda prompt"?

how are you running bash?

msys?

[heavy knot]

On a command prompt

Vscode integrated command prompt

[tawdry needle]

but how did you install bash?

[heavy knot]

I didn't, it's working laptop

[glacial temple]

My terminal in vscode will not work! Anyone ever been in that situation?

[indigo zenith]

There are all kinds of settings and things it can be so you'll need to be as specific as possible. If you had customizations, vs code updates sometimes wipe those out.

[scarlet whale]

can you make tool to create gmail in bulk?

[finite fulcrum]

Is there some way to force pre-commit to recreate an environment?

[cosmic breach]

Hey there, need someone help here with nginx
So I have a backend server running at localhost:8000, and at the same time I want to serve some static files from /var/www/html, The logic goes like this, if the requested uri starts with, api or auth I want to proxy pass it to backend, else serve the static file if it exists, else return 404 not found.

My current nginx configuration (in sites-available)

upstream backend_app {
    server localhost:8000;
}

server {
    server_name www.example.com;

    location / {
        root /var/www/html;
        try_files $uri =404;
    }

    location ^~ /(auth|api) {
        add_header X-debug-message "$request_uri was hit" always;
        proxy_pass http://backend_app;
    }
}

But it doesn't work, The static files are served fine but I cannot reach the backend endpoints whatsoever.

(Solved, it's the ^ before the tilde)

[indigo zenith]

My first thought is file permissions... If you attach to shell in your container the target files are accessible?

[cosmic breach]

Actually it was the ^ before the tilde. Didn't understand the exact reason though, you can see here: #help-peanut message

[heavy knot]

does anyone here have experience running postgresql on intel/m1 macOS for dev?

I have one m1 laptop with me and two on the way

I've been running benchmarks for the past week

and I'm lost

I have two laptops, m1/intel, and a couple of images

[abstract cape]

What's the canonical way to document dataclass attributes with Sphinx?
I'd like to just be able to use :param name: sample text as usual, but PyCharm complains about an "Unresolved reference 'name'"
And having to resort to name: str #: sample text after the docstring feels very unergonomic

[indigo zenith]

No but I would consider using Docker if you can. Postgres is just one of these apps (in my limited experience) that seems to work better that way half the time. Using a standard container image cuts out the guesswork

[tawdry needle]

i believe sphinx recognizes string literals after attribute definitions as "docstrings"

@dataclass
class Point:
    x: float
    """The first coordinate, on the horizontal axis."""
    y: float
    """The second coordinate, on the vertical axis."""

these aren't docstrings from the perspective of python, but sphinx should generate docs for them

[heavy knot]

Why pycharm so laggy?

[heavy knot]

anyone know how to show the column in a jupyter notebook in vscode? I wanna see the <80-90 char limit

[patent crow]

Hey, trying to do a custom dockerifle for pycharm

FROM python:3

WORKDIR /usr/src/app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD [ "python", "./your-daemon-or-script.py" ]

I get this,

but I can't specify the file

so is there an option for "run whatever file is in my window"

[rapid sparrow]

docker run image_name -it python3 your_any_script.py

You can replace CMD command at any time

[patent crow]

So when I click run on pycharm

It will run whatever file I have open

I don’t have to manually change the .py in the docket file

[rapid sparrow]

Docker volumes plus CMD?

Or just changeable CMD

[patent crow]

Hmm I will investigate

Thx!

[shadow crow]

!rule 5 @lucid roost asking for help with a botnet is not something you should do here

[rancid schooner]

Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

[lucid roost]

Well I am not breaking any laws?
I am testing it locally

To figure out how they work