#tools-and-devops

1 messages · Page 72 of 1

echo sapphire
#

if someone could help, that'd be great

echo sapphire
#

Got it working

#

I needed to do docker pull docker/getting-started

#

then run the command

#

my bad

#
=> ERROR [2/5] RUN apk add --no-cache python g++ make                                                                                                          6.2s 
------
> [2/5] RUN apk add --no-cache python g++ make:
#6 0.425 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
#6 5.314 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
#6 5.893 ERROR: unable to select packages:
#6 5.928   python (no such package):
#6 5.928     required by: world[python]
------
executor failed running [/bin/sh -c apk add --no-cache python g++ make]: exit code: 1
#

docker build failed....

#

Error : unable to select packages, Python?

#

I have python installed on my laptop

#

how to fix this error

rapid sparrow
#

otherwise you need to google how to install python in the chose OS for the image

#

for example this looks like fitting for alpine images

frail belfry
#

Does anyone have any Ai Dev Ideas?

rancid schoonerBOT
fast pollen
#

is there a way i can improve it?

covert sky
#

hi can i get some help with my terminal?

#

WARNING: The scripts pip, pip3 and pip3.8 are installed in '/Users/aarav/Library/Python/3.8/bin' which is not on PATH

#

im getting this error

#

how do i solve it

heavy knot
#

how to update the code after switching the branch for example i wrote some code via vs code and then created new branch and pushed the code but now i wanna see the code in my main branch again but the code is still the same after switching to main

dire quartz
heavy knot
#

git checkout main

#

is there a faster way to compare all changes at once in vs code? like instead of viewing every single change i just wanna revert all code to what it normally was on that branch

brazen forge
#

there's the discard button in the source control sidebar

#

ig it says discard all changes on hover

harsh basin
#

I’m having trouble with some code for a Mario-inspired video game

#

Do you guys know where I can get help

#

?

tawdry needle
tawdry needle
# covert sky WARNING: The scripts pip, pip3 and pip3.8 are installed in '/Users/aarav/Library...

when you execute a program on the command line (like python3), your shell looks up the full filename of the program in order to run it. the PATH environment variable is used to look it up.

the warning says: that directory is not in PATH, so you won't be able to run programs in it without using the full name. so if that directory contains a program hello, you will have to run ~/Library/Python/3.8/bin/hello, you won't be able to just run hello.

if you are OK with that, then you don't need to do anything.

since you appear to be using a mac, we need to know what shell you are using, if you do want to add it to PATH. what does echo $SHELL emit in your terminal?

tawdry needle
lyric token
#

Im not sure if this is the right chat for this, but i want to create a custom python module with some custom functions etc, but i dont know where to put it to make it accessible to all python files.

tawny temple
winter spruce
#

using setuptools can we have a monorepo?
in the js ecosystem, projects like gatsby have multiple packages per repo:
https://github.com/gatsbyjs/gatsby

is it possible to do something similar in Python? I know we have namespace
packages, but that's not what I'm looking for!

next spruce
#

I think setuptools is not setup to locate the dependencies in separate folders

sand gorge
#

Heya, setting up my setup.py and setup.cfg, what is the best way to enforce a dependency on gcc and cc1plus? I have tried to install my package on some machines where gcc was not installed and I had to install it manually using conda install -c conda-forge gcc gxx_linux-64

tawdry needle
#

i guess in setup.py you could check for the existence of gcc and cc1plus executables

#

maybe it would be interesting if pep517 frontends could check for the presence of a certain executable, eg using an env var

#

luarocks does that

#

eg if your lua package depends on readline you might have to pass luarocks install READLINE_DIR=/path/to/readline my-pkg

sand gorge
#

thanks for the tips! @tawdry needle will give some of these things a go

tawdry needle
#

putting "logic" in setup.py should be a last resort imo

#

don't put too much stock in the other stuff, it's a wishlist item 🙂

velvet spire
#

like, failed to build wheels

tawdry needle
#

i thought those had to be python package specifiers

velvet spire
#

i get that error trying to install pygit2==1.2.1 on a raspberry pi

#

one sec i'll do it

sand gorge
tawdry needle
sand gorge
#

yup - just googled it

#

thank you

tawdry needle
sand gorge
#

and then conda env create -f deps.yaml

tawdry needle
#

yes

velvet spire
#

i do know that pygit2 does depend on gcc or something when it is installed IIRC

tawdry needle
#

!pypi pygit2

rancid schoonerBOT
velvet spire
#

!pip pygit2

tawdry needle
velvet spire
#

yeah

tawdry needle
#

i guess by the time you are building wheels with c extensions all bets are off with respect to not putting logic in the setup file

velvet spire
#

but if trying to block an install because of that, you could do that

#

although at the same time, if its not required for install, then no one would like it if you blocked it then

tawdry needle
velvet spire
#

is that your github?

tawdry needle
#

yes

sand gorge
#

thank you that's useful

velvet spire
#

...

tawdry needle
#

but in the case of building up an environment specifically for building a wheel, i would argue against including pip deps in it, or actually against installing any python deps with conda

velvet spire
#
ERROR: Could not find a version that satisfies the requirement pygit2==1.2.1 (from versions: 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.17.2, 0.17.3, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.21.2, 0.21.3, 0.21.4, 0.22.0, 0.22.1, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.26.0, 0.26.1, 0.26.2, 0.26.3, 0.26.4, 0.27.0, 0.27.1, 0.27.2, 0.27.3, 0.27.4, 0.28.0, 0.28.1, 0.28.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.6.1, 1.7.0)
ERROR: No matching distribution found for pygit2==1.2.1
#

its in that list....

tawdry needle
#

lol i've had issues like this before. can't say i ever managed to fix them 😆

#

maybe some other package specifier?

velvet spire
#

idk

tawdry needle
#

so @sand gorge what i recommend is something like this:

conda create -n my-app
conda install -n my-app -c conda-forge gcc gxx_linux-64 python==3.9
conda env export > my-app.yml
pip install ./
velvet spire
#

i'm just going to try installing the entire problematic requirements.txt

#

i've never had that error before

tawdry needle
velvet spire
#

funny sad solution tho, if I install 1.5.1 I don't get an issue

tawdry needle
#

oh

#

so you need an arm build

velvet spire
#

raspberry pi os buster iirc

tawdry needle
#

and what version of python?

velvet spire
#

oh

#

huh

#

i should fix that

tawdry needle
#

plus the one source tarball

velvet spire
#

this is 3.9.6 but when i get the other issue its always been on 3.7 or 3.8

tawdry needle
#

hm, well if you do pip install --only-binary=pygit2 pygit2==1.2.1 on python 3.9 then yes, there won't be a wheel available for you

#

in fact i don't think there are arm/aarch wheels at all, i don't see them in the list

velvet spire
#

hang on, switching to 3.8 to fix that dumb identifier issue

velvet spire
#

piwheels

tawdry needle
#

huh, never knew about this

#

do they expose this via an index url that pip can use?

velvet spire
#

ye

tawdry needle
#

nice

velvet spire
tawdry needle
velvet spire
#

lmfao

tawdry needle
#

perhaps that answers your questions?

#

time for a raspi vm to build your wheels...

velvet spire
#

thing is, it builds successfully on pi with version 1.5

tawdry needle
#

wait, you get that error after building the wheel?

#

or before?

velvet spire
#

before

#

it fails to build

velvet spire
tawdry needle
#

huh

#

i wonder if there's some other version specifier that isn't satisfied

velvet spire
#

i stopped looking into it for the moment ablobsweats

#

remaking a py 3.8 venv

#

i've been working on some random scripts

velvet spire
#

@tawdry needleLOL

#
Collecting pygit2==1.2.1
  Using cached pygit2-1.2.1.tar.gz (235 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
velvet spire
#

;-;

#

we've collected the correct version
uh oh! we can't find that version!

velvet spire
# tawdry needle wait, you get that error after _building_ the wheel?

after trying to build it:
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include -I/home/pi/test/.venv/include -I/home/pi/.pyenv/versions/3.8.11/include/python3.8 -c src/blob.c -o build/temp.linux-armv7l-3.8/src/blob.o
In file included from src/blob.h:34,
from src/blob.c:30:
src/types.h:36:2: error: #error You need a compatible libgit2 version (1.0.x)
#error You need a compatible libgit2 version (1.0.x)
^~~~~
error: command 'gcc' failed with exit status 1

ERROR: Failed building wheel for pygit2
Failed to build pygit2
ERROR: Could not build wheels for pygit2, which is required to install pyproject.toml-based projects
(.venv) pi@rasp-server:~/test $ libgit2

tawdry needle
#

ah, that explains it

#

yeah, i think their attempt to make build failure non-fatal has led to confusing issues like this... need to read the entire error output

#

so the problem is that the build fails, then it can't find a suitable version and gives up?

velvet spire
#

the invalid version I only got on python 3.9.7

#

this error is from trying to install on 3.8.11

tawdry needle
velvet spire
tawdry needle
winter spruce
#

I've setup an nginx reverse proxy/ load balancer and another web server for the frontend right now.
I'll switch to traefik later when I learn more about it, but for now, the frontend throws up 404s
https://github.com/aryaniyaps/todos

I think that I haven't copied the compiled build properly, maybe?
I think that the reverse proxy is correctly setup, because the API works fine. It's the web server that is misconfigured, I think!
could someone please help me with this? thanks a lot!

GitHub

Todos w/ Flask + React. Contribute to aryaniyaps/todos development by creating an account on GitHub.

rapid sparrow
winter spruce
rapid sparrow
winter spruce
#

so nginx can handle the react app and haproxy as a balancer? is that good?

rapid sparrow
#

chain them 😉

rapid sparrow
winter spruce
#

well, this is just a local hobby project

#

I wont be deploying to cloud anytime soon

soft hare
#

This is my current release: https://github.com/doodspav/atomics/tree/v0.0.33a
In the build_py command (which gets called in build and bdist_wheel), it clones another repo, builds it, and places the built shared library into my source tree at src/atomics/_clib (before anything else happens)
Python setuptools then copies that file along with the rest of the sources into a wheel

#

It works on windows and linux

#

On Mac, everything I do works correctly, but then setuptools (or distutils) refuses to copy over the .dylib file, and i have no idea why

#
 [build_patomic] [DEBUG] Files in /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-itfnwrqz/src/atomics/_clib: [PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-itfnwrqz/src/atomics/_clib/libpatomic.dylib'), PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-itfnwrqz/src/atomics/_clib/README.txt')]

evidently both README.txt and libpatomic.dylib are there

#
     creating build/lib.macosx-10.9-x86_64-3.8/atomics/_clib
    copying src/atomics/_clib/README.txt -> build/lib.macosx-10.9-x86_64-3.8/atomics/_clib
    running build_ext
#

but it only copies over README.txt for some reason?

soft hare
#

ok well i think for now my workaround is going to be just naming the shared library file libpatomic.dylib.txt

#

wtf that doesn't work either

#

wtf is happening

#

ugh i'll find a mac and test this locally somehow

next spruce
#

and a #if LIBGIT2_VERSION < 100 #error you need a compatible version

#

you could run the same command with -dD -E to dump out the preprocessed code if you haven't already solved it

velvet spire
#

Huh, interesting

soft hare
#

On windows you only get .dll so that's easy
On linux you get .so.x.y.z and then the symlinks .so.x and .so
so i sort the file paths in reverse, and the first one is now .so.x.y.z
But on mac you get .x.y.z.dylib, with .dylib as a symlink
which sorted in reverse gave me .dylib as the first one
and then python does an isfile check, which that fails because it's a symlink and not a file

#

thank fuck that's over with, i had to debug by running it in github actions, took like 50 releases :/

serene elbow
#

I can't for the life of me figure out how to configure mypy to work properly with Ale

I have a config file at the root of my sourcefiles:

[mypy-*]
disable_error_code=501

Anyone know what I might be doing wrong?

#

... I think this didn't work because the error was flake8 not mypy... I guess always print which linter is generating the error in ale 😄

#

Solved...

mortal zodiac
#

👋🏿 Hi all! Anyone here have experience using BumpVersion, to update the release number? https://pypi.org/project/bumpversion/

This is my first time using it. It looks like all I have to do to update the minor release number is run bumpversion minor and it will update that version everywhere in my directory of files. Does that sound right?

For example, if my version is currently 3.1.0 and I want to bump the minor to 3.2.0, I'd just run bumpversion minor?

thick sinew
#

I think poetry has something similar built in to it

mortal zodiac
#

HI @thick sinew ok, I'll have to check out Poetry but I have to use BumpVersion for work.

#

I'd also like to know if I can undo a bumpversion if something get's messed up

thick sinew
#

oh if you have to use it for work that's different yeah, my mistake

mortal zodiac
#

That's ok 🙂

eager shard
#

I see python has the logging module... does this integrate with any logging systems?

tawdry needle
eager shard
#

I guess

#

Just not finding much info

#

We are in AWS

#

Ideally we could leverage some AWS service

#

I suppose everything is possible using a separate log agent, but was initially curious about something I didn't have to "install", just reference in python

umbral plaza
#

yes does anyone know how to clear the console in pycharm?

lyric token
#

is someone able to help me with custom modules?

rapid sparrow
lyric token
rapid sparrow
smoky stratus
#

Can someone explain why is round-robin the standard algorithm for load balancing for pretty much everything? I'm doing some kind of load balancing in all of my apps and round-robin was never ever a good idea for me. As in evenly distributing tasks between nodes/workers. Why evenly? If node-1 has a backlog of 100 tasks already and node-2 has zero, I don't want to distribute new tasks evenly, that makes no sense. It should always go either to the least used node, or if there's some kind of priority implemented, all should go to node-1 until it hits it utilization limit, then all the overflow to node-2 and so on

#

Is it just because it's easy to implement and not because it's the best generally?

#

I'm just curious as we're talking about load balancing but round-robin is doing zero balancing

smoky stratus
#

Great, thanks lol

dense trout
#

Anyone here knows ansible?

#

I have this repo of ansible that I have been assigned to recreate for my courseworl

#

However I am not able to figure out the repo

#

It anyone is aware do dm

rancid schoonerBOT
#

8. Do not help with ongoing exams. When helping with homework, help people learn how to do the assignment without doing it for them.

timber cipher
#

I need some advice for how to set up my powershell environment on windows so that it's able to execute some python scripts stored in a folder available on the $PATH.

I have a few scripts with a shebang #! /usr/bin/env python3 at the top which is recognized and used by the py launcher on windows.

I have a %USERPROFILE%\pythonbin folder containing a bunch of utility scripts which I want to be able to execute in powershell using py mytool.py, where mytool.py is residing in the pythonbin folder. I want mytool.py to run with the context of the current directory I'm on, like any cli tool.

I'm wracking my brains trying to figure it out, and have put the directory on the path, but whenever I run py mytool.py, it tries to look for the script in the current dir, not the one in the environment.

wooden ibex
#

Powershell won’t execute stuff in path normally

#

You will need to be very exact

smoky stratus
#

How can I force docker build to use the latest version of files? I'm building with docker build --no-cache -f Dockerfile -t consumer:latest . but the files in it are a day old

short peak
#

make sure you are copy them

smoky stratus
#

Copy what?

short peak
#

usually you do "COPY" in Dockerfile, are those the files you mean?

smoky stratus
#

Yeah of course I'm copying the source files lol

#
FROM python:3.10

COPY /backend/consumer/consumer.py /backend/consumer/
COPY /backend/consumer/requirements.txt /backend/consumer/
COPY /functions.py .
COPY /variables.py .
COPY /settings.py .

ENV PYTHONUNBUFFERED True

RUN pip install -r /backend/consumer/requirements.txt
#

I'm pushing the image then to my repo

#

And deploy to the swarm via compose

#

And make sure they pull the latest image (they do I've confirmed)

#

But the image itself used outdated source files

#

Which I can't figure out how

short peak
#

trying editing the file(s) you see outdated, do a build (no need to push), ~then build~, run and explore the container, check the edition you made is in the correct place. This will discard the problem is on build time

smoky stratus
#

You mean do two builds in a row without changing stuff between the builds?

short peak
#

no, just one.

#

I put "build" twice by mistake

smoky stratus
#

Hm it seems to have the up to date files like that

#

So the issue isn't with the build

#

That makes me even more confused

#

Because on the swarm I see the container match the image id

#

sha I mean

#

But when I inspect the volume the files are outdated

#

I do the exact same process every time and it worked fine until today

short peak
#

use different tags on images. Something like this

TAG=$(date "+TEST-%Y%m%d%H%M%S")
docker build --no-cache -t consumer:${TAG} .
docker tag consumer:${TAG} <your-repo-uri>:${TAG}
docker push <your-repo-uri>:${TAG}
#

no need to -f if filename is Dockerfile

smoky stratus
#

But why though

#

I've used the same tag since ever

short peak
#

which image is the latest if you always override it?

smoky stratus
#

It's always the one with the latest tag

short peak
#

how to track builds?

smoky stratus
#

I always just push the latest and redeploy with the latest

short peak
smoky stratus
#

But it isn't, I can see the sha match the image

short peak
#

mm

smoky stratus
#

The sha in the container matches the image sha I just built

#

Which is why I'm clueless lol

#

That's why I thought the image itself included outdated files

#

But that wasn't the case

#

Might be something with the volumes?

#

Do I need to do something extra to overwrite the files if they already exist on the volume?

#

Let me try deleting the file from the volume manually quick

short peak
#

I don't think binding volumes is the best way to overwrite files

smoky stratus
#

I'm not binding

#

At least I don't think so

#

I'm using

short peak
#

doesn't compose bind directories when mounting volumes?

smoky stratus
#

Under the service I have

volumes:
  - consumer:/backend/consumer

and in the root

volumes:
  consumer:
#

No idea

#

This is how I've done it and always worked

#

I've removed the files manually from the volume and gonna redeploy now with the latest image and see

#

Hm...

#

python: can't open file '/backend/consumer/consumer.py': [Errno 2] No such file or directory

#

But why

#

The dockerfile does copy the files I've seen them when I inspected it

#

But not in the swarm

short peak
#

when you mount the volume, binding overrides the content

smoky stratus
#

Oh

#

So how should I do it?

#

... and why did it always work until now?

short peak
smoky stratus
#

I'm deploying via compose

#

And I have it setup the way it shows in the example above that part

#

I.e.

services:
  frontend:
    image: node:lts
    volumes:
      - myapp:/home/node/app
volumes:
  myapp:
#

Do I need to add external?

#

That created a whole new volume

tawny temple
#

What I've gathered so far is that your new image contains updated files, but your existing volume is out of date. But if you delete your volume's content, then some files are missing?

#

Maybe what you need to do is make what you're mounting more narrow, so that your volume can use more of the files that are within the image. For example, by creating a subdirectory specifically for files that you expect to override or for files that are only need to be present in the mount but not the image.

#

But it's just a guess cause I am not sure what your goal is.

smoky stratus
# tawny temple Can you clarify what you want to achieve?

These are the files that are needed to be up to date with every update:

COPY /backend/consumer/consumer.py /backend/consumer/
COPY /backend/consumer/requirements.txt /backend/consumer/
COPY /functions.py .
COPY /variables.py .
COPY /settings.py .

In the compose file I've got something like this:

consumer:
  image: 192.168.0.100:5000/dev/consumer:latest
  volumes:
    - consumer:/backend/consumer
  working_dir: /backend/consumer
  command: ["python", "consumer.py"]

Then consumer.py uses /backend/consumer/logs/consumer.log for logs, which I need to preserve between updates and redeployments

#

The python script logs to /logs/consumer.py so since the working dir is /backend/consumer it should log to /backend/consumer/logs/consumer.log

#

Which it does

#

But if I delete consumer.py from the volume it breaks

#

(I don't want to delete it was just for testing)

#

(I do want to keep it up to date though)

tawny temple
#

Yeah, sounds like you need to narrow what's being mounted. You should try creating a volume just for /backend/consumer/logs/

#

Since it contains the files you want to preserve. Doing it for all of /backend/consumer/ is not necessary and is causing issues cause the volume is overriding the container.

#

@smoky stratus

wooden ibex
#

What is in those volumes?

smoky stratus
tawny temple
#

Yes, I believe so.

smoky stratus
#

Alright let me try

#

Hm that looks like it solved it

#

Thanks for the suggestion!

fathom crypt
#

Please tell me what to do in this situation? I did not find it in Google. I apologize in advance if I wrote to the wrong channel

tawny temple
urban ore
#

Hello, so I'm thinking on adding a workflow_dispatch event to my github action so I can run it manually. Will anyone who visits my repo be able to trigger the workflow or only the owner/admins can

velvet spire
#

!pypi virtualenv-clone

rancid schoonerBOT
velvet spire
#

😛

sage void
#

Anybody know of any good videos that might help me get my way through PyCharm? I want to understand how to create multiple files easily that run off the same editor? And use good techniques to do so.

rapid sparrow
vocal quiver
#

Hello, so in git I accidentally created a new branch and commited the changed and ran git checkout main. how can I enter the previous branch again (I cant remember the name) and merge it with the main brach?
I did a lot of changes in that branch, Thanks

vocal quiver
rapid sparrow
mortal panther
#

it took me a while to figure out how to get correct pythonPath, so I wonder if it works for everyone now

vocal quiver
rapid sparrow
#

no idea how to help you fixing it without knowing which commits you are expecting to have and what not
analyzie with git log -n 5 and e.t.c.

vocal quiver
#

as far as i remember the last commit i made has a message like this "Added ChannelViewSet"

vocal quiver
#

git reflog shows the commit I was searching for

#
(Chat app) C:\Users\Desktop\Programming\Chat app>git reflog
23e9484 (HEAD -> main) HEAD@{0}: checkout: moving from 99dec4e6e388a1cd50fc7182529c6adecf19d699 to main
99dec4e (origin/main) HEAD@{1}: checkout: moving from main to remotes/origin/main
23e9484 (HEAD -> main) HEAD@{2}: checkout: moving from 3c2982ec553e0b845da8bd81809fcc074ed1b4eb to main
3c2982e HEAD@{3}: commit: Implemented ChannelSerializer.messages
8190173 HEAD@{4}: commit: Added MessageCreateSerializer
f627351 HEAD@{5}: commit: Fixed AttributeError
rapid sparrow
#

push to it

#

compare branch with main

#

if there are just new commits, and everything else main has, just merge it from branch to main
if not... then pull from main to branch, resolve conflits, then can merge branch to main ;b

#

that's how I would have solved it at least.

rapid sparrow
vocal quiver
#

Thanks 👍, I'll try that

#

Thanks a lot!!!, it worked lemon_hyperpleased

#

I was trying to fix that for hours

rapid sparrow
#

well, u a welcome anyway

mortal panther
#

@vocal quiver git checkout -b creates a new branch from the one you are currently using. so checkout to main or whatever branch you need before running the command

mortal panther
#

👍

sand thistle
#

errr having some git issues

#
hint: Clones of the outer repository will not contain the contents of
hint: the embedded repository and will not know how to obtain it.
hint: If you meant to add a submodule, use:
hint: 
hint:   git submodule add <url> expedien-staffing_website/tf_for_exp
hint: 
hint: If you added this path by mistake, you can remove it from the
hint: index with:
hint: 
hint:   git rm --cached```
#

i deleted the file in my directory that contained my ill-imported code

#

but i guess the VCS still sees it as a sub module?

#

i follwed the instructions, and it's given me a loop

#

like i get a

#

fatal: not removing 'asdkasjdla' recursively without -r

#

so i use it and it appears to delete recursively

#

then I do a git add .

sand thistle
#

any advice would be appreciated

fluid venture
#

Does anybody know what signal is sent when the debugging session is stopped in vscode? I want to capture that signal

heavy knot
#

ok so how can i make in python voice text to speech with my own voice

indigo zenith
next spruce
fluid venture
next spruce
#

i don't know how they would stop you though

maiden shadow
#

I have a branch with two git commits.
I want to throw away the first commit and keep the changes from the second commit.
Basically, I only want the most recent commit.
How can I accomplish this?

next spruce
#

maybe at the C level it could be caught?

lusty finch
#

Do we agree that doing
docker run -itd XX YY
Does not make sense because the container will be detached and therefore no tty access?

maiden shadow
#

Anyone know git well?
I’m trying to do the following:

I have a branch with two git commits.
I want to throw away the first commit and keep the changes from the second commit.
Basically, I only want the most recent commit.

indigo zenith
crisp kettle
#

any good alternatives to invoke release for a version management system that's friendly with poetry?

#

there's versioneer but that seems to be stuck on distutils

#

(is this really a tools/devops question? dunno where else it'd go)

tawdry needle
#

@maiden shadow the tool for this is rebase or cherry-pick. If your branch is ahead of master by two commits, git rebase -i master and delete the first line (the commit you don't want) in the editor window that comes up

maiden shadow
#

@tawdry needle thanks! I’ll try the rebase approach

mossy spear
crisp kettle
#

yeah, I think we'd want something a bit more fully-featured, e.g. to cut a release branch and tags on git

velvet spire
#

....

#

@flat path yknow how i tell you to use poetry sometimes

#

but its times like this which make me regret it

#

poetry remove pydantic

uninstalled python-dotenv

knotty zodiac
#

No clue on which channel I need to ask this question but does anyone know where is the best place to start at while learning python? I'm new to it all and just want to find the best place to self teach myself.

wicked nimbus
#

hello people how do i fix fix Cannot connect to host www.somewebsite.com:443 ssl:default [[SSL: WRONG_SIGNATURE_TYPE] wrong signature type (_ssl.c:1129)] in Ubuntu 21.04 x64 i did my research and believe this is related to OpenSSL

indigo zenith
rancid schoonerBOT
#
Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

ocean flicker
#

Hello!

I want to read .xls files in my project. Also I want my project to be installed with ./setup.py install.
The problem is with the following setup.py. ./setup.py install crashes when executed the first time but completes successfully on the second execution.
Why does it fail on the first execution and how to fix that? Manually installing packages via pip works fine.

#!/usr/bin/env python3
from setuptools import setup, find_packages

setup(
    name='test-scripts',
    version='0.0.1',
    install_requires=[
        'pyexcel-xls',
        'pyexcel'
    ]
)

Excerpt from the log of ./setup.py install. It looks like it tries to install pyexcel-xls instead of pyexcel.

... part of the log omitted ...

Searching for pyexcel
Downloading https://files.pythonhosted.org/packages/71/20/a8945b1acc2076b78c3a2319389364f088a75474e07178ac8d9250dd0a48/pyexcel-xls-0.7.0.tar.gz#sha256=5ec606ef8667aafbb0c3fbd8242a7c23bf175ee7c10b08f70799b84fb2db84cb
Best match: pyexcel xls-0.7.0
Processing pyexcel-xls-0.7.0.tar.gz
Writing /tmp/easy_install-84mdsfn6/pyexcel-xls-0.7.0/setup.cfg
Running pyexcel-xls-0.7.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-84mdsfn6/pyexcel-xls-0.7.0/egg-dist-tmp-gse4h1ts
removing '/home/artur/xx/venv/lib/python3.8/site-packages/pyexcel_xls-0.7.0-py3.8.egg' (and everything under it)
creating /home/artur/xx/venv/lib/python3.8/site-packages/pyexcel_xls-0.7.0-py3.8.egg
Extracting pyexcel_xls-0.7.0-py3.8.egg to /home/artur/xx/venv/lib/python3.8/site-packages
pyexcel-xls 0.7.0 is already the active version in easy-install.pth

Installed /home/artur/xx/venv/lib/python3.8/site-packages/pyexcel_xls-0.7.0-py3.8.egg
error: The 'pyexcel' distribution was not found and is required by test-scripts

Python 3.8.10 is used with Ubuntu 20.04.

#

I managed to fix the problem by adding a lower bound for pyexcel version: pyexcel>=0.6.7. However I still don't understand the cause of the problem.

maiden shadow
#

Docker:
How can I containerize the following application?

Repository called: ‘env’
Repository called: ‘code 1’
Repository called: ‘code 2’

‘code 1’ and ‘code 2’ both need/depend on ‘env’ since that’s the Python libraries and licenses etc

Code 1 and code 2 can each be their own container since they’re separate services.

trim ermine
#

Just pull the python image. Install package Copy repos. Run repo

maiden shadow
#

@trim ermine I can’t just “install” anything

#

I need to specifically use the repo called env since those contain all the packages and software licenses

trim ermine
#

What do you mean by repo something hosted on git? And yes you can install packages with pip thats what its for. You build a docker file layer by layer. You can install depedencies. You can copy files. You change permissions. Thats the beauty of Dockerfiles

#

So if all of your code is local. You COPY files and folders over

#

If its in a repo. You pip install, ADD, or use a package manager.
If its in subversion you clone it.
If its in git you clone it or use PIP if its python.

#

What am i missing? @maiden shadow

mental steppe
#

Hey folks, I'm attempting to swap from a setup.py to a setup.cfg file, mostly it's straight forward, but I seem to have hit something of a snag. Our CI/CD pipelines uses a version.py file to extract and increment a __version__ string when we do various releases. In setup.py it's pretty simple to just say "read the value of __version__ from that file" but it's much less obvious how to do that in setutp.cfg. I'd prefer not to have to modify our CI/CD pipelines, so what's the best way to extract said value from the file into the setup.cfg . The documentation says you can use file: or attr: but I can find nothing on how those actually work.

mental steppe
urban pecan
#

if it ain't broke..

tawdry needle
gentle solstice
#

When making a utility docker image (like adding a root certificate), how should I name it?

#

Just make it under my own name?

visual ridge
#

Hi, I am making an open source project, but what I have noticed is people take the code and publish to their own github/gitlab without consent and proceed to change the liscense and claim the project is their own. I want a liscense similar to th MIT one, which allows all uses but the person must have the original liscense in their code unchanged

deep estuary
#

MIT says you have to distribute a copy of the license wherever you have taken code and redistributed it

#

if you want something stricter than MIT, you should look at GPL

velvet spire
#

was described there

deep estuary
visual ridge
deep estuary
#

I'm not sure of a reason why you can't modify a license, you'd have to check though, technically licenses themselves are copyrighted and not all behave the same. I'm not a lawyer, though.

visual ridge
#

Alright I will look into the same

#

Last question, will the gpl liscense force th party to retain the liscense unmodified? I could not find the answer online, and looking at the liscense itself was vague because it says the redistrobution and binary must have the liscense and disclaimer but it dosent say it cannot be modified

velvet spire
#

oh also

#

note that a license change only affects it going forward

#

so all of your previous code is still mit

silver summit
#

hey

#

i am trying to install one pip module and i am getting error

#
running build_ext
    building 'pdftotext' extension
    error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

#

i am trying in docker image

#

docker image is of ubuntu with wines

#
sudo docker pull cdrx/pyinstaller-windows
#

here is the image

#

after pulling when i am connecting the image shell

#

and trying to install pdftotext it keep showing error of need build tools 14 or above

#

i cant find any solution to fix this

ivory sierra
silver summit
#

i dont understand

silver summit
#

you can just try pull the image sudo docker pull cdrx/pyinstaller-windows and after connecting to the container with image ID try installing pip install pdftotext

tawdry needle
golden flax
#

does anybody know how wiremock works?

visual ridge
#

Hi, I am making a python project with a python backend, such as the flask library (specifically using streamlit). How would I be able to tell github to host the site? all thats needed is

pip install -r requirements.txt
cd src/
streamlit run app.py --port 80
gentle solstice
#

:/

+ docker build . -f docker/Dockerfile
Sending build context to Docker daemon  184.3kB

Step 1/43 : FROM python:3.9-slim as python-base
Error parsing reference: "python:3.9-slim as python-base" is not a valid repository/tag: invalid reference format
tawdry needle
#

@gentle solstice does it need to be upper-case AS?

gentle solstice
#

it works locally. It just doesn't when I run it in the docker image on kubernetes

tawdry needle
#

huh

gentle solstice
#

I tried using podman, but I get a different error.

+ podman build . -f docker/Dockerfile
time="2021-11-17T16:11:19Z" level=warning msg="\"/\" is not a shared mount, this could cause issues or missing mounts with rootless containers"
cannot clone: Invalid argument
user namespaces are not enabled in /proc/sys/user/max_user_namespaces
Error: cannot re-exec process
gentle solstice
#

It was because my kubernetes cluster was running docker 1.13

rancid schoonerBOT
#

Hey @golden flax!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

golden flax
#
order-service:
    container_name: api
    build:
      context: src
      dockerfile: Dockerfile
      target: dev-build
    environment:
      DEBUG: ${OS_DEBUG}
      DB_USER: ${OS_DB_USER}
      DB_NAME: ${OS_DB_NAME}
      DB_PASSWORD: ${OS_DB_PWD}
      DB_HOST: ${OS_DB_HOST}
      DB_PORT: ${OS_DB_PORT}
      ALLOWED_HOSTS: ${OS_ALLOWED_HOSTS}
      SECRET_KEY: ${SECRET_KEY}
      CORS_ORIGIN_ALLOW_ALL: ${OS_CORS_ORIGIN_ALLOW_ALL}
      CORS_ORIGIN_WHITELIST: ${OS_CORS_ORIGIN_WHITELIST}
      DJANGO_LOG_LEVEL: ${OS_DJANGO_LOG_LEVEL}
      VIRTUAL_ENV: /home/app/.venv
      PATH: /home/app/.venv/bin:/home/app/.poetry/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    command: bash -c "cd src && gunicorn --bind 0.0.0.0:8000 --timeout 600 --reload --env DJANGO_SETTINGS_MODULE=order_service.settings order_service.wsgi"
    links:
      - db
      - product-service
    depends_on:
      - db
      - product-service
    volumes:
      - ${OS_HOST_PROJECT_DIR}:/home/app/src
      - ${OS_HOST_PROJECT_DIR}/pyproject.toml:/home/app/pyproject.toml
      - ${OS_HOST_PROJECT_DIR}/poetry.lock:/home/app/poetry.lock
    ports:
      - 8000:8000
    networks:
      - internal
      - external

product-service:
    image: rodolpheche/wiremock:2.23.2-alpine
    command: --verbose
    hostname: productservice
    volumes:
      - ./wiremock/product-service/stubs:/home/wiremock
    networks:
      - internal
      - external
    ports:
      - "8081:8080"
#

i am unable to call api from order-service to product-service... this is my docker compose can someone help

vague silo
#

@golden flax do you get any error for the call? does it time out? is the connection refused?

molten lake
#

i have a venv with a bunch of dependencies, and I want to create a standalone exe for the python file in it

#

i tried pyinstaller inside the venv, but I am getting a dependency problem

velvet spire
#

how can i squash two merge commits from the same branch

supple venture
#

interactive rebase?

velvet spire
#

ended up just redoing the merge conflicts

untold crystal
#

always rebase and avoid them altogether 😛

velvet spire
#

I do now.... but this is an old branch

untold crystal
#

rebase the old branch

velvet spire
#

nope, too many merge conflicts to do that

untold crystal
#

haha, when thats the case, i squash the old branch into one commit and then just deal with the conflicts once

#

youre gonna deal with the conflicts no matter if you rebase or merge so, meh

tawdry needle
quasi star
#

why am i getting this error....
i have Kivy installed as a extension...

golden flax
untold crystal
golden flax
#

I'm getting "client unexpectedly closed TCP connection for rabbit mq" while publishing does anybody know whats goin on?

#

Im new to rabbit mq

untold crystal
untold crystal
#

check vs code's configured interpreter , control/cmd + p, then type python select and look for interpreter

#

make sure it matches where you installed kivy

quasi star
tawdry needle
quasi star
untold crystal
quasi star
quasi star
quasi star
trim ermine
#

You spelled widget wrong

#

wideget

#

Love me some wide gets

quasi star
quasi star
# trim ermine "WIDE GET"

do you know about kivy
bc if you do i really need help
the tutorial im following says i need to create a file and give it a ".kv" at the end but visual studio doesn't recognize it

trim ermine
#

Why would it? Its a text editor

winter spruce
#

Kubernetes has a lot of abstraction layers. Rn I'm familiar with docker and docker compose, but k8s is overwhelming, it seems like it might take a month to learn

#

Any tips to Speeden the process? I'm willing to put the hours in. The thing that bothers me is the yaml file boilerplate

rapid sparrow
#

which makes me putting it as at last thing to learn 😦

#

because there are so many easier and faster things to learn

trim ermine
heavy knot
#

Hello!

I am trying to build a small python code for productivity purposes (https://github.com/thomashirtz/notion-scholar). I would like to be able to make it easy for people to install it and use it. I have several requirements though, it must be easy to install, and I would also need an easy way for the user to set up some permanent variables for the configuration of the software.

Does someone know a good way to do that ? I found many information about PEX or else. However I didn't find much answer for the user config part

Thanks 🙂

tawdry needle
#

on windows and mac, maybe pyinstaller is the right tool for the job

#

i think pyinstaller works for linux too

urban pecan
#

hey

#

sent the guy an email to explain what actually are the problems with python packaging on distros

#

here's the answer

#

you're heavily complaining (quite rightfully so) about the state of
packaging in Python, but I don't see any mention or link to what exactly
are "the needs of any of the distros, despite our repeated pleas." -
could you please elaborate?

Here are a few common use-cases which are forgotten or explicitly
ignored by various newfangled Python build systems/package managers/bit
twiddlers:

  • Building & running tests without the network and without fetching
    dependencies during the build
  • Installing into an arbitrary prefix
  • Installing into an arbitrary prefix within a DESTDIR
  • Working with sysroots when linking to native code
  • Correct use of pkg-config
#

now, i'm wondering if there is anything that could be done to address those issues with justuse lemon_raised_eyebrow

#

i guess we're guilty of some of those points, too (at least having packages in ~/.justuse)

#

i'm not even sure what "installing into an arbitrary prefix" or "DESTDIR" means?

next spruce
#

prefix is where python is hard-coded to look for stuff like /usr vs /usr/local vs /opt, DESTDIR is the "staging area" where everything is copied to during "make install" (so you could use /tmp/output and then create an atrchive from the files there, which would include /tmp/output/usr/bin/python etc)

urban pecan
#

hmm.. couldn't we solve those dir questions via our config?

#

we'd still have a ~/.justuse folder, but only with the config, logs and registry

#

we don't need our packages to live in our home

#

i wonder if that would suffice to appease the distro maintainers

tawdry needle
#

I don't understand these complaints

#

Good idea to email and ask but his response makes no sense

#

They apply equally well to perl and ruby

#

And nobody is complaining about packaging perl applications

next spruce
#

how would you "fetch" dependencies without a network, I wonder

#

if you run pip with a specific, local index, and the .whl files are either all local or in pip's cache, no fetching should be required. so no network

#

I think they can be worked around -- sysroot can be avoided by using docker, which is the strategy cibuildwheel uses

tawdry needle
#

what's the concern with fetching exactly? pep 517 build deps?

#

having a full local mirror of pypi seems excessive (but maybe good for other reasons like not abusing their servers). can't you disable build isolation in pip, and otherwise declare the pep 517 build deps as build deps for the distro package?

next spruce
#

not sure

#

turns out pip calls itself when PEP 517 is used

#

which might be part of the pain point around passing flags and sysroot and such

rapid sparrow
#

Have containers in your docker registry history with different tags in order to have previous app versions

#

That makes u exposed only to be not having some dependency during development of future versions, but I think it is sacraficable enough thing. For all small products and up to medium it looks like having containers enough

rapid sparrow
#

If your target computer is not having docker, then download docker installers offline ;b

slender grove
#

what's the best way to get binaries for a specific micro version of python on linux? here's where i'm at:

  1. deadsnakes ppa works great in general, but i can't figure out how to trick it into giving me a micro version that isn't latest
  2. pyenv works great for dev, but in prod i want a python compiled with --enable-optimizations, because python is much slower without it. unfortunately, it then takes a long time for it to compile python
  3. conda? i've tried to stay away from conda ever since i had to spend a day debugging a pip + conda package clobber. but maybe it's my best option here?
rapid sparrow
next spruce
#

any standard dependencies certainly could be

#

you could do it with pip once and then reuse right

urban pecan
#

confusion and blame for the confusion going around 😄

next spruce
#

hopefully there will be some more answers from the maintainer

#

it would be nice to know exactly what he expects to do

urban pecan
#

yeah

tawdry needle
#

Ty I didn't know there was a public inbox thing

urban pecan
#

yeah, me neither

stone rivet
#

I have many nodes with random branches and versions of services, and whant to create universal jenkinsfile for them.
I make jenkinsfile with properties like node name and branch, etc.
But i don't need to display all of them (node names) in single select action on build page (by default).
How to create auto selection for node name from properties in jenkinsfile per project?
Maybe create jenkins pipeline with node name, and get its name to select node name in build menu? (And make other node name in list disable?)
lemon_exploding_head

wanton mirage
#

this might be a stupid question but I am learning docker and have a little python script that I am trying to package up and I was just curious if there was any way to force a container to always be run in interactive mode

civic grove
#

Does anyone know of any tool, that is free, for designing the architecture of a software?

#

basically graphs

rapid sparrow
civic grove
#

ohhh that looks very good

#

thank you

rapid sparrow
#

it can design anything

#

from database models, to UI interface 😉

civic grove
#

yeah that is pretty sick

sonic bay
#

shot in the dark but this seems like the right channel:

anyone have experience with talos/rancherOS/k3os/k0s? Looking to build my own multi-master k8s cluster the proper™ way and to not treat my previous single master/worker node like a precious pet any more

velvet spire
flat path
#

Honestly I wouldn't bother with caching for pip

#

Caching pip's cache is IMO not that useful since it's only downloads and local wheel builds you're saving yourself from doing. The installation is still slow (I'm eagerly waiting for parallelized wheel installs from pip).

#

You could cache the whole site-packages but that's actively discouraged as it's prone to breakage.

#

@velvet spire 😄

velvet spire
velvet spire
#

but it stores it as the key being the hash of the files that build it (pyproject.toml and poetry.lock) and rebuilds it when those don't hit a cache

velvet spire
flat path
#

And OS and Python version down to the bugfix version?

velvet spire
#

hmmmm

wooden ibex
velvet spire
#

will check

velvet spire
wooden ibex
#

The desire for “efficiency” can override common sense

#

How long does it really take for pip to run in your container?

velvet spire
#

2-3 minutes to install the dependencies iirc

wooden ibex
velvet spire
#

meh

#

nothing has broken yet due to that

#

and actually

#

I can add a small if to never hit the cache on a rerun of a workflow

wooden ibex
#

I’m getting my DevOps self out of this mess

velvet spire
#

lol

#

honestly tho

#

github actions has bigger problems

#

specifically that merging with the target branch on every pr workflow

#

like

#

why

velvet spire
#

if I ever get any problems from the caches I'll remove them

wooden ibex
velvet spire
#

lol I'll put a comment on it /srs

#

but also

velvet spire
next spruce
#

is there another way to do a PR?

velvet spire
#

wdym?

velvet spire
next spruce
#

it only would if someone opens a PR, right? and the PR was merged

#

is there some other merging?

velvet spire
#

no. every single workflow that github actions runs, if its checked out, it merges it with the base branch and runs the workflows

next spruce
#

ohhh yeah

velvet spire
#

which is utterly stupid

#

its also why workflows won't run if there are merge conflicts ;-;

next spruce
#

I don't even know if it warns about it. it's surprised me before

#

the worst I think was, I made a PR from my fork to an upstream repo, and somehow it ended up altering the PR and used main as its scratch space

#

was quite surprised to find commits on main (it had not been merged)

#
  • created a PR from my fork's topic to upstream main
  • made a PR to that PR (on upstream), say topic-patch into topic (details fuzzy, I realize it may not make complete sense)
  • merged the second PR in upstream
  • github actually committed the result on the upstream (!) main instead of making a topic branch to match the fork one
#

that is what I remember, not sure how I specifically did it

valid sonnet
#

hi all. can I ask here a question about an error I get in VS CODE and Github?

velvet spire
#

interesting, someone just showed pex to me, and now I'm curious, what are the ideas of using pex for test environments, to be able to test optional dependencies

#

!pip pex

rancid schoonerBOT
neon jungle
#

pylance be like

#

not sure if thats a bug on their end or what but yeah i'll just ignore it

untold crystal
neon jungle
#

but i'm pretty sure that's written mostly in rust

#

which could be the reason

untold crystal
#

oh, so its not in python.. then yeah i would expect odd pylance behavior

#

although they should have stubs setup

#

did you check "from_file" has a class method decorator though

wanton shale
#

I've got one for you guys

#

Light-mode or dark-mode for your IDE?

somber lion
#

Light mode content, dark mode interface.

wanton shale
#

Have done a fair bit of graphic design and UX/UI work, I know that certain things come into and out of fashion, and certain other things don't

#

Pair your serifs with sans serifs, never use a colored font, the only trustworthy color is blue

#

I can't shake the idea that when working with a complex problem, most people think better in dark mode

#

I'd bet a few dollars its not a fad is what I'm getting at

woeful dew
urban pecan
#

i like programming in dark mode, but reading texts i prefer light mode

velvet spire
#

i actually need to update

#

first world problems: need to update font

#

updated

urban pecan
#

yeah, not a bad font. i like

wanton shale
#

But yeah, for programming, monospace is best for technical reasons. I can't imagine using anything else (though, spacing and serifs are orthogonal to each other)

tawdry needle
#

the jetbrains font is pretty good too

velvet spire
#

I'll have to check it out

deep estuary
tawdry needle
#

looks like dejavu or something

#

or maybe adobe source code pro

velvet spire
#

lol

deep estuary
#

fira code

tawdry needle
#

huh no kidding

#

i always think of fira as being more "serif-ey"

deft summit
#

Hi, have any of you used a tool called mkdocs? I love it, but I'm not sure if any of you have tried to add a figma design to it? I'm trying to see export a UI/UX library into mkdocs for viewing. I don't really want to just add a pdf export but ideally a markup implementation but I don't need it to be data-driven or interactive like storybook

tawdry needle
waxen lily
#

tool for people who want to reverse a text:

#Imports
import PySimpleGUIQt as sg
import pyperclip

#theme
sg.theme("dark")

#layout
layout=[
    [sg.Text("Text Reverser by blabla_lab!")],
    [sg.Text("Text:")],
    [sg.Multiline("", size=(None, 10), key="-TEXT-")],
    [sg.OK(), sg.Exit()]
]
# show window
window = sg.Window('Text reverser', layout)

#Events
while True:
    event, values = window.read()
    if event == sg.WIN_CLOSED or event == 'Exit':
        break
    elif event == "OK":
        text = values["-TEXT-"]
        print(text)  # debug
        text = text.split(" ")
        print(text)  # debug
        text = text[::-1]
        results_popup_window = sg.popup_ok(f"result:\n{' '.join(text)}\n\nclick ok to copy the result", ) # ' '.join(text) to join text in list with a space
        pyperclip.copy(' '.join(text))
        sg.popup_no_border
window.close()
deft summit
molten lake
#
vast shore
#

Hello

tough fossil
#

Hi

velvet spire
#

I'm switching to coveralls from codecov 👀

#

and ensured I set up my workflow so it wouldn't fail when coveralls is down lol

brisk lake
#

okay so i got a small poetry project thats going to be used by a non technical person
right now its being invoked through a powershell script, what would be the easiest way to distribute this

#

ideally i would rather avoid having to install poetry on their machine

#

should i bake the powershell bit into the python project and use pyinstaller or something like that?

visual oxide
brisk lake
#

See im not sure what i can or cant do with it
We have a sharepoint site where we dump data files and "deploy" internal products like a tkinter app i work on and a bunch of macro books
I was thinking of just dumping it in there but the powershell script calls the python code through poetry atm

#

So i'd have to install poetry

velvet spire
#

I mean, you wouldn't have to

#

just make a mock poetry console script KEKW

flint lynx
velvet spire
mental crag
#

.

sharp wasp
#

hey

#

i want to connect to this wss://streamer.cryptocompare.com/v2 in python
and it connects in postman
but when i write my code it says getaddrinfo failed

HOST = 'wss://streamer.cryptocompare.com/v2
socket_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket_client.connect((HOST, 80))
ionic trellis
#

Can you guys help me to Scraping this web https://www.asahi.com/news/

vague silo
# sharp wasp i want to connect to this wss://streamer.cryptocompare.com/v2 in python and it c...

this is definitely not something you want to use the raw socket API for, thats far too low-level to sanely deal with websockets. use a websocket library like https://github.com/websocket-client/websocket-client instead

GitHub

WebSocket client for Python. Contribute to websocket-client/websocket-client development by creating an account on GitHub.

vague silo
#

most news websites are not happy with it. i can't read chinese / japanese though

drifting echo
#

Do we have any vim/gvim user in here?

#

map <F5> i{<Esc>:w}<Esc>
map <F6> i{<Esc>:!%:p}<Esc>

#

If so would these vimrc mappings work cause i cant make them work

#

I am trying to make f5 save and f6 run the code

#

ping me

drifting echo
#

Nvm i got my vim fully configured now :DDDD

sly sleet
heavy knot
#

Hello, I posted a question about celery in #help-bread id someone could help

static geyser
#

Having a little trouble, trying to use pip in shell but it 'is not recognized as an internal or external command', I type in python but don't get the version back

rapid sparrow
#

python -m pip --version

#

optionally perhaps python3 instead of python, and pip3 instead of pip

static geyser
#

"python was not found"

rapid sparrow
#

python3 --version ?

#

what's your operational system

#

python is not written into shell path then

static geyser
#

"python was not found" windows 10, and I was pretty certain of that just didn't know how to fix it, although when I was typing my message earlier i couldn't remember the term shell path so I just left that part out

rapid sparrow
#

find where it is installed

#

and request
"C:\full_path_to_python3.exe" --version

#

global python is needed only to use the one and only command anyway, to create local python environments with venv

#

"path to python\python3.exe" -m venv venv

#

venv/bin/activate.bat

#

and u have local project python environment

#

if venv is not existing, make sure to install, pip3 install python3-venv

static geyser
#

Thanks! I'll look that all over when I wake up.

wicked island
#

Hello. What programming language is needed to develop a web-based website like discord? Also discord has its own "sort like" language for its bot.

near mauve
#

do we need to use virtual env when using docker-compose?

rapid sparrow
rapid sparrow
orchid bramble
#

Accidentally switched off a warning or more like a suggestion where PyCharm asks to change every variable inside a function to small letters

Any idea how to turn it back on? (Ping while replying, Thank you)

celest compass
#

Hello! How can I create two packages within a single setup.py?

tawdry needle
#

a distribution package can contain several import packages, but each setup.py file can declare exactly 1 distribution package

celest compass
tawdry needle
#

however this is considered bad practice nowadays and is liable to confuse the hell out of users

#

if you want 2 separate entries on pypi.org, then you want 2 separate setup.py files

#

however, you can have two separate distribution packages that use the same namespace

#

look up "implicit namespace packages", you just have to change what you specify in setup.py

celest compass
#

understood, I'll have a look at that thank you. I think my_distribution_package containing my_distribution_package.reusable_stuff and my_distribution_package.more_reusable_stuff is the way to go for me

tawdry needle
rapid sparrow
#

I just remembered that I had a situation

#

When image had an older pip package six installed

#

I was forced to use venv in order to solve conflict versioning issue

shadow crow
#

Is it weird to proxy_pass to a lambda in a reverse proxy?

vague silo
shadow crow
#

There's just a static link to it

#

why does it matter?

vague silo
#

well, if the IP address (or well, your reference to it) would change every now and then, then that would be a problem. But if you have a static link, I don't see anything wrong with it

shadow crow
#

So this is my nginx setup:

  1. If the request matches location /foo, redirect to my magic lambda
  2. Otherwise, try to serve static content
    does this make sense, somewhat?
#

I'm redirecting to a lambda because the VPS is really crappy XD

vague silo
#

Yup, apart from the "magic lambda" part that's a common setup for sites that have static CSS etc.

shadow crow
#

TIL certbot automagically modifies your nginx configuration

vague silo
#

yup yup yup

#

just doesnt play super nicely with things like ansible

#

but you can tell it not to

shadow crow
#

@vague silo oof, on my other domain certbot successfully created an infinite redirect in my nginx conf

#

because I already had a handcrafted redirect in there

vague silo
#

for certbot?

shadow crow
#

hm?

vague silo
#

like, did you already have a handcrafted redirect in there for certbot

#

I have a configuration file like this:

location ^~ /.well-known/acme-challenge/ {
    root /var/www/_letsencrypt;
}

which is then included in the HTTP server (server 80) block for certbot

shadow crow
vague silo
#

that's where lets encrypt requests the challenge files from

smoky stratus
#

I have a repo on GitHub that has 2 apps in it, a consumer and a publisher. I deploy the two apps as a stack with 2 services on a Docker swarm via compose. Can I setup CI/CD in a way where if the commit changed consumer it will rebuild and redeploy that, but not publisher?

#

I have separate repos both both on the registry, but the Git is the same for the project it's not separated by app

heavy knot
#

It sounds possible, if you have some way of detecting if either changed, like checking specific subdirectories in a diff

smoky stratus
#

The reason I'm asking is because it's pointless to rebuild & redeploy publisher if it's not getting modified, only the other one

smoky stratus
#

They are in different directories

heavy knot
#

You should be able to create a diff between commit hashes, and compute something based on that, potentially in a step before the buidl that sets variables

#

And then make the build steps conditional on the result variables of that pre step

#

Not sure if that is the best way to do it, but it should work

smoky stratus
#

What would be the ideal way? I don't want to use multiple GitHub repos because the repo is for the same project just different parts of it, but I also don't want to redeploy the entire project then only one part of it changes because that would be inefficient

heavy knot
#

Honestly, they sound like different apps to me; I'd probably have placed them in a different, dedicated repository as well

#

But, you could use the diff strategy to look at if something changed in a subdir, I guess

tawdry needle
regal fjord
#

have a Pycharm/git question: Using pycharm I create a project and started working on it, went through the usual setup for a venv and such, then committed it to git using command line to push it to github (should have used the integrated approach, ohh well).

How do I add project settings/venv settings to the git repo? right now my python files, license, gitignore, readme, etc are all in git, but not my venv directory, nore any of the project settings, when opening it on another computer I have to go through trying to configure things to work (run configurations, etc), and no venv option shows up.

cosmic breach
shadow crow
#

The venv is built specifically for one system, so you can't really move or share it. Instead, you should save the list of packages (using poetry or just requirements.txt) that your project needs

short peak
cosmic breach
wooden ibex
shadow crow
#

so it's like alpine on caffeine?

smoky stratus
#

How can I push an image to Docker Hub in a way that actually updates it on the repo?

PS C:\Users\Martin> docker push markomartin/test:latest
The push refers to repository [docker.io/markomartin/test]
b7e5d2286d77: Layer already exists
9053d005a9e4: Layer already exists
48a1228e9de5: Layer already exists
6f06b418d75b: Layer already exists
42ce61e841fa: Layer already exists
a9b125f164c3: Layer already exists
e24045f8c247: Layer already exists
b7b662b31e70: Layer already exists
6f5234c0aacd: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
latest: digest: sha256:097e60c5940d3f0e4261e564b1c2af5d18b0ad19a9c635e8bd40acffedbc830b size: 3053

How can I update the image on the hub?

barren iron
#

So if you have made changes, re-build the docker-image, then push the changes

smoky stratus
barren iron
#

Can I see all the commands you are using

#

And how do you know it is out of date?

smoky stratus
barren iron
#

My guess is you have mistyped something somewhere

#

Ah

#

I'm sorry but I can't be of much more help if you can't get my commands. I don't use PyCharm

smoky stratus
barren iron
#

Is it a private repo?

smoky stratus
#

Yes

barren iron
#

Right. Again, can't really help you there either

#

Put the command you sent in suggests it is 100% up to date according to docker

smoky stratus
#

Also

#

LAST PUSHED
9 hours ago by markomartin

barren iron
#

Yeah that is normal

#

Cause there are no updates

smoky stratus
#

But I pushed an update a few minutes ago

barren iron
#

I don't think you have re-built the image

smoky stratus
#

Changed up the commands in the Dockerfile

#

I did

#

I rebuilt and pushed it to Hub

barren iron
smoky stratus
#

Let me try it manually with commands

barren iron
#

So it'll show the time as the last time the build changed, which, according to docker, must've been 9 hrs ago

smoky stratus
#

What should I run on the Dockerfile?

#

Let me see if I can see what PyCharm runs

#

I found it

#

docker build -t markomartin/test:latest .

#

Does that look correct?

#

And then docker push markomartin/test:latest

barren iron
#

Yep

#

It does

smoky stratus
#

So then what am I doing wrong?

barren iron
#

Can I see the output of both of those commands: can you change the Dockerfile to so that it does update?

smoky stratus
#

Sure 1 sec

#
[+] Building 36.3s (10/10) FINISHED
 => [internal] load build definition from Dockerfile                                                               0.2s
 => => transferring dockerfile: 304B                                                                               0.1s
 => [internal] load .dockerignore                                                                                  0.1s
 => => transferring context: 2B                                                                                    0.0s
 => [internal] load metadata for docker.io/library/python:3.10                                                     0.0s
 => [1/5] FROM docker.io/library/python:3.10                                                                       0.7s
 => [internal] load build context                                                                                  0.3s
 => => transferring context: 26.94kB                                                                               0.0s
 => [2/5] COPY /api_scraper/api_scraper.py /api_scraper/api_scraper.py                                             0.1s
 => [3/5] COPY /collection_scraper/collection_scraper.py /collection_scraper/collection_scraper.py                 0.1s
 => [4/5] COPY requirements.txt .                                                                                  0.1s
 => [5/5] RUN pip install -r requirements.txt                                                                     32.0s
 => exporting to image                                                                                             2.8s
 => => exporting layers                                                                                            2.8s
 => => writing image sha256:bb79f774ca1fbcb70b2040b5cdcde32bd1bb80c81f4f59611e00023486822d67                       0.0s
 => => naming to docker.io/markomartin/nft-scraper:latest                                                          0.0s
#
The push refers to repository [docker.io/markomartin/nft-scraper]
aa1bde3a80a0: Pushed
9053d005a9e4: Layer already exists
c2d42bf2a6fb: Pushed
c8a06d87a25e: Pushed
42ce61e841fa: Layer already exists
a9b125f164c3: Layer already exists
e24045f8c247: Layer already exists
b7b662b31e70: Layer already exists
6f5234c0aacd: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
latest: digest: sha256:a437841600a4aaa054904f8ab052c877f3683931f2f9cfbd147e00c351328485 size: 3053
#

Still 9 hours ago and the commands in the image layers are not updated

barren iron
#

Right so. This time you see how it was pushed?

#

Like those first couple steps

smoky stratus
#

But it's not lol

#

It is still outdated on the hub

barren iron
#

Well, now I think its something to do with private dockerhub limits?

smoky stratus
#

How the hell am I supposed to do it then?

#

It's a closed project for a client not public

barren iron
#

/markomartin/nft-scraper

#

Also your build has changed??

#

the name has changed

smoky stratus
#

Yeah I tried with the actual one I want to push not the test one

#

Neither works

#

COPY file:a29aecd233088a0cfdbcc34d9422c3fd7abf8446e5039943a115ce7df132e9a8 in /collection_scraper

barren iron
#

Can you show me screenshots from dockerhub?

#

With as much details as possible?

smoky stratus
#

I don't have that in my Dockerfile anymore

#

Sure let me make an example

#

In my Dockerfile:
ENV PYTHONUNBUFFERED False

barren iron
#

Nah I want to see what the tags say

#

Like on the main page

smoky stratus
#

Not sure what else to show

#

Local image:

#

So I am in indeed rebuilding it

#

And then pushing it to the repo which succeeds and says "pushed"

#

But it does nothing

barren iron
#

I don't think it is...

#

hub.docker.com is what I would say it should be?

smoky stratus
#

Well how would have I been able to push it then?

barren iron
#

I am not sure tho

smoky stratus
#

I just deleted the repo and pushed again, it worked like that

deep estuary
smoky stratus
#

But updating the image does absolutely nothing

#

Or does it just take forever for whatever reason?

barren iron
#

I don't see why it would take anytime...

#

But I have never used a private repo

smoky stratus
#

Neither do I but it most certainly not updating lol

barren iron
#

And I don't trust Pycharm

smoky stratus
#

I pushed manually, it had the same output

#

Most layers didn't get pushed (makes sense cause I only changed the Dockerfile)

#

Local image is fine, remote one isn't

#

Last pushed: a few seconds ago

#

Wait what

#

I did it like 10 minutes ago

#

It just updated

deep estuary
#

makes sense, it won't be immediate

#

it takes a bit to reindex

smoky stratus
#

Oh I didn't know that

#

My bad

deep estuary
#

generally the pull command should fetch the latest image for certain, but the site might take a few mins

smoky stratus
#

Ohh

#

That makes sense

#

I didn't try to pull was just waiting for it to update on the dashboard

barren iron
#

It would've updated in the 10mins me and him were talking tho joe?

smoky stratus
#

Thank Joe

barren iron
#

Ah sorry didn't read up high enough

smoky stratus
#

Thanks both in fact

#

Yeah I just tested it, when I push and pull it uses the fresh image instantly, but the site does take a bit to update and gives a false "last pushed" date (as in not accurate to the minute) - makes sense just confusing

#

In fact the "pushed" date makes no sense, it is jumping between 10 minutes, 45 minutes and 1 hour with every refresh

#

I broke their site I guess

deep estuary
#

this is why we use ghcr

#

lol

#

docker hub is bad

smoky stratus
deep estuary
#

maybe a year or two ago, gone very downhill lately

smoky stratus
#

I want to setup automatic build & deployment when I push to the Git

#

Can I do that with GHCR?

deep estuary
#

implemented some quite harsh ratelimits on how much you can pull images

#

yep

#

GHCR is just docker hub but provided by GitHub instead

#

you have ghcr.io/<gh-user>/<package>

smoky stratus
#

Thanks for the recommendation, was just about to pay for an annual docker hub but I already hated it lol

#

I'll look into it, thanks!

#

I use portainer to manage the Docker swarm, do I just add that as a custom repo and nothing much really changes?

#

Let me try and see

deep estuary
#

yea

barren iron
deep estuary
#

self-host a container registry?

barren iron
#

Yeah

#

I can think of a few reasons just wondering if they are the same

deep estuary
#

a) bandwidth costs b) integrations with github c) what benefit does it actually give us? we pay more for minimal benefit

#

there are many more reasons not to self-host than to self-host

#

it's not something where you get a benefit from self-hosting, you're just paying for more storage to host your images

barren iron
#

Yeah. I'd say a major reason for you guys is downtime. You don't want your docker repo to go down

#

Yeah cheers :)

deep estuary
#

on top of that ghcr is globally replicated, free, integrated with github permissions, works out of the box with github actions, a whole load of reasons why it's more suitable for any developer, not just organisations

barren iron
#

Yeah for your usecase it makes a lot of sense. Was just curious to get someone else's thoughts

#

Thanks

smoky stratus
#

I'm just stepping into CI/CD but this tech is amazing

deep estuary
#

awesome 😄

smoky stratus
#

First time using GitHub Actions also, it's insane how much GitHub gives us... for free

abstract osprey
#

Can someone help me with using my .gitignore

#

or on how to hide files...

inner pollen
#

created this small tool to delete the file/directory if it is create by this tool. I am looking for advise on how to kill the daemon process after it finishes its task of deleting the file
https://github.com/Agent-Hellboy/dswell

GitHub

Contribute to Agent-Hellboy/dswell development by creating an account on GitHub.

velvet spire
#

os._exit is best KEKW

indigo zenith
inner pollen
#

Looks like I have to reconsider the design or maybe move on with other things. 😜

tawdry needle
#

is there a standalone library for working with / generating ninja build files?

#

i see some build tools written in python that use ninja as a backend, but i don't see any standalone library for that purpose (as opposed to a "tool")

halcyon pelican
#

Hi...I have a task in my work.. we have a application mobil built with react native and flask as backend... Currently the application mobile is authenticating with Aws cognito service from backend. But my team determinated that this implementation is wrong. They want use AWS lambda like microservice that it can authenticate the user with diferentes providers of authentication such auth0 or cognito. They want have this flexibility when will be necessary. Finally the team too want social login. I have investigated other service such AWS ALB and AWS Api gateway these are others alternatives. I want read us about your opinions. how would you solve this problem? what are your tips?

analog kettle
#

Is there a way to make virtual environments that automatically install certain libraries, or should I just make a bash script for that?

velvet spire
#

uhhhh brb

analog kettle
#

I want to always update pip setuptools wheel and the install numpy, pandas, sklearn, and ipython.

velvet spire
#

no, not that i can tell

#

however virtualenv installed setuptools and wheel, but doesn't update them iirc

#

i've made a special requirements.txt in my home dir which has all of the packages, so i just invoke pip around that

smoky stratus
deep estuary
smoky stratus
velvet spire
smoky stratus
velvet spire
#

how would you do that on windows runner?

smoky stratus
#

Oh I thought you meant locally, my bad

deep estuary
#

lol got a 502 from google trying to find an answer, that's new

#

I know we use curl somewhere in one of our windows runners

velvet spire
#

i got a 500 with the USPS today lol

#

...pydis uses windows runners?

deep estuary
#

no

#

i have things other than pydis lol

velvet spire
#

smh its pydis server so i presume that's what you mean when you say 'we' here smh

smoky stratus
#

Same lol

deep estuary
velvet spire
#

probably

rich sequoia
#

What doee 502 mean tho?

deep estuary
#

when you make a request to a service like Google, you actually pass through a lot of servers that make other HTTP requests down the line, then once a response is created it's pushed back to you as the client

#

when one of those servers fails, a 502 is emitted, because the "gateway" to the application has failed

bleak ether
#

Anyone got a good reference for writing python dockerfiles? I get the feeling i should be using multistage builds

bleak ether
#

argh 4 gb

#

python images are disgusting

shadow crow
#

I want to learn how message queues work. What would be a good pet project for that?

bleak ether
#

typical use case is sending emails

tawdry needle
#

but a multi-stage build for python itself seems worthwhile

bleak ether
#

mainly for the size

#

im down to 3 gb

#

not sure what else to do though 😦

#

after playing with golang static binary + alpine 15 mb containters this is torture

tawdry needle
#

i found our base dockerfile

#

it's literally just FROM python

bleak ether
#
FROM continuumio/miniconda3 AS build

ARG ENV_FILE=dev-env.yml
ARG ENV_NAME=devenv

COPY ${ENV_FILE} .

RUN conda env create -f ${ENV_FILE}
RUN conda install -c conda-forge conda-pack

RUN conda-pack -n ${ENV_NAME} -o /tmp/env.tar && \
  mkdir /venv && cd /venv && tar xf /tmp/env.tar && \
  rm /tmp/env.tar

RUN /venv/bin/conda-unpack

#----------------------------------
FROM debian:buster AS runtime

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /app

RUN apt-get update && apt-get -yq upgrade && apt-get install -yq --no-install-recommends libgl1-mesa-glx && rm -rf /var/lib/apt/lists/*

COPY --from=build /venv /app/venv

CMD TODO```
tawdry needle
#

conda envs tend to be on the big side

bleak ether
#

yeah doesnt help that the dev pulls in giant libraries for just one function call

#

grrrrr

tawdry needle
#

i don't think you should use it unless you need it

#

there's no reason to use it if you're going to use venv anyway

bleak ether
#

we have some c deps that arent on pypi

tawdry needle
#

that are in conda-forge? interesting

bleak ether
#

maybe old news

tawdry needle
#

or are you using your own anaconda repo?

bleak ether
#

might of been a version conflict or seomthing

#

no just things like pypi doesnt have numpy with mkl

#

defaults to openblas

tawdry needle
#

ah, yep

#

what would the 2nd stage in your build even be?

bleak ether
#

no idea i figured it would cache more nicely

tawdry needle
#

those machine learning libraries tend to just be big

#

if you're pulling in pandas and tensorflow and stuff yeah you're looking at 1-2 gb

bleak ether
#

good to know, wont spend too much time on it

tawdry needle
#

that actually was an issue on my laptop at work in a past job, it was getting full from too many conda envs with those packages in it

bleak ether
#

yeah were using self-hosted github actions runners, they don't run in a container so you have to clean it up yourself (which is why im writing this docker file). I had a cron job to clean up the stuff everynow and then but of course it wasn't enough and the disk got filled

#

hopefully itll go more smoothly with the containersnow...

#

Thanks for the advice

twin hull
#
FROM ubuntu:20.04

RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y linux-perf
...

throws me the error #6 0.596 E: Unable to locate package linux-perf. Googling tells me I should update upgrade apt but that's what I'm doing already

bleak ether
#

not sure that package exists

#

what command are you trying to get

#

if its the perf command

twin hull
#

yeah it's perf I need

#

should probably switch to debian then

bleak ether
#

i mean ubuntu has it too the package is jsut named differently

twin hull
#

apt-get install linux-tools-$(uname -r) linux-tools-generic -y this didn't work

#

Debian worked though

bleak ether
#

kk

twin hull
#

thanks for the help!

clever wedge
#

every tool's a hammer

runic jewel
#

How to use GitHub actions or when I push the code to master branch it should be get automatically update to my nginx web server(on my instance) django

brisk lake
#

is there any tool out there to help me audit my packages for reported/known vulnerabilities/security concerns?

#

or do i just update to most recent version and hope for the best

#

first result i found is snyk.io but i dont really see an API/package for it and i'd rather do these things programmatically

silver vector
#

can someone explain to me what validation testing and unexpected failure testing is

silver summit
#
version: "3.7"

services:
  postgres:
    image: postgres
    environment:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test123
      POSTGRES_DB: idmaker
      PGDATA: /data/postgres
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
      - ./postgres-data:/data/postgres
    ports:
      - "5432:5432"
    restart: unless-stopped

thin jasper
#

seems like the docker user doesnt have permissions to write that dir

#

running the command with sudo has no effect on the user

#

change the write permissions of that directory you mount

silver summit
#

already did

#

nothing change

#

i want all the data in my local dir

#

so incase of any problem i can have everything in my drive

brisk lake
#

which it is

next spruce
#

but yeah, they have an agenda

brisk lake
#

they dont have an api from what i see

#

i gotta drag and drop my deps txt like a chump

smoky stratus
#

Docker compose question - I've got a service that goes like this

services:

  price_bot:
    image: ghcr.io/martin-marko/scraper:master
    volumes:
      - price_bot:/price_bot/logs
    working_dir: /price_bot
    command: ["python", "price_bot.py"]

    deploy:
      mode: replicated
      placement:
        constraints:
          - node.labels.price_bot==true

      << : *deploy

    << : *default
    logging: *logging
    secrets: *secrets
    environment: *environment

volumes:
  price_bot:

price_bot.py creates its log files at /logs so /logs/price_bot.log etc., but in the container the logs are in the root directory of the volume so at /var/lib/docker/volumes/scraper_price_bot/_data. What am I doing wrong and how should I mount the volume so it mounts the entire directory not just the logs in the directory?

flat path
#

!pypi pip-audit

rancid schoonerBOT